Changes between Version 6 and Version 8 of Ticket #1073


Ignore:
Timestamp:
Nov 19, 2015, 6:18:13 PM (8 years ago)
Author:
Dimitar Misev
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #1073 – Description

    v6 v8  
    2828* ntfs : `2^32-1` theoretically (same limit as number of files in a directory)
    2929
    30 Between 10,000 and 100,000 files per directory seems like a good number well supported across filesystems. If we take 100,000 on ext3 that gives us a lower limit of 3 billion tiles.
    31 
    32 = Single-level nesting =
    33 
    34 Distributing tiles in 100,000 per directory we have this organization:
    35 
    36 {{{
    37 $RASDATA
    38  |_ RASBASE
    39  |_ TILES
    40       |_ 0
    41       |  |_ 1
    42       |  |_ 2
    43       |  |_ 3
    44       |  |_ ...
    45       |   
    46       |_ 1
    47       |  |_ 100,000
    48       |  |_ 100,001
    49       |  |_ 100,002
    50       |  |_ ...
    51       | 
    52       |_ ...
    53 }}}
    54 
    55 The subdirectory index in TILES is dir_index = tile_index / 100,000. The 100,000 number can be a compile time constant that can be adjusted as necessary. By default it is maybe better if it is `2^16` or `2^17` so that the dir_index can be computed with a fast bit shift.
    56 
    57 With 30,000 subdirs this gives us a "lower" limit of ~12 PB (with 4MB tile size).
     30Between 10,000 and 100,000 files per directory seems like a good number well supported across filesystems.
    5831
    5932= Two-level nesting =
    … …  
    9063
    9164The subdirectory index in TILES is:
    92  - dir2_index = tile_index / max_files (100,000)
    93  - dir1_index = dir2_index / max_dirs (32,768)
     65 - dir2_index = tile_index / max_files (16,384)
     66 - dir1_index = dir2_index / max_dirs (16,384)
     67
     68I suggest we take a limit of `2^14` = 16,384 directories / files for the second and third level.
    9469 
    95 This sets a "lower" limit of ~400 EB (with 4MB tiles).
     70This gives us a "lower" limit of about ~20 EB (with 4MB tiles).
    9671
    9772= Backwards compatibility =