diff options
author | Ævar Arnfjörð Bjarmason <avarab@gmail.com> | 2022-08-04 18:28:35 +0200 |
---|---|---|
committer | Junio C Hamano <gitster@pobox.com> | 2022-08-04 14:12:23 -0700 |
commit | 8cbace93d270dc49f007a2c1922769240a22eca5 (patch) | |
tree | ea698398263906e6b210cbf1205dfed271173dc4 /Documentation/technical | |
parent | 844739ba275e451e44f09a61c4ce0458a8df6077 (diff) | |
download | git-8cbace93d270dc49f007a2c1922769240a22eca5.tar.gz |
docs: move commit-graph format docs to man section 5
Continue the move of existing Documentation/technical/* protocol and
file-format documentation into our main documentation space.
By moving the documentation for the commit-graph format into man
section 5 and the new "developerinterfaces" category. This change is
split from subsequent commits due to the relatively large amount of
ASCIIDOC formatting changes that are required.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Diffstat (limited to 'Documentation/technical')
-rw-r--r-- | Documentation/technical/chunk-format.txt | 4 | ||||
-rw-r--r-- | Documentation/technical/commit-graph-format.txt | 166 |
2 files changed, 2 insertions, 168 deletions
diff --git a/Documentation/technical/chunk-format.txt b/Documentation/technical/chunk-format.txt index 593614fced..f36ce42f37 100644 --- a/Documentation/technical/chunk-format.txt +++ b/Documentation/technical/chunk-format.txt @@ -6,7 +6,7 @@ sections of the file. This allows structured access to a large file by scanning a small "table of contents" for the remaining data. This common format is used by the `commit-graph` and `multi-pack-index` files. See link:technical/pack-format.html[the `multi-pack-index` format] and -link:technical/commit-graph-format.html[the `commit-graph` format] for +the `commit-graph` format in linkgit:gitformat-commit-graph[5] for how they use the chunks to describe structured data. A chunk-based file format begins with some header information custom to @@ -108,7 +108,7 @@ for future formats: * *commit-graph:* see `write_commit_graph_file()` and `parse_commit_graph()` in `commit-graph.c` for how the chunk-format API is used to write and parse the commit-graph file format documented in - link:technical/commit-graph-format.html[the commit-graph file format]. + the commit-graph file format in linkgit:gitformat-commit-graph[5]. * *multi-pack-index:* see `write_midx_internal()` and `load_multi_pack_index()` in `midx.c` for how the chunk-format API is used to write and diff --git a/Documentation/technical/commit-graph-format.txt b/Documentation/technical/commit-graph-format.txt deleted file mode 100644 index 484b185ba9..0000000000 --- a/Documentation/technical/commit-graph-format.txt +++ /dev/null @@ -1,166 +0,0 @@ -Git commit graph format -======================= - -The Git commit graph stores a list of commit OIDs and some associated -metadata, including: - -- The generation number of the commit. - -- The root tree OID. - -- The commit date. - -- The parents of the commit, stored using positional references within - the graph file. - -- The Bloom filter of the commit carrying the paths that were changed between - the commit and its first parent, if requested. - -These positional references are stored as unsigned 32-bit integers -corresponding to the array position within the list of commit OIDs. Due -to some special constants we use to track parents, we can store at most -(1 << 30) + (1 << 29) + (1 << 28) - 1 (around 1.8 billion) commits. - -== Commit graph files have the following format: - -In order to allow extensions that add extra data to the graph, we organize -the body into "chunks" and provide a binary lookup table at the beginning -of the body. The header includes certain values, such as number of chunks -and hash type. - -All multi-byte numbers are in network byte order. - -HEADER: - - 4-byte signature: - The signature is: {'C', 'G', 'P', 'H'} - - 1-byte version number: - Currently, the only valid version is 1. - - 1-byte Hash Version - We infer the hash length (H) from this value: - 1 => SHA-1 - 2 => SHA-256 - If the hash type does not match the repository's hash algorithm, the - commit-graph file should be ignored with a warning presented to the - user. - - 1-byte number (C) of "chunks" - - 1-byte number (B) of base commit-graphs - We infer the length (H*B) of the Base Graphs chunk - from this value. - -CHUNK LOOKUP: - - (C + 1) * 12 bytes listing the table of contents for the chunks: - First 4 bytes describe the chunk id. Value 0 is a terminating label. - Other 8 bytes provide the byte-offset in current file for chunk to - start. (Chunks are ordered contiguously in the file, so you can infer - the length using the next chunk position if necessary.) Each chunk - ID appears at most once. - - The CHUNK LOOKUP matches the table of contents from - link:technical/chunk-format.html[the chunk-based file format]. - - The remaining data in the body is described one chunk at a time, and - these chunks may be given in any order. Chunks are required unless - otherwise specified. - -CHUNK DATA: - - OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes) - The ith entry, F[i], stores the number of OIDs with first - byte at most i. Thus F[255] stores the total - number of commits (N). - - OID Lookup (ID: {'O', 'I', 'D', 'L'}) (N * H bytes) - The OIDs for all commits in the graph, sorted in ascending order. - - Commit Data (ID: {'C', 'D', 'A', 'T' }) (N * (H + 16) bytes) - * The first H bytes are for the OID of the root tree. - * The next 8 bytes are for the positions of the first two parents - of the ith commit. Stores value 0x70000000 if no parent in that - position. If there are more than two parents, the second value - has its most-significant bit on and the other bits store an array - position into the Extra Edge List chunk. - * The next 8 bytes store the topological level (generation number v1) - of the commit and - the commit time in seconds since EPOCH. The generation number - uses the higher 30 bits of the first 4 bytes, while the commit - time uses the 32 bits of the second 4 bytes, along with the lowest - 2 bits of the lowest byte, storing the 33rd and 34th bit of the - commit time. - - Generation Data (ID: {'G', 'D', 'A', '2' }) (N * 4 bytes) [Optional] - * This list of 4-byte values store corrected commit date offsets for the - commits, arranged in the same order as commit data chunk. - * If the corrected commit date offset cannot be stored within 31 bits, - the value has its most-significant bit on and the other bits store - the position of corrected commit date into the Generation Data Overflow - chunk. - * Generation Data chunk is present only when commit-graph file is written - by compatible versions of Git and in case of split commit-graph chains, - the topmost layer also has Generation Data chunk. - - Generation Data Overflow (ID: {'G', 'D', 'O', '2' }) [Optional] - * This list of 8-byte values stores the corrected commit date offsets - for commits with corrected commit date offsets that cannot be - stored within 31 bits. - * Generation Data Overflow chunk is present only when Generation Data - chunk is present and atleast one corrected commit date offset cannot - be stored within 31 bits. - - Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional] - This list of 4-byte values store the second through nth parents for - all octopus merges. The second parent value in the commit data stores - an array position within this list along with the most-significant bit - on. Starting at that array position, iterate through this list of commit - positions for the parents until reaching a value with the most-significant - bit on. The other bits correspond to the position of the last parent. - - Bloom Filter Index (ID: {'B', 'I', 'D', 'X'}) (N * 4 bytes) [Optional] - * The ith entry, BIDX[i], stores the number of bytes in all Bloom filters - from commit 0 to commit i (inclusive) in lexicographic order. The Bloom - filter for the i-th commit spans from BIDX[i-1] to BIDX[i] (plus header - length), where BIDX[-1] is 0. - * The BIDX chunk is ignored if the BDAT chunk is not present. - - Bloom Filter Data (ID: {'B', 'D', 'A', 'T'}) [Optional] - * It starts with header consisting of three unsigned 32-bit integers: - - Version of the hash algorithm being used. We currently only support - value 1 which corresponds to the 32-bit version of the murmur3 hash - implemented exactly as described in - https://en.wikipedia.org/wiki/MurmurHash#Algorithm and the double - hashing technique using seed values 0x293ae76f and 0x7e646e2 as - described in https://doi.org/10.1007/978-3-540-30494-4_26 "Bloom Filters - in Probabilistic Verification" - - The number of times a path is hashed and hence the number of bit positions - that cumulatively determine whether a file is present in the commit. - - The minimum number of bits 'b' per entry in the Bloom filter. If the filter - contains 'n' entries, then the filter size is the minimum number of 64-bit - words that contain n*b bits. - * The rest of the chunk is the concatenation of all the computed Bloom - filters for the commits in lexicographic order. - * Note: Commits with no changes or more than 512 changes have Bloom filters - of length one, with either all bits set to zero or one respectively. - * The BDAT chunk is present if and only if BIDX is present. - - Base Graphs List (ID: {'B', 'A', 'S', 'E'}) [Optional] - This list of H-byte hashes describe a set of B commit-graph files that - form a commit-graph chain. The graph position for the ith commit in this - file's OID Lookup chunk is equal to i plus the number of commits in all - base graphs. If B is non-zero, this chunk must exist. - -TRAILER: - - H-byte HASH-checksum of all of the above. - -== Historical Notes: - -The Generation Data (GDA2) and Generation Data Overflow (GDO2) chunks have -the number '2' in their chunk IDs because a previous version of Git wrote -possibly erroneous data in these chunks with the IDs "GDAT" and "GDOV". By -changing the IDs, newer versions of Git will silently ignore those older -chunks and write the new information without trusting the incorrect data. |