diff options
84 files changed, 2080 insertions, 1038 deletions
@@ -152,6 +152,7 @@ Lars Doelle <lars.doelle@on-line ! de> Lars Doelle <lars.doelle@on-line.de> Lars Noschinski <lars@public.noschinski.de> <lars.noschinski@rwth-aachen.de> Li Hong <leehong@pku.edu.cn> +Linus Arver <linus@ucla.edu> <linusa@google.com> Linus Torvalds <torvalds@linux-foundation.org> <torvalds@evo.osdl.org> Linus Torvalds <torvalds@linux-foundation.org> <torvalds@g5.osdl.org> Linus Torvalds <torvalds@linux-foundation.org> <torvalds@osdl.org> diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines index ab39509d59..1d92b2da03 100644 --- a/Documentation/CodingGuidelines +++ b/Documentation/CodingGuidelines @@ -188,6 +188,22 @@ For shell scripts specifically (not exhaustive): hopefully nobody starts using "local" before they are reimplemented in C ;-) + - Some versions of shell do not understand "export variable=value", + so we write "variable=value" and then "export variable" on two + separate lines. + + - Some versions of dash have broken variable assignment when prefixed + with "local", "export", and "readonly", in that the value to be + assigned goes through field splitting at $IFS unless quoted. + + (incorrect) + local variable=$value + local variable=$(command args) + + (correct) + local variable="$value" + local variable="$(command args)" + - Use octal escape sequences (e.g. "\302\242"), not hexadecimal (e.g. "\xc2\xa2") in printf format strings, since hexadecimal escape sequences are not portable. diff --git a/Documentation/RelNotes/2.45.0.txt b/Documentation/RelNotes/2.45.0.txt index 039cce7708..15704ff98f 100644 --- a/Documentation/RelNotes/2.45.0.txt +++ b/Documentation/RelNotes/2.45.0.txt @@ -77,13 +77,17 @@ UI, Workflows & Features skip showing the hunk immediately after it has already been shown, and an additional action to explicitly ask to reshow the current hunk. - * "git pack-refs" learned the "--auto" option, which is a useful - addition to be triggered from "git gc --auto". + * "git pack-refs" learned the "--auto" option, which defers the decision of + whether and how to pack to the ref backend. This is used by the reftable + backend to avoid repacking of an already-optimal ref database. The new mode + is triggered from "git gc --auto". * "git add -u <pathspec>" and "git commit [-i] <pathspec>" did not diagnose a pathspec element that did not match any files in certain situations, unlike "git add <pathspec>" did. + * The userdiff patterns for C# has been updated. + Performance, Internal Implementation, Development Support etc. @@ -98,7 +102,7 @@ Performance, Internal Implementation, Development Support etc. * The way placeholders are to be marked-up in documentation have been specified; use "_<placeholder>_" to typeset the word inside a pair - of <angle-brakets> emphasized. + of <angle-brackets> emphasized. * "git --no-lazy-fetch cmd" allows to run "cmd" while disabling lazy fetching of objects from the promisor remote, which may be handy @@ -108,9 +112,6 @@ Performance, Internal Implementation, Development Support etc. clean.requireForce has been simplified, together with the documentation. - * The code to iterate over refs with the reftable backend has seen - some optimization. - * Uses of xwrite() helper have been audited and updated for better error checking and simpler code. @@ -173,6 +174,16 @@ Performance, Internal Implementation, Development Support etc. under restrictive umask setting (e.g. 007), which has been corrected. + * Document and apply workaround for a buggy version of dash that + mishandles "local var=val" construct. + + * The codepaths that reach date_mode_from_type() have been updated to + pass "struct date_mode" by value to make them thread safe. + + * The strategy to compact multiple tables of reftables after many + operations accumulate many entries has been improved to avoid + accumulating too many tables uncollected. + Fixes since v2.44 ----------------- @@ -236,7 +247,7 @@ Fixes since v2.44 This has been corrected. (merge 199f44cb2e ps/remote-helper-repo-initialization-fix later to maint). - * Various parts of upload-pack has been updated to bound the resource + * Various parts of upload-pack have been updated to bound the resource consumption relative to the size of the repository to protect from abusive clients. (merge 6cd05e768b jk/upload-pack-bounded-resources later to maint). @@ -283,11 +294,11 @@ Fixes since v2.44 variable that is no longer used. (merge 72a8d3f027 pw/rebase-i-ignore-cherry-pick-help-environment later to maint). - * The code to find the effective end of log message can fall into an + * The code to find the effective end of log messages can fall into an endless loop, which has been corrected. (merge 2541cba2d6 fs/find-end-of-log-message-fix later to maint). - * Mark-ups used in the documentation has been improved for + * Mark-up used in the documentation has been improved for consistency. (merge 45d5ed3e50 ja/doc-markup-fixes later to maint). @@ -381,6 +392,15 @@ Fixes since v2.44 configuration variable used to leak, which has been corrected. (merge 0e0fefb29f jc/unleak-core-excludesfile later to maint). + * vreportf(), which is used by error() and friends, has been taught + to give the error message printf-format string when its vsnprintf() + call fails, instead of showing nothing useful to identify the + nature of the error. + (merge c63adab961 rs/usage-fallback-to-show-message-format later to maint). + + * Adjust to an upcoming changes to GNU make that breaks our Makefiles. + (merge 227b8fd902 tb/make-indent-conditional-with-non-spaces later to maint). + * Other code cleanup, docfix, build fix, etc. (merge f0e578c69c rs/use-xstrncmpz later to maint). (merge 83e6eb7d7a ba/credential-test-clean-fix later to maint). @@ -408,3 +428,6 @@ Fixes since v2.44 (merge 8b68b48d5c ds/typofix-core-config-doc later to maint). (merge 39bb692152 rs/imap-send-use-xsnprintf later to maint). (merge 8d320cec60 jc/t2104-style-fixes later to maint). + (merge b4454d5a7b pw/t3428-cleanup later to maint). + (merge 84a7c33a4b pf/commitish-committish later to maint). + (merge 8882ee9d68 la/mailmap-entry later to maint). diff --git a/Documentation/date-formats.txt b/Documentation/date-formats.txt index 67645cae64..e24517c496 100644 --- a/Documentation/date-formats.txt +++ b/Documentation/date-formats.txt @@ -11,7 +11,7 @@ Git internal format:: For example CET (which is 1 hour ahead of UTC) is `+0100`. RFC 2822:: - The standard email format as described by RFC 2822, for example + The standard date format as described by RFC 2822, for example `Thu, 07 Apr 2005 22:13:13 +0200`. ISO 8601:: diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index b2607366b9..3d435157a6 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -303,7 +303,7 @@ and some sanity checks on the numeric values may also be performed. with e.g. bogus timezone values. `rfc2822`:: - This is the standard email format as described by RFC 2822. + This is the standard date format as described by RFC 2822. + An example value is ``Tue Feb 6 11:22:18 2007 -0500''. The Git parser is accurate, but a little on the lenient side. It is the @@ -630,18 +630,28 @@ in octal. Git only supports the following modes: In both formats `<path>` is the complete path of the file to be added (if not already existing) or modified (if already existing). -A `<path>` string must use UNIX-style directory separators (forward -slash `/`), may contain any byte other than `LF`, and must not -start with double quote (`"`). - -A path can use C-style string quoting; this is accepted in all cases -and mandatory if the filename starts with double quote or contains -`LF`. In C-style quoting, the complete name should be surrounded with -double quotes, and any `LF`, backslash, or double quote characters -must be escaped by preceding them with a backslash (e.g., -`"path/with\n, \\ and \" in it"`). - -The value of `<path>` must be in canonical form. That is it must not: +A `<path>` can be written as unquoted bytes or a C-style quoted string. + +When a `<path>` does not start with a double quote (`"`), it is an +unquoted string and is parsed as literal bytes without any escape +sequences. However, if the filename contains `LF` or starts with double +quote, it cannot be represented as an unquoted string and must be +quoted. Additionally, the source `<path>` in `filecopy` or `filerename` +must be quoted if it contains SP. + +When a `<path>` starts with a double quote (`"`), it is a C-style quoted +string, where the complete filename is enclosed in a pair of double +quotes and escape sequences are used. Certain characters must be escaped +by preceding them with a backslash: `LF` is written as `\n`, backslash +as `\\`, and double quote as `\"`. Some characters may optionally be +written with escape sequences: `\a` for bell, `\b` for backspace, `\f` +for form feed, `\n` for line feed, `\r` for carriage return, `\t` for +horizontal tab, and `\v` for vertical tab. Any byte can be written with +3-digit octal codes (e.g., `\033`). All filenames can be represented as +quoted strings. + +A `<path>` must use UNIX-style directory separators (forward slash `/`) +and its value must be in canonical form. That is it must not: * contain an empty directory component (e.g. `foo//bar` is invalid), * end with a directory separator (e.g. `foo/` is invalid), @@ -651,6 +661,7 @@ The value of `<path>` must be in canonical form. That is it must not: The root of the tree can be represented by an empty string as `<path>`. +`<path>` cannot contain NUL, either literally or escaped as `\000`. It is recommended that `<path>` always be encoded using UTF-8. `filedelete` diff --git a/Documentation/git-replay.txt b/Documentation/git-replay.txt index f6c269c62d..8f3300c683 100644 --- a/Documentation/git-replay.txt +++ b/Documentation/git-replay.txt @@ -46,7 +46,7 @@ the new commits (in other words, this mimics a cherry-pick operation). Range of commits to replay. More than one <revision-range> can be passed, but in `--advance <branch>` mode, they should have a single tip, so that it's clear where <branch> should point - to. See "Specifying Ranges" in linkgit:git-rev-parse and the + to. See "Specifying Ranges" in linkgit:git-rev-parse[1] and the "Commit Limiting" options below. include::rev-list-options.txt[] diff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN index df788c764b..dabd2b5b89 100755 --- a/GIT-VERSION-GEN +++ b/GIT-VERSION-GEN @@ -1,7 +1,7 @@ #!/bin/sh GVF=GIT-VERSION-FILE -DEF_VER=v2.44.GIT +DEF_VER=v2.45.0-rc0 LF=' ' @@ -1558,23 +1558,23 @@ ifneq (,$(SOCKLEN_T)) endif ifeq ($(uname_S),Darwin) - ifndef NO_FINK - ifeq ($(shell test -d /sw/lib && echo y),y) + ifndef NO_FINK + ifeq ($(shell test -d /sw/lib && echo y),y) BASIC_CFLAGS += -I/sw/include BASIC_LDFLAGS += -L/sw/lib - endif - endif - ifndef NO_DARWIN_PORTS - ifeq ($(shell test -d /opt/local/lib && echo y),y) + endif + endif + ifndef NO_DARWIN_PORTS + ifeq ($(shell test -d /opt/local/lib && echo y),y) BASIC_CFLAGS += -I/opt/local/include BASIC_LDFLAGS += -L/opt/local/lib - endif - endif - ifndef NO_APPLE_COMMON_CRYPTO + endif + endif + ifndef NO_APPLE_COMMON_CRYPTO NO_OPENSSL = YesPlease APPLE_COMMON_CRYPTO = YesPlease COMPAT_CFLAGS += -DAPPLE_COMMON_CRYPTO - endif + endif PTHREAD_LIBS = endif @@ -1613,23 +1613,23 @@ ifdef NO_CURL REMOTE_CURL_NAMES = EXCLUDED_PROGRAMS += git-http-fetch git-http-push else - ifdef CURLDIR + ifdef CURLDIR # Try "-Wl,-rpath=$(CURLDIR)/$(lib)" in such a case. CURL_CFLAGS = -I$(CURLDIR)/include CURL_LIBCURL = $(call libpath_template,$(CURLDIR)/$(lib)) - else + else CURL_CFLAGS = CURL_LIBCURL = - endif + endif - ifndef CURL_LDFLAGS + ifndef CURL_LDFLAGS CURL_LDFLAGS = $(eval CURL_LDFLAGS := $$(shell $$(CURL_CONFIG) --libs))$(CURL_LDFLAGS) - endif + endif CURL_LIBCURL += $(CURL_LDFLAGS) - ifndef CURL_CFLAGS + ifndef CURL_CFLAGS CURL_CFLAGS = $(eval CURL_CFLAGS := $$(shell $$(CURL_CONFIG) --cflags))$(CURL_CFLAGS) - endif + endif BASIC_CFLAGS += $(CURL_CFLAGS) REMOTE_CURL_PRIMARY = git-remote-http$X @@ -1637,29 +1637,29 @@ else REMOTE_CURL_NAMES = $(REMOTE_CURL_PRIMARY) $(REMOTE_CURL_ALIASES) PROGRAM_OBJS += http-fetch.o PROGRAMS += $(REMOTE_CURL_NAMES) - ifndef NO_EXPAT + ifndef NO_EXPAT PROGRAM_OBJS += http-push.o - endif + endif curl_check := $(shell (echo 072200; $(CURL_CONFIG) --vernum | sed -e '/^70[BC]/s/^/0/') 2>/dev/null | sort -r | sed -ne 2p) - ifeq "$(curl_check)" "072200" + ifeq "$(curl_check)" "072200" USE_CURL_FOR_IMAP_SEND = YesPlease - endif - ifdef USE_CURL_FOR_IMAP_SEND + endif + ifdef USE_CURL_FOR_IMAP_SEND BASIC_CFLAGS += -DUSE_CURL_FOR_IMAP_SEND IMAP_SEND_BUILDDEPS = http.o IMAP_SEND_LDFLAGS += $(CURL_LIBCURL) - endif - ifndef NO_EXPAT - ifdef EXPATDIR + endif + ifndef NO_EXPAT + ifdef EXPATDIR BASIC_CFLAGS += -I$(EXPATDIR)/include EXPAT_LIBEXPAT = $(call libpath_template,$(EXPATDIR)/$(lib)) -lexpat - else + else EXPAT_LIBEXPAT = -lexpat - endif - ifdef EXPAT_NEEDS_XMLPARSE_H + endif + ifdef EXPAT_NEEDS_XMLPARSE_H BASIC_CFLAGS += -DEXPAT_NEEDS_XMLPARSE_H - endif - endif + endif + endif endif IMAP_SEND_LDFLAGS += $(OPENSSL_LINK) $(OPENSSL_LIBSSL) $(LIB_4_CRYPTO) @@ -1671,15 +1671,15 @@ EXTLIBS += -lz ifndef NO_OPENSSL OPENSSL_LIBSSL = -lssl - ifdef OPENSSLDIR + ifdef OPENSSLDIR BASIC_CFLAGS += -I$(OPENSSLDIR)/include OPENSSL_LINK = $(call libpath_template,$(OPENSSLDIR)/$(lib)) - else + else OPENSSL_LINK = - endif - ifdef NEEDS_CRYPTO_WITH_SSL + endif + ifdef NEEDS_CRYPTO_WITH_SSL OPENSSL_LIBSSL += -lcrypto - endif + endif else BASIC_CFLAGS += -DNO_OPENSSL OPENSSL_LIBSSL = @@ -1697,18 +1697,18 @@ ifdef APPLE_COMMON_CRYPTO endif endif ifndef NO_ICONV - ifdef NEEDS_LIBICONV - ifdef ICONVDIR + ifdef NEEDS_LIBICONV + ifdef ICONVDIR BASIC_CFLAGS += -I$(ICONVDIR)/include ICONV_LINK = $(call libpath_template,$(ICONVDIR)/$(lib)) - else + else ICONV_LINK = - endif - ifdef NEEDS_LIBINTL_BEFORE_LIBICONV + endif + ifdef NEEDS_LIBINTL_BEFORE_LIBICONV ICONV_LINK += -lintl - endif + endif EXTLIBS += $(ICONV_LINK) -liconv - endif + endif endif ifdef ICONV_OMITS_BOM BASIC_CFLAGS += -DICONV_OMITS_BOM @@ -1829,10 +1829,10 @@ ifdef NO_MMAP COMPAT_CFLAGS += -DNO_MMAP COMPAT_OBJS += compat/mmap.o else - ifdef USE_WIN32_MMAP + ifdef USE_WIN32_MMAP COMPAT_CFLAGS += -DUSE_WIN32_MMAP COMPAT_OBJS += compat/win32mmap.o - endif + endif endif ifdef MMAP_PREVENTS_DELETE BASIC_CFLAGS += -DMMAP_PREVENTS_DELETE @@ -1957,11 +1957,11 @@ else BASIC_CFLAGS += -DSHA1_DC LIB_OBJS += sha1dc_git.o ifdef DC_SHA1_EXTERNAL - ifdef DC_SHA1_SUBMODULE - ifneq ($(DC_SHA1_SUBMODULE),auto) + ifdef DC_SHA1_SUBMODULE + ifneq ($(DC_SHA1_SUBMODULE),auto) $(error Only set DC_SHA1_EXTERNAL or DC_SHA1_SUBMODULE, not both) - endif - endif + endif + endif BASIC_CFLAGS += -DDC_SHA1_EXTERNAL EXTLIBS += -lsha1detectcoll else @@ -738,7 +738,7 @@ static int submodule_create_branch(struct repository *r, } void create_branches_recursively(struct repository *r, const char *name, - const char *start_commitish, + const char *start_committish, const char *tracking_name, int force, int reflog, int quiet, enum branch_track track, int dry_run) @@ -748,8 +748,8 @@ void create_branches_recursively(struct repository *r, const char *name, struct object_id super_oid; struct submodule_entry_list submodule_entry_list; - /* Perform dwim on start_commitish to get super_oid and branch_point. */ - dwim_branch_start(r, start_commitish, BRANCH_TRACK_NEVER, + /* Perform dwim on start_committish to get super_oid and branch_point. */ + dwim_branch_start(r, start_committish, BRANCH_TRACK_NEVER, &branch_point, &super_oid); /* @@ -772,7 +772,7 @@ void create_branches_recursively(struct repository *r, const char *name, submodule_entry_list.entries[i].submodule->name); if (advice_enabled(ADVICE_SUBMODULES_NOT_UPDATED)) advise(_("You may try updating the submodules using 'git checkout --no-recurse-submodules %s && git submodule update --init'"), - start_commitish); + start_committish); exit(code); } @@ -787,7 +787,7 @@ void create_branches_recursively(struct repository *r, const char *name, name); } - create_branch(r, name, start_commitish, force, 0, reflog, quiet, + create_branch(r, name, start_committish, force, 0, reflog, quiet, BRANCH_TRACK_NEVER, dry_run); if (dry_run) return; @@ -78,26 +78,26 @@ void create_branch(struct repository *r, * those of create_branch() except for start_name, which is represented * by two different parameters: * - * - start_commitish is the commit-ish, in repository r, that determines + * - start_committish is the commit-ish, in repository r, that determines * which commits the branches will point to. The superproject branch - * will point to the commit of start_commitish and the submodule - * branches will point to the gitlink commit oids in start_commitish's + * will point to the commit of start_committish and the submodule + * branches will point to the gitlink commit oids in start_committish's * tree. * * - tracking_name is the name of the ref, in repository r, that will be * used to set up tracking information. This value is propagated to * all submodules, which will evaluate the ref using their own ref - * stores. If NULL, this defaults to start_commitish. + * stores. If NULL, this defaults to start_committish. * - * When this function is called on the superproject, start_commitish + * When this function is called on the superproject, start_committish * can be any user-provided ref and tracking_name can be NULL (similar * to create_branches()). But when recursing through submodules, - * start_commitish is the plain gitlink commit oid. Since the oid cannot + * start_committish is the plain gitlink commit oid. Since the oid cannot * be used for tracking information, tracking_name is propagated and * used for tracking instead. */ void create_branches_recursively(struct repository *r, const char *name, - const char *start_commitish, + const char *start_committish, const char *tracking_name, int force, int reflog, int quiet, enum branch_track track, int dry_run); diff --git a/builtin/blame.c b/builtin/blame.c index db1f56de61..9aa74680a3 100644 --- a/builtin/blame.c +++ b/builtin/blame.c @@ -316,7 +316,7 @@ static const char *format_time(timestamp_t time, const char *tz_str, size_t time_width; int tz; tz = atoi(tz_str); - time_str = show_date(time, tz, &blame_date_mode); + time_str = show_date(time, tz, blame_date_mode); strbuf_addstr(&time_buf, time_str); /* * Add space paddings to time_buf to display a fixed width @@ -1029,7 +1029,7 @@ parse_done: blame_date_width = sizeof("Thu Oct 19 16:00:04 2006 -0700"); break; case DATE_STRFTIME: - blame_date_width = strlen(show_date(0, 0, &blame_date_mode)) + 1; /* add the null */ + blame_date_width = strlen(show_date(0, 0, blame_date_mode)) + 1; /* add the null */ break; } blame_date_width -= 1; /* strip the null */ diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 782bda007c..dc5a9d32dd 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -2210,7 +2210,7 @@ static int parse_mapped_oid_hex(const char *hex, struct object_id *oid, const ch * * idnum ::= ':' bigint; * - * Return the first character after the value in *endptr. + * Update *endptr to point to the first character after the value. * * Complain if the following character is not what is expected, * either a space or end of the string. @@ -2243,8 +2243,8 @@ static uintmax_t parse_mark_ref_eol(const char *p) } /* - * Parse the mark reference, demanding a trailing space. Return a - * pointer to the space. + * Parse the mark reference, demanding a trailing space. Update *p to + * point to the first character after the space. */ static uintmax_t parse_mark_ref_space(const char **p) { @@ -2258,10 +2258,62 @@ static uintmax_t parse_mark_ref_space(const char **p) return mark; } +/* + * Parse the path string into the strbuf. The path can either be quoted with + * escape sequences or unquoted without escape sequences. Unquoted strings may + * contain spaces only if `is_last_field` is nonzero; otherwise, it stops + * parsing at the first space. + */ +static void parse_path(struct strbuf *sb, const char *p, const char **endp, + int is_last_field, const char *field) +{ + if (*p == '"') { + if (unquote_c_style(sb, p, endp)) + die("Invalid %s: %s", field, command_buf.buf); + if (strlen(sb->buf) != sb->len) + die("NUL in %s: %s", field, command_buf.buf); + } else { + /* + * Unless we are parsing the last field of a line, + * SP is the end of this field. + */ + *endp = is_last_field + ? p + strlen(p) + : strchrnul(p, ' '); + strbuf_add(sb, p, *endp - p); + } +} + +/* + * Parse the path string into the strbuf, and complain if this is not the end of + * the string. Unquoted strings may contain spaces. + */ +static void parse_path_eol(struct strbuf *sb, const char *p, const char *field) +{ + const char *end; + + parse_path(sb, p, &end, 1, field); + if (*end) + die("Garbage after %s: %s", field, command_buf.buf); +} + +/* + * Parse the path string into the strbuf, and ensure it is followed by a space. + * Unquoted strings may not contain spaces. Update *endp to point to the first + * character after the space. + */ +static void parse_path_space(struct strbuf *sb, const char *p, + const char **endp, const char *field) +{ + parse_path(sb, p, endp, 0, field); + if (**endp != ' ') + die("Missing space after %s: %s", field, command_buf.buf); + (*endp)++; +} + static void file_change_m(const char *p, struct branch *b) { - static struct strbuf uq = STRBUF_INIT; - const char *endp; + static struct strbuf path = STRBUF_INIT; struct object_entry *oe; struct object_id oid; uint16_t mode, inline_data = 0; @@ -2298,16 +2350,12 @@ static void file_change_m(const char *p, struct branch *b) die("Missing space after SHA1: %s", command_buf.buf); } - strbuf_reset(&uq); - if (!unquote_c_style(&uq, p, &endp)) { - if (*endp) - die("Garbage after path in: %s", command_buf.buf); - p = uq.buf; - } + strbuf_reset(&path); + parse_path_eol(&path, p, "path"); /* Git does not track empty, non-toplevel directories. */ - if (S_ISDIR(mode) && is_empty_tree_oid(&oid) && *p) { - tree_content_remove(&b->branch_tree, p, NULL, 0); + if (S_ISDIR(mode) && is_empty_tree_oid(&oid) && *path.buf) { + tree_content_remove(&b->branch_tree, path.buf, NULL, 0); return; } @@ -2328,10 +2376,6 @@ static void file_change_m(const char *p, struct branch *b) if (S_ISDIR(mode)) die("Directories cannot be specified 'inline': %s", command_buf.buf); - if (p != uq.buf) { - strbuf_addstr(&uq, p); - p = uq.buf; - } while (read_next_command() != EOF) { const char *v; if (skip_prefix(command_buf.buf, "cat-blob ", &v)) @@ -2357,74 +2401,48 @@ static void file_change_m(const char *p, struct branch *b) command_buf.buf); } - if (!*p) { + if (!*path.buf) { tree_content_replace(&b->branch_tree, &oid, mode, NULL); return; } - tree_content_set(&b->branch_tree, p, &oid, mode, NULL); + tree_content_set(&b->branch_tree, path.buf, &oid, mode, NULL); } static void file_change_d(const char *p, struct branch *b) { - static struct strbuf uq = STRBUF_INIT; - const char *endp; + static struct strbuf path = STRBUF_INIT; - strbuf_reset(&uq); - if (!unquote_c_style(&uq, p, &endp)) { - if (*endp) - die("Garbage after path in: %s", command_buf.buf); - p = uq.buf; - } - tree_content_remove(&b->branch_tree, p, NULL, 1); + strbuf_reset(&path); + parse_path_eol(&path, p, "path"); + tree_content_remove(&b->branch_tree, path.buf, NULL, 1); } -static void file_change_cr(const char *s, struct branch *b, int rename) +static void file_change_cr(const char *p, struct branch *b, int rename) { - const char *d; - static struct strbuf s_uq = STRBUF_INIT; - static struct strbuf d_uq = STRBUF_INIT; - const char *endp; + static struct strbuf source = STRBUF_INIT; + static struct strbuf dest = STRBUF_INIT; struct tree_entry leaf; - strbuf_reset(&s_uq); - if (!unquote_c_style(&s_uq, s, &endp)) { - if (*endp != ' ') - die("Missing space after source: %s", command_buf.buf); - } else { - endp = strchr(s, ' '); - if (!endp) - die("Missing space after source: %s", command_buf.buf); - strbuf_add(&s_uq, s, endp - s); - } - s = s_uq.buf; - - endp++; - if (!*endp) - die("Missing dest: %s", command_buf.buf); - - d = endp; - strbuf_reset(&d_uq); - if (!unquote_c_style(&d_uq, d, &endp)) { - if (*endp) - die("Garbage after dest in: %s", command_buf.buf); - d = d_uq.buf; - } + strbuf_reset(&source); + parse_path_space(&source, p, &p, "source"); + strbuf_reset(&dest); + parse_path_eol(&dest, p, "dest"); memset(&leaf, 0, sizeof(leaf)); if (rename) - tree_content_remove(&b->branch_tree, s, &leaf, 1); + tree_content_remove(&b->branch_tree, source.buf, &leaf, 1); else - tree_content_get(&b->branch_tree, s, &leaf, 1); + tree_content_get(&b->branch_tree, source.buf, &leaf, 1); if (!leaf.versions[1].mode) - die("Path %s not in branch", s); - if (!*d) { /* C "path/to/subdir" "" */ + die("Path %s not in branch", source.buf); + if (!*dest.buf) { /* C "path/to/subdir" "" */ tree_content_replace(&b->branch_tree, &leaf.versions[1].oid, leaf.versions[1].mode, leaf.tree); return; } - tree_content_set(&b->branch_tree, d, + tree_content_set(&b->branch_tree, dest.buf, &leaf.versions[1].oid, leaf.versions[1].mode, leaf.tree); @@ -2432,7 +2450,6 @@ static void file_change_cr(const char *s, struct branch *b, int rename) static void note_change_n(const char *p, struct branch *b, unsigned char *old_fanout) { - static struct strbuf uq = STRBUF_INIT; struct object_entry *oe; struct branch *s; struct object_id oid, commit_oid; @@ -2497,10 +2514,6 @@ static void note_change_n(const char *p, struct branch *b, unsigned char *old_fa die("Invalid ref name or SHA1 expression: %s", p); if (inline_data) { - if (p != uq.buf) { - strbuf_addstr(&uq, p); - p = uq.buf; - } read_next_command(); parse_and_store_blob(&last_blob, &oid, 0); } else if (oe) { @@ -3152,6 +3165,7 @@ static void print_ls(int mode, const unsigned char *hash, const char *path) static void parse_ls(const char *p, struct branch *b) { + static struct strbuf path = STRBUF_INIT; struct tree_entry *root = NULL; struct tree_entry leaf = {NULL}; @@ -3168,17 +3182,9 @@ static void parse_ls(const char *p, struct branch *b) root->versions[1].mode = S_IFDIR; load_tree(root); } - if (*p == '"') { - static struct strbuf uq = STRBUF_INIT; - const char *endp; - strbuf_reset(&uq); - if (unquote_c_style(&uq, p, &endp)) - die("Invalid path: %s", command_buf.buf); - if (*endp) - die("Garbage after path in: %s", command_buf.buf); - p = uq.buf; - } - tree_content_get(root, p, &leaf, 1); + strbuf_reset(&path); + parse_path_eol(&path, p, "path"); + tree_content_get(root, path.buf, &leaf, 1); /* * A directory in preparation would have a sha1 of zero * until it is saved. Save, for simplicity. @@ -3186,7 +3192,7 @@ static void parse_ls(const char *p, struct branch *b) if (S_ISDIR(leaf.versions[1].mode)) store_tree(&leaf); - print_ls(leaf.versions[1].mode, leaf.versions[1].oid.hash, p); + print_ls(leaf.versions[1].mode, leaf.versions[1].oid.hash, path.buf); if (leaf.tree) release_tree_content_recursive(leaf.tree); if (!b || root != &b->branch_tree) diff --git a/config.mak.uname b/config.mak.uname index fcf3e2d785..a7607a5676 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -65,9 +65,9 @@ ifeq ($(uname_S),Linux) HAVE_PLATFORM_PROCINFO = YesPlease COMPAT_OBJS += compat/linux/procinfo.o # centos7/rhel7 provides gcc 4.8.5 and zlib 1.2.7. - ifneq ($(findstring .el7.,$(uname_R)),) + ifneq ($(findstring .el7.,$(uname_R)),) BASIC_CFLAGS += -std=c99 - endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease @@ -95,13 +95,13 @@ ifeq ($(uname_S),UnixWare) NO_MEMMEM = YesPlease endif ifeq ($(uname_S),SCO_SV) - ifeq ($(uname_R),3.2) + ifeq ($(uname_R),3.2) CFLAGS = -O2 - endif - ifeq ($(uname_R),5) + endif + ifeq ($(uname_R),5) CC = cc BASIC_CFLAGS += -Kthread - endif + endif NEEDS_SOCKET = YesPlease NEEDS_NSL = YesPlease NEEDS_SSL_WITH_CRYPTO = YesPlease @@ -124,19 +124,19 @@ ifeq ($(uname_S),Darwin) # - MacOS 10.0.* and MacOS 10.1.0 = Darwin 1.* # - MacOS 10.x.* = Darwin (x+4).* for (1 <= x) # i.e. "begins with [15678] and a dot" means "10.4.* or older". - ifeq ($(shell expr "$(uname_R)" : '[15678]\.'),2) + ifeq ($(shell expr "$(uname_R)" : '[15678]\.'),2) OLD_ICONV = UnfortunatelyYes NO_APPLE_COMMON_CRYPTO = YesPlease - endif - ifeq ($(shell expr "$(uname_R)" : '[15]\.'),2) + endif + ifeq ($(shell expr "$(uname_R)" : '[15]\.'),2) NO_STRLCPY = YesPlease - endif - ifeq ($(shell test "`expr "$(uname_R)" : '\([0-9][0-9]*\)\.'`" -ge 11 && echo 1),1) + endif + ifeq ($(shell test "`expr "$(uname_R)" : '\([0-9][0-9]*\)\.'`" -ge 11 && echo 1),1) HAVE_GETDELIM = YesPlease - endif - ifeq ($(shell test "`expr "$(uname_R)" : '\([0-9][0-9]*\)\.'`" -ge 20 && echo 1),1) + endif + ifeq ($(shell test "`expr "$(uname_R)" : '\([0-9][0-9]*\)\.'`" -ge 20 && echo 1),1) OPEN_RETURNS_EINTR = UnfortunatelyYes - endif + endif NO_MEMMEM = YesPlease USE_ST_TIMESPEC = YesPlease HAVE_DEV_TTY = YesPlease @@ -152,12 +152,12 @@ ifeq ($(uname_S),Darwin) # Workaround for `gettext` being keg-only and not even being linked via # `brew link --force gettext`, should be obsolete as of # https://github.com/Homebrew/homebrew-core/pull/53489 - ifeq ($(shell test -d /usr/local/opt/gettext/ && echo y),y) + ifeq ($(shell test -d /usr/local/opt/gettext/ && echo y),y) BASIC_CFLAGS += -I/usr/local/include -I/usr/local/opt/gettext/include BASIC_LDFLAGS += -L/usr/local/lib -L/usr/local/opt/gettext/lib - ifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y) + ifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y) MSGFMT = /usr/local/opt/gettext/bin/msgfmt - endif + endif # On newer ARM-based machines the default installation path has changed to # /opt/homebrew. Include it in our search paths so that the user does not # have to configure this manually. @@ -165,22 +165,22 @@ ifeq ($(uname_S),Darwin) # Note that we do not employ the same workaround as above where we manually # add gettext. The issue was fixed more than three years ago by now, and at # that point there haven't been any ARM-based Macs yet. - else ifeq ($(shell test -d /opt/homebrew/ && echo y),y) + else ifeq ($(shell test -d /opt/homebrew/ && echo y),y) BASIC_CFLAGS += -I/opt/homebrew/include BASIC_LDFLAGS += -L/opt/homebrew/lib - ifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y) + ifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y) MSGFMT = /opt/homebrew/bin/msgfmt - endif - endif + endif + endif # The builtin FSMonitor on MacOS builds upon Simple-IPC. Both require # Unix domain sockets and PThreads. - ifndef NO_PTHREADS - ifndef NO_UNIX_SOCKETS + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin FSMONITOR_OS_SETTINGS = darwin - endif - endif + endif + endif BASIC_LDFLAGS += -framework CoreServices endif @@ -196,7 +196,7 @@ ifeq ($(uname_S),SunOS) NO_REGEX = YesPlease NO_MSGFMT_EXTENDED_OPTIONS = YesPlease HAVE_DEV_TTY = YesPlease - ifeq ($(uname_R),5.6) + ifeq ($(uname_R),5.6) SOCKLEN_T = int NO_HSTRERROR = YesPlease NO_IPV6 = YesPlease @@ -206,8 +206,8 @@ ifeq ($(uname_S),SunOS) NO_STRLCPY = YesPlease NO_STRTOUMAX = YesPlease GIT_TEST_CMP = cmp - endif - ifeq ($(uname_R),5.7) + endif + ifeq ($(uname_R),5.7) NEEDS_RESOLV = YesPlease NO_IPV6 = YesPlease NO_SOCKADDR_STORAGE = YesPlease @@ -216,25 +216,25 @@ ifeq ($(uname_S),SunOS) NO_STRLCPY = YesPlease NO_STRTOUMAX = YesPlease GIT_TEST_CMP = cmp - endif - ifeq ($(uname_R),5.8) + endif + ifeq ($(uname_R),5.8) NO_UNSETENV = YesPlease NO_SETENV = YesPlease NO_STRTOUMAX = YesPlease GIT_TEST_CMP = cmp - endif - ifeq ($(uname_R),5.9) + endif + ifeq ($(uname_R),5.9) NO_UNSETENV = YesPlease NO_SETENV = YesPlease NO_STRTOUMAX = YesPlease GIT_TEST_CMP = cmp - endif + endif INSTALL = /usr/ucb/install TAR = gtar BASIC_CFLAGS += -D__EXTENSIONS__ -D__sun__ endif ifeq ($(uname_O),Cygwin) - ifeq ($(shell expr "$(uname_R)" : '1\.[1-6]\.'),4) + ifeq ($(shell expr "$(uname_R)" : '1\.[1-6]\.'),4) NO_D_TYPE_IN_DIRENT = YesPlease NO_STRCASESTR = YesPlease NO_MEMMEM = YesPlease @@ -245,9 +245,9 @@ ifeq ($(uname_O),Cygwin) # On some boxes NO_MMAP is needed, and not so elsewhere. # Try commenting this out if you suspect MMAP is more efficient NO_MMAP = YesPlease - else + else NO_REGEX = UnfortunatelyYes - endif + endif HAVE_ALLOCA_H = YesPlease NEEDS_LIBICONV = YesPlease NO_FAST_WORKING_DIRECTORY = UnfortunatelyYes @@ -263,25 +263,25 @@ ifeq ($(uname_S),FreeBSD) NEEDS_LIBICONV = YesPlease # Versions up to 10.1 require OLD_ICONV; 10.2 and beyond don't. # A typical version string looks like "10.2-RELEASE". - ifeq ($(shell expr "$(uname_R)" : '[1-9]\.'),2) + ifeq ($(shell expr "$(uname_R)" : '[1-9]\.'),2) OLD_ICONV = YesPlease - endif - ifeq ($(firstword $(subst -, ,$(uname_R))),10.0) + endif + ifeq ($(firstword $(subst -, ,$(uname_R))),10.0) OLD_ICONV = YesPlease - endif - ifeq ($(firstword $(subst -, ,$(uname_R))),10.1) + endif + ifeq ($(firstword $(subst -, ,$(uname_R))),10.1) OLD_ICONV = YesPlease - endif + endif NO_MEMMEM = YesPlease BASIC_CFLAGS += -I/usr/local/include BASIC_LDFLAGS += -L/usr/local/lib DIR_HAS_BSD_GROUP_SEMANTICS = YesPlease USE_ST_TIMESPEC = YesPlease - ifeq ($(shell expr "$(uname_R)" : '4\.'),2) + ifeq ($(shell expr "$(uname_R)" : '4\.'),2) PTHREAD_LIBS = -pthread NO_UINTMAX_T = YesPlease NO_STRTOUMAX = YesPlease - endif + endif PYTHON_PATH = /usr/local/bin/python PERL_PATH = /usr/local/bin/perl HAVE_PATHS_H = YesPlease @@ -317,9 +317,9 @@ ifeq ($(uname_S),MirBSD) CSPRNG_METHOD = arc4random endif ifeq ($(uname_S),NetBSD) - ifeq ($(shell expr "$(uname_R)" : '[01]\.'),2) + ifeq ($(shell expr "$(uname_R)" : '[01]\.'),2) NEEDS_LIBICONV = YesPlease - endif + endif BASIC_CFLAGS += -I/usr/pkg/include BASIC_LDFLAGS += -L/usr/pkg/lib $(CC_LD_DYNPATH)/usr/pkg/lib USE_ST_TIMESPEC = YesPlease @@ -343,14 +343,14 @@ ifeq ($(uname_S),AIX) BASIC_CFLAGS += -D_LARGE_FILES FILENO_IS_A_MACRO = UnfortunatelyYes NEED_ACCESS_ROOT_HANDLER = UnfortunatelyYes - ifeq ($(shell expr "$(uname_V)" : '[1234]'),1) + ifeq ($(shell expr "$(uname_V)" : '[1234]'),1) NO_PTHREADS = YesPlease - else + else PTHREAD_LIBS = -lpthread - endif - ifeq ($(shell expr "$(uname_V).$(uname_R)" : '5\.1'),3) + endif + ifeq ($(shell expr "$(uname_V).$(uname_R)" : '5\.1'),3) INLINE = '' - endif + endif GIT_TEST_CMP = cmp endif ifeq ($(uname_S),GNU) @@ -410,29 +410,29 @@ ifeq ($(uname_S),HP-UX) NO_SYS_SELECT_H = YesPlease SNPRINTF_RETURNS_BOGUS = YesPlease NO_NSEC = YesPlease - ifeq ($(uname_R),B.11.00) + ifeq ($(uname_R),B.11.00) NO_INET_NTOP = YesPlease NO_INET_PTON = YesPlease - endif - ifeq ($(uname_R),B.10.20) + endif + ifeq ($(uname_R),B.10.20) # Override HP-UX 11.x setting: INLINE = SOCKLEN_T = size_t NO_PREAD = YesPlease NO_INET_NTOP = YesPlease NO_INET_PTON = YesPlease - endif + endif GIT_TEST_CMP = cmp endif ifeq ($(uname_S),Windows) GIT_VERSION := $(GIT_VERSION).MSVC pathsep = ; # Assume that this is built in Git for Windows' SDK - ifeq (MINGW32,$(MSYSTEM)) + ifeq (MINGW32,$(MSYSTEM)) prefix = /mingw32 - else + else prefix = /mingw64 - endif + endif # Prepend MSVC 64-bit tool-chain to PATH. # # A regular Git Bash *does not* have cl.exe in its $PATH. As there is a @@ -549,16 +549,16 @@ ifeq ($(uname_S),Interix) NO_MKDTEMP = YesPlease NO_STRTOUMAX = YesPlease NO_NSEC = YesPlease - ifeq ($(uname_R),3.5) + ifeq ($(uname_R),3.5) NO_INET_NTOP = YesPlease NO_INET_PTON = YesPlease NO_SOCKADDR_STORAGE = YesPlease - endif - ifeq ($(uname_R),5.2) + endif + ifeq ($(uname_R),5.2) NO_INET_NTOP = YesPlease NO_INET_PTON = YesPlease NO_SOCKADDR_STORAGE = YesPlease - endif + endif endif ifeq ($(uname_S),Minix) NO_IPV6 = YesPlease @@ -578,12 +578,12 @@ ifeq ($(uname_S),NONSTOP_KERNEL) # still not compile in c89 mode, due to non-const array initializations. CC = cc -c99 # Build down-rev compatible objects that don't use our new getopt_long. - ifeq ($(uname_R).$(uname_V),J06.21) + ifeq ($(uname_R).$(uname_V),J06.21) CC += -WRVU=J06.20 - endif - ifeq ($(uname_R).$(uname_V),L17.02) + endif + ifeq ($(uname_R).$(uname_V),L17.02) CC += -WRVU=L16.05 - endif + endif # Disable all optimization, seems to result in bad code, with -O or -O2 # or even -O1 (default), /usr/local/libexec/git-core/git-pack-objects # abends on "git push". Needs more investigation. @@ -650,9 +650,9 @@ ifeq ($(uname_S),OS/390) NEEDS_MODE_TRANSLATION = YesPlease endif ifeq ($(uname_S),MINGW) - ifeq ($(shell expr "$(uname_R)" : '1\.'),2) + ifeq ($(shell expr "$(uname_R)" : '1\.'),2) $(error "Building with MSys is no longer supported") - endif + endif pathsep = ; HAVE_ALLOCA_H = YesPlease NO_PREAD = YesPlease @@ -710,22 +710,22 @@ ifeq ($(uname_S),MINGW) # Enable DEP BASIC_LDFLAGS += -Wl,--nxcompat # Enable ASLR (unless debugging) - ifneq (,$(findstring -O,$(filter-out -O0 -Og,$(CFLAGS)))) + ifneq (,$(findstring -O,$(filter-out -O0 -Og,$(CFLAGS)))) BASIC_LDFLAGS += -Wl,--dynamicbase - endif - ifeq (MINGW32,$(MSYSTEM)) + endif + ifeq (MINGW32,$(MSYSTEM)) prefix = /mingw32 HOST_CPU = i686 BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup - endif - ifeq (MINGW64,$(MSYSTEM)) + endif + ifeq (MINGW64,$(MSYSTEM)) prefix = /mingw64 HOST_CPU = x86_64 BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup - else + else COMPAT_CFLAGS += -D_USE_32BIT_TIME_T BASIC_LDFLAGS += -Wl,--large-address-aware - endif + endif CC = gcc COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \ -fstack-protector-strong @@ -737,11 +737,11 @@ ifeq ($(uname_S),MINGW) USE_GETTEXT_SCHEME = fallthrough USE_LIBPCRE = YesPlease USE_NED_ALLOCATOR = YesPlease - ifeq (/mingw64,$(subst 32,64,$(prefix))) + ifeq (/mingw64,$(subst 32,64,$(prefix))) # Move system config into top-level /etc/ ETC_GITCONFIG = ../etc/gitconfig ETC_GITATTRIBUTES = ../etc/gitattributes - endif + endif endif ifeq ($(uname_S),QNX) COMPAT_CFLAGS += -DSA_RESTART=0 diff --git a/contrib/credential/osxkeychain/Makefile b/contrib/credential/osxkeychain/Makefile index 4b3a08a2ba..238f5f8c36 100644 --- a/contrib/credential/osxkeychain/Makefile +++ b/contrib/credential/osxkeychain/Makefile @@ -8,7 +8,8 @@ CFLAGS = -g -O2 -Wall -include ../../../config.mak git-credential-osxkeychain: git-credential-osxkeychain.o - $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) -Wl,-framework -Wl,Security + $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) \ + -framework Security -framework CoreFoundation git-credential-osxkeychain.o: git-credential-osxkeychain.c $(CC) -c $(CFLAGS) $< diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c index 5f2e5f16c8..6a40917b1e 100644 --- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c +++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c @@ -3,14 +3,51 @@ #include <stdlib.h> #include <Security/Security.h> -static SecProtocolType protocol; -static char *host; -static char *path; -static char *username; -static char *password; -static UInt16 port; - -__attribute__((format (printf, 1, 2))) +#define ENCODING kCFStringEncodingUTF8 +static CFStringRef protocol; /* Stores constant strings - not memory managed */ +static CFStringRef host; +static CFNumberRef port; +static CFStringRef path; +static CFStringRef username; +static CFDataRef password; +static CFDataRef password_expiry_utc; +static CFDataRef oauth_refresh_token; + +static void clear_credential(void) +{ + if (host) { + CFRelease(host); + host = NULL; + } + if (port) { + CFRelease(port); + port = NULL; + } + if (path) { + CFRelease(path); + path = NULL; + } + if (username) { + CFRelease(username); + username = NULL; + } + if (password) { + CFRelease(password); + password = NULL; + } + if (password_expiry_utc) { + CFRelease(password_expiry_utc); + password_expiry_utc = NULL; + } + if (oauth_refresh_token) { + CFRelease(oauth_refresh_token); + oauth_refresh_token = NULL; + } +} + +#define STRING_WITH_LENGTH(s) s, sizeof(s) - 1 + +__attribute__((format (printf, 1, 2), __noreturn__)) static void die(const char *err, ...) { char msg[4096]; @@ -19,70 +56,199 @@ static void die(const char *err, ...) vsnprintf(msg, sizeof(msg), err, params); fprintf(stderr, "%s\n", msg); va_end(params); + clear_credential(); exit(1); } -static void *xstrdup(const char *s1) +static void *xmalloc(size_t len) { - void *ret = strdup(s1); + void *ret = malloc(len); if (!ret) die("Out of memory"); return ret; } -#define KEYCHAIN_ITEM(x) (x ? strlen(x) : 0), x -#define KEYCHAIN_ARGS \ - NULL, /* default keychain */ \ - KEYCHAIN_ITEM(host), \ - 0, NULL, /* account domain */ \ - KEYCHAIN_ITEM(username), \ - KEYCHAIN_ITEM(path), \ - port, \ - protocol, \ - kSecAuthenticationTypeDefault - -static void write_item(const char *what, const char *buf, int len) +static CFDictionaryRef create_dictionary(CFAllocatorRef allocator, ...) +{ + va_list args; + const void *key; + CFMutableDictionaryRef result; + + result = CFDictionaryCreateMutable(allocator, + 0, + &kCFTypeDictionaryKeyCallBacks, + &kCFTypeDictionaryValueCallBacks); + + + va_start(args, allocator); + while ((key = va_arg(args, const void *)) != NULL) { + const void *value; + value = va_arg(args, const void *); + if (value) + CFDictionarySetValue(result, key, value); + } + va_end(args); + + return result; +} + +#define CREATE_SEC_ATTRIBUTES(...) \ + create_dictionary(kCFAllocatorDefault, \ + kSecClass, kSecClassInternetPassword, \ + kSecAttrServer, host, \ + kSecAttrAccount, username, \ + kSecAttrPath, path, \ + kSecAttrPort, port, \ + kSecAttrProtocol, protocol, \ + kSecAttrAuthenticationType, \ + kSecAttrAuthenticationTypeDefault, \ + __VA_ARGS__); + +static void write_item(const char *what, const char *buf, size_t len) { printf("%s=", what); fwrite(buf, 1, len, stdout); putchar('\n'); } -static void find_username_in_item(SecKeychainItemRef item) +static void find_username_in_item(CFDictionaryRef item) { - SecKeychainAttributeList list; - SecKeychainAttribute attr; + CFStringRef account_ref; + char *username_buf; + CFIndex buffer_len; - list.count = 1; - list.attr = &attr; - attr.tag = kSecAccountItemAttr; + account_ref = CFDictionaryGetValue(item, kSecAttrAccount); + if (!account_ref) + { + write_item("username", "", 0); + return; + } - if (SecKeychainItemCopyContent(item, NULL, &list, NULL, NULL)) + username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING); + if (username_buf) + { + write_item("username", username_buf, strlen(username_buf)); return; + } - write_item("username", attr.data, attr.length); - SecKeychainItemFreeContent(&list, NULL); + /* If we can't get a CString pointer then + * we need to allocate our own buffer */ + buffer_len = CFStringGetMaximumSizeForEncoding( + CFStringGetLength(account_ref), ENCODING) + 1; + username_buf = xmalloc(buffer_len); + if (CFStringGetCString(account_ref, + username_buf, + buffer_len, + ENCODING)) { + write_item("username", username_buf, buffer_len - 1); + } + free(username_buf); } -static void find_internet_password(void) +static OSStatus find_internet_password(void) { - void *buf; - UInt32 len; - SecKeychainItemRef item; + CFDictionaryRef attrs; + CFDictionaryRef item; + CFDataRef data; + OSStatus result; - if (SecKeychainFindInternetPassword(KEYCHAIN_ARGS, &len, &buf, &item)) - return; + attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitOne, + kSecReturnAttributes, kCFBooleanTrue, + kSecReturnData, kCFBooleanTrue, + NULL); + result = SecItemCopyMatching(attrs, (CFTypeRef *)&item); + if (result) { + goto out; + } + + data = CFDictionaryGetValue(item, kSecValueData); - write_item("password", buf, len); + write_item("password", + (const char *)CFDataGetBytePtr(data), + CFDataGetLength(data)); if (!username) find_username_in_item(item); - SecKeychainItemFreeContent(NULL, buf); + CFRelease(item); + +out: + CFRelease(attrs); + + /* We consider not found to not be an error */ + if (result == errSecItemNotFound) + result = errSecSuccess; + + return result; +} + +static OSStatus delete_ref(const void *itemRef) +{ + CFArrayRef item_ref_list; + CFDictionaryRef delete_query; + OSStatus result; + + item_ref_list = CFArrayCreate(kCFAllocatorDefault, + &itemRef, + 1, + &kCFTypeArrayCallBacks); + delete_query = create_dictionary(kCFAllocatorDefault, + kSecClass, kSecClassInternetPassword, + kSecMatchItemList, item_ref_list, + NULL); + + if (password) { + /* We only want to delete items with a matching password */ + CFIndex capacity; + CFMutableDictionaryRef query; + CFDataRef data; + + capacity = CFDictionaryGetCount(delete_query) + 1; + query = CFDictionaryCreateMutableCopy(kCFAllocatorDefault, + capacity, + delete_query); + CFDictionarySetValue(query, kSecReturnData, kCFBooleanTrue); + result = SecItemCopyMatching(query, (CFTypeRef *)&data); + if (!result) { + CFDataRef kc_password; + const UInt8 *raw_data; + const UInt8 *line; + + /* Don't match appended metadata */ + raw_data = CFDataGetBytePtr(data); + line = memchr(raw_data, '\n', CFDataGetLength(data)); + if (line) + kc_password = CFDataCreateWithBytesNoCopy( + kCFAllocatorDefault, + raw_data, + line - raw_data, + kCFAllocatorNull); + else + kc_password = data; + + if (CFEqual(kc_password, password)) + result = SecItemDelete(delete_query); + + if (line) + CFRelease(kc_password); + CFRelease(data); + } + + CFRelease(query); + } else { + result = SecItemDelete(delete_query); + } + + CFRelease(delete_query); + CFRelease(item_ref_list); + + return result; } -static void delete_internet_password(void) +static OSStatus delete_internet_password(void) { - SecKeychainItemRef item; + CFDictionaryRef attrs; + CFArrayRef refs; + OSStatus result; /* * Require at least a protocol and host for removal, which is what git @@ -90,25 +256,69 @@ static void delete_internet_password(void) * Keychain manager. */ if (!protocol || !host) - return; + return -1; - if (SecKeychainFindInternetPassword(KEYCHAIN_ARGS, 0, NULL, &item)) - return; + attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitAll, + kSecReturnRef, kCFBooleanTrue, + NULL); + result = SecItemCopyMatching(attrs, (CFTypeRef *)&refs); + CFRelease(attrs); + + if (!result) { + for (CFIndex i = 0; !result && i < CFArrayGetCount(refs); i++) + result = delete_ref(CFArrayGetValueAtIndex(refs, i)); - SecKeychainItemDelete(item); + CFRelease(refs); + } + + /* We consider not found to not be an error */ + if (result == errSecItemNotFound) + result = errSecSuccess; + + return result; } -static void add_internet_password(void) +static OSStatus add_internet_password(void) { + CFMutableDataRef data; + CFDictionaryRef attrs; + OSStatus result; + /* Only store complete credentials */ if (!protocol || !host || !username || !password) - return; + return -1; - if (SecKeychainAddInternetPassword( - KEYCHAIN_ARGS, - KEYCHAIN_ITEM(password), - NULL)) - return; + data = CFDataCreateMutableCopy(kCFAllocatorDefault, 0, password); + if (password_expiry_utc) { + CFDataAppendBytes(data, + (const UInt8 *)STRING_WITH_LENGTH("\npassword_expiry_utc=")); + CFDataAppendBytes(data, + CFDataGetBytePtr(password_expiry_utc), + CFDataGetLength(password_expiry_utc)); + } + if (oauth_refresh_token) { + CFDataAppendBytes(data, + (const UInt8 *)STRING_WITH_LENGTH("\noauth_refresh_token=")); + CFDataAppendBytes(data, + CFDataGetBytePtr(oauth_refresh_token), + CFDataGetLength(oauth_refresh_token)); + } + + attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, data, + NULL); + + result = SecItemAdd(attrs, NULL); + if (result == errSecDuplicateItem) { + CFDictionaryRef query; + query = CREATE_SEC_ATTRIBUTES(NULL); + result = SecItemUpdate(query, attrs); + CFRelease(query); + } + + CFRelease(data); + CFRelease(attrs); + + return result; } static void read_credential(void) @@ -131,36 +341,60 @@ static void read_credential(void) if (!strcmp(buf, "protocol")) { if (!strcmp(v, "imap")) - protocol = kSecProtocolTypeIMAP; + protocol = kSecAttrProtocolIMAP; else if (!strcmp(v, "imaps")) - protocol = kSecProtocolTypeIMAPS; + protocol = kSecAttrProtocolIMAPS; else if (!strcmp(v, "ftp")) - protocol = kSecProtocolTypeFTP; + protocol = kSecAttrProtocolFTP; else if (!strcmp(v, "ftps")) - protocol = kSecProtocolTypeFTPS; + protocol = kSecAttrProtocolFTPS; else if (!strcmp(v, "https")) - protocol = kSecProtocolTypeHTTPS; + protocol = kSecAttrProtocolHTTPS; else if (!strcmp(v, "http")) - protocol = kSecProtocolTypeHTTP; + protocol = kSecAttrProtocolHTTP; else if (!strcmp(v, "smtp")) - protocol = kSecProtocolTypeSMTP; - else /* we don't yet handle other protocols */ + protocol = kSecAttrProtocolSMTP; + else { + /* we don't yet handle other protocols */ + clear_credential(); exit(0); + } } else if (!strcmp(buf, "host")) { char *colon = strchr(v, ':'); if (colon) { + UInt16 port_i; *colon++ = '\0'; - port = atoi(colon); + port_i = atoi(colon); + port = CFNumberCreate(kCFAllocatorDefault, + kCFNumberShortType, + &port_i); } - host = xstrdup(v); + host = CFStringCreateWithCString(kCFAllocatorDefault, + v, + ENCODING); } else if (!strcmp(buf, "path")) - path = xstrdup(v); + path = CFStringCreateWithCString(kCFAllocatorDefault, + v, + ENCODING); else if (!strcmp(buf, "username")) - username = xstrdup(v); + username = CFStringCreateWithCString( + kCFAllocatorDefault, + v, + ENCODING); else if (!strcmp(buf, "password")) - password = xstrdup(v); + password = CFDataCreate(kCFAllocatorDefault, + (UInt8 *)v, + strlen(v)); + else if (!strcmp(buf, "password_expiry_utc")) + password_expiry_utc = CFDataCreate(kCFAllocatorDefault, + (UInt8 *)v, + strlen(v)); + else if (!strcmp(buf, "oauth_refresh_token")) + oauth_refresh_token = CFDataCreate(kCFAllocatorDefault, + (UInt8 *)v, + strlen(v)); /* * Ignore other lines; we don't know what they mean, but * this future-proofs us when later versions of git do @@ -173,6 +407,7 @@ static void read_credential(void) int main(int argc, const char **argv) { + OSStatus result = 0; const char *usage = "usage: git credential-osxkeychain <get|store|erase>"; @@ -182,12 +417,17 @@ int main(int argc, const char **argv) read_credential(); if (!strcmp(argv[1], "get")) - find_internet_password(); + result = find_internet_password(); else if (!strcmp(argv[1], "store")) - add_internet_password(); + result = add_internet_password(); else if (!strcmp(argv[1], "erase")) - delete_internet_password(); + result = delete_internet_password(); /* otherwise, ignore unknown action */ + if (result) + die("failed to %s: %d", argv[1], (int)result); + + clear_credential(); + return 0; } @@ -207,13 +207,13 @@ void show_date_relative(timestamp_t time, struct strbuf *timebuf) (diff + 183) / 365); } -struct date_mode *date_mode_from_type(enum date_mode_type type) +struct date_mode date_mode_from_type(enum date_mode_type type) { - static struct date_mode mode = DATE_MODE_INIT; + struct date_mode mode = DATE_MODE_INIT; if (type == DATE_STRFTIME) BUG("cannot create anonymous strftime date_mode struct"); mode.type = type; - return &mode; + return mode; } static void show_date_normal(struct strbuf *buf, timestamp_t time, struct tm *tm, int tz, struct tm *human_tm, int human_tz, int local) @@ -283,7 +283,7 @@ static void show_date_normal(struct strbuf *buf, timestamp_t time, struct tm *tm strbuf_addf(buf, " %+05d", tz); } -const char *show_date(timestamp_t time, int tz, const struct date_mode *mode) +const char *show_date(timestamp_t time, int tz, struct date_mode mode) { struct tm *tm; struct tm tmbuf = { 0 }; @@ -291,13 +291,13 @@ const char *show_date(timestamp_t time, int tz, const struct date_mode *mode) int human_tz = -1; static struct strbuf timebuf = STRBUF_INIT; - if (mode->type == DATE_UNIX) { + if (mode.type == DATE_UNIX) { strbuf_reset(&timebuf); strbuf_addf(&timebuf, "%"PRItime, time); return timebuf.buf; } - if (mode->type == DATE_HUMAN) { + if (mode.type == DATE_HUMAN) { struct timeval now; get_time(&now); @@ -306,22 +306,22 @@ const char *show_date(timestamp_t time, int tz, const struct date_mode *mode) human_tz = local_time_tzoffset(now.tv_sec, &human_tm); } - if (mode->local) + if (mode.local) tz = local_tzoffset(time); - if (mode->type == DATE_RAW) { + if (mode.type == DATE_RAW) { strbuf_reset(&timebuf); strbuf_addf(&timebuf, "%"PRItime" %+05d", time, tz); return timebuf.buf; } - if (mode->type == DATE_RELATIVE) { + if (mode.type == DATE_RELATIVE) { strbuf_reset(&timebuf); show_date_relative(time, &timebuf); return timebuf.buf; } - if (mode->local) + if (mode.local) tm = time_to_tm_local(time, &tmbuf); else tm = time_to_tm(time, tz, &tmbuf); @@ -331,17 +331,17 @@ const char *show_date(timestamp_t time, int tz, const struct date_mode *mode) } strbuf_reset(&timebuf); - if (mode->type == DATE_SHORT) + if (mode.type == DATE_SHORT) strbuf_addf(&timebuf, "%04d-%02d-%02d", tm->tm_year + 1900, tm->tm_mon + 1, tm->tm_mday); - else if (mode->type == DATE_ISO8601) + else if (mode.type == DATE_ISO8601) strbuf_addf(&timebuf, "%04d-%02d-%02d %02d:%02d:%02d %+05d", tm->tm_year + 1900, tm->tm_mon + 1, tm->tm_mday, tm->tm_hour, tm->tm_min, tm->tm_sec, tz); - else if (mode->type == DATE_ISO8601_STRICT) { + else if (mode.type == DATE_ISO8601_STRICT) { strbuf_addf(&timebuf, "%04d-%02d-%02dT%02d:%02d:%02d", tm->tm_year + 1900, tm->tm_mon + 1, @@ -354,16 +354,16 @@ const char *show_date(timestamp_t time, int tz, const struct date_mode *mode) tz = abs(tz); strbuf_addf(&timebuf, "%02d:%02d", tz / 100, tz % 100); } - } else if (mode->type == DATE_RFC2822) + } else if (mode.type == DATE_RFC2822) strbuf_addf(&timebuf, "%.3s, %d %.3s %d %02d:%02d:%02d %+05d", weekday_names[tm->tm_wday], tm->tm_mday, month_names[tm->tm_mon], tm->tm_year + 1900, tm->tm_hour, tm->tm_min, tm->tm_sec, tz); - else if (mode->type == DATE_STRFTIME) - strbuf_addftime(&timebuf, mode->strftime_fmt, tm, tz, - !mode->local); + else if (mode.type == DATE_STRFTIME) + strbuf_addftime(&timebuf, mode.strftime_fmt, tm, tz, + !mode.local); else - show_date_normal(&timebuf, time, tm, tz, &human_tm, human_tz, mode->local); + show_date_normal(&timebuf, time, tm, tz, &human_tm, human_tz, mode.local); return timebuf.buf; } @@ -22,8 +22,8 @@ enum date_mode_type { struct date_mode { enum date_mode_type type; - const char *strftime_fmt; int local; + const char *strftime_fmt; }; #define DATE_MODE_INIT { \ @@ -36,14 +36,14 @@ struct date_mode { * show_date(t, tz, DATE_MODE(NORMAL)); */ #define DATE_MODE(t) date_mode_from_type(DATE_##t) -struct date_mode *date_mode_from_type(enum date_mode_type type); +struct date_mode date_mode_from_type(enum date_mode_type type); /** * Format <'time', 'timezone'> into static memory according to 'mode' * and return it. The mode is an initialized "struct date_mode" * (usually from the DATE_MODE() macro). */ -const char *show_date(timestamp_t time, int timezone, const struct date_mode *mode); +const char *show_date(timestamp_t time, int timezone, struct date_mode mode); /** * Parse a date format for later use with show_date(). @@ -104,16 +104,15 @@ static int launch_specified_editor(const char *editor, const char *path, sigchain_pop(SIGQUIT); if (sig == SIGINT || sig == SIGQUIT) raise(sig); - if (ret) - return error("There was a problem with the editor '%s'.", - editor); - if (print_waiting_for_editor && !is_terminal_dumb()) /* * Erase the entire line to avoid wasting the * vertical space. */ term_clear_line(); + if (ret) + return error("there was a problem with the editor '%s'", + editor); } if (!buffer) diff --git a/git-compat-util.h b/git-compat-util.h index 044f87454a..ca7678a379 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -403,6 +403,7 @@ char *gitdirname(char *); #ifndef NO_OPENSSL #ifdef __APPLE__ +#undef __AVAILABILITY_MACROS_USES_AVAILABILITY #define __AVAILABILITY_MACROS_USES_AVAILABILITY 0 #include <AvailabilityMacros.h> #undef DEPRECATED_ATTRIBUTE diff --git a/git-gui/.gitattributes b/git-gui/.gitattributes index 59cd41dbff..118d56cfbd 100644 --- a/git-gui/.gitattributes +++ b/git-gui/.gitattributes @@ -3,3 +3,4 @@ git-gui.sh encoding=UTF-8 /po/*.po encoding=UTF-8 /GIT-VERSION-GEN eol=lf +Makefile whitespace=!indent,trail,space diff --git a/git-gui/Makefile b/git-gui/Makefile index 3f80435436..667c39ed56 100644 --- a/git-gui/Makefile +++ b/git-gui/Makefile @@ -107,12 +107,12 @@ endif ifeq ($(uname_S),Darwin) TKFRAMEWORK = /Library/Frameworks/Tk.framework/Resources/Wish.app - ifeq ($(shell echo "$(uname_R)" | awk -F. '{if ($$1 >= 9) print "y"}')_$(shell test -d $(TKFRAMEWORK) || echo n),y_n) + ifeq ($(shell echo "$(uname_R)" | awk -F. '{if ($$1 >= 9) print "y"}')_$(shell test -d $(TKFRAMEWORK) || echo n),y_n) TKFRAMEWORK = /System/Library/Frameworks/Tk.framework/Resources/Wish.app - ifeq ($(shell test -d $(TKFRAMEWORK) || echo n),n) + ifeq ($(shell test -d $(TKFRAMEWORK) || echo n),n) TKFRAMEWORK = /System/Library/Frameworks/Tk.framework/Resources/Wish\ Shell.app - endif - endif + endif + endif TKEXECUTABLE = $(shell basename "$(TKFRAMEWORK)" .app) endif @@ -143,9 +143,9 @@ ifeq ($(exedir),$(gg_libdir)) endif gg_libdir_sed_in := $(gg_libdir) ifeq ($(uname_S),Darwin) - ifeq ($(shell test -d $(TKFRAMEWORK) && echo y),y) + ifeq ($(shell test -d $(TKFRAMEWORK) && echo y),y) GITGUI_MACOSXAPP := YesPlease - endif + endif endif ifneq (,$(findstring MINGW,$(uname_S))) ifeq ($(shell expr "$(uname_R)" : '1\.'),2) @@ -220,9 +220,9 @@ ifdef NO_MSGFMT MSGFMT ?= $(TCL_PATH) po/po2msg.sh else MSGFMT ?= msgfmt - ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $$?),0) + ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $$?),0) MSGFMT := $(TCL_PATH) po/po2msg.sh - endif + endif endif msgsdir = $(gg_libdir)/msgs diff --git a/gitk-git/Makefile b/gitk-git/Makefile index 5bdd52a6eb..e1f0aff4a1 100644 --- a/gitk-git/Makefile +++ b/gitk-git/Makefile @@ -33,9 +33,9 @@ ifdef NO_MSGFMT MSGFMT ?= $(TCL_PATH) po/po2msg.sh else MSGFMT ?= msgfmt - ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $$?),0) + ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $$?),0) MSGFMT := $(TCL_PATH) po/po2msg.sh - endif + endif endif PO_TEMPLATE = po/gitk.pot diff --git a/gpg-interface.c b/gpg-interface.c index b5993385ff..1ff94266d2 100644 --- a/gpg-interface.c +++ b/gpg-interface.c @@ -483,7 +483,7 @@ static int verify_ssh_signed_buffer(struct signature_check *sigc, if (sigc->payload_timestamp) strbuf_addf(&verify_time, "-Overify-time=%s", - show_date(sigc->payload_timestamp, 0, &verify_date_mode)); + show_date(sigc->payload_timestamp, 0, verify_date_mode)); /* Find the principal from the signers */ strvec_pushl(&ssh_keygen.args, fmt->program, diff --git a/imap-send.c b/imap-send.c index 4caa8668e6..0afd088d8a 100644 --- a/imap-send.c +++ b/imap-send.c @@ -68,20 +68,6 @@ static void imap_warn(const char *, ...); static char *next_arg(char **); -static int nfvasprintf(char **strp, const char *fmt, va_list ap) -{ - int len; - char tmp[8192]; - - len = vsnprintf(tmp, sizeof(tmp), fmt, ap); - if (len < 0) - die("Fatal: Out of memory"); - if (len >= sizeof(tmp)) - die("imap command overflow!"); - *strp = xmemdupz(tmp, len); - return len; -} - struct imap_server_conf { const char *name; const char *tunnel; @@ -503,11 +489,11 @@ static struct imap_cmd *issue_imap_cmd(struct imap_store *ctx, { struct imap *imap = ctx->imap; struct imap_cmd *cmd; - int n, bufl; - char buf[1024]; + int n; + struct strbuf buf = STRBUF_INIT; cmd = xmalloc(sizeof(struct imap_cmd)); - nfvasprintf(&cmd->cmd, fmt, ap); + cmd->cmd = xstrvfmt(fmt, ap); cmd->tag = ++imap->nexttag; if (cb) @@ -519,27 +505,30 @@ static struct imap_cmd *issue_imap_cmd(struct imap_store *ctx, get_cmd_result(ctx, NULL); if (!cmd->cb.data) - bufl = xsnprintf(buf, sizeof(buf), "%d %s\r\n", cmd->tag, cmd->cmd); + strbuf_addf(&buf, "%d %s\r\n", cmd->tag, cmd->cmd); else - bufl = xsnprintf(buf, sizeof(buf), "%d %s{%d%s}\r\n", - cmd->tag, cmd->cmd, cmd->cb.dlen, - CAP(LITERALPLUS) ? "+" : ""); + strbuf_addf(&buf, "%d %s{%d%s}\r\n", cmd->tag, cmd->cmd, + cmd->cb.dlen, CAP(LITERALPLUS) ? "+" : ""); + if (buf.len > INT_MAX) + die("imap command overflow!"); if (0 < verbosity) { if (imap->num_in_progress) printf("(%d in progress) ", imap->num_in_progress); if (!starts_with(cmd->cmd, "LOGIN")) - printf(">>> %s", buf); + printf(">>> %s", buf.buf); else printf(">>> %d LOGIN <user> <pass>\n", cmd->tag); } - if (socket_write(&imap->buf.sock, buf, bufl) != bufl) { + if (socket_write(&imap->buf.sock, buf.buf, buf.len) != buf.len) { free(cmd->cmd); free(cmd); if (cb) free(cb->data); + strbuf_release(&buf); return NULL; } + strbuf_release(&buf); if (cmd->cb.data) { if (CAP(LITERALPLUS)) { n = socket_write(&imap->buf.sock, cmd->cb.data, cmd->cb.dlen); diff --git a/log-tree.c b/log-tree.c index 59eeaef1f7..16031b44e7 100644 --- a/log-tree.c +++ b/log-tree.c @@ -773,7 +773,7 @@ void show_log(struct rev_info *opt) */ show_reflog_message(opt->reflog_info, opt->commit_format == CMIT_FMT_ONELINE, - &opt->date_mode, + opt->date_mode, opt->date_mode_explicit); if (opt->commit_format == CMIT_FMT_ONELINE) return; @@ -170,9 +170,10 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local pair_chunk(cf, MIDX_CHUNKID_LARGEOFFSETS, &m->chunk_large_offsets, &m->chunk_large_offsets_len); - pair_chunk(cf, MIDX_CHUNKID_BITMAPPEDPACKS, - (const unsigned char **)&m->chunk_bitmapped_packs, - &m->chunk_bitmapped_packs_len); + if (git_env_bool("GIT_TEST_MIDX_READ_BTMP", 1)) + pair_chunk(cf, MIDX_CHUNKID_BITMAPPEDPACKS, + (const unsigned char **)&m->chunk_bitmapped_packs, + &m->chunk_bitmapped_packs_len); if (git_env_bool("GIT_TEST_MIDX_READ_RIDX", 1)) pair_chunk(cf, MIDX_CHUNKID_REVINDEX, &m->chunk_revindex, diff --git a/oss-fuzz/fuzz-date.c b/oss-fuzz/fuzz-date.c index 036378b946..9619dae40e 100644 --- a/oss-fuzz/fuzz-date.c +++ b/oss-fuzz/fuzz-date.c @@ -11,7 +11,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) int16_t tz; timestamp_t ts; enum date_mode_type dmtype; - struct date_mode *dm; + struct date_mode dm; if (size <= 4) /* @@ -40,10 +40,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) free(str); dm = date_mode_from_type(dmtype); - dm->local = local; + dm.local = local; show_date(ts, (int)tz, dm); - date_mode_release(dm); + date_mode_release(&dm); return 0; } diff --git a/pack-bitmap.c b/pack-bitmap.c index 2baeabacee..35c5ef9d3c 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -2049,7 +2049,10 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git, load_reverse_index(r, bitmap_git); - if (bitmap_is_midx(bitmap_git)) { + if (!bitmap_is_midx(bitmap_git) || !bitmap_git->midx->chunk_bitmapped_packs) + multi_pack_reuse = 0; + + if (multi_pack_reuse) { for (i = 0; i < bitmap_git->midx->num_packs; i++) { struct bitmapped_pack pack; if (nth_bitmapped_pack(r, bitmap_git->midx, &pack, i) < 0) { @@ -2062,34 +2065,32 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git, if (!pack.bitmap_nr) continue; - if (!multi_pack_reuse && pack.bitmap_pos) { - /* - * If we're only reusing a single pack, skip - * over any packs which are not positioned at - * the beginning of the MIDX bitmap. - * - * This is consistent with the existing - * single-pack reuse behavior, which only reuses - * parts of the MIDX's preferred pack. - */ - continue; - } - ALLOC_GROW(packs, packs_nr + 1, packs_alloc); memcpy(&packs[packs_nr++], &pack, sizeof(pack)); objects_nr += pack.p->num_objects; - - if (!multi_pack_reuse) - break; } QSORT(packs, packs_nr, bitmapped_pack_cmp); } else { - ALLOC_GROW(packs, packs_nr + 1, packs_alloc); + struct packed_git *pack; + + if (bitmap_is_midx(bitmap_git)) { + uint32_t preferred_pack_pos; + + if (midx_preferred_pack(bitmap_git->midx, &preferred_pack_pos) < 0) { + warning(_("unable to compute preferred pack, disabling pack-reuse")); + return; + } - packs[packs_nr].p = bitmap_git->pack; - packs[packs_nr].bitmap_nr = bitmap_git->pack->num_objects; + pack = bitmap_git->midx->packs[preferred_pack_pos]; + } else { + pack = bitmap_git->pack; + } + + ALLOC_GROW(packs, packs_nr + 1, packs_alloc); + packs[packs_nr].p = pack; + packs[packs_nr].bitmap_nr = pack->num_objects; packs[packs_nr].bitmap_pos = 0; objects_nr = packs[packs_nr++].bitmap_nr; @@ -20,7 +20,7 @@ # clone | klon(lamak) # # commit (ad) | işleme # # commit (eyl.) | işlemek # -# commitish | işlememsi # +# commit-ish | işlememsi # # conflict | çakışma # # cruft | süprüntü # # dangling object | sallanan nesne # @@ -428,7 +428,7 @@ static void add_rfc2047(struct strbuf *sb, const char *line, size_t len, } const char *show_ident_date(const struct ident_split *ident, - const struct date_mode *mode) + struct date_mode mode) { timestamp_t date = 0; long tz = 0; @@ -592,7 +592,7 @@ void pp_user_info(struct pretty_print_context *pp, switch (pp->fmt) { case CMIT_FMT_MEDIUM: strbuf_addf(sb, "Date: %s\n", - show_ident_date(&ident, &pp->date_mode)); + show_ident_date(&ident, pp->date_mode)); break; case CMIT_FMT_EMAIL: case CMIT_FMT_MBOXRD: @@ -601,7 +601,7 @@ void pp_user_info(struct pretty_print_context *pp, break; case CMIT_FMT_FULLER: strbuf_addf(sb, "%sDate: %s\n", what, - show_ident_date(&ident, &pp->date_mode)); + show_ident_date(&ident, pp->date_mode)); break; default: /* notin' */ @@ -775,7 +775,7 @@ static int mailmap_name(const char **email, size_t *email_len, static size_t format_person_part(struct strbuf *sb, char part, const char *msg, int len, - const struct date_mode *dmode) + struct date_mode dmode) { /* currently all placeholders have same length */ const int placeholder_len = 2; @@ -1034,7 +1034,7 @@ static void rewrap_message_tail(struct strbuf *sb, static int format_reflog_person(struct strbuf *sb, char part, struct reflog_walk_info *log, - const struct date_mode *dmode) + struct date_mode dmode) { const char *ident; @@ -1602,7 +1602,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */ if (c->pretty_ctx->reflog_info) get_reflog_selector(sb, c->pretty_ctx->reflog_info, - &c->pretty_ctx->date_mode, + c->pretty_ctx->date_mode, c->pretty_ctx->date_mode_explicit, (placeholder[1] == 'd')); return 2; @@ -1617,7 +1617,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */ return format_reflog_person(sb, placeholder[1], c->pretty_ctx->reflog_info, - &c->pretty_ctx->date_mode); + c->pretty_ctx->date_mode); } return 0; /* unknown %g placeholder */ case 'N': @@ -1712,11 +1712,11 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */ case 'a': /* author ... */ return format_person_part(sb, placeholder[1], msg + c->author.off, c->author.len, - &c->pretty_ctx->date_mode); + c->pretty_ctx->date_mode); case 'c': /* committer ... */ return format_person_part(sb, placeholder[1], msg + c->committer.off, c->committer.len, - &c->pretty_ctx->date_mode); + c->pretty_ctx->date_mode); case 'e': /* encoding */ if (c->commit_encoding) strbuf_addstr(sb, c->commit_encoding); @@ -167,7 +167,7 @@ int format_set_trailers_options(struct process_trailer_options *opts, * a well-known sentinel date if they appear bogus. */ const char *show_ident_date(const struct ident_split *id, - const struct date_mode *mode); + struct date_mode mode); #endif /* PRETTY_H */ diff --git a/ref-filter.c b/ref-filter.c index 03542d0236..59ad6f54dd 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -1627,7 +1627,7 @@ static void grab_date(const char *buf, struct atom_value *v, const char *atomnam tz = strtol(zone, NULL, 10); if ((tz == LONG_MIN || tz == LONG_MAX) && errno == ERANGE) goto bad; - v->s = xstrdup(show_date(timestamp, tz, &date_mode)); + v->s = xstrdup(show_date(timestamp, tz, date_mode)); v->value = timestamp; date_mode_release(&date_mode); return; diff --git a/reflog-walk.c b/reflog-walk.c index d216f6f966..66484f4f32 100644 --- a/reflog-walk.c +++ b/reflog-walk.c @@ -223,7 +223,7 @@ int add_reflog_for_walk(struct reflog_walk_info *info, void get_reflog_selector(struct strbuf *sb, struct reflog_walk_info *reflog_info, - const struct date_mode *dmode, int force_date, + struct date_mode dmode, int force_date, int shorten) { struct commit_reflog *commit_reflog = reflog_info->last_commit_reflog; @@ -297,7 +297,7 @@ timestamp_t get_reflog_timestamp(struct reflog_walk_info *reflog_info) } void show_reflog_message(struct reflog_walk_info *reflog_info, int oneline, - const struct date_mode *dmode, int force_date) + struct date_mode dmode, int force_date) { if (reflog_info && reflog_info->last_commit_reflog) { struct commit_reflog *commit_reflog = reflog_info->last_commit_reflog; diff --git a/reflog-walk.h b/reflog-walk.h index 4d93a26957..989583dc55 100644 --- a/reflog-walk.h +++ b/reflog-walk.h @@ -10,14 +10,14 @@ void reflog_walk_info_release(struct reflog_walk_info *info); int add_reflog_for_walk(struct reflog_walk_info *info, struct commit *commit, const char *name); void show_reflog_message(struct reflog_walk_info *info, int, - const struct date_mode *, int force_date); + struct date_mode, int force_date); void get_reflog_message(struct strbuf *sb, struct reflog_walk_info *reflog_info); const char *get_reflog_ident(struct reflog_walk_info *reflog_info); timestamp_t get_reflog_timestamp(struct reflog_walk_info *reflog_info); void get_reflog_selector(struct strbuf *sb, struct reflog_walk_info *reflog_info, - const struct date_mode *dmode, int force_date, + struct date_mode dmode, int force_date, int shorten); int reflog_walk_empty(struct reflog_walk_info *walk); diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 0bed6d2ab4..1cda48c504 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -18,6 +18,7 @@ #include "../reftable/reftable-merged.h" #include "../setup.h" #include "../strmap.h" +#include "parse.h" #include "refs-internal.h" /* @@ -247,6 +248,8 @@ static struct ref_store *reftable_be_init(struct repository *repo, refs->write_options.block_size = 4096; refs->write_options.hash_id = repo->hash_algo->format_id; refs->write_options.default_permissions = calc_shared_perm(0666 & ~mask); + refs->write_options.disable_auto_compact = + !git_env_bool("GIT_TEST_REFTABLE_AUTOCOMPACTION", 1); /* * Set up the main reftable stack that is hosted in GIT_COMMON_DIR. diff --git a/reftable/block.c b/reftable/block.c index 298e8c56b9..3e87460cba 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -175,11 +175,6 @@ int block_writer_finish(struct block_writer *w) return w->next; } -uint8_t block_reader_type(struct block_reader *r) -{ - return r->block.data[r->header_off]; -} - int block_reader_init(struct block_reader *br, struct reftable_block *block, uint32_t header_off, uint32_t table_block_size, int hash_size) @@ -191,7 +186,8 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, uint16_t restart_count = 0; uint32_t restart_start = 0; uint8_t *restart_bytes = NULL; - uint8_t *uncompressed = NULL; + + reftable_block_done(&br->block); if (!reftable_is_block_type(typ)) { err = REFTABLE_FORMAT_ERROR; @@ -199,37 +195,57 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, } if (typ == BLOCK_TYPE_LOG) { - int block_header_skip = 4 + header_off; - uLongf dst_len = sz - block_header_skip; /* total size of dest - buffer. */ - uLongf src_len = block->len - block_header_skip; + uint32_t block_header_skip = 4 + header_off; + uLong dst_len = sz - block_header_skip; + uLong src_len = block->len - block_header_skip; /* Log blocks specify the *uncompressed* size in their header. */ - REFTABLE_ALLOC_ARRAY(uncompressed, sz); + REFTABLE_ALLOC_GROW(br->uncompressed_data, sz, + br->uncompressed_cap); /* Copy over the block header verbatim. It's not compressed. */ - memcpy(uncompressed, block->data, block_header_skip); + memcpy(br->uncompressed_data, block->data, block_header_skip); - /* Uncompress */ - if (Z_OK != - uncompress2(uncompressed + block_header_skip, &dst_len, - block->data + block_header_skip, &src_len)) { + if (!br->zstream) { + REFTABLE_CALLOC_ARRAY(br->zstream, 1); + err = inflateInit(br->zstream); + } else { + err = inflateReset(br->zstream); + } + if (err != Z_OK) { err = REFTABLE_ZLIB_ERROR; goto done; } - if (dst_len + block_header_skip != sz) { + br->zstream->next_in = block->data + block_header_skip; + br->zstream->avail_in = src_len; + br->zstream->next_out = br->uncompressed_data + block_header_skip; + br->zstream->avail_out = dst_len; + + /* + * We know both input as well as output size, and we know that + * the sizes should never be bigger than `uInt_MAX` because + * blocks can at most be 16MB large. We can thus use `Z_FINISH` + * here to instruct zlib to inflate the data in one go, which + * is more efficient than using `Z_NO_FLUSH`. + */ + err = inflate(br->zstream, Z_FINISH); + if (err != Z_STREAM_END) { + err = REFTABLE_ZLIB_ERROR; + goto done; + } + err = 0; + + if (br->zstream->total_out + block_header_skip != sz) { err = REFTABLE_FORMAT_ERROR; goto done; } /* We're done with the input data. */ reftable_block_done(block); - block->data = uncompressed; - uncompressed = NULL; + block->data = br->uncompressed_data; block->len = sz; - block->source = malloc_block_source(); - full_block_size = src_len + block_header_skip; + full_block_size = src_len + block_header_skip - br->zstream->avail_in; } else if (full_block_size == 0) { full_block_size = sz; } else if (sz < full_block_size && sz < block->len && @@ -257,18 +273,52 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, br->restart_bytes = restart_bytes; done: - reftable_free(uncompressed); return err; } -static uint32_t block_reader_restart_offset(struct block_reader *br, int i) +void block_reader_release(struct block_reader *br) +{ + inflateEnd(br->zstream); + reftable_free(br->zstream); + reftable_free(br->uncompressed_data); + reftable_block_done(&br->block); +} + +uint8_t block_reader_type(const struct block_reader *r) +{ + return r->block.data[r->header_off]; +} + +int block_reader_first_key(const struct block_reader *br, struct strbuf *key) +{ + int off = br->header_off + 4, n; + struct string_view in = { + .buf = br->block.data + off, + .len = br->block_len - off, + }; + uint8_t extra = 0; + + strbuf_reset(key); + + n = reftable_decode_key(key, &extra, in); + if (n < 0) + return n; + if (!key->len) + return REFTABLE_FORMAT_ERROR; + + return 0; +} + +static uint32_t block_reader_restart_offset(const struct block_reader *br, int i) { return get_be24(br->restart_bytes + 3 * i); } -void block_reader_start(struct block_reader *br, struct block_iter *it) +void block_iter_seek_start(struct block_iter *it, const struct block_reader *br) { - it->br = br; + it->block = br->block.data; + it->block_len = br->block_len; + it->hash_size = br->hash_size; strbuf_reset(&it->last_key); it->next_off = br->header_off + 4; } @@ -276,7 +326,7 @@ void block_reader_start(struct block_reader *br, struct block_iter *it) struct restart_needle_less_args { int error; struct strbuf needle; - struct block_reader *reader; + const struct block_reader *reader; }; static int restart_needle_less(size_t idx, void *_args) @@ -315,25 +365,17 @@ static int restart_needle_less(size_t idx, void *_args) return args->needle.len < suffix_len; } -void block_iter_copy_from(struct block_iter *dest, struct block_iter *src) -{ - dest->br = src->br; - dest->next_off = src->next_off; - strbuf_reset(&dest->last_key); - strbuf_addbuf(&dest->last_key, &src->last_key); -} - int block_iter_next(struct block_iter *it, struct reftable_record *rec) { struct string_view in = { - .buf = it->br->block.data + it->next_off, - .len = it->br->block_len - it->next_off, + .buf = (unsigned char *) it->block + it->next_off, + .len = it->block_len - it->next_off, }; struct string_view start = in; uint8_t extra = 0; int n = 0; - if (it->next_off >= it->br->block_len) + if (it->next_off >= it->block_len) return 1; n = reftable_decode_key(&it->last_key, &extra, in); @@ -343,7 +385,7 @@ int block_iter_next(struct block_iter *it, struct reftable_record *rec) return REFTABLE_FORMAT_ERROR; string_view_consume(&in, n); - n = reftable_record_decode(rec, it->last_key, extra, in, it->br->hash_size, + n = reftable_record_decode(rec, it->last_key, extra, in, it->hash_size, &it->scratch); if (n < 0) return -1; @@ -353,29 +395,13 @@ int block_iter_next(struct block_iter *it, struct reftable_record *rec) return 0; } -int block_reader_first_key(struct block_reader *br, struct strbuf *key) +void block_iter_reset(struct block_iter *it) { - int off = br->header_off + 4, n; - struct string_view in = { - .buf = br->block.data + off, - .len = br->block_len - off, - }; - uint8_t extra = 0; - - strbuf_reset(key); - - n = reftable_decode_key(key, &extra, in); - if (n < 0) - return n; - if (!key->len) - return REFTABLE_FORMAT_ERROR; - - return 0; -} - -int block_iter_seek(struct block_iter *it, struct strbuf *want) -{ - return block_reader_seek(it->br, it, want); + strbuf_reset(&it->last_key); + it->next_off = 0; + it->block = NULL; + it->block_len = 0; + it->hash_size = 0; } void block_iter_close(struct block_iter *it) @@ -384,14 +410,13 @@ void block_iter_close(struct block_iter *it) strbuf_release(&it->scratch); } -int block_reader_seek(struct block_reader *br, struct block_iter *it, - struct strbuf *want) +int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, + struct strbuf *want) { struct restart_needle_less_args args = { .needle = *want, .reader = br, }; - struct block_iter next = BLOCK_ITER_INIT; struct reftable_record rec; int err = 0; size_t i; @@ -436,7 +461,9 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it, it->next_off = block_reader_restart_offset(br, i - 1); else it->next_off = br->header_off + 4; - it->br = br; + it->block = br->block.data; + it->block_len = br->block_len; + it->hash_size = br->hash_size; reftable_record_init(&rec, block_reader_type(br)); @@ -448,11 +475,13 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it, * far and then back up. */ while (1) { - block_iter_copy_from(&next, it); - err = block_iter_next(&next, &rec); + size_t prev_off = it->next_off; + + err = block_iter_next(it, &rec); if (err < 0) goto done; if (err > 0) { + it->next_off = prev_off; err = 0; goto done; } @@ -463,18 +492,23 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it, * record does not exist in the block and can thus abort early. * In case it is equal to the sought-after key we have found * the desired record. + * + * Note that we store the next record's key record directly in + * `last_key` without restoring the key of the preceding record + * in case we need to go one record back. This is safe to do as + * `block_iter_next()` would return the ref whose key is equal + * to `last_key` now, and naturally all keys share a prefix + * with themselves. */ reftable_record_key(&rec, &it->last_key); - if (strbuf_cmp(&it->last_key, want) >= 0) + if (strbuf_cmp(&it->last_key, want) >= 0) { + it->next_off = prev_off; goto done; - - block_iter_copy_from(it, &next); + } } done: - block_iter_close(&next); reftable_record_release(&rec); - return err; } diff --git a/reftable/block.h b/reftable/block.h index 47acc62c0a..ea4384a7e2 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -56,6 +56,8 @@ int block_writer_finish(struct block_writer *w); /* clears out internally allocated block_writer members. */ void block_writer_release(struct block_writer *bw); +struct z_stream; + /* Read a block. */ struct block_reader { /* offset of the block header; nonzero for the first block in a @@ -66,6 +68,11 @@ struct block_reader { struct reftable_block block; int hash_size; + /* Uncompressed data for log entries. */ + z_stream *zstream; + unsigned char *uncompressed_data; + size_t uncompressed_cap; + /* size of the data, excluding restart data. */ uint32_t block_len; uint8_t *restart_bytes; @@ -76,11 +83,26 @@ struct block_reader { uint32_t full_block_size; }; +/* initializes a block reader. */ +int block_reader_init(struct block_reader *br, struct reftable_block *bl, + uint32_t header_off, uint32_t table_block_size, + int hash_size); + +void block_reader_release(struct block_reader *br); + +/* Returns the block type (eg. 'r' for refs) */ +uint8_t block_reader_type(const struct block_reader *r); + +/* Decodes the first key in the block */ +int block_reader_first_key(const struct block_reader *br, struct strbuf *key); + /* Iterate over entries in a block */ struct block_iter { /* offset within the block of the next entry to read. */ uint32_t next_off; - struct block_reader *br; + const unsigned char *block; + size_t block_len; + int hash_size; /* key for last entry we read. */ struct strbuf last_key; @@ -92,31 +114,18 @@ struct block_iter { .scratch = STRBUF_INIT, \ } -/* initializes a block reader. */ -int block_reader_init(struct block_reader *br, struct reftable_block *bl, - uint32_t header_off, uint32_t table_block_size, - int hash_size); - /* Position `it` at start of the block */ -void block_reader_start(struct block_reader *br, struct block_iter *it); +void block_iter_seek_start(struct block_iter *it, const struct block_reader *br); /* Position `it` to the `want` key in the block */ -int block_reader_seek(struct block_reader *br, struct block_iter *it, - struct strbuf *want); - -/* Returns the block type (eg. 'r' for refs) */ -uint8_t block_reader_type(struct block_reader *r); - -/* Decodes the first key in the block */ -int block_reader_first_key(struct block_reader *br, struct strbuf *key); - -void block_iter_copy_from(struct block_iter *dest, struct block_iter *src); +int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, + struct strbuf *want); /* return < 0 for error, 0 for OK, > 0 for EOF. */ int block_iter_next(struct block_iter *it, struct reftable_record *rec); -/* Seek to `want` with in the block pointed to by `it` */ -int block_iter_seek(struct block_iter *it, struct strbuf *want); +/* Reset the block iterator to pristine state without releasing its memory. */ +void block_iter_reset(struct block_iter *it); /* deallocate memory for `it`. The block reader and its block is left intact. */ void block_iter_close(struct block_iter *it); diff --git a/reftable/block_test.c b/reftable/block_test.c index e162c6e33f..26a9cfbc83 100644 --- a/reftable/block_test.c +++ b/reftable/block_test.c @@ -69,7 +69,7 @@ static void test_block_read_write(void) block_reader_init(&br, &block, header_off, block_size, GIT_SHA1_RAWSZ); - block_reader_start(&br, &it); + block_iter_seek_start(&it, &br); while (1) { int r = block_iter_next(&it, &rec); @@ -89,7 +89,7 @@ static void test_block_read_write(void) strbuf_reset(&want); strbuf_addstr(&want, names[i]); - n = block_reader_seek(&br, &it, &want); + n = block_iter_seek_key(&it, &br, &want); EXPECT(n == 0); n = block_iter_next(&it, &rec); @@ -98,7 +98,7 @@ static void test_block_read_write(void) EXPECT_STREQ(names[i], rec.u.ref.refname); want.len--; - n = block_reader_seek(&br, &it, &want); + n = block_iter_seek_key(&it, &br, &want); EXPECT(n == 0); n = block_iter_next(&it, &rec); diff --git a/reftable/iter.c b/reftable/iter.c index 7aa30c4a51..aa9ac199b1 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -115,7 +115,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) /* indexed block does not exist. */ return REFTABLE_FORMAT_ERROR; } - block_reader_start(&it->block_reader, &it->cur); + block_iter_seek_start(&it->cur, &it->block_reader); return 0; } diff --git a/reftable/reader.c b/reftable/reader.c index b113daab77..481dff10d4 100644 --- a/reftable/reader.c +++ b/reftable/reader.c @@ -220,6 +220,7 @@ struct table_iter { struct reftable_reader *r; uint8_t typ; uint64_t block_off; + struct block_reader br; struct block_iter bi; int is_finished; }; @@ -227,16 +228,6 @@ struct table_iter { .bi = BLOCK_ITER_INIT \ } -static void table_iter_copy_from(struct table_iter *dest, - struct table_iter *src) -{ - dest->r = src->r; - dest->typ = src->typ; - dest->block_off = src->block_off; - dest->is_finished = src->is_finished; - block_iter_copy_from(&dest->bi, &src->bi); -} - static int table_iter_next_in_block(struct table_iter *ti, struct reftable_record *rec) { @@ -250,14 +241,8 @@ static int table_iter_next_in_block(struct table_iter *ti, static void table_iter_block_done(struct table_iter *ti) { - if (!ti->bi.br) { - return; - } - reftable_block_done(&ti->bi.br->block); - FREE_AND_NULL(ti->bi.br); - - ti->bi.last_key.len = 0; - ti->bi.next_off = 0; + block_reader_release(&ti->br); + block_iter_reset(&ti->bi); } static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off, @@ -321,32 +306,27 @@ done: return err; } -static int table_iter_next_block(struct table_iter *dest, - struct table_iter *src) +static void table_iter_close(struct table_iter *ti) { - uint64_t next_block_off = src->block_off + src->bi.br->full_block_size; - struct block_reader br = { 0 }; - int err = 0; + table_iter_block_done(ti); + block_iter_close(&ti->bi); +} - dest->r = src->r; - dest->typ = src->typ; - dest->block_off = next_block_off; +static int table_iter_next_block(struct table_iter *ti) +{ + uint64_t next_block_off = ti->block_off + ti->br.full_block_size; + int err; - err = reader_init_block_reader(src->r, &br, next_block_off, src->typ); - if (err > 0) { - dest->is_finished = 1; - return 1; - } - if (err != 0) + err = reader_init_block_reader(ti->r, &ti->br, next_block_off, ti->typ); + if (err > 0) + ti->is_finished = 1; + if (err) return err; - else { - struct block_reader *brp = - reftable_malloc(sizeof(struct block_reader)); - *brp = br; - dest->is_finished = 0; - block_reader_start(brp, &dest->bi); - } + ti->block_off = next_block_off; + ti->is_finished = 0; + block_iter_seek_start(&ti->bi, &ti->br); + return 0; } @@ -356,7 +336,6 @@ static int table_iter_next(struct table_iter *ti, struct reftable_record *rec) return REFTABLE_API_ERROR; while (1) { - struct table_iter next = TABLE_ITER_INIT; int err; if (ti->is_finished) @@ -376,15 +355,11 @@ static int table_iter_next(struct table_iter *ti, struct reftable_record *rec) * table and retry. If there are no more blocks then the * iterator is drained. */ - err = table_iter_next_block(&next, ti); - table_iter_block_done(ti); + err = table_iter_next_block(ti); if (err) { ti->is_finished = 1; return err; } - - table_iter_copy_from(ti, &next); - block_iter_close(&next.bi); } } @@ -393,16 +368,14 @@ static int table_iter_next_void(void *ti, struct reftable_record *rec) return table_iter_next(ti, rec); } -static void table_iter_close(void *p) +static void table_iter_close_void(void *ti) { - struct table_iter *ti = p; - table_iter_block_done(ti); - block_iter_close(&ti->bi); + table_iter_close(ti); } static struct reftable_iterator_vtable table_iter_vtable = { .next = &table_iter_next_void, - .close = &table_iter_close, + .close = &table_iter_close_void, }; static void iterator_from_table_iter(struct reftable_iterator *it, @@ -417,19 +390,16 @@ static int reader_table_iter_at(struct reftable_reader *r, struct table_iter *ti, uint64_t off, uint8_t typ) { - struct block_reader br = { 0 }; - struct block_reader *brp = NULL; + int err; - int err = reader_init_block_reader(r, &br, off, typ); + err = reader_init_block_reader(r, &ti->br, off, typ); if (err != 0) return err; - brp = reftable_malloc(sizeof(struct block_reader)); - *brp = br; ti->r = r; - ti->typ = block_reader_type(brp); + ti->typ = block_reader_type(&ti->br); ti->block_off = off; - block_reader_start(brp, &ti->bi); + block_iter_seek_start(&ti->bi, &ti->br); return 0; } @@ -454,23 +424,52 @@ static int reader_seek_linear(struct table_iter *ti, { struct strbuf want_key = STRBUF_INIT; struct strbuf got_key = STRBUF_INIT; - struct table_iter next = TABLE_ITER_INIT; struct reftable_record rec; int err = -1; reftable_record_init(&rec, reftable_record_type(want)); reftable_record_key(want, &want_key); + /* + * First we need to locate the block that must contain our record. To + * do so we scan through blocks linearly until we find the first block + * whose first key is bigger than our wanted key. Once we have found + * that block we know that the key must be contained in the preceding + * block. + * + * This algorithm is somewhat unfortunate because it means that we + * always have to seek one block too far and then back up. But as we + * can only decode the _first_ key of a block but not its _last_ key we + * have no other way to do this. + */ while (1) { - err = table_iter_next_block(&next, ti); + struct table_iter next = *ti; + + /* + * We must be careful to not modify underlying data of `ti` + * because we may find that `next` does not contain our desired + * block, but that `ti` does. In that case, we would discard + * `next` and continue with `ti`. + * + * This also means that we cannot reuse allocated memory for + * `next` here. While it would be great if we could, it should + * in practice not be too bad given that we should only ever + * end up doing linear seeks with at most three blocks. As soon + * as we have more than three blocks we would have an index, so + * we would not do a linear search there anymore. + */ + memset(&next.br.block, 0, sizeof(next.br.block)); + next.br.zstream = NULL; + next.br.uncompressed_data = NULL; + next.br.uncompressed_cap = 0; + + err = table_iter_next_block(&next); if (err < 0) goto done; - - if (err > 0) { + if (err > 0) break; - } - err = block_reader_first_key(next.bi.br, &got_key); + err = block_reader_first_key(&next.br, &got_key); if (err < 0) goto done; @@ -480,16 +479,20 @@ static int reader_seek_linear(struct table_iter *ti, } table_iter_block_done(ti); - table_iter_copy_from(ti, &next); + *ti = next; } - err = block_iter_seek(&ti->bi, &want_key); + /* + * We have located the block that must contain our record, so we seek + * the wanted key inside of it. If the block does not contain our key + * we know that the corresponding record does not exist. + */ + err = block_iter_seek_key(&ti->bi, &ti->br, &want_key); if (err < 0) goto done; err = 0; done: - block_iter_close(&next.bi); reftable_record_release(&rec); strbuf_release(&want_key); strbuf_release(&got_key); @@ -508,6 +511,7 @@ static int reader_seek_indexed(struct reftable_reader *r, .u.idx = { .last_key = STRBUF_INIT }, }; struct table_iter index_iter = TABLE_ITER_INIT; + struct table_iter empty = TABLE_ITER_INIT; struct table_iter next = TABLE_ITER_INIT; int err = 0; @@ -549,7 +553,6 @@ static int reader_seek_indexed(struct reftable_reader *r, * not exist. */ err = table_iter_next(&index_iter, &index_result); - table_iter_block_done(&index_iter); if (err != 0) goto done; @@ -558,7 +561,7 @@ static int reader_seek_indexed(struct reftable_reader *r, if (err != 0) goto done; - err = block_iter_seek(&next.bi, &want_index.u.idx.last_key); + err = block_iter_seek_key(&next.bi, &next.br, &want_index.u.idx.last_key); if (err < 0) goto done; @@ -572,18 +575,20 @@ static int reader_seek_indexed(struct reftable_reader *r, break; } - table_iter_copy_from(&index_iter, &next); + table_iter_close(&index_iter); + index_iter = next; + next = empty; } if (err == 0) { - struct table_iter empty = TABLE_ITER_INIT; struct table_iter *malloced = reftable_calloc(1, sizeof(*malloced)); - *malloced = empty; - table_iter_copy_from(malloced, &next); + *malloced = next; + next = empty; iterator_from_table_iter(it, malloced); } + done: - block_iter_close(&next.bi); + table_iter_close(&next); table_iter_close(&index_iter); reftable_record_release(&want_index); reftable_record_release(&index_result); @@ -597,25 +602,28 @@ static int reader_seek_internal(struct reftable_reader *r, struct reftable_reader_offsets *offs = reader_offsets_for(r, reftable_record_type(rec)); uint64_t idx = offs->index_offset; - struct table_iter ti = TABLE_ITER_INIT; - int err = 0; + struct table_iter ti = TABLE_ITER_INIT, *p; + int err; + if (idx > 0) return reader_seek_indexed(r, it, rec); err = reader_start(r, &ti, reftable_record_type(rec), 0); if (err < 0) - return err; + goto out; + err = reader_seek_linear(&ti, rec); if (err < 0) - return err; - else { - struct table_iter *p = - reftable_malloc(sizeof(struct table_iter)); - *p = ti; - iterator_from_table_iter(it, p); - } + goto out; - return 0; + REFTABLE_ALLOC_ARRAY(p, 1); + *p = ti; + iterator_from_table_iter(it, p); + +out: + if (err) + table_iter_close(&ti); + return err; } static int reader_seek(struct reftable_reader *r, struct reftable_iterator *it, diff --git a/reftable/reftable-writer.h b/reftable/reftable-writer.h index 7c7cae5f99..155bf0bbe2 100644 --- a/reftable/reftable-writer.h +++ b/reftable/reftable-writer.h @@ -46,6 +46,9 @@ struct reftable_write_options { * is a single line, and add '\n' if missing. */ unsigned exact_log_message : 1; + + /* boolean: Prevent auto-compaction of tables. */ + unsigned disable_auto_compact : 1; }; /* reftable_block_stats holds statistics for a single block type */ diff --git a/reftable/stack.c b/reftable/stack.c index dde50b61d6..80266bcbab 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -680,7 +680,7 @@ int reftable_addition_commit(struct reftable_addition *add) if (err) goto done; - if (!add->stack->disable_auto_compact) { + if (!add->stack->config.disable_auto_compact) { /* * Auto-compact the stack to keep the number of tables in * control. It is possible that a concurrent writer is already @@ -1216,75 +1216,76 @@ static int segment_size(struct segment *s) return s->end - s->start; } -int fastlog2(uint64_t sz) -{ - int l = 0; - if (sz == 0) - return 0; - for (; sz; sz /= 2) { - l++; - } - return l - 1; -} - -struct segment *sizes_to_segments(size_t *seglen, uint64_t *sizes, size_t n) -{ - struct segment *segs = reftable_calloc(n, sizeof(*segs)); - struct segment cur = { 0 }; - size_t next = 0, i; - - if (n == 0) { - *seglen = 0; - return segs; - } - for (i = 0; i < n; i++) { - int log = fastlog2(sizes[i]); - if (cur.log != log && cur.bytes > 0) { - struct segment fresh = { - .start = i, - }; - - segs[next++] = cur; - cur = fresh; - } - - cur.log = log; - cur.end = i + 1; - cur.bytes += sizes[i]; - } - segs[next++] = cur; - *seglen = next; - return segs; -} - struct segment suggest_compaction_segment(uint64_t *sizes, size_t n) { - struct segment min_seg = { - .log = 64, - }; - struct segment *segs; - size_t seglen = 0, i; - - segs = sizes_to_segments(&seglen, sizes, n); - for (i = 0; i < seglen; i++) { - if (segment_size(&segs[i]) == 1) - continue; + struct segment seg = { 0 }; + uint64_t bytes; + size_t i; - if (segs[i].log < min_seg.log) - min_seg = segs[i]; - } + /* + * If there are no tables or only a single one then we don't have to + * compact anything. The sequence is geometric by definition already. + */ + if (n <= 1) + return seg; - while (min_seg.start > 0) { - size_t prev = min_seg.start - 1; - if (fastlog2(min_seg.bytes) < fastlog2(sizes[prev])) + /* + * Find the ending table of the compaction segment needed to restore the + * geometric sequence. Note that the segment end is exclusive. + * + * To do so, we iterate backwards starting from the most recent table + * until a valid segment end is found. If the preceding table is smaller + * than the current table multiplied by the geometric factor (2), the + * compaction segment end has been identified. + * + * Tables after the ending point are not added to the byte count because + * they are already valid members of the geometric sequence. Due to the + * properties of a geometric sequence, it is not possible for the sum of + * these tables to exceed the value of the ending point table. + * + * Example table size sequence requiring no compaction: + * 64, 32, 16, 8, 4, 2, 1 + * + * Example table size sequence where compaction segment end is set to + * the last table. Since the segment end is exclusive, the last table is + * excluded during subsequent compaction and the table with size 3 is + * the final table included: + * 64, 32, 16, 8, 4, 3, 1 + */ + for (i = n - 1; i > 0; i--) { + if (sizes[i - 1] < sizes[i] * 2) { + seg.end = i + 1; + bytes = sizes[i]; break; + } + } - min_seg.start = prev; - min_seg.bytes += sizes[prev]; + /* + * Find the starting table of the compaction segment by iterating + * through the remaining tables and keeping track of the accumulated + * size of all tables seen from the segment end table. The previous + * table is compared to the accumulated size because the tables from the + * segment end are merged backwards recursively. + * + * Note that we keep iterating even after we have found the first + * starting point. This is because there may be tables in the stack + * preceding that first starting point which violate the geometric + * sequence. + * + * Example compaction segment start set to table with size 32: + * 128, 32, 16, 8, 4, 3, 1 + */ + for (; i > 0; i--) { + uint64_t curr = bytes; + bytes += sizes[i - 1]; + + if (sizes[i - 1] < curr * 2) { + seg.start = i - 1; + seg.bytes = bytes; + } } - reftable_free(segs); - return min_seg; + return seg; } static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st) diff --git a/reftable/stack.h b/reftable/stack.h index d919455669..d43efa4760 100644 --- a/reftable/stack.h +++ b/reftable/stack.h @@ -19,7 +19,6 @@ struct reftable_stack { int list_fd; char *reftable_dir; - int disable_auto_compact; struct reftable_write_options config; @@ -33,12 +32,9 @@ int read_lines(const char *filename, char ***lines); struct segment { size_t start, end; - int log; uint64_t bytes; }; -int fastlog2(uint64_t sz); -struct segment *sizes_to_segments(size_t *seglen, uint64_t *sizes, size_t n); struct segment suggest_compaction_segment(uint64_t *sizes, size_t n); #endif diff --git a/reftable/stack_test.c b/reftable/stack_test.c index 351e35bd86..1df3ffce52 100644 --- a/reftable/stack_test.c +++ b/reftable/stack_test.c @@ -325,7 +325,7 @@ static void test_reftable_stack_transaction_api_performs_auto_compaction(void) * we can ensure that we indeed honor this setting and have * better control over when exactly auto compaction runs. */ - st->disable_auto_compact = i != n; + st->config.disable_auto_compact = i != n; err = reftable_stack_new_addition(&add, st); EXPECT_ERR(err); @@ -497,6 +497,7 @@ static void test_reftable_stack_add(void) struct reftable_write_options cfg = { .exact_log_message = 1, .default_permissions = 0660, + .disable_auto_compact = 1, }; struct reftable_stack *st = NULL; char *dir = get_tmp_dir(__LINE__); @@ -508,7 +509,6 @@ static void test_reftable_stack_add(void) err = reftable_new_stack(&st, dir, cfg); EXPECT_ERR(err); - st->disable_auto_compact = 1; for (i = 0; i < N; i++) { char buf[256]; @@ -770,59 +770,13 @@ static void test_reftable_stack_hash_id(void) clear_dir(dir); } -static void test_log2(void) -{ - EXPECT(1 == fastlog2(3)); - EXPECT(2 == fastlog2(4)); - EXPECT(2 == fastlog2(5)); -} - -static void test_sizes_to_segments(void) -{ - uint64_t sizes[] = { 2, 3, 4, 5, 7, 9 }; - /* .................0 1 2 3 4 5 */ - - size_t seglen = 0; - struct segment *segs = - sizes_to_segments(&seglen, sizes, ARRAY_SIZE(sizes)); - EXPECT(segs[2].log == 3); - EXPECT(segs[2].start == 5); - EXPECT(segs[2].end == 6); - - EXPECT(segs[1].log == 2); - EXPECT(segs[1].start == 2); - EXPECT(segs[1].end == 5); - reftable_free(segs); -} - -static void test_sizes_to_segments_empty(void) -{ - size_t seglen = 0; - struct segment *segs = sizes_to_segments(&seglen, NULL, 0); - EXPECT(seglen == 0); - reftable_free(segs); -} - -static void test_sizes_to_segments_all_equal(void) -{ - uint64_t sizes[] = { 5, 5 }; - size_t seglen = 0; - struct segment *segs = - sizes_to_segments(&seglen, sizes, ARRAY_SIZE(sizes)); - EXPECT(seglen == 1); - EXPECT(segs[0].start == 0); - EXPECT(segs[0].end == 2); - reftable_free(segs); -} - static void test_suggest_compaction_segment(void) { - uint64_t sizes[] = { 128, 64, 17, 16, 9, 9, 9, 16, 16 }; - /* .................0 1 2 3 4 5 6 */ + uint64_t sizes[] = { 512, 64, 17, 16, 9, 9, 9, 16, 2, 16 }; struct segment min = suggest_compaction_segment(sizes, ARRAY_SIZE(sizes)); - EXPECT(min.start == 2); - EXPECT(min.end == 7); + EXPECT(min.start == 1); + EXPECT(min.end == 10); } static void test_suggest_compaction_segment_nothing(void) @@ -933,9 +887,21 @@ static void test_empty_add(void) reftable_stack_destroy(st2); } +static int fastlog2(uint64_t sz) +{ + int l = 0; + if (sz == 0) + return 0; + for (; sz; sz /= 2) + l++; + return l - 1; +} + static void test_reftable_stack_auto_compaction(void) { - struct reftable_write_options cfg = { 0 }; + struct reftable_write_options cfg = { + .disable_auto_compact = 1, + }; struct reftable_stack *st = NULL; char *dir = get_tmp_dir(__LINE__); @@ -945,7 +911,6 @@ static void test_reftable_stack_auto_compaction(void) err = reftable_new_stack(&st, dir, cfg); EXPECT_ERR(err); - st->disable_auto_compact = 1; /* call manually below for coverage. */ for (i = 0; i < N; i++) { char name[100]; struct reftable_ref_record ref = { @@ -994,7 +959,7 @@ static void test_reftable_stack_add_performs_auto_compaction(void) * we can ensure that we indeed honor this setting and have * better control over when exactly auto compaction runs. */ - st->disable_auto_compact = i != n; + st->config.disable_auto_compact = i != n; strbuf_reset(&refname); strbuf_addf(&refname, "branch-%04d", i); @@ -1121,7 +1086,6 @@ static void test_reftable_stack_compaction_concurrent_clean(void) int stack_test_main(int argc, const char *argv[]) { RUN_TEST(test_empty_add); - RUN_TEST(test_log2); RUN_TEST(test_names_equal); RUN_TEST(test_parse_names); RUN_TEST(test_read_file); @@ -1142,9 +1106,6 @@ int stack_test_main(int argc, const char *argv[]) RUN_TEST(test_reftable_stack_update_index_check); RUN_TEST(test_reftable_stack_uptodate); RUN_TEST(test_reftable_stack_validate_refname); - RUN_TEST(test_sizes_to_segments); - RUN_TEST(test_sizes_to_segments_all_equal); - RUN_TEST(test_sizes_to_segments_empty); RUN_TEST(test_suggest_compaction_segment); RUN_TEST(test_suggest_compaction_segment_nothing); return 0; @@ -219,6 +219,11 @@ static void read_rr(struct repository *r, struct string_list *rr) buf.buf[hexsz] = '\0'; id = new_rerere_id_hex(buf.buf); id->variant = variant; + /* + * make sure id->collection->status has enough space + * for the variant we are interested in + */ + fit_variant(id->collection, variant); string_list_insert(rr, path)->util = id; } strbuf_release(&buf); diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl index dd8107cd7d..b2b28c2ced 100755 --- a/t/check-non-portable-shell.pl +++ b/t/check-non-portable-shell.pl @@ -47,6 +47,8 @@ while (<>) { /\bgrep\b.*--file\b/ and err 'grep --file FILE is not portable (use grep -f FILE)'; /\b[ef]grep\b/ and err 'egrep/fgrep obsolescent (use grep -E/-F)'; /\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)'; + /\blocal\s+[A-Za-z0-9_]*=\$([A-Za-z0-9_{]|[(][^(])/ and + err q(quote "$val" in 'local var=$val'); /^\s*([A-Z0-9_]+=(\w*|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and err '"FOO=bar shell_func" assignment extends beyond "shell_func"'; $line = ''; diff --git a/t/helper/test-date.c b/t/helper/test-date.c index 0683d46574..f25512de9a 100644 --- a/t/helper/test-date.c +++ b/t/helper/test-date.c @@ -52,7 +52,7 @@ static void show_dates(const char **argv, const char *format) arg++; tz = atoi(arg); - printf("%s -> %s\n", *argv, show_date(t, tz, &mode)); + printf("%s -> %s\n", *argv, show_date(t, tz, mode)); } date_mode_release(&mode); diff --git a/t/lib-parallel-checkout.sh b/t/lib-parallel-checkout.sh index acaee9cbb6..8324d6c96d 100644 --- a/t/lib-parallel-checkout.sh +++ b/t/lib-parallel-checkout.sh @@ -20,7 +20,7 @@ test_checkout_workers () { BUG "too few arguments to test_checkout_workers" fi && - local expected_workers=$1 && + local expected_workers="$1" && shift && local trace_file=trace-test-checkout-workers && diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh index 38a535b041..178791e086 100755 --- a/t/t0610-reftable-basics.sh +++ b/t/t0610-reftable-basics.sh @@ -83,7 +83,7 @@ test_expect_success 'init: reinitializing reftable with files backend fails' ' test_expect_perms () { local perms="$1" local file="$2" - local actual=$(ls -l "$file") && + local actual="$(ls -l "$file")" && case "$actual" in $perms*) @@ -121,7 +121,7 @@ test_expect_reftable_perms () { umask $umask && git init --shared=$shared repo && test_commit -C repo A && - test_line_count = 3 repo/.git/reftable/tables.list && + test_line_count = 2 repo/.git/reftable/tables.list && git -C repo pack-refs ) && test_expect_perms "$expect" repo/.git/reftable/tables.list && @@ -324,12 +324,46 @@ test_expect_success 'ref transaction: writes cause auto-compaction' ' test_line_count = 1 repo/.git/reftable/tables.list && test_commit -C repo --no-tag A && - test_line_count = 2 repo/.git/reftable/tables.list && + test_line_count = 1 repo/.git/reftable/tables.list && test_commit -C repo --no-tag B && test_line_count = 1 repo/.git/reftable/tables.list ' +test_expect_success 'ref transaction: env var disables compaction' ' + test_when_finished "rm -rf repo" && + + git init repo && + test_commit -C repo A && + + start=$(wc -l <repo/.git/reftable/tables.list) && + iterations=5 && + expected=$((start + iterations)) && + + for i in $(test_seq $iterations) + do + GIT_TEST_REFTABLE_AUTOCOMPACTION=false \ + git -C repo update-ref branch-$i HEAD || return 1 + done && + test_line_count = $expected repo/.git/reftable/tables.list && + + git -C repo update-ref foo HEAD && + test_line_count -lt $expected repo/.git/reftable/tables.list +' + +test_expect_success 'ref transaction: alternating table sizes are compacted' ' + test_when_finished "rm -rf repo" && + + git init repo && + test_commit -C repo A && + for i in $(test_seq 5) + do + git -C repo branch -f foo && + git -C repo branch -d foo || return 1 + done && + test_line_count = 2 repo/.git/reftable/tables.list +' + check_fsync_events () { local trace="$1" && shift && @@ -355,7 +389,7 @@ test_expect_success 'ref transaction: writes are synced' ' git -C repo -c core.fsync=reference \ -c core.fsyncMethod=fsync update-ref refs/heads/branch HEAD && check_fsync_events trace2.txt <<-EOF - "name":"hardware-flush","count":2 + "name":"hardware-flush","count":4 EOF ' @@ -387,7 +421,7 @@ test_expect_success 'ref transaction: fails gracefully when auto compaction fail done || exit 1 done && - test_line_count = 13 .git/reftable/tables.list + test_line_count = 10 .git/reftable/tables.list ) ' @@ -397,8 +431,8 @@ test_expect_success 'pack-refs: compacts tables' ' test_commit -C repo A && ls -1 repo/.git/reftable >table-files && - test_line_count = 4 table-files && - test_line_count = 3 repo/.git/reftable/tables.list && + test_line_count = 3 table-files && + test_line_count = 2 repo/.git/reftable/tables.list && git -C repo pack-refs && ls -1 repo/.git/reftable >table-files && @@ -439,7 +473,7 @@ test_expect_success "$command: auto compaction" ' # The tables should have been auto-compacted, and thus auto # compaction should not have to do anything. ls -1 .git/reftable >tables-expect && - test_line_count = 4 tables-expect && + test_line_count = 3 tables-expect && git $command --auto && ls -1 .git/reftable >tables-actual && test_cmp tables-expect tables-actual && @@ -457,7 +491,7 @@ test_expect_success "$command: auto compaction" ' git branch B && git branch C && rm .git/reftable/*.lock && - test_line_count = 5 .git/reftable/tables.list && + test_line_count = 4 .git/reftable/tables.list && git $command --auto && test_line_count = 1 .git/reftable/tables.list @@ -837,12 +871,16 @@ test_expect_success 'worktree: pack-refs in main repo packs main refs' ' test_when_finished "rm -rf repo worktree" && git init repo && test_commit -C repo A && + + GIT_TEST_REFTABLE_AUTOCOMPACTION=false \ git -C repo worktree add ../worktree && + GIT_TEST_REFTABLE_AUTOCOMPACTION=false \ + git -C worktree update-ref refs/worktree/per-worktree HEAD && - test_line_count = 3 repo/.git/worktrees/worktree/reftable/tables.list && - test_line_count = 4 repo/.git/reftable/tables.list && + test_line_count = 4 repo/.git/worktrees/worktree/reftable/tables.list && + test_line_count = 3 repo/.git/reftable/tables.list && git -C repo pack-refs && - test_line_count = 3 repo/.git/worktrees/worktree/reftable/tables.list && + test_line_count = 4 repo/.git/worktrees/worktree/reftable/tables.list && test_line_count = 1 repo/.git/reftable/tables.list ' @@ -850,13 +888,17 @@ test_expect_success 'worktree: pack-refs in worktree packs worktree refs' ' test_when_finished "rm -rf repo worktree" && git init repo && test_commit -C repo A && + + GIT_TEST_REFTABLE_AUTOCOMPACTION=false \ git -C repo worktree add ../worktree && + GIT_TEST_REFTABLE_AUTOCOMPACTION=false \ + git -C worktree update-ref refs/worktree/per-worktree HEAD && - test_line_count = 3 repo/.git/worktrees/worktree/reftable/tables.list && - test_line_count = 4 repo/.git/reftable/tables.list && + test_line_count = 4 repo/.git/worktrees/worktree/reftable/tables.list && + test_line_count = 3 repo/.git/reftable/tables.list && git -C worktree pack-refs && test_line_count = 1 repo/.git/worktrees/worktree/reftable/tables.list && - test_line_count = 4 repo/.git/reftable/tables.list + test_line_count = 3 repo/.git/reftable/tables.list ' test_expect_success 'worktree: creating shared ref updates main stack' ' @@ -870,6 +912,7 @@ test_expect_success 'worktree: creating shared ref updates main stack' ' test_line_count = 1 repo/.git/worktrees/worktree/reftable/tables.list && test_line_count = 1 repo/.git/reftable/tables.list && + GIT_TEST_REFTABLE_AUTOCOMPACTION=false \ git -C worktree update-ref refs/heads/shared HEAD && test_line_count = 1 repo/.git/worktrees/worktree/reftable/tables.list && test_line_count = 2 repo/.git/reftable/tables.list diff --git a/t/t1016-compatObjectFormat.sh b/t/t1016-compatObjectFormat.sh index 8132cd37b8..be3206a16f 100755 --- a/t/t1016-compatObjectFormat.sh +++ b/t/t1016-compatObjectFormat.sh @@ -79,7 +79,7 @@ commit2_oid () { } del_sigcommit () { - local delete=$1 + local delete="$1" if test "$delete" = "sha256" ; then local pattern="gpgsig-sha256" @@ -91,8 +91,8 @@ del_sigcommit () { del_sigtag () { - local storage=$1 - local delete=$2 + local storage="$1" + local delete="$2" if test "$storage" = "$delete" ; then local pattern="trailer" @@ -181,7 +181,7 @@ done cd "$base" compare_oids () { - test "$#" = 5 && { local PREREQ=$1; shift; } || PREREQ= + test "$#" = 5 && { local PREREQ="$1"; shift; } || PREREQ= local type="$1" local name="$2" local sha1_oid="$3" @@ -193,8 +193,8 @@ compare_oids () { git --git-dir=repo-sha1/.git rev-parse --output-object-format=sha256 ${sha1_oid} > ${name}_sha1_sha256_found git --git-dir=repo-sha256/.git rev-parse --output-object-format=sha1 ${sha256_oid} > ${name}_sha256_sha1_found - local sha1_sha256_oid=$(cat ${name}_sha1_sha256_found) - local sha256_sha1_oid=$(cat ${name}_sha256_sha1_found) + local sha1_sha256_oid="$(cat ${name}_sha1_sha256_found)" + local sha256_sha1_oid="$(cat ${name}_sha256_sha1_found)" test_expect_success $PREREQ "Verify ${type} ${name}'s sha1 oid" ' git --git-dir=repo-sha256/.git rev-parse --output-object-format=sha1 ${sha256_oid} > ${name}_sha1 && diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index c28c04133c..ba320dc417 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -427,7 +427,7 @@ test_expect_success '"add" worktree with orphan branch, lock, and reason' ' # Note: Quoted arguments containing spaces are not supported. test_wt_add_orphan_hint () { local context="$1" && - local use_branch=$2 && + local use_branch="$2" && shift 2 && local opts="$*" && test_expect_success "'worktree add' show orphan hint in bad/orphan HEAD w/ $context" ' diff --git a/t/t3428-rebase-signoff.sh b/t/t3428-rebase-signoff.sh index e1b1e94764..1bebd1ce74 100755 --- a/t/t3428-rebase-signoff.sh +++ b/t/t3428-rebase-signoff.sh @@ -8,47 +8,45 @@ This test runs git rebase --signoff and make sure that it works. TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh -# A simple file to commit -cat >file <<EOF -a -EOF +test_expect_success 'setup' ' + git commit --allow-empty -m "Initial empty commit" && + test_commit first file a && + + ident="$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" && -# Expected commit message for initial commit after rebase --signoff -cat >expected-initial-signed <<EOF -Initial empty commit + # Expected commit message for initial commit after rebase --signoff + cat >expected-initial-signed <<-EOF && + Initial empty commit -Signed-off-by: $(git var GIT_COMMITTER_IDENT | sed -e "s/>.*/>/") -EOF + Signed-off-by: $ident + EOF -# Expected commit message after rebase --signoff -cat >expected-signed <<EOF -first + # Expected commit message after rebase --signoff + cat >expected-signed <<-EOF && + first -Signed-off-by: $(git var GIT_COMMITTER_IDENT | sed -e "s/>.*/>/") -EOF + Signed-off-by: $ident + EOF -# Expected commit message after rebase without --signoff (or with --no-signoff) -cat >expected-unsigned <<EOF -first -EOF + # Expected commit message after rebase without --signoff (or with --no-signoff) + cat >expected-unsigned <<-EOF && + first + EOF + git config alias.rbs "rebase --signoff" +' # We configure an alias to do the rebase --signoff so that # on the next subtest we can show that --no-signoff overrides the alias -test_expect_success 'rebase --signoff adds a sign-off line' ' - git commit --allow-empty -m "Initial empty commit" && - git add file && git commit -m first && - git config alias.rbs "rebase --signoff" && - git rbs HEAD^ && - git cat-file commit HEAD | sed -e "1,/^\$/d" > actual && - test_cmp expected-signed actual +test_expect_success 'rebase --apply --signoff adds a sign-off line' ' + git rbs --apply HEAD^ && + test_commit_message HEAD expected-signed ' test_expect_success 'rebase --no-signoff does not add a sign-off line' ' git commit --amend -m "first" && git rbs --no-signoff HEAD^ && - git cat-file commit HEAD | sed -e "1,/^\$/d" > actual && - test_cmp expected-unsigned actual + test_commit_message HEAD expected-unsigned ' test_expect_success 'rebase --exec --signoff adds a sign-off line' ' @@ -56,30 +54,25 @@ test_expect_success 'rebase --exec --signoff adds a sign-off line' ' git commit --amend -m "first" && git rebase --exec "touch exec" --signoff HEAD^ && test_path_is_file exec && - git cat-file commit HEAD | sed -e "1,/^\$/d" >actual && - test_cmp expected-signed actual + test_commit_message HEAD expected-signed ' test_expect_success 'rebase --root --signoff adds a sign-off line' ' git commit --amend -m "first" && git rebase --root --keep-empty --signoff && - git cat-file commit HEAD^ | sed -e "1,/^\$/d" >actual && - test_cmp expected-initial-signed actual && - git cat-file commit HEAD | sed -e "1,/^\$/d" >actual && - test_cmp expected-signed actual + test_commit_message HEAD^ expected-initial-signed && + test_commit_message HEAD expected-signed ' test_expect_success 'rebase -i --signoff fails' ' git commit --amend -m "first" && git rebase -i --signoff HEAD^ && - git cat-file commit HEAD | sed -e "1,/^\$/d" >actual && - test_cmp expected-signed actual + test_commit_message HEAD expected-signed ' test_expect_success 'rebase -m --signoff fails' ' git commit --amend -m "first" && git rebase -m --signoff HEAD^ && - git cat-file commit HEAD | sed -e "1,/^\$/d" >actual && - test_cmp expected-signed actual + test_commit_message HEAD expected-signed ' test_done diff --git a/t/t4011-diff-symlink.sh b/t/t4011-diff-symlink.sh index d7a5f7ae78..bc8ba88719 100755 --- a/t/t4011-diff-symlink.sh +++ b/t/t4011-diff-symlink.sh @@ -13,13 +13,13 @@ TEST_PASSES_SANITIZE_LEAK=true # Print the short OID of a symlink with the given name. symlink_oid () { - local oid=$(printf "%s" "$1" | git hash-object --stdin) && + local oid="$(printf "%s" "$1" | git hash-object --stdin)" && git rev-parse --short "$oid" } # Print the short OID of the given file. short_oid () { - local oid=$(git hash-object "$1") && + local oid="$(git hash-object "$1")" && git rev-parse --short "$oid" } diff --git a/t/t4018/csharp-exclude-assignments b/t/t4018/csharp-exclude-assignments new file mode 100644 index 0000000000..239f312963 --- /dev/null +++ b/t/t4018/csharp-exclude-assignments @@ -0,0 +1,20 @@ +class Example +{ + string Method(int RIGHT) + { + var constantAssignment = "test"; + var methodAssignment = MethodCall(); + var multiLineMethodAssignment = MethodCall( + ); + var multiLine = "first" + + MethodCall() + + + ( MethodCall() + ) + + MethodCall(); + + return "ChangeMe"; + } + + string MethodCall(int a = 0, int b = 0) => "test"; +} diff --git a/t/t4018/csharp-exclude-control-statements b/t/t4018/csharp-exclude-control-statements new file mode 100644 index 0000000000..3a0f404ee1 --- /dev/null +++ b/t/t4018/csharp-exclude-control-statements @@ -0,0 +1,34 @@ +class Example +{ + string Method(int RIGHT) + { + if (false) + { + return "out"; + } + else { } + if (true) MethodCall( + ); + else MethodCall( + ); + switch ("test") + { + case "one": + return MethodCall( + ); + case "two": + break; + } + (int, int) tuple = (1, 4); + switch (tuple) + { + case (1, 4): + MethodCall(); + break; + } + + return "ChangeMe"; + } + + string MethodCall(int a = 0, int b = 0) => "test"; +} diff --git a/t/t4018/csharp-exclude-exceptions b/t/t4018/csharp-exclude-exceptions new file mode 100644 index 0000000000..b1e64256cf --- /dev/null +++ b/t/t4018/csharp-exclude-exceptions @@ -0,0 +1,29 @@ +using System; + +class Example +{ + string Method(int RIGHT) + { + try + { + throw new Exception("fail"); + } + catch (Exception) + { + } + finally + { + } + try { } catch (Exception) {} + try + { + throw GetException( + ); + } + catch (Exception) { } + + return "ChangeMe"; + } + + Exception GetException() => new Exception("fail"); +} diff --git a/t/t4018/csharp-exclude-generic-method-calls b/t/t4018/csharp-exclude-generic-method-calls new file mode 100644 index 0000000000..31af546665 --- /dev/null +++ b/t/t4018/csharp-exclude-generic-method-calls @@ -0,0 +1,12 @@ +class Example +{ + string Method(int RIGHT) + { + GenericMethodCall<int, int>( + ); + + return "ChangeMe"; + } + + string GenericMethodCall<T, T2>() => "test"; +} diff --git a/t/t4018/csharp-exclude-init-dispose b/t/t4018/csharp-exclude-init-dispose new file mode 100644 index 0000000000..2bc8e194e2 --- /dev/null +++ b/t/t4018/csharp-exclude-init-dispose @@ -0,0 +1,22 @@ +using System; + +class Example : IDisposable +{ + string Method(int RIGHT) + { + new Example(); + new Example( + ); + new Example { }; + using (this) + { + } + var def = + this is default( + Example); + + return "ChangeMe"; + } + + public void Dispose() {} +} diff --git a/t/t4018/csharp-exclude-iterations b/t/t4018/csharp-exclude-iterations new file mode 100644 index 0000000000..960aa182ae --- /dev/null +++ b/t/t4018/csharp-exclude-iterations @@ -0,0 +1,26 @@ +using System.Linq; + +class Example +{ + string Method(int RIGHT) + { + do { } while (true); + do MethodCall( + ); while (true); + while (true); + while (true) { + break; + } + for (int i = 0; i < 10; ++i) + { + } + foreach (int i in Enumerable.Range(0, 10)) + { + } + int[] numbers = [5, 4, 1, 3, 9, 8, 6, 7, 2, 0]; + + return "ChangeMe"; + } + + string MethodCall(int a = 0, int b = 0) => "test"; +} diff --git a/t/t4018/csharp-exclude-method-calls b/t/t4018/csharp-exclude-method-calls new file mode 100644 index 0000000000..51e2dc2040 --- /dev/null +++ b/t/t4018/csharp-exclude-method-calls @@ -0,0 +1,20 @@ +class Example +{ + string Method(int RIGHT) + { + MethodCall(); + MethodCall(1, 2); + MethodCall( + 1, 2); + MethodCall( + 1, 2, + 3); + MethodCall( + 1, MethodCall(), + 2); + + return "ChangeMe"; + } + + int MethodCall(int a = 0, int b = 0, int c = 0) => 42; +} diff --git a/t/t4018/csharp-exclude-other b/t/t4018/csharp-exclude-other new file mode 100644 index 0000000000..4d5581cf3e --- /dev/null +++ b/t/t4018/csharp-exclude-other @@ -0,0 +1,18 @@ +class Example +{ + string Method(int RIGHT) + { + lock (this) + { + } + unsafe + { + byte[] bytes = [1, 2, 3]; + fixed (byte* pointerToFirst = bytes) + { + } + } + + return "ChangeMe"; + } +} diff --git a/t/t4018/csharp-method b/t/t4018/csharp-method new file mode 100644 index 0000000000..16b367aca2 --- /dev/null +++ b/t/t4018/csharp-method @@ -0,0 +1,10 @@ +class Example +{ + string Method(int RIGHT) + { + // Filler + // Filler + + return "ChangeMe"; + } +} diff --git a/t/t4018/csharp-method-array b/t/t4018/csharp-method-array new file mode 100644 index 0000000000..1126de8201 --- /dev/null +++ b/t/t4018/csharp-method-array @@ -0,0 +1,10 @@ +class Example +{ + string[] Method(int RIGHT) + { + // Filler + // Filler + + return ["ChangeMe"]; + } +} diff --git a/t/t4018/csharp-method-explicit b/t/t4018/csharp-method-explicit new file mode 100644 index 0000000000..5a710116cc --- /dev/null +++ b/t/t4018/csharp-method-explicit @@ -0,0 +1,12 @@ +using System; + +class Example : IDisposable +{ + void IDisposable.Dispose() // RIGHT + { + // Filler + // Filler + + // ChangeMe + } +} diff --git a/t/t4018/csharp-method-generics b/t/t4018/csharp-method-generics new file mode 100644 index 0000000000..b3216bfb2a --- /dev/null +++ b/t/t4018/csharp-method-generics @@ -0,0 +1,11 @@ +class Example<T1, T2> +{ + Example<int, string> Method<TA, TB>(TA RIGHT, TB b) + { + // Filler + // Filler + + // ChangeMe + return null; + } +} diff --git a/t/t4018/csharp-method-generics-alternate-spaces b/t/t4018/csharp-method-generics-alternate-spaces new file mode 100644 index 0000000000..9583621743 --- /dev/null +++ b/t/t4018/csharp-method-generics-alternate-spaces @@ -0,0 +1,11 @@ +class Example<T1, T2> +{ + Example<int,string> Method<TA ,TB>(TA RIGHT, TB b) + { + // Filler + // Filler + + // ChangeMe + return null; + } +} diff --git a/t/t4018/csharp-method-modifiers b/t/t4018/csharp-method-modifiers new file mode 100644 index 0000000000..caefa8ee99 --- /dev/null +++ b/t/t4018/csharp-method-modifiers @@ -0,0 +1,13 @@ +using System.Threading.Tasks; + +class Example +{ + static internal async Task Method(int RIGHT) + { + // Filler + // Filler + + // ChangeMe + await Task.Delay(1); + } +} diff --git a/t/t4018/csharp-method-multiline b/t/t4018/csharp-method-multiline new file mode 100644 index 0000000000..3983ff42f5 --- /dev/null +++ b/t/t4018/csharp-method-multiline @@ -0,0 +1,10 @@ +class Example +{ + string Method_RIGHT( + int a, + int b, + int c) + { + return "ChangeMe"; + } +} diff --git a/t/t4018/csharp-method-params b/t/t4018/csharp-method-params new file mode 100644 index 0000000000..3f00410ba1 --- /dev/null +++ b/t/t4018/csharp-method-params @@ -0,0 +1,10 @@ +class Example +{ + string Method(int RIGHT, int b, int c = 42) + { + // Filler + // Filler + + return "ChangeMe"; + } +} diff --git a/t/t4018/csharp-method-special-chars b/t/t4018/csharp-method-special-chars new file mode 100644 index 0000000000..e6c7bc01a1 --- /dev/null +++ b/t/t4018/csharp-method-special-chars @@ -0,0 +1,11 @@ +class @Some_Type +{ + @Some_Type @Method_With_Underscore(int RIGHT) + { + // Filler + // Filler + + // ChangeMe + return new @Some_Type(); + } +} diff --git a/t/t4018/csharp-method-with-spacing b/t/t4018/csharp-method-with-spacing new file mode 100644 index 0000000000..233bb976cc --- /dev/null +++ b/t/t4018/csharp-method-with-spacing @@ -0,0 +1,10 @@ +class Example +{ + string Method ( int RIGHT ) + { + // Filler + // Filler + + return "ChangeMe"; + } +} diff --git a/t/t4018/csharp-property b/t/t4018/csharp-property new file mode 100644 index 0000000000..e56dfce34c --- /dev/null +++ b/t/t4018/csharp-property @@ -0,0 +1,11 @@ +class Example +{ + public bool RIGHT + { + get { return true; } + set + { + // ChangeMe + } + } +} diff --git a/t/t4018/csharp-property-braces-same-line b/t/t4018/csharp-property-braces-same-line new file mode 100644 index 0000000000..608131d3d3 --- /dev/null +++ b/t/t4018/csharp-property-braces-same-line @@ -0,0 +1,10 @@ +class Example +{ + public bool RIGHT { + get { return true; } + set + { + // ChangeMe + } + } +} diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index fb53dddf79..b0a3e84984 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -671,4 +671,67 @@ test_expect_success 'test simple stage 1 handling' ' ) ' +test_expect_success 'rerere does not crash with missing preimage' ' + git config rerere.enabled true && + + echo bar >test && + git add test && + git commit -m "one" && + git branch rerere_no_crash && + + echo foo >>test && + git add test && + git commit -m "two" && + + git checkout rerere_no_crash && + echo "bar" >>test && + git add test && + git commit -m "three" && + + test_must_fail git rebase main && + rm .git/rr-cache/*/preimage && + git rebase --abort +' + +test_expect_success 'rerere does not crash with unmatched conflict marker' ' + git config rerere.enabled true && + + echo bar >test && + git add test && + git commit -m "one" && + git branch rerere_no_preimage && + + cat >test <<-EOF && + test + bar + foobar + EOF + git add test && + git commit -m "two" && + + git checkout rerere_no_preimage && + echo "bar" >>test && + git add test && + git commit -m "three" && + + cat >test <<-EOF && + foobar + bar + bar + EOF + git add test && + git commit -m "four" && + + test_must_fail git rebase main && + cat >test <<-EOF && + test + bar + <<<<<<< HEAD + foobar + bar + EOF + git add test && + test_must_fail git rebase --continue +' + test_done diff --git a/t/t4210-log-i18n.sh b/t/t4210-log-i18n.sh index d2dfcf164e..75216f19ce 100755 --- a/t/t4210-log-i18n.sh +++ b/t/t4210-log-i18n.sh @@ -64,7 +64,7 @@ test_expect_success 'log --grep does not find non-reencoded values (latin1)' ' ' triggers_undefined_behaviour () { - local engine=$1 + local engine="$1" case $engine in fixed) @@ -85,7 +85,7 @@ triggers_undefined_behaviour () { } mismatched_git_log () { - local pattern=$1 + local pattern="$1" LC_ALL=$is_IS_locale git log --encoding=ISO-8859-1 --format=%s \ --grep=$pattern diff --git a/t/t5326-multi-pack-bitmaps.sh b/t/t5326-multi-pack-bitmaps.sh index 70d1b58709..5d7d321840 100755 --- a/t/t5326-multi-pack-bitmaps.sh +++ b/t/t5326-multi-pack-bitmaps.sh @@ -513,4 +513,21 @@ test_expect_success 'corrupt MIDX with bitmap causes fallback' ' ) ' +for allow_pack_reuse in single multi +do + test_expect_success "reading MIDX without BTMP chunk does not complain with $allow_pack_reuse pack reuse" ' + test_when_finished "rm -rf midx-without-btmp" && + git init midx-without-btmp && + ( + cd midx-without-btmp && + test_commit initial && + + git repack -Adbl --write-bitmap-index --write-midx && + GIT_TEST_MIDX_READ_BTMP=false git -c pack.allowPackReuse=$allow_pack_reuse \ + pack-objects --all --use-bitmap-index --stdout </dev/null >/dev/null 2>err && + test_must_be_empty err + ) + ' +done + test_done diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh index 94f9f4a1da..127efe99f8 100755 --- a/t/t7700-repack.sh +++ b/t/t7700-repack.sh @@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' ' git log --format="create refs/tags/%s/%s %H" HEAD >refs && git update-ref --stdin <refs && + GIT_TEST_MULTI_PACK_INDEX=0 \ git repack --write-midx --write-bitmap-index && test_path_is_file $midx && test_path_is_file $midx-$(midx_checksum $objdir).bitmap && @@ -749,6 +750,7 @@ test_expect_success '--write-midx with --pack-kept-objects' ' keep="$objdir/pack/pack-$one.keep" && touch "$keep" && + GIT_TEST_MULTI_PACK_INDEX=0 \ git repack --write-midx --write-bitmap-index --geometric=2 -d \ --pack-kept-objects && diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 60e30fed3c..1e68426852 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -1059,30 +1059,33 @@ test_expect_success 'M: rename subdirectory to new subdirectory' ' compare_diff_raw expect actual ' -test_expect_success 'M: rename root to subdirectory' ' - cat >input <<-INPUT_END && - commit refs/heads/M4 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - rename root - COMMIT +for root in '""' '' +do + test_expect_success "M: rename root ($root) to subdirectory" ' + cat >input <<-INPUT_END && + commit refs/heads/M4 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + rename root + COMMIT - from refs/heads/M2^0 - R "" sub + from refs/heads/M2^0 + R $root sub - INPUT_END + INPUT_END - cat >expect <<-EOF && - :100644 100644 $oldf $oldf R100 file2/oldf sub/file2/oldf - :100755 100755 $f4id $f4id R100 file4 sub/file4 - :100755 100755 $newf $newf R100 i/am/new/to/you sub/i/am/new/to/you - :100755 100755 $f6id $f6id R100 newdir/exec.sh sub/newdir/exec.sh - :100644 100644 $f5id $f5id R100 newdir/interesting sub/newdir/interesting - EOF - git fast-import <input && - git diff-tree -M -r M4^ M4 >actual && - compare_diff_raw expect actual -' + cat >expect <<-EOF && + :100644 100644 $oldf $oldf R100 file2/oldf sub/file2/oldf + :100755 100755 $f4id $f4id R100 file4 sub/file4 + :100755 100755 $newf $newf R100 i/am/new/to/you sub/i/am/new/to/you + :100755 100755 $f6id $f6id R100 newdir/exec.sh sub/newdir/exec.sh + :100644 100644 $f5id $f5id R100 newdir/interesting sub/newdir/interesting + EOF + git fast-import <input && + git diff-tree -M -r M4^ M4 >actual && + compare_diff_raw expect actual + ' +done ### ### series N @@ -1259,49 +1262,52 @@ test_expect_success PIPE 'N: empty directory reads as missing' ' test_cmp expect actual ' -test_expect_success 'N: copy root directory by tree hash' ' - cat >expect <<-EOF && - :100755 000000 $newf $zero D file3/newf - :100644 000000 $oldf $zero D file3/oldf - EOF - root=$(git rev-parse refs/heads/branch^0^{tree}) && - cat >input <<-INPUT_END && - commit refs/heads/N6 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - copy root directory by tree hash - COMMIT - - from refs/heads/branch^0 - M 040000 $root "" - INPUT_END - git fast-import <input && - git diff-tree -C --find-copies-harder -r N4 N6 >actual && - compare_diff_raw expect actual -' +for root in '""' '' +do + test_expect_success "N: copy root ($root) by tree hash" ' + cat >expect <<-EOF && + :100755 000000 $newf $zero D file3/newf + :100644 000000 $oldf $zero D file3/oldf + EOF + root_tree=$(git rev-parse refs/heads/branch^0^{tree}) && + cat >input <<-INPUT_END && + commit refs/heads/N6 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + copy root directory by tree hash + COMMIT -test_expect_success 'N: copy root by path' ' - cat >expect <<-EOF && - :100755 100755 $newf $newf C100 file2/newf oldroot/file2/newf - :100644 100644 $oldf $oldf C100 file2/oldf oldroot/file2/oldf - :100755 100755 $f4id $f4id C100 file4 oldroot/file4 - :100755 100755 $f6id $f6id C100 newdir/exec.sh oldroot/newdir/exec.sh - :100644 100644 $f5id $f5id C100 newdir/interesting oldroot/newdir/interesting - EOF - cat >input <<-INPUT_END && - commit refs/heads/N-copy-root-path - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - copy root directory by (empty) path - COMMIT + from refs/heads/branch^0 + M 040000 $root_tree $root + INPUT_END + git fast-import <input && + git diff-tree -C --find-copies-harder -r N4 N6 >actual && + compare_diff_raw expect actual + ' + + test_expect_success "N: copy root ($root) by path" ' + cat >expect <<-EOF && + :100755 100755 $newf $newf C100 file2/newf oldroot/file2/newf + :100644 100644 $oldf $oldf C100 file2/oldf oldroot/file2/oldf + :100755 100755 $f4id $f4id C100 file4 oldroot/file4 + :100755 100755 $f6id $f6id C100 newdir/exec.sh oldroot/newdir/exec.sh + :100644 100644 $f5id $f5id C100 newdir/interesting oldroot/newdir/interesting + EOF + cat >input <<-INPUT_END && + commit refs/heads/N-copy-root-path + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + copy root directory by (empty) path + COMMIT - from refs/heads/branch^0 - C "" oldroot - INPUT_END - git fast-import <input && - git diff-tree -C --find-copies-harder -r branch N-copy-root-path >actual && - compare_diff_raw expect actual -' + from refs/heads/branch^0 + C $root oldroot + INPUT_END + git fast-import <input && + git diff-tree -C --find-copies-harder -r branch N-copy-root-path >actual && + compare_diff_raw expect actual + ' +done test_expect_success 'N: delete directory by copying' ' cat >expect <<-\EOF && @@ -1431,98 +1437,102 @@ test_expect_success 'N: reject foo/ syntax in ls argument' ' INPUT_END ' -test_expect_success 'N: copy to root by id and modify' ' - echo "hello, world" >expect.foo && - echo hello >expect.bar && - git fast-import <<-SETUP_END && - commit refs/heads/N7 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - hello, tree - COMMIT - - deleteall - M 644 inline foo/bar - data <<EOF - hello - EOF - SETUP_END - - tree=$(git rev-parse --verify N7:) && - git fast-import <<-INPUT_END && - commit refs/heads/N8 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - copy to root by id and modify - COMMIT +for root in '""' '' +do + test_expect_success "N: copy to root ($root) by id and modify" ' + echo "hello, world" >expect.foo && + echo hello >expect.bar && + git fast-import <<-SETUP_END && + commit refs/heads/N7 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + hello, tree + COMMIT - M 040000 $tree "" - M 644 inline foo/foo - data <<EOF - hello, world - EOF - INPUT_END - git show N8:foo/foo >actual.foo && - git show N8:foo/bar >actual.bar && - test_cmp expect.foo actual.foo && - test_cmp expect.bar actual.bar -' + deleteall + M 644 inline foo/bar + data <<EOF + hello + EOF + SETUP_END -test_expect_success 'N: extract subtree' ' - branch=$(git rev-parse --verify refs/heads/branch^{tree}) && - cat >input <<-INPUT_END && - commit refs/heads/N9 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - extract subtree branch:newdir - COMMIT + tree=$(git rev-parse --verify N7:) && + git fast-import <<-INPUT_END && + commit refs/heads/N8 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + copy to root by id and modify + COMMIT - M 040000 $branch "" - C "newdir" "" - INPUT_END - git fast-import <input && - git diff --exit-code branch:newdir N9 -' + M 040000 $tree $root + M 644 inline foo/foo + data <<EOF + hello, world + EOF + INPUT_END + git show N8:foo/foo >actual.foo && + git show N8:foo/bar >actual.bar && + test_cmp expect.foo actual.foo && + test_cmp expect.bar actual.bar + ' + + test_expect_success "N: extract subtree to the root ($root)" ' + branch=$(git rev-parse --verify refs/heads/branch^{tree}) && + cat >input <<-INPUT_END && + commit refs/heads/N9 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + extract subtree branch:newdir + COMMIT -test_expect_success 'N: modify subtree, extract it, and modify again' ' - echo hello >expect.baz && - echo hello, world >expect.qux && - git fast-import <<-SETUP_END && - commit refs/heads/N10 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - hello, tree - COMMIT + M 040000 $branch $root + C "newdir" $root + INPUT_END + git fast-import <input && + git diff --exit-code branch:newdir N9 + ' + + test_expect_success "N: modify subtree, extract it to the root ($root), and modify again" ' + echo hello >expect.baz && + echo hello, world >expect.qux && + git fast-import <<-SETUP_END && + commit refs/heads/N10 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + hello, tree + COMMIT - deleteall - M 644 inline foo/bar/baz - data <<EOF - hello - EOF - SETUP_END + deleteall + M 644 inline foo/bar/baz + data <<EOF + hello + EOF + SETUP_END - tree=$(git rev-parse --verify N10:) && - git fast-import <<-INPUT_END && - commit refs/heads/N11 - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - copy to root by id and modify - COMMIT + tree=$(git rev-parse --verify N10:) && + git fast-import <<-INPUT_END && + commit refs/heads/N11 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + copy to root by id and modify + COMMIT - M 040000 $tree "" - M 100644 inline foo/bar/qux - data <<EOF - hello, world - EOF - R "foo" "" - C "bar/qux" "bar/quux" - INPUT_END - git show N11:bar/baz >actual.baz && - git show N11:bar/qux >actual.qux && - git show N11:bar/quux >actual.quux && - test_cmp expect.baz actual.baz && - test_cmp expect.qux actual.qux && - test_cmp expect.qux actual.quux' + M 040000 $tree $root + M 100644 inline foo/bar/qux + data <<EOF + hello, world + EOF + R "foo" $root + C "bar/qux" "bar/quux" + INPUT_END + git show N11:bar/baz >actual.baz && + git show N11:bar/qux >actual.qux && + git show N11:bar/quux >actual.quux && + test_cmp expect.baz actual.baz && + test_cmp expect.qux actual.qux && + test_cmp expect.qux actual.quux + ' +done ### ### series O @@ -2142,6 +2152,7 @@ test_expect_success 'Q: deny note on empty branch' ' EOF test_must_fail git fast-import <input ' + ### ### series R (feature and option) ### @@ -2790,7 +2801,7 @@ test_expect_success 'R: blob appears only once' ' ' ### -### series S +### series S (mark and path parsing) ### # # Make sure missing spaces and EOLs after mark references @@ -3060,21 +3071,283 @@ test_expect_success 'S: ls with garbage after sha1 must fail' ' test_grep "space after tree-ish" err ' +# +# Path parsing +# +# There are two sorts of ways a path can be parsed, depending on whether it is +# the last field on the line. Additionally, ls without a <dataref> has a special +# case. Test every occurrence of <path> in the grammar against every error case. +# Paths for the root (empty strings) are tested elsewhere. +# + +# +# Valid paths at the end of a line: filemodify, filedelete, filecopy (dest), +# filerename (dest), and ls. +# +# commit :301 from root -- modify hello.c (for setup) +# commit :302 from :301 -- modify $path +# commit :303 from :302 -- delete $path +# commit :304 from :301 -- copy hello.c $path +# commit :305 from :301 -- rename hello.c $path +# ls :305 $path +# +test_path_eol_success () { + local test="$1" path="$2" unquoted_path="$3" + test_expect_success "S: paths at EOL with $test must work" ' + test_when_finished "git branch -D S-path-eol" && + + git fast-import --export-marks=marks.out <<-EOF >out 2>err && + blob + mark :401 + data <<BLOB + hello world + BLOB + + blob + mark :402 + data <<BLOB + hallo welt + BLOB + + commit refs/heads/S-path-eol + mark :301 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + initial commit + COMMIT + M 100644 :401 hello.c + + commit refs/heads/S-path-eol + mark :302 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit filemodify + COMMIT + from :301 + M 100644 :402 $path + + commit refs/heads/S-path-eol + mark :303 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit filedelete + COMMIT + from :302 + D $path + + commit refs/heads/S-path-eol + mark :304 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit filecopy dest + COMMIT + from :301 + C hello.c $path + + commit refs/heads/S-path-eol + mark :305 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit filerename dest + COMMIT + from :301 + R hello.c $path + + ls :305 $path + EOF + + commit_m=$(grep :302 marks.out | cut -d\ -f2) && + commit_d=$(grep :303 marks.out | cut -d\ -f2) && + commit_c=$(grep :304 marks.out | cut -d\ -f2) && + commit_r=$(grep :305 marks.out | cut -d\ -f2) && + blob1=$(grep :401 marks.out | cut -d\ -f2) && + blob2=$(grep :402 marks.out | cut -d\ -f2) && + + ( + printf "100644 blob $blob2\t$unquoted_path\n" && + printf "100644 blob $blob1\thello.c\n" + ) | sort >tree_m.exp && + git ls-tree $commit_m | sort >tree_m.out && + test_cmp tree_m.exp tree_m.out && + + printf "100644 blob $blob1\thello.c\n" >tree_d.exp && + git ls-tree $commit_d >tree_d.out && + test_cmp tree_d.exp tree_d.out && + + ( + printf "100644 blob $blob1\t$unquoted_path\n" && + printf "100644 blob $blob1\thello.c\n" + ) | sort >tree_c.exp && + git ls-tree $commit_c | sort >tree_c.out && + test_cmp tree_c.exp tree_c.out && + + printf "100644 blob $blob1\t$unquoted_path\n" >tree_r.exp && + git ls-tree $commit_r >tree_r.out && + test_cmp tree_r.exp tree_r.out && + + test_cmp out tree_r.exp + ' +} + +test_path_eol_success 'quoted spaces' '" hello world.c "' ' hello world.c ' +test_path_eol_success 'unquoted spaces' ' hello world.c ' ' hello world.c ' +test_path_eol_success 'octal escapes' '"\150\151\056\143"' 'hi.c' + +# +# Valid paths before a space: filecopy (source) and filerename (source). +# +# commit :301 from root -- modify $path (for setup) +# commit :302 from :301 -- copy $path hello2.c +# commit :303 from :301 -- rename $path hello2.c +# +test_path_space_success () { + local test="$1" path="$2" unquoted_path="$3" + test_expect_success "S: paths before space with $test must work" ' + test_when_finished "git branch -D S-path-space" && + + git fast-import --export-marks=marks.out <<-EOF 2>err && + blob + mark :401 + data <<BLOB + hello world + BLOB + + commit refs/heads/S-path-space + mark :301 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + initial commit + COMMIT + M 100644 :401 $path + + commit refs/heads/S-path-space + mark :302 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit filecopy source + COMMIT + from :301 + C $path hello2.c + + commit refs/heads/S-path-space + mark :303 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit filerename source + COMMIT + from :301 + R $path hello2.c + + EOF + + commit_c=$(grep :302 marks.out | cut -d\ -f2) && + commit_r=$(grep :303 marks.out | cut -d\ -f2) && + blob=$(grep :401 marks.out | cut -d\ -f2) && + + ( + printf "100644 blob $blob\t$unquoted_path\n" && + printf "100644 blob $blob\thello2.c\n" + ) | sort >tree_c.exp && + git ls-tree $commit_c | sort >tree_c.out && + test_cmp tree_c.exp tree_c.out && + + printf "100644 blob $blob\thello2.c\n" >tree_r.exp && + git ls-tree $commit_r >tree_r.out && + test_cmp tree_r.exp tree_r.out + ' +} + +test_path_space_success 'quoted spaces' '" hello world.c "' ' hello world.c ' +test_path_space_success 'no unquoted spaces' 'hello_world.c' 'hello_world.c' +test_path_space_success 'octal escapes' '"\150\151\056\143"' 'hi.c' + +# +# Test a single commit change with an invalid path. Run it with all occurrences +# of <path> in the grammar against all error kinds. +# +test_path_fail () { + local change="$1" what="$2" prefix="$3" path="$4" suffix="$5" err_grep="$6" + test_expect_success "S: $change with $what must fail" ' + test_must_fail git fast-import <<-EOF 2>err && + blob + mark :1 + data <<BLOB + hello world + BLOB + + commit refs/heads/S-path-fail + mark :2 + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit setup + COMMIT + M 100644 :1 hello.c + + commit refs/heads/S-path-fail + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + commit with bad path + COMMIT + from :2 + $prefix$path$suffix + EOF + + test_grep "$err_grep" err + ' +} + +test_path_base_fail () { + local change="$1" prefix="$2" field="$3" suffix="$4" + test_path_fail "$change" 'unclosed " in '"$field" "$prefix" '"hello.c' "$suffix" "Invalid $field" + test_path_fail "$change" "invalid escape in quoted $field" "$prefix" '"hello\xff"' "$suffix" "Invalid $field" + test_path_fail "$change" "escaped NUL in quoted $field" "$prefix" '"hello\000"' "$suffix" "NUL in $field" +} +test_path_eol_quoted_fail () { + local change="$1" prefix="$2" field="$3" + test_path_base_fail "$change" "$prefix" "$field" '' + test_path_fail "$change" "garbage after quoted $field" "$prefix" '"hello.c"' 'x' "Garbage after $field" + test_path_fail "$change" "space after quoted $field" "$prefix" '"hello.c"' ' ' "Garbage after $field" +} +test_path_eol_fail () { + local change="$1" prefix="$2" field="$3" + test_path_eol_quoted_fail "$change" "$prefix" "$field" +} +test_path_space_fail () { + local change="$1" prefix="$2" field="$3" + test_path_base_fail "$change" "$prefix" "$field" ' world.c' + test_path_fail "$change" "missing space after quoted $field" "$prefix" '"hello.c"' 'x world.c' "Missing space after $field" + test_path_fail "$change" "missing space after unquoted $field" "$prefix" 'hello.c' '' "Missing space after $field" +} + +test_path_eol_fail filemodify 'M 100644 :1 ' path +test_path_eol_fail filedelete 'D ' path +test_path_space_fail filecopy 'C ' source +test_path_eol_fail filecopy 'C hello.c ' dest +test_path_space_fail filerename 'R ' source +test_path_eol_fail filerename 'R hello.c ' dest +test_path_eol_fail 'ls (in commit)' 'ls :2 ' path + +# When 'ls' has no <dataref>, the <path> must be quoted. +test_path_eol_quoted_fail 'ls (without dataref in commit)' 'ls ' path + ### ### series T (ls) ### # Setup is carried over from series S. -test_expect_success 'T: ls root tree' ' - sed -e "s/Z\$//" >expect <<-EOF && - 040000 tree $(git rev-parse S^{tree}) Z - EOF - sha1=$(git rev-parse --verify S) && - git fast-import --import-marks=marks <<-EOF >actual && - ls $sha1 "" - EOF - test_cmp expect actual -' +for root in '""' '' +do + test_expect_success "T: ls root ($root) tree" ' + sed -e "s/Z\$//" >expect <<-EOF && + 040000 tree $(git rev-parse S^{tree}) Z + EOF + sha1=$(git rev-parse --verify S) && + git fast-import --import-marks=marks <<-EOF >actual && + ls $sha1 $root + EOF + test_cmp expect actual + ' +done test_expect_success 'T: delete branch' ' git branch to-delete && @@ -3176,30 +3449,33 @@ test_expect_success 'U: validate directory delete result' ' compare_diff_raw expect actual ' -test_expect_success 'U: filedelete root succeeds' ' - cat >input <<-INPUT_END && - commit refs/heads/U - committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE - data <<COMMIT - must succeed - COMMIT - from refs/heads/U^0 - D "" +for root in '""' '' +do + test_expect_success "U: filedelete root ($root) succeeds" ' + cat >input <<-INPUT_END && + commit refs/heads/U-delete-root + committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data <<COMMIT + must succeed + COMMIT + from refs/heads/U^0 + D $root - INPUT_END + INPUT_END - git fast-import <input -' + git fast-import <input + ' -test_expect_success 'U: validate root delete result' ' - cat >expect <<-EOF && - :100644 000000 $f7id $ZERO_OID D hello.c - EOF + test_expect_success "U: validate root ($root) delete result" ' + cat >expect <<-EOF && + :100644 000000 $f7id $ZERO_OID D hello.c + EOF - git diff-tree -M -r U^1 U >actual && + git diff-tree -M -r U U-delete-root >actual && - compare_diff_raw expect actual -' + compare_diff_raw expect actual + ' +done ### ### series V (checkpoint) diff --git a/t/t9604-cvsimport-timestamps.sh b/t/t9604-cvsimport-timestamps.sh index 2ff4aa932d..2d03259729 100755 --- a/t/t9604-cvsimport-timestamps.sh +++ b/t/t9604-cvsimport-timestamps.sh @@ -3,11 +3,28 @@ test_description='git cvsimport timestamps' . ./lib-cvs.sh +test_lazy_prereq POSIX_TIMEZONE ' + local tz=XST-1XDT,M3.5.0,M11.1.0 + echo "1711846799 -> 2024-03-31 01:59:59 +0100" >expected && + TZ="$tz" test-tool date show:iso-local 1711846799 >actual && + test_cmp expected actual && + echo "1711846800 -> 2024-03-31 03:00:00 +0200" >expected && + TZ="$tz" test-tool date show:iso-local 1711846800 >actual && + test_cmp expected actual && + echo "1730591999 -> 2024-11-03 01:59:59 +0200" >expected && + TZ="$tz" test-tool date show:iso-local 1730591999 >actual && + test_cmp expected actual && + echo "1730592000 -> 2024-11-03 01:00:00 +0100" >expected && + TZ="$tz" test-tool date show:iso-local 1730592000 >actual && + test_cmp expected actual +' + setup_cvs_test_repository t9604 -test_expect_success PERL 'check timestamps are UTC (TZ=CST6CDT)' ' +test_expect_success PERL,POSIX_TIMEZONE 'check timestamps are UTC' ' - TZ=CST6CDT git cvsimport -p"-x" -C module-1 module && + TZ=CST6CDT,M4.1.0,M10.5.0 \ + git cvsimport -p"-x" -C module-1 module && git cvsimport -p"-x" -C module-1 module && ( cd module-1 && @@ -34,13 +51,13 @@ test_expect_success PERL 'check timestamps are UTC (TZ=CST6CDT)' ' test_cmp expect-1 actual-1 ' -test_expect_success PERL 'check timestamps with author-specific timezones' ' +test_expect_success PERL,POSIX_TIMEZONE 'check timestamps with author-specific timezones' ' cat >cvs-authors <<-EOF && user1=User One <user1@domain.org> - user2=User Two <user2@domain.org> CST6CDT - user3=User Three <user3@domain.org> EST5EDT - user4=User Four <user4@domain.org> MST7MDT + user2=User Two <user2@domain.org> CST6CDT,M4.1.0,M10.5.0 + user3=User Three <user3@domain.org> EST5EDT,M4.1.0,M10.5.0 + user4=User Four <user4@domain.org> MST7MDT,M4.1.0,M10.5.0 EOF git cvsimport -p"-x" -A cvs-authors -C module-2 module && ( diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 2eccf100c0..862d80c974 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -385,7 +385,7 @@ test_commit () { shift done && indir=${indir:+"$indir"/} && - local file=${2:-"$1.t"} && + local file="${2:-"$1.t"}" && if test -n "$append" then $echo "${3-$1}" >>"$indir$file" @@ -1748,7 +1748,7 @@ test_oid () { # Insert a slash into an object ID so it can be used to reference a location # under ".git/objects". For example, "deadbeef..." becomes "de/adbeef..". test_oid_to_path () { - local basename=${1#??} + local basename="${1#??}" echo "${1%$basename}/$basename" } @@ -1765,7 +1765,7 @@ test_parse_ls_tree_oids () { # Choose a port number based on the test script's number and store it in # the given variable name, unless that variable already contains a number. test_set_port () { - local var=$1 port + local var="$1" port if test $# -ne 1 || test -z "$var" then @@ -1840,7 +1840,7 @@ test_subcommand () { shift fi - local expr=$(printf '"%s",' "$@") + local expr="$(printf '"%s",' "$@")" expr="${expr%,}" if test -n "$negate" @@ -1930,7 +1930,7 @@ test_readlink () { # An optional increment to the magic timestamp may be specified as second # argument. test_set_magic_mtime () { - local inc=${2:-0} && + local inc="${2:-0}" && local mtime=$((1234567890 + $inc)) && test-tool chmtime =$mtime "$1" && test_is_magic_mtime "$1" $inc @@ -1943,7 +1943,7 @@ test_set_magic_mtime () { # argument. Usually, this should be the same increment which was used for # the associated test_set_magic_mtime. test_is_magic_mtime () { - local inc=${2:-0} && + local inc="${2:-0}" && local mtime=$((1234567890 + $inc)) && echo $mtime >.git/test-mtime-expect && test-tool chmtime --get "$1" >.git/test-mtime-actual && @@ -144,38 +144,6 @@ static char last_non_space_char(const char *s) return '\0'; } -static void print_tok_val(struct strbuf *out, const char *tok, const char *val) -{ - char c; - - if (!tok) { - strbuf_addf(out, "%s\n", val); - return; - } - - c = last_non_space_char(tok); - if (!c) - return; - if (strchr(separators, c)) - strbuf_addf(out, "%s%s\n", tok, val); - else - strbuf_addf(out, "%s%c %s\n", tok, separators[0], val); -} - -void format_trailers(const struct process_trailer_options *opts, - struct list_head *trailers, - struct strbuf *out) -{ - struct list_head *pos; - struct trailer_item *item; - list_for_each(pos, trailers) { - item = list_entry(pos, struct trailer_item, list); - if ((!opts->trim_empty || strlen(item->value) > 0) && - (!opts->only_trailers || item->token)) - print_tok_val(out, item->token, item->value); - } -} - static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok) { struct trailer_item *new_item = xcalloc(1, sizeof(*new_item)); @@ -1084,26 +1052,32 @@ void trailer_info_release(struct trailer_info *info) free(info->trailers); } -static void format_trailer_info(const struct process_trailer_options *opts, - const struct trailer_info *info, - struct strbuf *out) +void format_trailers(const struct process_trailer_options *opts, + struct list_head *trailers, + struct strbuf *out) { size_t origlen = out->len; - size_t i; - - for (i = 0; i < info->trailer_nr; i++) { - char *trailer = info->trailers[i]; - ssize_t separator_pos = find_separator(trailer, separators); + struct list_head *pos; + struct trailer_item *item; - if (separator_pos >= 1) { + list_for_each(pos, trailers) { + item = list_entry(pos, struct trailer_item, list); + if (item->token) { struct strbuf tok = STRBUF_INIT; struct strbuf val = STRBUF_INIT; + strbuf_addstr(&tok, item->token); + strbuf_addstr(&val, item->value); + + /* + * Skip key/value pairs where the value was empty. This + * can happen from trailers specified without a + * separator, like `--trailer "Reviewed-by"` (no + * corresponding value). + */ + if (opts->trim_empty && !strlen(item->value)) + continue; - parse_trailer(&tok, &val, NULL, trailer, separator_pos); if (!opts->filter || opts->filter(&tok, opts->filter_data)) { - if (opts->unfold) - unfold_value(&val); - if (opts->separator && out->len != origlen) strbuf_addbuf(out, opts->separator); if (!opts->value_only) @@ -1111,8 +1085,11 @@ static void format_trailer_info(const struct process_trailer_options *opts, if (!opts->key_only && !opts->value_only) { if (opts->key_value_separator) strbuf_addbuf(out, opts->key_value_separator); - else - strbuf_addstr(out, ": "); + else { + char c = last_non_space_char(tok.buf); + if (c && !strchr(separators, c)) + strbuf_addf(out, "%c ", separators[0]); + } } if (!opts->key_only) strbuf_addbuf(out, &val); @@ -1126,13 +1103,13 @@ static void format_trailer_info(const struct process_trailer_options *opts, if (opts->separator && out->len != origlen) { strbuf_addbuf(out, opts->separator); } - strbuf_addstr(out, trailer); - if (opts->separator) { + strbuf_addstr(out, item->value); + if (opts->separator) strbuf_rtrim(out); - } + else + strbuf_addch(out, '\n'); } } - } void format_trailers_from_commit(const struct process_trailer_options *opts, @@ -1151,7 +1128,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts, strbuf_add(out, msg + info.trailer_block_start, info.trailer_block_end - info.trailer_block_start); } else - format_trailer_info(opts, &info, out); + format_trailers(opts, &trailer_objects, out); free_trailers(&trailer_objects); trailer_info_release(&info); @@ -107,17 +107,10 @@ void format_trailers(const struct process_trailer_options *, void free_trailers(struct list_head *); /* - * Format the trailers from the commit msg "msg" into the strbuf "out". - * Note two caveats about "opts": - * - * - this is primarily a helper for pretty.c, and not - * all of the flags are supported. - * - * - this differs from process_trailers slightly in that we always format - * only the trailer block itself, even if the "only_trailers" option is not - * set. + * Convenience function to format the trailers from the commit msg "msg" into + * the strbuf "out". Reuses format_trailers() internally. */ -void format_trailers_from_commit(const struct process_trailer_options *opts, +void format_trailers_from_commit(const struct process_trailer_options *, const char *msg, struct strbuf *out); @@ -19,8 +19,11 @@ static void vreportf(const char *prefix, const char *err, va_list params) } memcpy(msg, prefix, prefix_len); p = msg + prefix_len; - if (vsnprintf(p, pend - p, err, params) < 0) + if (vsnprintf(p, pend - p, err, params) < 0) { + fprintf(stderr, _("error: unable to format message: %s\n"), + err); *p = '\0'; /* vsnprintf() failed, clip at prefix */ + } for (; p != pend - 1 && *p; p++) { if (iscntrl(*p) && *p != '\t' && *p != '\n') diff --git a/userdiff.c b/userdiff.c index 92ef649c99..82bc76b910 100644 --- a/userdiff.c +++ b/userdiff.c @@ -90,12 +90,48 @@ PATTERNS("cpp", "|\\.[0-9][0-9]*([Ee][-+]?[0-9]+)?[fFlL]?" "|[-+*/<>%&^|=!]=|--|\\+\\+|<<=?|>>=?|&&|\\|\\||::|->\\*?|\\.\\*|<=>"), PATTERNS("csharp", - /* Keywords */ - "!^[ \t]*(do|while|for|if|else|instanceof|new|return|switch|case|throw|catch|using)\n" - /* Methods and constructors */ - "^[ \t]*(((static|public|internal|private|protected|new|virtual|sealed|override|unsafe|async)[ \t]+)*[][<>@.~_[:alnum:]]+[ \t]+[<>@._[:alnum:]]+[ \t]*\\(.*\\))[ \t]*$\n" - /* Properties */ - "^[ \t]*(((static|public|internal|private|protected|new|virtual|sealed|override|unsafe)[ \t]+)*[][<>@.~_[:alnum:]]+[ \t]+[@._[:alnum:]]+)[ \t]*$\n" + /* + * Jump over reserved keywords which are illegal method names, but which + * can be followed by parentheses without special characters in between, + * making them look like methods. + */ + "!(^|[ \t]+)" /* Start of line or whitespace. */ + "(do|while|for|foreach|if|else|new|default|return|switch|case|throw" + "|catch|using|lock|fixed)" + "([ \t(]+|$)\n" /* Whitespace, "(", or end of line. */ + /* + * Methods/constructors: + * The strategy is to identify a minimum of two groups (any combination + * of keywords/type/name) before the opening parenthesis, and without + * final unexpected characters, normally only used in ordinary statements. + */ + "^[ \t]*" /* Remove leading whitespace. */ + "(" /* Start chunk header capture. */ + "(" /* First group. */ + "[][[:alnum:]@_.]" /* Name. */ + "(<[][[:alnum:]@_, \t<>]+>)?" /* Optional generic parameters. */ + ")+" + "([ \t]+" /* Subsequent groups, prepended with space. */ + "([][[:alnum:]@_.](<[][[:alnum:]@_, \t<>]+>)?)+" + ")+" + "[ \t]*" /* Optional space before parameters start. */ + "\\(" /* Start of method parameters. */ + "[^;]*" /* Allow complex parameters, but exclude statements (;). */ + ")$\n" /* Close chunk header capture. */ + /* + * Properties: + * As with methods, expect a minimum of two groups. But, more trivial than + * methods, the vast majority of properties long enough to be worth + * showing a chunk header for don't include "=:;,()" on the line they are + * defined, since they don't have a parameter list. + */ + "^[ \t]*(" + "([][[:alnum:]@_.](<[][[:alnum:]@_, \t<>]+>)?)+" + "([ \t]+" + "([][[:alnum:]@_.](<[][[:alnum:]@_, \t<>]+>)?)+" + ")+" /* Up to here, same as methods regex. */ + "[^;=:,()]*" /* Compared to methods, no parameter list allowed. */ + ")$\n" /* Type definitions */ "^[ \t]*(((static|public|internal|private|protected|new|unsafe|sealed|abstract|partial)[ \t]+)*(class|enum|interface|struct|record)[ \t]+.*)$\n" /* Namespace */ |