Sync with 2.23.1

* maint-2.23: (44 commits)
  Git 2.23.1
  Git 2.22.2
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  ...

Documentation/RelNotes/2.14.6.txt

@@ -0,0 +1,54 @@
+Git v2.14.6 Release Notes
+=========================
+
+This release addresses the security issues CVE-2019-1348,
+CVE-2019-1349, CVE-2019-1350, CVE-2019-1351, CVE-2019-1352,
+CVE-2019-1353, CVE-2019-1354, and CVE-2019-1387.
+
+Fixes since v2.14.5
+-------------------
+
+ * CVE-2019-1348:
+   The --export-marks option of git fast-import is exposed also via
+   the in-stream command feature export-marks=... and it allows
+   overwriting arbitrary paths.
+
+ * CVE-2019-1349:
+   When submodules are cloned recursively, under certain circumstances
+   Git could be fooled into using the same Git directory twice. We now
+   require the directory to be empty.
+
+ * CVE-2019-1350:
+   Incorrect quoting of command-line arguments allowed remote code
+   execution during a recursive clone in conjunction with SSH URLs.
+
+ * CVE-2019-1351:
+   While the only permitted drive letters for physical drives on
+   Windows are letters of the US-English alphabet, this restriction
+   does not apply to virtual drives assigned via subst <letter>:
+   <path>. Git mistook such paths for relative paths, allowing writing
+   outside of the worktree while cloning.
+
+ * CVE-2019-1352:
+   Git was unaware of NTFS Alternate Data Streams, allowing files
+   inside the .git/ directory to be overwritten during a clone.
+
+ * CVE-2019-1353:
+   When running Git in the Windows Subsystem for Linux (also known as
+   "WSL") while accessing a working directory on a regular Windows
+   drive, none of the NTFS protections were active.
+
+ * CVE-2019-1354:
+   Filenames on Linux/Unix can contain backslashes. On Windows,
+   backslashes are directory separators. Git did not use to refuse to
+   write out tracked files with such filenames.
+
+ * CVE-2019-1387:
+   Recursive clones are currently affected by a vulnerability that is
+   caused by too-lax validation of submodule names, allowing very
+   targeted attacks via remote code execution in recursive clones.
+
+Credit for finding these vulnerabilities goes to Microsoft Security
+Response Center, in particular to Nicolas Joly. The `fast-import`
+fixes were provided by Jeff King, the other fixes by Johannes
+Schindelin with help from Garima Singh.

Documentation/RelNotes/2.15.4.txt

@@ -0,0 +1,11 @@
+Git v2.15.4 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6 to address
+the security issues CVE-2019-1348, CVE-2019-1349, CVE-2019-1350,
+CVE-2019-1351, CVE-2019-1352, CVE-2019-1353, CVE-2019-1354, and
+CVE-2019-1387; see the release notes for that version for details.
+
+In conjunction with a vulnerability that was fixed in v2.20.2,
+`.gitmodules` is no longer allowed to contain entries of the form
+`submodule.<name>.update=!command`.

Documentation/RelNotes/2.16.6.txt

@@ -0,0 +1,8 @@
+Git v2.16.6 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6 and in
+v2.15.4 addressing the security issues CVE-2019-1348, CVE-2019-1349,
+CVE-2019-1350, CVE-2019-1351, CVE-2019-1352, CVE-2019-1353,
+CVE-2019-1354, and CVE-2019-1387; see the release notes for those
+versions for details.

Documentation/RelNotes/2.17.3.txt

@@ -0,0 +1,12 @@
+Git v2.17.3 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6 and in
+v2.15.4 addressing the security issues CVE-2019-1348, CVE-2019-1349,
+CVE-2019-1350, CVE-2019-1351, CVE-2019-1352, CVE-2019-1353,
+CVE-2019-1354, and CVE-2019-1387; see the release notes for those
+versions for details.
+
+In addition, `git fsck` was taught to identify `.gitmodules` entries
+of the form `submodule.<name>.update=!command`, which have been
+disallowed in v2.15.4.

Documentation/RelNotes/2.18.2.txt

@@ -0,0 +1,8 @@
+Git v2.18.2 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6, v2.15.4
+and in v2.17.3, addressing the security issues CVE-2019-1348,
+CVE-2019-1349, CVE-2019-1350, CVE-2019-1351, CVE-2019-1352,
+CVE-2019-1353, CVE-2019-1354, and CVE-2019-1387; see the release notes
+for those versions for details.

Documentation/RelNotes/2.19.3.txt

@@ -0,0 +1,8 @@
+Git v2.19.3 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6, v2.15.4
+and in v2.17.3, addressing the security issues CVE-2019-1348,
+CVE-2019-1349, CVE-2019-1350, CVE-2019-1351, CVE-2019-1352,
+CVE-2019-1353, CVE-2019-1354, and CVE-2019-1387; see the release notes
+for those versions for details.

Documentation/RelNotes/2.20.2.txt

@@ -0,0 +1,18 @@
+Git v2.20.2 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6, v2.15.4
+and in v2.17.3, addressing the security issues CVE-2019-1348,
+CVE-2019-1349, CVE-2019-1350, CVE-2019-1351, CVE-2019-1352,
+CVE-2019-1353, CVE-2019-1354, and CVE-2019-1387; see the release notes
+for those versions for details.
+
+The change to disallow `submodule.<name>.update=!command` entries in
+`.gitmodules` which was introduced v2.15.4 (and for which v2.17.3
+added explicit fsck checks) fixes the vulnerability in v2.20.x where a
+recursive clone followed by a submodule update could execute code
+contained within the repository without the user explicitly having
+asked for that (CVE-2019-19604).
+
+Credit for finding this vulnerability goes to Joern Schneeweisz,
+credit for the fixes goes to Jonathan Nieder.

Documentation/RelNotes/2.21.1.txt

@@ -0,0 +1,12 @@
+Git v2.21.1 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6, v2.15.4,
+v2.17.3 and in v2.20.2, addressing the security issues CVE-2019-1348,
+CVE-2019-1349, CVE-2019-1350, CVE-2019-1351, CVE-2019-1352,
+CVE-2019-1353, CVE-2019-1354, CVE-2019-1387, and CVE-2019-19604;
+see the release notes for those versions for details.
+
+Additionally, this version also includes a couple of fixes for the
+Windows-specific quoting of command-line arguments when Git executes
+a Unix shell on Windows.

Documentation/RelNotes/2.22.2.txt

@@ -0,0 +1,8 @@
+Git v2.22.2 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6, v2.15.4,
+v2.17.3, v2.20.2 and in v2.21.1, addressing the security issues
+CVE-2019-1348, CVE-2019-1349, CVE-2019-1350, CVE-2019-1351,
+CVE-2019-1352, CVE-2019-1353, CVE-2019-1354, CVE-2019-1387, and
+CVE-2019-19604; see the release notes for those versions for details.

Documentation/RelNotes/2.23.1.txt

@@ -0,0 +1,8 @@
+Git v2.23.1 Release Notes
+=========================
+
+This release merges up the fixes that appear in v2.14.6, v2.15.4,
+v2.17.3, v2.20.2 and in v2.21.1, addressing the security issues
+CVE-2019-1348, CVE-2019-1349, CVE-2019-1350, CVE-2019-1351,
+CVE-2019-1352, CVE-2019-1353, CVE-2019-1354, CVE-2019-1387, and
+CVE-2019-19604; see the release notes for those versions for details.

Documentation/git-fast-import.txt

@@ -51,6 +51,21 @@ OPTIONS
 	memory used by fast-import during this run.  Showing this output
 	is currently the default, but can be disabled with --quiet.
 
+--allow-unsafe-features::
+	Many command-line options can be provided as part of the
+	fast-import stream itself by using the `feature` or `option`
+	commands. However, some of these options are unsafe (e.g.,
+	allowing fast-import to access the filesystem outside of the
+	repository). These options are disabled by default, but can be
+	allowed by providing this option on the command line.  This
+	currently impacts only the `export-marks`, `import-marks`, and
+	`import-marks-if-exists` feature commands.
++
+	Only enable this option if you trust the program generating the
+	fast-import stream! This option is enabled automatically for
+	remote-helpers that use the `import` capability, as they are
+	already trusted to run their own code.
+
 Options for Frontends
 ~~~~~~~~~~~~~~~~~~~~~
 

Documentation/gitmodules.txt

@@ -44,9 +44,8 @@ submodule.<name>.update::
 	submodule init` to initialize the configuration variable of
 	the same name. Allowed values here are 'checkout', 'rebase',
 	'merge' or 'none'. See description of 'update' command in
-	linkgit:git-submodule[1] for their meaning. Note that the
-	'!command' form is intentionally ignored here for security
-	reasons.
+	linkgit:git-submodule[1] for their meaning. For security
+	reasons, the '!command' form is not accepted here.
 
 submodule.<name>.branch::
 	A remote branch name for tracking updates in the upstream submodule.

builtin/clone.c

@@ -789,7 +789,7 @@ static int checkout(int submodule_progress)
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct argv_array args = ARGV_ARRAY_INIT;
-		argv_array_pushl(&args, "submodule", "update", "--init", "--recursive", NULL);
+		argv_array_pushl(&args, "submodule", "update", "--require-init", "--recursive", NULL);
 
 		if (option_shallow_submodules == 1)
 			argv_array_push(&args, "--depth=1");

builtin/submodule--helper.c

@@ -19,6 +19,7 @@
 #include "diffcore.h"
 #include "diff.h"
 #include "object-store.h"
+#include "dir.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
@@ -1359,7 +1360,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 	char *p, *path = NULL, *sm_gitdir;
 	struct strbuf sb = STRBUF_INIT;
 	struct string_list reference = STRING_LIST_INIT_NODUP;
-	int dissociate = 0;
+	int dissociate = 0, require_init = 0;
 	char *sm_alternate = NULL, *error_strategy = NULL;
 
 	struct option module_clone_options[] = {
@@ -1386,6 +1387,8 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 		OPT__QUIET(&quiet, "Suppress output for cloning a submodule"),
 		OPT_BOOL(0, "progress", &progress,
 			   N_("force cloning progress")),
+		OPT_BOOL(0, "require-init", &require_init,
+			   N_("disallow cloning into non-empty directory")),
 		OPT_END()
 	};
 
@@ -1413,6 +1416,10 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 	} else
 		path = xstrdup(path);
 
+	if (validate_submodule_git_dir(sm_gitdir, name) < 0)
+		die(_("refusing to create/use '%s' in another submodule's "
+			"git dir"), sm_gitdir);
+
 	if (!file_exists(sm_gitdir)) {
 		if (safe_create_leading_directories_const(sm_gitdir) < 0)
 			die(_("could not create directory '%s'"), sm_gitdir);
@@ -1424,6 +1431,8 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 			die(_("clone of '%s' into submodule path '%s' failed"),
 			    url, path);
 	} else {
+		if (require_init && !access(path, X_OK) && !is_empty_dir(path))
+			die(_("directory not empty: '%s'"), path);
 		if (safe_create_leading_directories_const(path) < 0)
 			die(_("could not create directory '%s'"), path);
 		strbuf_addf(&sb, "%s/index", sm_gitdir);
@@ -1478,6 +1487,8 @@ static void determine_submodule_update_strategy(struct repository *r,
 			die(_("Invalid update mode '%s' configured for submodule path '%s'"),
 				val, path);
 	} else if (sub->update_strategy.type != SM_UPDATE_UNSPECIFIED) {
+		if (sub->update_strategy.type == SM_UPDATE_COMMAND)
+			BUG("how did we read update = !command from .gitmodules?");
 		out->type = sub->update_strategy.type;
 		out->command = sub->update_strategy.command;
 	} else
@@ -1536,6 +1547,7 @@ struct submodule_update_clone {
 	int recommend_shallow;
 	struct string_list references;
 	int dissociate;
+	unsigned require_init;
 	const char *depth;
 	const char *recursive_prefix;
 	const char *prefix;
@@ -1554,7 +1566,7 @@ struct submodule_update_clone {
 	int max_jobs;
 };
 #define SUBMODULE_UPDATE_CLONE_INIT {0, MODULE_LIST_INIT, 0, \
-	SUBMODULE_UPDATE_STRATEGY_INIT, 0, 0, -1, STRING_LIST_INIT_DUP, 0, \
+	SUBMODULE_UPDATE_STRATEGY_INIT, 0, 0, -1, STRING_LIST_INIT_DUP, 0, 0, \
 	NULL, NULL, NULL, \
 	NULL, 0, 0, 0, NULL, 0, 0, 1}
 
@@ -1681,6 +1693,8 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 		argv_array_pushl(&child->args, "--prefix", suc->prefix, NULL);
 	if (suc->recommend_shallow && sub->recommend_shallow == 1)
 		argv_array_push(&child->args, "--depth=1");
+	if (suc->require_init)
+		argv_array_push(&child->args, "--require-init");
 	argv_array_pushl(&child->args, "--path", sub->path, NULL);
 	argv_array_pushl(&child->args, "--name", sub->name, NULL);
 	argv_array_pushl(&child->args, "--url", url, NULL);
@@ -1870,6 +1884,8 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 		OPT__QUIET(&suc.quiet, N_("don't print cloning progress")),
 		OPT_BOOL(0, "progress", &suc.progress,
 			    N_("force cloning progress")),
+		OPT_BOOL(0, "require-init", &suc.require_init,
+			   N_("disallow cloning into non-empty directory")),
 		OPT_END()
 	};
 

compat/mingw.c

@@ -392,6 +392,12 @@ int mingw_mkdir(const char *path, int mode)
 {
 	int ret;
 	wchar_t wpath[MAX_PATH];
+
+	if (!is_valid_win32_path(path)) {
+		errno = EINVAL;
+		return -1;
+	}
+
 	if (xutftowcs_path(wpath, path) < 0)
 		return -1;
 	ret = _wmkdir(wpath);
@@ -465,7 +471,7 @@ int mingw_open (const char *filename, int oflags, ...)
 	typedef int (*open_fn_t)(wchar_t const *wfilename, int oflags, ...);
 	va_list args;
 	unsigned mode;
-	int fd;
+	int fd, create = (oflags & (O_CREAT | O_EXCL)) == (O_CREAT | O_EXCL);
 	wchar_t wfilename[MAX_PATH];
 	open_fn_t open_fn;
 
@@ -473,6 +479,11 @@ int mingw_open (const char *filename, int oflags, ...)
 	mode = va_arg(args, int);
 	va_end(args);
 
+	if (!is_valid_win32_path(filename)) {
+		errno = create ? EINVAL : ENOENT;
+		return -1;
+	}
+
 	if (filename && !strcmp(filename, "/dev/null"))
 		filename = "nul";
 
@@ -539,6 +550,11 @@ FILE *mingw_fopen (const char *filename, const char *otype)
 	int hide = needs_hiding(filename);
 	FILE *file;
 	wchar_t wfilename[MAX_PATH], wotype[4];
+	if (!is_valid_win32_path(filename)) {
+		int create = otype && strchr(otype, 'w');
+		errno = create ? EINVAL : ENOENT;
+		return NULL;
+	}
 	if (filename && !strcmp(filename, "/dev/null"))
 		filename = "nul";
 	if (xutftowcs_path(wfilename, filename) < 0 ||
@@ -561,6 +577,11 @@ FILE *mingw_freopen (const char *filename, const char *otype, FILE *stream)
 	int hide = needs_hiding(filename);
 	FILE *file;
 	wchar_t wfilename[MAX_PATH], wotype[4];
+	if (!is_valid_win32_path(filename)) {
+		int create = otype && strchr(otype, 'w');
+		errno = create ? EINVAL : ENOENT;
+		return NULL;
+	}
 	if (filename && !strcmp(filename, "/dev/null"))
 		filename = "nul";
 	if (xutftowcs_path(wfilename, filename) < 0 ||
@@ -1054,7 +1075,7 @@ static const char *quote_arg_msvc(const char *arg)
 				p++;
 				len++;
 			}
-			if (*p == '"')
+			if (*p == '"' || !*p)
 				n += count*2 + 1;
 			continue;
 		}
@@ -1076,16 +1097,19 @@ static const char *quote_arg_msvc(const char *arg)
 				count++;
 				*d++ = *arg++;
 			}
-			if (*arg == '"') {
+			if (*arg == '"' || !*arg) {
 				while (count-- > 0)
 					*d++ = '\\';
+				/* don't escape the surrounding end quote */
+				if (!*arg)
+					break;
 				*d++ = '\\';
 			}
 		}
 		*d++ = *arg++;
 	}
 	*d++ = '"';
-	*d++ = 0;
+	*d++ = '\0';
 	return q;
 }
 
@@ -1098,13 +1122,14 @@ static const char *quote_arg_msys2(const char *arg)
 
 	for (p = arg; *p; p++) {
 		int ws = isspace(*p);
-		if (!ws && *p != '\\' && *p != '"' && *p != '{')
+		if (!ws && *p != '\\' && *p != '"' && *p != '{' && *p != '\'' &&
+		    *p != '?' && *p != '*' && *p != '~')
 			continue;
 		if (!buf.len)
 			strbuf_addch(&buf, '"');
 		if (p != p2)
 			strbuf_add(&buf, p2, p - p2);
-		if (!ws && *p != '{')
+		if (*p == '\\' || *p == '"')
 			strbuf_addch(&buf, '\\');
 		p2 = p;
 	}
@@ -1114,7 +1139,7 @@ static const char *quote_arg_msys2(const char *arg)
 	else if (!buf.len)
 		return arg;
 	else
-		strbuf_add(&buf, p2, p - p2),
+		strbuf_add(&buf, p2, p - p2);
 
 	strbuf_addch(&buf, '"');
 	return strbuf_detach(&buf, 0);
@@ -1371,7 +1396,10 @@ static inline int match_last_path_component(const char *path, size_t *len,
 
 static int is_msys2_sh(const char *cmd)
 {
-	if (cmd && !strcmp(cmd, "sh")) {
+	if (!cmd)
+		return 0;
+
+	if (!strcmp(cmd, "sh")) {
 		static int ret = -1;
 		char *p;
 
@@ -1391,6 +1419,16 @@ static int is_msys2_sh(const char *cmd)
 		}
 		return ret;
 	}
+
+	if (ends_with(cmd, "\\sh.exe")) {
+		static char *sh;
+
+		if (!sh)
+			sh = path_lookup("sh", 0);
+
+		return !fspathcmp(cmd, sh);
+	}
+
 	return 0;
 }
 
@@ -1406,7 +1444,8 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **deltaen
 	BOOL ret;
 	HANDLE cons;
 	const char *(*quote_arg)(const char *arg) =
-		is_msys2_sh(*argv) ? quote_arg_msys2 : quote_arg_msvc;
+		is_msys2_sh(cmd ? cmd : *argv) ?
+		quote_arg_msys2 : quote_arg_msvc;
 
 	do_unset_environment_variables();
 
@@ -2367,6 +2406,50 @@ static void setup_windows_environment(void)
 	}
 }
 
+int is_valid_win32_path(const char *path)
+{
+	int preceding_space_or_period = 0, i = 0, periods = 0;
+
+	if (!protect_ntfs)
+		return 1;
+
+	skip_dos_drive_prefix((char **)&path);
+
+	for (;;) {
+		char c = *(path++);
+		switch (c) {
+		case '\0':
+		case '/': case '\\':
+			/* cannot end in ` ` or `.`, except for `.` and `..` */
+			if (preceding_space_or_period &&
+			    (i != periods || periods > 2))
+				return 0;
+			if (!c)
+				return 1;
+
+			i = periods = preceding_space_or_period = 0;
+			continue;
+		case '.':
+			periods++;
+			/* fallthru */
+		case ' ':
+			preceding_space_or_period = 1;
+			i++;
+			continue;
+		case ':': /* DOS drive prefix was already skipped */
+		case '<': case '>': case '"': case '|': case '?': case '*':
+			/* illegal character */
+			return 0;
+		default:
+			if (c > '\0' && c < '\x20')
+				/* illegal character */
+				return 0;
+		}
+		preceding_space_or_period = 0;
+		i++;
+	}
+}
+
 #if !defined(_MSC_VER)
 /*
  * Disable MSVCRT command line wildcard expansion (__getmainargs called from

compat/mingw.h

@@ -452,6 +452,20 @@ char *mingw_query_user_email(void);
 #include <inttypes.h>
 #endif
 
+/**
+ * Verifies that the given path is a valid one on Windows.
+ *
+ * In particular, path segments are disallowed which
+ *
+ * - end in a period or a space (except the special directories `.` and `..`).
+ *
+ * - contain any of the reserved characters, e.g. `:`, `;`, `*`, etc
+ *
+ * Returns 1 upon success, otherwise 0.
+ */
+int is_valid_win32_path(const char *path);
+#define is_valid_path(path) is_valid_win32_path(path)
+
 /**
  * Converts UTF-8 encoded string to UTF-16LE.
  *

compat/win32/path-utils.c

@@ -1,5 +1,29 @@
 #include "../../git-compat-util.h"
 
+int win32_has_dos_drive_prefix(const char *path)
+{
+	int i;
+
+	/*
+	 * Does it start with an ASCII letter (i.e. highest bit not set),
+	 * followed by a colon?
+	 */
+	if (!(0x80 & (unsigned char)*path))
+		return *path && path[1] == ':' ? 2 : 0;
+
+	/*
+	 * While drive letters must be letters of the English alphabet, it is
+	 * possible to assign virtually _any_ Unicode character via `subst` as
+	 * a drive letter to "virtual drives". Even `1`, or `ä`. Or fun stuff
+	 * like this:
+	 *
+	 *      subst ֍: %USERPROFILE%\Desktop
+	 */
+	for (i = 1; i < 4 && (0x80 & (unsigned char)path[i]); i++)
+		; /* skip first UTF-8 character */
+	return path[i] == ':' ? i + 1 : 0;
+}
+
 int win32_skip_dos_drive_prefix(char **path)
 {
 	int ret = has_dos_drive_prefix(*path);

compat/win32/path-utils.h

@@ -1,8 +1,9 @@
 #ifndef WIN32_PATH_UTILS_H
 #define WIN32_PATH_UTILS_H
 
-#define has_dos_drive_prefix(path) \
-	(isalpha(*(path)) && (path)[1] == ':' ? 2 : 0)
+int win32_has_dos_drive_prefix(const char *path);
+#define has_dos_drive_prefix win32_has_dos_drive_prefix
+
 int win32_skip_dos_drive_prefix(char **path);
 #define skip_dos_drive_prefix win32_skip_dos_drive_prefix
 static inline int win32_is_dir_sep(int c)

config.mak.uname

@@ -454,7 +454,6 @@ ifneq ($(USE_MSVC_CRTDBG),)
 	# Optionally enable memory leak reporting.
 	BASIC_CFLAGS += -DUSE_MSVC_CRTDBG
 endif
-	BASIC_CFLAGS += -DPROTECT_NTFS_DEFAULT=1
 	# Always give "-Zi" to the compiler and "-debug" to linker (even in
 	# release mode) to force a PDB to be generated (like RelWithDebInfo).
 	BASIC_CFLAGS += -Zi
@@ -616,7 +615,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 		compat/win32/path-utils.o \
 		compat/win32/pthread.o compat/win32/syslog.o \
 		compat/win32/dirent.o
-	BASIC_CFLAGS += -DWIN32 -DPROTECT_NTFS_DEFAULT=1
+	BASIC_CFLAGS += -DWIN32
 	EXTLIBS += -lws2_32
 	GITLIBS += git.res
 	PTHREAD_LIBS =

connect.c

@@ -511,7 +511,7 @@ int url_is_local_not_ssh(const char *url)
 	const char *colon = strchr(url, ':');
 	const char *slash = strchr(url, '/');
 	return !colon || (slash && slash < colon) ||
-		has_dos_drive_prefix(url);
+		(has_dos_drive_prefix(url) && is_valid_path(url));
 }
 
 static const char *prot_name(enum protocol protocol)

environment.c

@@ -78,7 +78,7 @@ enum log_refs_config log_all_ref_updates = LOG_REFS_UNSET;
 int protect_hfs = PROTECT_HFS_DEFAULT;
 
 #ifndef PROTECT_NTFS_DEFAULT
-#define PROTECT_NTFS_DEFAULT 0
+#define PROTECT_NTFS_DEFAULT 1
 #endif
 int protect_ntfs = PROTECT_NTFS_DEFAULT;
 const char *core_fsmonitor;

fast-import.c

@@ -217,6 +217,7 @@ static uintmax_t next_mark;
 static struct strbuf new_data = STRBUF_INIT;
 static int seen_data_command;
 static int require_explicit_termination;
+static int allow_unsafe_features;
 
 /* Signal handling */
 static volatile sig_atomic_t checkpoint_requested;
@@ -1682,6 +1683,12 @@ static void dump_marks(void)
 	if (!export_marks_file || (import_marks_file && !import_marks_file_done))
 		return;
 
+	if (safe_create_leading_directories_const(export_marks_file)) {
+		failure |= error_errno("unable to create leading directories of %s",
+				       export_marks_file);
+		return;
+	}
+
 	if (hold_lock_file_for_update(&mark_lock, export_marks_file, 0) < 0) {
 		failure |= error_errno("Unable to write marks file %s",
 				       export_marks_file);
@@ -3146,7 +3153,6 @@ static void option_import_marks(const char *marks,
 	}
 
 	import_marks_file = make_fast_import_path(marks);
-	safe_create_leading_directories_const(import_marks_file);
 	import_marks_file_from_stream = from_stream;
 	import_marks_file_ignore_missing = ignore_missing;
 }
@@ -3187,7 +3193,6 @@ static void option_active_branches(const char *branches)
 static void option_export_marks(const char *marks)
 {
 	export_marks_file = make_fast_import_path(marks);
-	safe_create_leading_directories_const(export_marks_file);
 }
 
 static void option_cat_blob_fd(const char *fd)
@@ -3230,10 +3235,12 @@ static int parse_one_option(const char *option)
 		option_active_branches(option);
 	} else if (skip_prefix(option, "export-pack-edges=", &option)) {
 		option_export_pack_edges(option);
-	} else if (starts_with(option, "quiet")) {
+	} else if (!strcmp(option, "quiet")) {
 		show_stats = 0;
-	} else if (starts_with(option, "stats")) {
+	} else if (!strcmp(option, "stats")) {
 		show_stats = 1;
+	} else if (!strcmp(option, "allow-unsafe-features")) {
+		; /* already handled during early option parsing */
 	} else {
 		return 0;
 	}
@@ -3241,6 +3248,13 @@ static int parse_one_option(const char *option)
 	return 1;
 }
 
+static void check_unsafe_feature(const char *feature, int from_stream)
+{
+	if (from_stream && !allow_unsafe_features)
+		die(_("feature '%s' forbidden in input without --allow-unsafe-features"),
+		    feature);
+}
+
 static int parse_one_feature(const char *feature, int from_stream)
 {
 	const char *arg;
@@ -3248,10 +3262,13 @@ static int parse_one_feature(const char *feature, int from_stream)
 	if (skip_prefix(feature, "date-format=", &arg)) {
 		option_date_format(arg);
 	} else if (skip_prefix(feature, "import-marks=", &arg)) {
+		check_unsafe_feature("import-marks", from_stream);
 		option_import_marks(arg, from_stream, 0);
 	} else if (skip_prefix(feature, "import-marks-if-exists=", &arg)) {
+		check_unsafe_feature("import-marks-if-exists", from_stream);
 		option_import_marks(arg, from_stream, 1);
 	} else if (skip_prefix(feature, "export-marks=", &arg)) {
+		check_unsafe_feature(feature, from_stream);
 		option_export_marks(arg);
 	} else if (!strcmp(feature, "alias")) {
 		; /* Don't die - this feature is supported */
@@ -3380,6 +3397,20 @@ int cmd_main(int argc, const char **argv)
 	avail_tree_table = xcalloc(avail_tree_table_sz, sizeof(struct avail_tree_content*));
 	marks = mem_pool_calloc(&fi_mem_pool, 1, sizeof(struct mark_set));
 
+	/*
+	 * We don't parse most options until after we've seen the set of
+	 * "feature" lines at the start of the stream (which allows the command
+	 * line to override stream data). But we must do an early parse of any
+	 * command-line options that impact how we interpret the feature lines.
+	 */
+	for (i = 1; i < argc; i++) {
+		const char *arg = argv[i];
+		if (*arg != '-' || !strcmp(arg, "--"))
+			break;
+		if (!strcmp(arg, "--allow-unsafe-features"))
+			allow_unsafe_features = 1;
+	}
+
 	global_argc = argc;
 	global_argv = argv;
 

fsck.c

@@ -68,6 +68,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;
 	FUNC(GITMODULES_SYMLINK, ERROR) \
 	FUNC(GITMODULES_URL, ERROR) \
 	FUNC(GITMODULES_PATH, ERROR) \
+	FUNC(GITMODULES_UPDATE, ERROR) \
 	/* warnings */ \
 	FUNC(BAD_FILEMODE, WARN) \
 	FUNC(EMPTY_NAME, WARN) \
@@ -570,7 +571,7 @@ static int fsck_tree(struct tree *item, struct fsck_options *options)
 
 	while (desc.size) {
 		unsigned short mode;
-		const char *name;
+		const char *name, *backslash;
 		const struct object_id *oid;
 
 		oid = tree_entry_extract(&desc, &name, &mode);
@@ -592,6 +593,22 @@ static int fsck_tree(struct tree *item, struct fsck_options *options)
 						 ".gitmodules is a symbolic link");
 		}
 
+		if ((backslash = strchr(name, '\\'))) {
+			while (backslash) {
+				backslash++;
+				has_dotgit |= is_ntfs_dotgit(backslash);
+				if (is_ntfs_dotgitmodules(backslash)) {
+					if (!S_ISLNK(mode))
+						oidset_insert(&gitmodules_found, oid);
+					else
+						retval += report(options, &item->object,
+								 FSCK_MSG_GITMODULES_SYMLINK,
+								 ".gitmodules is a symbolic link");
+				}
+				backslash = strchr(backslash, '\\');
+			}
+		}
+
 		if (update_tree_entry_gently(&desc)) {
 			retval += report(options, &item->object, FSCK_MSG_BAD_TREE, "cannot be parsed as a tree");
 			break;
@@ -965,6 +982,12 @@ static int fsck_gitmodules_fn(const char *var, const char *value, void *vdata)
 				    FSCK_MSG_GITMODULES_PATH,
 				    "disallowed submodule path: %s",
 				    value);
+	if (!strcmp(key, "update") && value &&
+	    parse_submodule_update_type(value) == SM_UPDATE_COMMAND)
+		data->ret |= report(data->options, data->obj,
+				    FSCK_MSG_GITMODULES_UPDATE,
+				    "disallowed submodule update setting: %s",
+				    value);
 	free(name);
 
 	return 0;

git-compat-util.h

@@ -397,6 +397,10 @@ static inline int git_offset_1st_component(const char *path)
 #define offset_1st_component git_offset_1st_component
 #endif
 
+#ifndef is_valid_path
+#define is_valid_path(path) 1
+#endif
+
 #ifndef find_last_dir_sep
 static inline char *git_find_last_dir_sep(const char *path)
 {

git-submodule.sh

@@ -36,6 +36,7 @@ reference=
 cached=
 recursive=
 init=
+require_init=
 files=
 remote=
 nofetch=
@@ -466,6 +467,10 @@ cmd_update()
 		-i|--init)
 			init=1
 			;;
+		--require-init)
+			init=1
+			require_init=1
+			;;
 		--remote)
 			remote=1
 			;;
@@ -548,6 +553,7 @@ cmd_update()
 		${reference:+"$reference"} \
 		${dissociate:+"--dissociate"} \
 		${depth:+--depth "$depth"} \
+		${require_init:+--require-init} \
 		$recommend_shallow \
 		$jobs \
 		-- \

path.c

@@ -1316,37 +1316,77 @@ int daemon_avoid_alias(const char *p)
 	}
 }
 
-static int only_spaces_and_periods(const char *path, size_t len, size_t skip)
-{
-	if (len < skip)
-		return 0;
-	len -= skip;
-	path += skip;
-	while (len-- > 0) {
-		char c = *(path++);
-		if (c != ' ' && c != '.')
-			return 0;
-	}
-	return 1;
-}
-
+/*
+ * On NTFS, we need to be careful to disallow certain synonyms of the `.git/`
+ * directory:
+ *
+ * - For historical reasons, file names that end in spaces or periods are
+ *   automatically trimmed. Therefore, `.git . . ./` is a valid way to refer
+ *   to `.git/`.
+ *
+ * - For other historical reasons, file names that do not conform to the 8.3
+ *   format (up to eight characters for the basename, three for the file
+ *   extension, certain characters not allowed such as `+`, etc) are associated
+ *   with a so-called "short name", at least on the `C:` drive by default.
+ *   Which means that `git~1/` is a valid way to refer to `.git/`.
+ *
+ *   Note: Technically, `.git/` could receive the short name `git~2` if the
+ *   short name `git~1` were already used. In Git, however, we guarantee that
+ *   `.git` is the first item in a directory, therefore it will be associated
+ *   with the short name `git~1` (unless short names are disabled).
+ *
+ * - For yet other historical reasons, NTFS supports so-called "Alternate Data
+ *   Streams", i.e. metadata associated with a given file, referred to via
+ *   `<filename>:<stream-name>:<stream-type>`. There exists a default stream
+ *   type for directories, allowing `.git/` to be accessed via
+ *   `.git::$INDEX_ALLOCATION/`.
+ *
+ * When this function returns 1, it indicates that the specified file/directory
+ * name refers to a `.git` file or directory, or to any of these synonyms, and
+ * Git should therefore not track it.
+ *
+ * For performance reasons, _all_ Alternate Data Streams of `.git/` are
+ * forbidden, not just `::$INDEX_ALLOCATION`.
+ *
+ * This function is intended to be used by `git fsck` even on platforms where
+ * the backslash is a regular filename character, therefore it needs to handle
+ * backlash characters in the provided `name` specially: they are interpreted
+ * as directory separators.
+ */
 int is_ntfs_dotgit(const char *name)
 {
-	size_t len;
+	char c;
 
-	for (len = 0; ; len++)
-		if (!name[len] || name[len] == '\\' || is_dir_sep(name[len])) {
-			if (only_spaces_and_periods(name, len, 4) &&
-					!strncasecmp(name, ".git", 4))
-				return 1;
-			if (only_spaces_and_periods(name, len, 5) &&
-					!strncasecmp(name, "git~1", 5))
-				return 1;
-			if (name[len] != '\\')
-				return 0;
-			name += len + 1;
-			len = -1;
-		}
+	/*
+	 * Note that when we don't find `.git` or `git~1` we end up with `name`
+	 * advanced partway through the string. That's okay, though, as we
+	 * return immediately in those cases, without looking at `name` any
+	 * further.
+	 */
+	c = *(name++);
+	if (c == '.') {
+		/* .git */
+		if (((c = *(name++)) != 'g' && c != 'G') ||
+		    ((c = *(name++)) != 'i' && c != 'I') ||
+		    ((c = *(name++)) != 't' && c != 'T'))
+			return 0;
+	} else if (c == 'g' || c == 'G') {
+		/* git ~1 */
+		if (((c = *(name++)) != 'i' && c != 'I') ||
+		    ((c = *(name++)) != 't' && c != 'T') ||
+		    *(name++) != '~' ||
+		    *(name++) != '1')
+			return 0;
+	} else
+		return 0;
+
+	for (;;) {
+		c = *(name++);
+		if (!c || c == '\\' || c == '/' || c == ':')
+			return 1;
+		if (c != '.' && c != ' ')
+			return 0;
+	}
 }
 
 static int is_ntfs_dot_generic(const char *name,
@@ -1362,7 +1402,7 @@ static int is_ntfs_dot_generic(const char *name,
 only_spaces_and_periods:
 		for (;;) {
 			char c = name[i++];
-			if (!c)
+			if (!c || c == ':')
 				return 1;
 			if (c != ' ' && c != '.')
 				return 0;

read-cache.c

@@ -964,6 +964,9 @@ int verify_path(const char *path, unsigned mode)
 	if (has_dos_drive_prefix(path))
 		return 0;
 
+	if (!is_valid_path(path))
+		return 0;
+
 	goto inside;
 	for (;;) {
 		if (!c)
@@ -991,7 +994,15 @@ inside:
 			if ((c == '.' && !verify_dotfile(path, mode)) ||
 			    is_dir_sep(c) || c == '\0')
 				return 0;
+		} else if (c == '\\' && protect_ntfs) {
+			if (is_ntfs_dotgit(path))
+				return 0;
+			if (S_ISLNK(mode)) {
+				if (is_ntfs_dotgitmodules(path))
+					return 0;
+			}
 		}
+
 		c = *path++;
 	}
 }

submodule-config.c

@@ -409,6 +409,13 @@ struct parse_config_parameter {
 	int overwrite;
 };
 
+/*
+ * Parse a config item from .gitmodules.
+ *
+ * This does not handle submodule-related configuration from the main
+ * config store (.git/config, etc).  Callers are responsible for
+ * checking for overrides in the main config store when appropriate.
+ */
 static int parse_config(const char *var, const char *value, void *data)
 {
 	struct parse_config_parameter *me = data;
@@ -486,8 +493,9 @@ static int parse_config(const char *var, const char *value, void *data)
 			warn_multiple_config(me->treeish_name, submodule->name,
 					     "update");
 		else if (parse_submodule_update_strategy(value,
-			 &submodule->update_strategy) < 0)
-				die(_("invalid value for %s"), var);
+			 &submodule->update_strategy) < 0 ||
+			 submodule->update_strategy.type == SM_UPDATE_COMMAND)
+			die(_("invalid value for %s"), var);
 	} else if (!strcmp(item.buf, "shallow")) {
 		if (!me->overwrite && submodule->recommend_shallow != -1)
 			warn_multiple_config(me->treeish_name, submodule->name,

submodule.c

@@ -1993,6 +1993,47 @@ out:
 	return ret;
 }
 
+int validate_submodule_git_dir(char *git_dir, const char *submodule_name)
+{
+	size_t len = strlen(git_dir), suffix_len = strlen(submodule_name);
+	char *p;
+	int ret = 0;
+
+	if (len <= suffix_len || (p = git_dir + len - suffix_len)[-1] != '/' ||
+	    strcmp(p, submodule_name))
+		BUG("submodule name '%s' not a suffix of git dir '%s'",
+		    submodule_name, git_dir);
+
+	/*
+	 * We prevent the contents of sibling submodules' git directories to
+	 * clash.
+	 *
+	 * Example: having a submodule named `hippo` and another one named
+	 * `hippo/hooks` would result in the git directories
+	 * `.git/modules/hippo/` and `.git/modules/hippo/hooks/`, respectively,
+	 * but the latter directory is already designated to contain the hooks
+	 * of the former.
+	 */
+	for (; *p; p++) {
+		if (is_dir_sep(*p)) {
+			char c = *p;
+
+			*p = '\0';
+			if (is_git_directory(git_dir))
+				ret = -1;
+			*p = c;
+
+			if (ret < 0)
+				return error(_("submodule git dir '%s' is "
+					       "inside git dir '%.*s'"),
+					     git_dir,
+					     (int)(p - git_dir), git_dir);
+		}
+	}
+
+	return 0;
+}
+
 /*
  * Embeds a single submodules git directory into the superprojects git dir,
  * non recursively.
@@ -2000,7 +2041,7 @@ out:
 static void relocate_single_git_dir_into_superproject(const char *path)
 {
 	char *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;
-	const char *new_git_dir;
+	char *new_git_dir;
 	const struct submodule *sub;
 
 	if (submodule_uses_worktrees(path))
@@ -2018,10 +2059,14 @@ static void relocate_single_git_dir_into_superproject(const char *path)
 	if (!sub)
 		die(_("could not lookup name for submodule '%s'"), path);
 
-	new_git_dir = git_path("modules/%s", sub->name);
+	new_git_dir = git_pathdup("modules/%s", sub->name);
+	if (validate_submodule_git_dir(new_git_dir, sub->name) < 0)
+		die(_("refusing to move '%s' into an existing git dir"),
+		    real_old_git_dir);
 	if (safe_create_leading_directories_const(new_git_dir) < 0)
 		die(_("could not create directory '%s'"), new_git_dir);
 	real_new_git_dir = real_pathdup(new_git_dir, 1);
+	free(new_git_dir);
 
 	fprintf(stderr, _("Migrating git directory of '%s%s' from\n'%s' to\n'%s'\n"),
 		get_super_prefix_or_empty(), path,

submodule.h

@@ -124,6 +124,11 @@ int push_unpushed_submodules(struct repository *r,
  */
 int submodule_to_gitdir(struct strbuf *buf, const char *submodule);
 
+/*
+ * Make sure that no submodule's git dir is nested in a sibling submodule's.
+ */
+int validate_submodule_git_dir(char *git_dir, const char *submodule_name);
+
 #define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)
 #define SUBMODULE_MOVE_HEAD_FORCE   (1<<1)
 int submodule_move_head(const char *path,

t/helper/test-drop-caches.c

@@ -8,18 +8,21 @@ static int cmd_sync(void)
 {
 	char Buffer[MAX_PATH];
 	DWORD dwRet;
-	char szVolumeAccessPath[] = "\\\\.\\X:";
+	char szVolumeAccessPath[] = "\\\\.\\XXXX:";
 	HANDLE hVolWrite;
-	int success = 0;
+	int success = 0, dos_drive_prefix;
 
 	dwRet = GetCurrentDirectory(MAX_PATH, Buffer);
 	if ((0 == dwRet) || (dwRet > MAX_PATH))
 		return error("Error getting current directory");
 
-	if (!has_dos_drive_prefix(Buffer))
+	dos_drive_prefix = has_dos_drive_prefix(Buffer);
+	if (!dos_drive_prefix)
 		return error("'%s': invalid drive letter", Buffer);
 
-	szVolumeAccessPath[4] = Buffer[0];
+	memcpy(szVolumeAccessPath, Buffer, dos_drive_prefix);
+	szVolumeAccessPath[dos_drive_prefix] = '\0';
+
 	hVolWrite = CreateFile(szVolumeAccessPath, GENERIC_READ | GENERIC_WRITE,
 		FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);
 	if (INVALID_HANDLE_VALUE == hVolWrite)

t/helper/test-path-utils.c

@@ -185,6 +185,99 @@ static int cmp_by_st_size(const void *a, const void *b)
 	return x > y ? -1 : (x < y ? +1 : 0);
 }
 
+/*
+ * A very simple, reproducible pseudo-random generator. Copied from
+ * `test-genrandom.c`.
+ */
+static uint64_t my_random_value = 1234;
+
+static uint64_t my_random(void)
+{
+	my_random_value = my_random_value * 1103515245 + 12345;
+	return my_random_value;
+}
+
+/*
+ * A fast approximation of the square root, without requiring math.h.
+ *
+ * It uses Newton's method to approximate the solution of 0 = x^2 - value.
+ */
+static double my_sqrt(double value)
+{
+	const double epsilon = 1e-6;
+	double x = value;
+
+	if (value == 0)
+		return 0;
+
+	for (;;) {
+		double delta = (value / x - x) / 2;
+		if (delta < epsilon && delta > -epsilon)
+			return x + delta;
+		x += delta;
+	}
+}
+
+static int protect_ntfs_hfs_benchmark(int argc, const char **argv)
+{
+	size_t i, j, nr, min_len = 3, max_len = 20;
+	char **names;
+	int repetitions = 15, file_mode = 0100644;
+	uint64_t begin, end;
+	double m[3][2], v[3][2];
+	uint64_t cumul;
+	double cumul2;
+
+	if (argc > 1 && !strcmp(argv[1], "--with-symlink-mode")) {
+		file_mode = 0120000;
+		argc--;
+		argv++;
+	}
+
+	nr = argc > 1 ? strtoul(argv[1], NULL, 0) : 1000000;
+	ALLOC_ARRAY(names, nr);
+
+	if (argc > 2) {
+		min_len = strtoul(argv[2], NULL, 0);
+		if (argc > 3)
+			max_len = strtoul(argv[3], NULL, 0);
+		if (min_len > max_len)
+			die("min_len > max_len");
+	}
+
+	for (i = 0; i < nr; i++) {
+		size_t len = min_len + (my_random() % (max_len + 1 - min_len));
+
+		names[i] = xmallocz(len);
+		while (len > 0)
+			names[i][--len] = (char)(' ' + (my_random() % ('\x7f' - ' ')));
+	}
+
+	for (protect_ntfs = 0; protect_ntfs < 2; protect_ntfs++)
+		for (protect_hfs = 0; protect_hfs < 2; protect_hfs++) {
+			cumul = 0;
+			cumul2 = 0;
+			for (i = 0; i < repetitions; i++) {
+				begin = getnanotime();
+				for (j = 0; j < nr; j++)
+					verify_path(names[j], file_mode);
+				end = getnanotime();
+				printf("protect_ntfs = %d, protect_hfs = %d: %lfms\n", protect_ntfs, protect_hfs, (end-begin) / (double)1e6);
+				cumul += end - begin;
+				cumul2 += (end - begin) * (end - begin);
+			}
+			m[protect_ntfs][protect_hfs] = cumul / (double)repetitions;
+			v[protect_ntfs][protect_hfs] = my_sqrt(cumul2 / (double)repetitions - m[protect_ntfs][protect_hfs] * m[protect_ntfs][protect_hfs]);
+			printf("mean: %lfms, stddev: %lfms\n", m[protect_ntfs][protect_hfs] / (double)1e6, v[protect_ntfs][protect_hfs] / (double)1e6);
+		}
+
+	for (protect_ntfs = 0; protect_ntfs < 2; protect_ntfs++)
+		for (protect_hfs = 0; protect_hfs < 2; protect_hfs++)
+			printf("ntfs=%d/hfs=%d: %lf%% slower\n", protect_ntfs, protect_hfs, (m[protect_ntfs][protect_hfs] - m[0][0]) * 100 / m[0][0]);
+
+	return 0;
+}
+
 int cmd__path_utils(int argc, const char **argv)
 {
 	if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) {
@@ -355,6 +448,26 @@ int cmd__path_utils(int argc, const char **argv)
 		return !!res;
 	}
 
+	if (argc > 1 && !strcmp(argv[1], "protect_ntfs_hfs"))
+		return !!protect_ntfs_hfs_benchmark(argc - 1, argv + 1);
+
+	if (argc > 1 && !strcmp(argv[1], "is_valid_path")) {
+		int res = 0, expect = 1, i;
+
+		for (i = 2; i < argc; i++)
+			if (!strcmp("--not", argv[i]))
+				expect = 0;
+			else if (expect != is_valid_path(argv[i]))
+				res = error("'%s' is%s a valid path",
+					    argv[i], expect ? " not" : "");
+			else
+				fprintf(stderr,
+					"'%s' is%s a valid path\n",
+					argv[i], expect ? "" : " not");
+
+		return !!res;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

t/helper/test-run-command.c

@@ -18,8 +18,8 @@
 #include "string-list.h"
 #include "thread-utils.h"
 #include "wildmatch.h"
-#include <string.h>
-#include <errno.h>
+#include "gettext.h"
+#include "parse-options.h"
 
 static int number_callbacks;
 static int parallel_next(struct child_process *cp,
@@ -200,6 +200,134 @@ static int testsuite(int argc, const char **argv)
 	return !!ret;
 }
 
+static uint64_t my_random_next = 1234;
+
+static uint64_t my_random(void)
+{
+	uint64_t res = my_random_next;
+	my_random_next = my_random_next * 1103515245 + 12345;
+	return res;
+}
+
+static int quote_stress_test(int argc, const char **argv)
+{
+	/*
+	 * We are running a quote-stress test.
+	 * spawn a subprocess that runs quote-stress with a
+	 * special option that echoes back the arguments that
+	 * were passed in.
+	 */
+	char special[] = ".?*\\^_\"'`{}()[]<>@~&+:;$%"; // \t\r\n\a";
+	int i, j, k, trials = 100, skip = 0, msys2 = 0;
+	struct strbuf out = STRBUF_INIT;
+	struct argv_array args = ARGV_ARRAY_INIT;
+	struct option options[] = {
+		OPT_INTEGER('n', "trials", &trials, "Number of trials"),
+		OPT_INTEGER('s', "skip", &skip, "Skip <n> trials"),
+		OPT_BOOL('m', "msys2", &msys2, "Test quoting for MSYS2's sh"),
+		OPT_END()
+	};
+	const char * const usage[] = {
+		"test-tool run-command quote-stress-test <options>",
+		NULL
+	};
+
+	argc = parse_options(argc, argv, NULL, options, usage, 0);
+
+	setenv("MSYS_NO_PATHCONV", "1", 0);
+
+	for (i = 0; i < trials; i++) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		size_t arg_count, arg_offset;
+		int ret = 0;
+
+		argv_array_clear(&args);
+		if (msys2)
+			argv_array_pushl(&args, "sh", "-c",
+					 "printf %s\\\\0 \"$@\"", "skip", NULL);
+		else
+			argv_array_pushl(&args, "test-tool", "run-command",
+					 "quote-echo", NULL);
+		arg_offset = args.argc;
+
+		if (argc > 0) {
+			trials = 1;
+			arg_count = argc;
+			for (j = 0; j < arg_count; j++)
+				argv_array_push(&args, argv[j]);
+		} else {
+			arg_count = 1 + (my_random() % 5);
+			for (j = 0; j < arg_count; j++) {
+				char buf[20];
+				size_t min_len = 1;
+				size_t arg_len = min_len +
+					(my_random() % (ARRAY_SIZE(buf) - min_len));
+
+				for (k = 0; k < arg_len; k++)
+					buf[k] = special[my_random() %
+						ARRAY_SIZE(special)];
+				buf[arg_len] = '\0';
+
+				argv_array_push(&args, buf);
+			}
+		}
+
+		if (i < skip)
+			continue;
+
+		cp.argv = args.argv;
+		strbuf_reset(&out);
+		if (pipe_command(&cp, NULL, 0, &out, 0, NULL, 0) < 0)
+			return error("Failed to spawn child process");
+
+		for (j = 0, k = 0; j < arg_count; j++) {
+			const char *arg = args.argv[j + arg_offset];
+
+			if (strcmp(arg, out.buf + k))
+				ret = error("incorrectly quoted arg: '%s', "
+					    "echoed back as '%s'",
+					     arg, out.buf + k);
+			k += strlen(out.buf + k) + 1;
+		}
+
+		if (k != out.len)
+			ret = error("got %d bytes, but consumed only %d",
+				     (int)out.len, (int)k);
+
+		if (ret) {
+			fprintf(stderr, "Trial #%d failed. Arguments:\n", i);
+			for (j = 0; j < arg_count; j++)
+				fprintf(stderr, "arg #%d: '%s'\n",
+					(int)j, args.argv[j + arg_offset]);
+
+			strbuf_release(&out);
+			argv_array_clear(&args);
+
+			return ret;
+		}
+
+		if (i && (i % 100) == 0)
+			fprintf(stderr, "Trials completed: %d\n", (int)i);
+	}
+
+	strbuf_release(&out);
+	argv_array_clear(&args);
+
+	return 0;
+}
+
+static int quote_echo(int argc, const char **argv)
+{
+	while (argc > 1) {
+		fwrite(argv[1], strlen(argv[1]), 1, stdout);
+		fputc('\0', stdout);
+		argv++;
+		argc--;
+	}
+
+	return 0;
+}
+
 int cmd__run_command(int argc, const char **argv)
 {
 	struct child_process proc = CHILD_PROCESS_INIT;
@@ -208,6 +336,12 @@ int cmd__run_command(int argc, const char **argv)
 	if (argc > 1 && !strcmp(argv[1], "testsuite"))
 		exit(testsuite(argc - 1, argv + 1));
 
+	if (argc >= 2 && !strcmp(argv[1], "quote-stress-test"))
+		return !!quote_stress_test(argc - 1, argv + 1);
+
+	if (argc >= 2 && !strcmp(argv[1], "quote-echo"))
+		return !!quote_echo(argc - 1, argv + 1);
+
 	if (argc < 3)
 		return 1;
 	while (!strcmp(argv[1], "env")) {

t/t0060-path-utils.sh

@@ -165,6 +165,15 @@ test_expect_success 'absolute path rejects the empty string' '
 	test_must_fail test-tool path-utils absolute_path ""
 '
 
+test_expect_success MINGW '<drive-letter>:\\abc is an absolute path' '
+	for letter in : \" C Z 1 ä
+	do
+		path=$letter:\\abc &&
+		absolute="$(test-tool path-utils absolute_path "$path")" &&
+		test "$path" = "$absolute" || return 1
+	done
+'
+
 test_expect_success 'real path rejects the empty string' '
 	test_must_fail test-tool path-utils real_path ""
 '
@@ -423,6 +432,9 @@ test_expect_success 'match .gitmodules' '
 		~1000000 \
 		~9999999 \
 		\
+		.gitmodules:\$DATA \
+		"gitmod~4 . :\$DATA" \
+		\
 		--not \
 		".gitmodules x"  \
 		".gitmodules .x" \
@@ -447,7 +459,25 @@ test_expect_success 'match .gitmodules' '
 		\
 		GI7EB~1 \
 		GI7EB~01 \
-		GI7EB~1X
+		GI7EB~1X \
+		\
+		.gitmodules,:\$DATA
+'
+
+test_expect_success MINGW 'is_valid_path() on Windows' '
+       test-tool path-utils is_valid_path \
+		win32 \
+		"win32 x" \
+		../hello.txt \
+		C:\\git \
+		\
+		--not \
+		"win32 "  \
+		"win32 /x "  \
+		"win32."  \
+		"win32 . ." \
+		.../hello.txt \
+		colon:test
 '
 
 test_done

t/t1014-read-tree-confusing.sh

@@ -49,6 +49,7 @@ git~1
 .git.SPACE .git.{space}
 .\\\\.GIT\\\\foobar backslashes
 .git\\\\foobar backslashes2
+.git...:alternate-stream
 EOF
 
 test_expect_success 'utf-8 paths allowed with core.protectHFS off' '

t/t1450-fsck.sh

@@ -441,6 +441,7 @@ while read name path pretty; do
 		(
 			git init $name-$type &&
 			cd $name-$type &&
+			git config core.protectNTFS false &&
 			echo content >file &&
 			git add file &&
 			git commit -m base &&

t/t6130-pathspec-noglob.sh

@@ -10,6 +10,7 @@ test_expect_success 'create commits with glob characters' '
 	# the name "f*" in the worktree, because it is not allowed
 	# on Windows (the tests below do not depend on the presence
 	# of the file in the worktree)
+	git config core.protectNTFS false &&
 	git update-index --add --cacheinfo 100644 "$(git rev-parse HEAD:foo)" "f*" &&
 	test_tick &&
 	git commit -m star &&

t/t7406-submodule-update.sh

@@ -406,12 +406,26 @@ test_expect_success 'submodule update - command in .git/config' '
 	)
 '
 
-test_expect_success 'submodule update - command in .gitmodules is ignored' '
+test_expect_success 'submodule update - command in .gitmodules is rejected' '
 	test_when_finished "git -C super reset --hard HEAD^" &&
 	git -C super config -f .gitmodules submodule.submodule.update "!false" &&
 	git -C super commit -a -m "add command to .gitmodules file" &&
 	git -C super/submodule reset --hard $submodulesha1^ &&
-	git -C super submodule update submodule
+	test_must_fail git -C super submodule update submodule
+'
+
+test_expect_success 'fsck detects command in .gitmodules' '
+	git init command-in-gitmodules &&
+	(
+		cd command-in-gitmodules &&
+		git submodule add ../submodule submodule &&
+		test_commit adding-submodule &&
+
+		git config -f .gitmodules submodule.submodule.update "!false" &&
+		git add .gitmodules &&
+		test_commit configuring-update &&
+		test_must_fail git fsck
+	)
 '
 
 cat << EOF >expect
@@ -480,6 +494,9 @@ test_expect_success 'recursive submodule update - command in .git/config catches
 '
 
 test_expect_success 'submodule init does not copy command into .git/config' '
+	test_when_finished "git -C super update-index --force-remove submodule1" &&
+	test_when_finished git config -f super/.gitmodules \
+		--remove-section submodule.submodule1 &&
 	(cd super &&
 	 git ls-files -s submodule >out &&
 	 H=$(cut -d" " -f2 out) &&
@@ -488,10 +505,9 @@ test_expect_success 'submodule init does not copy command into .git/config' '
 	 git config -f .gitmodules submodule.submodule1.path submodule1 &&
 	 git config -f .gitmodules submodule.submodule1.url ../submodule &&
 	 git config -f .gitmodules submodule.submodule1.update !false &&
-	 git submodule init submodule1 &&
-	 echo "none" >expect &&
-	 git config submodule.submodule1.update >actual &&
-	 test_cmp expect actual
+	 test_must_fail git submodule init submodule1 &&
+	 test_expect_code 1 git config submodule.submodule1.update >actual &&
+	 test_must_be_empty actual
 	)
 '
 

t/t7415-submodule-names.sh

@@ -191,4 +191,60 @@ test_expect_success 'fsck detects corrupt .gitmodules' '
 	)
 '
 
+test_expect_success MINGW 'prevent git~1 squatting on Windows' '
+	git init squatting &&
+	(
+		cd squatting &&
+		mkdir a &&
+		touch a/..git &&
+		git add a/..git &&
+		test_tick &&
+		git commit -m initial &&
+
+		modules="$(test_write_lines \
+			"[submodule \"b.\"]" "url = ." "path = c" \
+			"[submodule \"b\"]" "url = ." "path = d\\\\a" |
+			git hash-object -w --stdin)" &&
+		rev="$(git rev-parse --verify HEAD)" &&
+		hash="$(echo x | git hash-object -w --stdin)" &&
+		git -c core.protectNTFS=false update-index --add \
+			--cacheinfo 100644,$modules,.gitmodules \
+			--cacheinfo 160000,$rev,c \
+			--cacheinfo 160000,$rev,d\\a \
+			--cacheinfo 100644,$hash,d./a/x \
+			--cacheinfo 100644,$hash,d./a/..git &&
+		test_tick &&
+		git -c core.protectNTFS=false commit -m "module" &&
+		test_must_fail git show HEAD: 2>err &&
+		test_i18ngrep backslash err
+	) &&
+	test_must_fail git -c core.protectNTFS=false \
+		clone --recurse-submodules squatting squatting-clone 2>err &&
+	test_i18ngrep -e "directory not empty" -e "not an empty directory" err &&
+	! grep gitdir squatting-clone/d/a/git~2
+'
+
+test_expect_success 'git dirs of sibling submodules must not be nested' '
+	git init nested &&
+	test_commit -C nested nested &&
+	(
+		cd nested &&
+		cat >.gitmodules <<-EOF &&
+		[submodule "hippo"]
+			url = .
+			path = thing1
+		[submodule "hippo/hooks"]
+			url = .
+			path = thing2
+		EOF
+		git clone . thing1 &&
+		git clone . thing2 &&
+		git add .gitmodules thing1 thing2 &&
+		test_tick &&
+		git commit -m nested
+	) &&
+	test_must_fail git clone --recurse-submodules nested clone 2>err &&
+	test_i18ngrep "is inside git dir" err
+'
+
 test_done

t/t7416-submodule-dash-url.sh

@@ -46,4 +46,18 @@ test_expect_success 'fsck rejects unprotected dash' '
 	grep gitmodulesUrl err
 '
 
+test_expect_success 'trailing backslash is handled correctly' '
+	git init testmodule &&
+	test_commit -C testmodule c &&
+	git submodule add ./testmodule &&
+	: ensure that the name ends in a double backslash &&
+	sed -e "s|\\(submodule \"testmodule\\)\"|\\1\\\\\\\\\"|" \
+		-e "s|url = .*|url = \" --should-not-be-an-option\"|" \
+		<.gitmodules >.new &&
+	mv .new .gitmodules &&
+	git commit -am "Add testmodule" &&
+	test_must_fail git clone --verbose --recurse-submodules . dolly 2>err &&
+	test_i18ngrep ! "unknown option" err
+'
+
 test_done

t/t7417-submodule-path-url.sh

@@ -25,4 +25,21 @@ test_expect_success 'fsck rejects unprotected dash' '
 	grep gitmodulesPath err
 '
 
+test_expect_success MINGW 'submodule paths disallows trailing spaces' '
+	git init super &&
+	test_must_fail git -C super submodule add ../upstream "sub " &&
+
+	: add "sub", then rename "sub" to "sub ", the hard way &&
+	git -C super submodule add ../upstream sub &&
+	tree=$(git -C super write-tree) &&
+	git -C super ls-tree $tree >tree &&
+	sed "s/sub/sub /" <tree >tree.new &&
+	tree=$(git -C super mktree <tree.new) &&
+	commit=$(echo with space | git -C super commit-tree $tree) &&
+	git -C super update-ref refs/heads/master $commit &&
+
+	test_must_fail git clone --recurse-submodules super dst 2>err &&
+	test_i18ngrep "sub " err
+'
+
 test_done

t/t9300-fast-import.sh

@@ -2143,12 +2143,27 @@ test_expect_success 'R: abort on receiving feature after data command' '
 	test_must_fail git fast-import <input
 '
 
+test_expect_success 'R: import-marks features forbidden by default' '
+	>git.marks &&
+	echo "feature import-marks=git.marks" >input &&
+	test_must_fail git fast-import <input &&
+	echo "feature import-marks-if-exists=git.marks" >input &&
+	test_must_fail git fast-import <input
+'
+
 test_expect_success 'R: only one import-marks feature allowed per stream' '
+	>git.marks &&
+	>git2.marks &&
 	cat >input <<-EOF &&
 	feature import-marks=git.marks
 	feature import-marks=git2.marks
 	EOF
 
+	test_must_fail git fast-import --allow-unsafe-features <input
+'
+
+test_expect_success 'R: export-marks feature forbidden by default' '
+	echo "feature export-marks=git.marks" >input &&
 	test_must_fail git fast-import <input
 '
 
@@ -2162,19 +2177,29 @@ test_expect_success 'R: export-marks feature results in a marks file being creat
 
 	EOF
 
-	cat input | git fast-import &&
+	git fast-import --allow-unsafe-features <input &&
 	grep :1 git.marks
 '
 
 test_expect_success 'R: export-marks options can be overridden by commandline options' '
-	cat input | git fast-import --export-marks=other.marks &&
-	grep :1 other.marks
+	cat >input <<-\EOF &&
+	feature export-marks=feature-sub/git.marks
+	blob
+	mark :1
+	data 3
+	hi
+
+	EOF
+	git fast-import --allow-unsafe-features \
+			--export-marks=cmdline-sub/other.marks <input &&
+	grep :1 cmdline-sub/other.marks &&
+	test_path_is_missing feature-sub
 '
 
 test_expect_success 'R: catch typo in marks file name' '
 	test_must_fail git fast-import --import-marks=nonexistent.marks </dev/null &&
 	echo "feature import-marks=nonexistent.marks" |
-	test_must_fail git fast-import
+	test_must_fail git fast-import --allow-unsafe-features
 '
 
 test_expect_success 'R: import and output marks can be the same file' '
@@ -2229,7 +2254,8 @@ test_expect_success 'R: --import-marks-if-exists' '
 test_expect_success 'R: feature import-marks-if-exists' '
 	rm -f io.marks &&
 
-	git fast-import --export-marks=io.marks <<-\EOF &&
+	git fast-import --export-marks=io.marks \
+			--allow-unsafe-features <<-\EOF &&
 	feature import-marks-if-exists=not_io.marks
 	EOF
 	test_must_be_empty io.marks &&
@@ -2240,7 +2266,8 @@ test_expect_success 'R: feature import-marks-if-exists' '
 	echo ":1 $blob" >expect &&
 	echo ":2 $blob" >>expect &&
 
-	git fast-import --export-marks=io.marks <<-\EOF &&
+	git fast-import --export-marks=io.marks \
+			--allow-unsafe-features <<-\EOF &&
 	feature import-marks-if-exists=io.marks
 	blob
 	mark :2
@@ -2253,7 +2280,8 @@ test_expect_success 'R: feature import-marks-if-exists' '
 	echo ":3 $blob" >>expect &&
 
 	git fast-import --import-marks=io.marks \
-			--export-marks=io.marks <<-\EOF &&
+			--export-marks=io.marks \
+			--allow-unsafe-features <<-\EOF &&
 	feature import-marks-if-exists=not_io.marks
 	blob
 	mark :3
@@ -2264,7 +2292,8 @@ test_expect_success 'R: feature import-marks-if-exists' '
 	test_cmp expect io.marks &&
 
 	git fast-import --import-marks-if-exists=not_io.marks \
-			--export-marks=io.marks <<-\EOF &&
+			--export-marks=io.marks \
+			--allow-unsafe-features <<-\EOF &&
 	feature import-marks-if-exists=io.marks
 	EOF
 	test_must_be_empty io.marks
@@ -2276,7 +2305,7 @@ test_expect_success 'R: import to output marks works without any content' '
 	feature export-marks=marks.new
 	EOF
 
-	cat input | git fast-import &&
+	git fast-import --allow-unsafe-features <input &&
 	test_cmp marks.out marks.new
 '
 
@@ -2286,7 +2315,7 @@ test_expect_success 'R: import marks prefers commandline marks file over the str
 	feature export-marks=marks.new
 	EOF
 
-	cat input | git fast-import --import-marks=marks.out &&
+	git fast-import --import-marks=marks.out --allow-unsafe-features <input &&
 	test_cmp marks.out marks.new
 '
 
@@ -2299,7 +2328,8 @@ test_expect_success 'R: multiple --import-marks= should be honoured' '
 
 	head -n2 marks.out > one.marks &&
 	tail -n +3 marks.out > two.marks &&
-	git fast-import --import-marks=one.marks --import-marks=two.marks <input &&
+	git fast-import --import-marks=one.marks --import-marks=two.marks \
+		--allow-unsafe-features <input &&
 	test_cmp marks.out combined.marks
 '
 
@@ -2312,7 +2342,7 @@ test_expect_success 'R: feature relative-marks should be honoured' '
 
 	mkdir -p .git/info/fast-import/ &&
 	cp marks.new .git/info/fast-import/relative.in &&
-	git fast-import <input &&
+	git fast-import --allow-unsafe-features <input &&
 	test_cmp marks.new .git/info/fast-import/relative.out
 '
 
@@ -2324,7 +2354,7 @@ test_expect_success 'R: feature no-relative-marks should be honoured' '
 	feature export-marks=non-relative.out
 	EOF
 
-	git fast-import <input &&
+	git fast-import --allow-unsafe-features <input &&
 	test_cmp marks.new non-relative.out
 '
 
@@ -2594,7 +2624,7 @@ test_expect_success 'R: quiet option results in no stats being output' '
 
 	EOF
 
-	cat input | git fast-import 2> output &&
+	git fast-import 2>output <input &&
 	test_must_be_empty output
 '
 

t/t9350-fast-export.sh

@@ -600,9 +600,10 @@ test_expect_success 'directory becomes symlink'        '
 
 test_expect_success 'fast-export quotes pathnames' '
 	git init crazy-paths &&
+	test_config -C crazy-paths core.protectNTFS false &&
 	(cd crazy-paths &&
 	 blob=$(echo foo | git hash-object -w --stdin) &&
-	 git update-index --add \
+	 git -c core.protectNTFS=false update-index --add \
 		--cacheinfo 100644 $blob "$(printf "path with\\nnewline")" \
 		--cacheinfo 100644 $blob "path with \"quote\"" \
 		--cacheinfo 100644 $blob "path with \\backslash" \

transport-helper.c

@@ -435,6 +435,7 @@ static int get_importer(struct transport *transport, struct child_process *fasti
 	child_process_init(fastimport);
 	fastimport->in = xdup(helper->out);
 	argv_array_push(&fastimport->args, "fast-import");
+	argv_array_push(&fastimport->args, "--allow-unsafe-features");
 	argv_array_push(&fastimport->args, debug ? "--stats" : "--quiet");
 
 	if (data->bidi_import) {

tree-walk.c

@@ -43,6 +43,12 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l
 		strbuf_addstr(err, _("empty filename in tree entry"));
 		return -1;
 	}
+#ifdef GIT_WINDOWS_NATIVE
+	if (protect_ntfs && strchr(path, '\\')) {
+		strbuf_addf(err, _("filename in tree entry contains backslash: '%s'"), path);
+		return -1;
+	}
+#endif
 	len = strlen(path) + 1;
 
 	/* Initialize the descriptor entry */

unpack-trees.c

@@ -2060,7 +2060,8 @@ static int merged_entry(const struct cache_entry *ce,
 		invalidate_ce_path(old, o);
 	}
 
-	do_add_entry(o, merge, update, CE_STAGEMASK);
+	if (do_add_entry(o, merge, update, CE_STAGEMASK) < 0)
+		return -1;
 	return 1;
 }