authorgravatar for squeek502@hotmail.comRyan Liptak <squeek502@hotmail.com> 2024-04-15 18:25:56-07:00
committergravatar for squeek502@hotmail.comRyan Liptak <squeek502@hotmail.com> 2024-04-23 03:21:51-07:00
log422464d54082b6a1fd5e40e4f9c1ec2a96c3f77e
tree9751a332699894e6dd94b502a917b5362399eb9b
parent84f4c5d9ccbebb6675c7366c4e1fdb661003356e

std.process.Child: Mitigate arbitrary command execution vulnerability on Windows (BatBadBut)

> Note: This first part is mostly a rephrasing of https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/ > See that article for more details On Windows, it is possible to execute `.bat`/`.cmd` scripts via CreateProcessW. When this happens, `CreateProcessW` will (under-the-hood) spawn a `cmd.exe` process with the path to the script and the args like so: cmd.exe /c script.bat arg1 arg2 This is a problem because: - `cmd.exe` has its own, separate, parsing/escaping rules for arguments - Environment variables in arguments will be expanded before the `cmd.exe` parsing rules are applied Together, this means that (1) maliciously constructed arguments can lead to arbitrary command execution via the APIs in `std.process.Child` and (2) escaping according to the rules of `cmd.exe` is not enough on its own. A basic example argv field that reproduces the vulnerability (this will erroneously spawn `calc.exe`): .argv = &.{ "test.bat", "\"&calc.exe" }, And one that takes advantage of environment variable expansion to still spawn calc.exe even if the args are properly escaped for `cmd.exe`: .argv = &.{ "test.bat", "%CMDCMDLINE:~-1%&calc.exe" }, (note: if these spawned e.g. `test.exe` instead of `test.bat`, they wouldn't be vulnerable; it's only `.bat`/`.cmd` scripts that are vulnerable since they go through `cmd.exe`) Zig allows passing `.bat`/`.cmd` scripts as `argv[0]` via `std.process.Child`, so the Zig API is affected by this vulnerability. Note also that Zig will search `PATH` for `.bat`/`.cmd` scripts, so spawning something like `foo` may end up executing `foo.bat` somewhere in the PATH (the PATH searching of Zig matches the behavior of cmd.exe). > Side note to keep in mind: On Windows, the extension is significant in terms of how Windows will try to execute the command. If the extension is not `.bat`/`.cmd`, we know that it will not attempt to be executed as a `.bat`/`.cmd` script (and vice versa). This means that we can just look at the extension to know if we are trying to execute a `.bat`/`.cmd` script. --- This general class of problem has been documented before in 2011 here: https://learn.microsoft.com/en-us/archive/blogs/twistylittlepassagesallalike/everyone-quotes-command-line-arguments-the-wrong-way and the course of action it suggests for escaping when executing .bat/.cmd files is: - Escape first using the non-cmd.exe rules - Then escape all cmd.exe 'metacharacters' (`(`, `)`, `%`, `!`, `^`, `"`, `<`, `>`, `&`, and `|`) with `^` However, escaping with ^ on its own is insufficient because it does not stop cmd.exe from expanding environment variables. For example: ``` args.bat %PATH% ``` escaped with ^ (and wrapped in quotes that are also escaped), it *will* stop cmd.exe from expanding `%PATH%`: ``` > args.bat ^"^%PATH^%^" "%PATH%" ``` but it will still try to expand `%PATH^%`: ``` set PATH^^=123 > args.bat ^"^%PATH^%^" "123" ``` The goal is to stop *all* environment variable expansion, so this won't work. Another problem with the ^ approach is that it does not seem to allow all possible command lines to round trip through cmd.exe (as far as I can tell at least). One known example: ``` args.bat ^"\^"key^=value\^"^" ``` where args.bat is: ``` @echo %1 %2 %3 %4 %5 %6 %7 %8 %9 ``` will print ``` "\"key value\"" ``` (it will turn the `=` into a space for an unknown reason; other minor variations do roundtrip, e.g. `\^"key^=value\^"`, `^"key^=value^"`, so it's unclear what's going on) It may actually be possible to escape with ^ such that every possible command line round trips correctly, but it's probably not worth the effort to figure it out, since the suggested mitigation for BatBadBut has better roundtripping and leads to less garbled command lines overall. --- Ultimately, the mitigation used here is the same as the one suggested in: https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/ The mitigation steps are reproduced here, noted with one deviation that Zig makes (following Rust's lead): 1. Replace percent sign (%) with %%cd:~,%. 2. Replace the backslash (\) in front of the double quote (") with two backslashes (\\). 3. Replace the double quote (") with two double quotes (""). 4. ~~Remove newline characters (\n).~~ - Instead, `\n`, `\r`, and NUL are disallowed and will trigger `error.InvalidBatchScriptArg` if they are found in `argv`. These three characters do not roundtrip through a `.bat` file and therefore are of dubious/no use. It's unclear to me if `\n` in particular is relevant to the BatBadBut vulnerability (I wasn't able to find a reproduction with \n and the post doesn't mention anything about it except in the suggested mitigation steps); it just seems to act as a 'end of arguments' marker and therefore anything after the `\n` is lost (and same with NUL). `\r` seems to be stripped from the command line arguments when passed through a `.bat`/`.cmd`, so that is also disallowed to ensure that `argv` can always fully roundtrip through `.bat`/`.cmd`. 5. Enclose the argument with double quotes ("). The escaped command line is then run as something like: cmd.exe /d /e:ON /v:OFF /c "foo.bat arg1 arg2" Note: Previously, we would pass `foo.bat arg1 arg2` as the command line and the path to `foo.bat` as the app name and let CreateProcessW handle the `cmd.exe` spawning for us, but because we need to pass `/e:ON` and `/v:OFF` to cmd.exe to ensure the mitigation is effective, that is no longer tenable. Instead, we now get the full path to `cmd.exe` and use that as the app name when executing `.bat`/`.cmd` files. --- A standalone test has also been added that tests two things: 1. Known reproductions of the vulnerability are tested to ensure that they do not reproduce the vulnerability 2. Randomly generated command line arguments roundtrip when passed to a `.bat` file and then are passed from the `.bat` file to a `.exe`. This fuzz test is as thorough as possible--it tests that things like arbitrary Unicode codepoints and unpaired surrogates roundtrip successfully. Note: In order for the `CreateProcessW` -> `.bat` -> `.exe` roundtripping to succeed, the .exe must split the arguments using the post-2008 C runtime argv splitting implementation, see https://github.com/ziglang/zig/pull/19655 for details on when that change was made in Zig.

7 files changed, 645 insertions(+), 12 deletions(-)

lib/std/child_process.zig+276-12
...@@ -136,6 +136,14 @@ pub const ChildProcess = struct {...@@ -136,6 +136,14 @@ pub const ChildProcess = struct {
136136
137 /// Windows-only. `cwd` was provided, but the path did not exist when spawning the child process.137 /// Windows-only. `cwd` was provided, but the path did not exist when spawning the child process.
138 CurrentWorkingDirectoryUnlinked,138 CurrentWorkingDirectoryUnlinked,
139
140 /// Windows-only. NUL (U+0000), LF (U+000A), CR (U+000D) are not allowed
141 /// within arguments when executing a `.bat`/`.cmd` script.
142 /// - NUL/LF signifiies end of arguments, so anything afterwards
143 /// would be lost after execution.
144 /// - CR is stripped by `cmd.exe`, so any CR codepoints
145 /// would be lost after execution.
146 InvalidBatchScriptArg,
139 } ||147 } ||
140 posix.ExecveError ||148 posix.ExecveError ||
141 posix.SetIdError ||149 posix.SetIdError ||
...@@ -814,17 +822,20 @@ pub const ChildProcess = struct {...@@ -814,17 +822,20 @@ pub const ChildProcess = struct {
814 const app_name_w = try unicode.wtf8ToWtf16LeAllocZ(self.allocator, app_basename_wtf8);822 const app_name_w = try unicode.wtf8ToWtf16LeAllocZ(self.allocator, app_basename_wtf8);
815 defer self.allocator.free(app_name_w);823 defer self.allocator.free(app_name_w);
816824
817 const cmd_line_w = argvToCommandLineWindows(self.allocator, self.argv) catch |err| switch (err) {
818 // argv[0] contains unsupported characters that will never resolve to a valid exe.
819 error.InvalidArg0 => return error.FileNotFound,
820 else => |e| return e,
821 };
822 defer self.allocator.free(cmd_line_w);
823
824 run: {825 run: {
825 const PATH: [:0]const u16 = std.process.getenvW(unicode.utf8ToUtf16LeStringLiteral("PATH")) orelse &[_:0]u16{};826 const PATH: [:0]const u16 = std.process.getenvW(unicode.utf8ToUtf16LeStringLiteral("PATH")) orelse &[_:0]u16{};
826 const PATHEXT: [:0]const u16 = std.process.getenvW(unicode.utf8ToUtf16LeStringLiteral("PATHEXT")) orelse &[_:0]u16{};827 const PATHEXT: [:0]const u16 = std.process.getenvW(unicode.utf8ToUtf16LeStringLiteral("PATHEXT")) orelse &[_:0]u16{};
827828
829 // In case the command ends up being a .bat/.cmd script, we need to escape things using the cmd.exe rules
830 // and invoke cmd.exe ourselves in order to mitigate arbitrary command execution from maliciously
831 // constructed arguments.
832 //
833 // We'll need to wait until we're actually trying to run the command to know for sure
834 // if the resolved command has the `.bat` or `.cmd` extension, so we defer actually
835 // serializing the command line until we determine how it should be serialized.
836 var cmd_line_cache = WindowsCommandLineCache.init(self.allocator, self.argv);
837 defer cmd_line_cache.deinit();
838
828 var app_buf = std.ArrayListUnmanaged(u16){};839 var app_buf = std.ArrayListUnmanaged(u16){};
829 defer app_buf.deinit(self.allocator);840 defer app_buf.deinit(self.allocator);
830841
...@@ -846,8 +857,10 @@ pub const ChildProcess = struct {...@@ -846,8 +857,10 @@ pub const ChildProcess = struct {
846 dir_buf.shrinkRetainingCapacity(normalized_len);857 dir_buf.shrinkRetainingCapacity(normalized_len);
847 }858 }
848859
849 windowsCreateProcessPathExt(self.allocator, &dir_buf, &app_buf, PATHEXT, cmd_line_w.ptr, envp_ptr, cwd_w_ptr, &siStartInfo, &piProcInfo) catch |no_path_err| {860 windowsCreateProcessPathExt(self.allocator, &dir_buf, &app_buf, PATHEXT, &cmd_line_cache, envp_ptr, cwd_w_ptr, &siStartInfo, &piProcInfo) catch |no_path_err| {
850 const original_err = switch (no_path_err) {861 const original_err = switch (no_path_err) {
862 // argv[0] contains unsupported characters that will never resolve to a valid exe.
863 error.InvalidArg0 => return error.FileNotFound,
851 error.FileNotFound, error.InvalidExe, error.AccessDenied => |e| e,864 error.FileNotFound, error.InvalidExe, error.AccessDenied => |e| e,
852 error.UnrecoverableInvalidExe => return error.InvalidExe,865 error.UnrecoverableInvalidExe => return error.InvalidExe,
853 else => |e| return e,866 else => |e| return e,
...@@ -872,9 +885,11 @@ pub const ChildProcess = struct {...@@ -872,9 +885,11 @@ pub const ChildProcess = struct {
872 const normalized_len = windows.normalizePath(u16, dir_buf.items) catch continue;885 const normalized_len = windows.normalizePath(u16, dir_buf.items) catch continue;
873 dir_buf.shrinkRetainingCapacity(normalized_len);886 dir_buf.shrinkRetainingCapacity(normalized_len);
874887
875 if (windowsCreateProcessPathExt(self.allocator, &dir_buf, &app_buf, PATHEXT, cmd_line_w.ptr, envp_ptr, cwd_w_ptr, &siStartInfo, &piProcInfo)) {888 if (windowsCreateProcessPathExt(self.allocator, &dir_buf, &app_buf, PATHEXT, &cmd_line_cache, envp_ptr, cwd_w_ptr, &siStartInfo, &piProcInfo)) {
876 break :run;889 break :run;
877 } else |err| switch (err) {890 } else |err| switch (err) {
891 // argv[0] contains unsupported characters that will never resolve to a valid exe.
892 error.InvalidArg0 => return error.FileNotFound,
878 error.FileNotFound, error.AccessDenied, error.InvalidExe => continue,893 error.FileNotFound, error.AccessDenied, error.InvalidExe => continue,
879 error.UnrecoverableInvalidExe => return error.InvalidExe,894 error.UnrecoverableInvalidExe => return error.InvalidExe,
880 else => |e| return e,895 else => |e| return e,
...@@ -935,7 +950,7 @@ fn windowsCreateProcessPathExt(...@@ -935,7 +950,7 @@ fn windowsCreateProcessPathExt(
935 dir_buf: *std.ArrayListUnmanaged(u16),950 dir_buf: *std.ArrayListUnmanaged(u16),
936 app_buf: *std.ArrayListUnmanaged(u16),951 app_buf: *std.ArrayListUnmanaged(u16),
937 pathext: [:0]const u16,952 pathext: [:0]const u16,
938 cmd_line: [*:0]u16,953 cmd_line_cache: *WindowsCommandLineCache,
939 envp_ptr: ?[*]u16,954 envp_ptr: ?[*]u16,
940 cwd_ptr: ?[*:0]u16,955 cwd_ptr: ?[*:0]u16,
941 lpStartupInfo: *windows.STARTUPINFOW,956 lpStartupInfo: *windows.STARTUPINFOW,
...@@ -1069,7 +1084,26 @@ fn windowsCreateProcessPathExt(...@@ -1069,7 +1084,26 @@ fn windowsCreateProcessPathExt(
1069 try dir_buf.append(allocator, 0);1084 try dir_buf.append(allocator, 0);
1070 const full_app_name = dir_buf.items[0 .. dir_buf.items.len - 1 :0];1085 const full_app_name = dir_buf.items[0 .. dir_buf.items.len - 1 :0];
10711086
1072 if (windowsCreateProcess(full_app_name.ptr, cmd_line, envp_ptr, cwd_ptr, lpStartupInfo, lpProcessInformation)) |_| {1087 const is_bat_or_cmd = bat_or_cmd: {
1088 const app_name = app_buf.items[0..app_name_len];
1089 const ext_start = std.mem.lastIndexOfScalar(u16, app_name, '.') orelse break :bat_or_cmd false;
1090 const ext = app_name[ext_start..];
1091 const ext_enum = windowsCreateProcessSupportsExtension(ext) orelse break :bat_or_cmd false;
1092 switch (ext_enum) {
1093 .cmd, .bat => break :bat_or_cmd true,
1094 else => break :bat_or_cmd false,
1095 }
1096 };
1097 const cmd_line_w = if (is_bat_or_cmd)
1098 try cmd_line_cache.scriptCommandLine(full_app_name)
1099 else
1100 try cmd_line_cache.commandLine();
1101 const app_name_w = if (is_bat_or_cmd)
1102 try cmd_line_cache.cmdExePath()
1103 else
1104 full_app_name;
1105
1106 if (windowsCreateProcess(app_name_w.ptr, cmd_line_w.ptr, envp_ptr, cwd_ptr, lpStartupInfo, lpProcessInformation)) |_| {
1073 return;1107 return;
1074 } else |err| switch (err) {1108 } else |err| switch (err) {
1075 error.FileNotFound,1109 error.FileNotFound,
...@@ -1111,7 +1145,20 @@ fn windowsCreateProcessPathExt(...@@ -1111,7 +1145,20 @@ fn windowsCreateProcessPathExt(
1111 try dir_buf.append(allocator, 0);1145 try dir_buf.append(allocator, 0);
1112 const full_app_name = dir_buf.items[0 .. dir_buf.items.len - 1 :0];1146 const full_app_name = dir_buf.items[0 .. dir_buf.items.len - 1 :0];
11131147
1114 if (windowsCreateProcess(full_app_name.ptr, cmd_line, envp_ptr, cwd_ptr, lpStartupInfo, lpProcessInformation)) |_| {1148 const is_bat_or_cmd = switch (ext_enum) {
1149 .cmd, .bat => true,
1150 else => false,
1151 };
1152 const cmd_line_w = if (is_bat_or_cmd)
1153 try cmd_line_cache.scriptCommandLine(full_app_name)
1154 else
1155 try cmd_line_cache.commandLine();
1156 const app_name_w = if (is_bat_or_cmd)
1157 try cmd_line_cache.cmdExePath()
1158 else
1159 full_app_name;
1160
1161 if (windowsCreateProcess(app_name_w.ptr, cmd_line_w.ptr, envp_ptr, cwd_ptr, lpStartupInfo, lpProcessInformation)) |_| {
1115 return;1162 return;
1116 } else |err| switch (err) {1163 } else |err| switch (err) {
1117 error.FileNotFound => continue,1164 error.FileNotFound => continue,
...@@ -1236,6 +1283,223 @@ test windowsCreateProcessSupportsExtension {...@@ -1236,6 +1283,223 @@ test windowsCreateProcessSupportsExtension {
1236 try std.testing.expect(windowsCreateProcessSupportsExtension(&[_]u16{ '.', 'e', 'X', 'e', 'c' }) == null);1283 try std.testing.expect(windowsCreateProcessSupportsExtension(&[_]u16{ '.', 'e', 'X', 'e', 'c' }) == null);
1237}1284}
12381285
1286/// Serializes argv into a WTF-16 encoded command-line string for use with CreateProcessW.
1287///
1288/// Serialization is done on-demand and the result is cached in order to allow for:
1289/// - Only serializing the particular type of command line needed (`.bat`/`.cmd`
1290/// command line serialization is different from `.exe`/etc)
1291/// - Reusing the serialized command lines if necessary (i.e. if the execution
1292/// of a command fails and the PATH is going to be continued to be searched
1293/// for more candidates)
1294pub const WindowsCommandLineCache = struct {
1295 cmd_line: ?[:0]u16 = null,
1296 script_cmd_line: ?[:0]u16 = null,
1297 cmd_exe_path: ?[:0]u16 = null,
1298 argv: []const []const u8,
1299 allocator: mem.Allocator,
1300
1301 pub fn init(allocator: mem.Allocator, argv: []const []const u8) WindowsCommandLineCache {
1302 return .{
1303 .allocator = allocator,
1304 .argv = argv,
1305 };
1306 }
1307
1308 pub fn deinit(self: *WindowsCommandLineCache) void {
1309 if (self.cmd_line) |cmd_line| self.allocator.free(cmd_line);
1310 if (self.script_cmd_line) |script_cmd_line| self.allocator.free(script_cmd_line);
1311 if (self.cmd_exe_path) |cmd_exe_path| self.allocator.free(cmd_exe_path);
1312 }
1313
1314 pub fn commandLine(self: *WindowsCommandLineCache) ![:0]u16 {
1315 if (self.cmd_line == null) {
1316 self.cmd_line = try argvToCommandLineWindows(self.allocator, self.argv);
1317 }
1318 return self.cmd_line.?;
1319 }
1320
1321 /// Not cached, since the path to the batch script will change during PATH searching.
1322 /// `script_path` should be as qualified as possible, e.g. if the PATH is being searched,
1323 /// then script_path should include both the search path and the script filename
1324 /// (this allows avoiding cmd.exe having to search the PATH again).
1325 pub fn scriptCommandLine(self: *WindowsCommandLineCache, script_path: []const u16) ![:0]u16 {
1326 if (self.script_cmd_line) |v| self.allocator.free(v);
1327 self.script_cmd_line = try argvToScriptCommandLineWindows(
1328 self.allocator,
1329 script_path,
1330 self.argv[1..],
1331 );
1332 return self.script_cmd_line.?;
1333 }
1334
1335 pub fn cmdExePath(self: *WindowsCommandLineCache) ![:0]u16 {
1336 if (self.cmd_exe_path == null) {
1337 self.cmd_exe_path = try windowsCmdExePath(self.allocator);
1338 }
1339 return self.cmd_exe_path.?;
1340 }
1341};
1342
1343pub fn windowsCmdExePath(allocator: mem.Allocator) error{ OutOfMemory, Unexpected }![:0]u16 {
1344 var buf = try std.ArrayListUnmanaged(u16).initCapacity(allocator, 128);
1345 errdefer buf.deinit(allocator);
1346 while (true) {
1347 const unused_slice = buf.unusedCapacitySlice();
1348 // TODO: Get the system directory from PEB.ReadOnlyStaticServerData
1349 const len = windows.kernel32.GetSystemDirectoryW(@ptrCast(unused_slice), @intCast(unused_slice.len));
1350 if (len == 0) {
1351 switch (windows.kernel32.GetLastError()) {
1352 else => |err| return windows.unexpectedError(err),
1353 }
1354 }
1355 if (len > unused_slice.len) {
1356 try buf.ensureUnusedCapacity(allocator, len);
1357 } else {
1358 buf.items.len = len;
1359 break;
1360 }
1361 }
1362 switch (buf.items[buf.items.len - 1]) {
1363 '/', '\\' => {},
1364 else => try buf.append(allocator, fs.path.sep),
1365 }
1366 try buf.appendSlice(allocator, std.unicode.utf8ToUtf16LeStringLiteral("cmd.exe"));
1367 return try buf.toOwnedSliceSentinel(allocator, 0);
1368}
1369
1370pub const ArgvToScriptCommandLineError = error{
1371 OutOfMemory,
1372 InvalidWtf8,
1373 /// NUL (U+0000), LF (U+000A), CR (U+000D) are not allowed
1374 /// within arguments when executing a `.bat`/`.cmd` script.
1375 /// - NUL/LF signifiies end of arguments, so anything afterwards
1376 /// would be lost after execution.
1377 /// - CR is stripped by `cmd.exe`, so any CR codepoints
1378 /// would be lost after execution.
1379 InvalidBatchScriptArg,
1380};
1381
1382/// Serializes `argv` to a Windows command-line string that uses `cmd.exe /c` and `cmd.exe`-specific
1383/// escaping rules. The caller owns the returned slice.
1384///
1385/// Escapes `argv` using the suggested mitigation against arbitrary command execution from:
1386/// https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/
1387pub fn argvToScriptCommandLineWindows(
1388 allocator: mem.Allocator,
1389 /// Path to the `.bat`/`.cmd` script. If this path is relative, it is assumed to be relative to the CWD.
1390 /// The script must have been verified to exist at this path before calling this function.
1391 script_path: []const u16,
1392 /// Arguments, not including the script name itself. Expected to be encoded as WTF-8.
1393 script_args: []const []const u8,
1394) ArgvToScriptCommandLineError![:0]u16 {
1395 var buf = try std.ArrayList(u8).initCapacity(allocator, 64);
1396 defer buf.deinit();
1397
1398 // `/d` disables execution of AutoRun commands.
1399 // `/e:ON` and `/v:OFF` are needed for BatBadBut mitigation:
1400 // > If delayed expansion is enabled via the registry value DelayedExpansion,
1401 // > it must be disabled by explicitly calling cmd.exe with the /V:OFF option.
1402 // > Escaping for % requires the command extension to be enabled.
1403 // > If it’s disabled via the registry value EnableExtensions, it must be enabled with the /E:ON option.
1404 // https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/
1405 buf.appendSliceAssumeCapacity("cmd.exe /d /e:ON /v:OFF /c \"");
1406
1407 // Always quote the path to the script arg
1408 buf.appendAssumeCapacity('"');
1409 // We always want the path to the batch script to include a path separator in order to
1410 // avoid cmd.exe searching the PATH for the script. This is not part of the arbitrary
1411 // command execution mitigation, we just know exactly what script we want to execute
1412 // at this point, and potentially making cmd.exe re-find it is unnecessary.
1413 //
1414 // If the script path does not have a path separator, then we know its relative to CWD and
1415 // we can just put `.\` in the front.
1416 if (mem.indexOfAny(u16, script_path, &[_]u16{ mem.nativeToLittle(u16, '\\'), mem.nativeToLittle(u16, '/') }) == null) {
1417 try buf.appendSlice(".\\");
1418 }
1419 // Note that we don't do any escaping/mitigations for this argument, since the relevant
1420 // characters (", %, etc) are illegal in file paths and this function should only be called
1421 // with script paths that have been verified to exist.
1422 try std.unicode.wtf16LeToWtf8ArrayList(&buf, script_path);
1423 buf.appendAssumeCapacity('"');
1424
1425 for (script_args) |arg| {
1426 // Literal carriage returns get stripped when run through cmd.exe
1427 // and NUL/newlines act as 'end of command.' Because of this, it's basically
1428 // always a mistake to include these characters in argv, so it's
1429 // an error condition in order to ensure that the return of this
1430 // function can always roundtrip through cmd.exe.
1431 if (std.mem.indexOfAny(u8, arg, "\x00\r\n") != null) {
1432 return error.InvalidBatchScriptArg;
1433 }
1434
1435 // Separate args with a space.
1436 try buf.append(' ');
1437
1438 // Need to quote if the argument is empty (otherwise the arg would just be lost)
1439 // or if the last character is a `\`, since then something like "%~2" in a .bat
1440 // script would cause the closing " to be escaped which we don't want.
1441 var needs_quotes = arg.len == 0 or arg[arg.len - 1] == '\\';
1442 if (!needs_quotes) {
1443 for (arg) |c| {
1444 switch (c) {
1445 // Known good characters that don't need to be quoted
1446 'A'...'Z', 'a'...'z', '0'...'9', '#', '$', '*', '+', '-', '.', '/', ':', '?', '@', '\\', '_' => {},
1447 // When in doubt, quote
1448 else => {
1449 needs_quotes = true;
1450 break;
1451 },
1452 }
1453 }
1454 }
1455 if (needs_quotes) {
1456 try buf.append('"');
1457 }
1458 var backslashes: usize = 0;
1459 for (arg) |c| {
1460 switch (c) {
1461 '\\' => {
1462 backslashes += 1;
1463 },
1464 '"' => {
1465 try buf.appendNTimes('\\', backslashes);
1466 try buf.append('"');
1467 backslashes = 0;
1468 },
1469 // Replace `%` with `%%cd:~,%`.
1470 //
1471 // cmd.exe allows extracting a substring from an environment
1472 // variable with the syntax: `%foo:~<start_index>,<end_index>%`.
1473 // Therefore, `%cd:~,%` will always expand to an empty string
1474 // since both the start and end index are blank, and it is assumed
1475 // that `%cd%` is always available since it is a built-in variable
1476 // that corresponds to the current directory.
1477 //
1478 // This means that replacing `%foo%` with `%%cd:~,%foo%%cd:~,%`
1479 // will stop `%foo%` from being expanded and *after* expansion
1480 // we'll still be left with `%foo%` (the literal string).
1481 '%' => {
1482 // the trailing `%` is appended outside the switch
1483 try buf.appendSlice("%%cd:~,");
1484 backslashes = 0;
1485 },
1486 else => {
1487 backslashes = 0;
1488 },
1489 }
1490 try buf.append(c);
1491 }
1492 if (needs_quotes) {
1493 try buf.appendNTimes('\\', backslashes);
1494 try buf.append('"');
1495 }
1496 }
1497
1498 try buf.append('"');
1499
1500 return try unicode.wtf8ToWtf16LeAllocZ(allocator, buf.items);
1501}
1502
1239pub const ArgvToCommandLineError = error{ OutOfMemory, InvalidWtf8, InvalidArg0 };1503pub const ArgvToCommandLineError = error{ OutOfMemory, InvalidWtf8, InvalidArg0 };
12401504
1241/// Serializes `argv` to a Windows command-line string suitable for passing to a child process and1505/// Serializes `argv` to a Windows command-line string suitable for passing to a child process and
lib/std/os/windows/kernel32.zig+2
...@@ -243,6 +243,8 @@ pub extern "kernel32" fn GetSystemInfo(lpSystemInfo: *SYSTEM_INFO) callconv(WINA...@@ -243,6 +243,8 @@ pub extern "kernel32" fn GetSystemInfo(lpSystemInfo: *SYSTEM_INFO) callconv(WINA
243pub extern "kernel32" fn GetSystemTimeAsFileTime(*FILETIME) callconv(WINAPI) void;243pub extern "kernel32" fn GetSystemTimeAsFileTime(*FILETIME) callconv(WINAPI) void;
244pub extern "kernel32" fn IsProcessorFeaturePresent(ProcessorFeature: DWORD) BOOL;244pub extern "kernel32" fn IsProcessorFeaturePresent(ProcessorFeature: DWORD) BOOL;
245245
246pub extern "kernel32" fn GetSystemDirectoryW(lpBuffer: LPWSTR, uSize: UINT) callconv(WINAPI) UINT;
247
246pub extern "kernel32" fn HeapCreate(flOptions: DWORD, dwInitialSize: SIZE_T, dwMaximumSize: SIZE_T) callconv(WINAPI) ?HANDLE;248pub extern "kernel32" fn HeapCreate(flOptions: DWORD, dwInitialSize: SIZE_T, dwMaximumSize: SIZE_T) callconv(WINAPI) ?HANDLE;
247pub extern "kernel32" fn HeapDestroy(hHeap: HANDLE) callconv(WINAPI) BOOL;249pub extern "kernel32" fn HeapDestroy(hHeap: HANDLE) callconv(WINAPI) BOOL;
248pub extern "kernel32" fn HeapReAlloc(hHeap: HANDLE, dwFlags: DWORD, lpMem: *anyopaque, dwBytes: SIZE_T) callconv(WINAPI) ?*anyopaque;250pub extern "kernel32" fn HeapReAlloc(hHeap: HANDLE, dwFlags: DWORD, lpMem: *anyopaque, dwBytes: SIZE_T) callconv(WINAPI) ?*anyopaque;
test/standalone/build.zig.zon+3
...@@ -107,6 +107,9 @@...@@ -107,6 +107,9 @@
107 .windows_argv = .{107 .windows_argv = .{
108 .path = "windows_argv",108 .path = "windows_argv",
109 },109 },
110 .windows_bat_args = .{
111 .path = "windows_bat_args",
112 },
110 .self_exe_symlink = .{113 .self_exe_symlink = .{
111 .path = "self_exe_symlink",114 .path = "self_exe_symlink",
112 },115 },
test/standalone/windows_bat_args/build.zig created+58
...@@ -0,0 +1,58 @@
1const std = @import("std");
2const builtin = @import("builtin");
3
4pub fn build(b: *std.Build) !void {
5 const test_step = b.step("test", "Test it");
6 b.default_step = test_step;
7
8 const optimize: std.builtin.OptimizeMode = .Debug;
9 const target = b.host;
10
11 if (builtin.os.tag != .windows) return;
12
13 const echo_args = b.addExecutable(.{
14 .name = "echo-args",
15 .root_source_file = b.path("echo-args.zig"),
16 .optimize = optimize,
17 .target = target,
18 });
19
20 const test_exe = b.addExecutable(.{
21 .name = "test",
22 .root_source_file = b.path("test.zig"),
23 .optimize = optimize,
24 .target = target,
25 });
26
27 const run = b.addRunArtifact(test_exe);
28 run.addArtifactArg(echo_args);
29 run.expectExitCode(0);
30 run.skip_foreign_checks = true;
31
32 test_step.dependOn(&run.step);
33
34 const fuzz = b.addExecutable(.{
35 .name = "fuzz",
36 .root_source_file = b.path("fuzz.zig"),
37 .optimize = optimize,
38 .target = target,
39 });
40
41 const fuzz_max_iterations = b.option(u64, "iterations", "The max fuzz iterations (default: 100)") orelse 100;
42 const fuzz_iterations_arg = std.fmt.allocPrint(b.allocator, "{}", .{fuzz_max_iterations}) catch @panic("oom");
43
44 const fuzz_seed = b.option(u64, "seed", "Seed to use for the PRNG (default: random)") orelse seed: {
45 var buf: [8]u8 = undefined;
46 try std.posix.getrandom(&buf);
47 break :seed std.mem.readInt(u64, &buf, builtin.cpu.arch.endian());
48 };
49 const fuzz_seed_arg = std.fmt.allocPrint(b.allocator, "{}", .{fuzz_seed}) catch @panic("oom");
50
51 const fuzz_run = b.addRunArtifact(fuzz);
52 fuzz_run.addArtifactArg(echo_args);
53 fuzz_run.addArgs(&.{ fuzz_iterations_arg, fuzz_seed_arg });
54 fuzz_run.expectExitCode(0);
55 fuzz_run.skip_foreign_checks = true;
56
57 test_step.dependOn(&fuzz_run.step);
58}
test/standalone/windows_bat_args/echo-args.zig created+14
...@@ -0,0 +1,14 @@
1const std = @import("std");
2
3pub fn main() !void {
4 var arena_state = std.heap.ArenaAllocator.init(std.heap.page_allocator);
5 defer arena_state.deinit();
6 const arena = arena_state.allocator();
7
8 const stdout = std.io.getStdOut().writer();
9 var args = try std.process.argsAlloc(arena);
10 for (args[1..], 1..) |arg, i| {
11 try stdout.writeAll(arg);
12 if (i != args.len - 1) try stdout.writeByte('\x00');
13 }
14}
test/standalone/windows_bat_args/fuzz.zig created+160
...@@ -0,0 +1,160 @@
1const std = @import("std");
2const builtin = @import("builtin");
3const Allocator = std.mem.Allocator;
4
5pub fn main() anyerror!void {
6 var gpa = std.heap.GeneralPurposeAllocator(.{}){};
7 defer if (gpa.deinit() == .leak) @panic("found memory leaks");
8 const allocator = gpa.allocator();
9
10 var it = try std.process.argsWithAllocator(allocator);
11 defer it.deinit();
12 _ = it.next() orelse unreachable; // skip binary name
13 const child_exe_path = it.next() orelse unreachable;
14
15 const iterations: u64 = iterations: {
16 const arg = it.next() orelse "0";
17 break :iterations try std.fmt.parseUnsigned(u64, arg, 10);
18 };
19
20 var rand_seed = false;
21 const seed: u64 = seed: {
22 const seed_arg = it.next() orelse {
23 rand_seed = true;
24 var buf: [8]u8 = undefined;
25 try std.posix.getrandom(&buf);
26 break :seed std.mem.readInt(u64, &buf, builtin.cpu.arch.endian());
27 };
28 break :seed try std.fmt.parseUnsigned(u64, seed_arg, 10);
29 };
30 var random = std.rand.DefaultPrng.init(seed);
31 const rand = random.random();
32
33 // If the seed was not given via the CLI, then output the
34 // randomly chosen seed so that this run can be reproduced
35 if (rand_seed) {
36 std.debug.print("rand seed: {}\n", .{seed});
37 }
38
39 var tmp = std.testing.tmpDir(.{});
40 defer tmp.cleanup();
41
42 try tmp.dir.setAsCwd();
43 defer tmp.parent_dir.setAsCwd() catch {};
44
45 var buf = try std.ArrayList(u8).initCapacity(allocator, 128);
46 defer buf.deinit();
47 try buf.appendSlice("@echo off\n");
48 try buf.append('"');
49 try buf.appendSlice(child_exe_path);
50 try buf.append('"');
51 const preamble_len = buf.items.len;
52
53 try buf.appendSlice(" %*");
54 try tmp.dir.writeFile("args1.bat", buf.items);
55 buf.shrinkRetainingCapacity(preamble_len);
56
57 try buf.appendSlice(" %1 %2 %3 %4 %5 %6 %7 %8 %9");
58 try tmp.dir.writeFile("args2.bat", buf.items);
59 buf.shrinkRetainingCapacity(preamble_len);
60
61 try buf.appendSlice(" \"%~1\" \"%~2\" \"%~3\" \"%~4\" \"%~5\" \"%~6\" \"%~7\" \"%~8\" \"%~9\"");
62 try tmp.dir.writeFile("args3.bat", buf.items);
63 buf.shrinkRetainingCapacity(preamble_len);
64
65 var i: u64 = 0;
66 while (iterations == 0 or i < iterations) {
67 const rand_arg = try randomArg(allocator, rand);
68 defer allocator.free(rand_arg);
69
70 try testExec(allocator, &.{rand_arg}, null);
71
72 i += 1;
73 }
74}
75
76fn testExec(allocator: std.mem.Allocator, args: []const []const u8, env: ?*std.process.EnvMap) !void {
77 try testExecBat(allocator, "args1.bat", args, env);
78 try testExecBat(allocator, "args2.bat", args, env);
79 try testExecBat(allocator, "args3.bat", args, env);
80}
81
82fn testExecBat(allocator: std.mem.Allocator, bat: []const u8, args: []const []const u8, env: ?*std.process.EnvMap) !void {
83 var argv = try std.ArrayList([]const u8).initCapacity(allocator, 1 + args.len);
84 defer argv.deinit();
85 argv.appendAssumeCapacity(bat);
86 argv.appendSliceAssumeCapacity(args);
87
88 const can_have_trailing_empty_args = std.mem.eql(u8, bat, "args3.bat");
89
90 const result = try std.ChildProcess.run(.{
91 .allocator = allocator,
92 .env_map = env,
93 .argv = argv.items,
94 });
95 defer allocator.free(result.stdout);
96 defer allocator.free(result.stderr);
97
98 try std.testing.expectEqualStrings("", result.stderr);
99 var it = std.mem.splitScalar(u8, result.stdout, '\x00');
100 var i: usize = 0;
101 while (it.next()) |actual_arg| {
102 if (i >= args.len and can_have_trailing_empty_args) {
103 try std.testing.expectEqualStrings("", actual_arg);
104 continue;
105 }
106 const expected_arg = args[i];
107 try std.testing.expectEqualSlices(u8, expected_arg, actual_arg);
108 i += 1;
109 }
110}
111
112fn randomArg(allocator: Allocator, rand: std.rand.Random) ![]const u8 {
113 const Choice = enum {
114 backslash,
115 quote,
116 space,
117 control,
118 printable,
119 surrogate_half,
120 non_ascii,
121 };
122
123 const choices = rand.uintAtMostBiased(u16, 256);
124 var buf = try std.ArrayList(u8).initCapacity(allocator, choices);
125 errdefer buf.deinit();
126
127 var last_codepoint: u21 = 0;
128 for (0..choices) |_| {
129 const choice = rand.enumValue(Choice);
130 const codepoint: u21 = switch (choice) {
131 .backslash => '\\',
132 .quote => '"',
133 .space => ' ',
134 .control => switch (rand.uintAtMostBiased(u8, 0x21)) {
135 // NUL/CR/LF can't roundtrip
136 '\x00', '\r', '\n' => ' ',
137 0x21 => '\x7F',
138 else => |b| b,
139 },
140 .printable => '!' + rand.uintAtMostBiased(u8, '~' - '!'),
141 .surrogate_half => rand.intRangeAtMostBiased(u16, 0xD800, 0xDFFF),
142 .non_ascii => rand.intRangeAtMostBiased(u21, 0x80, 0x10FFFF),
143 };
144 // Ensure that we always return well-formed WTF-8.
145 // Instead of concatenating to ensure well-formed WTF-8,
146 // we just skip encoding the low surrogate.
147 if (std.unicode.isSurrogateCodepoint(last_codepoint) and std.unicode.isSurrogateCodepoint(codepoint)) {
148 if (std.unicode.utf16IsHighSurrogate(@intCast(last_codepoint)) and std.unicode.utf16IsLowSurrogate(@intCast(codepoint))) {
149 continue;
150 }
151 }
152 try buf.ensureUnusedCapacity(4);
153 const unused_slice = buf.unusedCapacitySlice();
154 const len = std.unicode.wtf8Encode(codepoint, unused_slice) catch unreachable;
155 buf.items.len += len;
156 last_codepoint = codepoint;
157 }
158
159 return buf.toOwnedSlice();
160}
test/standalone/windows_bat_args/test.zig created+132
...@@ -0,0 +1,132 @@
1const std = @import("std");
2
3pub fn main() anyerror!void {
4 var gpa = std.heap.GeneralPurposeAllocator(.{}){};
5 defer if (gpa.deinit() == .leak) @panic("found memory leaks");
6 const allocator = gpa.allocator();
7
8 var it = try std.process.argsWithAllocator(allocator);
9 defer it.deinit();
10 _ = it.next() orelse unreachable; // skip binary name
11 const child_exe_path = it.next() orelse unreachable;
12
13 var tmp = std.testing.tmpDir(.{});
14 defer tmp.cleanup();
15
16 try tmp.dir.setAsCwd();
17 defer tmp.parent_dir.setAsCwd() catch {};
18
19 var buf = try std.ArrayList(u8).initCapacity(allocator, 128);
20 defer buf.deinit();
21 try buf.appendSlice("@echo off\n");
22 try buf.append('"');
23 try buf.appendSlice(child_exe_path);
24 try buf.append('"');
25 const preamble_len = buf.items.len;
26
27 try buf.appendSlice(" %*");
28 try tmp.dir.writeFile("args1.bat", buf.items);
29 buf.shrinkRetainingCapacity(preamble_len);
30
31 try buf.appendSlice(" %1 %2 %3 %4 %5 %6 %7 %8 %9");
32 try tmp.dir.writeFile("args2.bat", buf.items);
33 buf.shrinkRetainingCapacity(preamble_len);
34
35 try buf.appendSlice(" \"%~1\" \"%~2\" \"%~3\" \"%~4\" \"%~5\" \"%~6\" \"%~7\" \"%~8\" \"%~9\"");
36 try tmp.dir.writeFile("args3.bat", buf.items);
37 buf.shrinkRetainingCapacity(preamble_len);
38
39 // Test cases are from https://github.com/rust-lang/rust/blob/master/tests/ui/std/windows-bat-args.rs
40 try testExecError(error.InvalidBatchScriptArg, allocator, &.{"\x00"});
41 try testExecError(error.InvalidBatchScriptArg, allocator, &.{"\n"});
42 try testExecError(error.InvalidBatchScriptArg, allocator, &.{"\r"});
43 try testExec(allocator, &.{ "a", "b" }, null);
44 try testExec(allocator, &.{ "c is for cat", "d is for dog" }, null);
45 try testExec(allocator, &.{ "\"", " \"" }, null);
46 try testExec(allocator, &.{ "\\", "\\" }, null);
47 try testExec(allocator, &.{">file.txt"}, null);
48 try testExec(allocator, &.{"whoami.exe"}, null);
49 try testExec(allocator, &.{"&a.exe"}, null);
50 try testExec(allocator, &.{"&echo hello "}, null);
51 try testExec(allocator, &.{ "&echo hello", "&whoami", ">file.txt" }, null);
52 try testExec(allocator, &.{"!TMP!"}, null);
53 try testExec(allocator, &.{"key=value"}, null);
54 try testExec(allocator, &.{"\"key=value\""}, null);
55 try testExec(allocator, &.{"key = value"}, null);
56 try testExec(allocator, &.{"key=[\"value\"]"}, null);
57 try testExec(allocator, &.{ "", "a=b" }, null);
58 try testExec(allocator, &.{"key=\"foo bar\""}, null);
59 try testExec(allocator, &.{"key=[\"my_value]"}, null);
60 try testExec(allocator, &.{"key=[\"my_value\",\"other-value\"]"}, null);
61 try testExec(allocator, &.{"key\\=value"}, null);
62 try testExec(allocator, &.{"key=\"&whoami\""}, null);
63 try testExec(allocator, &.{"key=\"value\"=5"}, null);
64 try testExec(allocator, &.{"key=[\">file.txt\"]"}, null);
65 try testExec(allocator, &.{"%hello"}, null);
66 try testExec(allocator, &.{"%PATH%"}, null);
67 try testExec(allocator, &.{"%%cd:~,%"}, null);
68 try testExec(allocator, &.{"%PATH%PATH%"}, null);
69 try testExec(allocator, &.{"\">file.txt"}, null);
70 try testExec(allocator, &.{"abc\"&echo hello"}, null);
71 try testExec(allocator, &.{"123\">file.txt"}, null);
72 try testExec(allocator, &.{"\"&echo hello&whoami.exe"}, null);
73 try testExec(allocator, &.{ "\"hello^\"world\"", "hello &echo oh no >file.txt" }, null);
74 try testExec(allocator, &.{"&whoami.exe"}, null);
75
76 var env = env: {
77 var env = try std.process.getEnvMap(allocator);
78 errdefer env.deinit();
79 // No escaping
80 try env.put("FOO", "123");
81 // Some possible escaping of %FOO% that could be expanded
82 // when escaping cmd.exe meta characters with ^
83 try env.put("FOO^", "123"); // only escaping %
84 try env.put("^F^O^O^", "123"); // escaping every char
85 break :env env;
86 };
87 defer env.deinit();
88 try testExec(allocator, &.{"%FOO%"}, &env);
89
90 // Ensure that none of the `>file.txt`s have caused file.txt to be created
91 try std.testing.expectError(error.FileNotFound, tmp.dir.access("file.txt", .{}));
92}
93
94fn testExecError(err: anyerror, allocator: std.mem.Allocator, args: []const []const u8) !void {
95 return std.testing.expectError(err, testExec(allocator, args, null));
96}
97
98fn testExec(allocator: std.mem.Allocator, args: []const []const u8, env: ?*std.process.EnvMap) !void {
99 try testExecBat(allocator, "args1.bat", args, env);
100 try testExecBat(allocator, "args2.bat", args, env);
101 try testExecBat(allocator, "args3.bat", args, env);
102}
103
104fn testExecBat(allocator: std.mem.Allocator, bat: []const u8, args: []const []const u8, env: ?*std.process.EnvMap) !void {
105 var argv = try std.ArrayList([]const u8).initCapacity(allocator, 1 + args.len);
106 defer argv.deinit();
107 argv.appendAssumeCapacity(bat);
108 argv.appendSliceAssumeCapacity(args);
109
110 const can_have_trailing_empty_args = std.mem.eql(u8, bat, "args3.bat");
111
112 const result = try std.ChildProcess.run(.{
113 .allocator = allocator,
114 .env_map = env,
115 .argv = argv.items,
116 });
117 defer allocator.free(result.stdout);
118 defer allocator.free(result.stderr);
119
120 try std.testing.expectEqualStrings("", result.stderr);
121 var it = std.mem.splitScalar(u8, result.stdout, '\x00');
122 var i: usize = 0;
123 while (it.next()) |actual_arg| {
124 if (i >= args.len and can_have_trailing_empty_args) {
125 try std.testing.expectEqualStrings("", actual_arg);
126 continue;
127 }
128 const expected_arg = args[i];
129 try std.testing.expectEqualStrings(expected_arg, actual_arg);
130 i += 1;
131 }
132}