Commit c321b2f2a0257fa45262f1ece0a263b22e8c70f3 | zig

author	Ryan Liptak <squeek502@hotmail.com> 2022-06-23 17:20:24-07:00
committer	Ryan Liptak <squeek502@hotmail.com> 2022-06-23 17:20:24-07:00
log	c321b2f2a0257fa45262f1ece0a263b22e8c70f3
tree	cf0a1ada1de0b1cf7a3d4a69f784f052c33d3ef2
parent	def304a9a5b9d001e37c19509a9d2b2450088033

checkAllAllocationFailures: add possibility of SwallowedOutOfMemoryError (split from NondeterministicMemoryUsage)

Inducing failure but not getting OutOfMemory back is not as much of a problem as never inducing failure when it was expected to be induced, so treating them differently and allowing them to be handled differently by the caller is useful.

For example, the current implementation of `std.HashMapUnmanaged.getOrPutContextAdapted` always tries to grow and then recovers from OutOfMemory by attempting a lookup of an existing key. If this function is used (i.e. from `std.BufMap.putMove`) with `checkAllAllocationFailures`, then we'd have previously triggered `error.NondeterministicMemoryUsage`, but the real cause is that `OutOfMemory` is being recovered from and so the error is being swallowed. The new error allows us to both understand what's happening easier and to catch it and ignore it if we're okay with the code we're testing handling `error.OutOfMemory` without always bubbling it up.

1 files changed, 18 insertions(+), 5 deletions(-)

lib
- std
  - testing.zig

lib/std/testing.zig+18-5

 @@ -534,18 +534,27 @@ test {
 ///
 /// Any relevant state shared between runs of `test_fn` *must* be reset within `test_fn`.
 ///
-/// Expects that the `test_fn` has a deterministic number of memory allocations
-/// (an error will be returned if non-deterministic allocations are detected).
-///
 /// The strategy employed is to:
 /// - Run the test function once to get the total number of allocations.
 /// - Then, iterate and run the function X more times, incrementing
 ///   the failing index each iteration (where X is the total number of
 ///   allocations determined previously)
 ///
+/// Expects that `test_fn` has a deterministic number of memory allocations:
+/// - If an allocation was made to fail during a run of `test_fn`, but `test_fn`
+///   didn't return `error.OutOfMemory`, then `error.SwallowedOutOfMemoryError`
+///   is returned from `checkAllAllocationFailures`. You may want to ignore this
+///   depending on whether or not the code you're testing includes some strategies
+///   for recovering from `error.OutOfMemory`.
+/// - If a run of `test_fn` with an expected allocation failure executes without
+///   an allocation failure being induced, then `error.NondeterministicMemoryUsage`
+///   is returned. This error means that there are allocation points that won't be
+///   tested by the strategy this function employs (that is, there are sometimes more
+///   points of allocation than the initial run of `test_fn` detects).
+///
 /// ---
 ///
-/// Here's an example of using a simple test case that will cause a leak when the
+/// Here's an example using a simple test case that will cause a leak when the
 /// allocation of `bar` fails (but will pass normally):
 ///
 /// ```zig
 @@ -645,7 +654,11 @@ pub fn checkAllAllocationFailures(backing_allocator: std.mem.Allocator, comptime
         args.@"0" = failing_allocator_inst.allocator();
         if (@call(.{}, test_fn, args)) |_| {
-            return error.NondeterministicMemoryUsage;
+            if (failing_allocator_inst.has_induced_failure) {
+                return error.SwallowedOutOfMemoryError;
+            } else {
+                return error.NondeterministicMemoryUsage;
+            }
         } else |err| switch (err) {
             error.OutOfMemory => {
                 if (failing_allocator_inst.allocated_bytes != failing_allocator_inst.freed_bytes) {