Commit a217c764db0a1dae539c3b243ebc329350485eb5 | zig

author	Andrew Kelley <andrew@ziglang.org> 2018-03-01 22:25:15-05:00
committer	Andrew Kelley <andrew@ziglang.org> 2018-03-01 22:25:15-05:00
log	a217c764db0a1dae539c3b243ebc329350485eb5
tree	3e10e675ba5830d7da3e3fe7264ce9f3b2ca17de
parent	4955c4b8f99bc45ad9aacb13de691614c4e0ad38
parent	7d494b3e7b09403358232dc61f45374d6c26905f

Merge remote-tracking branch 'origin/master' into llvm6

22 files changed, 3270 insertions(+), 237 deletions(-)

doc/langref.html.in+118-22

...	@@ -2782,30 +2782,96 @@ test "fn reflection" {	...	@@ -2782,30 +2782,96 @@ test "fn reflection" {
2782	{#header_close#}	2782	{#header_close#}
2783	{#header_close#}	2783	{#header_close#}
2784	{#header_open\|Errors#}	2784	{#header_open\|Errors#}
		2785	{#header_open\|Error Set Type#}
2785	<p>	2786	<p>
2786	One of the distinguishing features of Zig is its exception handling strategy.	2787	An error set is like an {#link\|enum#}.
		2788	However, each error name across the entire compilation gets assigned an unsigned integer
		2789	greater than 0. You are allowed to declare the same error name more than once, and if you do, it
		2790	gets assigned the same integer value.
2787	</p>	2791	</p>
2788	<p>	2792	<p>
2789	TODO rewrite the errors section to take into account error sets	2793	The number of unique error values across the entire compilation should determine the size of the error set type.
		2794	However right now it is hard coded to be a <code>u16</code>. See <a href="https://github.com/zig-lang/zig/issues/786">#768</a>.
2790	</p>	2795	</p>
2791	<p>	2796	<p>
2792	These error values are assigned an unsigned integer value greater than 0 at	2797	You can implicitly cast an error from a subset to its superset:
2793	compile time. You are allowed to declare the same error value more than once,
2794	and if you do, it gets assigned the same integer value.
2795	</p>	2798	</p>
		2799	{#code_begin\|test#}
		2800	const std = @import("std");
		2801
		2802	const FileOpenError = error {
		2803	AccessDenied,
		2804	OutOfMemory,
		2805	FileNotFound,
		2806	};
		2807
		2808	const AllocationError = error {
		2809	OutOfMemory,
		2810	};
		2811
		2812	test "implicit cast subset to superset" {
		2813	const err = foo(AllocationError.OutOfMemory);
		2814	std.debug.assert(err == FileOpenError.OutOfMemory);
		2815	}
		2816
		2817	fn foo(err: AllocationError) FileOpenError {
		2818	return err;
		2819	}
		2820	{#code_end#}
		2821	<p>
		2822	But you cannot implicitly cast an error from a superset to a subset:
		2823	</p>
		2824	{#code_begin\|test_err\|not a member of destination error set#}
		2825	const FileOpenError = error {
		2826	AccessDenied,
		2827	OutOfMemory,
		2828	FileNotFound,
		2829	};
		2830
		2831	const AllocationError = error {
		2832	OutOfMemory,
		2833	};
		2834
		2835	test "implicit cast superset to subset" {
		2836	foo(FileOpenError.OutOfMemory) catch {};
		2837	}
		2838
		2839	fn foo(err: FileOpenError) AllocationError {
		2840	return err;
		2841	}
		2842	{#code_end#}
		2843	<p>
		2844	There is a shortcut for declaring an error set with only 1 value, and then getting that value:
		2845	</p>
		2846	{#code_begin\|syntax#}
		2847	const err = error.FileNotFound;
		2848	{#code_end#}
		2849	<p>This is equivalent to:</p>
		2850	{#code_begin\|syntax#}
		2851	const err = (error {FileNotFound}).FileNotFound;
		2852	{#code_end#}
2796	<p>	2853	<p>
2797	You can refer to these error values with the error namespace such as	2854	This becomes useful when using {#link\|Inferred Error Sets#}.
2798	<code>error.FileNotFound</code>.	2855	</p>
		2856	{#header_open\|The Global Error Set#}
		2857	<p><code>error</code> refers to the global error set.
		2858	This is the error set that contains all errors in the entire compilation unit.
		2859	It is a superset of all other error sets and a subset of none of them.
2799	</p>	2860	</p>
2800	<p>	2861	<p>
2801	Each error value across the entire compilation unit gets a unique integer,	2862	You can implicitly cast any error set to the global one, and you can explicitly
2802	and this determines the size of the error set type.	2863	cast an error of global error set to a non-global one. This inserts a language-level
		2864	assert to make sure the error value is in fact in the destination error set.
2803	</p>	2865	</p>
2804	<p>	2866	<p>
2805	The error set type is one of the error values, and in the same way that pointers	2867	The global error set should generally be avoided when possible, because it prevents
2806	cannot be null, a error set instance is always an error.	2868	the compiler from knowing what errors are possible at compile-time. Knowing
		2869	the error set at compile-time is better for generated documentationt and for
		2870	helpful error messages such as forgetting a possible error value in a {#link\|switch#}.
2807	</p>	2871	</p>
2808	{#code_begin\|syntax#}const pure_error = error.FileNotFound;{#code_end#}	2872	{#header_close#}
		2873	{#header_close#}
		2874	{#header_open\|Error Union Type#}
2809	<p>	2875	<p>
2810	Most of the time you will not find yourself using an error set type. Instead,	2876	Most of the time you will not find yourself using an error set type. Instead,
2811	likely you will be using the error union type. This is when you take an error set	2877	likely you will be using the error union type. This is when you take an error set
...	@@ -2918,7 +2984,6 @@ fn doAThing(str: []u8) !void {	...	@@ -2918,7 +2984,6 @@ fn doAThing(str: []u8) !void {
2918	a panic in Debug and ReleaseSafe modes and undefined behavior in ReleaseFast mode. So, while we're debugging the	2984	a panic in Debug and ReleaseSafe modes and undefined behavior in ReleaseFast mode. So, while we're debugging the
2919	application, if there <em>was</em> a surprise error here, the application would crash	2985	application, if there <em>was</em> a surprise error here, the application would crash
2920	appropriately.	2986	appropriately.
2921	TODO: mention error return traces
2922	</p>	2987	</p>
2923	<p>	2988	<p>
2924	Finally, you may want to take a different action for every situation. For that, we combine	2989	Finally, you may want to take a different action for every situation. For that, we combine
...	@@ -2986,7 +3051,7 @@ fn createFoo(param: i32) !Foo {	...	@@ -2986,7 +3051,7 @@ fn createFoo(param: i32) !Foo {
2986	</li>	3051	</li>
2987	</ul>	3052	</ul>
2988	{#see_also\|defer\|if\|switch#}	3053	{#see_also\|defer\|if\|switch#}
2989	{#header_open\|Error Union Type#}	3054
2990	<p>An error union is created with the <code>!</code> binary operator.	3055	<p>An error union is created with the <code>!</code> binary operator.
2991	You can use compile-time reflection to access the child type of an error union:</p>	3056	You can use compile-time reflection to access the child type of an error union:</p>
2992	{#code_begin\|test#}	3057	{#code_begin\|test#}
...	@@ -3008,8 +3073,12 @@ test "error union" {	...	@@ -3008,8 +3073,12 @@ test "error union" {
3008	comptime assert(@typeOf(foo).ErrorSet == error);	3073	comptime assert(@typeOf(foo).ErrorSet == error);
3009	}	3074	}
3010	{#code_end#}	3075	{#code_end#}
		3076	<p>TODO the <code>\|\|</code> operator for error sets</p>
		3077	{#header_open\|Inferred Error Sets#}
		3078	<p>TODO</p>
3011	{#header_close#}	3079	{#header_close#}
3012	{#header_open\|Error Set Type#}	3080	{#header_close#}
		3081	{#header_open\|Error Return Traces#}
3013	<p>TODO</p>	3082	<p>TODO</p>
3014	{#header_close#}	3083	{#header_close#}
3015	{#header_close#}	3084	{#header_close#}
...	@@ -3775,6 +3844,25 @@ pub fn main() void {	...	@@ -3775,6 +3844,25 @@ pub fn main() void {
3775	{#header_open\|@ArgType#}	3844	{#header_open\|@ArgType#}
3776	<p>TODO</p>	3845	<p>TODO</p>
3777	{#header_close#}	3846	{#header_close#}
		3847	{#header_open\|@atomicRmw#}
		3848	<pre><code class="zig">@atomicRmw(comptime T: type, ptr: &T, comptime op: builtin.AtomicRmwOp, operand: T, comptime ordering: builtin.AtomicOrder) -> T</code></pre>
		3849	<p>
		3850	This builtin function atomically modifies memory and then returns the previous value.
		3851	</p>
		3852	<p>
		3853	<code>T</code> must be a pointer type, a <code>bool</code>,
		3854	or an integer whose bit count meets these requirements:
		3855	</p>
		3856	<ul>
		3857	<li>At least 8</li>
		3858	<li>At most the same as usize</li>
		3859	<li>Power of 2</li>
		3860	</ul>
		3861	<p>
		3862	TODO right now bool is not accepted. Also I think we could make non powers of 2 work fine, maybe
		3863	we can remove this restriction
		3864	</p>
		3865	{#header_close#}
3778	{#header_open\|@bitCast#}	3866	{#header_open\|@bitCast#}
3779	<pre><code class="zig">@bitCast(comptime DestType: type, value: var) -> DestType</code></pre>	3867	<pre><code class="zig">@bitCast(comptime DestType: type, value: var) -> DestType</code></pre>
3780	<p>	3868	<p>
...	@@ -5645,7 +5733,7 @@ UseDecl = "use" Expression ";"	...	@@ -5645,7 +5733,7 @@ UseDecl = "use" Expression ";"
5645		5733
5646	ExternDecl = "extern" option(String) (FnProto \| VariableDeclaration) ";"	5734	ExternDecl = "extern" option(String) (FnProto \| VariableDeclaration) ";"
5647		5735
5648	FnProto = option("nakedcc" \| "stdcallcc" \| "extern") "fn" option(Symbol) ParamDeclList option("align" "(" Expression ")") option("section" "(" Expression ")") option("!") TypeExpr	5736	FnProto = option("nakedcc" \| "stdcallcc" \| "extern" \| ("async" option("(" Expression ")"))) "fn" option(Symbol) ParamDeclList option("align" "(" Expression ")") option("section" "(" Expression ")") option("!") TypeExpr
5649		5737
5650	FnDef = option("inline" \| "export") FnProto Block	5738	FnDef = option("inline" \| "export") FnProto Block
5651		5739
...	@@ -5663,7 +5751,7 @@ ErrorSetExpr = (PrefixOpExpression "!" PrefixOpExpression) \| PrefixOpExpression	...	@@ -5663,7 +5751,7 @@ ErrorSetExpr = (PrefixOpExpression "!" PrefixOpExpression) \| PrefixOpExpression
5663		5751
5664	BlockOrExpression = Block \| Expression	5752	BlockOrExpression = Block \| Expression
5665		5753
5666	Expression = TryExpression \| ReturnExpression \| BreakExpression \| AssignmentExpression	5754	Expression = TryExpression \| ReturnExpression \| BreakExpression \| AssignmentExpression \| CancelExpression \| ResumeExpression
5667		5755
5668	AsmExpression = "asm" option("volatile") "(" String option(AsmOutput) ")"	5756	AsmExpression = "asm" option("volatile") "(" String option(AsmOutput) ")"
5669		5757
...	@@ -5687,7 +5775,7 @@ AssignmentExpression = UnwrapExpression AssignmentOperator UnwrapExpression \| Un	...	@@ -5687,7 +5775,7 @@ AssignmentExpression = UnwrapExpression AssignmentOperator UnwrapExpression \| Un
5687		5775
5688	AssignmentOperator = "=" \| "=" \| "/=" \| "%=" \| "+=" \| "-=" \| "<<=" \| ">>=" \| "&=" \| "^=" \| "\|=" \| "%=" \| "+%=" \| "-%="	5776	AssignmentOperator = "=" \| "=" \| "/=" \| "%=" \| "+=" \| "-=" \| "<<=" \| ">>=" \| "&=" \| "^=" \| "\|=" \| "%=" \| "+%=" \| "-%="
5689		5777
5690	BlockExpression(body) = Block \| IfExpression(body) \| IfErrorExpression(body) \| TestExpression(body) \| WhileExpression(body) \| ForExpression(body) \| SwitchExpression \| CompTimeExpression(body)	5778	BlockExpression(body) = Block \| IfExpression(body) \| IfErrorExpression(body) \| TestExpression(body) \| WhileExpression(body) \| ForExpression(body) \| SwitchExpression \| CompTimeExpression(body) \| SuspendExpression(body)
5691		5779
5692	CompTimeExpression(body) = "comptime" body	5780	CompTimeExpression(body) = "comptime" body
5693		5781
...	@@ -5705,12 +5793,20 @@ ReturnExpression = "return" option(Expression)	...	@@ -5705,12 +5793,20 @@ ReturnExpression = "return" option(Expression)
5705		5793
5706	TryExpression = "try" Expression	5794	TryExpression = "try" Expression
5707		5795
		5796	AwaitExpression = "await" Expression
		5797
5708	BreakExpression = "break" option(":" Symbol) option(Expression)	5798	BreakExpression = "break" option(":" Symbol) option(Expression)
5709		5799
		5800	CancelExpression = "cancel" Expression;
		5801
		5802	ResumeExpression = "resume" Expression;
		5803
5710	Defer(body) = ("defer" \| "deferror") body	5804	Defer(body) = ("defer" \| "deferror") body
5711		5805
5712	IfExpression(body) = "if" "(" Expression ")" body option("else" BlockExpression(body))	5806	IfExpression(body) = "if" "(" Expression ")" body option("else" BlockExpression(body))
5713		5807
		5808	SuspendExpression(body) = "suspend" option(("\|" Symbol "\|" body))
		5809
5714	IfErrorExpression(body) = "if" "(" Expression ")" option("\|" option("*") Symbol "\|") body "else" "\|" Symbol "\|" BlockExpression(body)	5810	IfErrorExpression(body) = "if" "(" Expression ")" option("\|" option("*") Symbol "\|") body "else" "\|" Symbol "\|" BlockExpression(body)
5715		5811
5716	TestExpression(body) = "if" "(" Expression ")" option("\|" option("*") Symbol "\|") body option("else" BlockExpression(body))	5812	TestExpression(body) = "if" "(" Expression ")" option("\|" option("*") Symbol "\|") body option("else" BlockExpression(body))
...	@@ -5745,7 +5841,7 @@ MultiplyOperator = "\|\|" \| "" \| "/" \| "%" \| "" \| "%"	...	@@ -5745,7 +5841,7 @@ MultiplyOperator = "\|\|" \| "" \| "/" \| "%" \| "" \| "%"
5745		5841
5746	PrefixOpExpression = PrefixOp ErrorSetExpr \| SuffixOpExpression	5842	PrefixOpExpression = PrefixOp ErrorSetExpr \| SuffixOpExpression
5747		5843
5748	SuffixOpExpression = PrimaryExpression option(FnCallExpression \| ArrayAccessExpression \| FieldAccessExpression \| SliceExpression)	5844	SuffixOpExpression = ("async" option("(" Expression ")") PrimaryExpression FnCallExpression) \| PrimaryExpression option(FnCallExpression \| ArrayAccessExpression \| FieldAccessExpression \| SliceExpression)
5749		5845
5750	FieldAccessExpression = "." Symbol	5846	FieldAccessExpression = "." Symbol
5751		5847
...	@@ -5761,7 +5857,7 @@ ContainerInitBody = list(StructLiteralField, ",") \| list(Expression, ",")	...	@@ -5761,7 +5857,7 @@ ContainerInitBody = list(StructLiteralField, ",") \| list(Expression, ",")
5761		5857
5762	StructLiteralField = "." Symbol "=" Expression	5858	StructLiteralField = "." Symbol "=" Expression
5763		5859
5764	PrefixOp = "!" \| "-" \| "~" \| "*" \| ("&" option("align" "(" Expression option(":" Integer ":" Integer) ")" ) option("const") option("volatile")) \| "?" \| "??" \| "-%" \| "try"	5860	PrefixOp = "!" \| "-" \| "~" \| "*" \| ("&" option("align" "(" Expression option(":" Integer ":" Integer) ")" ) option("const") option("volatile")) \| "?" \| "??" \| "-%" \| "try" \| "await"
5765		5861
5766	PrimaryExpression = Integer \| Float \| String \| CharLiteral \| KeywordLiteral \| GroupedExpression \| BlockExpression(BlockOrExpression) \| Symbol \| ("@" Symbol FnCallExpression) \| ArrayType \| FnProto \| AsmExpression \| ContainerDecl \| ("continue" option(":" Symbol)) \| ErrorSetDecl	5862	PrimaryExpression = Integer \| Float \| String \| CharLiteral \| KeywordLiteral \| GroupedExpression \| BlockExpression(BlockOrExpression) \| Symbol \| ("@" Symbol FnCallExpression) \| ArrayType \| FnProto \| AsmExpression \| ContainerDecl \| ("continue" option(":" Symbol)) \| ErrorSetDecl
5767		5863
...	@@ -5769,7 +5865,7 @@ ArrayType : "[" option(Expression) "]" option("align" "(" Expression option(":"	...	@@ -5769,7 +5865,7 @@ ArrayType : "[" option(Expression) "]" option("align" "(" Expression option(":"
5769		5865
5770	GroupedExpression = "(" Expression ")"	5866	GroupedExpression = "(" Expression ")"
5771		5867
5772	KeywordLiteral = "true" \| "false" \| "null" \| "undefined" \| "error" \| "this" \| "unreachable"	5868	KeywordLiteral = "true" \| "false" \| "null" \| "undefined" \| "error" \| "this" \| "unreachable" \| "suspend"
5773		5869
5774	ErrorSetDecl = "error" "{" list(Symbol, ",") "}"	5870	ErrorSetDecl = "error" "{" list(Symbol, ",") "}"
5775		5871
...	@@ -5853,7 +5949,7 @@ hljs.registerLanguage("zig", function(t) {	...	@@ -5853,7 +5949,7 @@ hljs.registerLanguage("zig", function(t) {
5853	a = t.IR + "\\s*\\(",	5949	a = t.IR + "\\s*\\(",
5854	c = {	5950	c = {
5855	keyword: "const align var extern stdcallcc nakedcc volatile export pub noalias inline struct packed enum union break return try catch test continue unreachable comptime and or asm defer errdefer if else switch while for fn use bool f32 f64 void type noreturn error i8 u8 i16 u16 i32 u32 i64 u64 isize usize i8w u8w i16w i32w u32w i64w u64w isizew usizew c_short c_ushort c_int c_uint c_long c_ulong c_longlong c_ulonglong",	5951	keyword: "const align var extern stdcallcc nakedcc volatile export pub noalias inline struct packed enum union break return try catch test continue unreachable comptime and or asm defer errdefer if else switch while for fn use bool f32 f64 void type noreturn error i8 u8 i16 u16 i32 u32 i64 u64 isize usize i8w u8w i16w i32w u32w i64w u64w isizew usizew c_short c_ushort c_int c_uint c_long c_ulong c_longlong c_ulonglong",
5856	built_in: "breakpoint returnAddress frameAddress fieldParentPtr setFloatMode IntType OpaqueType compileError compileLog setCold setRuntimeSafety setEvalBranchQuota offsetOf memcpy inlineCall setGlobalLinkage setGlobalSection divTrunc divFloor enumTagName intToPtr ptrToInt panic canImplicitCast ptrCast bitCast rem mod memset sizeOf alignOf alignCast maxValue minValue memberCount memberName memberType typeOf addWithOverflow subWithOverflow mulWithOverflow shlWithOverflow shlExact shrExact cInclude cDefine cUndef ctz clz import cImport errorName embedFile cmpxchg fence divExact truncate",	5952	built_in: "breakpoint returnAddress frameAddress fieldParentPtr setFloatMode IntType OpaqueType compileError compileLog setCold setRuntimeSafety setEvalBranchQuota offsetOf memcpy inlineCall setGlobalLinkage setGlobalSection divTrunc divFloor enumTagName intToPtr ptrToInt panic canImplicitCast ptrCast bitCast rem mod memset sizeOf alignOf alignCast maxValue minValue memberCount memberName memberType typeOf addWithOverflow subWithOverflow mulWithOverflow shlWithOverflow shlExact shrExact cInclude cDefine cUndef ctz clz import cImport errorName embedFile cmpxchg fence divExact truncate atomicRmw",
5857	literal: "true false null undefined"	5953	literal: "true false null undefined"
5858	},	5954	},
5859	n = [e, t.CLCM, t.CBCM, s, r];	5955	n = [e, t.CLCM, t.CBCM, s, r];

src/all_types.hpp+210

...	@@ -56,6 +56,16 @@ struct IrExecutable {	...	@@ -56,6 +56,16 @@ struct IrExecutable {
56	IrAnalyze *analysis;	56	IrAnalyze *analysis;
57	Scope *begin_scope;	57	Scope *begin_scope;
58	ZigList<Tld *> tld_list;	58	ZigList<Tld *> tld_list;
		59
		60	IrInstruction *coro_handle;
		61	IrInstruction *coro_awaiter_field_ptr; // this one is shared and in the promise
		62	IrInstruction *coro_result_ptr_field_ptr;
		63	IrInstruction *await_handle_var_ptr; // this one is where we put the one we extracted from the promise
		64	IrBasicBlock *coro_early_final;
		65	IrBasicBlock *coro_normal_final;
		66	IrBasicBlock *coro_suspend_block;
		67	IrBasicBlock *coro_final_cleanup_block;
		68	VariableTableEntry *coro_allocator_var;
59	};	69	};
60		70
61	enum OutType {	71	enum OutType {
...	@@ -393,6 +403,10 @@ enum NodeType {	...	@@ -393,6 +403,10 @@ enum NodeType {
393	NodeTypeIfErrorExpr,	403	NodeTypeIfErrorExpr,
394	NodeTypeTestExpr,	404	NodeTypeTestExpr,
395	NodeTypeErrorSetDecl,	405	NodeTypeErrorSetDecl,
		406	NodeTypeCancel,
		407	NodeTypeResume,
		408	NodeTypeAwaitExpr,
		409	NodeTypeSuspend,
396	};	410	};
397		411
398	struct AstNodeRoot {	412	struct AstNodeRoot {
...	@@ -405,6 +419,7 @@ enum CallingConvention {	...	@@ -405,6 +419,7 @@ enum CallingConvention {
405	CallingConventionCold,	419	CallingConventionCold,
406	CallingConventionNaked,	420	CallingConventionNaked,
407	CallingConventionStdcall,	421	CallingConventionStdcall,
		422	CallingConventionAsync,
408	};	423	};
409		424
410	struct AstNodeFnProto {	425	struct AstNodeFnProto {
...	@@ -426,6 +441,7 @@ struct AstNodeFnProto {	...	@@ -426,6 +441,7 @@ struct AstNodeFnProto {
426	AstNode *section_expr;	441	AstNode *section_expr;
427		442
428	bool auto_err_set;	443	bool auto_err_set;
		444	AstNode *async_allocator_type;
429	};	445	};
430		446
431	struct AstNodeFnDef {	447	struct AstNodeFnDef {
...	@@ -567,6 +583,8 @@ struct AstNodeFnCallExpr {	...	@@ -567,6 +583,8 @@ struct AstNodeFnCallExpr {
567	AstNode *fn_ref_expr;	583	AstNode *fn_ref_expr;
568	ZigList<AstNode *> params;	584	ZigList<AstNode *> params;
569	bool is_builtin;	585	bool is_builtin;
		586	bool is_async;
		587	AstNode *async_allocator;
570	};	588	};
571		589
572	struct AstNodeArrayAccessExpr {	590	struct AstNodeArrayAccessExpr {
...	@@ -829,6 +847,14 @@ struct AstNodeBreakExpr {	...	@@ -829,6 +847,14 @@ struct AstNodeBreakExpr {
829	AstNode *expr; // may be null	847	AstNode *expr; // may be null
830	};	848	};
831		849
		850	struct AstNodeCancelExpr {
		851	AstNode *expr;
		852	};
		853
		854	struct AstNodeResumeExpr {
		855	AstNode *expr;
		856	};
		857
832	struct AstNodeContinueExpr {	858	struct AstNodeContinueExpr {
833	Buf *name;	859	Buf *name;
834	};	860	};
...	@@ -843,6 +869,15 @@ struct AstNodeErrorType {	...	@@ -843,6 +869,15 @@ struct AstNodeErrorType {
843	struct AstNodeVarLiteral {	869	struct AstNodeVarLiteral {
844	};	870	};
845		871
		872	struct AstNodeAwaitExpr {
		873	AstNode *expr;
		874	};
		875
		876	struct AstNodeSuspend {
		877	AstNode *block;
		878	AstNode *promise_symbol;
		879	};
		880
846	struct AstNode {	881	struct AstNode {
847	enum NodeType type;	882	enum NodeType type;
848	size_t line;	883	size_t line;
...	@@ -900,6 +935,10 @@ struct AstNode {	...	@@ -900,6 +935,10 @@ struct AstNode {
900	AstNodeErrorType error_type;	935	AstNodeErrorType error_type;
901	AstNodeVarLiteral var_literal;	936	AstNodeVarLiteral var_literal;
902	AstNodeErrorSetDecl err_set_decl;	937	AstNodeErrorSetDecl err_set_decl;
		938	AstNodeCancelExpr cancel_expr;
		939	AstNodeResumeExpr resume_expr;
		940	AstNodeAwaitExpr await_expr;
		941	AstNodeSuspend suspend;
903	} data;	942	} data;
904	};	943	};
905		944
...	@@ -926,6 +965,7 @@ struct FnTypeId {	...	@@ -926,6 +965,7 @@ struct FnTypeId {
926	bool is_var_args;	965	bool is_var_args;
927	CallingConvention cc;	966	CallingConvention cc;
928	uint32_t alignment;	967	uint32_t alignment;
		968	TypeTableEntry *async_allocator_type;
929	};	969	};
930		970
931	uint32_t fn_type_id_hash(FnTypeId*);	971	uint32_t fn_type_id_hash(FnTypeId*);
...	@@ -1087,6 +1127,11 @@ struct TypeTableEntryBoundFn {	...	@@ -1087,6 +1127,11 @@ struct TypeTableEntryBoundFn {
1087	TypeTableEntry *fn_type;	1127	TypeTableEntry *fn_type;
1088	};	1128	};
1089		1129
		1130	struct TypeTableEntryPromise {
		1131	// null if `promise` instead of `promise->T`
		1132	TypeTableEntry *result_type;
		1133	};
		1134
1090	enum TypeTableEntryId {	1135	enum TypeTableEntryId {
1091	TypeTableEntryIdInvalid,	1136	TypeTableEntryIdInvalid,
1092	TypeTableEntryIdVar,	1137	TypeTableEntryIdVar,
...	@@ -1114,6 +1159,7 @@ enum TypeTableEntryId {	...	@@ -1114,6 +1159,7 @@ enum TypeTableEntryId {
1114	TypeTableEntryIdBoundFn,	1159	TypeTableEntryIdBoundFn,
1115	TypeTableEntryIdArgTuple,	1160	TypeTableEntryIdArgTuple,
1116	TypeTableEntryIdOpaque,	1161	TypeTableEntryIdOpaque,
		1162	TypeTableEntryIdPromise,
1117	};	1163	};
1118		1164
1119	struct TypeTableEntry {	1165	struct TypeTableEntry {
...	@@ -1140,11 +1186,14 @@ struct TypeTableEntry {	...	@@ -1140,11 +1186,14 @@ struct TypeTableEntry {
1140	TypeTableEntryUnion unionation;	1186	TypeTableEntryUnion unionation;
1141	TypeTableEntryFn fn;	1187	TypeTableEntryFn fn;
1142	TypeTableEntryBoundFn bound_fn;	1188	TypeTableEntryBoundFn bound_fn;
		1189	TypeTableEntryPromise promise;
1143	} data;	1190	} data;
1144		1191
1145	// use these fields to make sure we don't duplicate type table entries for the same type	1192	// use these fields to make sure we don't duplicate type table entries for the same type
1146	TypeTableEntry *pointer_parent[2]; // [0 - mut, 1 - const]	1193	TypeTableEntry *pointer_parent[2]; // [0 - mut, 1 - const]
1147	TypeTableEntry *maybe_parent;	1194	TypeTableEntry *maybe_parent;
		1195	TypeTableEntry *promise_parent;
		1196	TypeTableEntry *promise_frame_parent;
1148	// If we generate a constant name value for this type, we memoize it here.	1197	// If we generate a constant name value for this type, we memoize it here.
1149	// The type of this is array	1198	// The type of this is array
1150	ConstExprValue *cached_const_name_val;	1199	ConstExprValue *cached_const_name_val;
...	@@ -1297,6 +1346,7 @@ enum BuiltinFnId {	...	@@ -1297,6 +1346,7 @@ enum BuiltinFnId {
1297	BuiltinFnIdArgType,	1346	BuiltinFnIdArgType,
1298	BuiltinFnIdExport,	1347	BuiltinFnIdExport,
1299	BuiltinFnIdErrorReturnTrace,	1348	BuiltinFnIdErrorReturnTrace,
		1349	BuiltinFnIdAtomicRmw,
1300	};	1350	};
1301		1351
1302	struct BuiltinFnEntry {	1352	struct BuiltinFnEntry {
...	@@ -1470,6 +1520,7 @@ struct CodeGen {	...	@@ -1470,6 +1520,7 @@ struct CodeGen {
1470	TypeTableEntry *entry_u8;	1520	TypeTableEntry *entry_u8;
1471	TypeTableEntry *entry_u16;	1521	TypeTableEntry *entry_u16;
1472	TypeTableEntry *entry_u32;	1522	TypeTableEntry *entry_u32;
		1523	TypeTableEntry *entry_u29;
1473	TypeTableEntry *entry_u64;	1524	TypeTableEntry *entry_u64;
1474	TypeTableEntry *entry_u128;	1525	TypeTableEntry *entry_u128;
1475	TypeTableEntry *entry_i8;	1526	TypeTableEntry *entry_i8;
...	@@ -1495,6 +1546,7 @@ struct CodeGen {	...	@@ -1495,6 +1546,7 @@ struct CodeGen {
1495	TypeTableEntry *entry_var;	1546	TypeTableEntry *entry_var;
1496	TypeTableEntry *entry_global_error_set;	1547	TypeTableEntry *entry_global_error_set;
1497	TypeTableEntry *entry_arg_tuple;	1548	TypeTableEntry *entry_arg_tuple;
		1549	TypeTableEntry *entry_promise;
1498	} builtin_types;	1550	} builtin_types;
1499		1551
1500	EmitFileType emit_file_type;	1552	EmitFileType emit_file_type;
...	@@ -1581,6 +1633,18 @@ struct CodeGen {	...	@@ -1581,6 +1633,18 @@ struct CodeGen {
1581	LLVMValueRef trap_fn_val;	1633	LLVMValueRef trap_fn_val;
1582	LLVMValueRef return_address_fn_val;	1634	LLVMValueRef return_address_fn_val;
1583	LLVMValueRef frame_address_fn_val;	1635	LLVMValueRef frame_address_fn_val;
		1636	LLVMValueRef coro_destroy_fn_val;
		1637	LLVMValueRef coro_id_fn_val;
		1638	LLVMValueRef coro_alloc_fn_val;
		1639	LLVMValueRef coro_size_fn_val;
		1640	LLVMValueRef coro_begin_fn_val;
		1641	LLVMValueRef coro_suspend_fn_val;
		1642	LLVMValueRef coro_end_fn_val;
		1643	LLVMValueRef coro_free_fn_val;
		1644	LLVMValueRef coro_resume_fn_val;
		1645	LLVMValueRef coro_save_fn_val;
		1646	LLVMValueRef coro_promise_fn_val;
		1647	LLVMValueRef coro_alloc_helper_fn_val;
1584	bool error_during_imports;	1648	bool error_during_imports;
1585		1649
1586	const char **clang_argv;	1650	const char **clang_argv;
...	@@ -1803,6 +1867,19 @@ enum AtomicOrder {	...	@@ -1803,6 +1867,19 @@ enum AtomicOrder {
1803	AtomicOrderSeqCst,	1867	AtomicOrderSeqCst,
1804	};	1868	};
1805		1869
		1870	// synchronized with the code in define_builtin_compile_vars
		1871	enum AtomicRmwOp {
		1872	AtomicRmwOp_xchg,
		1873	AtomicRmwOp_add,
		1874	AtomicRmwOp_sub,
		1875	AtomicRmwOp_and,
		1876	AtomicRmwOp_nand,
		1877	AtomicRmwOp_or,
		1878	AtomicRmwOp_xor,
		1879	AtomicRmwOp_max,
		1880	AtomicRmwOp_min,
		1881	};
		1882
1806	// A basic block contains no branching. Branches send control flow	1883	// A basic block contains no branching. Branches send control flow
1807	// to another basic block.	1884	// to another basic block.
1808	// Phi instructions must be first in a basic block.	1885	// Phi instructions must be first in a basic block.
...	@@ -1939,6 +2016,22 @@ enum IrInstructionId {	...	@@ -1939,6 +2016,22 @@ enum IrInstructionId {
1939	IrInstructionIdExport,	2016	IrInstructionIdExport,
1940	IrInstructionIdErrorReturnTrace,	2017	IrInstructionIdErrorReturnTrace,
1941	IrInstructionIdErrorUnion,	2018	IrInstructionIdErrorUnion,
		2019	IrInstructionIdCancel,
		2020	IrInstructionIdGetImplicitAllocator,
		2021	IrInstructionIdCoroId,
		2022	IrInstructionIdCoroAlloc,
		2023	IrInstructionIdCoroSize,
		2024	IrInstructionIdCoroBegin,
		2025	IrInstructionIdCoroAllocFail,
		2026	IrInstructionIdCoroSuspend,
		2027	IrInstructionIdCoroEnd,
		2028	IrInstructionIdCoroFree,
		2029	IrInstructionIdCoroResume,
		2030	IrInstructionIdCoroSave,
		2031	IrInstructionIdCoroPromise,
		2032	IrInstructionIdCoroAllocHelper,
		2033	IrInstructionIdAtomicRmw,
		2034	IrInstructionIdPromiseResultType,
1942	};	2035	};
1943		2036
1944	struct IrInstruction {	2037	struct IrInstruction {
...	@@ -2142,6 +2235,9 @@ struct IrInstructionCall {	...	@@ -2142,6 +2235,9 @@ struct IrInstructionCall {
2142	bool is_comptime;	2235	bool is_comptime;
2143	LLVMValueRef tmp_ptr;	2236	LLVMValueRef tmp_ptr;
2144	FnInline fn_inline;	2237	FnInline fn_inline;
		2238	bool is_async;
		2239
		2240	IrInstruction *async_allocator;
2145	};	2241	};
2146		2242
2147	struct IrInstructionConst {	2243	struct IrInstructionConst {
...	@@ -2776,6 +2872,113 @@ struct IrInstructionErrorUnion {	...	@@ -2776,6 +2872,113 @@ struct IrInstructionErrorUnion {
2776	IrInstruction *payload;	2872	IrInstruction *payload;
2777	};	2873	};
2778		2874
		2875	struct IrInstructionCancel {
		2876	IrInstruction base;
		2877
		2878	IrInstruction *target;
		2879	};
		2880
		2881	enum ImplicitAllocatorId {
		2882	ImplicitAllocatorIdArg,
		2883	ImplicitAllocatorIdLocalVar,
		2884	};
		2885
		2886	struct IrInstructionGetImplicitAllocator {
		2887	IrInstruction base;
		2888
		2889	ImplicitAllocatorId id;
		2890	};
		2891
		2892	struct IrInstructionCoroId {
		2893	IrInstruction base;
		2894
		2895	IrInstruction *promise_ptr;
		2896	};
		2897
		2898	struct IrInstructionCoroAlloc {
		2899	IrInstruction base;
		2900
		2901	IrInstruction *coro_id;
		2902	};
		2903
		2904	struct IrInstructionCoroSize {
		2905	IrInstruction base;
		2906	};
		2907
		2908	struct IrInstructionCoroBegin {
		2909	IrInstruction base;
		2910
		2911	IrInstruction *coro_id;
		2912	IrInstruction *coro_mem_ptr;
		2913	};
		2914
		2915	struct IrInstructionCoroAllocFail {
		2916	IrInstruction base;
		2917
		2918	IrInstruction *err_val;
		2919	};
		2920
		2921	struct IrInstructionCoroSuspend {
		2922	IrInstruction base;
		2923
		2924	IrInstruction *save_point;
		2925	IrInstruction *is_final;
		2926	};
		2927
		2928	struct IrInstructionCoroEnd {
		2929	IrInstruction base;
		2930	};
		2931
		2932	struct IrInstructionCoroFree {
		2933	IrInstruction base;
		2934
		2935	IrInstruction *coro_id;
		2936	IrInstruction *coro_handle;
		2937	};
		2938
		2939	struct IrInstructionCoroResume {
		2940	IrInstruction base;
		2941
		2942	IrInstruction *awaiter_handle;
		2943	};
		2944
		2945	struct IrInstructionCoroSave {
		2946	IrInstruction base;
		2947
		2948	IrInstruction *coro_handle;
		2949	};
		2950
		2951	struct IrInstructionCoroPromise {
		2952	IrInstruction base;
		2953
		2954	IrInstruction *coro_handle;
		2955	};
		2956
		2957	struct IrInstructionCoroAllocHelper {
		2958	IrInstruction base;
		2959
		2960	IrInstruction *alloc_fn;
		2961	IrInstruction *coro_size;
		2962	};
		2963
		2964	struct IrInstructionAtomicRmw {
		2965	IrInstruction base;
		2966
		2967	IrInstruction *operand_type;
		2968	IrInstruction *ptr;
		2969	IrInstruction *op;
		2970	AtomicRmwOp resolved_op;
		2971	IrInstruction *operand;
		2972	IrInstruction *ordering;
		2973	AtomicOrder resolved_ordering;
		2974	};
		2975
		2976	struct IrInstructionPromiseResultType {
		2977	IrInstruction base;
		2978
		2979	IrInstruction *promise_type;
		2980	};
		2981
2779	static const size_t slice_ptr_index = 0;	2982	static const size_t slice_ptr_index = 0;
2780	static const size_t slice_len_index = 1;	2983	static const size_t slice_len_index = 1;
2781		2984
...	@@ -2785,6 +2988,13 @@ static const size_t maybe_null_index = 1;	...	@@ -2785,6 +2988,13 @@ static const size_t maybe_null_index = 1;
2785	static const size_t err_union_err_index = 0;	2988	static const size_t err_union_err_index = 0;
2786	static const size_t err_union_payload_index = 1;	2989	static const size_t err_union_payload_index = 1;
2787		2990
		2991	#define ASYNC_ALLOC_FIELD_NAME "allocFn"
		2992	#define ASYNC_FREE_FIELD_NAME "freeFn"
		2993	#define AWAITER_HANDLE_FIELD_NAME "awaiter_handle"
		2994	#define RESULT_FIELD_NAME "result"
		2995	#define RESULT_PTR_FIELD_NAME "result_ptr"
		2996
		2997
2788	enum FloatMode {	2998	enum FloatMode {
2789	FloatModeOptimized,	2999	FloatModeOptimized,
2790	FloatModeStrict,	3000	FloatModeStrict,

src/analyze.cpp+182-31

 @@ -230,6 +230,7 @@ bool type_is_complete(TypeTableEntry *type_entry) {
         case TypeTableEntryIdBlock:
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdArgTuple:
+        case TypeTableEntryIdPromise:
             return true;
     }
     zig_unreachable();
 @@ -267,6 +268,7 @@ bool type_has_zero_bits_known(TypeTableEntry *type_entry) {
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdArgTuple:
         case TypeTableEntryIdOpaque:
+        case TypeTableEntryIdPromise:
             return true;
     }
     zig_unreachable();
 @@ -339,6 +341,32 @@ TypeTableEntry *get_smallest_unsigned_int_type(CodeGen *g, uint64_t x) {
     return get_int_type(g, false, bits_needed_for_unsigned(x));
 }
+TypeTableEntry *get_promise_type(CodeGen *g, TypeTableEntry *result_type) {
+    if (result_type != nullptr && result_type->promise_parent != nullptr) {
+        return result_type->promise_parent;
+    } else if (result_type == nullptr && g->builtin_types.entry_promise != nullptr) {
+        return g->builtin_types.entry_promise;
+    }
+    TypeTableEntry *u8_ptr_type = get_pointer_to_type(g, g->builtin_types.entry_u8, false);
+    TypeTableEntry *entry = new_type_table_entry(TypeTableEntryIdPromise);
+    entry->type_ref = u8_ptr_type->type_ref;
+    entry->zero_bits = false;
+    entry->data.promise.result_type = result_type;
+    buf_init_from_str(&entry->name, "promise");
+    if (result_type != nullptr) {
+        buf_appendf(&entry->name, "->%s", buf_ptr(&result_type->name));
+    }
+    entry->di_type = u8_ptr_type->di_type;
+    if (result_type != nullptr) {
+        result_type->promise_parent = entry;
+    } else if (result_type == nullptr) {
+        g->builtin_types.entry_promise = entry;
+    }
+    return entry;
+}
 TypeTableEntry *get_pointer_to_type_extra(CodeGen *g, TypeTableEntry *child_type, bool is_const,
         bool is_volatile, uint32_t byte_alignment, uint32_t bit_offset, uint32_t unaligned_bit_count)
 {
 @@ -429,6 +457,23 @@ TypeTableEntry *get_pointer_to_type(CodeGen *g, TypeTableEntry *child_type, bool
     return get_pointer_to_type_extra(g, child_type, is_const, false, get_abi_alignment(g, child_type), 0, 0);
 }
+TypeTableEntry *get_promise_frame_type(CodeGen *g, TypeTableEntry *return_type) {
+    if (return_type->promise_frame_parent != nullptr) {
+        return return_type->promise_frame_parent;
+    }
+    TypeTableEntry *awaiter_handle_type = get_maybe_type(g, g->builtin_types.entry_promise);
+    TypeTableEntry *result_ptr_type = get_pointer_to_type(g, return_type, false);
+    const char *field_names[] = {AWAITER_HANDLE_FIELD_NAME, RESULT_FIELD_NAME, RESULT_PTR_FIELD_NAME};
+    TypeTableEntry *field_types[] = {awaiter_handle_type, return_type, result_ptr_type};
+    size_t field_count = type_has_bits(result_ptr_type) ? 3 : 1;
+    Buf *name = buf_sprintf("AsyncFramePromise(%s)", buf_ptr(&return_type->name));
+    TypeTableEntry *entry = get_struct_type(g, buf_ptr(name), field_names, field_types, field_count);
+    return_type->promise_frame_parent = entry;
+    return entry;
+}
 TypeTableEntry *get_maybe_type(CodeGen *g, TypeTableEntry *child_type) {
     if (child_type->maybe_parent) {
         TypeTableEntry *entry = child_type->maybe_parent;
 @@ -447,9 +492,7 @@ TypeTableEntry *get_maybe_type(CodeGen *g, TypeTableEntry *child_type) {
         if (child_type->zero_bits) {
             entry->type_ref = LLVMInt1Type();
             entry->di_type = g->builtin_types.entry_bool->di_type;
-        } else if (child_type->id == TypeTableEntryIdPointer ||
+        } else if (type_is_codegen_pointer(child_type)) {
-            child_type->id == TypeTableEntryIdFn)
-        {
             // this is an optimization but also is necessary for calling C
             // functions where all pointers are maybe pointers
             // function types are technically pointers
 @@ -884,6 +927,7 @@ static const char *calling_convention_name(CallingConvention cc) {
         case CallingConventionCold: return "coldcc";
         case CallingConventionNaked: return "nakedcc";
         case CallingConventionStdcall: return "stdcallcc";
+        case CallingConventionAsync: return "async";
     }
     zig_unreachable();
 }
 @@ -895,6 +939,21 @@ static const char *calling_convention_fn_type_str(CallingConvention cc) {
         case CallingConventionCold: return "coldcc ";
         case CallingConventionNaked: return "nakedcc ";
         case CallingConventionStdcall: return "stdcallcc ";
+        case CallingConventionAsync: return "async ";
+    }
+    zig_unreachable();
+}
+static bool calling_convention_allows_zig_types(CallingConvention cc) {
+    switch (cc) {
+        case CallingConventionUnspecified:
+        case CallingConventionAsync:
+            return true;
+        case CallingConventionC:
+        case CallingConventionCold:
+        case CallingConventionNaked:
+        case CallingConventionStdcall:
+            return false;
     }
     zig_unreachable();
 }
 @@ -924,8 +983,13 @@ TypeTableEntry *get_fn_type(CodeGen *g, FnTypeId *fn_type_id) {
     // populate the name of the type
     buf_resize(&fn_type->name, 0);
-    const char *cc_str = calling_convention_fn_type_str(fn_type->data.fn.fn_type_id.cc);
+    if (fn_type->data.fn.fn_type_id.cc == CallingConventionAsync) {
-    buf_appendf(&fn_type->name, "%sfn(", cc_str);
+        buf_appendf(&fn_type->name, "async(%s) ", buf_ptr(&fn_type_id->async_allocator_type->name));
+    } else {
+        const char *cc_str = calling_convention_fn_type_str(fn_type->data.fn.fn_type_id.cc);
+        buf_appendf(&fn_type->name, "%s", cc_str);
+    }
+    buf_appendf(&fn_type->name, "fn(");
     for (size_t i = 0; i < fn_type_id->param_count; i += 1) {
         FnTypeParamInfo *param_info = &fn_type_id->param_info[i];
 @@ -953,20 +1017,23 @@ TypeTableEntry *get_fn_type(CodeGen *g, FnTypeId *fn_type_id) {
     if (!skip_debug_info) {
         bool first_arg_return = calling_convention_does_first_arg_return(fn_type_id->cc) &&
             handle_is_ptr(fn_type_id->return_type);
-        bool prefix_arg_error_return_trace = g->have_err_ret_tracing &&
+        bool is_async = fn_type_id->cc == CallingConventionAsync;
-            (fn_type_id->return_type->id == TypeTableEntryIdErrorUnion ||
+        bool prefix_arg_error_return_trace = g->have_err_ret_tracing && fn_type_can_fail(fn_type_id);
-            fn_type_id->return_type->id == TypeTableEntryIdErrorSet);
         // +1 for maybe making the first argument the return value
-        // +1 for maybe last argument the error return trace
+        // +1 for maybe first argument the error return trace
-        LLVMTypeRef *gen_param_types = allocate<LLVMTypeRef>(2 + fn_type_id->param_count);
+        // +2 for maybe arguments async allocator and error code pointer
+        LLVMTypeRef *gen_param_types = allocate<LLVMTypeRef>(4 + fn_type_id->param_count);
         // +1 because 0 is the return type and
         // +1 for maybe making first arg ret val and
-        // +1 for maybe last argument the error return trace
+        // +1 for maybe first argument the error return trace
-        ZigLLVMDIType **param_di_types = allocate<ZigLLVMDIType*>(3 + fn_type_id->param_count);
+        // +2 for maybe arguments async allocator and error code pointer
+        ZigLLVMDIType **param_di_types = allocate<ZigLLVMDIType*>(5 + fn_type_id->param_count);
         param_di_types[0] = fn_type_id->return_type->di_type;
         size_t gen_param_index = 0;
         TypeTableEntry *gen_return_type;
-        if (!type_has_bits(fn_type_id->return_type)) {
+        if (is_async) {
+            gen_return_type = get_pointer_to_type(g, g->builtin_types.entry_u8, false);
+        } else if (!type_has_bits(fn_type_id->return_type)) {
             gen_return_type = g->builtin_types.entry_void;
         } else if (first_arg_return) {
             TypeTableEntry *gen_type = get_pointer_to_type(g, fn_type_id->return_type, false);
 @@ -987,6 +1054,25 @@ TypeTableEntry *get_fn_type(CodeGen *g, FnTypeId *fn_type_id) {
             // after the gen_param_index += 1 because 0 is the return type
             param_di_types[gen_param_index] = gen_type->di_type;
         }
+        if (is_async) {
+            {
+                // async allocator param
+                TypeTableEntry *gen_type = fn_type_id->async_allocator_type;
+                gen_param_types[gen_param_index] = gen_type->type_ref;
+                gen_param_index += 1;
+                // after the gen_param_index += 1 because 0 is the return type
+                param_di_types[gen_param_index] = gen_type->di_type;
+            }
+            {
+                // error code pointer
+                TypeTableEntry *gen_type = get_pointer_to_type(g, g->builtin_types.entry_global_error_set, false);
+                gen_param_types[gen_param_index] = gen_type->type_ref;
+                gen_param_index += 1;
+                // after the gen_param_index += 1 because 0 is the return type
+                param_di_types[gen_param_index] = gen_type->di_type;
+            }
+        }
         fn_type->data.fn.gen_param_info = allocate<FnGenParamInfo>(fn_type_id->param_count);
         for (size_t i = 0; i < fn_type_id->param_count; i += 1) {
 @@ -997,7 +1083,7 @@ TypeTableEntry *get_fn_type(CodeGen *g, FnTypeId *fn_type_id) {
             gen_param_info->src_index = i;
             gen_param_info->gen_index = SIZE_MAX;
-            ensure_complete_type(g, type_entry);
+            type_ensure_zero_bits_known(g, type_entry);
             if (type_has_bits(type_entry)) {
                 TypeTableEntry *gen_type;
                 if (handle_is_ptr(type_entry)) {
 @@ -1096,7 +1182,16 @@ TypeTableEntry *analyze_type_expr(CodeGen *g, Scope *scope, AstNode *node) {
 TypeTableEntry *get_generic_fn_type(CodeGen *g, FnTypeId *fn_type_id) {
     TypeTableEntry *fn_type = new_type_table_entry(TypeTableEntryIdFn);
     fn_type->is_copyable = false;
-    buf_init_from_str(&fn_type->name, "fn(");
+    buf_resize(&fn_type->name, 0);
+    if (fn_type->data.fn.fn_type_id.cc == CallingConventionAsync) {
+        const char *async_allocator_type_str = (fn_type->data.fn.fn_type_id.async_allocator_type == nullptr) ?
+            "var" : buf_ptr(&fn_type_id->async_allocator_type->name);
+        buf_appendf(&fn_type->name, "async(%s) ", async_allocator_type_str);
+    } else {
+        const char *cc_str = calling_convention_fn_type_str(fn_type->data.fn.fn_type_id.cc);
+        buf_appendf(&fn_type->name, "%s", cc_str);
+    }
+    buf_appendf(&fn_type->name, "fn(");
     size_t i = 0;
     for (; i < fn_type_id->next_param_index; i += 1) {
         const char *comma_str = (i == 0) ? "" : ",";
 @@ -1201,6 +1296,7 @@ static bool type_allowed_in_packed_struct(TypeTableEntry *type_entry) {
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdArgTuple:
         case TypeTableEntryIdOpaque:
+        case TypeTableEntryIdPromise:
             return false;
         case TypeTableEntryIdVoid:
         case TypeTableEntryIdBool:
 @@ -1217,7 +1313,7 @@ static bool type_allowed_in_packed_struct(TypeTableEntry *type_entry) {
         case TypeTableEntryIdMaybe:
             {
                 TypeTableEntry *child_type = type_entry->data.maybe.child_type;
-                return child_type->id == TypeTableEntryIdPointer || child_type->id == TypeTableEntryIdFn;
+                return type_is_codegen_pointer(child_type);
             }
         case TypeTableEntryIdEnum:
             return type_entry->data.enumeration.decl_node->data.container_decl.init_arg_expr != nullptr;
 @@ -1241,6 +1337,7 @@ static bool type_allowed_in_extern(CodeGen *g, TypeTableEntry *type_entry) {
         case TypeTableEntryIdBlock:
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdArgTuple:
+        case TypeTableEntryIdPromise:
             return false;
         case TypeTableEntryIdOpaque:
         case TypeTableEntryIdUnreachable:
 @@ -1312,7 +1409,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
         bool param_is_var_args = param_node->data.param_decl.is_var_args;
         if (param_is_comptime) {
-            if (fn_type_id.cc != CallingConventionUnspecified) {
+            if (!calling_convention_allows_zig_types(fn_type_id.cc)) {
                 add_node_error(g, param_node,
                         buf_sprintf("comptime parameter not allowed in function with calling convention '%s'",
                             calling_convention_name(fn_type_id.cc)));
 @@ -1323,7 +1420,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
             if (fn_type_id.cc == CallingConventionC) {
                 fn_type_id.param_count = fn_type_id.next_param_index;
                 continue;
-            } else if (fn_type_id.cc == CallingConventionUnspecified) {
+            } else if (calling_convention_allows_zig_types(fn_type_id.cc)) {
                 return get_generic_fn_type(g, &fn_type_id);
             } else {
                 add_node_error(g, param_node,
 @@ -1337,7 +1434,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
         if (type_is_invalid(type_entry)) {
             return g->builtin_types.entry_invalid;
         }
-        if (fn_type_id.cc != CallingConventionUnspecified) {
+        if (!calling_convention_allows_zig_types(fn_type_id.cc)) {
             type_ensure_zero_bits_known(g, type_entry);
             if (!type_has_bits(type_entry)) {
                 add_node_error(g, param_node->data.param_decl.type,
 @@ -1347,7 +1444,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
             }
         }
-        if (fn_type_id.cc != CallingConventionUnspecified && !type_allowed_in_extern(g, type_entry)) {
+        if (!calling_convention_allows_zig_types(fn_type_id.cc) && !type_allowed_in_extern(g, type_entry)) {
             add_node_error(g, param_node->data.param_decl.type,
                     buf_sprintf("parameter of type '%s' not allowed in function with calling convention '%s'",
                         buf_ptr(&type_entry->name),
 @@ -1367,7 +1464,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
                     buf_sprintf("parameter of type '%s' not allowed", buf_ptr(&type_entry->name)));
                 return g->builtin_types.entry_invalid;
             case TypeTableEntryIdVar:
-                if (fn_type_id.cc != CallingConventionUnspecified) {
+                if (!calling_convention_allows_zig_types(fn_type_id.cc)) {
                     add_node_error(g, param_node->data.param_decl.type,
                             buf_sprintf("parameter of type 'var' not allowed in function with calling convention '%s'",
                                 calling_convention_name(fn_type_id.cc)));
 @@ -1381,7 +1478,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
             case TypeTableEntryIdBoundFn:
             case TypeTableEntryIdMetaType:
                 add_node_error(g, param_node->data.param_decl.type,
-                    buf_sprintf("parameter of type '%s' must be declared inline",
+                    buf_sprintf("parameter of type '%s' must be declared comptime",
                     buf_ptr(&type_entry->name)));
                 return g->builtin_types.entry_invalid;
             case TypeTableEntryIdVoid:
 @@ -1397,8 +1494,9 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
             case TypeTableEntryIdEnum:
             case TypeTableEntryIdUnion:
             case TypeTableEntryIdFn:
+            case TypeTableEntryIdPromise:
                 ensure_complete_type(g, type_entry);
-                if (fn_type_id.cc == CallingConventionUnspecified && !type_is_copyable(g, type_entry)) {
+                if (calling_convention_allows_zig_types(fn_type_id.cc) && !type_is_copyable(g, type_entry)) {
                     add_node_error(g, param_node->data.param_decl.type,
                         buf_sprintf("type '%s' is not copyable; cannot pass by value", buf_ptr(&type_entry->name)));
                     return g->builtin_types.entry_invalid;
 @@ -1429,7 +1527,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
         fn_type_id.return_type = specified_return_type;
     }
-    if (fn_type_id.cc != CallingConventionUnspecified && !type_allowed_in_extern(g, fn_type_id.return_type)) {
+    if (!calling_convention_allows_zig_types(fn_type_id.cc) && !type_allowed_in_extern(g, fn_type_id.return_type)) {
         add_node_error(g, fn_proto->return_type,
                 buf_sprintf("return type '%s' not allowed in function with calling convention '%s'",
                     buf_ptr(&fn_type_id.return_type->name),
 @@ -1456,7 +1554,7 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdVar:
         case TypeTableEntryIdMetaType:
-            if (fn_type_id.cc != CallingConventionUnspecified) {
+            if (!calling_convention_allows_zig_types(fn_type_id.cc)) {
                 add_node_error(g, fn_proto->return_type,
                     buf_sprintf("return type '%s' not allowed in function with calling convention '%s'",
                     buf_ptr(&fn_type_id.return_type->name),
 @@ -1478,9 +1576,20 @@ static TypeTableEntry *analyze_fn_type(CodeGen *g, AstNode *proto_node, Scope *c
         case TypeTableEntryIdEnum:
         case TypeTableEntryIdUnion:
         case TypeTableEntryIdFn:
+        case TypeTableEntryIdPromise:
             break;
     }
+    if (fn_type_id.cc == CallingConventionAsync) {
+        if (fn_proto->async_allocator_type == nullptr) {
+            return get_generic_fn_type(g, &fn_type_id);
+        }
+        fn_type_id.async_allocator_type = analyze_type_expr(g, child_scope, fn_proto->async_allocator_type);
+        if (type_is_invalid(fn_type_id.async_allocator_type)) {
+            return g->builtin_types.entry_invalid;
+        }
+    }
     return get_fn_type(g, &fn_type_id);
 }
 @@ -1615,6 +1724,8 @@ TypeTableEntry *get_struct_type(CodeGen *g, const char *type_name, const char *f
         field->src_index = i;
         field->gen_index = i;
+        assert(type_has_bits(field->type_entry));
         auto prev_entry = struct_type->data.structure.fields_by_name.put_unique(field->name, field);
         assert(prev_entry == nullptr);
     }
 @@ -2129,6 +2240,7 @@ static void resolve_enum_zero_bits(CodeGen *g, TypeTableEntry *enum_type) {
     if (enum_type->data.enumeration.zero_bits_loop_flag) {
         enum_type->data.enumeration.zero_bits_known = true;
+        enum_type->data.enumeration.zero_bits_loop_flag = false;
         return;
     }
 @@ -2283,6 +2395,7 @@ static void resolve_struct_zero_bits(CodeGen *g, TypeTableEntry *struct_type) {
         // the alignment is pointer width, then assert that the first field is within that
         // alignment
         struct_type->data.structure.zero_bits_known = true;
+        struct_type->data.structure.zero_bits_loop_flag = false;
         if (struct_type->data.structure.abi_alignment == 0) {
             if (struct_type->data.structure.layout == ContainerLayoutPacked) {
                 struct_type->data.structure.abi_alignment = 1;
 @@ -3117,6 +3230,10 @@ void scan_decls(CodeGen *g, ScopeDecls *decls_scope, AstNode *node) {
         case NodeTypeIfErrorExpr:
         case NodeTypeTestExpr:
         case NodeTypeErrorSetDecl:
+        case NodeTypeCancel:
+        case NodeTypeResume:
+        case NodeTypeAwaitExpr:
+        case NodeTypeSuspend:
             zig_unreachable();
     }
 }
 @@ -3172,6 +3289,7 @@ TypeTableEntry *validate_var_type(CodeGen *g, AstNode *source_node, TypeTableEnt
         case TypeTableEntryIdUnion:
         case TypeTableEntryIdFn:
         case TypeTableEntryIdBoundFn:
+        case TypeTableEntryIdPromise:
             return type_entry;
     }
     zig_unreachable();
 @@ -3550,6 +3668,7 @@ static bool is_container(TypeTableEntry *type_entry) {
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdArgTuple:
         case TypeTableEntryIdOpaque:
+        case TypeTableEntryIdPromise:
             return false;
     }
     zig_unreachable();
 @@ -3600,6 +3719,7 @@ void resolve_container_type(CodeGen *g, TypeTableEntry *type_entry) {
         case TypeTableEntryIdVar:
         case TypeTableEntryIdArgTuple:
         case TypeTableEntryIdOpaque:
+        case TypeTableEntryIdPromise:
             zig_unreachable();
     }
 }
 @@ -3607,15 +3727,17 @@ void resolve_container_type(CodeGen *g, TypeTableEntry *type_entry) {
 TypeTableEntry *get_codegen_ptr_type(TypeTableEntry *type) {
     if (type->id == TypeTableEntryIdPointer) return type;
     if (type->id == TypeTableEntryIdFn) return type;
+    if (type->id == TypeTableEntryIdPromise) return type;
     if (type->id == TypeTableEntryIdMaybe) {
         if (type->data.maybe.child_type->id == TypeTableEntryIdPointer) return type->data.maybe.child_type;
         if (type->data.maybe.child_type->id == TypeTableEntryIdFn) return type->data.maybe.child_type;
+        if (type->data.maybe.child_type->id == TypeTableEntryIdPromise) return type->data.maybe.child_type;
     }
     return nullptr;
 }
 bool type_is_codegen_pointer(TypeTableEntry *type) {
-    return get_codegen_ptr_type(type) != nullptr;
+    return get_codegen_ptr_type(type) == type;
 }
 uint32_t get_ptr_align(TypeTableEntry *type) {
 @@ -3624,6 +3746,8 @@ uint32_t get_ptr_align(TypeTableEntry *type) {
         return ptr_type->data.pointer.alignment;
     } else if (ptr_type->id == TypeTableEntryIdFn) {
         return (ptr_type->data.fn.fn_type_id.alignment == 0) ? 1 : ptr_type->data.fn.fn_type_id.alignment;
+    } else if (ptr_type->id == TypeTableEntryIdPromise) {
+        return 1;
     } else {
         zig_unreachable();
     }
 @@ -3638,7 +3762,7 @@ AstNode *get_param_decl_node(FnTableEntry *fn_entry, size_t index) {
         return nullptr;
 }
-void define_local_param_variables(CodeGen *g, FnTableEntry *fn_table_entry, VariableTableEntry **arg_vars) {
+static void define_local_param_variables(CodeGen *g, FnTableEntry *fn_table_entry, VariableTableEntry **arg_vars) {
     TypeTableEntry *fn_type = fn_table_entry->type_entry;
     assert(!fn_type->data.fn.is_generic);
     FnTypeId *fn_type_id = &fn_type->data.fn.fn_type_id;
 @@ -3659,7 +3783,7 @@ void define_local_param_variables(CodeGen *g, FnTableEntry *fn_table_entry, Vari
         TypeTableEntry *param_type = param_info->type;
         bool is_noalias = param_info->is_noalias;
-        if (is_noalias && !type_is_codegen_pointer(param_type)) {
+        if (is_noalias && get_codegen_ptr_type(param_type) == nullptr) {
             add_node_error(g, param_decl_node, buf_sprintf("noalias on non-pointer parameter"));
         }
 @@ -4092,6 +4216,7 @@ bool handle_is_ptr(TypeTableEntry *type_entry) {
         case TypeTableEntryIdErrorSet:
         case TypeTableEntryIdFn:
         case TypeTableEntryIdEnum:
+        case TypeTableEntryIdPromise:
              return false;
         case TypeTableEntryIdArray:
         case TypeTableEntryIdStruct:
 @@ -4100,8 +4225,7 @@ bool handle_is_ptr(TypeTableEntry *type_entry) {
              return type_has_bits(type_entry->data.error_union.payload_type);
         case TypeTableEntryIdMaybe:
              return type_has_bits(type_entry->data.maybe.child_type) &&
-                    type_entry->data.maybe.child_type->id != TypeTableEntryIdPointer &&
+                    !type_is_codegen_pointer(type_entry->data.maybe.child_type);
-                    type_entry->data.maybe.child_type->id != TypeTableEntryIdFn;
         case TypeTableEntryIdUnion:
              assert(type_entry->data.unionation.complete);
              if (type_entry->data.unionation.gen_field_count == 0)
 @@ -4203,6 +4327,7 @@ uint32_t fn_type_id_hash(FnTypeId *id) {
     result += ((uint32_t)(id->cc)) * (uint32_t)3349388391;
     result += id->is_var_args ? (uint32_t)1931444534 : 0;
     result += hash_ptr(id->return_type);
+    result += hash_ptr(id->async_allocator_type);
     result += id->alignment * 0xd3b3f3e2;
     for (size_t i = 0; i < id->param_count; i += 1) {
         FnTypeParamInfo *info = &id->param_info[i];
 @@ -4217,7 +4342,8 @@ bool fn_type_id_eql(FnTypeId *a, FnTypeId *b) {
         a->return_type != b->return_type ||
         a->is_var_args != b->is_var_args ||
         a->param_count != b->param_count ||
-        a->alignment != b->alignment)
+        a->alignment != b->alignment ||
+        a->async_allocator_type != b->async_allocator_type)
     {
         return false;
     }
 @@ -4339,6 +4465,9 @@ static uint32_t hash_const_val(ConstExprValue *const_val) {
                 }
                 zig_unreachable();
             }
+        case TypeTableEntryIdPromise:
+            // TODO better hashing algorithm
+            return 223048345;
         case TypeTableEntryIdUndefLit:
             return 162837799;
         case TypeTableEntryIdNullLit:
 @@ -4498,6 +4627,7 @@ bool type_requires_comptime(TypeTableEntry *type_entry) {
         case TypeTableEntryIdPointer:
         case TypeTableEntryIdVoid:
         case TypeTableEntryIdUnreachable:
+        case TypeTableEntryIdPromise:
             return false;
     }
     zig_unreachable();
 @@ -4967,6 +5097,7 @@ bool const_values_equal(ConstExprValue *a, ConstExprValue *b) {
         case TypeTableEntryIdInvalid:
         case TypeTableEntryIdUnreachable:
         case TypeTableEntryIdVar:
+        case TypeTableEntryIdPromise:
             zig_unreachable();
     }
     zig_unreachable();
 @@ -5241,6 +5372,8 @@ void render_const_value(CodeGen *g, Buf *buf, ConstExprValue *const_val) {
                 buf_appendf(buf, "(args value)");
                 return;
             }
+        case TypeTableEntryIdPromise:
+            zig_unreachable();
     }
     zig_unreachable();
 }
 @@ -5302,6 +5435,7 @@ uint32_t type_id_hash(TypeId x) {
         case TypeTableEntryIdBlock:
         case TypeTableEntryIdBoundFn:
         case TypeTableEntryIdArgTuple:
+        case TypeTableEntryIdPromise:
             zig_unreachable();
         case TypeTableEntryIdErrorUnion:
             return hash_ptr(x.data.error_union.err_set_type) ^ hash_ptr(x.data.error_union.payload_type);
 @@ -5339,6 +5473,7 @@ bool type_id_eql(TypeId a, TypeId b) {
         case TypeTableEntryIdUndefLit:
         case TypeTableEntryIdNullLit:
         case TypeTableEntryIdMaybe:
+        case TypeTableEntryIdPromise:
         case TypeTableEntryIdErrorSet:
         case TypeTableEntryIdEnum:
         case TypeTableEntryIdUnion:
 @@ -5466,6 +5601,7 @@ static const TypeTableEntryId all_type_ids[] = {
     TypeTableEntryIdBoundFn,
     TypeTableEntryIdArgTuple,
     TypeTableEntryIdOpaque,
+    TypeTableEntryIdPromise,
 };
 TypeTableEntryId type_id_at_index(size_t index) {
 @@ -5530,6 +5666,8 @@ size_t type_id_index(TypeTableEntryId id) {
             return 22;
         case TypeTableEntryIdOpaque:
             return 23;
+        case TypeTableEntryIdPromise:
+            return 24;
     }
     zig_unreachable();
 }
 @@ -5587,6 +5725,8 @@ const char *type_id_name(TypeTableEntryId id) {
             return "ArgTuple";
         case TypeTableEntryIdOpaque:
             return "Opaque";
+        case TypeTableEntryIdPromise:
+            return "Promise";
     }
     zig_unreachable();
 }
 @@ -5669,3 +5809,14 @@ bool type_is_global_error_set(TypeTableEntry *err_set_type) {
     assert(err_set_type->data.error_set.infer_fn == nullptr);
     return err_set_type->data.error_set.err_count == UINT32_MAX;
 }
+uint32_t get_coro_frame_align_bytes(CodeGen *g) {
+    return g->pointer_size_bytes * 2;
+}
+bool fn_type_can_fail(FnTypeId *fn_type_id) {
+    TypeTableEntry *return_type = fn_type_id->return_type;
+    return return_type->id == TypeTableEntryIdErrorUnion || return_type->id == TypeTableEntryIdErrorSet ||
+        fn_type_id->cc == CallingConventionAsync;
+}

src/analyze.hpp+6-1

src/ast_render.cpp+37

...	@@ -244,6 +244,14 @@ static const char *node_type_str(NodeType node_type) {	...	@@ -244,6 +244,14 @@ static const char *node_type_str(NodeType node_type) {
244	return "TestExpr";	244	return "TestExpr";
245	case NodeTypeErrorSetDecl:	245	case NodeTypeErrorSetDecl:
246	return "ErrorSetDecl";	246	return "ErrorSetDecl";
		247	case NodeTypeCancel:
		248	return "Cancel";
		249	case NodeTypeResume:
		250	return "Resume";
		251	case NodeTypeAwaitExpr:
		252	return "AwaitExpr";
		253	case NodeTypeSuspend:
		254	return "Suspend";
247	}	255	}
248	zig_unreachable();	256	zig_unreachable();
249	}	257	}
...	@@ -1037,6 +1045,35 @@ static void render_node_extra(AstRender ar, AstNode node, bool grouped) {	...	@@ -1037,6 +1045,35 @@ static void render_node_extra(AstRender ar, AstNode node, bool grouped) {
1037	fprintf(ar->f, "}");	1045	fprintf(ar->f, "}");
1038	break;	1046	break;
1039	}	1047	}
		1048	case NodeTypeCancel:
		1049	{
		1050	fprintf(ar->f, "cancel ");
		1051	render_node_grouped(ar, node->data.cancel_expr.expr);
		1052	break;
		1053	}
		1054	case NodeTypeResume:
		1055	{
		1056	fprintf(ar->f, "resume ");
		1057	render_node_grouped(ar, node->data.resume_expr.expr);
		1058	break;
		1059	}
		1060	case NodeTypeAwaitExpr:
		1061	{
		1062	fprintf(ar->f, "await ");
		1063	render_node_grouped(ar, node->data.await_expr.expr);
		1064	break;
		1065	}
		1066	case NodeTypeSuspend:
		1067	{
		1068	fprintf(ar->f, "suspend");
		1069	if (node->data.suspend.block != nullptr) {
		1070	fprintf(ar->f, " \|");
		1071	render_node_grouped(ar, node->data.suspend.promise_symbol);
		1072	fprintf(ar->f, "\| ");
		1073	render_node_grouped(ar, node->data.suspend.block);
		1074	}
		1075	break;
		1076	}
1040	case NodeTypeFnDecl:	1077	case NodeTypeFnDecl:
1041	case NodeTypeParamDecl:	1078	case NodeTypeParamDecl:
1042	case NodeTypeTestDecl:	1079	case NodeTypeTestDecl:

src/codegen.cpp+572-17

...	@@ -381,6 +381,8 @@ static LLVMCallConv get_llvm_cc(CodeGen *g, CallingConvention cc) {	...	@@ -381,6 +381,8 @@ static LLVMCallConv get_llvm_cc(CodeGen *g, CallingConvention cc) {
381	} else {	381	} else {
382	return LLVMCCallConv;	382	return LLVMCCallConv;
383	}	383	}
		384	case CallingConventionAsync:
		385	return LLVMFastCallConv;
384	}	386	}
385	zig_unreachable();	387	zig_unreachable();
386	}	388	}
...	@@ -410,10 +412,10 @@ static uint32_t get_err_ret_trace_arg_index(CodeGen g, FnTableEntry fn_table_e	...	@@ -410,10 +412,10 @@ static uint32_t get_err_ret_trace_arg_index(CodeGen g, FnTableEntry fn_table_e
410	return UINT32_MAX;	412	return UINT32_MAX;
411	}	413	}
412	TypeTableEntry *fn_type = fn_table_entry->type_entry;	414	TypeTableEntry *fn_type = fn_table_entry->type_entry;
413	TypeTableEntry *return_type = fn_type->data.fn.fn_type_id.return_type;	415	if (!fn_type_can_fail(&fn_type->data.fn.fn_type_id)) {
414	if (return_type->id != TypeTableEntryIdErrorUnion && return_type->id != TypeTableEntryIdErrorSet) {
415	return UINT32_MAX;	416	return UINT32_MAX;
416	}	417	}
		418	TypeTableEntry *return_type = fn_type->data.fn.fn_type_id.return_type;
417	bool first_arg_ret = type_has_bits(return_type) && handle_is_ptr(return_type);	419	bool first_arg_ret = type_has_bits(return_type) && handle_is_ptr(return_type);
418	return first_arg_ret ? 1 : 0;	420	return first_arg_ret ? 1 : 0;
419	}	421	}
...	@@ -540,7 +542,7 @@ static LLVMValueRef fn_llvm_value(CodeGen g, FnTableEntry fn_table_entry) {	...	@@ -540,7 +542,7 @@ static LLVMValueRef fn_llvm_value(CodeGen g, FnTableEntry fn_table_entry) {
540		542
541	if (!type_has_bits(return_type)) {	543	if (!type_has_bits(return_type)) {
542	// nothing to do	544	// nothing to do
543	} else if (return_type->id == TypeTableEntryIdPointer \|\| return_type->id == TypeTableEntryIdFn) {	545	} else if (type_is_codegen_pointer(return_type)) {
544	addLLVMAttr(fn_table_entry->llvm_value, 0, "nonnull");	546	addLLVMAttr(fn_table_entry->llvm_value, 0, "nonnull");
545	} else if (handle_is_ptr(return_type) &&	547	} else if (handle_is_ptr(return_type) &&
546	calling_convention_does_first_arg_return(fn_type->data.fn.fn_type_id.cc))	548	calling_convention_does_first_arg_return(fn_type->data.fn.fn_type_id.cc))
...	@@ -925,6 +927,177 @@ static LLVMValueRef get_memcpy_fn_val(CodeGen *g) {	...	@@ -925,6 +927,177 @@ static LLVMValueRef get_memcpy_fn_val(CodeGen *g) {
925	return g->memcpy_fn_val;	927	return g->memcpy_fn_val;
926	}	928	}
927		929
		930	static LLVMValueRef get_coro_destroy_fn_val(CodeGen *g) {
		931	if (g->coro_destroy_fn_val)
		932	return g->coro_destroy_fn_val;
		933
		934	LLVMTypeRef param_types[] = {
		935	LLVMPointerType(LLVMInt8Type(), 0),
		936	};
		937	LLVMTypeRef fn_type = LLVMFunctionType(LLVMVoidType(), param_types, 1, false);
		938	Buf *name = buf_sprintf("llvm.coro.destroy");
		939	g->coro_destroy_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		940	assert(LLVMGetIntrinsicID(g->coro_destroy_fn_val));
		941
		942	return g->coro_destroy_fn_val;
		943	}
		944
		945	static LLVMValueRef get_coro_id_fn_val(CodeGen *g) {
		946	if (g->coro_id_fn_val)
		947	return g->coro_id_fn_val;
		948
		949	LLVMTypeRef param_types[] = {
		950	LLVMInt32Type(),
		951	LLVMPointerType(LLVMInt8Type(), 0),
		952	LLVMPointerType(LLVMInt8Type(), 0),
		953	LLVMPointerType(LLVMInt8Type(), 0),
		954	};
		955	LLVMTypeRef fn_type = LLVMFunctionType(ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()), param_types, 4, false);
		956	Buf *name = buf_sprintf("llvm.coro.id");
		957	g->coro_id_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		958	assert(LLVMGetIntrinsicID(g->coro_id_fn_val));
		959
		960	return g->coro_id_fn_val;
		961	}
		962
		963	static LLVMValueRef get_coro_alloc_fn_val(CodeGen *g) {
		964	if (g->coro_alloc_fn_val)
		965	return g->coro_alloc_fn_val;
		966
		967	LLVMTypeRef param_types[] = {
		968	ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()),
		969	};
		970	LLVMTypeRef fn_type = LLVMFunctionType(LLVMInt1Type(), param_types, 1, false);
		971	Buf *name = buf_sprintf("llvm.coro.alloc");
		972	g->coro_alloc_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		973	assert(LLVMGetIntrinsicID(g->coro_alloc_fn_val));
		974
		975	return g->coro_alloc_fn_val;
		976	}
		977
		978	static LLVMValueRef get_coro_size_fn_val(CodeGen *g) {
		979	if (g->coro_size_fn_val)
		980	return g->coro_size_fn_val;
		981
		982	LLVMTypeRef fn_type = LLVMFunctionType(g->builtin_types.entry_usize->type_ref, nullptr, 0, false);
		983	Buf name = buf_sprintf("llvm.coro.size.i%d", g->pointer_size_bytes 8);
		984	g->coro_size_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		985	assert(LLVMGetIntrinsicID(g->coro_size_fn_val));
		986
		987	return g->coro_size_fn_val;
		988	}
		989
		990	static LLVMValueRef get_coro_begin_fn_val(CodeGen *g) {
		991	if (g->coro_begin_fn_val)
		992	return g->coro_begin_fn_val;
		993
		994	LLVMTypeRef param_types[] = {
		995	ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()),
		996	LLVMPointerType(LLVMInt8Type(), 0),
		997	};
		998	LLVMTypeRef fn_type = LLVMFunctionType(LLVMPointerType(LLVMInt8Type(), 0), param_types, 2, false);
		999	Buf *name = buf_sprintf("llvm.coro.begin");
		1000	g->coro_begin_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1001	assert(LLVMGetIntrinsicID(g->coro_begin_fn_val));
		1002
		1003	return g->coro_begin_fn_val;
		1004	}
		1005
		1006	static LLVMValueRef get_coro_suspend_fn_val(CodeGen *g) {
		1007	if (g->coro_suspend_fn_val)
		1008	return g->coro_suspend_fn_val;
		1009
		1010	LLVMTypeRef param_types[] = {
		1011	ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()),
		1012	LLVMInt1Type(),
		1013	};
		1014	LLVMTypeRef fn_type = LLVMFunctionType(LLVMInt8Type(), param_types, 2, false);
		1015	Buf *name = buf_sprintf("llvm.coro.suspend");
		1016	g->coro_suspend_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1017	assert(LLVMGetIntrinsicID(g->coro_suspend_fn_val));
		1018
		1019	return g->coro_suspend_fn_val;
		1020	}
		1021
		1022	static LLVMValueRef get_coro_end_fn_val(CodeGen *g) {
		1023	if (g->coro_end_fn_val)
		1024	return g->coro_end_fn_val;
		1025
		1026	LLVMTypeRef param_types[] = {
		1027	LLVMPointerType(LLVMInt8Type(), 0),
		1028	LLVMInt1Type(),
		1029	};
		1030	LLVMTypeRef fn_type = LLVMFunctionType(LLVMInt1Type(), param_types, 2, false);
		1031	Buf *name = buf_sprintf("llvm.coro.end");
		1032	g->coro_end_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1033	assert(LLVMGetIntrinsicID(g->coro_end_fn_val));
		1034
		1035	return g->coro_end_fn_val;
		1036	}
		1037
		1038	static LLVMValueRef get_coro_free_fn_val(CodeGen *g) {
		1039	if (g->coro_free_fn_val)
		1040	return g->coro_free_fn_val;
		1041
		1042	LLVMTypeRef param_types[] = {
		1043	ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()),
		1044	LLVMPointerType(LLVMInt8Type(), 0),
		1045	};
		1046	LLVMTypeRef fn_type = LLVMFunctionType(LLVMPointerType(LLVMInt8Type(), 0), param_types, 2, false);
		1047	Buf *name = buf_sprintf("llvm.coro.free");
		1048	g->coro_free_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1049	assert(LLVMGetIntrinsicID(g->coro_free_fn_val));
		1050
		1051	return g->coro_free_fn_val;
		1052	}
		1053
		1054	static LLVMValueRef get_coro_resume_fn_val(CodeGen *g) {
		1055	if (g->coro_resume_fn_val)
		1056	return g->coro_resume_fn_val;
		1057
		1058	LLVMTypeRef param_types[] = {
		1059	LLVMPointerType(LLVMInt8Type(), 0),
		1060	};
		1061	LLVMTypeRef fn_type = LLVMFunctionType(LLVMVoidType(), param_types, 1, false);
		1062	Buf *name = buf_sprintf("llvm.coro.resume");
		1063	g->coro_resume_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1064	assert(LLVMGetIntrinsicID(g->coro_resume_fn_val));
		1065
		1066	return g->coro_resume_fn_val;
		1067	}
		1068
		1069	static LLVMValueRef get_coro_save_fn_val(CodeGen *g) {
		1070	if (g->coro_save_fn_val)
		1071	return g->coro_save_fn_val;
		1072
		1073	LLVMTypeRef param_types[] = {
		1074	LLVMPointerType(LLVMInt8Type(), 0),
		1075	};
		1076	LLVMTypeRef fn_type = LLVMFunctionType(ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()), param_types, 1, false);
		1077	Buf *name = buf_sprintf("llvm.coro.save");
		1078	g->coro_save_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1079	assert(LLVMGetIntrinsicID(g->coro_save_fn_val));
		1080
		1081	return g->coro_save_fn_val;
		1082	}
		1083
		1084	static LLVMValueRef get_coro_promise_fn_val(CodeGen *g) {
		1085	if (g->coro_promise_fn_val)
		1086	return g->coro_promise_fn_val;
		1087
		1088	LLVMTypeRef param_types[] = {
		1089	LLVMPointerType(LLVMInt8Type(), 0),
		1090	LLVMInt32Type(),
		1091	LLVMInt1Type(),
		1092	};
		1093	LLVMTypeRef fn_type = LLVMFunctionType(LLVMPointerType(LLVMInt8Type(), 0), param_types, 3, false);
		1094	Buf *name = buf_sprintf("llvm.coro.promise");
		1095	g->coro_promise_fn_val = LLVMAddFunction(g->module, buf_ptr(name), fn_type);
		1096	assert(LLVMGetIntrinsicID(g->coro_promise_fn_val));
		1097
		1098	return g->coro_promise_fn_val;
		1099	}
		1100
928	static LLVMValueRef get_return_address_fn_val(CodeGen *g) {	1101	static LLVMValueRef get_return_address_fn_val(CodeGen *g) {
929	if (g->return_address_fn_val)	1102	if (g->return_address_fn_val)
930	return g->return_address_fn_val;	1103	return g->return_address_fn_val;
...	@@ -2506,6 +2679,25 @@ static LLVMValueRef ir_render_elem_ptr(CodeGen g, IrExecutable executable, IrI	...	@@ -2506,6 +2679,25 @@ static LLVMValueRef ir_render_elem_ptr(CodeGen g, IrExecutable executable, IrI
2506	}	2679	}
2507	}	2680	}
2508		2681
		2682	static bool get_prefix_arg_err_ret_stack(CodeGen g, FnTypeId fn_type_id) {
		2683	return g->have_err_ret_tracing &&
		2684	(fn_type_id->return_type->id == TypeTableEntryIdErrorUnion \|\|
		2685	fn_type_id->return_type->id == TypeTableEntryIdErrorSet \|\|
		2686	fn_type_id->cc == CallingConventionAsync);
		2687	}
		2688
		2689	static size_t get_async_allocator_arg_index(CodeGen g, FnTypeId fn_type_id) {
		2690	// 0 1 2 3
		2691	// err_ret_stack allocator_ptr err_code other_args...
		2692	return get_prefix_arg_err_ret_stack(g, fn_type_id) ? 1 : 0;
		2693	}
		2694
		2695	static size_t get_async_err_code_arg_index(CodeGen g, FnTypeId fn_type_id) {
		2696	// 0 1 2 3
		2697	// err_ret_stack allocator_ptr err_code other_args...
		2698	return 1 + get_async_allocator_arg_index(g, fn_type_id);
		2699	}
		2700
2509	static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstructionCall *instruction) {	2701	static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstructionCall *instruction) {
2510	LLVMValueRef fn_val;	2702	LLVMValueRef fn_val;
2511	TypeTableEntry *fn_type;	2703	TypeTableEntry *fn_type;
...	@@ -2519,11 +2711,15 @@ static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstr	...	@@ -2519,11 +2711,15 @@ static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstr
2519	}	2711	}
2520		2712
2521	FnTypeId *fn_type_id = &fn_type->data.fn.fn_type_id;	2713	FnTypeId *fn_type_id = &fn_type->data.fn.fn_type_id;
		2714
2522	TypeTableEntry *src_return_type = fn_type_id->return_type;	2715	TypeTableEntry *src_return_type = fn_type_id->return_type;
2523	bool ret_has_bits = type_has_bits(src_return_type);	2716	bool ret_has_bits = type_has_bits(src_return_type);
2524	bool first_arg_ret = ret_has_bits && handle_is_ptr(src_return_type);	2717
2525	bool prefix_arg_err_ret_stack = g->have_err_ret_tracing && (src_return_type->id == TypeTableEntryIdErrorUnion \|\| src_return_type->id == TypeTableEntryIdErrorSet);	2718	bool first_arg_ret = ret_has_bits && handle_is_ptr(src_return_type) &&
2526	size_t actual_param_count = instruction->arg_count + (first_arg_ret ? 1 : 0) + (prefix_arg_err_ret_stack ? 1 : 0);	2719	calling_convention_does_first_arg_return(fn_type->data.fn.fn_type_id.cc);
		2720	bool prefix_arg_err_ret_stack = get_prefix_arg_err_ret_stack(g, fn_type_id);
		2721	// +2 for the async args
		2722	size_t actual_param_count = instruction->arg_count + (first_arg_ret ? 1 : 0) + (prefix_arg_err_ret_stack ? 1 : 0) + 2;
2527	bool is_var_args = fn_type_id->is_var_args;	2723	bool is_var_args = fn_type_id->is_var_args;
2528	LLVMValueRef *gen_param_values = allocate<LLVMValueRef>(actual_param_count);	2724	LLVMValueRef *gen_param_values = allocate<LLVMValueRef>(actual_param_count);
2529	size_t gen_param_index = 0;	2725	size_t gen_param_index = 0;
...	@@ -2535,6 +2731,14 @@ static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstr	...	@@ -2535,6 +2731,14 @@ static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstr
2535	gen_param_values[gen_param_index] = g->cur_err_ret_trace_val;	2731	gen_param_values[gen_param_index] = g->cur_err_ret_trace_val;
2536	gen_param_index += 1;	2732	gen_param_index += 1;
2537	}	2733	}
		2734	if (instruction->is_async) {
		2735	gen_param_values[gen_param_index] = ir_llvm_value(g, instruction->async_allocator);
		2736	gen_param_index += 1;
		2737
		2738	LLVMValueRef err_val_ptr = LLVMBuildStructGEP(g->builder, instruction->tmp_ptr, err_union_err_index, "");
		2739	gen_param_values[gen_param_index] = err_val_ptr;
		2740	gen_param_index += 1;
		2741	}
2538	for (size_t call_i = 0; call_i < instruction->arg_count; call_i += 1) {	2742	for (size_t call_i = 0; call_i < instruction->arg_count; call_i += 1) {
2539	IrInstruction *param_instruction = instruction->args[call_i];	2743	IrInstruction *param_instruction = instruction->args[call_i];
2540	TypeTableEntry *param_type = param_instruction->value.type;	2744	TypeTableEntry *param_type = param_instruction->value.type;
...	@@ -2572,6 +2776,12 @@ static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstr	...	@@ -2572,6 +2776,12 @@ static LLVMValueRef ir_render_call(CodeGen g, IrExecutable executable, IrInstr
2572	}	2776	}
2573	}	2777	}
2574		2778
		2779	if (instruction->is_async) {
		2780	LLVMValueRef payload_ptr = LLVMBuildStructGEP(g->builder, instruction->tmp_ptr, err_union_payload_index, "");
		2781	LLVMBuildStore(g->builder, result, payload_ptr);
		2782	return instruction->tmp_ptr;
		2783	}
		2784
2575	if (src_return_type->id == TypeTableEntryIdUnreachable) {	2785	if (src_return_type->id == TypeTableEntryIdUnreachable) {
2576	return LLVMBuildUnreachable(g->builder);	2786	return LLVMBuildUnreachable(g->builder);
2577	} else if (!ret_has_bits) {	2787	} else if (!ret_has_bits) {
...	@@ -2783,7 +2993,7 @@ static LLVMValueRef gen_non_null_bit(CodeGen g, TypeTableEntry maybe_type, LLV	...	@@ -2783,7 +2993,7 @@ static LLVMValueRef gen_non_null_bit(CodeGen g, TypeTableEntry maybe_type, LLV
2783	if (child_type->zero_bits) {	2993	if (child_type->zero_bits) {
2784	return maybe_handle;	2994	return maybe_handle;
2785	} else {	2995	} else {
2786	bool maybe_is_ptr = (child_type->id == TypeTableEntryIdPointer \|\| child_type->id == TypeTableEntryIdFn);	2996	bool maybe_is_ptr = type_is_codegen_pointer(child_type);
2787	if (maybe_is_ptr) {	2997	if (maybe_is_ptr) {
2788	return LLVMBuildICmp(g->builder, LLVMIntNE, maybe_handle, LLVMConstNull(maybe_type->type_ref), "");	2998	return LLVMBuildICmp(g->builder, LLVMIntNE, maybe_handle, LLVMConstNull(maybe_type->type_ref), "");
2789	} else {	2999	} else {
...	@@ -2823,7 +3033,7 @@ static LLVMValueRef ir_render_unwrap_maybe(CodeGen g, IrExecutable executable,	...	@@ -2823,7 +3033,7 @@ static LLVMValueRef ir_render_unwrap_maybe(CodeGen g, IrExecutable executable,
2823	if (child_type->zero_bits) {	3033	if (child_type->zero_bits) {
2824	return nullptr;	3034	return nullptr;
2825	} else {	3035	} else {
2826	bool maybe_is_ptr = (child_type->id == TypeTableEntryIdPointer \|\| child_type->id == TypeTableEntryIdFn);	3036	bool maybe_is_ptr = type_is_codegen_pointer(child_type);
2827	if (maybe_is_ptr) {	3037	if (maybe_is_ptr) {
2828	return maybe_ptr;	3038	return maybe_ptr;
2829	} else {	3039	} else {
...	@@ -3046,6 +3256,10 @@ static LLVMValueRef ir_render_align_cast(CodeGen g, IrExecutable executable, I	...	@@ -3046,6 +3256,10 @@ static LLVMValueRef ir_render_align_cast(CodeGen g, IrExecutable executable, I
3046	{	3256	{
3047	align_bytes = target_type->data.maybe.child_type->data.fn.fn_type_id.alignment;	3257	align_bytes = target_type->data.maybe.child_type->data.fn.fn_type_id.alignment;
3048	ptr_val = target_val;	3258	ptr_val = target_val;
		3259	} else if (target_type->id == TypeTableEntryIdMaybe &&
		3260	target_type->data.maybe.child_type->id == TypeTableEntryIdPromise)
		3261	{
		3262	zig_panic("TODO audit this function");
3049	} else if (target_type->id == TypeTableEntryIdStruct && target_type->data.structure.is_slice) {	3263	} else if (target_type->id == TypeTableEntryIdStruct && target_type->data.structure.is_slice) {
3050	TypeTableEntry *slice_ptr_type = target_type->data.structure.fields[slice_ptr_index].type_entry;	3264	TypeTableEntry *slice_ptr_type = target_type->data.structure.fields[slice_ptr_index].type_entry;
3051	align_bytes = slice_ptr_type->data.pointer.alignment;	3265	align_bytes = slice_ptr_type->data.pointer.alignment;
...	@@ -3088,6 +3302,20 @@ static LLVMValueRef ir_render_error_return_trace(CodeGen g, IrExecutable execu	...	@@ -3088,6 +3302,20 @@ static LLVMValueRef ir_render_error_return_trace(CodeGen g, IrExecutable execu
3088	return g->cur_err_ret_trace_val;	3302	return g->cur_err_ret_trace_val;
3089	}	3303	}
3090		3304
		3305	static LLVMValueRef ir_render_cancel(CodeGen g, IrExecutable executable, IrInstructionCancel *instruction) {
		3306	LLVMValueRef target_handle = ir_llvm_value(g, instruction->target);
		3307	LLVMBuildCall(g->builder, get_coro_destroy_fn_val(g), &target_handle, 1, "");
		3308	return nullptr;
		3309	}
		3310
		3311	static LLVMValueRef ir_render_get_implicit_allocator(CodeGen g, IrExecutable executable,
		3312	IrInstructionGetImplicitAllocator *instruction)
		3313	{
		3314	assert(instruction->id == ImplicitAllocatorIdArg);
		3315	size_t allocator_arg_index = get_async_allocator_arg_index(g, &g->cur_fn->type_entry->data.fn.fn_type_id);
		3316	return LLVMGetParam(g->cur_fn_val, allocator_arg_index);
		3317	}
		3318
3091	static LLVMAtomicOrdering to_LLVMAtomicOrdering(AtomicOrder atomic_order) {	3319	static LLVMAtomicOrdering to_LLVMAtomicOrdering(AtomicOrder atomic_order) {
3092	switch (atomic_order) {	3320	switch (atomic_order) {
3093	case AtomicOrderUnordered: return LLVMAtomicOrderingUnordered;	3321	case AtomicOrderUnordered: return LLVMAtomicOrderingUnordered;
...	@@ -3100,6 +3328,23 @@ static LLVMAtomicOrdering to_LLVMAtomicOrdering(AtomicOrder atomic_order) {	...	@@ -3100,6 +3328,23 @@ static LLVMAtomicOrdering to_LLVMAtomicOrdering(AtomicOrder atomic_order) {
3100	zig_unreachable();	3328	zig_unreachable();
3101	}	3329	}
3102		3330
		3331	static LLVMAtomicRMWBinOp to_LLVMAtomicRMWBinOp(AtomicRmwOp op, bool is_signed) {
		3332	switch (op) {
		3333	case AtomicRmwOp_xchg: return LLVMAtomicRMWBinOpXchg;
		3334	case AtomicRmwOp_add: return LLVMAtomicRMWBinOpAdd;
		3335	case AtomicRmwOp_sub: return LLVMAtomicRMWBinOpSub;
		3336	case AtomicRmwOp_and: return LLVMAtomicRMWBinOpAnd;
		3337	case AtomicRmwOp_nand: return LLVMAtomicRMWBinOpNand;
		3338	case AtomicRmwOp_or: return LLVMAtomicRMWBinOpOr;
		3339	case AtomicRmwOp_xor: return LLVMAtomicRMWBinOpXor;
		3340	case AtomicRmwOp_max:
		3341	return is_signed ? LLVMAtomicRMWBinOpMax : LLVMAtomicRMWBinOpUMax;
		3342	case AtomicRmwOp_min:
		3343	return is_signed ? LLVMAtomicRMWBinOpMin : LLVMAtomicRMWBinOpUMin;
		3344	}
		3345	zig_unreachable();
		3346	}
		3347
3103	static LLVMValueRef ir_render_cmpxchg(CodeGen g, IrExecutable executable, IrInstructionCmpxchg *instruction) {	3348	static LLVMValueRef ir_render_cmpxchg(CodeGen g, IrExecutable executable, IrInstructionCmpxchg *instruction) {
3104	LLVMValueRef ptr_val = ir_llvm_value(g, instruction->ptr);	3349	LLVMValueRef ptr_val = ir_llvm_value(g, instruction->ptr);
3105	LLVMValueRef cmp_val = ir_llvm_value(g, instruction->cmp_value);	3350	LLVMValueRef cmp_val = ir_llvm_value(g, instruction->cmp_value);
...	@@ -3508,9 +3753,7 @@ static LLVMValueRef ir_render_maybe_wrap(CodeGen g, IrExecutable executable, I	...	@@ -3508,9 +3753,7 @@ static LLVMValueRef ir_render_maybe_wrap(CodeGen g, IrExecutable executable, I
3508	}	3753	}
3509		3754
3510	LLVMValueRef payload_val = ir_llvm_value(g, instruction->value);	3755	LLVMValueRef payload_val = ir_llvm_value(g, instruction->value);
3511	if (child_type->id == TypeTableEntryIdPointer \|\|	3756	if (type_is_codegen_pointer(child_type)) {
3512	child_type->id == TypeTableEntryIdFn)
3513	{
3514	return payload_val;	3757	return payload_val;
3515	}	3758	}
3516		3759
...	@@ -3682,6 +3925,264 @@ static LLVMValueRef ir_render_panic(CodeGen g, IrExecutable executable, IrInst	...	@@ -3682,6 +3925,264 @@ static LLVMValueRef ir_render_panic(CodeGen g, IrExecutable executable, IrInst
3682	return nullptr;	3925	return nullptr;
3683	}	3926	}
3684		3927
		3928	static LLVMValueRef ir_render_coro_id(CodeGen g, IrExecutable executable, IrInstructionCoroId *instruction) {
		3929	LLVMValueRef promise_ptr = ir_llvm_value(g, instruction->promise_ptr);
		3930	LLVMValueRef align_val = LLVMConstInt(LLVMInt32Type(), get_coro_frame_align_bytes(g), false);
		3931	LLVMValueRef null = LLVMConstIntToPtr(LLVMConstNull(g->builtin_types.entry_usize->type_ref),
		3932	LLVMPointerType(LLVMInt8Type(), 0));
		3933	LLVMValueRef params[] = {
		3934	align_val,
		3935	promise_ptr,
		3936	null,
		3937	null,
		3938	};
		3939	return LLVMBuildCall(g->builder, get_coro_id_fn_val(g), params, 4, "");
		3940	}
		3941
		3942	static LLVMValueRef ir_render_coro_alloc(CodeGen g, IrExecutable executable, IrInstructionCoroAlloc *instruction) {
		3943	LLVMValueRef token = ir_llvm_value(g, instruction->coro_id);
		3944	return LLVMBuildCall(g->builder, get_coro_alloc_fn_val(g), &token, 1, "");
		3945	}
		3946
		3947	static LLVMValueRef ir_render_coro_size(CodeGen g, IrExecutable executable, IrInstructionCoroSize *instruction) {
		3948	return LLVMBuildCall(g->builder, get_coro_size_fn_val(g), nullptr, 0, "");
		3949	}
		3950
		3951	static LLVMValueRef ir_render_coro_begin(CodeGen g, IrExecutable executable, IrInstructionCoroBegin *instruction) {
		3952	LLVMValueRef coro_id = ir_llvm_value(g, instruction->coro_id);
		3953	LLVMValueRef coro_mem_ptr = ir_llvm_value(g, instruction->coro_mem_ptr);
		3954	LLVMValueRef params[] = {
		3955	coro_id,
		3956	coro_mem_ptr,
		3957	};
		3958	return LLVMBuildCall(g->builder, get_coro_begin_fn_val(g), params, 2, "");
		3959	}
		3960
		3961	static LLVMValueRef ir_render_coro_alloc_fail(CodeGen g, IrExecutable executable,
		3962	IrInstructionCoroAllocFail *instruction)
		3963	{
		3964	size_t err_code_ptr_arg_index = get_async_err_code_arg_index(g, &g->cur_fn->type_entry->data.fn.fn_type_id);
		3965	LLVMValueRef err_code_ptr_val = LLVMGetParam(g->cur_fn_val, err_code_ptr_arg_index);
		3966	LLVMValueRef err_code = ir_llvm_value(g, instruction->err_val);
		3967	LLVMBuildStore(g->builder, err_code, err_code_ptr_val);
		3968
		3969	LLVMValueRef return_value;
		3970	if (ir_want_runtime_safety(g, &instruction->base)) {
		3971	return_value = LLVMConstNull(LLVMPointerType(LLVMInt8Type(), 0));
		3972	} else {
		3973	return_value = LLVMGetUndef(LLVMPointerType(LLVMInt8Type(), 0));
		3974	}
		3975	LLVMBuildRet(g->builder, return_value);
		3976	return nullptr;
		3977	}
		3978
		3979	static LLVMValueRef ir_render_coro_suspend(CodeGen g, IrExecutable executable, IrInstructionCoroSuspend *instruction) {
		3980	LLVMValueRef save_point;
		3981	if (instruction->save_point == nullptr) {
		3982	save_point = LLVMConstNull(ZigLLVMTokenTypeInContext(LLVMGetGlobalContext()));
		3983	} else {
		3984	save_point = ir_llvm_value(g, instruction->save_point);
		3985	}
		3986	LLVMValueRef is_final = ir_llvm_value(g, instruction->is_final);
		3987	LLVMValueRef params[] = {
		3988	save_point,
		3989	is_final,
		3990	};
		3991	return LLVMBuildCall(g->builder, get_coro_suspend_fn_val(g), params, 2, "");
		3992	}
		3993
		3994	static LLVMValueRef ir_render_coro_end(CodeGen g, IrExecutable executable, IrInstructionCoroEnd *instruction) {
		3995	LLVMValueRef params[] = {
		3996	LLVMConstNull(LLVMPointerType(LLVMInt8Type(), 0)),
		3997	LLVMConstNull(LLVMInt1Type()),
		3998	};
		3999	return LLVMBuildCall(g->builder, get_coro_end_fn_val(g), params, 2, "");
		4000	}
		4001
		4002	static LLVMValueRef ir_render_coro_free(CodeGen g, IrExecutable executable, IrInstructionCoroFree *instruction) {
		4003	LLVMValueRef coro_id = ir_llvm_value(g, instruction->coro_id);
		4004	LLVMValueRef coro_handle = ir_llvm_value(g, instruction->coro_handle);
		4005	LLVMValueRef params[] = {
		4006	coro_id,
		4007	coro_handle,
		4008	};
		4009	return LLVMBuildCall(g->builder, get_coro_free_fn_val(g), params, 2, "");
		4010	}
		4011
		4012	static LLVMValueRef ir_render_coro_resume(CodeGen g, IrExecutable executable, IrInstructionCoroResume *instruction) {
		4013	LLVMValueRef awaiter_handle = ir_llvm_value(g, instruction->awaiter_handle);
		4014	return LLVMBuildCall(g->builder, get_coro_resume_fn_val(g), &awaiter_handle, 1, "");
		4015	}
		4016
		4017	static LLVMValueRef ir_render_coro_save(CodeGen g, IrExecutable executable, IrInstructionCoroSave *instruction) {
		4018	LLVMValueRef coro_handle = ir_llvm_value(g, instruction->coro_handle);
		4019	return LLVMBuildCall(g->builder, get_coro_save_fn_val(g), &coro_handle, 1, "");
		4020	}
		4021
		4022	static LLVMValueRef ir_render_coro_promise(CodeGen g, IrExecutable executable, IrInstructionCoroPromise *instruction) {
		4023	LLVMValueRef coro_handle = ir_llvm_value(g, instruction->coro_handle);
		4024	LLVMValueRef params[] = {
		4025	coro_handle,
		4026	LLVMConstInt(LLVMInt32Type(), get_coro_frame_align_bytes(g), false),
		4027	LLVMConstNull(LLVMInt1Type()),
		4028	};
		4029	LLVMValueRef uncasted_result = LLVMBuildCall(g->builder, get_coro_promise_fn_val(g), params, 3, "");
		4030	return LLVMBuildBitCast(g->builder, uncasted_result, instruction->base.value.type->type_ref, "");
		4031	}
		4032
		4033	static LLVMValueRef get_coro_alloc_helper_fn_val(CodeGen g, LLVMTypeRef alloc_fn_type_ref, TypeTableEntry fn_type) {
		4034	if (g->coro_alloc_helper_fn_val != nullptr)
		4035	return g->coro_alloc_helper_fn_val;
		4036
		4037	assert(fn_type->id == TypeTableEntryIdFn);
		4038
		4039	TypeTableEntry *ptr_to_err_code_type = get_pointer_to_type(g, g->builtin_types.entry_global_error_set, false);
		4040
		4041	LLVMTypeRef alloc_raw_fn_type_ref = LLVMGetElementType(alloc_fn_type_ref);
		4042	LLVMTypeRef *alloc_fn_arg_types = allocate<LLVMTypeRef>(LLVMCountParamTypes(alloc_raw_fn_type_ref));
		4043	LLVMGetParamTypes(alloc_raw_fn_type_ref, alloc_fn_arg_types);
		4044
		4045	ZigList<LLVMTypeRef> arg_types = {};
		4046	arg_types.append(alloc_fn_type_ref);
		4047	if (g->have_err_ret_tracing) {
		4048	arg_types.append(alloc_fn_arg_types[1]);
		4049	}
		4050	arg_types.append(alloc_fn_arg_types[g->have_err_ret_tracing ? 2 : 1]);
		4051	arg_types.append(ptr_to_err_code_type->type_ref);
		4052	arg_types.append(g->builtin_types.entry_usize->type_ref);
		4053
		4054	LLVMTypeRef fn_type_ref = LLVMFunctionType(LLVMPointerType(LLVMInt8Type(), 0),
		4055	arg_types.items, arg_types.length, false);
		4056
		4057	Buf *fn_name = get_mangled_name(g, buf_create_from_str("__zig_coro_alloc_helper"), false);
		4058	LLVMValueRef fn_val = LLVMAddFunction(g->module, buf_ptr(fn_name), fn_type_ref);
		4059	LLVMSetLinkage(fn_val, LLVMInternalLinkage);
		4060	LLVMSetFunctionCallConv(fn_val, get_llvm_cc(g, CallingConventionUnspecified));
		4061	addLLVMFnAttr(fn_val, "nounwind");
		4062	addLLVMArgAttr(fn_val, (unsigned)0, "nonnull");
		4063	addLLVMArgAttr(fn_val, (unsigned)1, "nonnull");
		4064
		4065	LLVMBasicBlockRef prev_block = LLVMGetInsertBlock(g->builder);
		4066	LLVMValueRef prev_debug_location = LLVMGetCurrentDebugLocation(g->builder);
		4067	FnTableEntry *prev_cur_fn = g->cur_fn;
		4068	LLVMValueRef prev_cur_fn_val = g->cur_fn_val;
		4069
		4070	LLVMBasicBlockRef entry_block = LLVMAppendBasicBlock(fn_val, "Entry");
		4071	LLVMPositionBuilderAtEnd(g->builder, entry_block);
		4072	ZigLLVMClearCurrentDebugLocation(g->builder);
		4073	g->cur_fn = nullptr;
		4074	g->cur_fn_val = fn_val;
		4075
		4076	LLVMValueRef sret_ptr = LLVMBuildAlloca(g->builder, LLVMGetElementType(alloc_fn_arg_types[0]), "");
		4077
		4078	size_t next_arg = 0;
		4079	LLVMValueRef alloc_fn_val = LLVMGetParam(fn_val, next_arg);
		4080	next_arg += 1;
		4081
		4082	LLVMValueRef stack_trace_val;
		4083	if (g->have_err_ret_tracing) {
		4084	stack_trace_val = LLVMGetParam(fn_val, next_arg);
		4085	next_arg += 1;
		4086	}
		4087
		4088	LLVMValueRef allocator_val = LLVMGetParam(fn_val, next_arg);
		4089	next_arg += 1;
		4090	LLVMValueRef err_code_ptr = LLVMGetParam(fn_val, next_arg);
		4091	next_arg += 1;
		4092	LLVMValueRef coro_size = LLVMGetParam(fn_val, next_arg);
		4093	next_arg += 1;
		4094	LLVMValueRef alignment_val = LLVMConstInt(g->builtin_types.entry_u29->type_ref,
		4095	get_coro_frame_align_bytes(g), false);
		4096
		4097	ZigList<LLVMValueRef> args = {};
		4098	args.append(sret_ptr);
		4099	if (g->have_err_ret_tracing) {
		4100	args.append(stack_trace_val);
		4101	}
		4102	args.append(allocator_val);
		4103	args.append(coro_size);
		4104	args.append(alignment_val);
		4105	ZigLLVMBuildCall(g->builder, alloc_fn_val, args.items, args.length,
		4106	get_llvm_cc(g, CallingConventionUnspecified), ZigLLVM_FnInlineAuto, "");
		4107	LLVMValueRef err_val_ptr = LLVMBuildStructGEP(g->builder, sret_ptr, err_union_err_index, "");
		4108	LLVMValueRef err_val = LLVMBuildLoad(g->builder, err_val_ptr, "");
		4109	LLVMBuildStore(g->builder, err_val, err_code_ptr);
		4110	LLVMValueRef ok_bit = LLVMBuildICmp(g->builder, LLVMIntEQ, err_val, LLVMConstNull(LLVMTypeOf(err_val)), "");
		4111	LLVMBasicBlockRef ok_block = LLVMAppendBasicBlock(fn_val, "AllocOk");
		4112	LLVMBasicBlockRef fail_block = LLVMAppendBasicBlock(fn_val, "AllocFail");
		4113	LLVMBuildCondBr(g->builder, ok_bit, ok_block, fail_block);
		4114
		4115	LLVMPositionBuilderAtEnd(g->builder, ok_block);
		4116	LLVMValueRef payload_ptr = LLVMBuildStructGEP(g->builder, sret_ptr, err_union_payload_index, "");
		4117	TypeTableEntry *u8_ptr_type = get_pointer_to_type(g, g->builtin_types.entry_u8, false);
		4118	TypeTableEntry *slice_type = get_slice_type(g, u8_ptr_type);
		4119	size_t ptr_field_index = slice_type->data.structure.fields[slice_ptr_index].gen_index;
		4120	LLVMValueRef ptr_field_ptr = LLVMBuildStructGEP(g->builder, payload_ptr, ptr_field_index, "");
		4121	LLVMValueRef ptr_val = LLVMBuildLoad(g->builder, ptr_field_ptr, "");
		4122	LLVMBuildRet(g->builder, ptr_val);
		4123
		4124	LLVMPositionBuilderAtEnd(g->builder, fail_block);
		4125	LLVMBuildRet(g->builder, LLVMConstNull(LLVMPointerType(LLVMInt8Type(), 0)));
		4126
		4127	g->cur_fn = prev_cur_fn;
		4128	g->cur_fn_val = prev_cur_fn_val;
		4129	LLVMPositionBuilderAtEnd(g->builder, prev_block);
		4130	LLVMSetCurrentDebugLocation(g->builder, prev_debug_location);
		4131
		4132	g->coro_alloc_helper_fn_val = fn_val;
		4133	return fn_val;
		4134	}
		4135
		4136	static LLVMValueRef ir_render_coro_alloc_helper(CodeGen g, IrExecutable executable,
		4137	IrInstructionCoroAllocHelper *instruction)
		4138	{
		4139	LLVMValueRef alloc_fn = ir_llvm_value(g, instruction->alloc_fn);
		4140	LLVMValueRef coro_size = ir_llvm_value(g, instruction->coro_size);
		4141	LLVMValueRef fn_val = get_coro_alloc_helper_fn_val(g, LLVMTypeOf(alloc_fn), instruction->alloc_fn->value.type);
		4142	size_t err_code_ptr_arg_index = get_async_err_code_arg_index(g, &g->cur_fn->type_entry->data.fn.fn_type_id);
		4143	size_t allocator_arg_index = get_async_allocator_arg_index(g, &g->cur_fn->type_entry->data.fn.fn_type_id);
		4144
		4145	ZigList<LLVMValueRef> params = {};
		4146	params.append(alloc_fn);
		4147	uint32_t err_ret_trace_arg_index = get_err_ret_trace_arg_index(g, g->cur_fn);
		4148	if (err_ret_trace_arg_index != UINT32_MAX) {
		4149	params.append(LLVMGetParam(g->cur_fn_val, err_ret_trace_arg_index));
		4150	}
		4151	params.append(LLVMGetParam(g->cur_fn_val, allocator_arg_index));
		4152	params.append(LLVMGetParam(g->cur_fn_val, err_code_ptr_arg_index));
		4153	params.append(coro_size);
		4154
		4155	return ZigLLVMBuildCall(g->builder, fn_val, params.items, params.length,
		4156	get_llvm_cc(g, CallingConventionUnspecified), ZigLLVM_FnInlineAuto, "");
		4157	}
		4158
		4159	static LLVMValueRef ir_render_atomic_rmw(CodeGen g, IrExecutable executable,
		4160	IrInstructionAtomicRmw *instruction)
		4161	{
		4162	bool is_signed;
		4163	TypeTableEntry *operand_type = instruction->operand->value.type;
		4164	if (operand_type->id == TypeTableEntryIdInt) {
		4165	is_signed = operand_type->data.integral.is_signed;
		4166	} else {
		4167	is_signed = false;
		4168	}
		4169	LLVMAtomicRMWBinOp op = to_LLVMAtomicRMWBinOp(instruction->resolved_op, is_signed);
		4170	LLVMAtomicOrdering ordering = to_LLVMAtomicOrdering(instruction->resolved_ordering);
		4171	LLVMValueRef ptr = ir_llvm_value(g, instruction->ptr);
		4172	LLVMValueRef operand = ir_llvm_value(g, instruction->operand);
		4173
		4174	if (get_codegen_ptr_type(operand_type) == nullptr) {
		4175	return LLVMBuildAtomicRMW(g->builder, op, ptr, operand, ordering, false);
		4176	}
		4177
		4178	// it's a pointer but we need to treat it as an int
		4179	LLVMValueRef casted_ptr = LLVMBuildBitCast(g->builder, ptr,
		4180	LLVMPointerType(g->builtin_types.entry_usize->type_ref, 0), "");
		4181	LLVMValueRef casted_operand = LLVMBuildPtrToInt(g->builder, operand, g->builtin_types.entry_usize->type_ref, "");
		4182	LLVMValueRef uncasted_result = LLVMBuildAtomicRMW(g->builder, op, casted_ptr, casted_operand, ordering, false);
		4183	return LLVMBuildIntToPtr(g->builder, uncasted_result, operand_type->type_ref, "");
		4184	}
		4185
3685	static void set_debug_location(CodeGen g, IrInstruction instruction) {	4186	static void set_debug_location(CodeGen g, IrInstruction instruction) {
3686	AstNode *source_node = instruction->source_node;	4187	AstNode *source_node = instruction->source_node;
3687	Scope *scope = instruction->scope;	4188	Scope *scope = instruction->scope;
...	@@ -3745,7 +4246,9 @@ static LLVMValueRef ir_render_instruction(CodeGen g, IrExecutable executable,	...	@@ -3745,7 +4246,9 @@ static LLVMValueRef ir_render_instruction(CodeGen g, IrExecutable executable,
3745	case IrInstructionIdTagType:	4246	case IrInstructionIdTagType:
3746	case IrInstructionIdExport:	4247	case IrInstructionIdExport:
3747	case IrInstructionIdErrorUnion:	4248	case IrInstructionIdErrorUnion:
		4249	case IrInstructionIdPromiseResultType:
3748	zig_unreachable();	4250	zig_unreachable();
		4251
3749	case IrInstructionIdReturn:	4252	case IrInstructionIdReturn:
3750	return ir_render_return(g, executable, (IrInstructionReturn *)instruction);	4253	return ir_render_return(g, executable, (IrInstructionReturn *)instruction);
3751	case IrInstructionIdDeclVar:	4254	case IrInstructionIdDeclVar:
...	@@ -3862,12 +4365,43 @@ static LLVMValueRef ir_render_instruction(CodeGen g, IrExecutable executable,	...	@@ -3862,12 +4365,43 @@ static LLVMValueRef ir_render_instruction(CodeGen g, IrExecutable executable,
3862	return ir_render_align_cast(g, executable, (IrInstructionAlignCast *)instruction);	4365	return ir_render_align_cast(g, executable, (IrInstructionAlignCast *)instruction);
3863	case IrInstructionIdErrorReturnTrace:	4366	case IrInstructionIdErrorReturnTrace:
3864	return ir_render_error_return_trace(g, executable, (IrInstructionErrorReturnTrace *)instruction);	4367	return ir_render_error_return_trace(g, executable, (IrInstructionErrorReturnTrace *)instruction);
		4368	case IrInstructionIdCancel:
		4369	return ir_render_cancel(g, executable, (IrInstructionCancel *)instruction);
		4370	case IrInstructionIdGetImplicitAllocator:
		4371	return ir_render_get_implicit_allocator(g, executable, (IrInstructionGetImplicitAllocator *)instruction);
		4372	case IrInstructionIdCoroId:
		4373	return ir_render_coro_id(g, executable, (IrInstructionCoroId *)instruction);
		4374	case IrInstructionIdCoroAlloc:
		4375	return ir_render_coro_alloc(g, executable, (IrInstructionCoroAlloc *)instruction);
		4376	case IrInstructionIdCoroSize:
		4377	return ir_render_coro_size(g, executable, (IrInstructionCoroSize *)instruction);
		4378	case IrInstructionIdCoroBegin:
		4379	return ir_render_coro_begin(g, executable, (IrInstructionCoroBegin *)instruction);
		4380	case IrInstructionIdCoroAllocFail:
		4381	return ir_render_coro_alloc_fail(g, executable, (IrInstructionCoroAllocFail *)instruction);
		4382	case IrInstructionIdCoroSuspend:
		4383	return ir_render_coro_suspend(g, executable, (IrInstructionCoroSuspend *)instruction);
		4384	case IrInstructionIdCoroEnd:
		4385	return ir_render_coro_end(g, executable, (IrInstructionCoroEnd *)instruction);
		4386	case IrInstructionIdCoroFree:
		4387	return ir_render_coro_free(g, executable, (IrInstructionCoroFree *)instruction);
		4388	case IrInstructionIdCoroResume:
		4389	return ir_render_coro_resume(g, executable, (IrInstructionCoroResume *)instruction);
		4390	case IrInstructionIdCoroSave:
		4391	return ir_render_coro_save(g, executable, (IrInstructionCoroSave *)instruction);
		4392	case IrInstructionIdCoroPromise:
		4393	return ir_render_coro_promise(g, executable, (IrInstructionCoroPromise *)instruction);
		4394	case IrInstructionIdCoroAllocHelper:
		4395	return ir_render_coro_alloc_helper(g, executable, (IrInstructionCoroAllocHelper *)instruction);
		4396	case IrInstructionIdAtomicRmw:
		4397	return ir_render_atomic_rmw(g, executable, (IrInstructionAtomicRmw *)instruction);
3865	}	4398	}
3866	zig_unreachable();	4399	zig_unreachable();
3867	}	4400	}
3868		4401
3869	static void ir_render(CodeGen g, FnTableEntry fn_entry) {	4402	static void ir_render(CodeGen g, FnTableEntry fn_entry) {
3870	assert(fn_entry);	4403	assert(fn_entry);
		4404
3871	IrExecutable *executable = &fn_entry->analyzed_executable;	4405	IrExecutable *executable = &fn_entry->analyzed_executable;
3872	assert(executable->basic_block_list.length > 0);	4406	assert(executable->basic_block_list.length > 0);
3873	for (size_t block_i = 0; block_i < executable->basic_block_list.length; block_i += 1) {	4407	for (size_t block_i = 0; block_i < executable->basic_block_list.length; block_i += 1) {
...	@@ -4009,6 +4543,7 @@ static LLVMValueRef pack_const_int(CodeGen *g, LLVMTypeRef big_int_type_ref, Con	...	@@ -4009,6 +4543,7 @@ static LLVMValueRef pack_const_int(CodeGen *g, LLVMTypeRef big_int_type_ref, Con
4009	case TypeTableEntryIdPointer:	4543	case TypeTableEntryIdPointer:
4010	case TypeTableEntryIdFn:	4544	case TypeTableEntryIdFn:
4011	case TypeTableEntryIdMaybe:	4545	case TypeTableEntryIdMaybe:
		4546	case TypeTableEntryIdPromise:
4012	{	4547	{
4013	LLVMValueRef ptr_val = gen_const_val(g, const_val, "");	4548	LLVMValueRef ptr_val = gen_const_val(g, const_val, "");
4014	LLVMValueRef ptr_size_int_val = LLVMConstPtrToInt(ptr_val, g->builtin_types.entry_usize->type_ref);	4549	LLVMValueRef ptr_size_int_val = LLVMConstPtrToInt(ptr_val, g->builtin_types.entry_usize->type_ref);
...	@@ -4104,9 +4639,7 @@ static LLVMValueRef gen_const_val(CodeGen g, ConstExprValue const_val, const c	...	@@ -4104,9 +4639,7 @@ static LLVMValueRef gen_const_val(CodeGen g, ConstExprValue const_val, const c
4104	TypeTableEntry *child_type = type_entry->data.maybe.child_type;	4639	TypeTableEntry *child_type = type_entry->data.maybe.child_type;
4105	if (child_type->zero_bits) {	4640	if (child_type->zero_bits) {
4106	return LLVMConstInt(LLVMInt1Type(), const_val->data.x_maybe ? 1 : 0, false);	4641	return LLVMConstInt(LLVMInt1Type(), const_val->data.x_maybe ? 1 : 0, false);
4107	} else if (child_type->id == TypeTableEntryIdPointer \|\|	4642	} else if (type_is_codegen_pointer(child_type)) {
4108	child_type->id == TypeTableEntryIdFn)
4109	{
4110	if (const_val->data.x_maybe) {	4643	if (const_val->data.x_maybe) {
4111	return gen_const_val(g, const_val->data.x_maybe, "");	4644	return gen_const_val(g, const_val->data.x_maybe, "");
4112	} else {	4645	} else {
...	@@ -4426,6 +4959,7 @@ static LLVMValueRef gen_const_val(CodeGen g, ConstExprValue const_val, const c	...	@@ -4426,6 +4959,7 @@ static LLVMValueRef gen_const_val(CodeGen g, ConstExprValue const_val, const c
4426	case TypeTableEntryIdVar:	4959	case TypeTableEntryIdVar:
4427	case TypeTableEntryIdArgTuple:	4960	case TypeTableEntryIdArgTuple:
4428	case TypeTableEntryIdOpaque:	4961	case TypeTableEntryIdOpaque:
		4962	case TypeTableEntryIdPromise:
4429	zig_unreachable();	4963	zig_unreachable();
4430		4964
4431	}	4965	}
...	@@ -5235,6 +5769,7 @@ static void define_builtin_types(CodeGen *g) {	...	@@ -5235,6 +5769,7 @@ static void define_builtin_types(CodeGen *g) {
5235		5769
5236	g->builtin_types.entry_u8 = get_int_type(g, false, 8);	5770	g->builtin_types.entry_u8 = get_int_type(g, false, 8);
5237	g->builtin_types.entry_u16 = get_int_type(g, false, 16);	5771	g->builtin_types.entry_u16 = get_int_type(g, false, 16);
		5772	g->builtin_types.entry_u29 = get_int_type(g, false, 29);
5238	g->builtin_types.entry_u32 = get_int_type(g, false, 32);	5773	g->builtin_types.entry_u32 = get_int_type(g, false, 32);
5239	g->builtin_types.entry_u64 = get_int_type(g, false, 64);	5774	g->builtin_types.entry_u64 = get_int_type(g, false, 64);
5240	g->builtin_types.entry_u128 = get_int_type(g, false, 128);	5775	g->builtin_types.entry_u128 = get_int_type(g, false, 128);
...	@@ -5271,6 +5806,10 @@ static void define_builtin_types(CodeGen *g) {	...	@@ -5271,6 +5806,10 @@ static void define_builtin_types(CodeGen *g) {
5271		5806
5272	g->primitive_type_table.put(&entry->name, entry);	5807	g->primitive_type_table.put(&entry->name, entry);
5273	}	5808	}
		5809	{
		5810	TypeTableEntry *entry = get_promise_type(g, nullptr);
		5811	g->primitive_type_table.put(&entry->name, entry);
		5812	}
5274		5813
5275	}	5814	}
5276		5815
...	@@ -5348,6 +5887,7 @@ static void define_builtin_fns(CodeGen *g) {	...	@@ -5348,6 +5887,7 @@ static void define_builtin_fns(CodeGen *g) {
5348	create_builtin_fn(g, BuiltinFnIdArgType, "ArgType", 2);	5887	create_builtin_fn(g, BuiltinFnIdArgType, "ArgType", 2);
5349	create_builtin_fn(g, BuiltinFnIdExport, "export", 3);	5888	create_builtin_fn(g, BuiltinFnIdExport, "export", 3);
5350	create_builtin_fn(g, BuiltinFnIdErrorReturnTrace, "errorReturnTrace", 0);	5889	create_builtin_fn(g, BuiltinFnIdErrorReturnTrace, "errorReturnTrace", 0);
		5890	create_builtin_fn(g, BuiltinFnIdAtomicRmw, "atomicRmw", 5);
5351	}	5891	}
5352		5892
5353	static const char *bool_to_str(bool b) {	5893	static const char *bool_to_str(bool b) {
...	@@ -5477,6 +6017,20 @@ static void define_builtin_compile_vars(CodeGen *g) {	...	@@ -5477,6 +6017,20 @@ static void define_builtin_compile_vars(CodeGen *g) {
5477	" SeqCst,\n"	6017	" SeqCst,\n"
5478	"};\n\n");	6018	"};\n\n");
5479	}	6019	}
		6020	{
		6021	buf_appendf(contents,
		6022	"pub const AtomicRmwOp = enum {\n"
		6023	" Xchg,\n"
		6024	" Add,\n"
		6025	" Sub,\n"
		6026	" And,\n"
		6027	" Nand,\n"
		6028	" Or,\n"
		6029	" Xor,\n"
		6030	" Max,\n"
		6031	" Min,\n"
		6032	"};\n\n");
		6033	}
5480	{	6034	{
5481	buf_appendf(contents,	6035	buf_appendf(contents,
5482	"pub const Mode = enum {\n"	6036	"pub const Mode = enum {\n"
...	@@ -5898,6 +6452,7 @@ static void prepend_c_type_to_decl_list(CodeGen g, GenH gen_h, TypeTableEntry	...	@@ -5898,6 +6452,7 @@ static void prepend_c_type_to_decl_list(CodeGen g, GenH gen_h, TypeTableEntry
5898	case TypeTableEntryIdArgTuple:	6452	case TypeTableEntryIdArgTuple:
5899	case TypeTableEntryIdErrorUnion:	6453	case TypeTableEntryIdErrorUnion:
5900	case TypeTableEntryIdErrorSet:	6454	case TypeTableEntryIdErrorSet:
		6455	case TypeTableEntryIdPromise:
5901	zig_unreachable();	6456	zig_unreachable();
5902	case TypeTableEntryIdVoid:	6457	case TypeTableEntryIdVoid:
5903	case TypeTableEntryIdUnreachable:	6458	case TypeTableEntryIdUnreachable:
...	@@ -6027,9 +6582,7 @@ static void get_c_type(CodeGen g, GenH gen_h, TypeTableEntry *type_entry, Buf	...	@@ -6027,9 +6582,7 @@ static void get_c_type(CodeGen g, GenH gen_h, TypeTableEntry *type_entry, Buf
6027	if (child_type->zero_bits) {	6582	if (child_type->zero_bits) {
6028	buf_init_from_str(out_buf, "bool");	6583	buf_init_from_str(out_buf, "bool");
6029	return;	6584	return;
6030	} else if (child_type->id == TypeTableEntryIdPointer \|\|	6585	} else if (type_is_codegen_pointer(child_type)) {
6031	child_type->id == TypeTableEntryIdFn)
6032	{
6033	return get_c_type(g, gen_h, child_type, out_buf);	6586	return get_c_type(g, gen_h, child_type, out_buf);
6034	} else {	6587	} else {
6035	zig_unreachable();	6588	zig_unreachable();
...	@@ -6084,6 +6637,7 @@ static void get_c_type(CodeGen g, GenH gen_h, TypeTableEntry *type_entry, Buf	...	@@ -6084,6 +6637,7 @@ static void get_c_type(CodeGen g, GenH gen_h, TypeTableEntry *type_entry, Buf
6084	case TypeTableEntryIdNullLit:	6637	case TypeTableEntryIdNullLit:
6085	case TypeTableEntryIdVar:	6638	case TypeTableEntryIdVar:
6086	case TypeTableEntryIdArgTuple:	6639	case TypeTableEntryIdArgTuple:
		6640	case TypeTableEntryIdPromise:
6087	zig_unreachable();	6641	zig_unreachable();
6088	}	6642	}
6089	}	6643	}
...	@@ -6244,6 +6798,7 @@ static void gen_h_file(CodeGen *g) {	...	@@ -6244,6 +6798,7 @@ static void gen_h_file(CodeGen *g) {
6244	case TypeTableEntryIdArgTuple:	6798	case TypeTableEntryIdArgTuple:
6245	case TypeTableEntryIdMaybe:	6799	case TypeTableEntryIdMaybe:
6246	case TypeTableEntryIdFn:	6800	case TypeTableEntryIdFn:
		6801	case TypeTableEntryIdPromise:
6247	zig_unreachable();	6802	zig_unreachable();
6248	case TypeTableEntryIdEnum:	6803	case TypeTableEntryIdEnum:
6249	assert(type_entry->data.enumeration.layout == ContainerLayoutExtern);	6804	assert(type_entry->data.enumeration.layout == ContainerLayoutExtern);

src/ir.cpp+1336-85

...	@@ -65,6 +65,7 @@ enum ConstCastResultId {	...	@@ -65,6 +65,7 @@ enum ConstCastResultId {
65	ConstCastResultIdFnArgNoAlias,	65	ConstCastResultIdFnArgNoAlias,
66	ConstCastResultIdType,	66	ConstCastResultIdType,
67	ConstCastResultIdUnresolvedInferredErrSet,	67	ConstCastResultIdUnresolvedInferredErrSet,
		68	ConstCastResultIdAsyncAllocatorType,
68	};	69	};
69		70
70	struct ConstCastErrSetMismatch {	71	struct ConstCastErrSetMismatch {
...	@@ -92,6 +93,7 @@ struct ConstCastOnly {	...	@@ -92,6 +93,7 @@ struct ConstCastOnly {
92	ConstCastOnly *error_union_payload;	93	ConstCastOnly *error_union_payload;
93	ConstCastOnly *error_union_error_set;	94	ConstCastOnly *error_union_error_set;
94	ConstCastOnly *return_type;	95	ConstCastOnly *return_type;
		96	ConstCastOnly *async_allocator_type;
95	ConstCastArg fn_arg;	97	ConstCastArg fn_arg;
96	ConstCastArgNoAlias arg_no_alias;	98	ConstCastArgNoAlias arg_no_alias;
97	} data;	99	} data;
...	@@ -104,6 +106,10 @@ static TypeTableEntry ir_analyze_instruction(IrAnalyze ira, IrInstruction *ins	...	@@ -104,6 +106,10 @@ static TypeTableEntry ir_analyze_instruction(IrAnalyze ira, IrInstruction *ins
104	static IrInstruction ir_implicit_cast(IrAnalyze ira, IrInstruction value, TypeTableEntry expected_type);	106	static IrInstruction ir_implicit_cast(IrAnalyze ira, IrInstruction value, TypeTableEntry expected_type);
105	static IrInstruction ir_get_deref(IrAnalyze ira, IrInstruction source_instruction, IrInstruction ptr);	107	static IrInstruction ir_get_deref(IrAnalyze ira, IrInstruction source_instruction, IrInstruction ptr);
106	static ErrorMsg exec_add_error_node(CodeGen codegen, IrExecutable exec, AstNode source_node, Buf *msg);	108	static ErrorMsg exec_add_error_node(CodeGen codegen, IrExecutable exec, AstNode source_node, Buf *msg);
		109	static IrInstruction ir_analyze_container_field_ptr(IrAnalyze ira, Buf *field_name,
		110	IrInstruction source_instr, IrInstruction container_ptr, TypeTableEntry *container_type);
		111	static IrInstruction ir_get_var_ptr(IrAnalyze ira, IrInstruction *instruction,
		112	VariableTableEntry *var, bool is_const_ptr, bool is_volatile_ptr);
107		113
108	ConstExprValue const_ptr_pointee(CodeGen g, ConstExprValue *const_val) {	114	ConstExprValue const_ptr_pointee(CodeGen g, ConstExprValue *const_val) {
109	assert(const_val->type->id == TypeTableEntryIdPointer);	115	assert(const_val->type->id == TypeTableEntryIdPointer);
...	@@ -637,6 +643,70 @@ static constexpr IrInstructionId ir_instruction_id(IrInstructionErrorUnion *) {	...	@@ -637,6 +643,70 @@ static constexpr IrInstructionId ir_instruction_id(IrInstructionErrorUnion *) {
637	return IrInstructionIdErrorUnion;	643	return IrInstructionIdErrorUnion;
638	}	644	}
639		645
		646	static constexpr IrInstructionId ir_instruction_id(IrInstructionCancel *) {
		647	return IrInstructionIdCancel;
		648	}
		649
		650	static constexpr IrInstructionId ir_instruction_id(IrInstructionGetImplicitAllocator *) {
		651	return IrInstructionIdGetImplicitAllocator;
		652	}
		653
		654	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroId *) {
		655	return IrInstructionIdCoroId;
		656	}
		657
		658	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroAlloc *) {
		659	return IrInstructionIdCoroAlloc;
		660	}
		661
		662	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroSize *) {
		663	return IrInstructionIdCoroSize;
		664	}
		665
		666	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroBegin *) {
		667	return IrInstructionIdCoroBegin;
		668	}
		669
		670	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroAllocFail *) {
		671	return IrInstructionIdCoroAllocFail;
		672	}
		673
		674	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroSuspend *) {
		675	return IrInstructionIdCoroSuspend;
		676	}
		677
		678	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroEnd *) {
		679	return IrInstructionIdCoroEnd;
		680	}
		681
		682	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroFree *) {
		683	return IrInstructionIdCoroFree;
		684	}
		685
		686	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroResume *) {
		687	return IrInstructionIdCoroResume;
		688	}
		689
		690	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroSave *) {
		691	return IrInstructionIdCoroSave;
		692	}
		693
		694	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroPromise *) {
		695	return IrInstructionIdCoroPromise;
		696	}
		697
		698	static constexpr IrInstructionId ir_instruction_id(IrInstructionCoroAllocHelper *) {
		699	return IrInstructionIdCoroAllocHelper;
		700	}
		701
		702	static constexpr IrInstructionId ir_instruction_id(IrInstructionAtomicRmw *) {
		703	return IrInstructionIdAtomicRmw;
		704	}
		705
		706	static constexpr IrInstructionId ir_instruction_id(IrInstructionPromiseResultType *) {
		707	return IrInstructionIdPromiseResultType;
		708	}
		709
640	template<typename T>	710	template<typename T>
641	static T ir_create_instruction(IrBuilder irb, Scope scope, AstNode source_node) {	711	static T ir_create_instruction(IrBuilder irb, Scope scope, AstNode source_node) {
642	T *special_instruction = allocate<T>(1);	712	T *special_instruction = allocate<T>(1);
...	@@ -708,14 +778,6 @@ static IrInstruction ir_build_return(IrBuilder irb, Scope scope, AstNode sou	...	@@ -708,14 +778,6 @@ static IrInstruction ir_build_return(IrBuilder irb, Scope scope, AstNode sou
708	return &return_instruction->base;	778	return &return_instruction->base;
709	}	779	}
710		780
711	static IrInstruction ir_build_return_from(IrBuilder irb, IrInstruction *old_instruction,
712	IrInstruction *return_value)
713	{
714	IrInstruction *new_instruction = ir_build_return(irb, old_instruction->scope, old_instruction->source_node, return_value);
715	ir_link_new_instruction(new_instruction, old_instruction);
716	return new_instruction;
717	}
718
719	static IrInstruction ir_create_const(IrBuilder irb, Scope scope, AstNode source_node,	781	static IrInstruction ir_create_const(IrBuilder irb, Scope scope, AstNode source_node,
720	TypeTableEntry *type_entry)	782	TypeTableEntry *type_entry)
721	{	783	{
...	@@ -779,6 +841,14 @@ static IrInstruction ir_build_const_usize(IrBuilder irb, Scope *scope, AstNode	...	@@ -779,6 +841,14 @@ static IrInstruction ir_build_const_usize(IrBuilder irb, Scope *scope, AstNode
779	return &const_instruction->base;	841	return &const_instruction->base;
780	}	842	}
781		843
		844	static IrInstruction ir_build_const_u8(IrBuilder irb, Scope scope, AstNode source_node, uint8_t value) {
		845	IrInstructionConst *const_instruction = ir_build_instruction<IrInstructionConst>(irb, scope, source_node);
		846	const_instruction->base.value.type = irb->codegen->builtin_types.entry_u8;
		847	const_instruction->base.value.special = ConstValSpecialStatic;
		848	bigint_init_unsigned(&const_instruction->base.value.data.x_bigint, value);
		849	return &const_instruction->base;
		850	}
		851
782	static IrInstruction ir_create_const_type(IrBuilder irb, Scope scope, AstNode source_node,	852	static IrInstruction ir_create_const_type(IrBuilder irb, Scope scope, AstNode source_node,
783	TypeTableEntry *type_entry)	853	TypeTableEntry *type_entry)
784	{	854	{
...	@@ -866,6 +936,27 @@ static IrInstruction ir_build_const_c_str_lit(IrBuilder irb, Scope *scope, Ast	...	@@ -866,6 +936,27 @@ static IrInstruction ir_build_const_c_str_lit(IrBuilder irb, Scope *scope, Ast
866	return &const_instruction->base;	936	return &const_instruction->base;
867	}	937	}
868		938
		939	static IrInstruction ir_build_const_promise_init(IrBuilder irb, Scope scope, AstNode source_node,
		940	TypeTableEntry *return_type)
		941	{
		942	TypeTableEntry *struct_type = get_promise_frame_type(irb->codegen, return_type);
		943
		944	IrInstructionConst *const_instruction = ir_build_instruction<IrInstructionConst>(irb, scope, source_node);
		945	const_instruction->base.value.type = struct_type;
		946	const_instruction->base.value.special = ConstValSpecialStatic;
		947	const_instruction->base.value.data.x_struct.fields = allocate<ConstExprValue>(struct_type->data.structure.src_field_count);
		948	const_instruction->base.value.data.x_struct.fields[0].type = struct_type->data.structure.fields[0].type_entry;
		949	const_instruction->base.value.data.x_struct.fields[0].special = ConstValSpecialStatic;
		950	const_instruction->base.value.data.x_struct.fields[0].data.x_maybe = nullptr;
		951	if (struct_type->data.structure.src_field_count > 1) {
		952	const_instruction->base.value.data.x_struct.fields[1].type = return_type;
		953	const_instruction->base.value.data.x_struct.fields[1].special = ConstValSpecialUndef;
		954	const_instruction->base.value.data.x_struct.fields[2].type = struct_type->data.structure.fields[2].type_entry;
		955	const_instruction->base.value.data.x_struct.fields[2].special = ConstValSpecialUndef;
		956	}
		957	return &const_instruction->base;
		958	}
		959
869	static IrInstruction ir_build_bin_op(IrBuilder irb, Scope scope, AstNode source_node, IrBinOp op_id,	960	static IrInstruction ir_build_bin_op(IrBuilder irb, Scope scope, AstNode source_node, IrBinOp op_id,
870	IrInstruction op1, IrInstruction op2, bool safety_check_on)	961	IrInstruction op1, IrInstruction op2, bool safety_check_on)
871	{	962	{
...	@@ -950,15 +1041,6 @@ static IrInstruction ir_build_struct_field_ptr(IrBuilder irb, Scope *scope, As	...	@@ -950,15 +1041,6 @@ static IrInstruction ir_build_struct_field_ptr(IrBuilder irb, Scope *scope, As
950	return &instruction->base;	1041	return &instruction->base;
951	}	1042	}
952		1043
953	static IrInstruction ir_build_struct_field_ptr_from(IrBuilder irb, IrInstruction *old_instruction,
954	IrInstruction struct_ptr, TypeStructField type_struct_field)
955	{
956	IrInstruction *new_instruction = ir_build_struct_field_ptr(irb, old_instruction->scope,
957	old_instruction->source_node, struct_ptr, type_struct_field);
958	ir_link_new_instruction(new_instruction, old_instruction);
959	return new_instruction;
960	}
961
962	static IrInstruction ir_build_union_field_ptr(IrBuilder irb, Scope scope, AstNode source_node,	1044	static IrInstruction ir_build_union_field_ptr(IrBuilder irb, Scope scope, AstNode source_node,
963	IrInstruction union_ptr, TypeUnionField field)	1045	IrInstruction union_ptr, TypeUnionField field)
964	{	1046	{
...	@@ -982,7 +1064,7 @@ static IrInstruction ir_build_union_field_ptr_from(IrBuilder irb, IrInstructio	...	@@ -982,7 +1064,7 @@ static IrInstruction ir_build_union_field_ptr_from(IrBuilder irb, IrInstructio
982		1064
983	static IrInstruction ir_build_call(IrBuilder irb, Scope scope, AstNode source_node,	1065	static IrInstruction ir_build_call(IrBuilder irb, Scope scope, AstNode source_node,
984	FnTableEntry fn_entry, IrInstruction fn_ref, size_t arg_count, IrInstruction **args,	1066	FnTableEntry fn_entry, IrInstruction fn_ref, size_t arg_count, IrInstruction **args,
985	bool is_comptime, FnInline fn_inline)	1067	bool is_comptime, FnInline fn_inline, bool is_async, IrInstruction *async_allocator)
986	{	1068	{
987	IrInstructionCall *call_instruction = ir_build_instruction<IrInstructionCall>(irb, scope, source_node);	1069	IrInstructionCall *call_instruction = ir_build_instruction<IrInstructionCall>(irb, scope, source_node);
988	call_instruction->fn_entry = fn_entry;	1070	call_instruction->fn_entry = fn_entry;
...	@@ -991,21 +1073,25 @@ static IrInstruction ir_build_call(IrBuilder irb, Scope scope, AstNode sourc	...	@@ -991,21 +1073,25 @@ static IrInstruction ir_build_call(IrBuilder irb, Scope scope, AstNode sourc
991	call_instruction->fn_inline = fn_inline;	1073	call_instruction->fn_inline = fn_inline;
992	call_instruction->args = args;	1074	call_instruction->args = args;
993	call_instruction->arg_count = arg_count;	1075	call_instruction->arg_count = arg_count;
		1076	call_instruction->is_async = is_async;
		1077	call_instruction->async_allocator = async_allocator;
994		1078
995	if (fn_ref)	1079	if (fn_ref)
996	ir_ref_instruction(fn_ref, irb->current_basic_block);	1080	ir_ref_instruction(fn_ref, irb->current_basic_block);
997	for (size_t i = 0; i < arg_count; i += 1)	1081	for (size_t i = 0; i < arg_count; i += 1)
998	ir_ref_instruction(args[i], irb->current_basic_block);	1082	ir_ref_instruction(args[i], irb->current_basic_block);
		1083	if (async_allocator)
		1084	ir_ref_instruction(async_allocator, irb->current_basic_block);
999		1085
1000	return &call_instruction->base;	1086	return &call_instruction->base;
1001	}	1087	}
1002		1088
1003	static IrInstruction ir_build_call_from(IrBuilder irb, IrInstruction *old_instruction,	1089	static IrInstruction ir_build_call_from(IrBuilder irb, IrInstruction *old_instruction,
1004	FnTableEntry fn_entry, IrInstruction fn_ref, size_t arg_count, IrInstruction **args,	1090	FnTableEntry fn_entry, IrInstruction fn_ref, size_t arg_count, IrInstruction **args,
1005	bool is_comptime, FnInline fn_inline)	1091	bool is_comptime, FnInline fn_inline, bool is_async, IrInstruction *async_allocator)
1006	{	1092	{
1007	IrInstruction *new_instruction = ir_build_call(irb, old_instruction->scope,	1093	IrInstruction *new_instruction = ir_build_call(irb, old_instruction->scope,
1008	old_instruction->source_node, fn_entry, fn_ref, arg_count, args, is_comptime, fn_inline);	1094	old_instruction->source_node, fn_entry, fn_ref, arg_count, args, is_comptime, fn_inline, is_async, async_allocator);
1009	ir_link_new_instruction(new_instruction, old_instruction);	1095	ir_link_new_instruction(new_instruction, old_instruction);
1010	return new_instruction;	1096	return new_instruction;
1011	}	1097	}
...	@@ -2396,6 +2482,182 @@ static IrInstruction ir_build_error_union(IrBuilder irb, Scope *scope, AstNode	...	@@ -2396,6 +2482,182 @@ static IrInstruction ir_build_error_union(IrBuilder irb, Scope *scope, AstNode
2396	return &instruction->base;	2482	return &instruction->base;
2397	}	2483	}
2398		2484
		2485	static IrInstruction ir_build_cancel(IrBuilder irb, Scope scope, AstNode source_node,
		2486	IrInstruction *target)
		2487	{
		2488	IrInstructionCancel *instruction = ir_build_instruction<IrInstructionCancel>(irb, scope, source_node);
		2489	instruction->target = target;
		2490
		2491	ir_ref_instruction(target, irb->current_basic_block);
		2492
		2493	return &instruction->base;
		2494	}
		2495
		2496	static IrInstruction ir_build_get_implicit_allocator(IrBuilder irb, Scope scope, AstNode source_node,
		2497	ImplicitAllocatorId id)
		2498	{
		2499	IrInstructionGetImplicitAllocator *instruction = ir_build_instruction<IrInstructionGetImplicitAllocator>(irb, scope, source_node);
		2500	instruction->id = id;
		2501
		2502	return &instruction->base;
		2503	}
		2504
		2505	static IrInstruction ir_build_coro_id(IrBuilder irb, Scope scope, AstNode source_node, IrInstruction *promise_ptr) {
		2506	IrInstructionCoroId *instruction = ir_build_instruction<IrInstructionCoroId>(irb, scope, source_node);
		2507	instruction->promise_ptr = promise_ptr;
		2508
		2509	ir_ref_instruction(promise_ptr, irb->current_basic_block);
		2510
		2511	return &instruction->base;
		2512	}
		2513
		2514	static IrInstruction ir_build_coro_alloc(IrBuilder irb, Scope scope, AstNode source_node, IrInstruction *coro_id) {
		2515	IrInstructionCoroAlloc *instruction = ir_build_instruction<IrInstructionCoroAlloc>(irb, scope, source_node);
		2516	instruction->coro_id = coro_id;
		2517
		2518	ir_ref_instruction(coro_id, irb->current_basic_block);
		2519
		2520	return &instruction->base;
		2521	}
		2522
		2523	static IrInstruction ir_build_coro_size(IrBuilder irb, Scope scope, AstNode source_node) {
		2524	IrInstructionCoroSize *instruction = ir_build_instruction<IrInstructionCoroSize>(irb, scope, source_node);
		2525
		2526	return &instruction->base;
		2527	}
		2528
		2529	static IrInstruction ir_build_coro_begin(IrBuilder irb, Scope scope, AstNode source_node, IrInstruction coro_id, IrInstruction coro_mem_ptr) {
		2530	IrInstructionCoroBegin *instruction = ir_build_instruction<IrInstructionCoroBegin>(irb, scope, source_node);
		2531	instruction->coro_id = coro_id;
		2532	instruction->coro_mem_ptr = coro_mem_ptr;
		2533
		2534	ir_ref_instruction(coro_id, irb->current_basic_block);
		2535	ir_ref_instruction(coro_mem_ptr, irb->current_basic_block);
		2536
		2537	return &instruction->base;
		2538	}
		2539
		2540	static IrInstruction ir_build_coro_alloc_fail(IrBuilder irb, Scope scope, AstNode source_node, IrInstruction *err_val) {
		2541	IrInstructionCoroAllocFail *instruction = ir_build_instruction<IrInstructionCoroAllocFail>(irb, scope, source_node);
		2542	instruction->base.value.type = irb->codegen->builtin_types.entry_unreachable;
		2543	instruction->base.value.special = ConstValSpecialStatic;
		2544	instruction->err_val = err_val;
		2545
		2546	ir_ref_instruction(err_val, irb->current_basic_block);
		2547
		2548	return &instruction->base;
		2549	}
		2550
		2551	static IrInstruction ir_build_coro_suspend(IrBuilder irb, Scope scope, AstNode source_node,
		2552	IrInstruction save_point, IrInstruction is_final)
		2553	{
		2554	IrInstructionCoroSuspend *instruction = ir_build_instruction<IrInstructionCoroSuspend>(irb, scope, source_node);
		2555	instruction->save_point = save_point;
		2556	instruction->is_final = is_final;
		2557
		2558	if (save_point != nullptr) ir_ref_instruction(save_point, irb->current_basic_block);
		2559	ir_ref_instruction(is_final, irb->current_basic_block);
		2560
		2561	return &instruction->base;
		2562	}
		2563
		2564	static IrInstruction ir_build_coro_end(IrBuilder irb, Scope scope, AstNode source_node) {
		2565	IrInstructionCoroEnd *instruction = ir_build_instruction<IrInstructionCoroEnd>(irb, scope, source_node);
		2566	return &instruction->base;
		2567	}
		2568
		2569	static IrInstruction ir_build_coro_free(IrBuilder irb, Scope scope, AstNode source_node,
		2570	IrInstruction coro_id, IrInstruction coro_handle)
		2571	{
		2572	IrInstructionCoroFree *instruction = ir_build_instruction<IrInstructionCoroFree>(irb, scope, source_node);
		2573	instruction->coro_id = coro_id;
		2574	instruction->coro_handle = coro_handle;
		2575
		2576	ir_ref_instruction(coro_id, irb->current_basic_block);
		2577	ir_ref_instruction(coro_handle, irb->current_basic_block);
		2578
		2579	return &instruction->base;
		2580	}
		2581
		2582	static IrInstruction ir_build_coro_resume(IrBuilder irb, Scope scope, AstNode source_node,
		2583	IrInstruction *awaiter_handle)
		2584	{
		2585	IrInstructionCoroResume *instruction = ir_build_instruction<IrInstructionCoroResume>(irb, scope, source_node);
		2586	instruction->awaiter_handle = awaiter_handle;
		2587
		2588	ir_ref_instruction(awaiter_handle, irb->current_basic_block);
		2589
		2590	return &instruction->base;
		2591	}
		2592
		2593	static IrInstruction ir_build_coro_save(IrBuilder irb, Scope scope, AstNode source_node,
		2594	IrInstruction *coro_handle)
		2595	{
		2596	IrInstructionCoroSave *instruction = ir_build_instruction<IrInstructionCoroSave>(irb, scope, source_node);
		2597	instruction->coro_handle = coro_handle;
		2598
		2599	ir_ref_instruction(coro_handle, irb->current_basic_block);
		2600
		2601	return &instruction->base;
		2602	}
		2603
		2604	static IrInstruction ir_build_coro_promise(IrBuilder irb, Scope scope, AstNode source_node,
		2605	IrInstruction *coro_handle)
		2606	{
		2607	IrInstructionCoroPromise *instruction = ir_build_instruction<IrInstructionCoroPromise>(irb, scope, source_node);
		2608	instruction->coro_handle = coro_handle;
		2609
		2610	ir_ref_instruction(coro_handle, irb->current_basic_block);
		2611
		2612	return &instruction->base;
		2613	}
		2614
		2615	static IrInstruction ir_build_coro_alloc_helper(IrBuilder irb, Scope scope, AstNode source_node,
		2616	IrInstruction alloc_fn, IrInstruction coro_size)
		2617	{
		2618	IrInstructionCoroAllocHelper *instruction = ir_build_instruction<IrInstructionCoroAllocHelper>(irb, scope, source_node);
		2619	instruction->alloc_fn = alloc_fn;
		2620	instruction->coro_size = coro_size;
		2621
		2622	ir_ref_instruction(alloc_fn, irb->current_basic_block);
		2623	ir_ref_instruction(coro_size, irb->current_basic_block);
		2624
		2625	return &instruction->base;
		2626	}
		2627
		2628	static IrInstruction ir_build_atomic_rmw(IrBuilder irb, Scope scope, AstNode source_node,
		2629	IrInstruction operand_type, IrInstruction ptr, IrInstruction op, IrInstruction operand,
		2630	IrInstruction *ordering, AtomicRmwOp resolved_op, AtomicOrder resolved_ordering)
		2631	{
		2632	IrInstructionAtomicRmw *instruction = ir_build_instruction<IrInstructionAtomicRmw>(irb, scope, source_node);
		2633	instruction->operand_type = operand_type;
		2634	instruction->ptr = ptr;
		2635	instruction->op = op;
		2636	instruction->operand = operand;
		2637	instruction->ordering = ordering;
		2638	instruction->resolved_op = resolved_op;
		2639	instruction->resolved_ordering = resolved_ordering;
		2640
		2641	if (operand_type != nullptr) ir_ref_instruction(operand_type, irb->current_basic_block);
		2642	ir_ref_instruction(ptr, irb->current_basic_block);
		2643	if (op != nullptr) ir_ref_instruction(op, irb->current_basic_block);
		2644	ir_ref_instruction(operand, irb->current_basic_block);
		2645	if (ordering != nullptr) ir_ref_instruction(ordering, irb->current_basic_block);
		2646
		2647	return &instruction->base;
		2648	}
		2649
		2650	static IrInstruction ir_build_promise_result_type(IrBuilder irb, Scope scope, AstNode source_node,
		2651	IrInstruction *promise_type)
		2652	{
		2653	IrInstructionPromiseResultType *instruction = ir_build_instruction<IrInstructionPromiseResultType>(irb, scope, source_node);
		2654	instruction->promise_type = promise_type;
		2655
		2656	ir_ref_instruction(promise_type, irb->current_basic_block);
		2657
		2658	return &instruction->base;
		2659	}
		2660
2399	static void ir_count_defers(IrBuilder irb, Scope inner_scope, Scope outer_scope, size_t results) {	2661	static void ir_count_defers(IrBuilder irb, Scope inner_scope, Scope outer_scope, size_t results) {
2400	results[ReturnKindUnconditional] = 0;	2662	results[ReturnKindUnconditional] = 0;
2401	results[ReturnKindError] = 0;	2663	results[ReturnKindError] = 0;
...	@@ -2468,6 +2730,36 @@ static ScopeDeferExpr get_scope_defer_expr(Scope scope) {	...	@@ -2468,6 +2730,36 @@ static ScopeDeferExpr get_scope_defer_expr(Scope scope) {
2468	return nullptr;	2730	return nullptr;
2469	}	2731	}
2470		2732
		2733	static IrInstruction ir_gen_async_return(IrBuilder irb, Scope scope, AstNode node, IrInstruction *return_value,
		2734	bool is_generated_code)
		2735	{
		2736	FnTableEntry *fn_entry = exec_fn_entry(irb->exec);
		2737	bool is_async = fn_entry != nullptr && fn_entry->type_entry->data.fn.fn_type_id.cc == CallingConventionAsync;
		2738	if (!is_async) {
		2739	IrInstruction *return_inst = ir_build_return(irb, scope, node, return_value);
		2740	return_inst->is_gen = is_generated_code;
		2741	return return_inst;
		2742	}
		2743
		2744	if (irb->exec->coro_result_ptr_field_ptr) {
		2745	IrInstruction *result_ptr = ir_build_load_ptr(irb, scope, node, irb->exec->coro_result_ptr_field_ptr);
		2746	ir_build_store_ptr(irb, scope, node, result_ptr, return_value);
		2747	}
		2748	IrInstruction *promise_type_val = ir_build_const_type(irb, scope, node,
		2749	get_maybe_type(irb->codegen, irb->codegen->builtin_types.entry_promise));
		2750	// TODO replace replacement_value with @intToPtr(?promise, 0x1) when it doesn't crash zig
		2751	IrInstruction *replacement_value = irb->exec->coro_handle;
		2752	IrInstruction *maybe_await_handle = ir_build_atomic_rmw(irb, scope, node,
		2753	promise_type_val, irb->exec->coro_awaiter_field_ptr, nullptr, replacement_value, nullptr,
		2754	AtomicRmwOp_xchg, AtomicOrderSeqCst);
		2755	ir_build_store_ptr(irb, scope, node, irb->exec->await_handle_var_ptr, maybe_await_handle);
		2756	IrInstruction *is_non_null = ir_build_test_nonnull(irb, scope, node, maybe_await_handle);
		2757	IrInstruction *is_comptime = ir_build_const_bool(irb, scope, node, false);
		2758	return ir_build_cond_br(irb, scope, node, is_non_null, irb->exec->coro_normal_final, irb->exec->coro_early_final,
		2759	is_comptime);
		2760	// the above blocks are rendered by ir_gen after the rest of codegen
		2761	}
		2762
2471	static IrInstruction ir_gen_return(IrBuilder irb, Scope scope, AstNode node, LVal lval) {	2763	static IrInstruction ir_gen_return(IrBuilder irb, Scope scope, AstNode node, LVal lval) {
2472	assert(node->type == NodeTypeReturnExpr);	2764	assert(node->type == NodeTypeReturnExpr);
2473		2765
...	@@ -2517,18 +2809,22 @@ static IrInstruction ir_gen_return(IrBuilder irb, Scope scope, AstNode node,	...	@@ -2517,18 +2809,22 @@ static IrInstruction ir_gen_return(IrBuilder irb, Scope scope, AstNode node,
2517	}	2809	}
2518		2810
2519	ir_mark_gen(ir_build_cond_br(irb, scope, node, is_err, err_block, ok_block, is_comptime));	2811	ir_mark_gen(ir_build_cond_br(irb, scope, node, is_err, err_block, ok_block, is_comptime));
		2812	IrBasicBlock *ret_stmt_block = ir_create_basic_block(irb, scope, "RetStmt");
2520		2813
2521	ir_set_cursor_at_end_and_append_block(irb, err_block);	2814	ir_set_cursor_at_end_and_append_block(irb, err_block);
2522	ir_gen_defers_for_block(irb, scope, outer_scope, true);	2815	ir_gen_defers_for_block(irb, scope, outer_scope, true);
2523	ir_build_return(irb, scope, node, return_value);	2816	ir_build_br(irb, scope, node, ret_stmt_block, is_comptime);
2524		2817
2525	ir_set_cursor_at_end_and_append_block(irb, ok_block);	2818	ir_set_cursor_at_end_and_append_block(irb, ok_block);
2526	ir_gen_defers_for_block(irb, scope, outer_scope, false);	2819	ir_gen_defers_for_block(irb, scope, outer_scope, false);
2527	return ir_build_return(irb, scope, node, return_value);	2820	ir_build_br(irb, scope, node, ret_stmt_block, is_comptime);
		2821
		2822	ir_set_cursor_at_end_and_append_block(irb, ret_stmt_block);
		2823	return ir_gen_async_return(irb, scope, node, return_value, false);
2528	} else {	2824	} else {
2529	// generate unconditional defers	2825	// generate unconditional defers
2530	ir_gen_defers_for_block(irb, scope, outer_scope, false);	2826	ir_gen_defers_for_block(irb, scope, outer_scope, false);
2531	return ir_build_return(irb, scope, node, return_value);	2827	return ir_gen_async_return(irb, scope, node, return_value, false);
2532	}	2828	}
2533	}	2829	}
2534	case ReturnKindError:	2830	case ReturnKindError:
...	@@ -2548,7 +2844,7 @@ static IrInstruction ir_gen_return(IrBuilder irb, Scope scope, AstNode node,	...	@@ -2548,7 +2844,7 @@ static IrInstruction ir_gen_return(IrBuilder irb, Scope scope, AstNode node,
2548	ir_set_cursor_at_end_and_append_block(irb, return_block);	2844	ir_set_cursor_at_end_and_append_block(irb, return_block);
2549	ir_gen_defers_for_block(irb, scope, outer_scope, true);	2845	ir_gen_defers_for_block(irb, scope, outer_scope, true);
2550	IrInstruction *err_val = ir_build_unwrap_err_code(irb, scope, node, err_union_ptr);	2846	IrInstruction *err_val = ir_build_unwrap_err_code(irb, scope, node, err_union_ptr);
2551	ir_build_return(irb, scope, node, err_val);	2847	ir_gen_async_return(irb, scope, node, err_val, false);
2552		2848
2553	ir_set_cursor_at_end_and_append_block(irb, continue_block);	2849	ir_set_cursor_at_end_and_append_block(irb, continue_block);
2554	IrInstruction *unwrapped_ptr = ir_build_unwrap_err_payload(irb, scope, node, err_union_ptr, false);	2850	IrInstruction *unwrapped_ptr = ir_build_unwrap_err_payload(irb, scope, node, err_union_ptr, false);
...	@@ -3739,7 +4035,7 @@ static IrInstruction ir_gen_builtin_fn_call(IrBuilder irb, Scope *scope, AstNo	...	@@ -3739,7 +4035,7 @@ static IrInstruction ir_gen_builtin_fn_call(IrBuilder irb, Scope *scope, AstNo
3739	}	4035	}
3740	FnInline fn_inline = (builtin_fn->id == BuiltinFnIdInlineCall) ? FnInlineAlways : FnInlineNever;	4036	FnInline fn_inline = (builtin_fn->id == BuiltinFnIdInlineCall) ? FnInlineAlways : FnInlineNever;
3741		4037
3742	return ir_build_call(irb, scope, node, nullptr, fn_ref, arg_count, args, false, fn_inline);	4038	return ir_build_call(irb, scope, node, nullptr, fn_ref, arg_count, args, false, fn_inline, false, nullptr);
3743	}	4039	}
3744	case BuiltinFnIdTypeId:	4040	case BuiltinFnIdTypeId:
3745	{	4041	{
...	@@ -3849,6 +4145,38 @@ static IrInstruction ir_gen_builtin_fn_call(IrBuilder irb, Scope *scope, AstNo	...	@@ -3849,6 +4145,38 @@ static IrInstruction ir_gen_builtin_fn_call(IrBuilder irb, Scope *scope, AstNo
3849	{	4145	{
3850	return ir_build_error_return_trace(irb, scope, node);	4146	return ir_build_error_return_trace(irb, scope, node);
3851	}	4147	}
		4148	case BuiltinFnIdAtomicRmw:
		4149	{
		4150	AstNode *arg0_node = node->data.fn_call_expr.params.at(0);
		4151	IrInstruction *arg0_value = ir_gen_node(irb, arg0_node, scope);
		4152	if (arg0_value == irb->codegen->invalid_instruction)
		4153	return arg0_value;
		4154
		4155	AstNode *arg1_node = node->data.fn_call_expr.params.at(1);
		4156	IrInstruction *arg1_value = ir_gen_node(irb, arg1_node, scope);
		4157	if (arg1_value == irb->codegen->invalid_instruction)
		4158	return arg1_value;
		4159
		4160	AstNode *arg2_node = node->data.fn_call_expr.params.at(2);
		4161	IrInstruction *arg2_value = ir_gen_node(irb, arg2_node, scope);
		4162	if (arg2_value == irb->codegen->invalid_instruction)
		4163	return arg2_value;
		4164
		4165	AstNode *arg3_node = node->data.fn_call_expr.params.at(3);
		4166	IrInstruction *arg3_value = ir_gen_node(irb, arg3_node, scope);
		4167	if (arg3_value == irb->codegen->invalid_instruction)
		4168	return arg3_value;
		4169
		4170	AstNode *arg4_node = node->data.fn_call_expr.params.at(4);
		4171	IrInstruction *arg4_value = ir_gen_node(irb, arg4_node, scope);
		4172	if (arg4_value == irb->codegen->invalid_instruction)
		4173	return arg4_value;
		4174
		4175	return ir_build_atomic_rmw(irb, scope, node, arg0_value, arg1_value, arg2_value, arg3_value,
		4176	arg4_value,
		4177	// these 2 values don't mean anything since we passed non-null values for other args
		4178	AtomicRmwOp_xchg, AtomicOrderMonotonic);
		4179	}
3852	}	4180	}
3853	zig_unreachable();	4181	zig_unreachable();
3854	}	4182	}
...	@@ -3873,7 +4201,17 @@ static IrInstruction ir_gen_fn_call(IrBuilder irb, Scope scope, AstNode node	...	@@ -3873,7 +4201,17 @@ static IrInstruction ir_gen_fn_call(IrBuilder irb, Scope scope, AstNode node
3873	return args[i];	4201	return args[i];
3874	}	4202	}
3875		4203
3876	return ir_build_call(irb, scope, node, nullptr, fn_ref, arg_count, args, false, FnInlineAuto);	4204	bool is_async = node->data.fn_call_expr.is_async;
		4205	IrInstruction *async_allocator = nullptr;
		4206	if (is_async) {
		4207	if (node->data.fn_call_expr.async_allocator) {
		4208	async_allocator = ir_gen_node(irb, node->data.fn_call_expr.async_allocator, scope);
		4209	if (async_allocator == irb->codegen->invalid_instruction)
		4210	return async_allocator;
		4211	}
		4212	}
		4213
		4214	return ir_build_call(irb, scope, node, nullptr, fn_ref, arg_count, args, false, FnInlineAuto, is_async, async_allocator);
3877	}	4215	}
3878		4216
3879	static IrInstruction ir_gen_if_bool_expr(IrBuilder irb, Scope scope, AstNode node) {	4217	static IrInstruction ir_gen_if_bool_expr(IrBuilder irb, Scope scope, AstNode node) {
...	@@ -5604,6 +5942,187 @@ static IrInstruction ir_gen_fn_proto(IrBuilder irb, Scope *parent_scope, AstNo	...	@@ -5604,6 +5942,187 @@ static IrInstruction ir_gen_fn_proto(IrBuilder irb, Scope *parent_scope, AstNo
5604	return ir_build_fn_proto(irb, parent_scope, node, param_types, align_value, return_type, is_var_args);	5942	return ir_build_fn_proto(irb, parent_scope, node, param_types, align_value, return_type, is_var_args);
5605	}	5943	}
5606		5944
		5945	static IrInstruction ir_gen_cancel(IrBuilder irb, Scope parent_scope, AstNode node) {
		5946	assert(node->type == NodeTypeCancel);
		5947
		5948	IrInstruction *target_inst = ir_gen_node(irb, node->data.cancel_expr.expr, parent_scope);
		5949	if (target_inst == irb->codegen->invalid_instruction)
		5950	return irb->codegen->invalid_instruction;
		5951
		5952	return ir_build_cancel(irb, parent_scope, node, target_inst);
		5953	}
		5954
		5955	static IrInstruction ir_gen_resume(IrBuilder irb, Scope parent_scope, AstNode node) {
		5956	assert(node->type == NodeTypeResume);
		5957
		5958	IrInstruction *target_inst = ir_gen_node(irb, node->data.resume_expr.expr, parent_scope);
		5959	if (target_inst == irb->codegen->invalid_instruction)
		5960	return irb->codegen->invalid_instruction;
		5961
		5962	return ir_build_coro_resume(irb, parent_scope, node, target_inst);
		5963	}
		5964
		5965	static IrInstruction ir_gen_await_expr(IrBuilder irb, Scope parent_scope, AstNode node) {
		5966	assert(node->type == NodeTypeAwaitExpr);
		5967
		5968	IrInstruction *target_inst = ir_gen_node(irb, node->data.await_expr.expr, parent_scope);
		5969	if (target_inst == irb->codegen->invalid_instruction)
		5970	return irb->codegen->invalid_instruction;
		5971
		5972	FnTableEntry *fn_entry = exec_fn_entry(irb->exec);
		5973	if (!fn_entry) {
		5974	add_node_error(irb->codegen, node, buf_sprintf("await outside function definition"));
		5975	return irb->codegen->invalid_instruction;
		5976	}
		5977	if (fn_entry->type_entry->data.fn.fn_type_id.cc != CallingConventionAsync) {
		5978	add_node_error(irb->codegen, node, buf_sprintf("await in non-async function"));
		5979	return irb->codegen->invalid_instruction;
		5980	}
		5981
		5982	ScopeDeferExpr *scope_defer_expr = get_scope_defer_expr(parent_scope);
		5983	if (scope_defer_expr) {
		5984	if (!scope_defer_expr->reported_err) {
		5985	add_node_error(irb->codegen, node, buf_sprintf("cannot await inside defer expression"));
		5986	scope_defer_expr->reported_err = true;
		5987	}
		5988	return irb->codegen->invalid_instruction;
		5989	}
		5990
		5991	Scope *outer_scope = irb->exec->begin_scope;
		5992
		5993	IrInstruction *coro_promise_ptr = ir_build_coro_promise(irb, parent_scope, node, target_inst);
		5994	Buf *result_ptr_field_name = buf_create_from_str(RESULT_PTR_FIELD_NAME);
		5995	IrInstruction *result_ptr_field_ptr = ir_build_field_ptr(irb, parent_scope, node, coro_promise_ptr, result_ptr_field_name);
		5996
		5997	Buf *awaiter_handle_field_name = buf_create_from_str(AWAITER_HANDLE_FIELD_NAME);
		5998	IrInstruction *awaiter_field_ptr = ir_build_field_ptr(irb, parent_scope, node, coro_promise_ptr,
		5999	awaiter_handle_field_name);
		6000
		6001	IrInstruction *const_bool_false = ir_build_const_bool(irb, parent_scope, node, false);
		6002	VariableTableEntry *result_var = ir_create_var(irb, node, parent_scope, nullptr,
		6003	false, false, true, const_bool_false);
		6004	IrInstruction *undefined_value = ir_build_const_undefined(irb, parent_scope, node);
		6005	IrInstruction *target_promise_type = ir_build_typeof(irb, parent_scope, node, target_inst);
		6006	IrInstruction *promise_result_type = ir_build_promise_result_type(irb, parent_scope, node, target_promise_type);
		6007	ir_build_var_decl(irb, parent_scope, node, result_var, promise_result_type, nullptr, undefined_value);
		6008	IrInstruction *my_result_var_ptr = ir_build_var_ptr(irb, parent_scope, node, result_var, false, false);
		6009	ir_build_store_ptr(irb, parent_scope, node, result_ptr_field_ptr, my_result_var_ptr);
		6010	IrInstruction *save_token = ir_build_coro_save(irb, parent_scope, node, irb->exec->coro_handle);
		6011	IrInstruction *promise_type_val = ir_build_const_type(irb, parent_scope, node,
		6012	get_maybe_type(irb->codegen, irb->codegen->builtin_types.entry_promise));
		6013	IrInstruction *maybe_await_handle = ir_build_atomic_rmw(irb, parent_scope, node,
		6014	promise_type_val, awaiter_field_ptr, nullptr, irb->exec->coro_handle, nullptr,
		6015	AtomicRmwOp_xchg, AtomicOrderSeqCst);
		6016	IrInstruction *is_non_null = ir_build_test_nonnull(irb, parent_scope, node, maybe_await_handle);
		6017	IrBasicBlock *yes_suspend_block = ir_create_basic_block(irb, parent_scope, "YesSuspend");
		6018	IrBasicBlock *no_suspend_block = ir_create_basic_block(irb, parent_scope, "NoSuspend");
		6019	IrBasicBlock *merge_block = ir_create_basic_block(irb, parent_scope, "Merge");
		6020	ir_build_cond_br(irb, parent_scope, node, is_non_null, no_suspend_block, yes_suspend_block, const_bool_false);
		6021
		6022	ir_set_cursor_at_end_and_append_block(irb, no_suspend_block);
		6023	Buf *result_field_name = buf_create_from_str(RESULT_FIELD_NAME);
		6024	IrInstruction *promise_result_ptr = ir_build_field_ptr(irb, parent_scope, node, coro_promise_ptr, result_field_name);
		6025	IrInstruction *no_suspend_result = ir_build_load_ptr(irb, parent_scope, node, promise_result_ptr);
		6026	ir_build_cancel(irb, parent_scope, node, target_inst);
		6027	ir_build_br(irb, parent_scope, node, merge_block, const_bool_false);
		6028
		6029	ir_set_cursor_at_end_and_append_block(irb, yes_suspend_block);
		6030	IrInstruction *suspend_code = ir_build_coro_suspend(irb, parent_scope, node, save_token, const_bool_false);
		6031	IrBasicBlock *cleanup_block = ir_create_basic_block(irb, parent_scope, "SuspendCleanup");
		6032	IrBasicBlock *resume_block = ir_create_basic_block(irb, parent_scope, "SuspendResume");
		6033
		6034	IrInstructionSwitchBrCase *cases = allocate<IrInstructionSwitchBrCase>(2);
		6035	cases[0].value = ir_build_const_u8(irb, parent_scope, node, 0);
		6036	cases[0].block = resume_block;
		6037	cases[1].value = ir_build_const_u8(irb, parent_scope, node, 1);
		6038	cases[1].block = cleanup_block;
		6039	ir_build_switch_br(irb, parent_scope, node, suspend_code, irb->exec->coro_suspend_block,
		6040	2, cases, const_bool_false);
		6041
		6042	ir_set_cursor_at_end_and_append_block(irb, cleanup_block);
		6043	ir_gen_defers_for_block(irb, parent_scope, outer_scope, true);
		6044	ir_build_br(irb, parent_scope, node, irb->exec->coro_final_cleanup_block, const_bool_false);
		6045
		6046	ir_set_cursor_at_end_and_append_block(irb, resume_block);
		6047	IrInstruction *yes_suspend_result = ir_build_load_ptr(irb, parent_scope, node, my_result_var_ptr);
		6048	ir_build_br(irb, parent_scope, node, merge_block, const_bool_false);
		6049
		6050	ir_set_cursor_at_end_and_append_block(irb, merge_block);
		6051	IrBasicBlock *incoming_blocks = allocate<IrBasicBlock >(2);
		6052	IrInstruction *incoming_values = allocate<IrInstruction >(2);
		6053	incoming_blocks[0] = resume_block;
		6054	incoming_values[0] = yes_suspend_result;
		6055	incoming_blocks[1] = no_suspend_block;
		6056	incoming_values[1] = no_suspend_result;
		6057	return ir_build_phi(irb, parent_scope, node, 2, incoming_blocks, incoming_values);
		6058	}
		6059
		6060	static IrInstruction ir_gen_suspend(IrBuilder irb, Scope parent_scope, AstNode node) {
		6061	assert(node->type == NodeTypeSuspend);
		6062
		6063	FnTableEntry *fn_entry = exec_fn_entry(irb->exec);
		6064	if (!fn_entry) {
		6065	add_node_error(irb->codegen, node, buf_sprintf("suspend outside function definition"));
		6066	return irb->codegen->invalid_instruction;
		6067	}
		6068	if (fn_entry->type_entry->data.fn.fn_type_id.cc != CallingConventionAsync) {
		6069	add_node_error(irb->codegen, node, buf_sprintf("suspend in non-async function"));
		6070	return irb->codegen->invalid_instruction;
		6071	}
		6072
		6073	ScopeDeferExpr *scope_defer_expr = get_scope_defer_expr(parent_scope);
		6074	if (scope_defer_expr) {
		6075	if (!scope_defer_expr->reported_err) {
		6076	add_node_error(irb->codegen, node, buf_sprintf("cannot suspend inside defer expression"));
		6077	scope_defer_expr->reported_err = true;
		6078	}
		6079	return irb->codegen->invalid_instruction;
		6080	}
		6081
		6082	Scope *outer_scope = irb->exec->begin_scope;
		6083
		6084
		6085	IrInstruction *suspend_code;
		6086	IrInstruction *const_bool_false = ir_build_const_bool(irb, parent_scope, node, false);
		6087	if (node->data.suspend.block == nullptr) {
		6088	suspend_code = ir_build_coro_suspend(irb, parent_scope, node, nullptr, const_bool_false);
		6089	} else {
		6090	assert(node->data.suspend.promise_symbol != nullptr);
		6091	assert(node->data.suspend.promise_symbol->type == NodeTypeSymbol);
		6092	Buf *promise_symbol_name = node->data.suspend.promise_symbol->data.symbol_expr.symbol;
		6093	Scope *child_scope;
		6094	if (!buf_eql_str(promise_symbol_name, "_")) {
		6095	VariableTableEntry *promise_var = ir_create_var(irb, node, parent_scope, promise_symbol_name,
		6096	true, true, false, const_bool_false);
		6097	ir_build_var_decl(irb, parent_scope, node, promise_var, nullptr, nullptr, irb->exec->coro_handle);
		6098	child_scope = promise_var->child_scope;
		6099	} else {
		6100	child_scope = parent_scope;
		6101	}
		6102	IrInstruction *save_token = ir_build_coro_save(irb, child_scope, node, irb->exec->coro_handle);
		6103	ir_gen_node(irb, node->data.suspend.block, child_scope);
		6104	suspend_code = ir_build_coro_suspend(irb, parent_scope, node, save_token, const_bool_false);
		6105	}
		6106
		6107	IrBasicBlock *cleanup_block = ir_create_basic_block(irb, parent_scope, "SuspendCleanup");
		6108	IrBasicBlock *resume_block = ir_create_basic_block(irb, parent_scope, "SuspendResume");
		6109
		6110	IrInstructionSwitchBrCase *cases = allocate<IrInstructionSwitchBrCase>(2);
		6111	cases[0].value = ir_build_const_u8(irb, parent_scope, node, 0);
		6112	cases[0].block = resume_block;
		6113	cases[1].value = ir_build_const_u8(irb, parent_scope, node, 1);
		6114	cases[1].block = cleanup_block;
		6115	ir_build_switch_br(irb, parent_scope, node, suspend_code, irb->exec->coro_suspend_block,
		6116	2, cases, const_bool_false);
		6117
		6118	ir_set_cursor_at_end_and_append_block(irb, cleanup_block);
		6119	ir_gen_defers_for_block(irb, parent_scope, outer_scope, true);
		6120	ir_build_br(irb, parent_scope, node, irb->exec->coro_final_cleanup_block, const_bool_false);
		6121
		6122	ir_set_cursor_at_end_and_append_block(irb, resume_block);
		6123	return ir_build_const_void(irb, parent_scope, node);
		6124	}
		6125
5607	static IrInstruction ir_gen_node_raw(IrBuilder irb, AstNode node, Scope scope,	6126	static IrInstruction ir_gen_node_raw(IrBuilder irb, AstNode node, Scope scope,
5608	LVal lval)	6127	LVal lval)
5609	{	6128	{
...	@@ -5700,6 +6219,14 @@ static IrInstruction ir_gen_node_raw(IrBuilder irb, AstNode node, Scope scop	...	@@ -5700,6 +6219,14 @@ static IrInstruction ir_gen_node_raw(IrBuilder irb, AstNode node, Scope scop
5700	return ir_lval_wrap(irb, scope, ir_gen_fn_proto(irb, scope, node), lval);	6219	return ir_lval_wrap(irb, scope, ir_gen_fn_proto(irb, scope, node), lval);
5701	case NodeTypeErrorSetDecl:	6220	case NodeTypeErrorSetDecl:
5702	return ir_lval_wrap(irb, scope, ir_gen_err_set_decl(irb, scope, node), lval);	6221	return ir_lval_wrap(irb, scope, ir_gen_err_set_decl(irb, scope, node), lval);
		6222	case NodeTypeCancel:
		6223	return ir_lval_wrap(irb, scope, ir_gen_cancel(irb, scope, node), lval);
		6224	case NodeTypeResume:
		6225	return ir_lval_wrap(irb, scope, ir_gen_resume(irb, scope, node), lval);
		6226	case NodeTypeAwaitExpr:
		6227	return ir_lval_wrap(irb, scope, ir_gen_await_expr(irb, scope, node), lval);
		6228	case NodeTypeSuspend:
		6229	return ir_lval_wrap(irb, scope, ir_gen_suspend(irb, scope, node), lval);
5703	}	6230	}
5704	zig_unreachable();	6231	zig_unreachable();
5705	}	6232	}
...	@@ -5728,6 +6255,7 @@ static void invalidate_exec(IrExecutable *exec) {	...	@@ -5728,6 +6255,7 @@ static void invalidate_exec(IrExecutable *exec) {
5728	invalidate_exec(exec->source_exec);	6255	invalidate_exec(exec->source_exec);
5729	}	6256	}
5730		6257
		6258
5731	bool ir_gen(CodeGen codegen, AstNode node, Scope scope, IrExecutable ir_executable) {	6259	bool ir_gen(CodeGen codegen, AstNode node, Scope scope, IrExecutable ir_executable) {
5732	assert(node->owner);	6260	assert(node->owner);
5733		6261
...	@@ -5742,13 +6270,162 @@ bool ir_gen(CodeGen codegen, AstNode node, Scope scope, IrExecutable ir_exec	...	@@ -5742,13 +6270,162 @@ bool ir_gen(CodeGen codegen, AstNode node, Scope scope, IrExecutable ir_exec
5742	// Entry block gets a reference because we enter it to begin.	6270	// Entry block gets a reference because we enter it to begin.
5743	ir_ref_bb(irb->current_basic_block);	6271	ir_ref_bb(irb->current_basic_block);
5744		6272
		6273	FnTableEntry *fn_entry = exec_fn_entry(irb->exec);
		6274	bool is_async = fn_entry != nullptr && fn_entry->type_entry->data.fn.fn_type_id.cc == CallingConventionAsync;
		6275	IrInstruction *coro_id;
		6276	IrInstruction *u8_ptr_type;
		6277	IrInstruction *const_bool_false;
		6278	IrInstruction *coro_result_field_ptr;
		6279	TypeTableEntry *return_type;
		6280	Buf *result_ptr_field_name;
		6281	VariableTableEntry *coro_size_var;
		6282	if (is_async) {
		6283	// create the coro promise
		6284	const_bool_false = ir_build_const_bool(irb, scope, node, false);
		6285	VariableTableEntry *promise_var = ir_create_var(irb, node, scope, nullptr, false, false, true, const_bool_false);
		6286
		6287	return_type = fn_entry->type_entry->data.fn.fn_type_id.return_type;
		6288	IrInstruction *promise_init = ir_build_const_promise_init(irb, scope, node, return_type);
		6289	ir_build_var_decl(irb, scope, node, promise_var, nullptr, nullptr, promise_init);
		6290	IrInstruction *coro_promise_ptr = ir_build_var_ptr(irb, scope, node, promise_var, false, false);
		6291
		6292	VariableTableEntry *await_handle_var = ir_create_var(irb, node, scope, nullptr, false, false, true, const_bool_false);
		6293	IrInstruction *null_value = ir_build_const_null(irb, scope, node);
		6294	IrInstruction *await_handle_type_val = ir_build_const_type(irb, scope, node,
		6295	get_maybe_type(irb->codegen, irb->codegen->builtin_types.entry_promise));
		6296	ir_build_var_decl(irb, scope, node, await_handle_var, await_handle_type_val, nullptr, null_value);
		6297	irb->exec->await_handle_var_ptr = ir_build_var_ptr(irb, scope, node,
		6298	await_handle_var, false, false);
		6299
		6300	u8_ptr_type = ir_build_const_type(irb, scope, node,
		6301	get_pointer_to_type(irb->codegen, irb->codegen->builtin_types.entry_u8, false));
		6302	IrInstruction *promise_as_u8_ptr = ir_build_ptr_cast(irb, scope, node, u8_ptr_type, coro_promise_ptr);
		6303	coro_id = ir_build_coro_id(irb, scope, node, promise_as_u8_ptr);
		6304	coro_size_var = ir_create_var(irb, node, scope, nullptr, false, false, true, const_bool_false);
		6305	IrInstruction *coro_size = ir_build_coro_size(irb, scope, node);
		6306	ir_build_var_decl(irb, scope, node, coro_size_var, nullptr, nullptr, coro_size);
		6307	IrInstruction *implicit_allocator_ptr = ir_build_get_implicit_allocator(irb, scope, node,
		6308	ImplicitAllocatorIdArg);
		6309	irb->exec->coro_allocator_var = ir_create_var(irb, node, scope, nullptr, true, true, true, const_bool_false);
		6310	ir_build_var_decl(irb, scope, node, irb->exec->coro_allocator_var, nullptr, nullptr, implicit_allocator_ptr);
		6311	Buf *alloc_field_name = buf_create_from_str(ASYNC_ALLOC_FIELD_NAME);
		6312	IrInstruction *alloc_fn_ptr = ir_build_field_ptr(irb, scope, node, implicit_allocator_ptr, alloc_field_name);
		6313	IrInstruction *alloc_fn = ir_build_load_ptr(irb, scope, node, alloc_fn_ptr);
		6314	IrInstruction *maybe_coro_mem_ptr = ir_build_coro_alloc_helper(irb, scope, node, alloc_fn, coro_size);
		6315	IrInstruction *alloc_result_is_ok = ir_build_test_nonnull(irb, scope, node, maybe_coro_mem_ptr);
		6316	IrBasicBlock *alloc_err_block = ir_create_basic_block(irb, scope, "AllocError");
		6317	IrBasicBlock *alloc_ok_block = ir_create_basic_block(irb, scope, "AllocOk");
		6318	ir_build_cond_br(irb, scope, node, alloc_result_is_ok, alloc_ok_block, alloc_err_block, const_bool_false);
		6319
		6320	ir_set_cursor_at_end_and_append_block(irb, alloc_err_block);
		6321	IrInstruction *undef = ir_build_const_undefined(irb, scope, node);
		6322	ir_build_return(irb, scope, node, undef);
		6323
		6324	ir_set_cursor_at_end_and_append_block(irb, alloc_ok_block);
		6325	IrInstruction *coro_mem_ptr = ir_build_ptr_cast(irb, scope, node, u8_ptr_type, maybe_coro_mem_ptr);
		6326	irb->exec->coro_handle = ir_build_coro_begin(irb, scope, node, coro_id, coro_mem_ptr);
		6327
		6328	Buf *awaiter_handle_field_name = buf_create_from_str(AWAITER_HANDLE_FIELD_NAME);
		6329	irb->exec->coro_awaiter_field_ptr = ir_build_field_ptr(irb, scope, node, coro_promise_ptr,
		6330	awaiter_handle_field_name);
		6331	if (type_has_bits(return_type)) {
		6332	Buf *result_field_name = buf_create_from_str(RESULT_FIELD_NAME);
		6333	coro_result_field_ptr = ir_build_field_ptr(irb, scope, node, coro_promise_ptr, result_field_name);
		6334	result_ptr_field_name = buf_create_from_str(RESULT_PTR_FIELD_NAME);
		6335	irb->exec->coro_result_ptr_field_ptr = ir_build_field_ptr(irb, scope, node, coro_promise_ptr,
		6336	result_ptr_field_name);
		6337	ir_build_store_ptr(irb, scope, node, irb->exec->coro_result_ptr_field_ptr, coro_result_field_ptr);
		6338	}
		6339
		6340
		6341	irb->exec->coro_early_final = ir_create_basic_block(irb, scope, "CoroEarlyFinal");
		6342	irb->exec->coro_normal_final = ir_create_basic_block(irb, scope, "CoroNormalFinal");
		6343	irb->exec->coro_suspend_block = ir_create_basic_block(irb, scope, "Suspend");
		6344	irb->exec->coro_final_cleanup_block = ir_create_basic_block(irb, scope, "FinalCleanup");
		6345	}
		6346
5745	IrInstruction *result = ir_gen_node_extra(irb, node, scope, LVAL_NONE);	6347	IrInstruction *result = ir_gen_node_extra(irb, node, scope, LVAL_NONE);
5746	assert(result);	6348	assert(result);
5747	if (irb->exec->invalid)	6349	if (irb->exec->invalid)
5748	return false;	6350	return false;
5749		6351
5750	if (!instr_is_unreachable(result)) {	6352	if (!instr_is_unreachable(result)) {
5751	ir_mark_gen(ir_build_return(irb, scope, result->source_node, result));	6353	ir_gen_async_return(irb, scope, result->source_node, result, true);
		6354	}
		6355
		6356	if (is_async) {
		6357	IrBasicBlock *invalid_resume_block = ir_create_basic_block(irb, scope, "InvalidResume");
		6358	IrBasicBlock *check_free_block = ir_create_basic_block(irb, scope, "CheckFree");
		6359
		6360	ir_set_cursor_at_end_and_append_block(irb, irb->exec->coro_early_final);
		6361	IrInstruction *const_bool_true = ir_build_const_bool(irb, scope, node, true);
		6362	IrInstruction *suspend_code = ir_build_coro_suspend(irb, scope, node, nullptr, const_bool_true);
		6363	IrInstructionSwitchBrCase *cases = allocate<IrInstructionSwitchBrCase>(2);
		6364	cases[0].value = ir_build_const_u8(irb, scope, node, 0);
		6365	cases[0].block = invalid_resume_block;
		6366	cases[1].value = ir_build_const_u8(irb, scope, node, 1);
		6367	cases[1].block = irb->exec->coro_final_cleanup_block;
		6368	ir_build_switch_br(irb, scope, node, suspend_code, irb->exec->coro_suspend_block, 2, cases, const_bool_false);
		6369
		6370	ir_set_cursor_at_end_and_append_block(irb, irb->exec->coro_suspend_block);
		6371	ir_build_coro_end(irb, scope, node);
		6372	ir_build_return(irb, scope, node, irb->exec->coro_handle);
		6373
		6374	ir_set_cursor_at_end_and_append_block(irb, invalid_resume_block);
		6375	ir_build_unreachable(irb, scope, node);
		6376
		6377	ir_set_cursor_at_end_and_append_block(irb, irb->exec->coro_normal_final);
		6378	ir_build_br(irb, scope, node, check_free_block, const_bool_false);
		6379
		6380	ir_set_cursor_at_end_and_append_block(irb, irb->exec->coro_final_cleanup_block);
		6381	if (type_has_bits(return_type)) {
		6382	IrInstruction *result_ptr = ir_build_load_ptr(irb, scope, node, irb->exec->coro_result_ptr_field_ptr);
		6383	IrInstruction *result_ptr_as_u8_ptr = ir_build_ptr_cast(irb, scope, node, u8_ptr_type, result_ptr);
		6384	IrInstruction *return_value_ptr_as_u8_ptr = ir_build_ptr_cast(irb, scope, node, u8_ptr_type,
		6385	coro_result_field_ptr);
		6386	IrInstruction *return_type_inst = ir_build_const_type(irb, scope, node,
		6387	fn_entry->type_entry->data.fn.fn_type_id.return_type);
		6388	IrInstruction *size_of_ret_val = ir_build_size_of(irb, scope, node, return_type_inst);
		6389	ir_build_memcpy(irb, scope, node, result_ptr_as_u8_ptr, return_value_ptr_as_u8_ptr, size_of_ret_val);
		6390	}
		6391	ir_build_br(irb, scope, node, check_free_block, const_bool_false);
		6392
		6393	ir_set_cursor_at_end_and_append_block(irb, check_free_block);
		6394	IrBasicBlock *incoming_blocks = allocate<IrBasicBlock >(2);
		6395	IrInstruction *incoming_values = allocate<IrInstruction >(2);
		6396	incoming_blocks[0] = irb->exec->coro_final_cleanup_block;
		6397	incoming_values[0] = const_bool_false;
		6398	incoming_blocks[1] = irb->exec->coro_normal_final;
		6399	incoming_values[1] = const_bool_true;
		6400	IrInstruction *resume_awaiter = ir_build_phi(irb, scope, node, 2, incoming_blocks, incoming_values);
		6401
		6402	Buf *free_field_name = buf_create_from_str(ASYNC_FREE_FIELD_NAME);
		6403	IrInstruction *implicit_allocator_ptr = ir_build_get_implicit_allocator(irb, scope, node,
		6404	ImplicitAllocatorIdLocalVar);
		6405	IrInstruction *free_fn_ptr = ir_build_field_ptr(irb, scope, node, implicit_allocator_ptr, free_field_name);
		6406	IrInstruction *free_fn = ir_build_load_ptr(irb, scope, node, free_fn_ptr);
		6407	IrInstruction *zero = ir_build_const_usize(irb, scope, node, 0);
		6408	IrInstruction *coro_mem_ptr_maybe = ir_build_coro_free(irb, scope, node, coro_id, irb->exec->coro_handle);
		6409	IrInstruction *coro_mem_ptr = ir_build_ptr_cast(irb, scope, node, u8_ptr_type, coro_mem_ptr_maybe);
		6410	IrInstruction *coro_mem_ptr_ref = ir_build_ref(irb, scope, node, coro_mem_ptr, true, false);
		6411	IrInstruction *coro_size_ptr = ir_build_var_ptr(irb, scope, node, coro_size_var, true, false);
		6412	IrInstruction *coro_size = ir_build_load_ptr(irb, scope, node, coro_size_ptr);
		6413	IrInstruction *mem_slice = ir_build_slice(irb, scope, node, coro_mem_ptr_ref, zero, coro_size, false);
		6414	size_t arg_count = 2;
		6415	IrInstruction *args = allocate<IrInstruction >(arg_count);
		6416	args[0] = implicit_allocator_ptr; // self
		6417	args[1] = mem_slice; // old_mem
		6418	ir_build_call(irb, scope, node, nullptr, free_fn, arg_count, args, false, FnInlineAuto, false, nullptr);
		6419
		6420	IrBasicBlock *resume_block = ir_create_basic_block(irb, scope, "Resume");
		6421	ir_build_cond_br(irb, scope, node, resume_awaiter, resume_block, irb->exec->coro_suspend_block, const_bool_false);
		6422
		6423	ir_set_cursor_at_end_and_append_block(irb, resume_block);
		6424	IrInstruction *unwrapped_await_handle_ptr = ir_build_unwrap_maybe(irb, scope, node,
		6425	irb->exec->await_handle_var_ptr, false);
		6426	IrInstruction *awaiter_handle = ir_build_load_ptr(irb, scope, node, unwrapped_await_handle_ptr);
		6427	ir_build_coro_resume(irb, scope, node, awaiter_handle);
		6428	ir_build_br(irb, scope, node, irb->exec->coro_suspend_block, const_bool_false);
5752	}	6429	}
5753		6430
5754	return true;	6431	return true;
...	@@ -6705,6 +7382,12 @@ static ConstCastOnly types_match_const_cast_only(IrAnalyze *ira, TypeTableEntry	...	@@ -6705,6 +7382,12 @@ static ConstCastOnly types_match_const_cast_only(IrAnalyze *ira, TypeTableEntry
6705	return result;	7382	return result;
6706	}	7383	}
6707		7384
		7385	if (expected_type == ira->codegen->builtin_types.entry_promise &&
		7386	actual_type->id == TypeTableEntryIdPromise)
		7387	{
		7388	return result;
		7389	}
		7390
6708	// fn	7391	// fn
6709	if (expected_type->id == TypeTableEntryIdFn &&	7392	if (expected_type->id == TypeTableEntryIdFn &&
6710	actual_type->id == TypeTableEntryIdFn)	7393	actual_type->id == TypeTableEntryIdFn)
...	@@ -6736,6 +7419,16 @@ static ConstCastOnly types_match_const_cast_only(IrAnalyze *ira, TypeTableEntry	...	@@ -6736,6 +7419,16 @@ static ConstCastOnly types_match_const_cast_only(IrAnalyze *ira, TypeTableEntry
6736	return result;	7419	return result;
6737	}	7420	}
6738	}	7421	}
		7422	if (!expected_type->data.fn.is_generic && expected_type->data.fn.fn_type_id.cc == CallingConventionAsync) {
		7423	ConstCastOnly child = types_match_const_cast_only(ira, actual_type->data.fn.fn_type_id.async_allocator_type,
		7424	expected_type->data.fn.fn_type_id.async_allocator_type, source_node);
		7425	if (child.id != ConstCastResultIdOk) {
		7426	result.id = ConstCastResultIdAsyncAllocatorType;
		7427	result.data.async_allocator_type = allocate_nonzero<ConstCastOnly>(1);
		7428	*result.data.async_allocator_type = child;
		7429	return result;
		7430	}
		7431	}
6739	if (expected_type->data.fn.fn_type_id.param_count != actual_type->data.fn.fn_type_id.param_count) {	7432	if (expected_type->data.fn.fn_type_id.param_count != actual_type->data.fn.fn_type_id.param_count) {
6740	result.id = ConstCastResultIdFnArgCount;	7433	result.id = ConstCastResultIdFnArgCount;
6741	return result;	7434	return result;
...	@@ -8817,16 +9510,30 @@ static IrInstruction ir_analyze_cast(IrAnalyze ira, IrInstruction *source_inst	...	@@ -8817,16 +9510,30 @@ static IrInstruction ir_analyze_cast(IrAnalyze ira, IrInstruction *source_inst
8817		9510
8818	// explicit cast from child type of maybe type to maybe type	9511	// explicit cast from child type of maybe type to maybe type
8819	if (wanted_type->id == TypeTableEntryIdMaybe) {	9512	if (wanted_type->id == TypeTableEntryIdMaybe) {
8820	if (types_match_const_cast_only(ira, wanted_type->data.maybe.child_type, actual_type, source_node).id == ConstCastResultIdOk) {	9513	TypeTableEntry *wanted_child_type = wanted_type->data.maybe.child_type;
		9514	if (types_match_const_cast_only(ira, wanted_child_type, actual_type, source_node).id == ConstCastResultIdOk) {
8821	return ir_analyze_maybe_wrap(ira, source_instr, value, wanted_type);	9515	return ir_analyze_maybe_wrap(ira, source_instr, value, wanted_type);
8822	} else if (actual_type->id == TypeTableEntryIdNumLitInt \|\|	9516	} else if (actual_type->id == TypeTableEntryIdNumLitInt \|\|
8823	actual_type->id == TypeTableEntryIdNumLitFloat)	9517	actual_type->id == TypeTableEntryIdNumLitFloat)
8824	{	9518	{
8825	if (ir_num_lit_fits_in_other_type(ira, value, wanted_type->data.maybe.child_type, true)) {	9519	if (ir_num_lit_fits_in_other_type(ira, value, wanted_child_type, true)) {
8826	return ir_analyze_maybe_wrap(ira, source_instr, value, wanted_type);	9520	return ir_analyze_maybe_wrap(ira, source_instr, value, wanted_type);
8827	} else {	9521	} else {
8828	return ira->codegen->invalid_instruction;	9522	return ira->codegen->invalid_instruction;
8829	}	9523	}
		9524	} else if (wanted_child_type->id == TypeTableEntryIdPointer &&
		9525	wanted_child_type->data.pointer.is_const &&
		9526	(actual_type->id == TypeTableEntryIdPointer \|\| is_container(actual_type)))
		9527	{
		9528	IrInstruction *cast1 = ir_analyze_cast(ira, source_instr, wanted_child_type, value);
		9529	if (type_is_invalid(cast1->value.type))
		9530	return ira->codegen->invalid_instruction;
		9531
		9532	IrInstruction *cast2 = ir_analyze_cast(ira, source_instr, wanted_type, cast1);
		9533	if (type_is_invalid(cast2->value.type))
		9534	return ira->codegen->invalid_instruction;
		9535
		9536	return cast2;
8830	}	9537	}
8831	}	9538	}
8832		9539
...	@@ -9210,15 +9917,35 @@ static bool ir_resolve_comptime(IrAnalyze ira, IrInstruction value, bool *out)	...	@@ -9210,15 +9917,35 @@ static bool ir_resolve_comptime(IrAnalyze ira, IrInstruction value, bool *out)
9210	return ir_resolve_bool(ira, value, out);	9917	return ir_resolve_bool(ira, value, out);
9211	}	9918	}
9212		9919
9213	static bool ir_resolve_atomic_order(IrAnalyze ira, IrInstruction value, AtomicOrder *out) {	9920	static bool ir_resolve_atomic_order(IrAnalyze ira, IrInstruction value, AtomicOrder *out) {
		9921	if (type_is_invalid(value->value.type))
		9922	return false;
		9923
		9924	ConstExprValue *atomic_order_val = get_builtin_value(ira->codegen, "AtomicOrder");
		9925	assert(atomic_order_val->type->id == TypeTableEntryIdMetaType);
		9926	TypeTableEntry *atomic_order_type = atomic_order_val->data.x_type;
		9927
		9928	IrInstruction *casted_value = ir_implicit_cast(ira, value, atomic_order_type);
		9929	if (type_is_invalid(casted_value->value.type))
		9930	return false;
		9931
		9932	ConstExprValue *const_val = ir_resolve_const(ira, casted_value, UndefBad);
		9933	if (!const_val)
		9934	return false;
		9935
		9936	*out = (AtomicOrder)bigint_as_unsigned(&const_val->data.x_enum_tag);
		9937	return true;
		9938	}
		9939
		9940	static bool ir_resolve_atomic_rmw_op(IrAnalyze ira, IrInstruction value, AtomicRmwOp *out) {
9214	if (type_is_invalid(value->value.type))	9941	if (type_is_invalid(value->value.type))
9215	return false;	9942	return false;
9216		9943
9217	ConstExprValue *atomic_order_val = get_builtin_value(ira->codegen, "AtomicOrder");	9944	ConstExprValue *atomic_rmw_op_val = get_builtin_value(ira->codegen, "AtomicRmwOp");
9218	assert(atomic_order_val->type->id == TypeTableEntryIdMetaType);	9945	assert(atomic_rmw_op_val->type->id == TypeTableEntryIdMetaType);
9219	TypeTableEntry *atomic_order_type = atomic_order_val->data.x_type;	9946	TypeTableEntry *atomic_rmw_op_type = atomic_rmw_op_val->data.x_type;
9220		9947
9221	IrInstruction *casted_value = ir_implicit_cast(ira, value, atomic_order_type);	9948	IrInstruction *casted_value = ir_implicit_cast(ira, value, atomic_rmw_op_type);
9222	if (type_is_invalid(casted_value->value.type))	9949	if (type_is_invalid(casted_value->value.type))
9223	return false;	9950	return false;
9224		9951
...	@@ -9226,7 +9953,7 @@ static bool ir_resolve_atomic_order(IrAnalyze ira, IrInstruction value, Atomic	...	@@ -9226,7 +9953,7 @@ static bool ir_resolve_atomic_order(IrAnalyze ira, IrInstruction value, Atomic
9226	if (!const_val)	9953	if (!const_val)
9227	return false;	9954	return false;
9228		9955
9229	*out = (AtomicOrder)bigint_as_unsigned(&const_val->data.x_enum_tag);	9956	*out = (AtomicRmwOp)bigint_as_unsigned(&const_val->data.x_enum_tag);
9230	return true;	9957	return true;
9231	}	9958	}
9232		9959
...	@@ -9328,8 +10055,11 @@ static TypeTableEntry ir_analyze_instruction_return(IrAnalyze ira,	...	@@ -9328,8 +10055,11 @@ static TypeTableEntry ir_analyze_instruction_return(IrAnalyze ira,
9328	ir_add_error(ira, casted_value, buf_sprintf("function returns address of local variable"));	10055	ir_add_error(ira, casted_value, buf_sprintf("function returns address of local variable"));
9329	return ir_unreach_error(ira);	10056	return ir_unreach_error(ira);
9330	}	10057	}
9331	ir_build_return_from(&ira->new_irb, &return_instruction->base, casted_value);	10058	IrInstruction *result = ir_build_return(&ira->new_irb, return_instruction->base.scope,
9332	return ir_finish_anal(ira, ira->codegen->builtin_types.entry_unreachable);	10059	return_instruction->base.source_node, casted_value);
		10060	result->value.type = ira->codegen->builtin_types.entry_unreachable;
		10061	ir_link_new_instruction(result, &return_instruction->base);
		10062	return ir_finish_anal(ira, result->value.type);
9333	}	10063	}
9334		10064
9335	static TypeTableEntry ir_analyze_instruction_const(IrAnalyze ira, IrInstructionConst *const_instruction) {	10065	static TypeTableEntry ir_analyze_instruction_const(IrAnalyze ira, IrInstructionConst *const_instruction) {
...	@@ -9554,6 +10284,7 @@ static TypeTableEntry ir_analyze_bin_op_cmp(IrAnalyze ira, IrInstructionBinOp	...	@@ -9554,6 +10284,7 @@ static TypeTableEntry ir_analyze_bin_op_cmp(IrAnalyze ira, IrInstructionBinOp
9554	case TypeTableEntryIdBlock:	10284	case TypeTableEntryIdBlock:
9555	case TypeTableEntryIdBoundFn:	10285	case TypeTableEntryIdBoundFn:
9556	case TypeTableEntryIdArgTuple:	10286	case TypeTableEntryIdArgTuple:
		10287	case TypeTableEntryIdPromise:
9557	if (!is_equality_cmp) {	10288	if (!is_equality_cmp) {
9558	ir_add_error_node(ira, source_node,	10289	ir_add_error_node(ira, source_node,
9559	buf_sprintf("operator not allowed for type '%s'", buf_ptr(&resolved_type->name)));	10290	buf_sprintf("operator not allowed for type '%s'", buf_ptr(&resolved_type->name)));
...	@@ -10383,6 +11114,7 @@ static VarClassRequired get_var_class_required(TypeTableEntry *type_entry) {	...	@@ -10383,6 +11114,7 @@ static VarClassRequired get_var_class_required(TypeTableEntry *type_entry) {
10383	case TypeTableEntryIdVoid:	11114	case TypeTableEntryIdVoid:
10384	case TypeTableEntryIdErrorSet:	11115	case TypeTableEntryIdErrorSet:
10385	case TypeTableEntryIdFn:	11116	case TypeTableEntryIdFn:
		11117	case TypeTableEntryIdPromise:
10386	return VarClassRequiredAny;	11118	return VarClassRequiredAny;
10387	case TypeTableEntryIdNumLitFloat:	11119	case TypeTableEntryIdNumLitFloat:
10388	case TypeTableEntryIdNumLitInt:	11120	case TypeTableEntryIdNumLitInt:
...	@@ -10559,6 +11291,11 @@ static TypeTableEntry ir_analyze_instruction_export(IrAnalyze ira, IrInstructi	...	@@ -10559,6 +11291,11 @@ static TypeTableEntry ir_analyze_instruction_export(IrAnalyze ira, IrInstructi
10559	buf_sprintf("exported function must specify calling convention"));	11291	buf_sprintf("exported function must specify calling convention"));
10560	add_error_note(ira->codegen, msg, fn_entry->proto_node, buf_sprintf("declared here"));	11292	add_error_note(ira->codegen, msg, fn_entry->proto_node, buf_sprintf("declared here"));
10561	} break;	11293	} break;
		11294	case CallingConventionAsync: {
		11295	ErrorMsg *msg = ir_add_error(ira, target,
		11296	buf_sprintf("exported function cannot be async"));
		11297	add_error_note(ira->codegen, msg, fn_entry->proto_node, buf_sprintf("declared here"));
		11298	} break;
10562	case CallingConventionC:	11299	case CallingConventionC:
10563	case CallingConventionNaked:	11300	case CallingConventionNaked:
10564	case CallingConventionCold:	11301	case CallingConventionCold:
...	@@ -10648,6 +11385,7 @@ static TypeTableEntry ir_analyze_instruction_export(IrAnalyze ira, IrInstructi	...	@@ -10648,6 +11385,7 @@ static TypeTableEntry ir_analyze_instruction_export(IrAnalyze ira, IrInstructi
10648	case TypeTableEntryIdBoundFn:	11385	case TypeTableEntryIdBoundFn:
10649	case TypeTableEntryIdArgTuple:	11386	case TypeTableEntryIdArgTuple:
10650	case TypeTableEntryIdOpaque:	11387	case TypeTableEntryIdOpaque:
		11388	case TypeTableEntryIdPromise:
10651	ir_add_error(ira, target,	11389	ir_add_error(ira, target,
10652	buf_sprintf("invalid export target '%s'", buf_ptr(&type_value->name)));	11390	buf_sprintf("invalid export target '%s'", buf_ptr(&type_value->name)));
10653	break;	11391	break;
...	@@ -10672,6 +11410,7 @@ static TypeTableEntry ir_analyze_instruction_export(IrAnalyze ira, IrInstructi	...	@@ -10672,6 +11410,7 @@ static TypeTableEntry ir_analyze_instruction_export(IrAnalyze ira, IrInstructi
10672	case TypeTableEntryIdBoundFn:	11410	case TypeTableEntryIdBoundFn:
10673	case TypeTableEntryIdArgTuple:	11411	case TypeTableEntryIdArgTuple:
10674	case TypeTableEntryIdOpaque:	11412	case TypeTableEntryIdOpaque:
		11413	case TypeTableEntryIdPromise:
10675	ir_add_error(ira, target,	11414	ir_add_error(ira, target,
10676	buf_sprintf("invalid export target type '%s'", buf_ptr(&target->value.type->name)));	11415	buf_sprintf("invalid export target type '%s'", buf_ptr(&target->value.type->name)));
10677	break;	11416	break;
...	@@ -10724,6 +11463,81 @@ static TypeTableEntry ir_analyze_instruction_error_union(IrAnalyze ira,	...	@@ -10724,6 +11463,81 @@ static TypeTableEntry ir_analyze_instruction_error_union(IrAnalyze ira,
10724	return ira->codegen->builtin_types.entry_type;	11463	return ira->codegen->builtin_types.entry_type;
10725	}	11464	}
10726		11465
		11466	IrInstruction ir_get_implicit_allocator(IrAnalyze ira, IrInstruction *source_instr, ImplicitAllocatorId id) {
		11467	FnTableEntry *parent_fn_entry = exec_fn_entry(ira->new_irb.exec);
		11468	if (parent_fn_entry == nullptr) {
		11469	ir_add_error(ira, source_instr, buf_sprintf("no implicit allocator available"));
		11470	return ira->codegen->invalid_instruction;
		11471	}
		11472
		11473	FnTypeId *parent_fn_type = &parent_fn_entry->type_entry->data.fn.fn_type_id;
		11474	if (parent_fn_type->cc != CallingConventionAsync) {
		11475	ir_add_error(ira, source_instr, buf_sprintf("async function call from non-async caller requires allocator parameter"));
		11476	return ira->codegen->invalid_instruction;
		11477	}
		11478
		11479	assert(parent_fn_type->async_allocator_type != nullptr);
		11480
		11481	switch (id) {
		11482	case ImplicitAllocatorIdArg:
		11483	{
		11484	IrInstruction *result = ir_build_get_implicit_allocator(&ira->new_irb, source_instr->scope,
		11485	source_instr->source_node, ImplicitAllocatorIdArg);
		11486	result->value.type = parent_fn_type->async_allocator_type;
		11487	return result;
		11488	}
		11489	case ImplicitAllocatorIdLocalVar:
		11490	{
		11491	VariableTableEntry *coro_allocator_var = ira->old_irb.exec->coro_allocator_var;
		11492	assert(coro_allocator_var != nullptr);
		11493	IrInstruction *var_ptr_inst = ir_get_var_ptr(ira, source_instr, coro_allocator_var, true, false);
		11494	IrInstruction *result = ir_get_deref(ira, source_instr, var_ptr_inst);
		11495	assert(result->value.type != nullptr);
		11496	return result;
		11497	}
		11498	}
		11499	zig_unreachable();
		11500	}
		11501
		11502	static IrInstruction ir_analyze_async_call(IrAnalyze ira, IrInstructionCall call_instruction, FnTableEntry fn_entry, TypeTableEntry *fn_type,
		11503	IrInstruction fn_ref, IrInstruction casted_args, size_t arg_count, IrInstruction async_allocator_inst)
		11504	{
		11505	Buf *alloc_field_name = buf_create_from_str(ASYNC_ALLOC_FIELD_NAME);
		11506	//Buf *free_field_name = buf_create_from_str("freeFn");
		11507	assert(async_allocator_inst->value.type->id == TypeTableEntryIdPointer);
		11508	TypeTableEntry *container_type = async_allocator_inst->value.type->data.pointer.child_type;
		11509	IrInstruction *field_ptr_inst = ir_analyze_container_field_ptr(ira, alloc_field_name, &call_instruction->base,
		11510	async_allocator_inst, container_type);
		11511	if (type_is_invalid(field_ptr_inst->value.type)) {
		11512	return ira->codegen->invalid_instruction;
		11513	}
		11514	TypeTableEntry *ptr_to_alloc_fn_type = field_ptr_inst->value.type;
		11515	assert(ptr_to_alloc_fn_type->id == TypeTableEntryIdPointer);
		11516
		11517	TypeTableEntry *alloc_fn_type = ptr_to_alloc_fn_type->data.pointer.child_type;
		11518	if (alloc_fn_type->id != TypeTableEntryIdFn) {
		11519	ir_add_error(ira, &call_instruction->base,
		11520	buf_sprintf("expected allocation function, found '%s'", buf_ptr(&alloc_fn_type->name)));
		11521	return ira->codegen->invalid_instruction;
		11522	}
		11523
		11524	TypeTableEntry *alloc_fn_return_type = alloc_fn_type->data.fn.fn_type_id.return_type;
		11525	if (alloc_fn_return_type->id != TypeTableEntryIdErrorUnion) {
		11526	ir_add_error(ira, fn_ref,
		11527	buf_sprintf("expected allocation function to return error union, but it returns '%s'", buf_ptr(&alloc_fn_return_type->name)));
		11528	return ira->codegen->invalid_instruction;
		11529	}
		11530	TypeTableEntry *alloc_fn_error_set_type = alloc_fn_return_type->data.error_union.err_set_type;
		11531	TypeTableEntry *return_type = fn_type->data.fn.fn_type_id.return_type;
		11532	TypeTableEntry *promise_type = get_promise_type(ira->codegen, return_type);
		11533	TypeTableEntry *async_return_type = get_error_union_type(ira->codegen, alloc_fn_error_set_type, promise_type);
		11534
		11535	IrInstruction *result = ir_build_call(&ira->new_irb, call_instruction->base.scope, call_instruction->base.source_node,
		11536	fn_entry, fn_ref, arg_count, casted_args, false, FnInlineAuto, true, async_allocator_inst);
		11537	result->value.type = async_return_type;
		11538	return result;
		11539	}
		11540
10727	static bool ir_analyze_fn_call_inline_arg(IrAnalyze ira, AstNode fn_proto_node,	11541	static bool ir_analyze_fn_call_inline_arg(IrAnalyze ira, AstNode fn_proto_node,
10728	IrInstruction arg, Scope exec_scope, size_t next_proto_i)	11542	IrInstruction arg, Scope exec_scope, size_t next_proto_i)
10729	{	11543	{
...	@@ -10938,6 +11752,20 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal	...	@@ -10938,6 +11752,20 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal
10938	}	11752	}
10939	return ira->codegen->builtin_types.entry_invalid;	11753	return ira->codegen->builtin_types.entry_invalid;
10940	}	11754	}
		11755	if (fn_type_id->cc == CallingConventionAsync && !call_instruction->is_async) {
		11756	ErrorMsg *msg = ir_add_error(ira, fn_ref, buf_sprintf("must use async keyword to call async function"));
		11757	if (fn_proto_node) {
		11758	add_error_note(ira->codegen, msg, fn_proto_node, buf_sprintf("declared here"));
		11759	}
		11760	return ira->codegen->builtin_types.entry_invalid;
		11761	}
		11762	if (fn_type_id->cc != CallingConventionAsync && call_instruction->is_async) {
		11763	ErrorMsg *msg = ir_add_error(ira, fn_ref, buf_sprintf("cannot use async keyword to call non-async function"));
		11764	if (fn_proto_node) {
		11765	add_error_note(ira->codegen, msg, fn_proto_node, buf_sprintf("declared here"));
		11766	}
		11767	return ira->codegen->builtin_types.entry_invalid;
		11768	}
10941		11769
10942		11770
10943	if (fn_type_id->is_var_args) {	11771	if (fn_type_id->is_var_args) {
...	@@ -11064,6 +11892,11 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal	...	@@ -11064,6 +11892,11 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal
11064	buf_sprintf("calling a generic function requires compile-time known function value"));	11892	buf_sprintf("calling a generic function requires compile-time known function value"));
11065	return ira->codegen->builtin_types.entry_invalid;	11893	return ira->codegen->builtin_types.entry_invalid;
11066	}	11894	}
		11895	if (call_instruction->is_async && fn_type_id->is_var_args) {
		11896	ir_add_error(ira, call_instruction->fn_ref,
		11897	buf_sprintf("compiler bug: TODO: implement var args async functions. https://github.com/zig-lang/zig/issues/557"));
		11898	return ira->codegen->builtin_types.entry_invalid;
		11899	}
11067		11900
11068	// Count the arguments of the function type id we are creating	11901	// Count the arguments of the function type id we are creating
11069	size_t new_fn_arg_count = first_arg_1_or_0;	11902	size_t new_fn_arg_count = first_arg_1_or_0;
...	@@ -11212,6 +12045,36 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal	...	@@ -11212,6 +12045,36 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal
11212	return ir_analyze_fn_call(ira, call_instruction, fn_entry, fn_type, fn_ref, first_arg_ptr, true, FnInlineAuto);	12045	return ir_analyze_fn_call(ira, call_instruction, fn_entry, fn_type, fn_ref, first_arg_ptr, true, FnInlineAuto);
11213	}	12046	}
11214	}	12047	}
		12048	IrInstruction *async_allocator_inst = nullptr;
		12049	if (call_instruction->is_async) {
		12050	AstNode *async_allocator_type_node = fn_proto_node->data.fn_proto.async_allocator_type;
		12051	if (async_allocator_type_node != nullptr) {
		12052	TypeTableEntry *async_allocator_type = analyze_type_expr(ira->codegen, impl_fn->child_scope, async_allocator_type_node);
		12053	if (type_is_invalid(async_allocator_type))
		12054	return ira->codegen->builtin_types.entry_invalid;
		12055	inst_fn_type_id.async_allocator_type = async_allocator_type;
		12056	}
		12057	IrInstruction *uncasted_async_allocator_inst;
		12058	if (call_instruction->async_allocator == nullptr) {
		12059	uncasted_async_allocator_inst = ir_get_implicit_allocator(ira, &call_instruction->base,
		12060	ImplicitAllocatorIdLocalVar);
		12061	if (type_is_invalid(uncasted_async_allocator_inst->value.type))
		12062	return ira->codegen->builtin_types.entry_invalid;
		12063	} else {
		12064	uncasted_async_allocator_inst = call_instruction->async_allocator->other;
		12065	if (type_is_invalid(uncasted_async_allocator_inst->value.type))
		12066	return ira->codegen->builtin_types.entry_invalid;
		12067	}
		12068	if (inst_fn_type_id.async_allocator_type == nullptr) {
		12069	IrInstruction *casted_inst = ir_implicit_byval_const_ref_cast(ira, uncasted_async_allocator_inst);
		12070	if (type_is_invalid(casted_inst->value.type))
		12071	return ira->codegen->builtin_types.entry_invalid;
		12072	inst_fn_type_id.async_allocator_type = casted_inst->value.type;
		12073	}
		12074	async_allocator_inst = ir_implicit_cast(ira, uncasted_async_allocator_inst, inst_fn_type_id.async_allocator_type);
		12075	if (type_is_invalid(async_allocator_inst->value.type))
		12076	return ira->codegen->builtin_types.entry_invalid;
		12077	}
11215		12078
11216	auto existing_entry = ira->codegen->generic_table.put_unique(generic_id, impl_fn);	12079	auto existing_entry = ira->codegen->generic_table.put_unique(generic_id, impl_fn);
11217	if (existing_entry) {	12080	if (existing_entry) {
...	@@ -11231,24 +12094,34 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal	...	@@ -11231,24 +12094,34 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal
11231	ira->codegen->fn_defs.append(impl_fn);	12094	ira->codegen->fn_defs.append(impl_fn);
11232	}	12095	}
11233		12096
		12097	TypeTableEntry *return_type = impl_fn->type_entry->data.fn.fn_type_id.return_type;
		12098	if (fn_type_can_fail(&impl_fn->type_entry->data.fn.fn_type_id)) {
		12099	parent_fn_entry->calls_errorable_function = true;
		12100	}
		12101
11234	size_t impl_param_count = impl_fn->type_entry->data.fn.fn_type_id.param_count;	12102	size_t impl_param_count = impl_fn->type_entry->data.fn.fn_type_id.param_count;
		12103	if (call_instruction->is_async) {
		12104	IrInstruction *result = ir_analyze_async_call(ira, call_instruction, impl_fn, impl_fn->type_entry, fn_ref, casted_args, impl_param_count,
		12105	async_allocator_inst);
		12106	ir_link_new_instruction(result, &call_instruction->base);
		12107	ir_add_alloca(ira, result, result->value.type);
		12108	return ir_finish_anal(ira, result->value.type);
		12109	}
		12110
		12111	assert(async_allocator_inst == nullptr);
11235	IrInstruction *new_call_instruction = ir_build_call_from(&ira->new_irb, &call_instruction->base,	12112	IrInstruction *new_call_instruction = ir_build_call_from(&ira->new_irb, &call_instruction->base,
11236	impl_fn, nullptr, impl_param_count, casted_args, false, fn_inline);	12113	impl_fn, nullptr, impl_param_count, casted_args, false, fn_inline,
		12114	call_instruction->is_async, nullptr);
11237		12115
11238	TypeTableEntry *return_type = impl_fn->type_entry->data.fn.fn_type_id.return_type;
11239	ir_add_alloca(ira, new_call_instruction, return_type);	12116	ir_add_alloca(ira, new_call_instruction, return_type);
11240		12117
11241	if (return_type->id == TypeTableEntryIdErrorSet \|\| return_type->id == TypeTableEntryIdErrorUnion) {
11242	parent_fn_entry->calls_errorable_function = true;
11243	}
11244
11245	return ir_finish_anal(ira, return_type);	12118	return ir_finish_anal(ira, return_type);
11246	}	12119	}
11247		12120
11248	FnTableEntry *parent_fn_entry = exec_fn_entry(ira->new_irb.exec);	12121	FnTableEntry *parent_fn_entry = exec_fn_entry(ira->new_irb.exec);
11249	assert(fn_type_id->return_type != nullptr);	12122	assert(fn_type_id->return_type != nullptr);
11250	assert(parent_fn_entry != nullptr);	12123	assert(parent_fn_entry != nullptr);
11251	if (fn_type_id->return_type->id == TypeTableEntryIdErrorSet \|\| fn_type_id->return_type->id == TypeTableEntryIdErrorUnion) {	12124	if (fn_type_can_fail(fn_type_id)) {
11252	parent_fn_entry->calls_errorable_function = true;	12125	parent_fn_entry->calls_errorable_function = true;
11253	}	12126	}
11254		12127
...	@@ -11303,8 +12176,33 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal	...	@@ -11303,8 +12176,33 @@ static TypeTableEntry ir_analyze_fn_call(IrAnalyze ira, IrInstructionCall *cal
11303	if (type_is_invalid(return_type))	12176	if (type_is_invalid(return_type))
11304	return ira->codegen->builtin_types.entry_invalid;	12177	return ira->codegen->builtin_types.entry_invalid;
11305		12178
		12179	if (call_instruction->is_async) {
		12180	IrInstruction *uncasted_async_allocator_inst;
		12181	if (call_instruction->async_allocator == nullptr) {
		12182	uncasted_async_allocator_inst = ir_get_implicit_allocator(ira, &call_instruction->base,
		12183	ImplicitAllocatorIdLocalVar);
		12184	if (type_is_invalid(uncasted_async_allocator_inst->value.type))
		12185	return ira->codegen->builtin_types.entry_invalid;
		12186	} else {
		12187	uncasted_async_allocator_inst = call_instruction->async_allocator->other;
		12188	if (type_is_invalid(uncasted_async_allocator_inst->value.type))
		12189	return ira->codegen->builtin_types.entry_invalid;
		12190
		12191	}
		12192	IrInstruction *async_allocator_inst = ir_implicit_cast(ira, uncasted_async_allocator_inst, fn_type_id->async_allocator_type);
		12193	if (type_is_invalid(async_allocator_inst->value.type))
		12194	return ira->codegen->builtin_types.entry_invalid;
		12195
		12196	IrInstruction *result = ir_analyze_async_call(ira, call_instruction, fn_entry, fn_type, fn_ref, casted_args, call_param_count,
		12197	async_allocator_inst);
		12198	ir_link_new_instruction(result, &call_instruction->base);
		12199	ir_add_alloca(ira, result, result->value.type);
		12200	return ir_finish_anal(ira, result->value.type);
		12201	}
		12202
		12203
11306	IrInstruction *new_call_instruction = ir_build_call_from(&ira->new_irb, &call_instruction->base,	12204	IrInstruction *new_call_instruction = ir_build_call_from(&ira->new_irb, &call_instruction->base,
11307	fn_entry, fn_ref, call_param_count, casted_args, false, fn_inline);	12205	fn_entry, fn_ref, call_param_count, casted_args, false, fn_inline, false, nullptr);
11308		12206
11309	ir_add_alloca(ira, new_call_instruction, return_type);	12207	ir_add_alloca(ira, new_call_instruction, return_type);
11310	return ir_finish_anal(ira, return_type);	12208	return ir_finish_anal(ira, return_type);
...	@@ -11430,6 +12328,7 @@ static TypeTableEntry ir_analyze_maybe(IrAnalyze ira, IrInstructionUnOp *un_op	...	@@ -11430,6 +12328,7 @@ static TypeTableEntry ir_analyze_maybe(IrAnalyze ira, IrInstructionUnOp *un_op
11430	case TypeTableEntryIdBlock:	12328	case TypeTableEntryIdBlock:
11431	case TypeTableEntryIdBoundFn:	12329	case TypeTableEntryIdBoundFn:
11432	case TypeTableEntryIdArgTuple:	12330	case TypeTableEntryIdArgTuple:
		12331	case TypeTableEntryIdPromise:
11433	{	12332	{
11434	ConstExprValue *out_val = ir_build_const_from(ira, &un_op_instruction->base);	12333	ConstExprValue *out_val = ir_build_const_from(ira, &un_op_instruction->base);
11435	out_val->data.x_type = get_maybe_type(ira->codegen, type_entry);	12334	out_val->data.x_type = get_maybe_type(ira->codegen, type_entry);
...	@@ -11998,8 +12897,8 @@ static TypeTableEntry ir_analyze_instruction_elem_ptr(IrAnalyze ira, IrInstruc	...	@@ -11998,8 +12897,8 @@ static TypeTableEntry ir_analyze_instruction_elem_ptr(IrAnalyze ira, IrInstruc
11998	return return_type;	12897	return return_type;
11999	}	12898	}
12000		12899
12001	static TypeTableEntry ir_analyze_container_member_access_inner(IrAnalyze ira,	12900	static IrInstruction ir_analyze_container_member_access_inner(IrAnalyze ira,
12002	TypeTableEntry bare_struct_type, Buf field_name, IrInstructionFieldPtr *field_ptr_instruction,	12901	TypeTableEntry bare_struct_type, Buf field_name, IrInstruction *source_instr,
12003	IrInstruction container_ptr, TypeTableEntry container_type)	12902	IrInstruction container_ptr, TypeTableEntry container_type)
12004	{	12903	{
12005	if (!is_slice(bare_struct_type)) {	12904	if (!is_slice(bare_struct_type)) {
...	@@ -12007,17 +12906,17 @@ static TypeTableEntry ir_analyze_container_member_access_inner(IrAnalyze ira,	...	@@ -12007,17 +12906,17 @@ static TypeTableEntry ir_analyze_container_member_access_inner(IrAnalyze ira,
12007	auto entry = container_scope->decl_table.maybe_get(field_name);	12906	auto entry = container_scope->decl_table.maybe_get(field_name);
12008	Tld *tld = entry ? entry->value : nullptr;	12907	Tld *tld = entry ? entry->value : nullptr;
12009	if (tld && tld->id == TldIdFn) {	12908	if (tld && tld->id == TldIdFn) {
12010	resolve_top_level_decl(ira->codegen, tld, false, field_ptr_instruction->base.source_node);	12909	resolve_top_level_decl(ira->codegen, tld, false, source_instr->source_node);
12011	if (tld->resolution == TldResolutionInvalid)	12910	if (tld->resolution == TldResolutionInvalid)
12012	return ira->codegen->builtin_types.entry_invalid;	12911	return ira->codegen->invalid_instruction;
12013	TldFn tld_fn = (TldFn )tld;	12912	TldFn tld_fn = (TldFn )tld;
12014	FnTableEntry *fn_entry = tld_fn->fn_entry;	12913	FnTableEntry *fn_entry = tld_fn->fn_entry;
12015	if (type_is_invalid(fn_entry->type_entry))	12914	if (type_is_invalid(fn_entry->type_entry))
12016	return ira->codegen->builtin_types.entry_invalid;	12915	return ira->codegen->invalid_instruction;
12017		12916
12018	IrInstruction *bound_fn_value = ir_build_const_bound_fn(&ira->new_irb, field_ptr_instruction->base.scope,	12917	IrInstruction *bound_fn_value = ir_build_const_bound_fn(&ira->new_irb, source_instr->scope,
12019	field_ptr_instruction->base.source_node, fn_entry, container_ptr);	12918	source_instr->source_node, fn_entry, container_ptr);
12020	return ir_analyze_ref(ira, &field_ptr_instruction->base, bound_fn_value, true, false);	12919	return ir_get_ref(ira, source_instr, bound_fn_value, true, false);
12021	}	12920	}
12022	}	12921	}
12023	const char *prefix_name;	12922	const char *prefix_name;
...	@@ -12032,19 +12931,19 @@ static TypeTableEntry ir_analyze_container_member_access_inner(IrAnalyze ira,	...	@@ -12032,19 +12931,19 @@ static TypeTableEntry ir_analyze_container_member_access_inner(IrAnalyze ira,
12032	} else {	12931	} else {
12033	prefix_name = "";	12932	prefix_name = "";
12034	}	12933	}
12035	ir_add_error_node(ira, field_ptr_instruction->base.source_node,	12934	ir_add_error_node(ira, source_instr->source_node,
12036	buf_sprintf("no member named '%s' in %s'%s'", buf_ptr(field_name), prefix_name, buf_ptr(&bare_struct_type->name)));	12935	buf_sprintf("no member named '%s' in %s'%s'", buf_ptr(field_name), prefix_name, buf_ptr(&bare_struct_type->name)));
12037	return ira->codegen->builtin_types.entry_invalid;	12936	return ira->codegen->invalid_instruction;
12038	}	12937	}
12039		12938
12040		12939
12041	static TypeTableEntry ir_analyze_container_field_ptr(IrAnalyze ira, Buf *field_name,	12940	static IrInstruction ir_analyze_container_field_ptr(IrAnalyze ira, Buf *field_name,
12042	IrInstructionFieldPtr field_ptr_instruction, IrInstruction container_ptr, TypeTableEntry *container_type)	12941	IrInstruction source_instr, IrInstruction container_ptr, TypeTableEntry *container_type)
12043	{	12942	{
12044	TypeTableEntry *bare_type = container_ref_type(container_type);	12943	TypeTableEntry *bare_type = container_ref_type(container_type);
12045	ensure_complete_type(ira->codegen, bare_type);	12944	ensure_complete_type(ira->codegen, bare_type);
12046	if (type_is_invalid(bare_type))	12945	if (type_is_invalid(bare_type))
12047	return ira->codegen->builtin_types.entry_invalid;	12946	return ira->codegen->invalid_instruction;
12048		12947
12049	assert(container_ptr->value.type->id == TypeTableEntryIdPointer);	12948	assert(container_ptr->value.type->id == TypeTableEntryIdPointer);
12050	bool is_const = container_ptr->value.type->data.pointer.is_const;	12949	bool is_const = container_ptr->value.type->data.pointer.is_const;
...	@@ -12061,46 +12960,51 @@ static TypeTableEntry ir_analyze_container_field_ptr(IrAnalyze ira, Buf *field	...	@@ -12061,46 +12960,51 @@ static TypeTableEntry ir_analyze_container_field_ptr(IrAnalyze ira, Buf *field
12061	if (instr_is_comptime(container_ptr)) {	12960	if (instr_is_comptime(container_ptr)) {
12062	ConstExprValue *ptr_val = ir_resolve_const(ira, container_ptr, UndefBad);	12961	ConstExprValue *ptr_val = ir_resolve_const(ira, container_ptr, UndefBad);
12063	if (!ptr_val)	12962	if (!ptr_val)
12064	return ira->codegen->builtin_types.entry_invalid;	12963	return ira->codegen->invalid_instruction;
12065		12964
12066	if (ptr_val->data.x_ptr.special != ConstPtrSpecialHardCodedAddr) {	12965	if (ptr_val->data.x_ptr.special != ConstPtrSpecialHardCodedAddr) {
12067	ConstExprValue *struct_val = const_ptr_pointee(ira->codegen, ptr_val);	12966	ConstExprValue *struct_val = const_ptr_pointee(ira->codegen, ptr_val);
12068	if (type_is_invalid(struct_val->type))	12967	if (type_is_invalid(struct_val->type))
12069	return ira->codegen->builtin_types.entry_invalid;	12968	return ira->codegen->invalid_instruction;
12070	ConstExprValue *field_val = &struct_val->data.x_struct.fields[field->src_index];	12969	ConstExprValue *field_val = &struct_val->data.x_struct.fields[field->src_index];
12071	TypeTableEntry *ptr_type = get_pointer_to_type_extra(ira->codegen, field_val->type,	12970	TypeTableEntry *ptr_type = get_pointer_to_type_extra(ira->codegen, field_val->type,
12072	is_const, is_volatile, align_bytes,	12971	is_const, is_volatile, align_bytes,
12073	(uint32_t)(ptr_bit_offset + field->packed_bits_offset),	12972	(uint32_t)(ptr_bit_offset + field->packed_bits_offset),
12074	(uint32_t)unaligned_bit_count_for_result_type);	12973	(uint32_t)unaligned_bit_count_for_result_type);
12075	ConstExprValue *const_val = ir_build_const_from(ira, &field_ptr_instruction->base);	12974	IrInstruction *result = ir_get_const(ira, source_instr);
		12975	ConstExprValue *const_val = &result->value;
12076	const_val->data.x_ptr.special = ConstPtrSpecialBaseStruct;	12976	const_val->data.x_ptr.special = ConstPtrSpecialBaseStruct;
12077	const_val->data.x_ptr.mut = container_ptr->value.data.x_ptr.mut;	12977	const_val->data.x_ptr.mut = container_ptr->value.data.x_ptr.mut;
12078	const_val->data.x_ptr.data.base_struct.struct_val = struct_val;	12978	const_val->data.x_ptr.data.base_struct.struct_val = struct_val;
12079	const_val->data.x_ptr.data.base_struct.field_index = field->src_index;	12979	const_val->data.x_ptr.data.base_struct.field_index = field->src_index;
12080	return ptr_type;	12980	const_val->type = ptr_type;
		12981	return result;
12081	}	12982	}
12082	}	12983	}
12083	ir_build_struct_field_ptr_from(&ira->new_irb, &field_ptr_instruction->base, container_ptr, field);	12984	IrInstruction *result = ir_build_struct_field_ptr(&ira->new_irb, source_instr->scope, source_instr->source_node,
12084	return get_pointer_to_type_extra(ira->codegen, field->type_entry, is_const, is_volatile,	12985	container_ptr, field);
		12986	result->value.type = get_pointer_to_type_extra(ira->codegen, field->type_entry, is_const, is_volatile,
12085	align_bytes,	12987	align_bytes,
12086	(uint32_t)(ptr_bit_offset + field->packed_bits_offset),	12988	(uint32_t)(ptr_bit_offset + field->packed_bits_offset),
12087	(uint32_t)unaligned_bit_count_for_result_type);	12989	(uint32_t)unaligned_bit_count_for_result_type);
		12990	return result;
12088	} else {	12991	} else {
12089	return ir_analyze_container_member_access_inner(ira, bare_type, field_name,	12992	return ir_analyze_container_member_access_inner(ira, bare_type, field_name,
12090	field_ptr_instruction, container_ptr, container_type);	12993	source_instr, container_ptr, container_type);
12091	}	12994	}
12092	} else if (bare_type->id == TypeTableEntryIdEnum) {	12995	} else if (bare_type->id == TypeTableEntryIdEnum) {
12093	return ir_analyze_container_member_access_inner(ira, bare_type, field_name,	12996	return ir_analyze_container_member_access_inner(ira, bare_type, field_name,
12094	field_ptr_instruction, container_ptr, container_type);	12997	source_instr, container_ptr, container_type);
12095	} else if (bare_type->id == TypeTableEntryIdUnion) {	12998	} else if (bare_type->id == TypeTableEntryIdUnion) {
12096	TypeUnionField *field = find_union_type_field(bare_type, field_name);	12999	TypeUnionField *field = find_union_type_field(bare_type, field_name);
12097	if (field) {	13000	if (field) {
12098	ir_build_union_field_ptr_from(&ira->new_irb, &field_ptr_instruction->base, container_ptr, field);	13001	IrInstruction *result = ir_build_union_field_ptr(&ira->new_irb, source_instr->scope, source_instr->source_node, container_ptr, field);
12099	return get_pointer_to_type_extra(ira->codegen, field->type_entry, is_const, is_volatile,	13002	result->value.type = get_pointer_to_type_extra(ira->codegen, field->type_entry, is_const, is_volatile,
12100	get_abi_alignment(ira->codegen, field->type_entry), 0, 0);	13003	get_abi_alignment(ira->codegen, field->type_entry), 0, 0);
		13004	return result;
12101	} else {	13005	} else {
12102	return ir_analyze_container_member_access_inner(ira, bare_type, field_name,	13006	return ir_analyze_container_member_access_inner(ira, bare_type, field_name,
12103	field_ptr_instruction, container_ptr, container_type);	13007	source_instr, container_ptr, container_type);
12104	}	13008	}
12105	} else {	13009	} else {
12106	zig_unreachable();	13010	zig_unreachable();
...	@@ -12210,9 +13114,13 @@ static TypeTableEntry ir_analyze_instruction_field_ptr(IrAnalyze ira, IrInstru	...	@@ -12210,9 +13114,13 @@ static TypeTableEntry ir_analyze_instruction_field_ptr(IrAnalyze ira, IrInstru
12210	if (container_type->id == TypeTableEntryIdPointer) {	13114	if (container_type->id == TypeTableEntryIdPointer) {
12211	TypeTableEntry *bare_type = container_ref_type(container_type);	13115	TypeTableEntry *bare_type = container_ref_type(container_type);
12212	IrInstruction *container_child = ir_get_deref(ira, &field_ptr_instruction->base, container_ptr);	13116	IrInstruction *container_child = ir_get_deref(ira, &field_ptr_instruction->base, container_ptr);
12213	return ir_analyze_container_field_ptr(ira, field_name, field_ptr_instruction, container_child, bare_type);	13117	IrInstruction *result = ir_analyze_container_field_ptr(ira, field_name, &field_ptr_instruction->base, container_child, bare_type);
		13118	ir_link_new_instruction(result, &field_ptr_instruction->base);
		13119	return result->value.type;
12214	} else {	13120	} else {
12215	return ir_analyze_container_field_ptr(ira, field_name, field_ptr_instruction, container_ptr, container_type);	13121	IrInstruction *result = ir_analyze_container_field_ptr(ira, field_name, &field_ptr_instruction->base, container_ptr, container_type);
		13122	ir_link_new_instruction(result, &field_ptr_instruction->base);
		13123	return result->value.type;
12216	}	13124	}
12217	} else if (container_type->id == TypeTableEntryIdArray) {	13125	} else if (container_type->id == TypeTableEntryIdArray) {
12218	if (buf_eql_str(field_name, "len")) {	13126	if (buf_eql_str(field_name, "len")) {
...	@@ -12659,6 +13567,7 @@ static TypeTableEntry ir_analyze_instruction_typeof(IrAnalyze ira, IrInstructi	...	@@ -12659,6 +13567,7 @@ static TypeTableEntry ir_analyze_instruction_typeof(IrAnalyze ira, IrInstructi
12659	case TypeTableEntryIdFn:	13567	case TypeTableEntryIdFn:
12660	case TypeTableEntryIdArgTuple:	13568	case TypeTableEntryIdArgTuple:
12661	case TypeTableEntryIdOpaque:	13569	case TypeTableEntryIdOpaque:
		13570	case TypeTableEntryIdPromise:
12662	{	13571	{
12663	ConstExprValue *out_val = ir_build_const_from(ira, &typeof_instruction->base);	13572	ConstExprValue *out_val = ir_build_const_from(ira, &typeof_instruction->base);
12664	out_val->data.x_type = type_entry;	13573	out_val->data.x_type = type_entry;
...	@@ -12926,6 +13835,7 @@ static TypeTableEntry ir_analyze_instruction_slice_type(IrAnalyze ira,	...	@@ -12926,6 +13835,7 @@ static TypeTableEntry ir_analyze_instruction_slice_type(IrAnalyze ira,
12926	case TypeTableEntryIdFn:	13835	case TypeTableEntryIdFn:
12927	case TypeTableEntryIdNamespace:	13836	case TypeTableEntryIdNamespace:
12928	case TypeTableEntryIdBoundFn:	13837	case TypeTableEntryIdBoundFn:
		13838	case TypeTableEntryIdPromise:
12929	{	13839	{
12930	type_ensure_zero_bits_known(ira->codegen, child_type);	13840	type_ensure_zero_bits_known(ira->codegen, child_type);
12931	TypeTableEntry *slice_ptr_type = get_pointer_to_type_extra(ira->codegen, child_type,	13841	TypeTableEntry *slice_ptr_type = get_pointer_to_type_extra(ira->codegen, child_type,
...	@@ -13034,6 +13944,7 @@ static TypeTableEntry ir_analyze_instruction_array_type(IrAnalyze ira,	...	@@ -13034,6 +13944,7 @@ static TypeTableEntry ir_analyze_instruction_array_type(IrAnalyze ira,
13034	case TypeTableEntryIdFn:	13944	case TypeTableEntryIdFn:
13035	case TypeTableEntryIdNamespace:	13945	case TypeTableEntryIdNamespace:
13036	case TypeTableEntryIdBoundFn:	13946	case TypeTableEntryIdBoundFn:
		13947	case TypeTableEntryIdPromise:
13037	{	13948	{
13038	TypeTableEntry *result_type = get_array_type(ira->codegen, child_type, size);	13949	TypeTableEntry *result_type = get_array_type(ira->codegen, child_type, size);
13039	ConstExprValue *out_val = ir_build_const_from(ira, &array_type_instruction->base);	13950	ConstExprValue *out_val = ir_build_const_from(ira, &array_type_instruction->base);
...	@@ -13085,6 +13996,7 @@ static TypeTableEntry ir_analyze_instruction_size_of(IrAnalyze ira,	...	@@ -13085,6 +13996,7 @@ static TypeTableEntry ir_analyze_instruction_size_of(IrAnalyze ira,
13085	case TypeTableEntryIdEnum:	13996	case TypeTableEntryIdEnum:
13086	case TypeTableEntryIdUnion:	13997	case TypeTableEntryIdUnion:
13087	case TypeTableEntryIdFn:	13998	case TypeTableEntryIdFn:
		13999	case TypeTableEntryIdPromise:
13088	{	14000	{
13089	uint64_t size_in_bytes = type_size(ira->codegen, type_entry);	14001	uint64_t size_in_bytes = type_size(ira->codegen, type_entry);
13090	ConstExprValue *out_val = ir_build_const_from(ira, &size_of_instruction->base);	14002	ConstExprValue *out_val = ir_build_const_from(ira, &size_of_instruction->base);
...	@@ -13414,6 +14326,7 @@ static TypeTableEntry ir_analyze_instruction_switch_target(IrAnalyze ira,	...	@@ -13414,6 +14326,7 @@ static TypeTableEntry ir_analyze_instruction_switch_target(IrAnalyze ira,
13414	case TypeTableEntryIdNumLitFloat:	14326	case TypeTableEntryIdNumLitFloat:
13415	case TypeTableEntryIdNumLitInt:	14327	case TypeTableEntryIdNumLitInt:
13416	case TypeTableEntryIdPointer:	14328	case TypeTableEntryIdPointer:
		14329	case TypeTableEntryIdPromise:
13417	case TypeTableEntryIdFn:	14330	case TypeTableEntryIdFn:
13418	case TypeTableEntryIdNamespace:	14331	case TypeTableEntryIdNamespace:
13419	case TypeTableEntryIdErrorSet:	14332	case TypeTableEntryIdErrorSet:
...	@@ -14002,6 +14915,7 @@ static TypeTableEntry ir_analyze_min_max(IrAnalyze ira, IrInstruction *source_	...	@@ -14002,6 +14915,7 @@ static TypeTableEntry ir_analyze_min_max(IrAnalyze ira, IrInstruction *source_
14002	case TypeTableEntryIdMetaType:	14915	case TypeTableEntryIdMetaType:
14003	case TypeTableEntryIdUnreachable:	14916	case TypeTableEntryIdUnreachable:
14004	case TypeTableEntryIdPointer:	14917	case TypeTableEntryIdPointer:
		14918	case TypeTableEntryIdPromise:
14005	case TypeTableEntryIdArray:	14919	case TypeTableEntryIdArray:
14006	case TypeTableEntryIdStruct:	14920	case TypeTableEntryIdStruct:
14007	case TypeTableEntryIdNumLitFloat:	14921	case TypeTableEntryIdNumLitFloat:
...	@@ -15262,6 +16176,7 @@ static TypeTableEntry ir_analyze_instruction_align_of(IrAnalyze ira, IrInstruc	...	@@ -15262,6 +16176,7 @@ static TypeTableEntry ir_analyze_instruction_align_of(IrAnalyze ira, IrInstruc
15262	case TypeTableEntryIdInt:	16176	case TypeTableEntryIdInt:
15263	case TypeTableEntryIdFloat:	16177	case TypeTableEntryIdFloat:
15264	case TypeTableEntryIdPointer:	16178	case TypeTableEntryIdPointer:
		16179	case TypeTableEntryIdPromise:
15265	case TypeTableEntryIdArray:	16180	case TypeTableEntryIdArray:
15266	case TypeTableEntryIdStruct:	16181	case TypeTableEntryIdStruct:
15267	case TypeTableEntryIdMaybe:	16182	case TypeTableEntryIdMaybe:
...	@@ -15890,12 +16805,12 @@ static TypeTableEntry ir_analyze_instruction_ptr_cast(IrAnalyze ira, IrInstruc	...	@@ -15890,12 +16805,12 @@ static TypeTableEntry ir_analyze_instruction_ptr_cast(IrAnalyze ira, IrInstruc
15890	if (type_is_invalid(src_type))	16805	if (type_is_invalid(src_type))
15891	return ira->codegen->builtin_types.entry_invalid;	16806	return ira->codegen->builtin_types.entry_invalid;
15892		16807
15893	if (!type_is_codegen_pointer(src_type)) {	16808	if (get_codegen_ptr_type(src_type) == nullptr) {
15894	ir_add_error(ira, ptr, buf_sprintf("expected pointer, found '%s'", buf_ptr(&src_type->name)));	16809	ir_add_error(ira, ptr, buf_sprintf("expected pointer, found '%s'", buf_ptr(&src_type->name)));
15895	return ira->codegen->builtin_types.entry_invalid;	16810	return ira->codegen->builtin_types.entry_invalid;
15896	}	16811	}
15897		16812
15898	if (!type_is_codegen_pointer(dest_type)) {	16813	if (get_codegen_ptr_type(dest_type) == nullptr) {
15899	ir_add_error(ira, dest_type_value,	16814	ir_add_error(ira, dest_type_value,
15900	buf_sprintf("expected pointer, found '%s'", buf_ptr(&dest_type->name)));	16815	buf_sprintf("expected pointer, found '%s'", buf_ptr(&dest_type->name)));
15901	return ira->codegen->builtin_types.entry_invalid;	16816	return ira->codegen->builtin_types.entry_invalid;
...	@@ -15957,6 +16872,7 @@ static void buf_write_value_bytes(CodeGen codegen, uint8_t buf, ConstExprValue	...	@@ -15957,6 +16872,7 @@ static void buf_write_value_bytes(CodeGen codegen, uint8_t buf, ConstExprValue
15957	case TypeTableEntryIdNumLitInt:	16872	case TypeTableEntryIdNumLitInt:
15958	case TypeTableEntryIdUndefLit:	16873	case TypeTableEntryIdUndefLit:
15959	case TypeTableEntryIdNullLit:	16874	case TypeTableEntryIdNullLit:
		16875	case TypeTableEntryIdPromise:
15960	zig_unreachable();	16876	zig_unreachable();
15961	case TypeTableEntryIdVoid:	16877	case TypeTableEntryIdVoid:
15962	return;	16878	return;
...	@@ -16024,6 +16940,7 @@ static void buf_read_value_bytes(CodeGen codegen, uint8_t buf, ConstExprValue	...	@@ -16024,6 +16940,7 @@ static void buf_read_value_bytes(CodeGen codegen, uint8_t buf, ConstExprValue
16024	case TypeTableEntryIdNumLitInt:	16940	case TypeTableEntryIdNumLitInt:
16025	case TypeTableEntryIdUndefLit:	16941	case TypeTableEntryIdUndefLit:
16026	case TypeTableEntryIdNullLit:	16942	case TypeTableEntryIdNullLit:
		16943	case TypeTableEntryIdPromise:
16027	zig_unreachable();	16944	zig_unreachable();
16028	case TypeTableEntryIdVoid:	16945	case TypeTableEntryIdVoid:
16029	return;	16946	return;
...	@@ -16080,9 +16997,9 @@ static TypeTableEntry ir_analyze_instruction_bit_cast(IrAnalyze ira, IrInstruc	...	@@ -16080,9 +16997,9 @@ static TypeTableEntry ir_analyze_instruction_bit_cast(IrAnalyze ira, IrInstruc
16080	ensure_complete_type(ira->codegen, dest_type);	16997	ensure_complete_type(ira->codegen, dest_type);
16081	ensure_complete_type(ira->codegen, src_type);	16998	ensure_complete_type(ira->codegen, src_type);
16082		16999
16083	if (type_is_codegen_pointer(src_type)) {	17000	if (get_codegen_ptr_type(src_type) != nullptr) {
16084	ir_add_error(ira, value,	17001	ir_add_error(ira, value,
16085	buf_sprintf("unable to @bitCast from type '%s'", buf_ptr(&src_type->name)));	17002	buf_sprintf("unable to @bitCast from pointer type '%s'", buf_ptr(&src_type->name)));
16086	return ira->codegen->builtin_types.entry_invalid;	17003	return ira->codegen->builtin_types.entry_invalid;
16087	}	17004	}
16088		17005
...	@@ -16107,9 +17024,9 @@ static TypeTableEntry ir_analyze_instruction_bit_cast(IrAnalyze ira, IrInstruc	...	@@ -16107,9 +17024,9 @@ static TypeTableEntry ir_analyze_instruction_bit_cast(IrAnalyze ira, IrInstruc
16107	break;	17024	break;
16108	}	17025	}
16109		17026
16110	if (type_is_codegen_pointer(dest_type)) {	17027	if (get_codegen_ptr_type(dest_type) != nullptr) {
16111	ir_add_error(ira, dest_type_value,	17028	ir_add_error(ira, dest_type_value,
16112	buf_sprintf("unable to @bitCast to type '%s'", buf_ptr(&dest_type->name)));	17029	buf_sprintf("unable to @bitCast to pointer type '%s'", buf_ptr(&dest_type->name)));
16113	return ira->codegen->builtin_types.entry_invalid;	17030	return ira->codegen->builtin_types.entry_invalid;
16114	}	17031	}
16115		17032
...	@@ -16170,7 +17087,7 @@ static TypeTableEntry ir_analyze_instruction_int_to_ptr(IrAnalyze ira, IrInstr	...	@@ -16170,7 +17087,7 @@ static TypeTableEntry ir_analyze_instruction_int_to_ptr(IrAnalyze ira, IrInstr
16170	if (type_is_invalid(dest_type))	17087	if (type_is_invalid(dest_type))
16171	return ira->codegen->builtin_types.entry_invalid;	17088	return ira->codegen->builtin_types.entry_invalid;
16172		17089
16173	if (!type_is_codegen_pointer(dest_type)) {	17090	if (get_codegen_ptr_type(dest_type) == nullptr) {
16174	ir_add_error(ira, dest_type_value, buf_sprintf("expected pointer, found '%s'", buf_ptr(&dest_type->name)));	17091	ir_add_error(ira, dest_type_value, buf_sprintf("expected pointer, found '%s'", buf_ptr(&dest_type->name)));
16175	return ira->codegen->builtin_types.entry_invalid;	17092	return ira->codegen->builtin_types.entry_invalid;
16176	}	17093	}
...	@@ -16276,12 +17193,7 @@ static TypeTableEntry ir_analyze_instruction_ptr_to_int(IrAnalyze ira, IrInstr	...	@@ -16276,12 +17193,7 @@ static TypeTableEntry ir_analyze_instruction_ptr_to_int(IrAnalyze ira, IrInstr
16276		17193
16277	TypeTableEntry *usize = ira->codegen->builtin_types.entry_usize;	17194	TypeTableEntry *usize = ira->codegen->builtin_types.entry_usize;
16278		17195
16279	if (!(target->value.type->id == TypeTableEntryIdPointer \|\|	17196	if (get_codegen_ptr_type(target->value.type) == nullptr) {
16280	target->value.type->id == TypeTableEntryIdFn \|\|
16281	(target->value.type->id == TypeTableEntryIdMaybe &&
16282	(target->value.type->data.maybe.child_type->id == TypeTableEntryIdPointer \|\|
16283	target->value.type->data.maybe.child_type->id == TypeTableEntryIdFn))))
16284	{
16285	ir_add_error(ira, target,	17197	ir_add_error(ira, target,
16286	buf_sprintf("expected pointer, found '%s'", buf_ptr(&target->value.type->name)));	17198	buf_sprintf("expected pointer, found '%s'", buf_ptr(&target->value.type->name)));
16287	return ira->codegen->builtin_types.entry_invalid;	17199	return ira->codegen->builtin_types.entry_invalid;
...	@@ -16465,6 +17377,292 @@ static TypeTableEntry ir_analyze_instruction_tag_type(IrAnalyze ira, IrInstruc	...	@@ -16465,6 +17377,292 @@ static TypeTableEntry ir_analyze_instruction_tag_type(IrAnalyze ira, IrInstruc
16465	}	17377	}
16466	}	17378	}
16467		17379
		17380	static TypeTableEntry ir_analyze_instruction_cancel(IrAnalyze ira, IrInstructionCancel *instruction) {
		17381	IrInstruction *target_inst = instruction->target->other;
		17382	if (type_is_invalid(target_inst->value.type))
		17383	return ira->codegen->builtin_types.entry_invalid;
		17384	IrInstruction *casted_target = ir_implicit_cast(ira, target_inst, ira->codegen->builtin_types.entry_promise);
		17385	if (type_is_invalid(casted_target->value.type))
		17386	return ira->codegen->builtin_types.entry_invalid;
		17387
		17388	IrInstruction *result = ir_build_cancel(&ira->new_irb, instruction->base.scope, instruction->base.source_node, casted_target);
		17389	result->value.type = ira->codegen->builtin_types.entry_void;
		17390	result->value.special = ConstValSpecialStatic;
		17391	ir_link_new_instruction(result, &instruction->base);
		17392	return result->value.type;
		17393	}
		17394
		17395	static TypeTableEntry ir_analyze_instruction_coro_id(IrAnalyze ira, IrInstructionCoroId *instruction) {
		17396	IrInstruction *promise_ptr = instruction->promise_ptr->other;
		17397	if (type_is_invalid(promise_ptr->value.type))
		17398	return ira->codegen->builtin_types.entry_invalid;
		17399
		17400	IrInstruction *result = ir_build_coro_id(&ira->new_irb, instruction->base.scope, instruction->base.source_node,
		17401	promise_ptr);
		17402	ir_link_new_instruction(result, &instruction->base);
		17403	result->value.type = ira->codegen->builtin_types.entry_usize;
		17404	return result->value.type;
		17405	}
		17406
		17407	static TypeTableEntry ir_analyze_instruction_coro_alloc(IrAnalyze ira, IrInstructionCoroAlloc *instruction) {
		17408	IrInstruction *coro_id = instruction->coro_id->other;
		17409	if (type_is_invalid(coro_id->value.type))
		17410	return ira->codegen->builtin_types.entry_invalid;
		17411
		17412	IrInstruction *result = ir_build_coro_alloc(&ira->new_irb, instruction->base.scope, instruction->base.source_node,
		17413	coro_id);
		17414	ir_link_new_instruction(result, &instruction->base);
		17415	result->value.type = ira->codegen->builtin_types.entry_bool;
		17416	return result->value.type;
		17417	}
		17418
		17419	static TypeTableEntry ir_analyze_instruction_coro_size(IrAnalyze ira, IrInstructionCoroSize *instruction) {
		17420	IrInstruction *result = ir_build_coro_size(&ira->new_irb, instruction->base.scope, instruction->base.source_node);
		17421	ir_link_new_instruction(result, &instruction->base);
		17422	result->value.type = ira->codegen->builtin_types.entry_usize;
		17423	return result->value.type;
		17424	}
		17425
		17426	static TypeTableEntry ir_analyze_instruction_coro_begin(IrAnalyze ira, IrInstructionCoroBegin *instruction) {
		17427	IrInstruction *coro_id = instruction->coro_id->other;
		17428	if (type_is_invalid(coro_id->value.type))
		17429	return ira->codegen->builtin_types.entry_invalid;
		17430
		17431	IrInstruction *coro_mem_ptr = instruction->coro_mem_ptr->other;
		17432	if (type_is_invalid(coro_mem_ptr->value.type))
		17433	return ira->codegen->builtin_types.entry_invalid;
		17434
		17435	FnTableEntry *fn_entry = exec_fn_entry(ira->new_irb.exec);
		17436	assert(fn_entry != nullptr);
		17437	IrInstruction *result = ir_build_coro_begin(&ira->new_irb, instruction->base.scope, instruction->base.source_node,
		17438	coro_id, coro_mem_ptr);
		17439	ir_link_new_instruction(result, &instruction->base);
		17440	result->value.type = get_promise_type(ira->codegen, fn_entry->type_entry->data.fn.fn_type_id.return_type);
		17441	return result->value.type;
		17442	}
		17443
		17444	static TypeTableEntry ir_analyze_instruction_get_implicit_allocator(IrAnalyze ira, IrInstructionGetImplicitAllocator *instruction) {
		17445	IrInstruction *result = ir_get_implicit_allocator(ira, &instruction->base, instruction->id);
		17446	ir_link_new_instruction(result, &instruction->base);
		17447	return result->value.type;
		17448	}
		17449
		17450	static TypeTableEntry ir_analyze_instruction_coro_alloc_fail(IrAnalyze ira, IrInstructionCoroAllocFail *instruction) {
		17451	IrInstruction *err_val = instruction->err_val->other;
		17452	if (type_is_invalid(err_val->value.type))
		17453	return ir_unreach_error(ira);
		17454
		17455	IrInstruction *result = ir_build_coro_alloc_fail(&ira->new_irb, instruction->base.scope, instruction->base.source_node, err_val);
		17456	ir_link_new_instruction(result, &instruction->base);
		17457	result->value.type = ira->codegen->builtin_types.entry_unreachable;
		17458	return ir_finish_anal(ira, result->value.type);
		17459	}
		17460
		17461	static TypeTableEntry ir_analyze_instruction_coro_suspend(IrAnalyze ira, IrInstructionCoroSuspend *instruction) {
		17462	IrInstruction *save_point = nullptr;
		17463	if (instruction->save_point != nullptr) {
		17464	save_point = instruction->save_point->other;
		17465	if (type_is_invalid(save_point->value.type))
		17466	return ira->codegen->builtin_types.entry_invalid;
		17467	}
		17468
		17469	IrInstruction *is_final = instruction->is_final->other;
		17470	if (type_is_invalid(is_final->value.type))
		17471	return ira->codegen->builtin_types.entry_invalid;
		17472
		17473	IrInstruction *result = ir_build_coro_suspend(&ira->new_irb, instruction->base.scope,
		17474	instruction->base.source_node, save_point, is_final);
		17475	ir_link_new_instruction(result, &instruction->base);
		17476	result->value.type = ira->codegen->builtin_types.entry_u8;
		17477	return result->value.type;
		17478	}
		17479
		17480	static TypeTableEntry ir_analyze_instruction_coro_end(IrAnalyze ira, IrInstructionCoroEnd *instruction) {
		17481	IrInstruction *result = ir_build_coro_end(&ira->new_irb, instruction->base.scope,
		17482	instruction->base.source_node);
		17483	ir_link_new_instruction(result, &instruction->base);
		17484	result->value.type = ira->codegen->builtin_types.entry_void;
		17485	return result->value.type;
		17486	}
		17487
		17488	static TypeTableEntry ir_analyze_instruction_coro_free(IrAnalyze ira, IrInstructionCoroFree *instruction) {
		17489	IrInstruction *coro_id = instruction->coro_id->other;
		17490	if (type_is_invalid(coro_id->value.type))
		17491	return ira->codegen->builtin_types.entry_invalid;
		17492
		17493	IrInstruction *coro_handle = instruction->coro_handle->other;
		17494	if (type_is_invalid(coro_handle->value.type))
		17495	return ira->codegen->builtin_types.entry_invalid;
		17496
		17497	IrInstruction *result = ir_build_coro_free(&ira->new_irb, instruction->base.scope,
		17498	instruction->base.source_node, coro_id, coro_handle);
		17499	ir_link_new_instruction(result, &instruction->base);
		17500	TypeTableEntry *ptr_type = get_pointer_to_type(ira->codegen, ira->codegen->builtin_types.entry_u8, false);
		17501	result->value.type = get_maybe_type(ira->codegen, ptr_type);
		17502	return result->value.type;
		17503	}
		17504
		17505	static TypeTableEntry ir_analyze_instruction_coro_resume(IrAnalyze ira, IrInstructionCoroResume *instruction) {
		17506	IrInstruction *awaiter_handle = instruction->awaiter_handle->other;
		17507	if (type_is_invalid(awaiter_handle->value.type))
		17508	return ira->codegen->builtin_types.entry_invalid;
		17509
		17510	IrInstruction *casted_target = ir_implicit_cast(ira, awaiter_handle, ira->codegen->builtin_types.entry_promise);
		17511	if (type_is_invalid(casted_target->value.type))
		17512	return ira->codegen->builtin_types.entry_invalid;
		17513
		17514	IrInstruction *result = ir_build_coro_resume(&ira->new_irb, instruction->base.scope,
		17515	instruction->base.source_node, casted_target);
		17516	ir_link_new_instruction(result, &instruction->base);
		17517	result->value.type = ira->codegen->builtin_types.entry_void;
		17518	return result->value.type;
		17519	}
		17520
		17521	static TypeTableEntry ir_analyze_instruction_coro_save(IrAnalyze ira, IrInstructionCoroSave *instruction) {
		17522	IrInstruction *coro_handle = instruction->coro_handle->other;
		17523	if (type_is_invalid(coro_handle->value.type))
		17524	return ira->codegen->builtin_types.entry_invalid;
		17525
		17526	IrInstruction *result = ir_build_coro_save(&ira->new_irb, instruction->base.scope,
		17527	instruction->base.source_node, coro_handle);
		17528	ir_link_new_instruction(result, &instruction->base);
		17529	result->value.type = ira->codegen->builtin_types.entry_usize;
		17530	return result->value.type;
		17531	}
		17532
		17533	static TypeTableEntry ir_analyze_instruction_coro_promise(IrAnalyze ira, IrInstructionCoroPromise *instruction) {
		17534	IrInstruction *coro_handle = instruction->coro_handle->other;
		17535	if (type_is_invalid(coro_handle->value.type))
		17536	return ira->codegen->builtin_types.entry_invalid;
		17537
		17538	if (coro_handle->value.type->id != TypeTableEntryIdPromise \|\|
		17539	coro_handle->value.type->data.promise.result_type == nullptr)
		17540	{
		17541	ir_add_error(ira, &instruction->base, buf_sprintf("expected promise->T, found '%s'",
		17542	buf_ptr(&coro_handle->value.type->name)));
		17543	return ira->codegen->builtin_types.entry_invalid;
		17544	}
		17545
		17546	TypeTableEntry *coro_frame_type = get_promise_frame_type(ira->codegen,
		17547	coro_handle->value.type->data.promise.result_type);
		17548
		17549	IrInstruction *result = ir_build_coro_promise(&ira->new_irb, instruction->base.scope,
		17550	instruction->base.source_node, coro_handle);
		17551	ir_link_new_instruction(result, &instruction->base);
		17552	result->value.type = get_pointer_to_type(ira->codegen, coro_frame_type, false);
		17553	return result->value.type;
		17554	}
		17555
		17556	static TypeTableEntry ir_analyze_instruction_coro_alloc_helper(IrAnalyze ira, IrInstructionCoroAllocHelper *instruction) {
		17557	IrInstruction *alloc_fn = instruction->alloc_fn->other;
		17558	if (type_is_invalid(alloc_fn->value.type))
		17559	return ira->codegen->builtin_types.entry_invalid;
		17560
		17561	IrInstruction *coro_size = instruction->coro_size->other;
		17562	if (type_is_invalid(coro_size->value.type))
		17563	return ira->codegen->builtin_types.entry_invalid;
		17564
		17565	IrInstruction *result = ir_build_coro_alloc_helper(&ira->new_irb, instruction->base.scope,
		17566	instruction->base.source_node, alloc_fn, coro_size);
		17567	ir_link_new_instruction(result, &instruction->base);
		17568	TypeTableEntry *u8_ptr_type = get_pointer_to_type(ira->codegen, ira->codegen->builtin_types.entry_u8, false);
		17569	result->value.type = get_maybe_type(ira->codegen, u8_ptr_type);
		17570	return result->value.type;
		17571	}
		17572
		17573	static TypeTableEntry ir_analyze_instruction_atomic_rmw(IrAnalyze ira, IrInstructionAtomicRmw *instruction) {
		17574	TypeTableEntry *operand_type = ir_resolve_type(ira, instruction->operand_type->other);
		17575	if (type_is_invalid(operand_type)) {
		17576	return ira->codegen->builtin_types.entry_invalid;
		17577	}
		17578	if (operand_type->id == TypeTableEntryIdInt) {
		17579	if (operand_type->data.integral.bit_count < 8) {
		17580	ir_add_error(ira, &instruction->base,
		17581	buf_sprintf("expected integer type 8 bits or larger, found %" PRIu32 "-bit integer type",
		17582	operand_type->data.integral.bit_count));
		17583	return ira->codegen->builtin_types.entry_invalid;
		17584	}
		17585	if (operand_type->data.integral.bit_count > ira->codegen->pointer_size_bytes * 8) {
		17586	ir_add_error(ira, &instruction->base,
		17587	buf_sprintf("expected integer type pointer size or smaller, found %" PRIu32 "-bit integer type",
		17588	operand_type->data.integral.bit_count));
		17589	return ira->codegen->builtin_types.entry_invalid;
		17590	}
		17591	if (!is_power_of_2(operand_type->data.integral.bit_count)) {
		17592	ir_add_error(ira, &instruction->base,
		17593	buf_sprintf("%" PRIu32 "-bit integer type is not a power of 2", operand_type->data.integral.bit_count));
		17594	return ira->codegen->builtin_types.entry_invalid;
		17595	}
		17596	} else if (get_codegen_ptr_type(operand_type) == nullptr) {
		17597	ir_add_error(ira, &instruction->base,
		17598	buf_sprintf("expected integer or pointer type, found '%s'", buf_ptr(&operand_type->name)));
		17599	return ira->codegen->builtin_types.entry_invalid;
		17600	}
		17601
		17602	IrInstruction *ptr_inst = instruction->ptr->other;
		17603	if (type_is_invalid(ptr_inst->value.type))
		17604	return ira->codegen->builtin_types.entry_invalid;
		17605
		17606	TypeTableEntry *ptr_type = get_pointer_to_type(ira->codegen, operand_type, false);
		17607	IrInstruction *casted_ptr = ir_implicit_cast(ira, ptr_inst, ptr_type);
		17608	if (type_is_invalid(casted_ptr->value.type))
		17609	return ira->codegen->builtin_types.entry_invalid;
		17610
		17611	AtomicRmwOp op;
		17612	if (instruction->op == nullptr) {
		17613	op = instruction->resolved_op;
		17614	} else {
		17615	if (!ir_resolve_atomic_rmw_op(ira, instruction->op->other, &op)) {
		17616	return ira->codegen->builtin_types.entry_invalid;
		17617	}
		17618	}
		17619
		17620	IrInstruction *operand = instruction->operand->other;
		17621	if (type_is_invalid(operand->value.type))
		17622	return ira->codegen->builtin_types.entry_invalid;
		17623
		17624	IrInstruction *casted_operand = ir_implicit_cast(ira, operand, operand_type);
		17625	if (type_is_invalid(casted_ptr->value.type))
		17626	return ira->codegen->builtin_types.entry_invalid;
		17627
		17628	AtomicOrder ordering;
		17629	if (instruction->ordering == nullptr) {
		17630	ordering = instruction->resolved_ordering;
		17631	} else {
		17632	if (!ir_resolve_atomic_order(ira, instruction->ordering->other, &ordering))
		17633	return ira->codegen->builtin_types.entry_invalid;
		17634	}
		17635
		17636	if (instr_is_comptime(casted_operand) && instr_is_comptime(casted_ptr) && casted_ptr->value.data.x_ptr.mut == ConstPtrMutComptimeVar)
		17637	{
		17638	zig_panic("TODO compile-time execution of atomicRmw");
		17639	}
		17640
		17641	IrInstruction *result = ir_build_atomic_rmw(&ira->new_irb, instruction->base.scope,
		17642	instruction->base.source_node, nullptr, casted_ptr, nullptr, casted_operand, nullptr,
		17643	op, ordering);
		17644	ir_link_new_instruction(result, &instruction->base);
		17645	result->value.type = operand_type;
		17646	return result->value.type;
		17647	}
		17648
		17649	static TypeTableEntry ir_analyze_instruction_promise_result_type(IrAnalyze ira, IrInstructionPromiseResultType *instruction) {
		17650	TypeTableEntry *promise_type = ir_resolve_type(ira, instruction->promise_type->other);
		17651	if (type_is_invalid(promise_type))
		17652	return ira->codegen->builtin_types.entry_invalid;
		17653
		17654	if (promise_type->id != TypeTableEntryIdPromise \|\| promise_type->data.promise.result_type == nullptr) {
		17655	ir_add_error(ira, &instruction->base, buf_sprintf("expected promise->T, found '%s'",
		17656	buf_ptr(&promise_type->name)));
		17657	return ira->codegen->builtin_types.entry_invalid;
		17658	}
		17659
		17660	ConstExprValue *out_val = ir_build_const_from(ira, &instruction->base);
		17661	out_val->data.x_type = promise_type->data.promise.result_type;
		17662	return ira->codegen->builtin_types.entry_type;
		17663	}
		17664
		17665
16468	static TypeTableEntry ir_analyze_instruction_nocast(IrAnalyze ira, IrInstruction *instruction) {	17666	static TypeTableEntry ir_analyze_instruction_nocast(IrAnalyze ira, IrInstruction *instruction) {
16469	switch (instruction->id) {	17667	switch (instruction->id) {
16470	case IrInstructionIdInvalid:	17668	case IrInstructionIdInvalid:
...	@@ -16667,6 +17865,38 @@ static TypeTableEntry ir_analyze_instruction_nocast(IrAnalyze ira, IrInstructi	...	@@ -16667,6 +17865,38 @@ static TypeTableEntry ir_analyze_instruction_nocast(IrAnalyze ira, IrInstructi
16667	return ir_analyze_instruction_error_return_trace(ira, (IrInstructionErrorReturnTrace *)instruction);	17865	return ir_analyze_instruction_error_return_trace(ira, (IrInstructionErrorReturnTrace *)instruction);
16668	case IrInstructionIdErrorUnion:	17866	case IrInstructionIdErrorUnion:
16669	return ir_analyze_instruction_error_union(ira, (IrInstructionErrorUnion *)instruction);	17867	return ir_analyze_instruction_error_union(ira, (IrInstructionErrorUnion *)instruction);
		17868	case IrInstructionIdCancel:
		17869	return ir_analyze_instruction_cancel(ira, (IrInstructionCancel *)instruction);
		17870	case IrInstructionIdCoroId:
		17871	return ir_analyze_instruction_coro_id(ira, (IrInstructionCoroId *)instruction);
		17872	case IrInstructionIdCoroAlloc:
		17873	return ir_analyze_instruction_coro_alloc(ira, (IrInstructionCoroAlloc *)instruction);
		17874	case IrInstructionIdCoroSize:
		17875	return ir_analyze_instruction_coro_size(ira, (IrInstructionCoroSize *)instruction);
		17876	case IrInstructionIdCoroBegin:
		17877	return ir_analyze_instruction_coro_begin(ira, (IrInstructionCoroBegin *)instruction);
		17878	case IrInstructionIdGetImplicitAllocator:
		17879	return ir_analyze_instruction_get_implicit_allocator(ira, (IrInstructionGetImplicitAllocator *)instruction);
		17880	case IrInstructionIdCoroAllocFail:
		17881	return ir_analyze_instruction_coro_alloc_fail(ira, (IrInstructionCoroAllocFail *)instruction);
		17882	case IrInstructionIdCoroSuspend:
		17883	return ir_analyze_instruction_coro_suspend(ira, (IrInstructionCoroSuspend *)instruction);
		17884	case IrInstructionIdCoroEnd:
		17885	return ir_analyze_instruction_coro_end(ira, (IrInstructionCoroEnd *)instruction);
		17886	case IrInstructionIdCoroFree:
		17887	return ir_analyze_instruction_coro_free(ira, (IrInstructionCoroFree *)instruction);
		17888	case IrInstructionIdCoroResume:
		17889	return ir_analyze_instruction_coro_resume(ira, (IrInstructionCoroResume *)instruction);
		17890	case IrInstructionIdCoroSave:
		17891	return ir_analyze_instruction_coro_save(ira, (IrInstructionCoroSave *)instruction);
		17892	case IrInstructionIdCoroPromise:
		17893	return ir_analyze_instruction_coro_promise(ira, (IrInstructionCoroPromise *)instruction);
		17894	case IrInstructionIdCoroAllocHelper:
		17895	return ir_analyze_instruction_coro_alloc_helper(ira, (IrInstructionCoroAllocHelper *)instruction);
		17896	case IrInstructionIdAtomicRmw:
		17897	return ir_analyze_instruction_atomic_rmw(ira, (IrInstructionAtomicRmw *)instruction);
		17898	case IrInstructionIdPromiseResultType:
		17899	return ir_analyze_instruction_promise_result_type(ira, (IrInstructionPromiseResultType *)instruction);
16670	}	17900	}
16671	zig_unreachable();	17901	zig_unreachable();
16672	}	17902	}
...	@@ -16696,7 +17926,10 @@ TypeTableEntry ir_analyze(CodeGen codegen, IrExecutable *old_exec, IrExecutabl	...	@@ -16696,7 +17926,10 @@ TypeTableEntry ir_analyze(CodeGen codegen, IrExecutable *old_exec, IrExecutabl
16696	IrAnalyze *ira = allocate<IrAnalyze>(1);	17926	IrAnalyze *ira = allocate<IrAnalyze>(1);
16697	old_exec->analysis = ira;	17927	old_exec->analysis = ira;
16698	ira->codegen = codegen;	17928	ira->codegen = codegen;
16699	ira->explicit_return_type = expected_type;	17929
		17930	FnTableEntry *fn_entry = exec_fn_entry(old_exec);
		17931	bool is_async = fn_entry != nullptr && fn_entry->type_entry->data.fn.fn_type_id.cc == CallingConventionAsync;
		17932	ira->explicit_return_type = is_async ? get_promise_type(codegen, expected_type) : expected_type;
16700		17933
16701	ira->old_irb.codegen = codegen;	17934	ira->old_irb.codegen = codegen;
16702	ira->old_irb.exec = old_exec;	17935	ira->old_irb.exec = old_exec;
...	@@ -16780,7 +18013,16 @@ bool ir_has_side_effects(IrInstruction *instruction) {	...	@@ -16780,7 +18013,16 @@ bool ir_has_side_effects(IrInstruction *instruction) {
16780	case IrInstructionIdPtrTypeOf:	18013	case IrInstructionIdPtrTypeOf:
16781	case IrInstructionIdSetAlignStack:	18014	case IrInstructionIdSetAlignStack:
16782	case IrInstructionIdExport:	18015	case IrInstructionIdExport:
		18016	case IrInstructionIdCancel:
		18017	case IrInstructionIdCoroId:
		18018	case IrInstructionIdCoroBegin:
		18019	case IrInstructionIdCoroAllocFail:
		18020	case IrInstructionIdCoroEnd:
		18021	case IrInstructionIdCoroResume:
		18022	case IrInstructionIdCoroSave:
		18023	case IrInstructionIdCoroAllocHelper:
16783	return true;	18024	return true;
		18025
16784	case IrInstructionIdPhi:	18026	case IrInstructionIdPhi:
16785	case IrInstructionIdUnOp:	18027	case IrInstructionIdUnOp:
16786	case IrInstructionIdBinOp:	18028	case IrInstructionIdBinOp:
...	@@ -16853,7 +18095,16 @@ bool ir_has_side_effects(IrInstruction *instruction) {	...	@@ -16853,7 +18095,16 @@ bool ir_has_side_effects(IrInstruction *instruction) {
16853	case IrInstructionIdTagType:	18095	case IrInstructionIdTagType:
16854	case IrInstructionIdErrorReturnTrace:	18096	case IrInstructionIdErrorReturnTrace:
16855	case IrInstructionIdErrorUnion:	18097	case IrInstructionIdErrorUnion:
		18098	case IrInstructionIdGetImplicitAllocator:
		18099	case IrInstructionIdCoroAlloc:
		18100	case IrInstructionIdCoroSize:
		18101	case IrInstructionIdCoroSuspend:
		18102	case IrInstructionIdCoroFree:
		18103	case IrInstructionIdAtomicRmw:
		18104	case IrInstructionIdCoroPromise:
		18105	case IrInstructionIdPromiseResultType:
16856	return false;	18106	return false;
		18107
16857	case IrInstructionIdAsm:	18108	case IrInstructionIdAsm:
16858	{	18109	{
16859	IrInstructionAsm asm_instruction = (IrInstructionAsm )instruction;	18110	IrInstructionAsm asm_instruction = (IrInstructionAsm )instruction;

src/ir_print.cpp+193

...	@@ -198,6 +198,15 @@ static void ir_print_cast(IrPrint irp, IrInstructionCast cast_instruction) {	...	@@ -198,6 +198,15 @@ static void ir_print_cast(IrPrint irp, IrInstructionCast cast_instruction) {
198	}	198	}
199		199
200	static void ir_print_call(IrPrint irp, IrInstructionCall call_instruction) {	200	static void ir_print_call(IrPrint irp, IrInstructionCall call_instruction) {
		201	if (call_instruction->is_async) {
		202	fprintf(irp->f, "async");
		203	if (call_instruction->async_allocator != nullptr) {
		204	fprintf(irp->f, "(");
		205	ir_print_other_instruction(irp, call_instruction->async_allocator);
		206	fprintf(irp->f, ")");
		207	}
		208	fprintf(irp->f, " ");
		209	}
201	if (call_instruction->fn_entry) {	210	if (call_instruction->fn_entry) {
202	fprintf(irp->f, "%s", buf_ptr(&call_instruction->fn_entry->symbol_name));	211	fprintf(irp->f, "%s", buf_ptr(&call_instruction->fn_entry->symbol_name));
203	} else {	212	} else {
...	@@ -830,6 +839,12 @@ static void ir_print_ptr_to_int(IrPrint irp, IrInstructionPtrToInt instruction	...	@@ -830,6 +839,12 @@ static void ir_print_ptr_to_int(IrPrint irp, IrInstructionPtrToInt instruction
830		839
831	static void ir_print_int_to_ptr(IrPrint irp, IrInstructionIntToPtr instruction) {	840	static void ir_print_int_to_ptr(IrPrint irp, IrInstructionIntToPtr instruction) {
832	fprintf(irp->f, "@intToPtr(");	841	fprintf(irp->f, "@intToPtr(");
		842	if (instruction->dest_type == nullptr) {
		843	fprintf(irp->f, "(null)");
		844	} else {
		845	ir_print_other_instruction(irp, instruction->dest_type);
		846	}
		847	fprintf(irp->f, ",");
833	ir_print_other_instruction(irp, instruction->target);	848	ir_print_other_instruction(irp, instruction->target);
834	fprintf(irp->f, ")");	849	fprintf(irp->f, ")");
835	}	850	}
...	@@ -1010,6 +1025,136 @@ static void ir_print_error_union(IrPrint irp, IrInstructionErrorUnion instruct	...	@@ -1010,6 +1025,136 @@ static void ir_print_error_union(IrPrint irp, IrInstructionErrorUnion instruct
1010	ir_print_other_instruction(irp, instruction->payload);	1025	ir_print_other_instruction(irp, instruction->payload);
1011	}	1026	}
1012		1027
		1028	static void ir_print_cancel(IrPrint irp, IrInstructionCancel instruction) {
		1029	fprintf(irp->f, "cancel ");
		1030	ir_print_other_instruction(irp, instruction->target);
		1031	}
		1032
		1033	static void ir_print_get_implicit_allocator(IrPrint irp, IrInstructionGetImplicitAllocator instruction) {
		1034	fprintf(irp->f, "@getImplicitAllocator(");
		1035	switch (instruction->id) {
		1036	case ImplicitAllocatorIdArg:
		1037	fprintf(irp->f, "Arg");
		1038	break;
		1039	case ImplicitAllocatorIdLocalVar:
		1040	fprintf(irp->f, "LocalVar");
		1041	break;
		1042	}
		1043	fprintf(irp->f, ")");
		1044	}
		1045
		1046	static void ir_print_coro_id(IrPrint irp, IrInstructionCoroId instruction) {
		1047	fprintf(irp->f, "@coroId(");
		1048	ir_print_other_instruction(irp, instruction->promise_ptr);
		1049	fprintf(irp->f, ")");
		1050	}
		1051
		1052	static void ir_print_coro_alloc(IrPrint irp, IrInstructionCoroAlloc instruction) {
		1053	fprintf(irp->f, "@coroAlloc(");
		1054	ir_print_other_instruction(irp, instruction->coro_id);
		1055	fprintf(irp->f, ")");
		1056	}
		1057
		1058	static void ir_print_coro_size(IrPrint irp, IrInstructionCoroSize instruction) {
		1059	fprintf(irp->f, "@coroSize()");
		1060	}
		1061
		1062	static void ir_print_coro_begin(IrPrint irp, IrInstructionCoroBegin instruction) {
		1063	fprintf(irp->f, "@coroBegin(");
		1064	ir_print_other_instruction(irp, instruction->coro_id);
		1065	fprintf(irp->f, ",");
		1066	ir_print_other_instruction(irp, instruction->coro_mem_ptr);
		1067	fprintf(irp->f, ")");
		1068	}
		1069
		1070	static void ir_print_coro_alloc_fail(IrPrint irp, IrInstructionCoroAllocFail instruction) {
		1071	fprintf(irp->f, "@coroAllocFail(");
		1072	ir_print_other_instruction(irp, instruction->err_val);
		1073	fprintf(irp->f, ")");
		1074	}
		1075
		1076	static void ir_print_coro_suspend(IrPrint irp, IrInstructionCoroSuspend instruction) {
		1077	fprintf(irp->f, "@coroSuspend(");
		1078	if (instruction->save_point != nullptr) {
		1079	ir_print_other_instruction(irp, instruction->save_point);
		1080	} else {
		1081	fprintf(irp->f, "null");
		1082	}
		1083	fprintf(irp->f, ",");
		1084	ir_print_other_instruction(irp, instruction->is_final);
		1085	fprintf(irp->f, ")");
		1086	}
		1087
		1088	static void ir_print_coro_end(IrPrint irp, IrInstructionCoroEnd instruction) {
		1089	fprintf(irp->f, "@coroEnd()");
		1090	}
		1091
		1092	static void ir_print_coro_free(IrPrint irp, IrInstructionCoroFree instruction) {
		1093	fprintf(irp->f, "@coroFree(");
		1094	ir_print_other_instruction(irp, instruction->coro_id);
		1095	fprintf(irp->f, ",");
		1096	ir_print_other_instruction(irp, instruction->coro_handle);
		1097	fprintf(irp->f, ")");
		1098	}
		1099
		1100	static void ir_print_coro_resume(IrPrint irp, IrInstructionCoroResume instruction) {
		1101	fprintf(irp->f, "@coroResume(");
		1102	ir_print_other_instruction(irp, instruction->awaiter_handle);
		1103	fprintf(irp->f, ")");
		1104	}
		1105
		1106	static void ir_print_coro_save(IrPrint irp, IrInstructionCoroSave instruction) {
		1107	fprintf(irp->f, "@coroSave(");
		1108	ir_print_other_instruction(irp, instruction->coro_handle);
		1109	fprintf(irp->f, ")");
		1110	}
		1111
		1112	static void ir_print_coro_promise(IrPrint irp, IrInstructionCoroPromise instruction) {
		1113	fprintf(irp->f, "@coroPromise(");
		1114	ir_print_other_instruction(irp, instruction->coro_handle);
		1115	fprintf(irp->f, ")");
		1116	}
		1117
		1118	static void ir_print_promise_result_type(IrPrint irp, IrInstructionPromiseResultType instruction) {
		1119	fprintf(irp->f, "@PromiseResultType(");
		1120	ir_print_other_instruction(irp, instruction->promise_type);
		1121	fprintf(irp->f, ")");
		1122	}
		1123
		1124	static void ir_print_coro_alloc_helper(IrPrint irp, IrInstructionCoroAllocHelper instruction) {
		1125	fprintf(irp->f, "@coroAllocHelper(");
		1126	ir_print_other_instruction(irp, instruction->alloc_fn);
		1127	fprintf(irp->f, ",");
		1128	ir_print_other_instruction(irp, instruction->coro_size);
		1129	fprintf(irp->f, ")");
		1130	}
		1131
		1132	static void ir_print_atomic_rmw(IrPrint irp, IrInstructionAtomicRmw instruction) {
		1133	fprintf(irp->f, "@atomicRmw(");
		1134	if (instruction->operand_type != nullptr) {
		1135	ir_print_other_instruction(irp, instruction->operand_type);
		1136	} else {
		1137	fprintf(irp->f, "[TODO print]");
		1138	}
		1139	fprintf(irp->f, ",");
		1140	ir_print_other_instruction(irp, instruction->ptr);
		1141	fprintf(irp->f, ",");
		1142	if (instruction->op != nullptr) {
		1143	ir_print_other_instruction(irp, instruction->op);
		1144	} else {
		1145	fprintf(irp->f, "[TODO print]");
		1146	}
		1147	fprintf(irp->f, ",");
		1148	ir_print_other_instruction(irp, instruction->operand);
		1149	fprintf(irp->f, ",");
		1150	if (instruction->ordering != nullptr) {
		1151	ir_print_other_instruction(irp, instruction->ordering);
		1152	} else {
		1153	fprintf(irp->f, "[TODO print]");
		1154	}
		1155	fprintf(irp->f, ")");
		1156	}
		1157
1013	static void ir_print_instruction(IrPrint irp, IrInstruction instruction) {	1158	static void ir_print_instruction(IrPrint irp, IrInstruction instruction) {
1014	ir_print_prefix(irp, instruction);	1159	ir_print_prefix(irp, instruction);
1015	switch (instruction->id) {	1160	switch (instruction->id) {
...	@@ -1330,6 +1475,54 @@ static void ir_print_instruction(IrPrint irp, IrInstruction instruction) {	...	@@ -1330,6 +1475,54 @@ static void ir_print_instruction(IrPrint irp, IrInstruction instruction) {
1330	case IrInstructionIdErrorUnion:	1475	case IrInstructionIdErrorUnion:
1331	ir_print_error_union(irp, (IrInstructionErrorUnion *)instruction);	1476	ir_print_error_union(irp, (IrInstructionErrorUnion *)instruction);
1332	break;	1477	break;
		1478	case IrInstructionIdCancel:
		1479	ir_print_cancel(irp, (IrInstructionCancel *)instruction);
		1480	break;
		1481	case IrInstructionIdGetImplicitAllocator:
		1482	ir_print_get_implicit_allocator(irp, (IrInstructionGetImplicitAllocator *)instruction);
		1483	break;
		1484	case IrInstructionIdCoroId:
		1485	ir_print_coro_id(irp, (IrInstructionCoroId *)instruction);
		1486	break;
		1487	case IrInstructionIdCoroAlloc:
		1488	ir_print_coro_alloc(irp, (IrInstructionCoroAlloc *)instruction);
		1489	break;
		1490	case IrInstructionIdCoroSize:
		1491	ir_print_coro_size(irp, (IrInstructionCoroSize *)instruction);
		1492	break;
		1493	case IrInstructionIdCoroBegin:
		1494	ir_print_coro_begin(irp, (IrInstructionCoroBegin *)instruction);
		1495	break;
		1496	case IrInstructionIdCoroAllocFail:
		1497	ir_print_coro_alloc_fail(irp, (IrInstructionCoroAllocFail *)instruction);
		1498	break;
		1499	case IrInstructionIdCoroSuspend:
		1500	ir_print_coro_suspend(irp, (IrInstructionCoroSuspend *)instruction);
		1501	break;
		1502	case IrInstructionIdCoroEnd:
		1503	ir_print_coro_end(irp, (IrInstructionCoroEnd *)instruction);
		1504	break;
		1505	case IrInstructionIdCoroFree:
		1506	ir_print_coro_free(irp, (IrInstructionCoroFree *)instruction);
		1507	break;
		1508	case IrInstructionIdCoroResume:
		1509	ir_print_coro_resume(irp, (IrInstructionCoroResume *)instruction);
		1510	break;
		1511	case IrInstructionIdCoroSave:
		1512	ir_print_coro_save(irp, (IrInstructionCoroSave *)instruction);
		1513	break;
		1514	case IrInstructionIdCoroAllocHelper:
		1515	ir_print_coro_alloc_helper(irp, (IrInstructionCoroAllocHelper *)instruction);
		1516	break;
		1517	case IrInstructionIdAtomicRmw:
		1518	ir_print_atomic_rmw(irp, (IrInstructionAtomicRmw *)instruction);
		1519	break;
		1520	case IrInstructionIdCoroPromise:
		1521	ir_print_coro_promise(irp, (IrInstructionCoroPromise *)instruction);
		1522	break;
		1523	case IrInstructionIdPromiseResultType:
		1524	ir_print_promise_result_type(irp, (IrInstructionPromiseResultType *)instruction);
		1525	break;
1333	}	1526	}
1334	fprintf(irp->f, "\n");	1527	fprintf(irp->f, "\n");
1335	}	1528	}

src/parser.cpp+170-9

...	@@ -221,6 +221,7 @@ static AstNode ast_parse_grouped_expr(ParseContext pc, size_t *token_index, bo	...	@@ -221,6 +221,7 @@ static AstNode ast_parse_grouped_expr(ParseContext pc, size_t *token_index, bo
221	static AstNode ast_parse_container_decl(ParseContext pc, size_t *token_index, bool mandatory);	221	static AstNode ast_parse_container_decl(ParseContext pc, size_t *token_index, bool mandatory);
222	static AstNode ast_parse_primary_expr(ParseContext pc, size_t *token_index, bool mandatory);	222	static AstNode ast_parse_primary_expr(ParseContext pc, size_t *token_index, bool mandatory);
223	static AstNode ast_parse_try_expr(ParseContext pc, size_t *token_index);	223	static AstNode ast_parse_try_expr(ParseContext pc, size_t *token_index);
		224	static AstNode ast_parse_await_expr(ParseContext pc, size_t *token_index);
224	static AstNode ast_parse_symbol(ParseContext pc, size_t *token_index);	225	static AstNode ast_parse_symbol(ParseContext pc, size_t *token_index);
225		226
226	static void ast_expect_token(ParseContext pc, Token token, TokenId token_id) {	227	static void ast_expect_token(ParseContext pc, Token token, TokenId token_id) {
...	@@ -650,6 +651,41 @@ static AstNode ast_parse_asm_expr(ParseContext pc, size_t *token_index, bool m	...	@@ -650,6 +651,41 @@ static AstNode ast_parse_asm_expr(ParseContext pc, size_t *token_index, bool m
650	return node;	651	return node;
651	}	652	}
652		653
		654	/*
		655	SuspendExpression(body) = "suspend" "\|" Symbol "\|" body
		656	*/
		657	static AstNode ast_parse_suspend_block(ParseContext pc, size_t *token_index, bool mandatory) {
		658	size_t orig_token_index = *token_index;
		659
		660	Token suspend_token = &pc->tokens->at(token_index);
		661	if (suspend_token->id == TokenIdKeywordSuspend) {
		662	*token_index += 1;
		663	} else if (mandatory) {
		664	ast_expect_token(pc, suspend_token, TokenIdKeywordSuspend);
		665	zig_unreachable();
		666	} else {
		667	return nullptr;
		668	}
		669
		670	Token bar_token = &pc->tokens->at(token_index);
		671	if (bar_token->id == TokenIdBinOr) {
		672	*token_index += 1;
		673	} else if (mandatory) {
		674	ast_expect_token(pc, suspend_token, TokenIdBinOr);
		675	zig_unreachable();
		676	} else {
		677	*token_index = orig_token_index;
		678	return nullptr;
		679	}
		680
		681	AstNode *node = ast_create_node(pc, NodeTypeSuspend, suspend_token);
		682	node->data.suspend.promise_symbol = ast_parse_symbol(pc, token_index);
		683	ast_eat_token(pc, token_index, TokenIdBinOr);
		684	node->data.suspend.block = ast_parse_block(pc, token_index, true);
		685
		686	return node;
		687	}
		688
653	/*	689	/*
654	CompTimeExpression(body) = "comptime" body	690	CompTimeExpression(body) = "comptime" body
655	*/	691	*/
...	@@ -674,7 +710,7 @@ static AstNode ast_parse_comptime_expr(ParseContext pc, size_t *token_index, b	...	@@ -674,7 +710,7 @@ static AstNode ast_parse_comptime_expr(ParseContext pc, size_t *token_index, b
674		710
675	/*	711	/*
676	PrimaryExpression = Integer \| Float \| String \| CharLiteral \| KeywordLiteral \| GroupedExpression \| BlockExpression(BlockOrExpression) \| Symbol \| ("@" Symbol FnCallExpression) \| ArrayType \| FnProto \| AsmExpression \| ContainerDecl \| ("continue" option(":" Symbol)) \| ErrorSetDecl	712	PrimaryExpression = Integer \| Float \| String \| CharLiteral \| KeywordLiteral \| GroupedExpression \| BlockExpression(BlockOrExpression) \| Symbol \| ("@" Symbol FnCallExpression) \| ArrayType \| FnProto \| AsmExpression \| ContainerDecl \| ("continue" option(":" Symbol)) \| ErrorSetDecl
677	KeywordLiteral = "true" \| "false" \| "null" \| "undefined" \| "error" \| "this" \| "unreachable"	713	KeywordLiteral = "true" \| "false" \| "null" \| "undefined" \| "error" \| "this" \| "unreachable" \| "suspend"
678	ErrorSetDecl = "error" "{" list(Symbol, ",") "}"	714	ErrorSetDecl = "error" "{" list(Symbol, ",") "}"
679	*/	715	*/
680	static AstNode ast_parse_primary_expr(ParseContext pc, size_t *token_index, bool mandatory) {	716	static AstNode ast_parse_primary_expr(ParseContext pc, size_t *token_index, bool mandatory) {
...	@@ -738,6 +774,10 @@ static AstNode ast_parse_primary_expr(ParseContext pc, size_t *token_index, bo	...	@@ -738,6 +774,10 @@ static AstNode ast_parse_primary_expr(ParseContext pc, size_t *token_index, bo
738	AstNode *node = ast_create_node(pc, NodeTypeUnreachable, token);	774	AstNode *node = ast_create_node(pc, NodeTypeUnreachable, token);
739	*token_index += 1;	775	*token_index += 1;
740	return node;	776	return node;
		777	} else if (token->id == TokenIdKeywordSuspend) {
		778	AstNode *node = ast_create_node(pc, NodeTypeSuspend, token);
		779	*token_index += 1;
		780	return node;
741	} else if (token->id == TokenIdKeywordError) {	781	} else if (token->id == TokenIdKeywordError) {
742	Token next_token = &pc->tokens->at(token_index + 1);	782	Token next_token = &pc->tokens->at(token_index + 1);
743	if (next_token->id == TokenIdLBrace) {	783	if (next_token->id == TokenIdLBrace) {
...	@@ -920,7 +960,7 @@ static AstNode ast_parse_curly_suffix_expr(ParseContext pc, size_t *token_inde	...	@@ -920,7 +960,7 @@ static AstNode ast_parse_curly_suffix_expr(ParseContext pc, size_t *token_inde
920	}	960	}
921		961
922	/*	962	/*
923	SuffixOpExpression = PrimaryExpression option(FnCallExpression \| ArrayAccessExpression \| FieldAccessExpression \| SliceExpression)	963	SuffixOpExpression = ("async" option("(" Expression ")") PrimaryExpression FnCallExpression) \| PrimaryExpression option(FnCallExpression \| ArrayAccessExpression \| FieldAccessExpression \| SliceExpression)
924	FnCallExpression : token(LParen) list(Expression, token(Comma)) token(RParen)	964	FnCallExpression : token(LParen) list(Expression, token(Comma)) token(RParen)
925	ArrayAccessExpression : token(LBracket) Expression token(RBracket)	965	ArrayAccessExpression : token(LBracket) Expression token(RBracket)
926	SliceExpression = "[" Expression ".." option(Expression) "]"	966	SliceExpression = "[" Expression ".." option(Expression) "]"
...	@@ -928,9 +968,34 @@ FieldAccessExpression : token(Dot) token(Symbol)	...	@@ -928,9 +968,34 @@ FieldAccessExpression : token(Dot) token(Symbol)
928	StructLiteralField : token(Dot) token(Symbol) token(Eq) Expression	968	StructLiteralField : token(Dot) token(Symbol) token(Eq) Expression
929	*/	969	*/
930	static AstNode ast_parse_suffix_op_expr(ParseContext pc, size_t *token_index, bool mandatory) {	970	static AstNode ast_parse_suffix_op_expr(ParseContext pc, size_t *token_index, bool mandatory) {
931	AstNode *primary_expr = ast_parse_primary_expr(pc, token_index, mandatory);	971	AstNode *primary_expr;
932	if (!primary_expr)	972
933	return nullptr;	973	Token async_token = &pc->tokens->at(token_index);
		974	if (async_token->id == TokenIdKeywordAsync) {
		975	*token_index += 1;
		976
		977	AstNode *allocator_expr_node = nullptr;
		978	Token async_lparen_tok = &pc->tokens->at(token_index);
		979	if (async_lparen_tok->id == TokenIdLParen) {
		980	*token_index += 1;
		981	allocator_expr_node = ast_parse_expression(pc, token_index, true);
		982	ast_eat_token(pc, token_index, TokenIdRParen);
		983	}
		984
		985	AstNode *fn_ref_expr_node = ast_parse_primary_expr(pc, token_index, true);
		986	Token *lparen_tok = ast_eat_token(pc, token_index, TokenIdLParen);
		987	AstNode *node = ast_create_node(pc, NodeTypeFnCallExpr, lparen_tok);
		988	node->data.fn_call_expr.is_async = true;
		989	node->data.fn_call_expr.async_allocator = allocator_expr_node;
		990	node->data.fn_call_expr.fn_ref_expr = fn_ref_expr_node;
		991	ast_parse_fn_call_param_list(pc, token_index, &node->data.fn_call_expr.params);
		992
		993	primary_expr = node;
		994	} else {
		995	primary_expr = ast_parse_primary_expr(pc, token_index, mandatory);
		996	if (!primary_expr)
		997	return nullptr;
		998	}
934		999
935	while (true) {	1000	while (true) {
936	Token first_token = &pc->tokens->at(token_index);	1001	Token first_token = &pc->tokens->at(token_index);
...	@@ -1042,7 +1107,7 @@ static AstNode ast_parse_addr_of(ParseContext pc, size_t *token_index) {	...	@@ -1042,7 +1107,7 @@ static AstNode ast_parse_addr_of(ParseContext pc, size_t *token_index) {
1042		1107
1043	/*	1108	/*
1044	PrefixOpExpression = PrefixOp ErrorSetExpr \| SuffixOpExpression	1109	PrefixOpExpression = PrefixOp ErrorSetExpr \| SuffixOpExpression
1045	PrefixOp = "!" \| "-" \| "~" \| "*" \| ("&" option("align" "(" Expression option(":" Integer ":" Integer) ")" ) option("const") option("volatile")) \| "?" \| "??" \| "-%" \| "try"	1110	PrefixOp = "!" \| "-" \| "~" \| "*" \| ("&" option("align" "(" Expression option(":" Integer ":" Integer) ")" ) option("const") option("volatile")) \| "?" \| "??" \| "-%" \| "try" \| "await"
1046	*/	1111	*/
1047	static AstNode ast_parse_prefix_op_expr(ParseContext pc, size_t *token_index, bool mandatory) {	1112	static AstNode ast_parse_prefix_op_expr(ParseContext pc, size_t *token_index, bool mandatory) {
1048	Token token = &pc->tokens->at(token_index);	1113	Token token = &pc->tokens->at(token_index);
...	@@ -1052,6 +1117,9 @@ static AstNode ast_parse_prefix_op_expr(ParseContext pc, size_t *token_index,	...	@@ -1052,6 +1117,9 @@ static AstNode ast_parse_prefix_op_expr(ParseContext pc, size_t *token_index,
1052	if (token->id == TokenIdKeywordTry) {	1117	if (token->id == TokenIdKeywordTry) {
1053	return ast_parse_try_expr(pc, token_index);	1118	return ast_parse_try_expr(pc, token_index);
1054	}	1119	}
		1120	if (token->id == TokenIdKeywordAwait) {
		1121	return ast_parse_await_expr(pc, token_index);
		1122	}
1055	PrefixOp prefix_op = tok_to_prefix_op(token);	1123	PrefixOp prefix_op = tok_to_prefix_op(token);
1056	if (prefix_op == PrefixOpInvalid) {	1124	if (prefix_op == PrefixOpInvalid) {
1057	return ast_parse_suffix_op_expr(pc, token_index, mandatory);	1125	return ast_parse_suffix_op_expr(pc, token_index, mandatory);
...	@@ -1510,6 +1578,23 @@ static AstNode ast_parse_try_expr(ParseContext pc, size_t *token_index) {	...	@@ -1510,6 +1578,23 @@ static AstNode ast_parse_try_expr(ParseContext pc, size_t *token_index) {
1510	return node;	1578	return node;
1511	}	1579	}
1512		1580
		1581	/*
		1582	AwaitExpression : "await" Expression
		1583	*/
		1584	static AstNode ast_parse_await_expr(ParseContext pc, size_t *token_index) {
		1585	Token token = &pc->tokens->at(token_index);
		1586
		1587	if (token->id != TokenIdKeywordAwait) {
		1588	return nullptr;
		1589	}
		1590	*token_index += 1;
		1591
		1592	AstNode *node = ast_create_node(pc, NodeTypeAwaitExpr, token);
		1593	node->data.await_expr.expr = ast_parse_expression(pc, token_index, true);
		1594
		1595	return node;
		1596	}
		1597
1513	/*	1598	/*
1514	BreakExpression = "break" option(":" Symbol) option(Expression)	1599	BreakExpression = "break" option(":" Symbol) option(Expression)
1515	*/	1600	*/
...	@@ -1535,6 +1620,42 @@ static AstNode ast_parse_break_expr(ParseContext pc, size_t *token_index) {	...	@@ -1535,6 +1620,42 @@ static AstNode ast_parse_break_expr(ParseContext pc, size_t *token_index) {
1535	return node;	1620	return node;
1536	}	1621	}
1537		1622
		1623	/*
		1624	CancelExpression = "cancel" Expression;
		1625	*/
		1626	static AstNode ast_parse_cancel_expr(ParseContext pc, size_t *token_index) {
		1627	Token token = &pc->tokens->at(token_index);
		1628
		1629	if (token->id != TokenIdKeywordCancel) {
		1630	return nullptr;
		1631	}
		1632	*token_index += 1;
		1633
		1634	AstNode *node = ast_create_node(pc, NodeTypeCancel, token);
		1635
		1636	node->data.cancel_expr.expr = ast_parse_expression(pc, token_index, false);
		1637
		1638	return node;
		1639	}
		1640
		1641	/*
		1642	ResumeExpression = "resume" Expression;
		1643	*/
		1644	static AstNode ast_parse_resume_expr(ParseContext pc, size_t *token_index) {
		1645	Token token = &pc->tokens->at(token_index);
		1646
		1647	if (token->id != TokenIdKeywordResume) {
		1648	return nullptr;
		1649	}
		1650	*token_index += 1;
		1651
		1652	AstNode *node = ast_create_node(pc, NodeTypeResume, token);
		1653
		1654	node->data.resume_expr.expr = ast_parse_expression(pc, token_index, false);
		1655
		1656	return node;
		1657	}
		1658
1538	/*	1659	/*
1539	Defer(body) = ("defer" \| "errdefer") body	1660	Defer(body) = ("defer" \| "errdefer") body
1540	*/	1661	*/
...	@@ -2001,7 +2122,7 @@ static AstNode ast_parse_switch_expr(ParseContext pc, size_t *token_index, boo	...	@@ -2001,7 +2122,7 @@ static AstNode ast_parse_switch_expr(ParseContext pc, size_t *token_index, boo
2001	}	2122	}
2002		2123
2003	/*	2124	/*
2004	BlockExpression(body) = Block \| IfExpression(body) \| TryExpression(body) \| TestExpression(body) \| WhileExpression(body) \| ForExpression(body) \| SwitchExpression \| CompTimeExpression(body)	2125	BlockExpression(body) = Block \| IfExpression(body) \| IfErrorExpression(body) \| TestExpression(body) \| WhileExpression(body) \| ForExpression(body) \| SwitchExpression \| CompTimeExpression(body) \| SuspendExpression(body)
2005	*/	2126	*/
2006	static AstNode ast_parse_block_expr(ParseContext pc, size_t *token_index, bool mandatory) {	2127	static AstNode ast_parse_block_expr(ParseContext pc, size_t *token_index, bool mandatory) {
2007	Token token = &pc->tokens->at(token_index);	2128	Token token = &pc->tokens->at(token_index);
...	@@ -2030,6 +2151,10 @@ static AstNode ast_parse_block_expr(ParseContext pc, size_t *token_index, bool	...	@@ -2030,6 +2151,10 @@ static AstNode ast_parse_block_expr(ParseContext pc, size_t *token_index, bool
2030	if (comptime_node)	2151	if (comptime_node)
2031	return comptime_node;	2152	return comptime_node;
2032		2153
		2154	AstNode *suspend_node = ast_parse_suspend_block(pc, token_index, false);
		2155	if (suspend_node)
		2156	return suspend_node;
		2157
2033	if (mandatory)	2158	if (mandatory)
2034	ast_invalid_token_error(pc, token);	2159	ast_invalid_token_error(pc, token);
2035		2160
...	@@ -2159,7 +2284,7 @@ static AstNode ast_parse_block_or_expression(ParseContext pc, size_t *token_in	...	@@ -2159,7 +2284,7 @@ static AstNode ast_parse_block_or_expression(ParseContext pc, size_t *token_in
2159	}	2284	}
2160		2285
2161	/*	2286	/*
2162	Expression = TryExpression \| ReturnExpression \| BreakExpression \| AssignmentExpression	2287	Expression = TryExpression \| ReturnExpression \| BreakExpression \| AssignmentExpression \| CancelExpression \| ResumeExpression
2163	*/	2288	*/
2164	static AstNode ast_parse_expression(ParseContext pc, size_t *token_index, bool mandatory) {	2289	static AstNode ast_parse_expression(ParseContext pc, size_t *token_index, bool mandatory) {
2165	Token token = &pc->tokens->at(token_index);	2290	Token token = &pc->tokens->at(token_index);
...	@@ -2176,6 +2301,14 @@ static AstNode ast_parse_expression(ParseContext pc, size_t *token_index, bool	...	@@ -2176,6 +2301,14 @@ static AstNode ast_parse_expression(ParseContext pc, size_t *token_index, bool
2176	if (break_expr)	2301	if (break_expr)
2177	return break_expr;	2302	return break_expr;
2178		2303
		2304	AstNode *cancel_expr = ast_parse_cancel_expr(pc, token_index);
		2305	if (cancel_expr)
		2306	return cancel_expr;
		2307
		2308	AstNode *resume_expr = ast_parse_resume_expr(pc, token_index);
		2309	if (resume_expr)
		2310	return resume_expr;
		2311
2179	AstNode *ass_expr = ast_parse_ass_expr(pc, token_index, false);	2312	AstNode *ass_expr = ast_parse_ass_expr(pc, token_index, false);
2180	if (ass_expr)	2313	if (ass_expr)
2181	return ass_expr;	2314	return ass_expr;
...	@@ -2208,6 +2341,8 @@ static bool statement_terminates_without_semicolon(AstNode *node) {	...	@@ -2208,6 +2341,8 @@ static bool statement_terminates_without_semicolon(AstNode *node) {
2208	return node->data.comptime_expr.expr->type == NodeTypeBlock;	2341	return node->data.comptime_expr.expr->type == NodeTypeBlock;
2209	case NodeTypeDefer:	2342	case NodeTypeDefer:
2210	return node->data.defer.expr->type == NodeTypeBlock;	2343	return node->data.defer.expr->type == NodeTypeBlock;
		2344	case NodeTypeSuspend:
		2345	return node->data.suspend.block != nullptr && node->data.suspend.block->type == NodeTypeBlock;
2211	case NodeTypeSwitchExpr:	2346	case NodeTypeSwitchExpr:
2212	case NodeTypeBlock:	2347	case NodeTypeBlock:
2213	return true;	2348	return true;
...	@@ -2286,7 +2421,7 @@ static AstNode ast_parse_block(ParseContext pc, size_t *token_index, bool mand	...	@@ -2286,7 +2421,7 @@ static AstNode ast_parse_block(ParseContext pc, size_t *token_index, bool mand
2286	}	2421	}
2287		2422
2288	/*	2423	/*
2289	FnProto = option("nakedcc" \| "stdcallcc" \| "extern") "fn" option(Symbol) ParamDeclList option("align" "(" Expression ")") option("section" "(" Expression ")") option("!") TypeExpr	2424	FnProto = option("nakedcc" \| "stdcallcc" \| "extern" \| ("async" option("(" Expression ")"))) "fn" option(Symbol) ParamDeclList option("align" "(" Expression ")") option("section" "(" Expression ")") option("!") TypeExpr
2290	*/	2425	*/
2291	static AstNode ast_parse_fn_proto(ParseContext pc, size_t *token_index, bool mandatory, VisibMod visib_mod) {	2426	static AstNode ast_parse_fn_proto(ParseContext pc, size_t *token_index, bool mandatory, VisibMod visib_mod) {
2292	Token first_token = &pc->tokens->at(token_index);	2427	Token first_token = &pc->tokens->at(token_index);
...	@@ -2294,10 +2429,20 @@ static AstNode ast_parse_fn_proto(ParseContext pc, size_t *token_index, bool m	...	@@ -2294,10 +2429,20 @@ static AstNode ast_parse_fn_proto(ParseContext pc, size_t *token_index, bool m
2294		2429
2295	CallingConvention cc;	2430	CallingConvention cc;
2296	bool is_extern = false;	2431	bool is_extern = false;
		2432	AstNode *async_allocator_type_node = nullptr;
2297	if (first_token->id == TokenIdKeywordNakedCC) {	2433	if (first_token->id == TokenIdKeywordNakedCC) {
2298	*token_index += 1;	2434	*token_index += 1;
2299	fn_token = ast_eat_token(pc, token_index, TokenIdKeywordFn);	2435	fn_token = ast_eat_token(pc, token_index, TokenIdKeywordFn);
2300	cc = CallingConventionNaked;	2436	cc = CallingConventionNaked;
		2437	} else if (first_token->id == TokenIdKeywordAsync) {
		2438	*token_index += 1;
		2439	Token next_token = &pc->tokens->at(token_index);
		2440	if (next_token->id == TokenIdLParen) {
		2441	async_allocator_type_node = ast_parse_type_expr(pc, token_index, true);
		2442	ast_eat_token(pc, token_index, TokenIdRParen);
		2443	}
		2444	fn_token = ast_eat_token(pc, token_index, TokenIdKeywordFn);
		2445	cc = CallingConventionAsync;
2301	} else if (first_token->id == TokenIdKeywordStdcallCC) {	2446	} else if (first_token->id == TokenIdKeywordStdcallCC) {
2302	*token_index += 1;	2447	*token_index += 1;
2303	fn_token = ast_eat_token(pc, token_index, TokenIdKeywordFn);	2448	fn_token = ast_eat_token(pc, token_index, TokenIdKeywordFn);
...	@@ -2332,6 +2477,7 @@ static AstNode ast_parse_fn_proto(ParseContext pc, size_t *token_index, bool m	...	@@ -2332,6 +2477,7 @@ static AstNode ast_parse_fn_proto(ParseContext pc, size_t *token_index, bool m
2332	node->data.fn_proto.visib_mod = visib_mod;	2477	node->data.fn_proto.visib_mod = visib_mod;
2333	node->data.fn_proto.cc = cc;	2478	node->data.fn_proto.cc = cc;
2334	node->data.fn_proto.is_extern = is_extern;	2479	node->data.fn_proto.is_extern = is_extern;
		2480	node->data.fn_proto.async_allocator_type = async_allocator_type_node;
2335		2481
2336	Token fn_name = &pc->tokens->at(token_index);	2482	Token fn_name = &pc->tokens->at(token_index);
2337		2483
...	@@ -2747,6 +2893,7 @@ void ast_visit_node_children(AstNode node, void (visit)(AstNode *, void cont	...	@@ -2747,6 +2893,7 @@ void ast_visit_node_children(AstNode node, void (visit)(AstNode *, void cont
2747	visit_node_list(&node->data.fn_proto.params, visit, context);	2893	visit_node_list(&node->data.fn_proto.params, visit, context);
2748	visit_field(&node->data.fn_proto.align_expr, visit, context);	2894	visit_field(&node->data.fn_proto.align_expr, visit, context);
2749	visit_field(&node->data.fn_proto.section_expr, visit, context);	2895	visit_field(&node->data.fn_proto.section_expr, visit, context);
		2896	visit_field(&node->data.fn_proto.async_allocator_type, visit, context);
2750	break;	2897	break;
2751	case NodeTypeFnDef:	2898	case NodeTypeFnDef:
2752	visit_field(&node->data.fn_def.fn_proto, visit, context);	2899	visit_field(&node->data.fn_def.fn_proto, visit, context);
...	@@ -2809,6 +2956,7 @@ void ast_visit_node_children(AstNode node, void (visit)(AstNode *, void cont	...	@@ -2809,6 +2956,7 @@ void ast_visit_node_children(AstNode node, void (visit)(AstNode *, void cont
2809	case NodeTypeFnCallExpr:	2956	case NodeTypeFnCallExpr:
2810	visit_field(&node->data.fn_call_expr.fn_ref_expr, visit, context);	2957	visit_field(&node->data.fn_call_expr.fn_ref_expr, visit, context);
2811	visit_node_list(&node->data.fn_call_expr.params, visit, context);	2958	visit_node_list(&node->data.fn_call_expr.params, visit, context);
		2959	visit_field(&node->data.fn_call_expr.async_allocator, visit, context);
2812	break;	2960	break;
2813	case NodeTypeArrayAccessExpr:	2961	case NodeTypeArrayAccessExpr:
2814	visit_field(&node->data.array_access_expr.array_ref_expr, visit, context);	2962	visit_field(&node->data.array_access_expr.array_ref_expr, visit, context);
...	@@ -2931,5 +3079,18 @@ void ast_visit_node_children(AstNode node, void (visit)(AstNode *, void cont	...	@@ -2931,5 +3079,18 @@ void ast_visit_node_children(AstNode node, void (visit)(AstNode *, void cont
2931	case NodeTypeErrorSetDecl:	3079	case NodeTypeErrorSetDecl:
2932	visit_node_list(&node->data.err_set_decl.decls, visit, context);	3080	visit_node_list(&node->data.err_set_decl.decls, visit, context);
2933	break;	3081	break;
		3082	case NodeTypeCancel:
		3083	visit_field(&node->data.cancel_expr.expr, visit, context);
		3084	break;
		3085	case NodeTypeResume:
		3086	visit_field(&node->data.resume_expr.expr, visit, context);
		3087	break;
		3088	case NodeTypeAwaitExpr:
		3089	visit_field(&node->data.await_expr.expr, visit, context);
		3090	break;
		3091	case NodeTypeSuspend:
		3092	visit_field(&node->data.suspend.promise_symbol, visit, context);
		3093	visit_field(&node->data.suspend.block, visit, context);
		3094	break;
2934	}	3095	}
2935	}	3096	}

src/tokenizer.cpp+10

 @@ -110,7 +110,10 @@ static const struct ZigKeyword zig_keywords[] = {
     {"align", TokenIdKeywordAlign},
     {"and", TokenIdKeywordAnd},
     {"asm", TokenIdKeywordAsm},
+    {"async", TokenIdKeywordAsync},
+    {"await", TokenIdKeywordAwait},
     {"break", TokenIdKeywordBreak},
+    {"cancel", TokenIdKeywordCancel},
     {"catch", TokenIdKeywordCatch},
     {"comptime", TokenIdKeywordCompTime},
     {"const", TokenIdKeywordConst},
 @@ -133,10 +136,12 @@ static const struct ZigKeyword zig_keywords[] = {
     {"or", TokenIdKeywordOr},
     {"packed", TokenIdKeywordPacked},
     {"pub", TokenIdKeywordPub},
+    {"resume", TokenIdKeywordResume},
     {"return", TokenIdKeywordReturn},
     {"section", TokenIdKeywordSection},
     {"stdcallcc", TokenIdKeywordStdcallCC},
     {"struct", TokenIdKeywordStruct},
+    {"suspend", TokenIdKeywordSuspend},
     {"switch", TokenIdKeywordSwitch},
     {"test", TokenIdKeywordTest},
     {"this", TokenIdKeywordThis},
 @@ -1523,6 +1528,11 @@ const char * token_name(TokenId id) {
         case TokenIdFatArrow: return "=>";
         case TokenIdFloatLiteral: return "FloatLiteral";
         case TokenIdIntLiteral: return "IntLiteral";
+        case TokenIdKeywordAsync: return "async";
+        case TokenIdKeywordAwait: return "await";
+        case TokenIdKeywordResume: return "resume";
+        case TokenIdKeywordSuspend: return "suspend";
+        case TokenIdKeywordCancel: return "cancel";
         case TokenIdKeywordAlign: return "align";
         case TokenIdKeywordAnd: return "and";
         case TokenIdKeywordAsm: return "asm";

src/tokenizer.hpp+5

 @@ -51,7 +51,10 @@ enum TokenId {
     TokenIdKeywordAlign,
     TokenIdKeywordAnd,
     TokenIdKeywordAsm,
+    TokenIdKeywordAsync,
+    TokenIdKeywordAwait,
     TokenIdKeywordBreak,
+    TokenIdKeywordCancel,
     TokenIdKeywordCatch,
     TokenIdKeywordCompTime,
     TokenIdKeywordConst,
 @@ -74,10 +77,12 @@ enum TokenId {
     TokenIdKeywordOr,
     TokenIdKeywordPacked,
     TokenIdKeywordPub,
+    TokenIdKeywordResume,
     TokenIdKeywordReturn,
     TokenIdKeywordSection,
     TokenIdKeywordStdcallCC,
     TokenIdKeywordStruct,
+    TokenIdKeywordSuspend,
     TokenIdKeywordSwitch,
     TokenIdKeywordTest,
     TokenIdKeywordThis,

src/zig_llvm.cpp+6

src/zig_llvm.h+2

...	@@ -54,6 +54,8 @@ enum ZigLLVM_EmitOutputType {	...	@@ -54,6 +54,8 @@ enum ZigLLVM_EmitOutputType {
54	ZIG_EXTERN_C bool ZigLLVMTargetMachineEmitToFile(LLVMTargetMachineRef targ_machine_ref, LLVMModuleRef module_ref,	54	ZIG_EXTERN_C bool ZigLLVMTargetMachineEmitToFile(LLVMTargetMachineRef targ_machine_ref, LLVMModuleRef module_ref,
55	const char filename, enum ZigLLVM_EmitOutputType output_type, char *error_message, bool is_debug);	55	const char filename, enum ZigLLVM_EmitOutputType output_type, char *error_message, bool is_debug);
56		56
		57	ZIG_EXTERN_C LLVMTypeRef ZigLLVMTokenTypeInContext(LLVMContextRef context_ref);
		58
57	enum ZigLLVM_FnInline {	59	enum ZigLLVM_FnInline {
58	ZigLLVM_FnInlineAuto,	60	ZigLLVM_FnInlineAuto,
59	ZigLLVM_FnInlineAlways,	61	ZigLLVM_FnInlineAlways,

std/debug/index.zig+8-10

 @@ -98,21 +98,18 @@ pub fn assertOrPanic(ok: bool) void {
     }
 }
-var panicking = false;
+var panicking: u8 = 0; // TODO make this a bool
 /// This is the default panic implementation.
 pub fn panic(comptime format: []const u8, args: ...) noreturn {
-    // TODO an intrinsic that labels this as unlikely to be reached
+    @setCold(true);
-    // TODO
+    if (@atomicRmw(u8, &panicking, builtin.AtomicRmwOp.Xchg, 1, builtin.AtomicOrder.SeqCst) == 1) {
-    // if (@atomicRmw(AtomicOp.XChg, &panicking, true, AtomicOrder.SeqCst)) { }
-    if (panicking) {
         // Panicked during a panic.
         // TODO detect if a different thread caused the panic, because in that case
         // we would want to return here instead of calling abort, so that the thread
         // which first called panic can finish printing a stack trace.
         os.abort();
-    } else {
-        panicking = true;
     }
     const stderr = getStderrStream() catch os.abort();
 @@ -123,10 +120,11 @@ pub fn panic(comptime format: []const u8, args: ...) noreturn {
 }
 pub fn panicWithTrace(trace: &const builtin.StackTrace, comptime format: []const u8, args: ...) noreturn {
-    if (panicking) {
+    @setCold(true);
+    if (@atomicRmw(u8, &panicking, builtin.AtomicRmwOp.Xchg, 1, builtin.AtomicOrder.SeqCst) == 1) {
+        // See TODO in above function
         os.abort();
-    } else {
-        panicking = true;
     }
     const stderr = getStderrStream() catch os.abort();
     stderr.print(format ++ "\n", args) catch os.abort();

std/hash_map.zig+1-1

...	@@ -235,7 +235,7 @@ pub fn HashMap(comptime K: type, comptime V: type,	...	@@ -235,7 +235,7 @@ pub fn HashMap(comptime K: type, comptime V: type,
235	};	235	};
236	}	236	}
237		237
238	test "basicHashMapTest" {	238	test "basic hash map usage" {
239	var map = HashMap(i32, i32, hash_i32, eql_i32).init(debug.global_allocator);	239	var map = HashMap(i32, i32, hash_i32, eql_i32).init(debug.global_allocator);
240	defer map.deinit();	240	defer map.deinit();
241		241

std/os/child_process.zig-50

 @@ -32,9 +32,6 @@ pub const ChildProcess = struct {
     pub argv: []const []const u8,
-    /// Possibly called from a signal handler. Must set this before calling `spawn`.
-    pub onTerm: ?fn(&ChildProcess)void,
     /// Leave as null to use the current env map using the supplied allocator.
     pub env_map: ?&const BufMap,
 @@ -102,7 +99,6 @@ pub const ChildProcess = struct {
             .err_pipe = undefined,
             .llnode = undefined,
             .term = null,
-            .onTerm = null,
             .env_map = null,
             .cwd = null,
             .uid = if (is_windows) {} else null,
 @@ -124,7 +120,6 @@ pub const ChildProcess = struct {
         self.gid = user_info.gid;
     }
-    /// onTerm can be called before `spawn` returns.
     /// On success must call `kill` or `wait`.
     pub fn spawn(self: &ChildProcess) !void {
         if (is_windows) {
 @@ -165,9 +160,6 @@ pub const ChildProcess = struct {
     }
     pub fn killPosix(self: &ChildProcess) !Term {
-        block_SIGCHLD();
-        defer restore_SIGCHLD();
         if (self.term) |term| {
             self.cleanupStreams();
             return term;
 @@ -246,9 +238,6 @@ pub const ChildProcess = struct {
     }
     fn waitPosix(self: &ChildProcess) !Term {
-        block_SIGCHLD();
-        defer restore_SIGCHLD();
         if (self.term) |term| {
             self.cleanupStreams();
             return term;
 @@ -298,10 +287,6 @@ pub const ChildProcess = struct {
     fn handleWaitResult(self: &ChildProcess, status: i32) void {
         self.term = self.cleanupAfterWait(status);
-        if (self.onTerm) |onTerm| {
-            onTerm(self);
-        }
     }
     fn cleanupStreams(self: &ChildProcess) void {
 @@ -347,9 +332,6 @@ pub const ChildProcess = struct {
     }
     fn spawnPosix(self: &ChildProcess) !void {
-        // TODO atomically set a flag saying that we already did this
-        install_SIGCHLD_handler();
         const stdin_pipe = if (self.stdin_behavior == StdIo.Pipe) try makePipe() else undefined;
         errdefer if (self.stdin_behavior == StdIo.Pipe) { destroyPipe(stdin_pipe); };
 @@ -387,11 +369,9 @@ pub const ChildProcess = struct {
         const err_pipe = try makePipe();
         errdefer destroyPipe(err_pipe);
-        block_SIGCHLD();
         const pid_result = posix.fork();
         const pid_err = posix.getErrno(pid_result);
         if (pid_err > 0) {
-            restore_SIGCHLD();
             return switch (pid_err) {
                 posix.EAGAIN, posix.ENOMEM, posix.ENOSYS => error.SystemResources,
                 else => os.unexpectedErrorPosix(pid_err),
 @@ -399,7 +379,6 @@ pub const ChildProcess = struct {
         }
         if (pid_result == 0) {
             // we are the child
-            restore_SIGCHLD();
             setUpChildIo(self.stdin_behavior, stdin_pipe[0], posix.STDIN_FILENO, dev_null_fd) catch
                 |err| forkChildErrReport(err_pipe[1], err);
 @@ -451,8 +430,6 @@ pub const ChildProcess = struct {
         // TODO make this atomic so it works even with threads
         children_nodes.prepend(&self.llnode);
-        restore_SIGCHLD();
         if (self.stdin_behavior == StdIo.Pipe) { os.close(stdin_pipe[0]); }
         if (self.stdout_behavior == StdIo.Pipe) { os.close(stdout_pipe[1]); }
         if (self.stderr_behavior == StdIo.Pipe) { os.close(stderr_pipe[1]); }
 @@ -824,30 +801,3 @@ fn handleTerm(pid: i32, status: i32) void {
         }
     }
 }
-const sigchld_set = x: {
-    var signal_set = posix.empty_sigset;
-    posix.sigaddset(&signal_set, posix.SIGCHLD);
-    break :x signal_set;
-};
-fn block_SIGCHLD() void {
-    const err = posix.getErrno(posix.sigprocmask(posix.SIG_BLOCK, &sigchld_set, null));
-    assert(err == 0);
-}
-fn restore_SIGCHLD() void {
-    const err = posix.getErrno(posix.sigprocmask(posix.SIG_UNBLOCK, &sigchld_set, null));
-    assert(err == 0);
-}
-const sigchld_action = posix.Sigaction {
-    .handler = sigchld_handler,
-    .mask = posix.empty_sigset,
-    .flags = posix.SA_RESTART | posix.SA_NOCLDSTOP,
-};
-fn install_SIGCHLD_handler() void {
-    const err = posix.getErrno(posix.sigaction(posix.SIGCHLD, &sigchld_action, null));
-    assert(err == 0);
-}

std/unicode.zig+140-9

...	@@ -1,4 +1,5 @@	...	@@ -1,4 +1,5 @@
1	const std = @import("./index.zig");	1	const std = @import("./index.zig");
		2	const debug = std.debug;
2		3
3	/// Given the first byte of a UTF-8 codepoint,	4	/// Given the first byte of a UTF-8 codepoint,
4	/// returns a number 1-4 indicating the total length of the codepoint in bytes.	5	/// returns a number 1-4 indicating the total length of the codepoint in bytes.
...	@@ -25,8 +26,8 @@ pub fn utf8Decode(bytes: []const u8) !u32 {	...	@@ -25,8 +26,8 @@ pub fn utf8Decode(bytes: []const u8) !u32 {
25	};	26	};
26	}	27	}
27	pub fn utf8Decode2(bytes: []const u8) !u32 {	28	pub fn utf8Decode2(bytes: []const u8) !u32 {
28	std.debug.assert(bytes.len == 2);	29	debug.assert(bytes.len == 2);
29	std.debug.assert(bytes[0] & 0b11100000 == 0b11000000);	30	debug.assert(bytes[0] & 0b11100000 == 0b11000000);
30	var value: u32 = bytes[0] & 0b00011111;	31	var value: u32 = bytes[0] & 0b00011111;
31		32
32	if (bytes[1] & 0b11000000 != 0b10000000) return error.Utf8ExpectedContinuation;	33	if (bytes[1] & 0b11000000 != 0b10000000) return error.Utf8ExpectedContinuation;
...	@@ -38,8 +39,8 @@ pub fn utf8Decode2(bytes: []const u8) !u32 {	...	@@ -38,8 +39,8 @@ pub fn utf8Decode2(bytes: []const u8) !u32 {
38	return value;	39	return value;
39	}	40	}
40	pub fn utf8Decode3(bytes: []const u8) !u32 {	41	pub fn utf8Decode3(bytes: []const u8) !u32 {
41	std.debug.assert(bytes.len == 3);	42	debug.assert(bytes.len == 3);
42	std.debug.assert(bytes[0] & 0b11110000 == 0b11100000);	43	debug.assert(bytes[0] & 0b11110000 == 0b11100000);
43	var value: u32 = bytes[0] & 0b00001111;	44	var value: u32 = bytes[0] & 0b00001111;
44		45
45	if (bytes[1] & 0b11000000 != 0b10000000) return error.Utf8ExpectedContinuation;	46	if (bytes[1] & 0b11000000 != 0b10000000) return error.Utf8ExpectedContinuation;
...	@@ -56,8 +57,8 @@ pub fn utf8Decode3(bytes: []const u8) !u32 {	...	@@ -56,8 +57,8 @@ pub fn utf8Decode3(bytes: []const u8) !u32 {
56	return value;	57	return value;
57	}	58	}
58	pub fn utf8Decode4(bytes: []const u8) !u32 {	59	pub fn utf8Decode4(bytes: []const u8) !u32 {
59	std.debug.assert(bytes.len == 4);	60	debug.assert(bytes.len == 4);
60	std.debug.assert(bytes[0] & 0b11111000 == 0b11110000);	61	debug.assert(bytes[0] & 0b11111000 == 0b11110000);
61	var value: u32 = bytes[0] & 0b00000111;	62	var value: u32 = bytes[0] & 0b00000111;
62		63
63	if (bytes[1] & 0b11000000 != 0b10000000) return error.Utf8ExpectedContinuation;	64	if (bytes[1] & 0b11000000 != 0b10000000) return error.Utf8ExpectedContinuation;
...	@@ -78,6 +79,136 @@ pub fn utf8Decode4(bytes: []const u8) !u32 {	...	@@ -78,6 +79,136 @@ pub fn utf8Decode4(bytes: []const u8) !u32 {
78	return value;	79	return value;
79	}	80	}
80		81
		82	pub fn utf8ValidateSlice(s: []const u8) bool {
		83	var i: usize = 0;
		84	while (i < s.len) {
		85	if (utf8ByteSequenceLength(s[i])) \|cp_len\| {
		86	if (i + cp_len > s.len) {
		87	return false;
		88	}
		89
		90	if (utf8Decode(s[i..i+cp_len])) \|_\| {} else \|_\| { return false; }
		91	i += cp_len;
		92	} else \|err\| {
		93	return false;
		94	}
		95	}
		96	return true;
		97	}
		98
		99	const Utf8View = struct {
		100	bytes: []const u8,
		101
		102	pub fn init(s: []const u8) !Utf8View {
		103	if (!utf8ValidateSlice(s)) {
		104	return error.InvalidUtf8;
		105	}
		106
		107	return initUnchecked(s);
		108	}
		109
		110	pub fn initUnchecked(s: []const u8) Utf8View {
		111	return Utf8View {
		112	.bytes = s,
		113	};
		114	}
		115
		116	pub fn initComptime(comptime s: []const u8) Utf8View {
		117	if (comptime init(s)) \|r\| {
		118	return r;
		119	} else \|err\| switch (err) {
		120	error.InvalidUtf8 => {
		121	@compileError("invalid utf8");
		122	unreachable;
		123	}
		124	}
		125	}
		126
		127	pub fn Iterator(s: &const Utf8View) Utf8Iterator {
		128	return Utf8Iterator {
		129	.bytes = s.bytes,
		130	.i = 0,
		131	};
		132	}
		133	};
		134
		135	const Utf8Iterator = struct {
		136	bytes: []const u8,
		137	i: usize,
		138
		139	pub fn nextCodepointSlice(it: &Utf8Iterator) ?[]const u8 {
		140	if (it.i >= it.bytes.len) {
		141	return null;
		142	}
		143
		144	const cp_len = utf8ByteSequenceLength(it.bytes[it.i]) catch unreachable;
		145
		146	it.i += cp_len;
		147	return it.bytes[it.i-cp_len..it.i];
		148	}
		149
		150	pub fn nextCodepoint(it: &Utf8Iterator) ?u32 {
		151	const slice = it.nextCodepointSlice() ?? return null;
		152
		153	const r = switch (slice.len) {
		154	1 => u32(slice[0]),
		155	2 => utf8Decode2(slice),
		156	3 => utf8Decode3(slice),
		157	4 => utf8Decode4(slice),
		158	else => unreachable,
		159	};
		160
		161	return r catch unreachable;
		162	}
		163	};
		164
		165	test "utf8 iterator on ascii" {
		166	const s = Utf8View.initComptime("abc");
		167
		168	var it1 = s.Iterator();
		169	debug.assert(std.mem.eql(u8, "a", ??it1.nextCodepointSlice()));
		170	debug.assert(std.mem.eql(u8, "b", ??it1.nextCodepointSlice()));
		171	debug.assert(std.mem.eql(u8, "c", ??it1.nextCodepointSlice()));
		172	debug.assert(it1.nextCodepointSlice() == null);
		173
		174	var it2 = s.Iterator();
		175	debug.assert(??it2.nextCodepoint() == 'a');
		176	debug.assert(??it2.nextCodepoint() == 'b');
		177	debug.assert(??it2.nextCodepoint() == 'c');
		178	debug.assert(it2.nextCodepoint() == null);
		179	}
		180
		181	test "utf8 view bad" {
		182	// Compile-time error.
		183	// const s3 = Utf8View.initComptime("\xfe\xf2");
		184
		185	const s = Utf8View.init("hel\xadlo");
		186	if (s) \|_\| { unreachable; } else \|err\| { debug.assert(err == error.InvalidUtf8); }
		187	}
		188
		189	test "utf8 view ok" {
		190	const s = Utf8View.initComptime("東京市");
		191
		192	var it1 = s.Iterator();
		193	debug.assert(std.mem.eql(u8, "東", ??it1.nextCodepointSlice()));
		194	debug.assert(std.mem.eql(u8, "京", ??it1.nextCodepointSlice()));
		195	debug.assert(std.mem.eql(u8, "市", ??it1.nextCodepointSlice()));
		196	debug.assert(it1.nextCodepointSlice() == null);
		197
		198	var it2 = s.Iterator();
		199	debug.assert(??it2.nextCodepoint() == 0x6771);
		200	debug.assert(??it2.nextCodepoint() == 0x4eac);
		201	debug.assert(??it2.nextCodepoint() == 0x5e02);
		202	debug.assert(it2.nextCodepoint() == null);
		203	}
		204
		205	test "bad utf8 slice" {
		206	debug.assert(utf8ValidateSlice("abc"));
		207	debug.assert(!utf8ValidateSlice("abc\xc0"));
		208	debug.assert(!utf8ValidateSlice("abc\xc0abc"));
		209	debug.assert(utf8ValidateSlice("abc\xdf\xbf"));
		210	}
		211
81	test "valid utf8" {	212	test "valid utf8" {
82	testValid("\x00", 0x0);	213	testValid("\x00", 0x0);
83	testValid("\x20", 0x20);	214	testValid("\x20", 0x20);
...	@@ -145,17 +276,17 @@ fn testError(bytes: []const u8, expected_err: error) void {	...	@@ -145,17 +276,17 @@ fn testError(bytes: []const u8, expected_err: error) void {
145	if (testDecode(bytes)) \|_\| {	276	if (testDecode(bytes)) \|_\| {
146	unreachable;	277	unreachable;
147	} else \|err\| {	278	} else \|err\| {
148	std.debug.assert(err == expected_err);	279	debug.assert(err == expected_err);
149	}	280	}
150	}	281	}
151		282
152	fn testValid(bytes: []const u8, expected_codepoint: u32) void {	283	fn testValid(bytes: []const u8, expected_codepoint: u32) void {
153	std.debug.assert((testDecode(bytes) catch unreachable) == expected_codepoint);	284	debug.assert((testDecode(bytes) catch unreachable) == expected_codepoint);
154	}	285	}
155		286
156	fn testDecode(bytes: []const u8) !u32 {	287	fn testDecode(bytes: []const u8) !u32 {
157	const length = try utf8ByteSequenceLength(bytes[0]);	288	const length = try utf8ByteSequenceLength(bytes[0]);
158	if (bytes.len < length) return error.UnexpectedEof;	289	if (bytes.len < length) return error.UnexpectedEof;
159	std.debug.assert(bytes.len == length);	290	debug.assert(bytes.len == length);
160	return utf8Decode(bytes);	291	return utf8Decode(bytes);
161	}	292	}

test/behavior.zig+14-1

...	@@ -1,3 +1,5 @@	...	@@ -1,3 +1,5 @@
		1	const builtin = @import("builtin");
		2
1	comptime {	3	comptime {
2	_ = @import("cases/align.zig");	4	_ = @import("cases/align.zig");
3	_ = @import("cases/alignof.zig");	5	_ = @import("cases/alignof.zig");
...	@@ -34,8 +36,8 @@ comptime {	...	@@ -34,8 +36,8 @@ comptime {
34	_ = @import("cases/sizeof_and_typeof.zig");	36	_ = @import("cases/sizeof_and_typeof.zig");
35	_ = @import("cases/slice.zig");	37	_ = @import("cases/slice.zig");
36	_ = @import("cases/struct.zig");	38	_ = @import("cases/struct.zig");
37	_ = @import("cases/struct_contains_slice_of_itself.zig");
38	_ = @import("cases/struct_contains_null_ptr_itself.zig");	39	_ = @import("cases/struct_contains_null_ptr_itself.zig");
		40	_ = @import("cases/struct_contains_slice_of_itself.zig");
39	_ = @import("cases/switch.zig");	41	_ = @import("cases/switch.zig");
40	_ = @import("cases/switch_prong_err_enum.zig");	42	_ = @import("cases/switch_prong_err_enum.zig");
41	_ = @import("cases/switch_prong_implicit_cast.zig");	43	_ = @import("cases/switch_prong_implicit_cast.zig");
...	@@ -47,4 +49,15 @@ comptime {	...	@@ -47,4 +49,15 @@ comptime {
47	_ = @import("cases/var_args.zig");	49	_ = @import("cases/var_args.zig");
48	_ = @import("cases/void.zig");	50	_ = @import("cases/void.zig");
49	_ = @import("cases/while.zig");	51	_ = @import("cases/while.zig");
		52
		53
		54	// LLVM 5.0.1, 6.0.0, and trunk crash when attempting to optimize coroutine code.
		55	// So, Zig does not support ReleaseFast or ReleaseSafe for coroutines yet.
		56	// Luckily, Clang users are running into the same crashes, so folks from the LLVM
		57	// community are working on fixes. If we're really lucky they'll be fixed in 6.0.1.
		58	// Otherwise we can hope for 7.0.0.
		59	if (builtin.mode == builtin.Mode.Debug) {
		60	_ = @import("cases/coroutines.zig");
		61	}
		62
50	}	63	}

test/cases/atomics.zig+14-1

...	@@ -1,5 +1,7 @@	...	@@ -1,5 +1,7 @@
1	const assert = @import("std").debug.assert;	1	const assert = @import("std").debug.assert;
2	const AtomicOrder = @import("builtin").AtomicOrder;	2	const builtin = @import("builtin");
		3	const AtomicRmwOp = builtin.AtomicRmwOp;
		4	const AtomicOrder = builtin.AtomicOrder;
3		5
4	test "cmpxchg" {	6	test "cmpxchg" {
5	var x: i32 = 1234;	7	var x: i32 = 1234;
...	@@ -12,3 +14,14 @@ test "fence" {	...	@@ -12,3 +14,14 @@ test "fence" {
12	@fence(AtomicOrder.SeqCst);	14	@fence(AtomicOrder.SeqCst);
13	x = 5678;	15	x = 5678;
14	}	16	}
		17
		18	test "atomicrmw" {
		19	var data: u8 = 200;
		20	testAtomicRmw(&data);
		21	assert(data == 42);
		22	}
		23
		24	fn testAtomicRmw(ptr: &u8) void {
		25	const prev_value = @atomicRmw(u8, ptr, AtomicRmwOp.Xchg, 42, AtomicOrder.SeqCst);
		26	assert(prev_value == 200);
		27	}

test/cases/cast.zig+102

...	@@ -32,6 +32,108 @@ fn funcWithConstPtrPtr(x: &const &i32) void {	...	@@ -32,6 +32,108 @@ fn funcWithConstPtrPtr(x: &const &i32) void {
32	**x += 1;	32	**x += 1;
33	}	33	}
34		34
		35	test "implicitly cast a container to a const pointer of it" {
		36	const z = Struct(void) { .x = void{} };
		37	assert(0 == @sizeOf(@typeOf(z)));
		38	assert(void{} == Struct(void).pointer(z).x);
		39	assert(void{} == Struct(void).pointer(&z).x);
		40	assert(void{} == Struct(void).maybePointer(z).x);
		41	assert(void{} == Struct(void).maybePointer(&z).x);
		42	assert(void{} == Struct(void).maybePointer(null).x);
		43	const s = Struct(u8) { .x = 42 };
		44	assert(0 != @sizeOf(@typeOf(s)));
		45	assert(42 == Struct(u8).pointer(s).x);
		46	assert(42 == Struct(u8).pointer(&s).x);
		47	assert(42 == Struct(u8).maybePointer(s).x);
		48	assert(42 == Struct(u8).maybePointer(&s).x);
		49	assert(0 == Struct(u8).maybePointer(null).x);
		50	const u = Union { .x = 42 };
		51	assert(42 == Union.pointer(u).x);
		52	assert(42 == Union.pointer(&u).x);
		53	assert(42 == Union.maybePointer(u).x);
		54	assert(42 == Union.maybePointer(&u).x);
		55	assert(0 == Union.maybePointer(null).x);
		56	const e = Enum.Some;
		57	assert(Enum.Some == Enum.pointer(e));
		58	assert(Enum.Some == Enum.pointer(&e));
		59	assert(Enum.Some == Enum.maybePointer(e));
		60	assert(Enum.Some == Enum.maybePointer(&e));
		61	assert(Enum.None == Enum.maybePointer(null));
		62	}
		63
		64	fn Struct(comptime T: type) type {
		65	return struct {
		66	const Self = this;
		67	x: T,
		68
		69	fn pointer(self: &const Self) Self {
		70	return *self;
		71	}
		72
		73	fn maybePointer(self: ?&const Self) Self {
		74	const none = Self { .x = if (T == void) void{} else 0 };
		75	return *(self ?? &none);
		76	}
		77	};
		78	}
		79
		80	const Union = union {
		81	x: u8,
		82
		83	fn pointer(self: &const Union) Union {
		84	return *self;
		85	}
		86
		87	fn maybePointer(self: ?&const Union) Union {
		88	const none = Union { .x = 0 };
		89	return *(self ?? &none);
		90	}
		91	};
		92
		93	const Enum = enum {
		94	None,
		95	Some,
		96
		97	fn pointer(self: &const Enum) Enum {
		98	return *self;
		99	}
		100
		101	fn maybePointer(self: ?&const Enum) Enum {
		102	return *(self ?? &Enum.None);
		103	}
		104	};
		105
		106	test "implicitly cast indirect pointer to maybe-indirect pointer" {
		107	const S = struct {
		108	const Self = this;
		109	x: u8,
		110	fn constConst(p: &const &const Self) u8 {
		111	return (*p).x;
		112	}
		113	fn maybeConstConst(p: ?&const &const Self) u8 {
		114	return (*??p).x;
		115	}
		116	fn constConstConst(p: &const &const &const Self) u8 {
		117	return (**p).x;
		118	}
		119	fn maybeConstConstConst(p: ?&const &const &const Self) u8 {
		120	return (**??p).x;
		121	}
		122	};
		123	const s = S { .x = 42 };
		124	const p = &s;
		125	const q = &p;
		126	const r = &q;
		127	assert(42 == S.constConst(p));
		128	assert(42 == S.constConst(q));
		129	assert(42 == S.maybeConstConst(p));
		130	assert(42 == S.maybeConstConst(q));
		131	assert(42 == S.constConstConst(q));
		132	assert(42 == S.constConstConst(r));
		133	assert(42 == S.maybeConstConstConst(q));
		134	assert(42 == S.maybeConstConstConst(r));
		135	}
		136
35	test "explicit cast from integer to error type" {	137	test "explicit cast from integer to error type" {
36	testCastIntToErr(error.ItBroke);	138	testCastIntToErr(error.ItBroke);
37	comptime testCastIntToErr(error.ItBroke);	139	comptime testCastIntToErr(error.ItBroke);

test/cases/coroutines.zig created+132

...	@@ -0,0 +1,132 @@
1	const std = @import("std");
2	const assert = std.debug.assert;
3
4	var x: i32 = 1;
5
6	test "create a coroutine and cancel it" {
7	const p = try async(std.debug.global_allocator) simpleAsyncFn();
8	cancel p;
9	assert(x == 2);
10	}
11
12	async fn simpleAsyncFn() void {
13	x += 1;
14	suspend;
15	x += 1;
16	}
17
18	test "coroutine suspend, resume, cancel" {
19	seq('a');
20	const p = try async(std.debug.global_allocator) testAsyncSeq();
21	seq('c');
22	resume p;
23	seq('f');
24	cancel p;
25	seq('g');
26
27	assert(std.mem.eql(u8, points, "abcdefg"));
28	}
29
30	async fn testAsyncSeq() void {
31	defer seq('e');
32
33	seq('b');
34	suspend;
35	seq('d');
36	}
37	var points = []u8{0} ** "abcdefg".len;
38	var index: usize = 0;
39
40	fn seq(c: u8) void {
41	points[index] = c;
42	index += 1;
43	}
44
45	test "coroutine suspend with block" {
46	const p = try async(std.debug.global_allocator) testSuspendBlock();
47	std.debug.assert(!result);
48	resume a_promise;
49	std.debug.assert(result);
50	cancel p;
51	}
52
53	var a_promise: promise = undefined;
54	var result = false;
55
56	async fn testSuspendBlock() void {
57	suspend \|p\| {
58	a_promise = p;
59	}
60	result = true;
61	}
62
63	var await_a_promise: promise = undefined;
64	var await_final_result: i32 = 0;
65
66	test "coroutine await" {
67	await_seq('a');
68	const p = async(std.debug.global_allocator) await_amain() catch unreachable;
69	await_seq('f');
70	resume await_a_promise;
71	await_seq('i');
72	assert(await_final_result == 1234);
73	assert(std.mem.eql(u8, await_points, "abcdefghi"));
74	}
75
76	async fn await_amain() void {
77	await_seq('b');
78	const p = async await_another() catch unreachable;
79	await_seq('e');
80	await_final_result = await p;
81	await_seq('h');
82	}
83
84	async fn await_another() i32 {
85	await_seq('c');
86	suspend \|p\| {
87	await_seq('d');
88	await_a_promise = p;
89	}
90	await_seq('g');
91	return 1234;
92	}
93
94	var await_points = []u8{0} ** "abcdefghi".len;
95	var await_seq_index: usize = 0;
96
97	fn await_seq(c: u8) void {
98	await_points[await_seq_index] = c;
99	await_seq_index += 1;
100	}
101
102
103	var early_final_result: i32 = 0;
104
105	test "coroutine await early return" {
106	early_seq('a');
107	const p = async(std.debug.global_allocator) early_amain() catch unreachable;
108	early_seq('f');
109	assert(early_final_result == 1234);
110	assert(std.mem.eql(u8, early_points, "abcdef"));
111	}
112
113	async fn early_amain() void {
114	early_seq('b');
115	const p = async early_another() catch unreachable;
116	early_seq('d');
117	early_final_result = await p;
118	early_seq('e');
119	}
120
121	async fn early_another() i32 {
122	early_seq('c');
123	return 1234;
124	}
125
126	var early_points = []u8{0} ** "abcdef".len;
127	var early_seq_index: usize = 0;
128
129	fn early_seq(c: u8) void {
130	early_points[early_seq_index] = c;
131	early_seq_index += 1;
132	}

test/compile_errors.zig+12

...	@@ -3090,4 +3090,16 @@ pub fn addCases(cases: &tests.CompileErrorContext) void {	...	@@ -3090,4 +3090,16 @@ pub fn addCases(cases: &tests.CompileErrorContext) void {
3090	,	3090	,
3091	".tmp_source.zig:11:20: error: runtime cast to union 'Value' which has non-void fields",	3091	".tmp_source.zig:11:20: error: runtime cast to union 'Value' which has non-void fields",
3092	".tmp_source.zig:3:5: note: field 'A' has type 'i32'");	3092	".tmp_source.zig:3:5: note: field 'A' has type 'i32'");
		3093
		3094	cases.add("self-referencing function pointer field",
		3095	\\const S = struct {
		3096	\\ f: fn(_: S) void,
		3097	\\};
		3098	\\fn f(_: S) void {
		3099	\\}
		3100	\\export fn entry() void {
		3101	\\ var _ = S { .f = f };
		3102	\\}
		3103	,
		3104	".tmp_source.zig:4:9: error: type 'S' is not copyable; cannot pass by value");
3093	}	3105	}

...	@@ -35,6 +35,8 @@ TypeTableEntry get_bound_fn_type(CodeGen g, FnTableEntry *fn_entry);	...	@@ -35,6 +35,8 @@ TypeTableEntry get_bound_fn_type(CodeGen g, FnTableEntry *fn_entry);
35	TypeTableEntry get_opaque_type(CodeGen g, Scope scope, AstNode source_node, const char *name);	35	TypeTableEntry get_opaque_type(CodeGen g, Scope scope, AstNode source_node, const char *name);
36	TypeTableEntry get_struct_type(CodeGen g, const char type_name, const char field_names[],	36	TypeTableEntry get_struct_type(CodeGen g, const char type_name, const char field_names[],
37	TypeTableEntry *field_types[], size_t field_count);	37	TypeTableEntry *field_types[], size_t field_count);
		38	TypeTableEntry get_promise_type(CodeGen g, TypeTableEntry *result_type);
		39	TypeTableEntry get_promise_frame_type(CodeGen g, TypeTableEntry *return_type);
38	TypeTableEntry get_test_fn_type(CodeGen g);	40	TypeTableEntry get_test_fn_type(CodeGen g);
39	bool handle_is_ptr(TypeTableEntry *type_entry);	41	bool handle_is_ptr(TypeTableEntry *type_entry);
40	void find_libc_include_path(CodeGen *g);	42	void find_libc_include_path(CodeGen *g);
...	@@ -50,6 +52,7 @@ VariableTableEntry find_variable(CodeGen g, Scope orig_context, Buf name);	...	@@ -50,6 +52,7 @@ VariableTableEntry find_variable(CodeGen g, Scope orig_context, Buf name);
50	Tld find_decl(CodeGen g, Scope scope, Buf name);	52	Tld find_decl(CodeGen g, Scope scope, Buf name);
51	void resolve_top_level_decl(CodeGen g, Tld tld, bool pointer_only, AstNode *source_node);	53	void resolve_top_level_decl(CodeGen g, Tld tld, bool pointer_only, AstNode *source_node);
52	bool type_is_codegen_pointer(TypeTableEntry *type);	54	bool type_is_codegen_pointer(TypeTableEntry *type);
		55
53	TypeTableEntry get_codegen_ptr_type(TypeTableEntry type);	56	TypeTableEntry get_codegen_ptr_type(TypeTableEntry type);
54	uint32_t get_ptr_align(TypeTableEntry *type);	57	uint32_t get_ptr_align(TypeTableEntry *type);
55	TypeTableEntry validate_var_type(CodeGen g, AstNode source_node, TypeTableEntry type_entry);	58	TypeTableEntry validate_var_type(CodeGen g, AstNode source_node, TypeTableEntry type_entry);
...	@@ -92,7 +95,6 @@ void eval_min_max_value(CodeGen g, TypeTableEntry type_entry, ConstExprValue *	...	@@ -92,7 +95,6 @@ void eval_min_max_value(CodeGen g, TypeTableEntry type_entry, ConstExprValue *
92	void eval_min_max_value_int(CodeGen g, TypeTableEntry int_type, BigInt *bigint, bool is_max);	95	void eval_min_max_value_int(CodeGen g, TypeTableEntry int_type, BigInt *bigint, bool is_max);
93		96
94	void render_const_value(CodeGen g, Buf buf, ConstExprValue *const_val);	97	void render_const_value(CodeGen g, Buf buf, ConstExprValue *const_val);
95	void define_local_param_variables(CodeGen g, FnTableEntry fn_table_entry, VariableTableEntry **arg_vars);
96	void analyze_fn_ir(CodeGen g, FnTableEntry fn_table_entry, AstNode *return_type_node);	98	void analyze_fn_ir(CodeGen g, FnTableEntry fn_table_entry, AstNode *return_type_node);
97		99
98	ScopeBlock create_block_scope(AstNode node, Scope *parent);	100	ScopeBlock create_block_scope(AstNode node, Scope *parent);
...	@@ -190,4 +192,7 @@ void analyze_fn_body(CodeGen g, FnTableEntry fn_table_entry);	...	@@ -190,4 +192,7 @@ void analyze_fn_body(CodeGen g, FnTableEntry fn_table_entry);
190		192
191	TypeTableEntry get_auto_err_set_type(CodeGen g, FnTableEntry *fn_entry);	193	TypeTableEntry get_auto_err_set_type(CodeGen g, FnTableEntry *fn_entry);
192		194
		195	uint32_t get_coro_frame_align_bytes(CodeGen *g);
		196	bool fn_type_can_fail(FnTypeId *fn_type_id);
		197
193	#endif	198	#endif

...	@@ -32,6 +32,7 @@	...	@@ -32,6 +32,7 @@
32	#include <llvm/Support/TargetParser.h>	32	#include <llvm/Support/TargetParser.h>
33	#include <llvm/Support/raw_ostream.h>	33	#include <llvm/Support/raw_ostream.h>
34	#include <llvm/Target/TargetMachine.h>	34	#include <llvm/Target/TargetMachine.h>
		35	#include <llvm/Transforms/Coroutines.h>
35	#include <llvm/Transforms/IPO.h>	36	#include <llvm/Transforms/IPO.h>
36	#include <llvm/Transforms/IPO/PassManagerBuilder.h>	37	#include <llvm/Transforms/IPO/PassManagerBuilder.h>
37	#include <llvm/Transforms/IPO/AlwaysInliner.h>	38	#include <llvm/Transforms/IPO/AlwaysInliner.h>
...	@@ -129,6 +130,8 @@ bool ZigLLVMTargetMachineEmitToFile(LLVMTargetMachineRef targ_machine_ref, LLVMM	...	@@ -129,6 +130,8 @@ bool ZigLLVMTargetMachineEmitToFile(LLVMTargetMachineRef targ_machine_ref, LLVMM
129	PMBuilder->Inliner = createFunctionInliningPass(PMBuilder->OptLevel, PMBuilder->SizeLevel, false);	130	PMBuilder->Inliner = createFunctionInliningPass(PMBuilder->OptLevel, PMBuilder->SizeLevel, false);
130	}	131	}
131		132
		133	addCoroutinePassesToExtensionPoints(*PMBuilder);
		134
132	// Set up the per-function pass manager.	135	// Set up the per-function pass manager.
133	legacy::FunctionPassManager FPM = legacy::FunctionPassManager(module);	136	legacy::FunctionPassManager FPM = legacy::FunctionPassManager(module);
134	auto tliwp = new(std::nothrow) TargetLibraryInfoWrapperPass(tlii);	137	auto tliwp = new(std::nothrow) TargetLibraryInfoWrapperPass(tlii);
...	@@ -182,6 +185,9 @@ bool ZigLLVMTargetMachineEmitToFile(LLVMTargetMachineRef targ_machine_ref, LLVMM	...	@@ -182,6 +185,9 @@ bool ZigLLVMTargetMachineEmitToFile(LLVMTargetMachineRef targ_machine_ref, LLVMM
182	return false;	185	return false;
183	}	186	}
184		187
		188	ZIG_EXTERN_C LLVMTypeRef ZigLLVMTokenTypeInContext(LLVMContextRef context_ref) {
		189	return wrap(Type::getTokenTy(*unwrap(context_ref)));
		190	}
185		191
186	LLVMValueRef ZigLLVMBuildCall(LLVMBuilderRef B, LLVMValueRef Fn, LLVMValueRef *Args,	192	LLVMValueRef ZigLLVMBuildCall(LLVMBuilderRef B, LLVMValueRef Fn, LLVMValueRef *Args,
187	unsigned NumArgs, unsigned CC, ZigLLVM_FnInline fn_inline, const char *Name)	193	unsigned NumArgs, unsigned CC, ZigLLVM_FnInline fn_inline, const char *Name)