Commit 4dca99d3f6b732c415d270f0c97def144ed6d3b7 | zig

author	Andrew Kelley <andrew@ziglang.org> 2021-01-30 20:16:59-07:00
committer	Andrew Kelley <andrew@ziglang.org> 2021-01-30 20:16:59-07:00
log	4dca99d3f6b732c415d270f0c97def144ed6d3b7
tree	e1ebee7fd7ce3718efce59b73cce802496021281
parent	766b315b3888f0f9ac1ece69131cdf23f98b2c14

stage2: rework AST memory layout

This is a proof-of-concept of switching to a new memory layout for
tokens and AST nodes. The goal is threefold:

 * smaller memory footprint
 * faster performance for tokenization and parsing
 * most importantly, a proof-of-concept that can be also applied to ZIR
   and TZIR to improve the entire compiler pipeline in this way.

I had a few key insights here:

 * Underlying premise: using less memory will make things faster, because
   of fewer allocations and better cache utilization. Also using less
   memory is valuable in and of itself.
 * Using a Struct-Of-Arrays for tokens and AST nodes, saves the bytes of
   padding between the enum tag (which kind of token is it; which kind
   of AST node is it) and the next fields in the struct. It also improves
   cache coherence, since one can peek ahead in the tokens array without
   having to load the source locations of tokens.
 * Token memory can be conserved by only having the tag (1 byte) and byte
   offset (4 bytes) for a total of 5 bytes per token. It is not necessary
   to store the token ending byte offset because one can always re-tokenize
   later, but also most tokens the length can be trivially determined from
   the tag alone, and for ones where it doesn't, string literals for
   example, one must parse the string literal again later anyway in
   astgen, making it free to re-tokenize.
 * AST nodes do not actually need to store more than 1 token index because
   one can poke left and right in the tokens array very cheaply.

So far we are left with one big problem though: how can we put AST nodes
into an array, since different AST nodes are different sizes?

This is where my key observation comes in: one can have a hash table for
the extra data for the less common AST nodes! But it gets even better than
that:

I defined this data that is always present for every AST Node:

 * tag (1 byte)
   - which AST node is it
 * main_token (4 bytes, index into tokens array)
   - the tag determines which token this points to
 * struct{lhs: u32, rhs: u32}
   - enough to store 2 indexes to other AST nodes, the tag determines
     how to interpret this data

You can see how a binary operation, such as `a * b` would fit into this
structure perfectly. A unary operation, such as `*a` would also fit,
and leave `rhs` unused. So this is a total of 13 bytes per AST node.
And again, we don't have to pay for the padding to round up to 16 because
we store in struct-of-arrays format.

I made a further observation: the only kind of data AST nodes need to
store other than the main_token is indexes to sub-expressions. That's it.
The only purpose of an AST is to bring a tree structure to a list of tokens.
This observation means all the data that nodes store are only sets of u32
indexes to other nodes. The other tokens can be found later by the compiler,
by poking around in the tokens array, which again is super fast because it
is struct-of-arrays, so you often only need to look at the token tags array,
which is an array of bytes, very cache friendly.

So for nearly every kind of AST node, you can store it in 13 bytes. For the
rarer AST nodes that have 3 or more indexes to other nodes to store, either
the lhs or the rhs will be repurposed to be an index into an extra_data array
which contains the extra AST node indexes. In other words, no hash table needed,
it's just 1 big ArrayList with the extra data for AST Nodes.

Final observation, no need to have a canonical tag for a given AST. For example:
The expression `foo(bar)` is a function call. Function calls can have any
number of parameters. However in this example, we can encode the function
call into the AST with a tag called `FunctionCallOnlyOneParam`, and use lhs
for the function expr and rhs for the only parameter expr. Meanwhile if the
code was `foo(bar, baz)` then the AST node would have to be `FunctionCall`
with lhs still being the function expr, but rhs being the index into
`extra_data`. Then because the tag is `FunctionCall` it means
`extra_data[rhs]` is the "start" and `extra_data[rhs+1]` is the "end".
Now the range `extra_data[start..end]` describes the list of parameters
to the function.

Point being, you only have to pay for the extra bytes if the AST actually
requires it. There's no limit to the number of different AST tag encodings.

Preliminary results:

 * 15% improvement on cache-misses
 * 28% improvement on total instructions executed
 * 26% improvement on total CPU cycles
 * 22% improvement on wall clock time

This is 1/4 items on the checklist before this can actually be merged:

 * [x] parser
 * [ ] render (zig fmt)
 * [ ] astgen
 * [ ] translate-c

4 files changed, 3873 insertions(+), 6164 deletions(-)

lib
- std
  - zig
    ast.zig
    parse.zig
    parser_test.zig
    tokenizer.zig

lib/std/zig/ast.zig+518-2960

...	...	@@ -9,48 +9,30 @@ const testing = std.testing;
9	9	const mem = std.mem;
10	10	const Token = std.zig.Token;
11	11
12		pub const TokenIndex = usize;
13		pub const NodeIndex = usize;
	12	pub const TokenIndex = u32;
	13	pub const ByteOffset = u32;
	14
	15	pub const TokenList = std.MultiArrayList(struct {
	16	tag: Token.Tag,
	17	start: ByteOffset,
	18	});
	19	pub const NodeList = std.MultiArrayList(struct {
	20	tag: Node.Tag,
	21	main_token: TokenIndex,
	22	data: Node.Data,
	23	});
14	24
15	25	pub const Tree = struct {
16	26	/// Reference to externally-owned data.
17	27	source: []const u8,
18		token_ids: []const Token.Id,
19		token_locs: []const Token.Loc,
20		errors: []const Error,
21		root_node: *Node.Root,
22
23		arena: std.heap.ArenaAllocator.State,
24		gpa: *mem.Allocator,
25
26		/// translate-c uses this to avoid having to emit correct newlines
27		/// TODO get rid of this hack
28		generated: bool = false,
29
30		pub fn deinit(self: *Tree) void {
31		self.gpa.free(self.token_ids);
32		self.gpa.free(self.token_locs);
33		self.gpa.free(self.errors);
34		self.arena.promote(self.gpa).deinit();
35		}
36
37		pub fn renderError(self: Tree, parse_error: const Error, stream: anytype) !void {
38		return parse_error.render(self.token_ids, stream);
39		}
40
41		pub fn tokenSlice(self: *Tree, token_index: TokenIndex) []const u8 {
42		return self.tokenSliceLoc(self.token_locs[token_index]);
43		}
44	28
45		pub fn tokenSliceLoc(self: *Tree, token: Token.Loc) []const u8 {
46		return self.source[token.start..token.end];
47		}
	29	tokens: TokenList.Slice,
	30	/// The root AST node is assumed to be index 0. Since there can be no
	31	/// references to the root node, this means 0 is available to indicate null.
	32	nodes: NodeList.Slice,
	33	extra_data: []Node.Index,
48	34
49		pub fn getNodeSource(self: const Tree, node: const Node) []const u8 {
50		const first_token = self.token_locs[node.firstToken()];
51		const last_token = self.token_locs[node.lastToken()];
52		return self.source[first_token.start..last_token.end];
53		}
	35	errors: []const Error,
54	36
55	37	pub const Location = struct {
56	38	line: usize,
...	...	@@ -59,21 +41,28 @@ pub const Tree = struct {
59	41	line_end: usize,
60	42	};
61	43
62		/// Return the Location of the token relative to the offset specified by `start_index`.
63		pub fn tokenLocationLoc(self: *Tree, start_index: usize, token: Token.Loc) Location {
	44	pub fn deinit(tree: Tree, gpa: mem.Allocator) void {
	45	tree.tokens.deinit(gpa);
	46	tree.nodes.deinit(gpa);
	47	gpa.free(tree.extra_data);
	48	gpa.free(tree.errors);
	49	tree.* = undefined;
	50	}
	51
	52	pub fn tokenLocation(self: Tree, start_offset: ByteOffset, token_index: TokenIndex) Location {
64	53	var loc = Location{
65	54	.line = 0,
66	55	.column = 0,
67		.line_start = start_index,
	56	.line_start = start_offset,
68	57	.line_end = self.source.len,
69	58	};
70		if (self.generated)
71		return loc;
72		const token_start = token.start;
73		for (self.source[start_index..]) \|c, i\| {
74		if (i + start_index == token_start) {
75		loc.line_end = i + start_index;
76		while (loc.line_end < self.source.len and self.source[loc.line_end] != '\n') : (loc.line_end += 1) {}
	59	const token_start = self.tokens.items(.start)[token_index];
	60	for (self.source[start_offset..]) \|c, i\| {
	61	if (i + start_offset == token_start) {
	62	loc.line_end = i + start_offset;
	63	while (loc.line_end < self.source.len and self.source[loc.line_end] != '\n') {
	64	loc.line_end += 1;
	65	}
77	66	return loc;
78	67	}
79	68	if (c == '\n') {
...	...	@@ -87,94 +76,9 @@ pub const Tree = struct {
87	76	return loc;
88	77	}
89	78
90		pub fn tokenLocation(self: *Tree, start_index: usize, token_index: TokenIndex) Location {
91		return self.tokenLocationLoc(start_index, self.token_locs[token_index]);
92		}
93
94		pub fn tokensOnSameLine(self: *Tree, token1_index: TokenIndex, token2_index: TokenIndex) bool {
95		return self.tokensOnSameLineLoc(self.token_locs[token1_index], self.token_locs[token2_index]);
96		}
97
98		pub fn tokensOnSameLineLoc(self: *Tree, token1: Token.Loc, token2: Token.Loc) bool {
99		return mem.indexOfScalar(u8, self.source[token1.end..token2.start], '\n') == null;
100		}
101
102		pub fn dump(self: *Tree) void {
103		self.root_node.base.dump(0);
104		}
105
106		/// Skips over comments
107		pub fn prevToken(self: *Tree, token_index: TokenIndex) TokenIndex {
108		var index = token_index - 1;
109		while (self.token_ids[index] == Token.Id.LineComment) {
110		index -= 1;
111		}
112		return index;
113		}
114
115		/// Skips over comments
116		pub fn nextToken(self: *Tree, token_index: TokenIndex) TokenIndex {
117		var index = token_index + 1;
118		while (self.token_ids[index] == Token.Id.LineComment) {
119		index += 1;
120		}
121		return index;
122		}
123		};
124
125		pub const Error = union(enum) {
126		InvalidToken: InvalidToken,
127		ExpectedContainerMembers: ExpectedContainerMembers,
128		ExpectedStringLiteral: ExpectedStringLiteral,
129		ExpectedIntegerLiteral: ExpectedIntegerLiteral,
130		ExpectedPubItem: ExpectedPubItem,
131		ExpectedIdentifier: ExpectedIdentifier,
132		ExpectedStatement: ExpectedStatement,
133		ExpectedVarDeclOrFn: ExpectedVarDeclOrFn,
134		ExpectedVarDecl: ExpectedVarDecl,
135		ExpectedFn: ExpectedFn,
136		ExpectedReturnType: ExpectedReturnType,
137		ExpectedAggregateKw: ExpectedAggregateKw,
138		UnattachedDocComment: UnattachedDocComment,
139		ExpectedEqOrSemi: ExpectedEqOrSemi,
140		ExpectedSemiOrLBrace: ExpectedSemiOrLBrace,
141		ExpectedSemiOrElse: ExpectedSemiOrElse,
142		ExpectedLabelOrLBrace: ExpectedLabelOrLBrace,
143		ExpectedLBrace: ExpectedLBrace,
144		ExpectedColonOrRParen: ExpectedColonOrRParen,
145		ExpectedLabelable: ExpectedLabelable,
146		ExpectedInlinable: ExpectedInlinable,
147		ExpectedAsmOutputReturnOrType: ExpectedAsmOutputReturnOrType,
148		ExpectedCall: ExpectedCall,
149		ExpectedCallOrFnProto: ExpectedCallOrFnProto,
150		ExpectedSliceOrRBracket: ExpectedSliceOrRBracket,
151		ExtraAlignQualifier: ExtraAlignQualifier,
152		ExtraConstQualifier: ExtraConstQualifier,
153		ExtraVolatileQualifier: ExtraVolatileQualifier,
154		ExtraAllowZeroQualifier: ExtraAllowZeroQualifier,
155		ExpectedTypeExpr: ExpectedTypeExpr,
156		ExpectedPrimaryTypeExpr: ExpectedPrimaryTypeExpr,
157		ExpectedParamType: ExpectedParamType,
158		ExpectedExpr: ExpectedExpr,
159		ExpectedPrimaryExpr: ExpectedPrimaryExpr,
160		ExpectedToken: ExpectedToken,
161		ExpectedCommaOrEnd: ExpectedCommaOrEnd,
162		ExpectedParamList: ExpectedParamList,
163		ExpectedPayload: ExpectedPayload,
164		ExpectedBlockOrAssignment: ExpectedBlockOrAssignment,
165		ExpectedBlockOrExpression: ExpectedBlockOrExpression,
166		ExpectedExprOrAssignment: ExpectedExprOrAssignment,
167		ExpectedPrefixExpr: ExpectedPrefixExpr,
168		ExpectedLoopExpr: ExpectedLoopExpr,
169		ExpectedDerefOrUnwrap: ExpectedDerefOrUnwrap,
170		ExpectedSuffixOp: ExpectedSuffixOp,
171		ExpectedBlockOrField: ExpectedBlockOrField,
172		DeclBetweenFields: DeclBetweenFields,
173		InvalidAnd: InvalidAnd,
174		AsteriskAfterPointerDereference: AsteriskAfterPointerDereference,
175
176		pub fn render(self: *const Error, tokens: []const Token.Id, stream: anytype) !void {
177		switch (self.*) {
	79	pub fn renderError(tree: Tree, parse_error: Error, stream: anytype) !void {
	80	const tokens = tree.tokens.items(.tag);
	81	switch (parse_error) {
178	82	.InvalidToken => \|*x\| return x.render(tokens, stream),
179	83	.ExpectedContainerMembers => \|*x\| return x.render(tokens, stream),
180	84	.ExpectedStringLiteral => \|*x\| return x.render(tokens, stream),
...	...	@@ -197,8 +101,8 @@ pub const Error = union(enum) {
197	101	.ExpectedLabelable => \|*x\| return x.render(tokens, stream),
198	102	.ExpectedInlinable => \|*x\| return x.render(tokens, stream),
199	103	.ExpectedAsmOutputReturnOrType => \|*x\| return x.render(tokens, stream),
200		.ExpectedCall => \|*x\| return x.render(tokens, stream),
201		.ExpectedCallOrFnProto => \|*x\| return x.render(tokens, stream),
	104	.ExpectedCall => \|x\| return x.render(tree, stream),
	105	.ExpectedCallOrFnProto => \|x\| return x.render(tree, stream),
202	106	.ExpectedSliceOrRBracket => \|*x\| return x.render(tokens, stream),
203	107	.ExtraAlignQualifier => \|*x\| return x.render(tokens, stream),
204	108	.ExtraConstQualifier => \|*x\| return x.render(tokens, stream),
...	...	@@ -227,8 +131,8 @@ pub const Error = union(enum) {
227	131	}
228	132	}
229	133
230		pub fn loc(self: *const Error) TokenIndex {
231		switch (self.*) {
	134	pub fn errorToken(tree: Tree, parse_error: Error) TokenIndex {
	135	switch (parse_error) {
232	136	.InvalidToken => \|x\| return x.token,
233	137	.ExpectedContainerMembers => \|x\| return x.token,
234	138	.ExpectedStringLiteral => \|x\| return x.token,
...	...	@@ -251,8 +155,8 @@ pub const Error = union(enum) {
251	155	.ExpectedLabelable => \|x\| return x.token,
252	156	.ExpectedInlinable => \|x\| return x.token,
253	157	.ExpectedAsmOutputReturnOrType => \|x\| return x.token,
254		.ExpectedCall => \|x\| return x.node.firstToken(),
255		.ExpectedCallOrFnProto => \|x\| return x.node.firstToken(),
	158	.ExpectedCall => \|x\| return tree.nodes.items(.main_token)[x.node],
	159	.ExpectedCallOrFnProto => \|x\| return tree.nodes.items(.main_token)[x.node],
256	160	.ExpectedSliceOrRBracket => \|x\| return x.token,
257	161	.ExtraAlignQualifier => \|x\| return x.token,
258	162	.ExtraConstQualifier => \|x\| return x.token,
...	...	@@ -281,6 +185,78 @@ pub const Error = union(enum) {
281	185	}
282	186	}
283	187
	188	/// Skips over comments.
	189	pub fn prevToken(self: *const Tree, token_index: TokenIndex) TokenIndex {
	190	const token_tags = self.tokens.items(.tag);
	191	var index = token_index - 1;
	192	while (token_tags[index] == .LineComment) {
	193	index -= 1;
	194	}
	195	return index;
	196	}
	197
	198	/// Skips over comments.
	199	pub fn nextToken(self: *const Tree, token_index: TokenIndex) TokenIndex {
	200	const token_tags = self.tokens.items(.tag);
	201	var index = token_index + 1;
	202	while (token_tags[index] == .LineComment) {
	203	index += 1;
	204	}
	205	return index;
	206	}
	207	};
	208
	209	pub const Error = union(enum) {
	210	InvalidToken: InvalidToken,
	211	ExpectedContainerMembers: ExpectedContainerMembers,
	212	ExpectedStringLiteral: ExpectedStringLiteral,
	213	ExpectedIntegerLiteral: ExpectedIntegerLiteral,
	214	ExpectedPubItem: ExpectedPubItem,
	215	ExpectedIdentifier: ExpectedIdentifier,
	216	ExpectedStatement: ExpectedStatement,
	217	ExpectedVarDeclOrFn: ExpectedVarDeclOrFn,
	218	ExpectedVarDecl: ExpectedVarDecl,
	219	ExpectedFn: ExpectedFn,
	220	ExpectedReturnType: ExpectedReturnType,
	221	ExpectedAggregateKw: ExpectedAggregateKw,
	222	UnattachedDocComment: UnattachedDocComment,
	223	ExpectedEqOrSemi: ExpectedEqOrSemi,
	224	ExpectedSemiOrLBrace: ExpectedSemiOrLBrace,
	225	ExpectedSemiOrElse: ExpectedSemiOrElse,
	226	ExpectedLabelOrLBrace: ExpectedLabelOrLBrace,
	227	ExpectedLBrace: ExpectedLBrace,
	228	ExpectedColonOrRParen: ExpectedColonOrRParen,
	229	ExpectedLabelable: ExpectedLabelable,
	230	ExpectedInlinable: ExpectedInlinable,
	231	ExpectedAsmOutputReturnOrType: ExpectedAsmOutputReturnOrType,
	232	ExpectedCall: ExpectedCall,
	233	ExpectedCallOrFnProto: ExpectedCallOrFnProto,
	234	ExpectedSliceOrRBracket: ExpectedSliceOrRBracket,
	235	ExtraAlignQualifier: ExtraAlignQualifier,
	236	ExtraConstQualifier: ExtraConstQualifier,
	237	ExtraVolatileQualifier: ExtraVolatileQualifier,
	238	ExtraAllowZeroQualifier: ExtraAllowZeroQualifier,
	239	ExpectedTypeExpr: ExpectedTypeExpr,
	240	ExpectedPrimaryTypeExpr: ExpectedPrimaryTypeExpr,
	241	ExpectedParamType: ExpectedParamType,
	242	ExpectedExpr: ExpectedExpr,
	243	ExpectedPrimaryExpr: ExpectedPrimaryExpr,
	244	ExpectedToken: ExpectedToken,
	245	ExpectedCommaOrEnd: ExpectedCommaOrEnd,
	246	ExpectedParamList: ExpectedParamList,
	247	ExpectedPayload: ExpectedPayload,
	248	ExpectedBlockOrAssignment: ExpectedBlockOrAssignment,
	249	ExpectedBlockOrExpression: ExpectedBlockOrExpression,
	250	ExpectedExprOrAssignment: ExpectedExprOrAssignment,
	251	ExpectedPrefixExpr: ExpectedPrefixExpr,
	252	ExpectedLoopExpr: ExpectedLoopExpr,
	253	ExpectedDerefOrUnwrap: ExpectedDerefOrUnwrap,
	254	ExpectedSuffixOp: ExpectedSuffixOp,
	255	ExpectedBlockOrField: ExpectedBlockOrField,
	256	DeclBetweenFields: DeclBetweenFields,
	257	InvalidAnd: InvalidAnd,
	258	AsteriskAfterPointerDereference: AsteriskAfterPointerDereference,
	259
284	260	pub const InvalidToken = SingleTokenError("Invalid token '{s}'");
285	261	pub const ExpectedContainerMembers = SingleTokenError("Expected test, comptime, var decl, or container field, found '{s}'");
286	262	pub const ExpectedStringLiteral = SingleTokenError("Expected string literal, found '{s}'");
...	...	@@ -291,7 +267,7 @@ pub const Error = union(enum) {
291	267	pub const ExpectedVarDecl = SingleTokenError("Expected variable declaration, found '{s}'");
292	268	pub const ExpectedFn = SingleTokenError("Expected function, found '{s}'");
293	269	pub const ExpectedReturnType = SingleTokenError("Expected 'var' or return type expression, found '{s}'");
294		pub const ExpectedAggregateKw = SingleTokenError("Expected '" ++ Token.Id.Keyword_struct.symbol() ++ "', '" ++ Token.Id.Keyword_union.symbol() ++ "', '" ++ Token.Id.Keyword_enum.symbol() ++ "', or '" ++ Token.Id.Keyword_opaque.symbol() ++ "', found '{s}'");
	270	pub const ExpectedAggregateKw = SingleTokenError("Expected '" ++ Token.Tag.Keyword_struct.symbol() ++ "', '" ++ Token.Tag.Keyword_union.symbol() ++ "', '" ++ Token.Tag.Keyword_enum.symbol() ++ "', or '" ++ Token.Tag.Keyword_opaque.symbol() ++ "', found '{s}'");
295	271	pub const ExpectedEqOrSemi = SingleTokenError("Expected '=' or ';', found '{s}'");
296	272	pub const ExpectedSemiOrLBrace = SingleTokenError("Expected ';' or '{{', found '{s}'");
297	273	pub const ExpectedSemiOrElse = SingleTokenError("Expected ';' or 'else', found '{s}'");
...	...	@@ -300,7 +276,7 @@ pub const Error = union(enum) {
300	276	pub const ExpectedColonOrRParen = SingleTokenError("Expected ':' or ')', found '{s}'");
301	277	pub const ExpectedLabelable = SingleTokenError("Expected 'while', 'for', 'inline', 'suspend', or '{{', found '{s}'");
302	278	pub const ExpectedInlinable = SingleTokenError("Expected 'while' or 'for', found '{s}'");
303		pub const ExpectedAsmOutputReturnOrType = SingleTokenError("Expected '->' or '" ++ Token.Id.Identifier.symbol() ++ "', found '{s}'");
	279	pub const ExpectedAsmOutputReturnOrType = SingleTokenError("Expected '->' or '" ++ Token.Tag.Identifier.symbol() ++ "', found '{s}'");
304	280	pub const ExpectedSliceOrRBracket = SingleTokenError("Expected ']' or '..', found '{s}'");
305	281	pub const ExpectedTypeExpr = SingleTokenError("Expected type expression, found '{s}'");
306	282	pub const ExpectedPrimaryTypeExpr = SingleTokenError("Expected primary type expression, found '{s}'");
...	...	@@ -329,29 +305,31 @@ pub const Error = union(enum) {
329	305	pub const AsteriskAfterPointerDereference = SimpleError("`.` can't be followed by ``. Are you missing a space?");
330	306
331	307	pub const ExpectedCall = struct {
332		node: *Node,
	308	node: Node.Index,
333	309
334		pub fn render(self: *const ExpectedCall, tokens: []const Token.Id, stream: anytype) !void {
	310	pub fn render(self: ExpectedCall, tree: Tree, stream: anytype) !void {
	311	const node_tag = tree.nodes.items(.tag)[self.node];
335	312	return stream.print("expected " ++ @tagName(Node.Tag.Call) ++ ", found {s}", .{
336		@tagName(self.node.tag),
	313	@tagName(node_tag),
337	314	});
338	315	}
339	316	};
340	317
341	318	pub const ExpectedCallOrFnProto = struct {
342		node: *Node,
	319	node: Node.Index,
343	320
344		pub fn render(self: *const ExpectedCallOrFnProto, tokens: []const Token.Id, stream: anytype) !void {
	321	pub fn render(self: ExpectedCallOrFnProto, tree: Tree, stream: anytype) !void {
	322	const node_tag = tree.nodes.items(.tag)[self.node];
345	323	return stream.print("expected " ++ @tagName(Node.Tag.Call) ++ " or " ++
346		@tagName(Node.Tag.FnProto) ++ ", found {s}", .{@tagName(self.node.tag)});
	324	@tagName(Node.Tag.FnProto) ++ ", found {s}", .{@tagName(node_tag)});
347	325	}
348	326	};
349	327
350	328	pub const ExpectedToken = struct {
351	329	token: TokenIndex,
352		expected_id: Token.Id,
	330	expected_id: Token.Tag,
353	331
354		pub fn render(self: *const ExpectedToken, tokens: []const Token.Id, stream: anytype) !void {
	332	pub fn render(self: *const ExpectedToken, tokens: []const Token.Tag, stream: anytype) !void {
355	333	const found_token = tokens[self.token];
356	334	switch (found_token) {
357	335	.Invalid => {
...	...	@@ -367,9 +345,9 @@ pub const Error = union(enum) {
367	345
368	346	pub const ExpectedCommaOrEnd = struct {
369	347	token: TokenIndex,
370		end_id: Token.Id,
	348	end_id: Token.Tag,
371	349
372		pub fn render(self: *const ExpectedCommaOrEnd, tokens: []const Token.Id, stream: anytype) !void {
	350	pub fn render(self: *const ExpectedCommaOrEnd, tokens: []const Token.Tag, stream: anytype) !void {
373	351	const actual_token = tokens[self.token];
374	352	return stream.print("expected ',' or '{s}', found '{s}'", .{
375	353	self.end_id.symbol(),
...	...	@@ -384,7 +362,7 @@ pub const Error = union(enum) {
384	362
385	363	token: TokenIndex,
386	364
387		pub fn render(self: *const ThisError, tokens: []const Token.Id, stream: anytype) !void {
	365	pub fn render(self: *const ThisError, tokens: []const Token.Tag, stream: anytype) !void {
388	366	const actual_token = tokens[self.token];
389	367	return stream.print(msg, .{actual_token.symbol()});
390	368	}
...	...	@@ -397,2886 +375,466 @@ pub const Error = union(enum) {
397	375
398	376	token: TokenIndex,
399	377
400		pub fn render(self: *const ThisError, tokens: []const Token.Id, stream: anytype) !void {
	378	pub fn render(self: *const ThisError, tokens: []const Token.Tag, stream: anytype) !void {
401	379	return stream.writeAll(msg);
402	380	}
403	381	};
404	382	}
	383
	384	pub fn loc(self: Error) TokenIndex {
	385	switch (self) {
	386	.InvalidToken => \|x\| return x.token,
	387	.ExpectedContainerMembers => \|x\| return x.token,
	388	.ExpectedStringLiteral => \|x\| return x.token,
	389	.ExpectedIntegerLiteral => \|x\| return x.token,
	390	.ExpectedPubItem => \|x\| return x.token,
	391	.ExpectedIdentifier => \|x\| return x.token,
	392	.ExpectedStatement => \|x\| return x.token,
	393	.ExpectedVarDeclOrFn => \|x\| return x.token,
	394	.ExpectedVarDecl => \|x\| return x.token,
	395	.ExpectedFn => \|x\| return x.token,
	396	.ExpectedReturnType => \|x\| return x.token,
	397	.ExpectedAggregateKw => \|x\| return x.token,
	398	.UnattachedDocComment => \|x\| return x.token,
	399	.ExpectedEqOrSemi => \|x\| return x.token,
	400	.ExpectedSemiOrLBrace => \|x\| return x.token,
	401	.ExpectedSemiOrElse => \|x\| return x.token,
	402	.ExpectedLabelOrLBrace => \|x\| return x.token,
	403	.ExpectedLBrace => \|x\| return x.token,
	404	.ExpectedColonOrRParen => \|x\| return x.token,
	405	.ExpectedLabelable => \|x\| return x.token,
	406	.ExpectedInlinable => \|x\| return x.token,
	407	.ExpectedAsmOutputReturnOrType => \|x\| return x.token,
	408	.ExpectedCall => \|x\| @panic("TODO redo ast errors"),
	409	.ExpectedCallOrFnProto => \|x\| @panic("TODO redo ast errors"),
	410	.ExpectedSliceOrRBracket => \|x\| return x.token,
	411	.ExtraAlignQualifier => \|x\| return x.token,
	412	.ExtraConstQualifier => \|x\| return x.token,
	413	.ExtraVolatileQualifier => \|x\| return x.token,
	414	.ExtraAllowZeroQualifier => \|x\| return x.token,
	415	.ExpectedTypeExpr => \|x\| return x.token,
	416	.ExpectedPrimaryTypeExpr => \|x\| return x.token,
	417	.ExpectedParamType => \|x\| return x.token,
	418	.ExpectedExpr => \|x\| return x.token,
	419	.ExpectedPrimaryExpr => \|x\| return x.token,
	420	.ExpectedToken => \|x\| return x.token,
	421	.ExpectedCommaOrEnd => \|x\| return x.token,
	422	.ExpectedParamList => \|x\| return x.token,
	423	.ExpectedPayload => \|x\| return x.token,
	424	.ExpectedBlockOrAssignment => \|x\| return x.token,
	425	.ExpectedBlockOrExpression => \|x\| return x.token,
	426	.ExpectedExprOrAssignment => \|x\| return x.token,
	427	.ExpectedPrefixExpr => \|x\| return x.token,
	428	.ExpectedLoopExpr => \|x\| return x.token,
	429	.ExpectedDerefOrUnwrap => \|x\| return x.token,
	430	.ExpectedSuffixOp => \|x\| return x.token,
	431	.ExpectedBlockOrField => \|x\| return x.token,
	432	.DeclBetweenFields => \|x\| return x.token,
	433	.InvalidAnd => \|x\| return x.token,
	434	.AsteriskAfterPointerDereference => \|x\| return x.token,
	435	}
	436	}
405	437	};
406	438
407	439	pub const Node = struct {
408		tag: Tag,
	440	index: Index,
	441
	442	pub const Index = u32;
	443
	444	comptime {
	445	// Goal is to keep this under one byte for efficiency.
	446	assert(@sizeOf(Tag) == 1);
	447	}
409	448
410	449	pub const Tag = enum {
411		// Top level
	450	/// sub_list[lhs...rhs]
412	451	Root,
413		Use,
	452	/// lhs is the sub-expression. rhs is unused.
	453	UsingNamespace,
	454	/// lhs is test name token (must be string literal), if any.
	455	/// rhs is the body node.
414	456	TestDecl,
415
416		// Statements
417		VarDecl,
	457	/// lhs is the index into global_var_decl_list.
	458	/// rhs is the initialization expression, if any.
	459	GlobalVarDecl,
	460	/// `var a: x align(y) = rhs`
	461	/// lhs is the index into local_var_decl_list.
	462	LocalVarDecl,
	463	/// `var a: lhs = rhs`. lhs and rhs may be unused.
	464	/// Can be local or global.
	465	SimpleVarDecl,
	466	/// `var a align(lhs) = rhs`. lhs and rhs may be unused.
	467	/// Can be local or global.
	468	AlignedVarDecl,
	469	/// lhs is the identifier token payload if any,
	470	/// rhs is the deferred expression.
	471	ErrDefer,
	472	/// lhs is unused.
	473	/// rhs is the deferred expression.
418	474	Defer,
419
420		// Infix operators
	475	/// lhs is target expr; rhs is fallback expr.
	476	/// payload is determined by looking at the prev tokens before rhs.
421	477	Catch,
422
423		// SimpleInfixOp
424		Add,
425		AddWrap,
426		ArrayCat,
427		ArrayMult,
428		Assign,
429		AssignBitAnd,
430		AssignBitOr,
431		AssignBitShiftLeft,
432		AssignBitShiftRight,
433		AssignBitXor,
	478	/// `lhs.a`. main_token is the dot. rhs is the identifier token index.
	479	FieldAccess,
	480	/// `lhs.?`. main_token is the dot. rhs is the `?` token index.
	481	UnwrapOptional,
	482	/// `lhs == rhs`. main_token is op.
	483	EqualEqual,
	484	/// `lhs != rhs`. main_token is op.
	485	BangEqual,
	486	/// `lhs < rhs`. main_token is op.
	487	LessThan,
	488	/// `lhs > rhs`. main_token is op.
	489	GreaterThan,
	490	/// `lhs <= rhs`. main_token is op.
	491	LessOrEqual,
	492	/// `lhs >= rhs`. main_token is op.
	493	GreaterOrEqual,
	494	/// `lhs *= rhs`. main_token is op.
	495	AssignMul,
	496	/// `lhs /= rhs`. main_token is op.
434	497	AssignDiv,
435		AssignSub,
436		AssignSubWrap,
	498	/// `lhs *= rhs`. main_token is op.
437	499	AssignMod,
	500	/// `lhs += rhs`. main_token is op.
438	501	AssignAdd,
439		AssignAddWrap,
440		AssignMul,
	502	/// `lhs -= rhs`. main_token is op.
	503	AssignSub,
	504	/// `lhs <<= rhs`. main_token is op.
	505	AssignBitShiftLeft,
	506	/// `lhs >>= rhs`. main_token is op.
	507	AssignBitShiftRight,
	508	/// `lhs &= rhs`. main_token is op.
	509	AssignBitAnd,
	510	/// `lhs ^= rhs`. main_token is op.
	511	AssignBitXor,
	512	/// `lhs \|= rhs`. main_token is op.
	513	AssignBitOr,
	514	/// `lhs *%= rhs`. main_token is op.
441	515	AssignMulWrap,
442		BangEqual,
443		BitAnd,
444		BitOr,
	516	/// `lhs +%= rhs`. main_token is op.
	517	AssignAddWrap,
	518	/// `lhs -%= rhs`. main_token is op.
	519	AssignSubWrap,
	520	/// `lhs = rhs`. main_token is op.
	521	Assign,
	522	/// `lhs \|\| rhs`. main_token is the `\|\|`.
	523	MergeErrorSets,
	524	/// `lhs * rhs`. main_token is the `*`.
	525	Mul,
	526	/// `lhs / rhs`. main_token is the `/`.
	527	Div,
	528	/// `lhs % rhs`. main_token is the `%`.
	529	Mod,
	530	/// `lhs rhs`. main_token is the ``.
	531	ArrayMult,
	532	/// `lhs % rhs`. main_token is the `%`.
	533	MulWrap,
	534	/// `lhs + rhs`. main_token is the `+`.
	535	Add,
	536	/// `lhs - rhs`. main_token is the `-`.
	537	Sub,
	538	/// `lhs ++ rhs`. main_token is the `++`.
	539	ArrayCat,
	540	/// `lhs +% rhs`. main_token is the `+%`.
	541	AddWrap,
	542	/// `lhs -% rhs`. main_token is the `-%`.
	543	SubWrap,
	544	/// `lhs << rhs`. main_token is the `<<`.
445	545	BitShiftLeft,
	546	/// `lhs >> rhs`. main_token is the `>>`.
446	547	BitShiftRight,
	548	/// `lhs & rhs`. main_token is the `&`.
	549	BitAnd,
	550	/// `lhs ^ rhs`. main_token is the `^`.
447	551	BitXor,
	552	/// `lhs \| rhs`. main_token is the `\|`.
	553	BitOr,
	554	/// `lhs orelse rhs`. main_token is the `orelse`.
	555	OrElse,
	556	/// `lhs and rhs`. main_token is the `and`.
448	557	BoolAnd,
	558	/// `lhs or rhs`. main_token is the `or`.
449	559	BoolOr,
450		Div,
451		EqualEqual,
452		ErrorUnion,
453		GreaterOrEqual,
454		GreaterThan,
455		LessOrEqual,
456		LessThan,
457		MergeErrorSets,
458		Mod,
459		Mul,
460		MulWrap,
461		Period,
462		Range,
463		Sub,
464		SubWrap,
465		OrElse,
466
467		// SimplePrefixOp
468		AddressOf,
469		Await,
470		BitNot,
	560	/// `op lhs`. rhs unused. main_token is op.
471	561	BoolNot,
472		OptionalType,
	562	/// `op lhs`. rhs unused. main_token is op.
473	563	Negation,
	564	/// `op lhs`. rhs unused. main_token is op.
	565	BitNot,
	566	/// `op lhs`. rhs unused. main_token is op.
474	567	NegationWrap,
475		Resume,
	568	/// `op lhs`. rhs unused. main_token is op.
	569	AddressOf,
	570	/// `op lhs`. rhs unused. main_token is op.
476	571	Try,
477
	572	/// `op lhs`. rhs unused. main_token is op.
	573	Await,
	574	/// `?lhs`. rhs unused. main_token is the `?`.
	575	OptionalType,
	576	/// `[lhs]rhs`. lhs can be omitted to make it a slice.
478	577	ArrayType,
479		/// ArrayType but has a sentinel node.
	578	/// `[lhs:a]b`. `ArrayTypeSentinel[rhs]`.
480	579	ArrayTypeSentinel,
	580	/// `[*]align(lhs) rhs`. lhs can be omitted.
	581	/// `*align(lhs) rhs`. lhs can be omitted.
	582	/// `[]rhs`.
	583	PtrTypeAligned,
	584	/// `[*:lhs]rhs`. lhs can be omitted.
	585	/// `*rhs`.
	586	/// `[:lhs]rhs`.
	587	PtrTypeSentinel,
	588	/// lhs is index into PtrType. rhs is the element type expression.
481	589	PtrType,
	590	/// lhs is index into SliceType. rhs is the element type expression.
	591	/// Can be pointer or slice, depending on main_token.
482	592	SliceType,
483		/// `a[b..c]`
	593	/// `lhs[rhs..]`
	594	/// main_token is the `[`.
	595	SliceOpen,
	596	/// `lhs[b..c :d]`. `slice_list[rhs]`.
	597	/// main_token is the `[`.
484	598	Slice,
485		/// `a.*`
	599	/// `lhs.*`. rhs is unused.
486	600	Deref,
487		/// `a.?`
488		UnwrapOptional,
489		/// `a[b]`
	601	/// `lhs[rhs]`.
490	602	ArrayAccess,
491		/// `T{a, b}`
492		ArrayInitializer,
493		/// ArrayInitializer but with `.` instead of a left-hand-side operand.
494		ArrayInitializerDot,
495		/// `T{.a = b}`
496		StructInitializer,
497		/// StructInitializer but with `.` instead of a left-hand-side operand.
498		StructInitializerDot,
499		/// `foo()`
	603	/// `lhs{rhs}`. rhs can be omitted.
	604	ArrayInitOne,
	605	/// `.{lhs, rhs}`. lhs and rhs can be omitted.
	606	ArrayInitDotTwo,
	607	/// `.{a, b}`. `sub_list[lhs..rhs]`.
	608	ArrayInitDot,
	609	/// `lhs{a, b}`. `sub_range_list[rhs]`. lhs can be omitted which means `.{a, b}`.
	610	ArrayInit,
	611	/// `lhs{.a = rhs}`. rhs can be omitted making it empty.
	612	StructInitOne,
	613	/// `.{.a = lhs, .b = rhs}`. lhs and rhs can be omitted.
	614	StructInitDotTwo,
	615	/// `.{.a = b, .c = d}`. `sub_list[lhs..rhs]`.
	616	StructInitDot,
	617	/// `lhs{.a = b, .c = d}`. `sub_range_list[rhs]`.
	618	/// lhs can be omitted which means `.{.a = b, .c = d}`.
	619	StructInit,
	620	/// `lhs(rhs)`. rhs can be omitted.
	621	CallOne,
	622	/// `lhs(a, b, c)`. `sub_range_list[rhs]`.
	623	/// main_token is the `(`.
500	624	Call,
501
502		// Control flow
	625	/// `switch(lhs) {}`. `sub_range_list[rhs]`.
503	626	Switch,
	627	/// `lhs => rhs`. If lhs is omitted it means `else`.
	628	/// main_token is the `=>`
	629	SwitchCaseOne,
	630	/// `a, b, c => rhs`. `sub_range_list[lhs]`.
	631	SwitchCaseMulti,
	632	/// `lhs...rhs`.
	633	SwitchRange,
	634	/// `while (lhs) rhs`.
	635	WhileSimple,
	636	/// `while (lhs) \|x\| rhs`.
	637	WhileSimpleOptional,
	638	/// `while (lhs) : (a) b`. `WhileCont[rhs]`.
	639	WhileCont,
	640	/// `while (lhs) : (a) b`. `WhileCont[rhs]`.
	641	WhileContOptional,
	642	/// `while (lhs) : (a) b else c`. `While[rhs]`.
504	643	While,
	644	/// `while (lhs) \|x\| : (a) b else c`. `While[rhs]`.
	645	WhileOptional,
	646	/// `while (lhs) \|x\| : (a) b else \|y\| c`. `While[rhs]`.
	647	WhileError,
	648	/// `for (lhs) rhs`.
	649	ForSimple,
	650	/// `for (lhs) a else b`. `if_list[rhs]`.
505	651	For,
	652	/// `if (lhs) rhs`.
	653	IfSimple,
	654	/// `if (lhs) \|a\| rhs`.
	655	IfSimpleOptional,
	656	/// `if (lhs) a else b`. `if_list[rhs]`.
506	657	If,
	658	/// `if (lhs) \|x\| a else b`. `if_list[rhs]`.
	659	IfOptional,
	660	/// `if (lhs) \|x\| a else \|y\| b`. `if_list[rhs]`.
	661	IfError,
	662	/// `suspend lhs`. lhs can be omitted. rhs is unused.
507	663	Suspend,
	664	/// `resume lhs`. rhs is unused.
	665	Resume,
	666	/// `continue`. lhs is token index of label if any. rhs is unused.
508	667	Continue,
	668	/// `break rhs`. rhs can be omitted. lhs is label token index, if any.
509	669	Break,
	670	/// `return lhs`. lhs can be omitted. rhs is unused.
510	671	Return,
511
512		// Type expressions
513		AnyType,
514		ErrorType,
	672	/// `fn(a: lhs) rhs`. lhs can be omitted.
	673	/// anytype and ... parameters are omitted from the AST tree.
	674	FnProtoSimple,
	675	/// `fn(a: b, c: d) rhs`. `sub_range_list[lhs]`.
	676	/// anytype and ... parameters are omitted from the AST tree.
	677	FnProtoSimpleMulti,
	678	/// `fn(a: b) rhs linksection(e) callconv(f)`. lhs is index into extra_data.
	679	/// zero or one parameters.
	680	/// anytype and ... parameters are omitted from the AST tree.
	681	FnProtoOne,
	682	/// `fn(a: b, c: d) rhs linksection(e) callconv(f)`. `fn_proto_list[lhs]`.
	683	/// anytype and ... parameters are omitted from the AST tree.
515	684	FnProto,
	685	/// lhs is the FnProto, rhs is the function body block.
	686	FnDecl,
	687	/// `anyframe->rhs`. main_token is `anyframe`. `lhs` is arrow token index.
516	688	AnyFrameType,
517
518		// Primary expressions
519		IntegerLiteral,
520		FloatLiteral,
	689	/// Could be integer literal, float literal, char literal, bool literal,
	690	/// null literal, undefined literal, unreachable, depending on the token.
	691	/// Both lhs and rhs unused.
	692	OneToken,
	693	/// Both lhs and rhs unused.
	694	/// Most identifiers will not have explicit AST nodes, however for expressions
	695	/// which could be one of many different kinds of AST nodes, there will be an
	696	/// Identifier AST node for it.
	697	Identifier,
	698	/// lhs is the dot token index, rhs unused, main_token is the identifier.
521	699	EnumLiteral,
522		StringLiteral,
	700	/// Both lhs and rhs unused.
523	701	MultilineStringLiteral,
524		CharLiteral,
525		BoolLiteral,
526		NullLiteral,
527		UndefinedLiteral,
528		Unreachable,
529		Identifier,
	702	/// `(lhs)`. main_token is the `(`; rhs is the token index of the `)`.
530	703	GroupedExpression,
	704	/// `@a(lhs, rhs)`. lhs and rhs may be omitted.
	705	BuiltinCallTwo,
	706	/// `@a(b, c)`. `sub_list[lhs..rhs]`.
531	707	BuiltinCall,
	708	/// `error{a, b}`.
	709	/// lhs and rhs both unused.
532	710	ErrorSetDecl,
	711	/// `struct {}`, `union {}`, etc. `sub_list[lhs..rhs]`.
533	712	ContainerDecl,
534		Asm,
	713	/// `union(lhs)` / `enum(lhs)`. `sub_range_list[rhs]`.
	714	ContainerDeclArg,
	715	/// `union(enum) {}`. `sub_list[lhs..rhs]`.
	716	/// Note that tagged unions with explicitly provided enums are represented
	717	/// by `ContainerDeclArg`.
	718	TaggedUnion,
	719	/// `union(enum(lhs)) {}`. `sub_list_range[rhs]`.
	720	TaggedUnionEnumTag,
	721	/// `a: lhs = rhs,`. lhs and rhs can be omitted.
	722	ContainerFieldInit,
	723	/// `a: lhs align(rhs),`. rhs can be omitted.
	724	ContainerFieldAlign,
	725	/// `a: lhs align(c) = d,`. `container_field_list[rhs]`.
	726	ContainerField,
	727	/// `anytype`. both lhs and rhs unused.
	728	/// Used by `ContainerField`.
	729	AnyType,
	730	/// `comptime lhs`. rhs unused.
535	731	Comptime,
	732	/// `nosuspend lhs`. rhs unused.
536	733	Nosuspend,
	734	/// `{}`. `sub_list[lhs..rhs]`.
537	735	Block,
538		LabeledBlock,
539
540		// Misc
541		DocComment,
542		SwitchCase, // TODO make this not a child of AST Node
543		SwitchElse, // TODO make this not a child of AST Node
544		Else, // TODO make this not a child of AST Node
545		Payload, // TODO make this not a child of AST Node
546		PointerPayload, // TODO make this not a child of AST Node
547		PointerIndexPayload, // TODO make this not a child of AST Node
548		ContainerField,
549		ErrorTag, // TODO make this not a child of AST Node
550		FieldInitializer, // TODO make this not a child of AST Node
551
552		pub fn Type(tag: Tag) type {
553		return switch (tag) {
554		.Root => Root,
555		.Use => Use,
556		.TestDecl => TestDecl,
557		.VarDecl => VarDecl,
558		.Defer => Defer,
559		.Catch => Catch,
560
561		.Add,
562		.AddWrap,
563		.ArrayCat,
564		.ArrayMult,
565		.Assign,
566		.AssignBitAnd,
567		.AssignBitOr,
568		.AssignBitShiftLeft,
569		.AssignBitShiftRight,
570		.AssignBitXor,
571		.AssignDiv,
572		.AssignSub,
573		.AssignSubWrap,
574		.AssignMod,
575		.AssignAdd,
576		.AssignAddWrap,
577		.AssignMul,
578		.AssignMulWrap,
579		.BangEqual,
580		.BitAnd,
581		.BitOr,
582		.BitShiftLeft,
583		.BitShiftRight,
584		.BitXor,
585		.BoolAnd,
586		.BoolOr,
587		.Div,
588		.EqualEqual,
589		.ErrorUnion,
590		.GreaterOrEqual,
591		.GreaterThan,
592		.LessOrEqual,
593		.LessThan,
594		.MergeErrorSets,
595		.Mod,
596		.Mul,
597		.MulWrap,
598		.Period,
599		.Range,
600		.Sub,
601		.SubWrap,
602		.OrElse,
603		=> SimpleInfixOp,
604
605		.AddressOf,
606		.Await,
607		.BitNot,
608		.BoolNot,
609		.OptionalType,
610		.Negation,
611		.NegationWrap,
612		.Resume,
613		.Try,
614		=> SimplePrefixOp,
615
616		.Identifier,
617		.BoolLiteral,
618		.NullLiteral,
619		.UndefinedLiteral,
620		.Unreachable,
621		.AnyType,
622		.ErrorType,
623		.IntegerLiteral,
624		.FloatLiteral,
625		.StringLiteral,
626		.CharLiteral,
627		=> OneToken,
628
629		.Continue,
630		.Break,
631		.Return,
632		=> ControlFlowExpression,
633
634		.ArrayType => ArrayType,
635		.ArrayTypeSentinel => ArrayTypeSentinel,
636
637		.PtrType => PtrType,
638		.SliceType => SliceType,
639		.Slice => Slice,
640		.Deref, .UnwrapOptional => SimpleSuffixOp,
641		.ArrayAccess => ArrayAccess,
642
643		.ArrayInitializer => ArrayInitializer,
644		.ArrayInitializerDot => ArrayInitializerDot,
645
646		.StructInitializer => StructInitializer,
647		.StructInitializerDot => StructInitializerDot,
648
649		.Call => Call,
650		.Switch => Switch,
651		.While => While,
652		.For => For,
653		.If => If,
654		.Suspend => Suspend,
655		.FnProto => FnProto,
656		.AnyFrameType => AnyFrameType,
657		.EnumLiteral => EnumLiteral,
658		.MultilineStringLiteral => MultilineStringLiteral,
659		.GroupedExpression => GroupedExpression,
660		.BuiltinCall => BuiltinCall,
661		.ErrorSetDecl => ErrorSetDecl,
662		.ContainerDecl => ContainerDecl,
663		.Asm => Asm,
664		.Comptime => Comptime,
665		.Nosuspend => Nosuspend,
666		.Block => Block,
667		.LabeledBlock => LabeledBlock,
668		.DocComment => DocComment,
669		.SwitchCase => SwitchCase,
670		.SwitchElse => SwitchElse,
671		.Else => Else,
672		.Payload => Payload,
673		.PointerPayload => PointerPayload,
674		.PointerIndexPayload => PointerIndexPayload,
675		.ContainerField => ContainerField,
676		.ErrorTag => ErrorTag,
677		.FieldInitializer => FieldInitializer,
678		};
679		}
680
681		pub fn isBlock(tag: Tag) bool {
682		return switch (tag) {
683		.Block, .LabeledBlock => true,
684		else => false,
685		};
686		}
	736	/// `asm(lhs)`. rhs unused.
	737	AsmSimple,
	738	/// `asm(lhs, a)`. `sub_range_list[rhs]`.
	739	Asm,
	740	/// `[a] "b" (c)`. lhs is string literal token index, rhs is 0.
	741	/// `[a] "b" (-> rhs)`. lhs is the string literal token index, rhs is type expr.
	742	/// main_token is `a`.
	743	AsmOutput,
	744	/// `[a] "b" (rhs)`. lhs is string literal token index.
	745	/// main_token is `a`.
	746	AsmInput,
	747	/// `error.a`. lhs is token index of `.`. rhs is token index of `a`.
	748	ErrorValue,
	749	/// `lhs!rhs`. main_token is the `!`.
	750	ErrorUnion,
687	751	};
688	752
689		/// Prefer `castTag` to this.
690		pub fn cast(base: Node, comptime T: type) ?T {
691		if (std.meta.fieldInfo(T, .base).default_value) \|default_base\| {
692		return base.castTag(default_base.tag);
693		}
694		inline for (@typeInfo(Tag).Enum.fields) \|field\| {
695		const tag = @intToEnum(Tag, field.value);
696		if (base.tag == tag) {
697		if (T == tag.Type()) {
698		return @fieldParentPtr(T, "base", base);
699		}
700		return null;
701		}
702		}
703		unreachable;
704		}
705
706		pub fn castTag(base: Node, comptime tag: Tag) ?tag.Type() {
707		if (base.tag == tag) {
708		return @fieldParentPtr(tag.Type(), "base", base);
709		}
710		return null;
711		}
712
713		pub fn iterate(base: Node, index: usize) ?Node {
714		inline for (@typeInfo(Tag).Enum.fields) \|field\| {
715		const tag = @intToEnum(Tag, field.value);
716		if (base.tag == tag) {
717		return @fieldParentPtr(tag.Type(), "base", base).iterate(index);
718		}
719		}
720		unreachable;
721		}
722
723		pub fn firstToken(base: *const Node) TokenIndex {
724		inline for (@typeInfo(Tag).Enum.fields) \|field\| {
725		const tag = @intToEnum(Tag, field.value);
726		if (base.tag == tag) {
727		return @fieldParentPtr(tag.Type(), "base", base).firstToken();
728		}
729		}
730		unreachable;
731		}
732
733		pub fn lastToken(base: *const Node) TokenIndex {
734		inline for (@typeInfo(Tag).Enum.fields) \|field\| {
735		const tag = @intToEnum(Tag, field.value);
736		if (base.tag == tag) {
737		return @fieldParentPtr(tag.Type(), "base", base).lastToken();
738		}
739		}
740		unreachable;
741		}
742
743		pub fn requireSemiColon(base: *const Node) bool {
744		var n = base;
745		while (true) {
746		switch (n.tag) {
747		.Root,
748		.ContainerField,
749		.Block,
750		.LabeledBlock,
751		.Payload,
752		.PointerPayload,
753		.PointerIndexPayload,
754		.Switch,
755		.SwitchCase,
756		.SwitchElse,
757		.FieldInitializer,
758		.DocComment,
759		.TestDecl,
760		=> return false,
761
762		.While => {
763		const while_node = @fieldParentPtr(While, "base", n);
764		if (while_node.@"else") \|@"else"\| {
765		n = &@"else".base;
766		continue;
767		}
768
769		return !while_node.body.tag.isBlock();
770		},
771		.For => {
772		const for_node = @fieldParentPtr(For, "base", n);
773		if (for_node.@"else") \|@"else"\| {
774		n = &@"else".base;
775		continue;
776		}
777
778		return !for_node.body.tag.isBlock();
779		},
780		.If => {
781		const if_node = @fieldParentPtr(If, "base", n);
782		if (if_node.@"else") \|@"else"\| {
783		n = &@"else".base;
784		continue;
785		}
786
787		return !if_node.body.tag.isBlock();
788		},
789		.Else => {
790		const else_node = @fieldParentPtr(Else, "base", n);
791		n = else_node.body;
792		continue;
793		},
794		.Defer => {
795		const defer_node = @fieldParentPtr(Defer, "base", n);
796		return !defer_node.expr.tag.isBlock();
797		},
798		.Comptime => {
799		const comptime_node = @fieldParentPtr(Comptime, "base", n);
800		return !comptime_node.expr.tag.isBlock();
801		},
802		.Suspend => {
803		const suspend_node = @fieldParentPtr(Suspend, "base", n);
804		if (suspend_node.body) \|body\| {
805		return !body.tag.isBlock();
806		}
807
808		return true;
809		},
810		.Nosuspend => {
811		const nosuspend_node = @fieldParentPtr(Nosuspend, "base", n);
812		return !nosuspend_node.expr.tag.isBlock();
813		},
814		else => return true,
815		}
816		}
817		}
818
819		/// Asserts the node is a Block or LabeledBlock and returns the statements slice.
820		pub fn blockStatements(base: Node) []Node {
821		if (base.castTag(.Block)) \|block\| {
822		return block.statements();
823		} else if (base.castTag(.LabeledBlock)) \|labeled_block\| {
824		return labeled_block.statements();
825		} else {
826		unreachable;
827		}
828		}
829
830		pub fn findFirstWithId(self: Node, id: Id) ?Node {
831		if (self.id == id) return self;
832		var child_i: usize = 0;
833		while (self.iterate(child_i)) \|child\| : (child_i += 1) {
834		if (child.findFirstWithId(id)) \|result\| return result;
835		}
836		return null;
837		}
838
839		pub fn dump(self: *Node, indent: usize) void {
840		{
841		var i: usize = 0;
842		while (i < indent) : (i += 1) {
843		std.debug.warn(" ", .{});
844		}
845		}
846		std.debug.warn("{s}\n", .{@tagName(self.tag)});
847
848		var child_i: usize = 0;
849		while (self.iterate(child_i)) \|child\| : (child_i += 1) {
850		child.dump(indent + 2);
851		}
852		}
853
854		/// The decls data follows this struct in memory as an array of Node pointers.
855		pub const Root = struct {
856		base: Node = Node{ .tag = .Root },
857		eof_token: TokenIndex,
858		decls_len: NodeIndex,
859
860		/// After this the caller must initialize the decls list.
861		pub fn create(allocator: mem.Allocator, decls_len: NodeIndex, eof_token: TokenIndex) !Root {
862		const bytes = try allocator.alignedAlloc(u8, @alignOf(Root), sizeInBytes(decls_len));
863		const self = @ptrCast(*Root, bytes.ptr);
864		self.* = .{
865		.eof_token = eof_token,
866		.decls_len = decls_len,
867		};
868		return self;
869		}
870
871		pub fn destroy(self: Decl, allocator: mem.Allocator) void {
872		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.decls_len)];
873		allocator.free(bytes);
874		}
875
876		pub fn iterate(self: const Root, index: usize) ?Node {
877		var i = index;
878
879		if (i < self.decls_len) return self.declsConst()[i];
880		return null;
881		}
882
883		pub fn decls(self: Root) []Node {
884		const decls_start = @ptrCast([*]u8, self) + @sizeOf(Root);
885		return @ptrCast([]Node, decls_start)[0..self.decls_len];
886		}
887
888		pub fn declsConst(self: const Root) []const Node {
889		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(Root);
890		return @ptrCast([]const Node, decls_start)[0..self.decls_len];
891		}
892
893		pub fn firstToken(self: *const Root) TokenIndex {
894		if (self.decls_len == 0) return self.eof_token;
895		return self.declsConst()[0].firstToken();
896		}
897
898		pub fn lastToken(self: *const Root) TokenIndex {
899		if (self.decls_len == 0) return self.eof_token;
900		return self.declsConst()[self.decls_len - 1].lastToken();
901		}
902
903		fn sizeInBytes(decls_len: NodeIndex) usize {
904		return @sizeOf(Root) + @sizeOf(Node) @as(usize, decls_len);
905		}
	753	pub const Data = struct {
	754	lhs: Index,
	755	rhs: Index,
906	756	};
907	757
908		/// Trailed in memory by possibly many things, with each optional thing
909		/// determined by a bit in `trailer_flags`.
910		pub const VarDecl = struct {
911		base: Node = Node{ .tag = .VarDecl },
912		trailer_flags: TrailerFlags,
913		mut_token: TokenIndex,
914		name_token: TokenIndex,
915		semicolon_token: TokenIndex,
916
917		pub const TrailerFlags = std.meta.TrailerFlags(struct {
918		doc_comments: *DocComment,
919		visib_token: TokenIndex,
920		thread_local_token: TokenIndex,
921		eq_token: TokenIndex,
922		comptime_token: TokenIndex,
923		extern_export_token: TokenIndex,
924		lib_name: *Node,
925		type_node: *Node,
926		align_node: *Node,
927		section_node: *Node,
928		init_node: *Node,
929		});
930
931		pub fn getDocComments(self: const VarDecl) ?DocComment {
932		return self.getTrailer(.doc_comments);
933		}
934
935		pub fn setDocComments(self: VarDecl, value: DocComment) void {
936		self.setTrailer(.doc_comments, value);
937		}
938
939		pub fn getVisibToken(self: *const VarDecl) ?TokenIndex {
940		return self.getTrailer(.visib_token);
941		}
942
943		pub fn setVisibToken(self: *VarDecl, value: TokenIndex) void {
944		self.setTrailer(.visib_token, value);
945		}
946
947		pub fn getThreadLocalToken(self: *const VarDecl) ?TokenIndex {
948		return self.getTrailer(.thread_local_token);
949		}
950
951		pub fn setThreadLocalToken(self: *VarDecl, value: TokenIndex) void {
952		self.setTrailer(.thread_local_token, value);
953		}
954
955		pub fn getEqToken(self: *const VarDecl) ?TokenIndex {
956		return self.getTrailer(.eq_token);
957		}
958
959		pub fn setEqToken(self: *VarDecl, value: TokenIndex) void {
960		self.setTrailer(.eq_token, value);
961		}
962
963		pub fn getComptimeToken(self: *const VarDecl) ?TokenIndex {
964		return self.getTrailer(.comptime_token);
965		}
966
967		pub fn setComptimeToken(self: *VarDecl, value: TokenIndex) void {
968		self.setTrailer(.comptime_token, value);
969		}
970
971		pub fn getExternExportToken(self: *const VarDecl) ?TokenIndex {
972		return self.getTrailer(.extern_export_token);
973		}
974
975		pub fn setExternExportToken(self: *VarDecl, value: TokenIndex) void {
976		self.setTrailer(.extern_export_token, value);
977		}
978
979		pub fn getLibName(self: const VarDecl) ?Node {
980		return self.getTrailer(.lib_name);
981		}
982
983		pub fn setLibName(self: VarDecl, value: Node) void {
984		self.setTrailer(.lib_name, value);
985		}
986
987		pub fn getTypeNode(self: const VarDecl) ?Node {
988		return self.getTrailer(.type_node);
989		}
990
991		pub fn setTypeNode(self: VarDecl, value: Node) void {
992		self.setTrailer(.type_node, value);
993		}
994
995		pub fn getAlignNode(self: const VarDecl) ?Node {
996		return self.getTrailer(.align_node);
997		}
998
999		pub fn setAlignNode(self: VarDecl, value: Node) void {
1000		self.setTrailer(.align_node, value);
1001		}
1002
1003		pub fn getSectionNode(self: const VarDecl) ?Node {
1004		return self.getTrailer(.section_node);
1005		}
1006
1007		pub fn setSectionNode(self: VarDecl, value: Node) void {
1008		self.setTrailer(.section_node, value);
1009		}
1010
1011		pub fn getInitNode(self: const VarDecl) ?Node {
1012		return self.getTrailer(.init_node);
1013		}
1014
1015		pub fn setInitNode(self: VarDecl, value: Node) void {
1016		self.setTrailer(.init_node, value);
1017		}
1018
1019		pub const RequiredFields = struct {
1020		mut_token: TokenIndex,
1021		name_token: TokenIndex,
1022		semicolon_token: TokenIndex,
1023		};
1024
1025		fn getTrailer(self: *const VarDecl, comptime field: TrailerFlags.FieldEnum) ?TrailerFlags.Field(field) {
1026		const trailers_start = @ptrCast([*]const u8, self) + @sizeOf(VarDecl);
1027		return self.trailer_flags.get(trailers_start, field);
1028		}
1029
1030		fn setTrailer(self: *VarDecl, comptime field: TrailerFlags.FieldEnum, value: TrailerFlags.Field(field)) void {
1031		const trailers_start = @ptrCast([*]u8, self) + @sizeOf(VarDecl);
1032		self.trailer_flags.set(trailers_start, field, value);
1033		}
1034
1035		pub fn create(allocator: mem.Allocator, required: RequiredFields, trailers: TrailerFlags.InitStruct) !VarDecl {
1036		const trailer_flags = TrailerFlags.init(trailers);
1037		const bytes = try allocator.alignedAlloc(u8, @alignOf(VarDecl), sizeInBytes(trailer_flags));
1038		const var_decl = @ptrCast(*VarDecl, bytes.ptr);
1039		var_decl.* = .{
1040		.trailer_flags = trailer_flags,
1041		.mut_token = required.mut_token,
1042		.name_token = required.name_token,
1043		.semicolon_token = required.semicolon_token,
1044		};
1045		const trailers_start = bytes.ptr + @sizeOf(VarDecl);
1046		trailer_flags.setMany(trailers_start, trailers);
1047		return var_decl;
1048		}
1049
1050		pub fn destroy(self: VarDecl, allocator: mem.Allocator) void {
1051		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.trailer_flags)];
1052		allocator.free(bytes);
1053		}
1054
1055		pub fn iterate(self: const VarDecl, index: usize) ?Node {
1056		var i = index;
1057
1058		if (self.getTypeNode()) \|type_node\| {
1059		if (i < 1) return type_node;
1060		i -= 1;
1061		}
1062
1063		if (self.getAlignNode()) \|align_node\| {
1064		if (i < 1) return align_node;
1065		i -= 1;
1066		}
1067
1068		if (self.getSectionNode()) \|section_node\| {
1069		if (i < 1) return section_node;
1070		i -= 1;
1071		}
1072
1073		if (self.getInitNode()) \|init_node\| {
1074		if (i < 1) return init_node;
1075		i -= 1;
1076		}
1077
1078		return null;
1079		}
1080
1081		pub fn firstToken(self: *const VarDecl) TokenIndex {
1082		if (self.getVisibToken()) \|visib_token\| return visib_token;
1083		if (self.getThreadLocalToken()) \|thread_local_token\| return thread_local_token;
1084		if (self.getComptimeToken()) \|comptime_token\| return comptime_token;
1085		if (self.getExternExportToken()) \|extern_export_token\| return extern_export_token;
1086		assert(self.getLibName() == null);
1087		return self.mut_token;
1088		}
1089
1090		pub fn lastToken(self: *const VarDecl) TokenIndex {
1091		return self.semicolon_token;
1092		}
1093
1094		fn sizeInBytes(trailer_flags: TrailerFlags) usize {
1095		return @sizeOf(VarDecl) + trailer_flags.sizeInBytes();
1096		}
	758	pub const LocalVarDecl = struct {
	759	type_node: Index,
	760	align_node: Index,
1097	761	};
1098	762
1099		pub const Use = struct {
1100		base: Node = Node{ .tag = .Use },
1101		doc_comments: ?*DocComment,
1102		visib_token: ?TokenIndex,
1103		use_token: TokenIndex,
1104		expr: *Node,
1105		semicolon_token: TokenIndex,
1106
1107		pub fn iterate(self: const Use, index: usize) ?Node {
1108		var i = index;
1109
1110		if (i < 1) return self.expr;
1111		i -= 1;
1112
1113		return null;
1114		}
1115
1116		pub fn firstToken(self: *const Use) TokenIndex {
1117		if (self.visib_token) \|visib_token\| return visib_token;
1118		return self.use_token;
1119		}
1120
1121		pub fn lastToken(self: *const Use) TokenIndex {
1122		return self.semicolon_token;
1123		}
	763	pub const ArrayTypeSentinel = struct {
	764	elem_type: Index,
	765	sentinel: Index,
1124	766	};
1125	767
1126		pub const ErrorSetDecl = struct {
1127		base: Node = Node{ .tag = .ErrorSetDecl },
1128		error_token: TokenIndex,
1129		rbrace_token: TokenIndex,
1130		decls_len: NodeIndex,
1131
1132		/// After this the caller must initialize the decls list.
1133		pub fn alloc(allocator: mem.Allocator, decls_len: NodeIndex) !ErrorSetDecl {
1134		const bytes = try allocator.alignedAlloc(u8, @alignOf(ErrorSetDecl), sizeInBytes(decls_len));
1135		return @ptrCast(*ErrorSetDecl, bytes.ptr);
1136		}
1137
1138		pub fn free(self: ErrorSetDecl, allocator: mem.Allocator) void {
1139		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.decls_len)];
1140		allocator.free(bytes);
1141		}
1142
1143		pub fn iterate(self: const ErrorSetDecl, index: usize) ?Node {
1144		var i = index;
1145
1146		if (i < self.decls_len) return self.declsConst()[i];
1147		i -= self.decls_len;
1148
1149		return null;
1150		}
1151
1152		pub fn firstToken(self: *const ErrorSetDecl) TokenIndex {
1153		return self.error_token;
1154		}
1155
1156		pub fn lastToken(self: *const ErrorSetDecl) TokenIndex {
1157		return self.rbrace_token;
1158		}
1159
1160		pub fn decls(self: ErrorSetDecl) []Node {
1161		const decls_start = @ptrCast([*]u8, self) + @sizeOf(ErrorSetDecl);
1162		return @ptrCast([]Node, decls_start)[0..self.decls_len];
1163		}
1164
1165		pub fn declsConst(self: const ErrorSetDecl) []const Node {
1166		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(ErrorSetDecl);
1167		return @ptrCast([]const Node, decls_start)[0..self.decls_len];
1168		}
1169
1170		fn sizeInBytes(decls_len: NodeIndex) usize {
1171		return @sizeOf(ErrorSetDecl) + @sizeOf(Node) @as(usize, decls_len);
1172		}
	768	pub const PtrType = struct {
	769	sentinel: Index,
	770	align_node: Index,
	771	bit_range_start: Index,
	772	bit_range_end: Index,
1173	773	};
1174	774
1175		/// The fields and decls Node pointers directly follow this struct in memory.
1176		pub const ContainerDecl = struct {
1177		base: Node = Node{ .tag = .ContainerDecl },
1178		kind_token: TokenIndex,
1179		layout_token: ?TokenIndex,
1180		lbrace_token: TokenIndex,
1181		rbrace_token: TokenIndex,
1182		fields_and_decls_len: NodeIndex,
1183		init_arg_expr: InitArg,
1184
1185		pub const InitArg = union(enum) {
1186		None,
1187		Enum: ?*Node,
1188		Type: *Node,
1189		};
1190
1191		/// After this the caller must initialize the fields_and_decls list.
1192		pub fn alloc(allocator: mem.Allocator, fields_and_decls_len: NodeIndex) !ContainerDecl {
1193		const bytes = try allocator.alignedAlloc(u8, @alignOf(ContainerDecl), sizeInBytes(fields_and_decls_len));
1194		return @ptrCast(*ContainerDecl, bytes.ptr);
1195		}
1196
1197		pub fn free(self: ContainerDecl, allocator: mem.Allocator) void {
1198		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.fields_and_decls_len)];
1199		allocator.free(bytes);
1200		}
1201
1202		pub fn iterate(self: const ContainerDecl, index: usize) ?Node {
1203		var i = index;
1204
1205		switch (self.init_arg_expr) {
1206		.Type => \|t\| {
1207		if (i < 1) return t;
1208		i -= 1;
1209		},
1210		.None, .Enum => {},
1211		}
1212
1213		if (i < self.fields_and_decls_len) return self.fieldsAndDeclsConst()[i];
1214		i -= self.fields_and_decls_len;
1215
1216		return null;
1217		}
1218
1219		pub fn firstToken(self: *const ContainerDecl) TokenIndex {
1220		if (self.layout_token) \|layout_token\| {
1221		return layout_token;
1222		}
1223		return self.kind_token;
1224		}
1225
1226		pub fn lastToken(self: *const ContainerDecl) TokenIndex {
1227		return self.rbrace_token;
1228		}
1229
1230		pub fn fieldsAndDecls(self: ContainerDecl) []Node {
1231		const decls_start = @ptrCast([*]u8, self) + @sizeOf(ContainerDecl);
1232		return @ptrCast([]Node, decls_start)[0..self.fields_and_decls_len];
1233		}
1234
1235		pub fn fieldsAndDeclsConst(self: const ContainerDecl) []const Node {
1236		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(ContainerDecl);
1237		return @ptrCast([]const Node, decls_start)[0..self.fields_and_decls_len];
1238		}
	775	pub const SliceType = struct {
	776	sentinel: Index,
	777	align_node: Index,
	778	};
	779	pub const SubRange = struct {
	780	/// Index into sub_list.
	781	start: Index,
	782	/// Index into sub_list.
	783	end: Index,
	784	};
1239	785
1240		fn sizeInBytes(fields_and_decls_len: NodeIndex) usize {
1241		return @sizeOf(ContainerDecl) + @sizeOf(Node) @as(usize, fields_and_decls_len);
1242		}
	786	pub const If = struct {
	787	then_expr: Index,
	788	else_expr: Index,
1243	789	};
1244	790
1245	791	pub const ContainerField = struct {
1246		base: Node = Node{ .tag = .ContainerField },
1247		doc_comments: ?*DocComment,
1248		comptime_token: ?TokenIndex,
1249		name_token: TokenIndex,
1250		type_expr: ?*Node,
1251		value_expr: ?*Node,
1252		align_expr: ?*Node,
	792	value_expr: Index,
	793	align_expr: Index,
	794	};
1253	795
1254		pub fn iterate(self: const ContainerField, index: usize) ?Node {
1255		var i = index;
	796	pub const GlobalVarDecl = struct {
	797	type_node: Index,
	798	align_node: Index,
	799	section_node: Index,
	800	};
1256	801
1257		if (self.type_expr) \|type_expr\| {
1258		if (i < 1) return type_expr;
1259		i -= 1;
1260		}
	802	pub const Slice = struct {
	803	start: Index,
	804	end: Index,
	805	sentinel: Index,
	806	};
1261	807
1262		if (self.align_expr) \|align_expr\| {
1263		if (i < 1) return align_expr;
1264		i -= 1;
1265		}
	808	pub const While = struct {
	809	continue_expr: Index,
	810	then_expr: Index,
	811	else_expr: Index,
	812	};
1266	813
1267		if (self.value_expr) \|value_expr\| {
1268		if (i < 1) return value_expr;
1269		i -= 1;
1270		}
	814	pub const WhileCont = struct {
	815	continue_expr: Index,
	816	then_expr: Index,
	817	};
1271	818
1272		return null;
1273		}
1274
1275		pub fn firstToken(self: *const ContainerField) TokenIndex {
1276		return self.comptime_token orelse self.name_token;
1277		}
1278
1279		pub fn lastToken(self: *const ContainerField) TokenIndex {
1280		if (self.value_expr) \|value_expr\| {
1281		return value_expr.lastToken();
1282		}
1283		if (self.align_expr) \|align_expr\| {
1284		// The expression refers to what's inside the parenthesis, the
1285		// last token is the closing one
1286		return align_expr.lastToken() + 1;
1287		}
1288		if (self.type_expr) \|type_expr\| {
1289		return type_expr.lastToken();
1290		}
1291
1292		return self.name_token;
1293		}
1294		};
1295
1296		pub const ErrorTag = struct {
1297		base: Node = Node{ .tag = .ErrorTag },
1298		doc_comments: ?*DocComment,
1299		name_token: TokenIndex,
1300
1301		pub fn iterate(self: const ErrorTag, index: usize) ?Node {
1302		var i = index;
1303
1304		if (self.doc_comments) \|comments\| {
1305		if (i < 1) return &comments.base;
1306		i -= 1;
1307		}
1308
1309		return null;
1310		}
1311
1312		pub fn firstToken(self: *const ErrorTag) TokenIndex {
1313		return self.name_token;
1314		}
1315
1316		pub fn lastToken(self: *const ErrorTag) TokenIndex {
1317		return self.name_token;
1318		}
1319		};
1320
1321		pub const OneToken = struct {
1322		base: Node,
1323		token: TokenIndex,
1324
1325		pub fn iterate(self: const OneToken, index: usize) ?Node {
1326		return null;
1327		}
1328
1329		pub fn firstToken(self: *const OneToken) TokenIndex {
1330		return self.token;
1331		}
1332
1333		pub fn lastToken(self: *const OneToken) TokenIndex {
1334		return self.token;
1335		}
1336		};
1337
1338		/// The params are directly after the FnProto in memory.
1339		/// Next, each optional thing determined by a bit in `trailer_flags`.
1340		pub const FnProto = struct {
1341		base: Node = Node{ .tag = .FnProto },
1342		trailer_flags: TrailerFlags,
1343		fn_token: TokenIndex,
1344		params_len: NodeIndex,
1345		return_type: ReturnType,
1346
1347		pub const TrailerFlags = std.meta.TrailerFlags(struct {
1348		doc_comments: *DocComment,
1349		body_node: *Node,
1350		lib_name: *Node, // populated if this is an extern declaration
1351		align_expr: *Node, // populated if align(A) is present
1352		section_expr: *Node, // populated if linksection(A) is present
1353		callconv_expr: *Node, // populated if callconv(A) is present
1354		visib_token: TokenIndex,
1355		name_token: TokenIndex,
1356		var_args_token: TokenIndex,
1357		extern_export_inline_token: TokenIndex,
1358		is_extern_prototype: void, // TODO: Remove once extern fn rewriting is
1359		is_async: void, // TODO: remove once async fn rewriting is
1360		});
1361
1362		pub const RequiredFields = struct {
1363		fn_token: TokenIndex,
1364		params_len: NodeIndex,
1365		return_type: ReturnType,
1366		};
1367
1368		pub const ReturnType = union(enum) {
1369		Explicit: *Node,
1370		InferErrorSet: *Node,
1371		Invalid: TokenIndex,
1372		};
1373
1374		pub const ParamDecl = struct {
1375		doc_comments: ?*DocComment,
1376		comptime_token: ?TokenIndex,
1377		noalias_token: ?TokenIndex,
1378		name_token: ?TokenIndex,
1379		param_type: ParamType,
1380
1381		pub const ParamType = union(enum) {
1382		any_type: *Node,
1383		type_expr: *Node,
1384		};
1385
1386		pub fn iterate(self: const ParamDecl, index: usize) ?Node {
1387		var i = index;
1388
1389		if (i < 1) {
1390		switch (self.param_type) {
1391		.any_type, .type_expr => \|node\| return node,
1392		}
1393		}
1394		i -= 1;
1395
1396		return null;
1397		}
1398
1399		pub fn firstToken(self: *const ParamDecl) TokenIndex {
1400		if (self.comptime_token) \|comptime_token\| return comptime_token;
1401		if (self.noalias_token) \|noalias_token\| return noalias_token;
1402		if (self.name_token) \|name_token\| return name_token;
1403		switch (self.param_type) {
1404		.any_type, .type_expr => \|node\| return node.firstToken(),
1405		}
1406		}
1407
1408		pub fn lastToken(self: *const ParamDecl) TokenIndex {
1409		switch (self.param_type) {
1410		.any_type, .type_expr => \|node\| return node.lastToken(),
1411		}
1412		}
1413		};
1414
1415		/// For debugging purposes.
1416		pub fn dump(self: *const FnProto) void {
1417		const trailers_start = @alignCast(
1418		@alignOf(ParamDecl),
1419		@ptrCast([]const u8, self) + @sizeOf(FnProto) + @sizeOf(ParamDecl) self.params_len,
1420		);
1421		std.debug.print("{} flags: {b} name_token: {s} {} params_len: {d}\n", .{
1422		self,
1423		self.trailer_flags.bits,
1424		self.getNameToken(),
1425		self.trailer_flags.ptrConst(trailers_start, .name_token),
1426		self.params_len,
1427		});
1428		}
1429
1430		pub fn getDocComments(self: const FnProto) ?DocComment {
1431		return self.getTrailer(.doc_comments);
1432		}
1433
1434		pub fn setDocComments(self: FnProto, value: DocComment) void {
1435		self.setTrailer(.doc_comments, value);
1436		}
1437
1438		pub fn getBodyNode(self: const FnProto) ?Node {
1439		return self.getTrailer(.body_node);
1440		}
1441
1442		pub fn setBodyNode(self: FnProto, value: Node) void {
1443		self.setTrailer(.body_node, value);
1444		}
1445
1446		pub fn getLibName(self: const FnProto) ?Node {
1447		return self.getTrailer(.lib_name);
1448		}
1449
1450		pub fn setLibName(self: FnProto, value: Node) void {
1451		self.setTrailer(.lib_name, value);
1452		}
1453
1454		pub fn getAlignExpr(self: const FnProto) ?Node {
1455		return self.getTrailer(.align_expr);
1456		}
1457
1458		pub fn setAlignExpr(self: FnProto, value: Node) void {
1459		self.setTrailer(.align_expr, value);
1460		}
1461
1462		pub fn getSectionExpr(self: const FnProto) ?Node {
1463		return self.getTrailer(.section_expr);
1464		}
1465
1466		pub fn setSectionExpr(self: FnProto, value: Node) void {
1467		self.setTrailer(.section_expr, value);
1468		}
1469
1470		pub fn getCallconvExpr(self: const FnProto) ?Node {
1471		return self.getTrailer(.callconv_expr);
1472		}
1473
1474		pub fn setCallconvExpr(self: FnProto, value: Node) void {
1475		self.setTrailer(.callconv_expr, value);
1476		}
1477
1478		pub fn getVisibToken(self: *const FnProto) ?TokenIndex {
1479		return self.getTrailer(.visib_token);
1480		}
1481
1482		pub fn setVisibToken(self: *FnProto, value: TokenIndex) void {
1483		self.setTrailer(.visib_token, value);
1484		}
1485
1486		pub fn getNameToken(self: *const FnProto) ?TokenIndex {
1487		return self.getTrailer(.name_token);
1488		}
1489
1490		pub fn setNameToken(self: *FnProto, value: TokenIndex) void {
1491		self.setTrailer(.name_token, value);
1492		}
1493
1494		pub fn getVarArgsToken(self: *const FnProto) ?TokenIndex {
1495		return self.getTrailer(.var_args_token);
1496		}
1497
1498		pub fn setVarArgsToken(self: *FnProto, value: TokenIndex) void {
1499		self.setTrailer(.var_args_token, value);
1500		}
1501
1502		pub fn getExternExportInlineToken(self: *const FnProto) ?TokenIndex {
1503		return self.getTrailer(.extern_export_inline_token);
1504		}
1505
1506		pub fn setExternExportInlineToken(self: *FnProto, value: TokenIndex) void {
1507		self.setTrailer(.extern_export_inline_token, value);
1508		}
1509
1510		pub fn getIsExternPrototype(self: *const FnProto) ?void {
1511		return self.getTrailer(.is_extern_prototype);
1512		}
1513
1514		pub fn setIsExternPrototype(self: *FnProto, value: void) void {
1515		self.setTrailer(.is_extern_prototype, value);
1516		}
1517
1518		pub fn getIsAsync(self: *const FnProto) ?void {
1519		return self.getTrailer(.is_async);
1520		}
1521
1522		pub fn setIsAsync(self: *FnProto, value: void) void {
1523		self.setTrailer(.is_async, value);
1524		}
1525
1526		fn getTrailer(self: *const FnProto, comptime field: TrailerFlags.FieldEnum) ?TrailerFlags.Field(field) {
1527		const trailers_start = @alignCast(
1528		@alignOf(ParamDecl),
1529		@ptrCast([]const u8, self) + @sizeOf(FnProto) + @sizeOf(ParamDecl) self.params_len,
1530		);
1531		return self.trailer_flags.get(trailers_start, field);
1532		}
1533
1534		fn setTrailer(self: *FnProto, comptime field: TrailerFlags.FieldEnum, value: TrailerFlags.Field(field)) void {
1535		const trailers_start = @alignCast(
1536		@alignOf(ParamDecl),
1537		@ptrCast([]u8, self) + @sizeOf(FnProto) + @sizeOf(ParamDecl) self.params_len,
1538		);
1539		self.trailer_flags.set(trailers_start, field, value);
1540		}
1541
1542		/// After this the caller must initialize the params list.
1543		pub fn create(allocator: mem.Allocator, required: RequiredFields, trailers: TrailerFlags.InitStruct) !FnProto {
1544		const trailer_flags = TrailerFlags.init(trailers);
1545		const bytes = try allocator.alignedAlloc(u8, @alignOf(FnProto), sizeInBytes(
1546		required.params_len,
1547		trailer_flags,
1548		));
1549		const fn_proto = @ptrCast(*FnProto, bytes.ptr);
1550		fn_proto.* = .{
1551		.trailer_flags = trailer_flags,
1552		.fn_token = required.fn_token,
1553		.params_len = required.params_len,
1554		.return_type = required.return_type,
1555		};
1556		const trailers_start = @alignCast(
1557		@alignOf(ParamDecl),
1558		bytes.ptr + @sizeOf(FnProto) + @sizeOf(ParamDecl) * required.params_len,
1559		);
1560		trailer_flags.setMany(trailers_start, trailers);
1561		return fn_proto;
1562		}
1563
1564		pub fn destroy(self: FnProto, allocator: mem.Allocator) void {
1565		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.params_len, self.trailer_flags)];
1566		allocator.free(bytes);
1567		}
1568
1569		pub fn iterate(self: const FnProto, index: usize) ?Node {
1570		var i = index;
1571
1572		if (self.getLibName()) \|lib_name\| {
1573		if (i < 1) return lib_name;
1574		i -= 1;
1575		}
1576
1577		const params_len: usize = if (self.params_len == 0)
1578		0
1579		else switch (self.paramsConst()[self.params_len - 1].param_type) {
1580		.any_type, .type_expr => self.params_len,
1581		};
1582		if (i < params_len) {
1583		switch (self.paramsConst()[i].param_type) {
1584		.any_type => \|n\| return n,
1585		.type_expr => \|n\| return n,
1586		}
1587		}
1588		i -= params_len;
1589
1590		if (self.getAlignExpr()) \|align_expr\| {
1591		if (i < 1) return align_expr;
1592		i -= 1;
1593		}
1594
1595		if (self.getSectionExpr()) \|section_expr\| {
1596		if (i < 1) return section_expr;
1597		i -= 1;
1598		}
1599
1600		switch (self.return_type) {
1601		.Explicit, .InferErrorSet => \|node\| {
1602		if (i < 1) return node;
1603		i -= 1;
1604		},
1605		.Invalid => {},
1606		}
1607
1608		if (self.getBodyNode()) \|body_node\| {
1609		if (i < 1) return body_node;
1610		i -= 1;
1611		}
1612
1613		return null;
1614		}
1615
1616		pub fn firstToken(self: *const FnProto) TokenIndex {
1617		if (self.getVisibToken()) \|visib_token\| return visib_token;
1618		if (self.getExternExportInlineToken()) \|extern_export_inline_token\| return extern_export_inline_token;
1619		assert(self.getLibName() == null);
1620		return self.fn_token;
1621		}
1622
1623		pub fn lastToken(self: *const FnProto) TokenIndex {
1624		if (self.getBodyNode()) \|body_node\| return body_node.lastToken();
1625		switch (self.return_type) {
1626		.Explicit, .InferErrorSet => \|node\| return node.lastToken(),
1627		.Invalid => \|tok\| return tok,
1628		}
1629		}
1630
1631		pub fn params(self: *FnProto) []ParamDecl {
1632		const params_start = @ptrCast([*]u8, self) + @sizeOf(FnProto);
1633		return @ptrCast([*]ParamDecl, params_start)[0..self.params_len];
1634		}
1635
1636		pub fn paramsConst(self: *const FnProto) []const ParamDecl {
1637		const params_start = @ptrCast([*]const u8, self) + @sizeOf(FnProto);
1638		return @ptrCast([*]const ParamDecl, params_start)[0..self.params_len];
1639		}
1640
1641		fn sizeInBytes(params_len: NodeIndex, trailer_flags: TrailerFlags) usize {
1642		return @sizeOf(FnProto) + @sizeOf(ParamDecl) * @as(usize, params_len) + trailer_flags.sizeInBytes();
1643		}
1644		};
1645
1646		pub const AnyFrameType = struct {
1647		base: Node = Node{ .tag = .AnyFrameType },
1648		anyframe_token: TokenIndex,
1649		result: ?Result,
1650
1651		pub const Result = struct {
1652		arrow_token: TokenIndex,
1653		return_type: *Node,
1654		};
1655
1656		pub fn iterate(self: const AnyFrameType, index: usize) ?Node {
1657		var i = index;
1658
1659		if (self.result) \|result\| {
1660		if (i < 1) return result.return_type;
1661		i -= 1;
1662		}
1663
1664		return null;
1665		}
1666
1667		pub fn firstToken(self: *const AnyFrameType) TokenIndex {
1668		return self.anyframe_token;
1669		}
1670
1671		pub fn lastToken(self: *const AnyFrameType) TokenIndex {
1672		if (self.result) \|result\| return result.return_type.lastToken();
1673		return self.anyframe_token;
1674		}
1675		};
1676
1677		/// The statements of the block follow Block directly in memory.
1678		pub const Block = struct {
1679		base: Node = Node{ .tag = .Block },
1680		statements_len: NodeIndex,
1681		lbrace: TokenIndex,
1682		rbrace: TokenIndex,
1683
1684		/// After this the caller must initialize the statements list.
1685		pub fn alloc(allocator: mem.Allocator, statements_len: NodeIndex) !Block {
1686		const bytes = try allocator.alignedAlloc(u8, @alignOf(Block), sizeInBytes(statements_len));
1687		return @ptrCast(*Block, bytes.ptr);
1688		}
1689
1690		pub fn free(self: Block, allocator: mem.Allocator) void {
1691		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.statements_len)];
1692		allocator.free(bytes);
1693		}
1694
1695		pub fn iterate(self: const Block, index: usize) ?Node {
1696		var i = index;
1697
1698		if (i < self.statements_len) return self.statementsConst()[i];
1699		i -= self.statements_len;
1700
1701		return null;
1702		}
1703
1704		pub fn firstToken(self: *const Block) TokenIndex {
1705		return self.lbrace;
1706		}
1707
1708		pub fn lastToken(self: *const Block) TokenIndex {
1709		return self.rbrace;
1710		}
1711
1712		pub fn statements(self: Block) []Node {
1713		const decls_start = @ptrCast([*]u8, self) + @sizeOf(Block);
1714		return @ptrCast([]Node, decls_start)[0..self.statements_len];
1715		}
1716
1717		pub fn statementsConst(self: const Block) []const Node {
1718		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(Block);
1719		return @ptrCast([]const Node, decls_start)[0..self.statements_len];
1720		}
1721
1722		fn sizeInBytes(statements_len: NodeIndex) usize {
1723		return @sizeOf(Block) + @sizeOf(Node) @as(usize, statements_len);
1724		}
1725		};
1726
1727		/// The statements of the block follow LabeledBlock directly in memory.
1728		pub const LabeledBlock = struct {
1729		base: Node = Node{ .tag = .LabeledBlock },
1730		statements_len: NodeIndex,
1731		lbrace: TokenIndex,
1732		rbrace: TokenIndex,
1733		label: TokenIndex,
1734
1735		/// After this the caller must initialize the statements list.
1736		pub fn alloc(allocator: mem.Allocator, statements_len: NodeIndex) !LabeledBlock {
1737		const bytes = try allocator.alignedAlloc(u8, @alignOf(LabeledBlock), sizeInBytes(statements_len));
1738		return @ptrCast(*LabeledBlock, bytes.ptr);
1739		}
1740
1741		pub fn free(self: LabeledBlock, allocator: mem.Allocator) void {
1742		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.statements_len)];
1743		allocator.free(bytes);
1744		}
1745
1746		pub fn iterate(self: const LabeledBlock, index: usize) ?Node {
1747		var i = index;
1748
1749		if (i < self.statements_len) return self.statementsConst()[i];
1750		i -= self.statements_len;
1751
1752		return null;
1753		}
1754
1755		pub fn firstToken(self: *const LabeledBlock) TokenIndex {
1756		return self.label;
1757		}
1758
1759		pub fn lastToken(self: *const LabeledBlock) TokenIndex {
1760		return self.rbrace;
1761		}
1762
1763		pub fn statements(self: LabeledBlock) []Node {
1764		const decls_start = @ptrCast([*]u8, self) + @sizeOf(LabeledBlock);
1765		return @ptrCast([]Node, decls_start)[0..self.statements_len];
1766		}
1767
1768		pub fn statementsConst(self: const LabeledBlock) []const Node {
1769		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(LabeledBlock);
1770		return @ptrCast([]const Node, decls_start)[0..self.statements_len];
1771		}
1772
1773		fn sizeInBytes(statements_len: NodeIndex) usize {
1774		return @sizeOf(LabeledBlock) + @sizeOf(Node) @as(usize, statements_len);
1775		}
1776		};
1777
1778		pub const Defer = struct {
1779		base: Node = Node{ .tag = .Defer },
1780		defer_token: TokenIndex,
1781		payload: ?*Node,
1782		expr: *Node,
1783
1784		pub fn iterate(self: const Defer, index: usize) ?Node {
1785		var i = index;
1786
1787		if (i < 1) return self.expr;
1788		i -= 1;
1789
1790		return null;
1791		}
1792
1793		pub fn firstToken(self: *const Defer) TokenIndex {
1794		return self.defer_token;
1795		}
1796
1797		pub fn lastToken(self: *const Defer) TokenIndex {
1798		return self.expr.lastToken();
1799		}
1800		};
1801
1802		pub const Comptime = struct {
1803		base: Node = Node{ .tag = .Comptime },
1804		doc_comments: ?*DocComment,
1805		comptime_token: TokenIndex,
1806		expr: *Node,
1807
1808		pub fn iterate(self: const Comptime, index: usize) ?Node {
1809		var i = index;
1810
1811		if (i < 1) return self.expr;
1812		i -= 1;
1813
1814		return null;
1815		}
1816
1817		pub fn firstToken(self: *const Comptime) TokenIndex {
1818		return self.comptime_token;
1819		}
1820
1821		pub fn lastToken(self: *const Comptime) TokenIndex {
1822		return self.expr.lastToken();
1823		}
1824		};
1825
1826		pub const Nosuspend = struct {
1827		base: Node = Node{ .tag = .Nosuspend },
1828		nosuspend_token: TokenIndex,
1829		expr: *Node,
1830
1831		pub fn iterate(self: const Nosuspend, index: usize) ?Node {
1832		var i = index;
1833
1834		if (i < 1) return self.expr;
1835		i -= 1;
1836
1837		return null;
1838		}
1839
1840		pub fn firstToken(self: *const Nosuspend) TokenIndex {
1841		return self.nosuspend_token;
1842		}
1843
1844		pub fn lastToken(self: *const Nosuspend) TokenIndex {
1845		return self.expr.lastToken();
1846		}
1847		};
1848
1849		pub const Payload = struct {
1850		base: Node = Node{ .tag = .Payload },
1851		lpipe: TokenIndex,
1852		error_symbol: *Node,
1853		rpipe: TokenIndex,
1854
1855		pub fn iterate(self: const Payload, index: usize) ?Node {
1856		var i = index;
1857
1858		if (i < 1) return self.error_symbol;
1859		i -= 1;
1860
1861		return null;
1862		}
1863
1864		pub fn firstToken(self: *const Payload) TokenIndex {
1865		return self.lpipe;
1866		}
1867
1868		pub fn lastToken(self: *const Payload) TokenIndex {
1869		return self.rpipe;
1870		}
1871		};
1872
1873		pub const PointerPayload = struct {
1874		base: Node = Node{ .tag = .PointerPayload },
1875		lpipe: TokenIndex,
1876		ptr_token: ?TokenIndex,
1877		value_symbol: *Node,
1878		rpipe: TokenIndex,
1879
1880		pub fn iterate(self: const PointerPayload, index: usize) ?Node {
1881		var i = index;
1882
1883		if (i < 1) return self.value_symbol;
1884		i -= 1;
1885
1886		return null;
1887		}
1888
1889		pub fn firstToken(self: *const PointerPayload) TokenIndex {
1890		return self.lpipe;
1891		}
1892
1893		pub fn lastToken(self: *const PointerPayload) TokenIndex {
1894		return self.rpipe;
1895		}
1896		};
1897
1898		pub const PointerIndexPayload = struct {
1899		base: Node = Node{ .tag = .PointerIndexPayload },
1900		lpipe: TokenIndex,
1901		ptr_token: ?TokenIndex,
1902		value_symbol: *Node,
1903		index_symbol: ?*Node,
1904		rpipe: TokenIndex,
1905
1906		pub fn iterate(self: const PointerIndexPayload, index: usize) ?Node {
1907		var i = index;
1908
1909		if (i < 1) return self.value_symbol;
1910		i -= 1;
1911
1912		if (self.index_symbol) \|index_symbol\| {
1913		if (i < 1) return index_symbol;
1914		i -= 1;
1915		}
1916
1917		return null;
1918		}
1919
1920		pub fn firstToken(self: *const PointerIndexPayload) TokenIndex {
1921		return self.lpipe;
1922		}
1923
1924		pub fn lastToken(self: *const PointerIndexPayload) TokenIndex {
1925		return self.rpipe;
1926		}
1927		};
1928
1929		pub const Else = struct {
1930		base: Node = Node{ .tag = .Else },
1931		else_token: TokenIndex,
1932		payload: ?*Node,
1933		body: *Node,
1934
1935		pub fn iterate(self: const Else, index: usize) ?Node {
1936		var i = index;
1937
1938		if (self.payload) \|payload\| {
1939		if (i < 1) return payload;
1940		i -= 1;
1941		}
1942
1943		if (i < 1) return self.body;
1944		i -= 1;
1945
1946		return null;
1947		}
1948
1949		pub fn firstToken(self: *const Else) TokenIndex {
1950		return self.else_token;
1951		}
1952
1953		pub fn lastToken(self: *const Else) TokenIndex {
1954		return self.body.lastToken();
1955		}
1956		};
1957
1958		/// The cases node pointers are found in memory after Switch.
1959		/// They must be SwitchCase or SwitchElse nodes.
1960		pub const Switch = struct {
1961		base: Node = Node{ .tag = .Switch },
1962		switch_token: TokenIndex,
1963		rbrace: TokenIndex,
1964		cases_len: NodeIndex,
1965		expr: *Node,
1966
1967		/// After this the caller must initialize the fields_and_decls list.
1968		pub fn alloc(allocator: mem.Allocator, cases_len: NodeIndex) !Switch {
1969		const bytes = try allocator.alignedAlloc(u8, @alignOf(Switch), sizeInBytes(cases_len));
1970		return @ptrCast(*Switch, bytes.ptr);
1971		}
1972
1973		pub fn free(self: Switch, allocator: mem.Allocator) void {
1974		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.cases_len)];
1975		allocator.free(bytes);
1976		}
1977
1978		pub fn iterate(self: const Switch, index: usize) ?Node {
1979		var i = index;
1980
1981		if (i < 1) return self.expr;
1982		i -= 1;
1983
1984		if (i < self.cases_len) return self.casesConst()[i];
1985		i -= self.cases_len;
1986
1987		return null;
1988		}
1989
1990		pub fn firstToken(self: *const Switch) TokenIndex {
1991		return self.switch_token;
1992		}
1993
1994		pub fn lastToken(self: *const Switch) TokenIndex {
1995		return self.rbrace;
1996		}
1997
1998		pub fn cases(self: Switch) []Node {
1999		const decls_start = @ptrCast([*]u8, self) + @sizeOf(Switch);
2000		return @ptrCast([]Node, decls_start)[0..self.cases_len];
2001		}
2002
2003		pub fn casesConst(self: const Switch) []const Node {
2004		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(Switch);
2005		return @ptrCast([]const Node, decls_start)[0..self.cases_len];
2006		}
2007
2008		fn sizeInBytes(cases_len: NodeIndex) usize {
2009		return @sizeOf(Switch) + @sizeOf(Node) @as(usize, cases_len);
2010		}
2011		};
2012
2013		/// Items sub-nodes appear in memory directly following SwitchCase.
2014		pub const SwitchCase = struct {
2015		base: Node = Node{ .tag = .SwitchCase },
2016		arrow_token: TokenIndex,
2017		payload: ?*Node,
2018		expr: *Node,
2019		items_len: NodeIndex,
2020
2021		/// After this the caller must initialize the fields_and_decls list.
2022		pub fn alloc(allocator: mem.Allocator, items_len: NodeIndex) !SwitchCase {
2023		const bytes = try allocator.alignedAlloc(u8, @alignOf(SwitchCase), sizeInBytes(items_len));
2024		return @ptrCast(*SwitchCase, bytes.ptr);
2025		}
2026
2027		pub fn free(self: SwitchCase, allocator: mem.Allocator) void {
2028		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.items_len)];
2029		allocator.free(bytes);
2030		}
2031
2032		pub fn iterate(self: const SwitchCase, index: usize) ?Node {
2033		var i = index;
2034
2035		if (i < self.items_len) return self.itemsConst()[i];
2036		i -= self.items_len;
2037
2038		if (self.payload) \|payload\| {
2039		if (i < 1) return payload;
2040		i -= 1;
2041		}
2042
2043		if (i < 1) return self.expr;
2044		i -= 1;
2045
2046		return null;
2047		}
2048
2049		pub fn firstToken(self: *const SwitchCase) TokenIndex {
2050		return self.itemsConst()[0].firstToken();
2051		}
2052
2053		pub fn lastToken(self: *const SwitchCase) TokenIndex {
2054		return self.expr.lastToken();
2055		}
2056
2057		pub fn items(self: SwitchCase) []Node {
2058		const decls_start = @ptrCast([*]u8, self) + @sizeOf(SwitchCase);
2059		return @ptrCast([]Node, decls_start)[0..self.items_len];
2060		}
2061
2062		pub fn itemsConst(self: const SwitchCase) []const Node {
2063		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(SwitchCase);
2064		return @ptrCast([]const Node, decls_start)[0..self.items_len];
2065		}
2066
2067		fn sizeInBytes(items_len: NodeIndex) usize {
2068		return @sizeOf(SwitchCase) + @sizeOf(Node) @as(usize, items_len);
2069		}
2070		};
2071
2072		pub const SwitchElse = struct {
2073		base: Node = Node{ .tag = .SwitchElse },
2074		token: TokenIndex,
2075
2076		pub fn iterate(self: const SwitchElse, index: usize) ?Node {
2077		return null;
2078		}
2079
2080		pub fn firstToken(self: *const SwitchElse) TokenIndex {
2081		return self.token;
2082		}
2083
2084		pub fn lastToken(self: *const SwitchElse) TokenIndex {
2085		return self.token;
2086		}
2087		};
2088
2089		pub const While = struct {
2090		base: Node = Node{ .tag = .While },
2091		label: ?TokenIndex,
2092		inline_token: ?TokenIndex,
2093		while_token: TokenIndex,
2094		condition: *Node,
2095		payload: ?*Node,
2096		continue_expr: ?*Node,
2097		body: *Node,
2098		@"else": ?*Else,
2099
2100		pub fn iterate(self: const While, index: usize) ?Node {
2101		var i = index;
2102
2103		if (i < 1) return self.condition;
2104		i -= 1;
2105
2106		if (self.payload) \|payload\| {
2107		if (i < 1) return payload;
2108		i -= 1;
2109		}
2110
2111		if (self.continue_expr) \|continue_expr\| {
2112		if (i < 1) return continue_expr;
2113		i -= 1;
2114		}
2115
2116		if (i < 1) return self.body;
2117		i -= 1;
2118
2119		if (self.@"else") \|@"else"\| {
2120		if (i < 1) return &@"else".base;
2121		i -= 1;
2122		}
2123
2124		return null;
2125		}
2126
2127		pub fn firstToken(self: *const While) TokenIndex {
2128		if (self.label) \|label\| {
2129		return label;
2130		}
2131
2132		if (self.inline_token) \|inline_token\| {
2133		return inline_token;
2134		}
2135
2136		return self.while_token;
2137		}
2138
2139		pub fn lastToken(self: *const While) TokenIndex {
2140		if (self.@"else") \|@"else"\| {
2141		return @"else".body.lastToken();
2142		}
2143
2144		return self.body.lastToken();
2145		}
2146		};
2147
2148		pub const For = struct {
2149		base: Node = Node{ .tag = .For },
2150		label: ?TokenIndex,
2151		inline_token: ?TokenIndex,
2152		for_token: TokenIndex,
2153		array_expr: *Node,
2154		payload: *Node,
2155		body: *Node,
2156		@"else": ?*Else,
2157
2158		pub fn iterate(self: const For, index: usize) ?Node {
2159		var i = index;
2160
2161		if (i < 1) return self.array_expr;
2162		i -= 1;
2163
2164		if (i < 1) return self.payload;
2165		i -= 1;
2166
2167		if (i < 1) return self.body;
2168		i -= 1;
2169
2170		if (self.@"else") \|@"else"\| {
2171		if (i < 1) return &@"else".base;
2172		i -= 1;
2173		}
2174
2175		return null;
2176		}
2177
2178		pub fn firstToken(self: *const For) TokenIndex {
2179		if (self.label) \|label\| {
2180		return label;
2181		}
2182
2183		if (self.inline_token) \|inline_token\| {
2184		return inline_token;
2185		}
2186
2187		return self.for_token;
2188		}
2189
2190		pub fn lastToken(self: *const For) TokenIndex {
2191		if (self.@"else") \|@"else"\| {
2192		return @"else".body.lastToken();
2193		}
2194
2195		return self.body.lastToken();
2196		}
2197		};
2198
2199		pub const If = struct {
2200		base: Node = Node{ .tag = .If },
2201		if_token: TokenIndex,
2202		condition: *Node,
2203		payload: ?*Node,
2204		body: *Node,
2205		@"else": ?*Else,
2206
2207		pub fn iterate(self: const If, index: usize) ?Node {
2208		var i = index;
2209
2210		if (i < 1) return self.condition;
2211		i -= 1;
2212
2213		if (self.payload) \|payload\| {
2214		if (i < 1) return payload;
2215		i -= 1;
2216		}
2217
2218		if (i < 1) return self.body;
2219		i -= 1;
2220
2221		if (self.@"else") \|@"else"\| {
2222		if (i < 1) return &@"else".base;
2223		i -= 1;
2224		}
2225
2226		return null;
2227		}
2228
2229		pub fn firstToken(self: *const If) TokenIndex {
2230		return self.if_token;
2231		}
2232
2233		pub fn lastToken(self: *const If) TokenIndex {
2234		if (self.@"else") \|@"else"\| {
2235		return @"else".body.lastToken();
2236		}
2237
2238		return self.body.lastToken();
2239		}
2240		};
2241
2242		pub const Catch = struct {
2243		base: Node = Node{ .tag = .Catch },
2244		op_token: TokenIndex,
2245		lhs: *Node,
2246		rhs: *Node,
2247		payload: ?*Node,
2248
2249		pub fn iterate(self: const Catch, index: usize) ?Node {
2250		var i = index;
2251
2252		if (i < 1) return self.lhs;
2253		i -= 1;
2254
2255		if (self.payload) \|payload\| {
2256		if (i < 1) return payload;
2257		i -= 1;
2258		}
2259
2260		if (i < 1) return self.rhs;
2261		i -= 1;
2262
2263		return null;
2264		}
2265
2266		pub fn firstToken(self: *const Catch) TokenIndex {
2267		return self.lhs.firstToken();
2268		}
2269
2270		pub fn lastToken(self: *const Catch) TokenIndex {
2271		return self.rhs.lastToken();
2272		}
2273		};
2274
2275		pub const SimpleInfixOp = struct {
2276		base: Node,
2277		op_token: TokenIndex,
2278		lhs: *Node,
2279		rhs: *Node,
2280
2281		pub fn iterate(self: const SimpleInfixOp, index: usize) ?Node {
2282		var i = index;
2283
2284		if (i < 1) return self.lhs;
2285		i -= 1;
2286
2287		if (i < 1) return self.rhs;
2288		i -= 1;
2289
2290		return null;
2291		}
2292
2293		pub fn firstToken(self: *const SimpleInfixOp) TokenIndex {
2294		return self.lhs.firstToken();
2295		}
2296
2297		pub fn lastToken(self: *const SimpleInfixOp) TokenIndex {
2298		return self.rhs.lastToken();
2299		}
2300		};
2301
2302		pub const SimplePrefixOp = struct {
2303		base: Node,
2304		op_token: TokenIndex,
2305		rhs: *Node,
2306
2307		const Self = @This();
2308
2309		pub fn iterate(self: const Self, index: usize) ?Node {
2310		if (index == 0) return self.rhs;
2311		return null;
2312		}
2313
2314		pub fn firstToken(self: *const Self) TokenIndex {
2315		return self.op_token;
2316		}
2317
2318		pub fn lastToken(self: *const Self) TokenIndex {
2319		return self.rhs.lastToken();
2320		}
2321		};
2322
2323		pub const ArrayType = struct {
2324		base: Node = Node{ .tag = .ArrayType },
2325		op_token: TokenIndex,
2326		rhs: *Node,
2327		len_expr: *Node,
2328
2329		pub fn iterate(self: const ArrayType, index: usize) ?Node {
2330		var i = index;
2331
2332		if (i < 1) return self.len_expr;
2333		i -= 1;
2334
2335		if (i < 1) return self.rhs;
2336		i -= 1;
2337
2338		return null;
2339		}
2340
2341		pub fn firstToken(self: *const ArrayType) TokenIndex {
2342		return self.op_token;
2343		}
2344
2345		pub fn lastToken(self: *const ArrayType) TokenIndex {
2346		return self.rhs.lastToken();
2347		}
2348		};
2349
2350		pub const ArrayTypeSentinel = struct {
2351		base: Node = Node{ .tag = .ArrayTypeSentinel },
2352		op_token: TokenIndex,
2353		rhs: *Node,
2354		len_expr: *Node,
2355		sentinel: *Node,
2356
2357		pub fn iterate(self: const ArrayTypeSentinel, index: usize) ?Node {
2358		var i = index;
2359
2360		if (i < 1) return self.len_expr;
2361		i -= 1;
2362
2363		if (i < 1) return self.sentinel;
2364		i -= 1;
2365
2366		if (i < 1) return self.rhs;
2367		i -= 1;
2368
2369		return null;
2370		}
2371
2372		pub fn firstToken(self: *const ArrayTypeSentinel) TokenIndex {
2373		return self.op_token;
2374		}
2375
2376		pub fn lastToken(self: *const ArrayTypeSentinel) TokenIndex {
2377		return self.rhs.lastToken();
2378		}
2379		};
2380
2381		pub const PtrType = struct {
2382		base: Node = Node{ .tag = .PtrType },
2383		op_token: TokenIndex,
2384		rhs: *Node,
2385		/// TODO Add a u8 flags field to Node where it would otherwise be padding, and each bit represents
2386		/// one of these possibly-null things. Then we have them directly follow the PtrType in memory.
2387		ptr_info: PtrInfo = .{},
2388
2389		pub fn iterate(self: const PtrType, index: usize) ?Node {
2390		var i = index;
2391
2392		if (self.ptr_info.sentinel) \|sentinel\| {
2393		if (i < 1) return sentinel;
2394		i -= 1;
2395		}
2396
2397		if (self.ptr_info.align_info) \|align_info\| {
2398		if (i < 1) return align_info.node;
2399		i -= 1;
2400		}
2401
2402		if (i < 1) return self.rhs;
2403		i -= 1;
2404
2405		return null;
2406		}
2407
2408		pub fn firstToken(self: *const PtrType) TokenIndex {
2409		return self.op_token;
2410		}
2411
2412		pub fn lastToken(self: *const PtrType) TokenIndex {
2413		return self.rhs.lastToken();
2414		}
2415		};
2416
2417		pub const SliceType = struct {
2418		base: Node = Node{ .tag = .SliceType },
2419		op_token: TokenIndex,
2420		rhs: *Node,
2421		/// TODO Add a u8 flags field to Node where it would otherwise be padding, and each bit represents
2422		/// one of these possibly-null things. Then we have them directly follow the SliceType in memory.
2423		ptr_info: PtrInfo = .{},
2424
2425		pub fn iterate(self: const SliceType, index: usize) ?Node {
2426		var i = index;
2427
2428		if (self.ptr_info.sentinel) \|sentinel\| {
2429		if (i < 1) return sentinel;
2430		i -= 1;
2431		}
2432
2433		if (self.ptr_info.align_info) \|align_info\| {
2434		if (i < 1) return align_info.node;
2435		i -= 1;
2436		}
2437
2438		if (i < 1) return self.rhs;
2439		i -= 1;
2440
2441		return null;
2442		}
2443
2444		pub fn firstToken(self: *const SliceType) TokenIndex {
2445		return self.op_token;
2446		}
2447
2448		pub fn lastToken(self: *const SliceType) TokenIndex {
2449		return self.rhs.lastToken();
2450		}
2451		};
2452
2453		pub const FieldInitializer = struct {
2454		base: Node = Node{ .tag = .FieldInitializer },
2455		period_token: TokenIndex,
2456		name_token: TokenIndex,
2457		expr: *Node,
2458
2459		pub fn iterate(self: const FieldInitializer, index: usize) ?Node {
2460		var i = index;
2461
2462		if (i < 1) return self.expr;
2463		i -= 1;
2464
2465		return null;
2466		}
2467
2468		pub fn firstToken(self: *const FieldInitializer) TokenIndex {
2469		return self.period_token;
2470		}
2471
2472		pub fn lastToken(self: *const FieldInitializer) TokenIndex {
2473		return self.expr.lastToken();
2474		}
2475		};
2476
2477		/// Elements occur directly in memory after ArrayInitializer.
2478		pub const ArrayInitializer = struct {
2479		base: Node = Node{ .tag = .ArrayInitializer },
2480		rtoken: TokenIndex,
2481		list_len: NodeIndex,
2482		lhs: *Node,
2483
2484		/// After this the caller must initialize the fields_and_decls list.
2485		pub fn alloc(allocator: mem.Allocator, list_len: NodeIndex) !ArrayInitializer {
2486		const bytes = try allocator.alignedAlloc(u8, @alignOf(ArrayInitializer), sizeInBytes(list_len));
2487		return @ptrCast(*ArrayInitializer, bytes.ptr);
2488		}
2489
2490		pub fn free(self: ArrayInitializer, allocator: mem.Allocator) void {
2491		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.list_len)];
2492		allocator.free(bytes);
2493		}
2494
2495		pub fn iterate(self: const ArrayInitializer, index: usize) ?Node {
2496		var i = index;
2497
2498		if (i < 1) return self.lhs;
2499		i -= 1;
2500
2501		if (i < self.list_len) return self.listConst()[i];
2502		i -= self.list_len;
2503
2504		return null;
2505		}
2506
2507		pub fn firstToken(self: *const ArrayInitializer) TokenIndex {
2508		return self.lhs.firstToken();
2509		}
2510
2511		pub fn lastToken(self: *const ArrayInitializer) TokenIndex {
2512		return self.rtoken;
2513		}
2514
2515		pub fn list(self: ArrayInitializer) []Node {
2516		const decls_start = @ptrCast([*]u8, self) + @sizeOf(ArrayInitializer);
2517		return @ptrCast([]Node, decls_start)[0..self.list_len];
2518		}
2519
2520		pub fn listConst(self: const ArrayInitializer) []const Node {
2521		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(ArrayInitializer);
2522		return @ptrCast([]const Node, decls_start)[0..self.list_len];
2523		}
2524
2525		fn sizeInBytes(list_len: NodeIndex) usize {
2526		return @sizeOf(ArrayInitializer) + @sizeOf(Node) @as(usize, list_len);
2527		}
2528		};
2529
2530		/// Elements occur directly in memory after ArrayInitializerDot.
2531		pub const ArrayInitializerDot = struct {
2532		base: Node = Node{ .tag = .ArrayInitializerDot },
2533		dot: TokenIndex,
2534		rtoken: TokenIndex,
2535		list_len: NodeIndex,
2536
2537		/// After this the caller must initialize the fields_and_decls list.
2538		pub fn alloc(allocator: mem.Allocator, list_len: NodeIndex) !ArrayInitializerDot {
2539		const bytes = try allocator.alignedAlloc(u8, @alignOf(ArrayInitializerDot), sizeInBytes(list_len));
2540		return @ptrCast(*ArrayInitializerDot, bytes.ptr);
2541		}
2542
2543		pub fn free(self: ArrayInitializerDot, allocator: mem.Allocator) void {
2544		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.list_len)];
2545		allocator.free(bytes);
2546		}
2547
2548		pub fn iterate(self: const ArrayInitializerDot, index: usize) ?Node {
2549		var i = index;
2550
2551		if (i < self.list_len) return self.listConst()[i];
2552		i -= self.list_len;
2553
2554		return null;
2555		}
2556
2557		pub fn firstToken(self: *const ArrayInitializerDot) TokenIndex {
2558		return self.dot;
2559		}
2560
2561		pub fn lastToken(self: *const ArrayInitializerDot) TokenIndex {
2562		return self.rtoken;
2563		}
2564
2565		pub fn list(self: ArrayInitializerDot) []Node {
2566		const decls_start = @ptrCast([*]u8, self) + @sizeOf(ArrayInitializerDot);
2567		return @ptrCast([]Node, decls_start)[0..self.list_len];
2568		}
2569
2570		pub fn listConst(self: const ArrayInitializerDot) []const Node {
2571		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(ArrayInitializerDot);
2572		return @ptrCast([]const Node, decls_start)[0..self.list_len];
2573		}
2574
2575		fn sizeInBytes(list_len: NodeIndex) usize {
2576		return @sizeOf(ArrayInitializerDot) + @sizeOf(Node) @as(usize, list_len);
2577		}
2578		};
2579
2580		/// Elements occur directly in memory after StructInitializer.
2581		pub const StructInitializer = struct {
2582		base: Node = Node{ .tag = .StructInitializer },
2583		rtoken: TokenIndex,
2584		list_len: NodeIndex,
2585		lhs: *Node,
2586
2587		/// After this the caller must initialize the fields_and_decls list.
2588		pub fn alloc(allocator: mem.Allocator, list_len: NodeIndex) !StructInitializer {
2589		const bytes = try allocator.alignedAlloc(u8, @alignOf(StructInitializer), sizeInBytes(list_len));
2590		return @ptrCast(*StructInitializer, bytes.ptr);
2591		}
2592
2593		pub fn free(self: StructInitializer, allocator: mem.Allocator) void {
2594		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.list_len)];
2595		allocator.free(bytes);
2596		}
2597
2598		pub fn iterate(self: const StructInitializer, index: usize) ?Node {
2599		var i = index;
2600
2601		if (i < 1) return self.lhs;
2602		i -= 1;
2603
2604		if (i < self.list_len) return self.listConst()[i];
2605		i -= self.list_len;
2606
2607		return null;
2608		}
2609
2610		pub fn firstToken(self: *const StructInitializer) TokenIndex {
2611		return self.lhs.firstToken();
2612		}
2613
2614		pub fn lastToken(self: *const StructInitializer) TokenIndex {
2615		return self.rtoken;
2616		}
2617
2618		pub fn list(self: StructInitializer) []Node {
2619		const decls_start = @ptrCast([*]u8, self) + @sizeOf(StructInitializer);
2620		return @ptrCast([]Node, decls_start)[0..self.list_len];
2621		}
2622
2623		pub fn listConst(self: const StructInitializer) []const Node {
2624		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(StructInitializer);
2625		return @ptrCast([]const Node, decls_start)[0..self.list_len];
2626		}
2627
2628		fn sizeInBytes(list_len: NodeIndex) usize {
2629		return @sizeOf(StructInitializer) + @sizeOf(Node) @as(usize, list_len);
2630		}
2631		};
2632
2633		/// Elements occur directly in memory after StructInitializerDot.
2634		pub const StructInitializerDot = struct {
2635		base: Node = Node{ .tag = .StructInitializerDot },
2636		dot: TokenIndex,
2637		rtoken: TokenIndex,
2638		list_len: NodeIndex,
2639
2640		/// After this the caller must initialize the fields_and_decls list.
2641		pub fn alloc(allocator: mem.Allocator, list_len: NodeIndex) !StructInitializerDot {
2642		const bytes = try allocator.alignedAlloc(u8, @alignOf(StructInitializerDot), sizeInBytes(list_len));
2643		return @ptrCast(*StructInitializerDot, bytes.ptr);
2644		}
2645
2646		pub fn free(self: StructInitializerDot, allocator: mem.Allocator) void {
2647		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.list_len)];
2648		allocator.free(bytes);
2649		}
2650
2651		pub fn iterate(self: const StructInitializerDot, index: usize) ?Node {
2652		var i = index;
2653
2654		if (i < self.list_len) return self.listConst()[i];
2655		i -= self.list_len;
2656
2657		return null;
2658		}
2659
2660		pub fn firstToken(self: *const StructInitializerDot) TokenIndex {
2661		return self.dot;
2662		}
2663
2664		pub fn lastToken(self: *const StructInitializerDot) TokenIndex {
2665		return self.rtoken;
2666		}
2667
2668		pub fn list(self: StructInitializerDot) []Node {
2669		const decls_start = @ptrCast([*]u8, self) + @sizeOf(StructInitializerDot);
2670		return @ptrCast([]Node, decls_start)[0..self.list_len];
2671		}
2672
2673		pub fn listConst(self: const StructInitializerDot) []const Node {
2674		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(StructInitializerDot);
2675		return @ptrCast([]const Node, decls_start)[0..self.list_len];
2676		}
2677
2678		fn sizeInBytes(list_len: NodeIndex) usize {
2679		return @sizeOf(StructInitializerDot) + @sizeOf(Node) @as(usize, list_len);
2680		}
2681		};
2682
2683		/// Parameter nodes directly follow Call in memory.
2684		pub const Call = struct {
2685		base: Node = Node{ .tag = .Call },
2686		rtoken: TokenIndex,
2687		lhs: *Node,
2688		params_len: NodeIndex,
2689		async_token: ?TokenIndex,
2690
2691		/// After this the caller must initialize the fields_and_decls list.
2692		pub fn alloc(allocator: mem.Allocator, params_len: NodeIndex) !Call {
2693		const bytes = try allocator.alignedAlloc(u8, @alignOf(Call), sizeInBytes(params_len));
2694		return @ptrCast(*Call, bytes.ptr);
2695		}
2696
2697		pub fn free(self: Call, allocator: mem.Allocator) void {
2698		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.params_len)];
2699		allocator.free(bytes);
2700		}
2701
2702		pub fn iterate(self: const Call, index: usize) ?Node {
2703		var i = index;
2704
2705		if (i < 1) return self.lhs;
2706		i -= 1;
2707
2708		if (i < self.params_len) return self.paramsConst()[i];
2709		i -= self.params_len;
2710
2711		return null;
2712		}
2713
2714		pub fn firstToken(self: *const Call) TokenIndex {
2715		if (self.async_token) \|async_token\| return async_token;
2716		return self.lhs.firstToken();
2717		}
2718
2719		pub fn lastToken(self: *const Call) TokenIndex {
2720		return self.rtoken;
2721		}
2722
2723		pub fn params(self: Call) []Node {
2724		const decls_start = @ptrCast([*]u8, self) + @sizeOf(Call);
2725		return @ptrCast([]Node, decls_start)[0..self.params_len];
2726		}
2727
2728		pub fn paramsConst(self: const Call) []const Node {
2729		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(Call);
2730		return @ptrCast([]const Node, decls_start)[0..self.params_len];
2731		}
2732
2733		fn sizeInBytes(params_len: NodeIndex) usize {
2734		return @sizeOf(Call) + @sizeOf(Node) @as(usize, params_len);
2735		}
2736		};
2737
2738		pub const ArrayAccess = struct {
2739		base: Node = Node{ .tag = .ArrayAccess },
2740		rtoken: TokenIndex,
2741		lhs: *Node,
2742		index_expr: *Node,
2743
2744		pub fn iterate(self: const ArrayAccess, index: usize) ?Node {
2745		var i = index;
2746
2747		if (i < 1) return self.lhs;
2748		i -= 1;
2749
2750		if (i < 1) return self.index_expr;
2751		i -= 1;
2752
2753		return null;
2754		}
2755
2756		pub fn firstToken(self: *const ArrayAccess) TokenIndex {
2757		return self.lhs.firstToken();
2758		}
2759
2760		pub fn lastToken(self: *const ArrayAccess) TokenIndex {
2761		return self.rtoken;
2762		}
2763		};
2764
2765		pub const SimpleSuffixOp = struct {
2766		base: Node,
2767		rtoken: TokenIndex,
2768		lhs: *Node,
2769
2770		pub fn iterate(self: const SimpleSuffixOp, index: usize) ?Node {
2771		var i = index;
2772
2773		if (i < 1) return self.lhs;
2774		i -= 1;
2775
2776		return null;
2777		}
2778
2779		pub fn firstToken(self: *const SimpleSuffixOp) TokenIndex {
2780		return self.lhs.firstToken();
2781		}
2782
2783		pub fn lastToken(self: *const SimpleSuffixOp) TokenIndex {
2784		return self.rtoken;
2785		}
2786		};
2787
2788		pub const Slice = struct {
2789		base: Node = Node{ .tag = .Slice },
2790		rtoken: TokenIndex,
2791		lhs: *Node,
2792		start: *Node,
2793		end: ?*Node,
2794		sentinel: ?*Node,
2795
2796		pub fn iterate(self: const Slice, index: usize) ?Node {
2797		var i = index;
2798
2799		if (i < 1) return self.lhs;
2800		i -= 1;
2801
2802		if (i < 1) return self.start;
2803		i -= 1;
2804
2805		if (self.end) \|end\| {
2806		if (i < 1) return end;
2807		i -= 1;
2808		}
2809		if (self.sentinel) \|sentinel\| {
2810		if (i < 1) return sentinel;
2811		i -= 1;
2812		}
2813
2814		return null;
2815		}
2816
2817		pub fn firstToken(self: *const Slice) TokenIndex {
2818		return self.lhs.firstToken();
2819		}
2820
2821		pub fn lastToken(self: *const Slice) TokenIndex {
2822		return self.rtoken;
2823		}
2824		};
2825
2826		pub const GroupedExpression = struct {
2827		base: Node = Node{ .tag = .GroupedExpression },
2828		lparen: TokenIndex,
2829		expr: *Node,
2830		rparen: TokenIndex,
2831
2832		pub fn iterate(self: const GroupedExpression, index: usize) ?Node {
2833		var i = index;
2834
2835		if (i < 1) return self.expr;
2836		i -= 1;
2837
2838		return null;
2839		}
2840
2841		pub fn firstToken(self: *const GroupedExpression) TokenIndex {
2842		return self.lparen;
2843		}
2844
2845		pub fn lastToken(self: *const GroupedExpression) TokenIndex {
2846		return self.rparen;
2847		}
2848		};
2849
2850		/// Trailed in memory by possibly many things, with each optional thing
2851		/// determined by a bit in `trailer_flags`.
2852		/// Can be: return, break, continue
2853		pub const ControlFlowExpression = struct {
2854		base: Node,
2855		trailer_flags: TrailerFlags,
2856		ltoken: TokenIndex,
2857
2858		pub const TrailerFlags = std.meta.TrailerFlags(struct {
2859		rhs: *Node,
2860		label: TokenIndex,
2861		});
2862
2863		pub const RequiredFields = struct {
2864		tag: Tag,
2865		ltoken: TokenIndex,
2866		};
2867
2868		pub fn getRHS(self: const ControlFlowExpression) ?Node {
2869		return self.getTrailer(.rhs);
2870		}
2871
2872		pub fn setRHS(self: ControlFlowExpression, value: Node) void {
2873		self.setTrailer(.rhs, value);
2874		}
2875
2876		pub fn getLabel(self: *const ControlFlowExpression) ?TokenIndex {
2877		return self.getTrailer(.label);
2878		}
2879
2880		pub fn setLabel(self: *ControlFlowExpression, value: TokenIndex) void {
2881		self.setTrailer(.label, value);
2882		}
2883
2884		fn getTrailer(self: *const ControlFlowExpression, comptime field: TrailerFlags.FieldEnum) ?TrailerFlags.Field(field) {
2885		const trailers_start = @ptrCast([*]const u8, self) + @sizeOf(ControlFlowExpression);
2886		return self.trailer_flags.get(trailers_start, field);
2887		}
2888
2889		fn setTrailer(self: *ControlFlowExpression, comptime field: TrailerFlags.FieldEnum, value: TrailerFlags.Field(field)) void {
2890		const trailers_start = @ptrCast([*]u8, self) + @sizeOf(ControlFlowExpression);
2891		self.trailer_flags.set(trailers_start, field, value);
2892		}
2893
2894		pub fn create(allocator: mem.Allocator, required: RequiredFields, trailers: TrailerFlags.InitStruct) !ControlFlowExpression {
2895		const trailer_flags = TrailerFlags.init(trailers);
2896		const bytes = try allocator.alignedAlloc(u8, @alignOf(ControlFlowExpression), sizeInBytes(trailer_flags));
2897		const ctrl_flow_expr = @ptrCast(*ControlFlowExpression, bytes.ptr);
2898		ctrl_flow_expr.* = .{
2899		.base = .{ .tag = required.tag },
2900		.trailer_flags = trailer_flags,
2901		.ltoken = required.ltoken,
2902		};
2903		const trailers_start = bytes.ptr + @sizeOf(ControlFlowExpression);
2904		trailer_flags.setMany(trailers_start, trailers);
2905		return ctrl_flow_expr;
2906		}
2907
2908		pub fn destroy(self: ControlFlowExpression, allocator: mem.Allocator) void {
2909		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.trailer_flags)];
2910		allocator.free(bytes);
2911		}
2912
2913		pub fn iterate(self: const ControlFlowExpression, index: usize) ?Node {
2914		var i = index;
2915
2916		if (self.getRHS()) \|rhs\| {
2917		if (i < 1) return rhs;
2918		i -= 1;
2919		}
2920
2921		return null;
2922		}
2923
2924		pub fn firstToken(self: *const ControlFlowExpression) TokenIndex {
2925		return self.ltoken;
2926		}
2927
2928		pub fn lastToken(self: *const ControlFlowExpression) TokenIndex {
2929		if (self.getRHS()) \|rhs\| {
2930		return rhs.lastToken();
2931		}
2932
2933		if (self.getLabel()) \|label\| {
2934		return label;
2935		}
2936
2937		return self.ltoken;
2938		}
2939
2940		fn sizeInBytes(trailer_flags: TrailerFlags) usize {
2941		return @sizeOf(ControlFlowExpression) + trailer_flags.sizeInBytes();
2942		}
2943		};
2944
2945		pub const Suspend = struct {
2946		base: Node = Node{ .tag = .Suspend },
2947		suspend_token: TokenIndex,
2948		body: ?*Node,
2949
2950		pub fn iterate(self: const Suspend, index: usize) ?Node {
2951		var i = index;
2952
2953		if (self.body) \|body\| {
2954		if (i < 1) return body;
2955		i -= 1;
2956		}
2957
2958		return null;
2959		}
2960
2961		pub fn firstToken(self: *const Suspend) TokenIndex {
2962		return self.suspend_token;
2963		}
2964
2965		pub fn lastToken(self: *const Suspend) TokenIndex {
2966		if (self.body) \|body\| {
2967		return body.lastToken();
2968		}
2969
2970		return self.suspend_token;
2971		}
2972		};
2973
2974		pub const EnumLiteral = struct {
2975		base: Node = Node{ .tag = .EnumLiteral },
2976		dot: TokenIndex,
2977		name: TokenIndex,
2978
2979		pub fn iterate(self: const EnumLiteral, index: usize) ?Node {
2980		return null;
2981		}
2982
2983		pub fn firstToken(self: *const EnumLiteral) TokenIndex {
2984		return self.dot;
2985		}
2986
2987		pub fn lastToken(self: *const EnumLiteral) TokenIndex {
2988		return self.name;
2989		}
2990		};
2991
2992		/// Parameters are in memory following BuiltinCall.
2993		pub const BuiltinCall = struct {
2994		base: Node = Node{ .tag = .BuiltinCall },
2995		params_len: NodeIndex,
2996		builtin_token: TokenIndex,
2997		rparen_token: TokenIndex,
2998
2999		/// After this the caller must initialize the fields_and_decls list.
3000		pub fn alloc(allocator: mem.Allocator, params_len: NodeIndex) !BuiltinCall {
3001		const bytes = try allocator.alignedAlloc(u8, @alignOf(BuiltinCall), sizeInBytes(params_len));
3002		return @ptrCast(*BuiltinCall, bytes.ptr);
3003		}
3004
3005		pub fn free(self: BuiltinCall, allocator: mem.Allocator) void {
3006		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.params_len)];
3007		allocator.free(bytes);
3008		}
3009
3010		pub fn iterate(self: const BuiltinCall, index: usize) ?Node {
3011		var i = index;
3012
3013		if (i < self.params_len) return self.paramsConst()[i];
3014		i -= self.params_len;
3015
3016		return null;
3017		}
3018
3019		pub fn firstToken(self: *const BuiltinCall) TokenIndex {
3020		return self.builtin_token;
3021		}
3022
3023		pub fn lastToken(self: *const BuiltinCall) TokenIndex {
3024		return self.rparen_token;
3025		}
3026
3027		pub fn params(self: BuiltinCall) []Node {
3028		const decls_start = @ptrCast([*]u8, self) + @sizeOf(BuiltinCall);
3029		return @ptrCast([]Node, decls_start)[0..self.params_len];
3030		}
3031
3032		pub fn paramsConst(self: const BuiltinCall) []const Node {
3033		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(BuiltinCall);
3034		return @ptrCast([]const Node, decls_start)[0..self.params_len];
3035		}
3036
3037		fn sizeInBytes(params_len: NodeIndex) usize {
3038		return @sizeOf(BuiltinCall) + @sizeOf(Node) @as(usize, params_len);
3039		}
3040		};
3041
3042		/// The string literal tokens appear directly in memory after MultilineStringLiteral.
3043		pub const MultilineStringLiteral = struct {
3044		base: Node = Node{ .tag = .MultilineStringLiteral },
3045		lines_len: TokenIndex,
3046
3047		/// After this the caller must initialize the lines list.
3048		pub fn alloc(allocator: mem.Allocator, lines_len: NodeIndex) !MultilineStringLiteral {
3049		const bytes = try allocator.alignedAlloc(u8, @alignOf(MultilineStringLiteral), sizeInBytes(lines_len));
3050		return @ptrCast(*MultilineStringLiteral, bytes.ptr);
3051		}
3052
3053		pub fn free(self: MultilineStringLiteral, allocator: mem.Allocator) void {
3054		const bytes = @ptrCast([*]u8, self)[0..sizeInBytes(self.lines_len)];
3055		allocator.free(bytes);
3056		}
3057
3058		pub fn iterate(self: const MultilineStringLiteral, index: usize) ?Node {
3059		return null;
3060		}
3061
3062		pub fn firstToken(self: *const MultilineStringLiteral) TokenIndex {
3063		return self.linesConst()[0];
3064		}
3065
3066		pub fn lastToken(self: *const MultilineStringLiteral) TokenIndex {
3067		return self.linesConst()[self.lines_len - 1];
3068		}
3069
3070		pub fn lines(self: *MultilineStringLiteral) []TokenIndex {
3071		const decls_start = @ptrCast([*]u8, self) + @sizeOf(MultilineStringLiteral);
3072		return @ptrCast([*]TokenIndex, decls_start)[0..self.lines_len];
3073		}
3074
3075		pub fn linesConst(self: *const MultilineStringLiteral) []const TokenIndex {
3076		const decls_start = @ptrCast([*]const u8, self) + @sizeOf(MultilineStringLiteral);
3077		return @ptrCast([*]const TokenIndex, decls_start)[0..self.lines_len];
3078		}
3079
3080		fn sizeInBytes(lines_len: NodeIndex) usize {
3081		return @sizeOf(MultilineStringLiteral) + @sizeOf(TokenIndex) * @as(usize, lines_len);
3082		}
3083		};
3084
3085		pub const Asm = struct {
3086		base: Node = Node{ .tag = .Asm },
3087		asm_token: TokenIndex,
3088		rparen: TokenIndex,
3089		volatile_token: ?TokenIndex,
3090		template: *Node,
3091		outputs: []Output,
3092		inputs: []Input,
3093		/// A clobber node must be a StringLiteral or MultilineStringLiteral.
3094		clobbers: []*Node,
3095
3096		pub const Output = struct {
3097		lbracket: TokenIndex,
3098		symbolic_name: *Node,
3099		constraint: *Node,
3100		kind: Kind,
3101		rparen: TokenIndex,
3102
3103		pub const Kind = union(enum) {
3104		Variable: *OneToken,
3105		Return: *Node,
3106		};
3107
3108		pub fn iterate(self: const Output, index: usize) ?Node {
3109		var i = index;
3110
3111		if (i < 1) return self.symbolic_name;
3112		i -= 1;
3113
3114		if (i < 1) return self.constraint;
3115		i -= 1;
3116
3117		switch (self.kind) {
3118		.Variable => \|variable_name\| {
3119		if (i < 1) return &variable_name.base;
3120		i -= 1;
3121		},
3122		.Return => \|return_type\| {
3123		if (i < 1) return return_type;
3124		i -= 1;
3125		},
3126		}
3127
3128		return null;
3129		}
3130
3131		pub fn firstToken(self: *const Output) TokenIndex {
3132		return self.lbracket;
3133		}
3134
3135		pub fn lastToken(self: *const Output) TokenIndex {
3136		return self.rparen;
3137		}
3138		};
3139
3140		pub const Input = struct {
3141		lbracket: TokenIndex,
3142		symbolic_name: *Node,
3143		constraint: *Node,
3144		expr: *Node,
3145		rparen: TokenIndex,
3146
3147		pub fn iterate(self: const Input, index: usize) ?Node {
3148		var i = index;
3149
3150		if (i < 1) return self.symbolic_name;
3151		i -= 1;
3152
3153		if (i < 1) return self.constraint;
3154		i -= 1;
3155
3156		if (i < 1) return self.expr;
3157		i -= 1;
3158
3159		return null;
3160		}
3161
3162		pub fn firstToken(self: *const Input) TokenIndex {
3163		return self.lbracket;
3164		}
3165
3166		pub fn lastToken(self: *const Input) TokenIndex {
3167		return self.rparen;
3168		}
3169		};
3170
3171		pub fn iterate(self: const Asm, index: usize) ?Node {
3172		var i = index;
3173
3174		if (i < self.outputs.len * 3) switch (i % 3) {
3175		0 => return self.outputs[i / 3].symbolic_name,
3176		1 => return self.outputs[i / 3].constraint,
3177		2 => switch (self.outputs[i / 3].kind) {
3178		.Variable => \|variable_name\| return &variable_name.base,
3179		.Return => \|return_type\| return return_type,
3180		},
3181		else => unreachable,
3182		};
3183		i -= self.outputs.len * 3;
3184
3185		if (i < self.inputs.len * 3) switch (i % 3) {
3186		0 => return self.inputs[i / 3].symbolic_name,
3187		1 => return self.inputs[i / 3].constraint,
3188		2 => return self.inputs[i / 3].expr,
3189		else => unreachable,
3190		};
3191		i -= self.inputs.len * 3;
3192
3193		return null;
3194		}
3195
3196		pub fn firstToken(self: *const Asm) TokenIndex {
3197		return self.asm_token;
3198		}
3199
3200		pub fn lastToken(self: *const Asm) TokenIndex {
3201		return self.rparen;
3202		}
3203		};
3204
3205		/// TODO remove from the Node base struct
3206		/// TODO actually maybe remove entirely in favor of iterating backward from Node.firstToken()
3207		/// and forwards to find same-line doc comments.
3208		pub const DocComment = struct {
3209		base: Node = Node{ .tag = .DocComment },
3210		/// Points to the first doc comment token. API users are expected to iterate over the
3211		/// tokens array, looking for more doc comments, ignoring line comments, and stopping
3212		/// at the first other token.
3213		first_line: TokenIndex,
3214
3215		pub fn iterate(self: const DocComment, index: usize) ?Node {
3216		return null;
3217		}
3218
3219		pub fn firstToken(self: *const DocComment) TokenIndex {
3220		return self.first_line;
3221		}
3222
3223		/// Returns the first doc comment line. Be careful, this may not be the desired behavior,
3224		/// which would require the tokens array.
3225		pub fn lastToken(self: *const DocComment) TokenIndex {
3226		return self.first_line;
3227		}
3228		};
3229
3230		pub const TestDecl = struct {
3231		base: Node = Node{ .tag = .TestDecl },
3232		doc_comments: ?*DocComment,
3233		test_token: TokenIndex,
3234		name: ?*Node,
3235		body_node: *Node,
3236
3237		pub fn iterate(self: const TestDecl, index: usize) ?Node {
3238		var i = index;
3239
3240		if (i < 1) return self.body_node;
3241		i -= 1;
3242
3243		return null;
3244		}
3245
3246		pub fn firstToken(self: *const TestDecl) TokenIndex {
3247		return self.test_token;
3248		}
3249
3250		pub fn lastToken(self: *const TestDecl) TokenIndex {
3251		return self.body_node.lastToken();
3252		}
	819	pub const FnProtoOne = struct {
	820	/// Populated if there is exactly 1 parameter. Otherwise there are 0 parameters.
	821	param: Index,
	822	/// Populated if align(A) is present.
	823	align_expr: Index,
	824	/// Populated if linksection(A) is present.
	825	section_expr: Index,
	826	/// Populated if callconv(A) is present.
	827	callconv_expr: Index,
3253	828	};
3254		};
3255
3256		pub const PtrInfo = struct {
3257		allowzero_token: ?TokenIndex = null,
3258		align_info: ?Align = null,
3259		const_token: ?TokenIndex = null,
3260		volatile_token: ?TokenIndex = null,
3261		sentinel: ?*Node = null,
3262
3263		pub const Align = struct {
3264		node: *Node,
3265		bit_range: ?BitRange = null,
3266	829
3267		pub const BitRange = struct {
3268		start: *Node,
3269		end: *Node,
3270		};
	830	pub const FnProto = struct {
	831	params_start: Index,
	832	params_end: Index,
	833	/// Populated if align(A) is present.
	834	align_expr: Index,
	835	/// Populated if linksection(A) is present.
	836	section_expr: Index,
	837	/// Populated if callconv(A) is present.
	838	callconv_expr: Index,
3271	839	};
3272	840	};
3273
3274		test "iterate" {
3275		var root = Node.Root{
3276		.base = Node{ .tag = Node.Tag.Root },
3277		.decls_len = 0,
3278		.eof_token = 0,
3279		};
3280		var base = &root.base;
3281		testing.expect(base.iterate(0) == null);
3282		}

lib/std/zig/parse.zig+2911-2755

...	...	@@ -11,85 +11,138 @@ const Node = ast.Node;
11	11	const Tree = ast.Tree;
12	12	const AstError = ast.Error;
13	13	const TokenIndex = ast.TokenIndex;
14		const NodeIndex = ast.NodeIndex;
15	14	const Token = std.zig.Token;
16	15
17	16	pub const Error = error{ParseError} \|\| Allocator.Error;
18	17
19	18	/// Result should be freed with tree.deinit() when there are
20	19	/// no more references to any of the tokens or nodes.
21		pub fn parse(gpa: Allocator, source: []const u8) Allocator.Error!Tree {
22		var token_ids = std.ArrayList(Token.Id).init(gpa);
23		defer token_ids.deinit();
24		var token_locs = std.ArrayList(Token.Loc).init(gpa);
25		defer token_locs.deinit();
	20	pub fn parse(gpa: *Allocator, source: []const u8) Allocator.Error!Tree {
	21	var tokens = ast.TokenList{};
	22	defer tokens.deinit(gpa);
26	23
27	24	// Empirically, the zig std lib has an 8:1 ratio of source bytes to token count.
28	25	const estimated_token_count = source.len / 8;
29		try token_ids.ensureCapacity(estimated_token_count);
30		try token_locs.ensureCapacity(estimated_token_count);
	26	try tokens.ensureCapacity(gpa, estimated_token_count);
31	27
32	28	var tokenizer = std.zig.Tokenizer.init(source);
33	29	while (true) {
34	30	const token = tokenizer.next();
35		try token_ids.append(token.id);
36		try token_locs.append(token.loc);
37		if (token.id == .Eof) break;
	31	if (token.tag == .LineComment) continue;
	32	try tokens.append(gpa, .{
	33	.tag = token.tag,
	34	.start = @intCast(u32, token.loc.start),
	35	});
	36	if (token.tag == .Eof) break;
38	37	}
39	38
40	39	var parser: Parser = .{
41	40	.source = source,
42		.arena = std.heap.ArenaAllocator.init(gpa),
43	41	.gpa = gpa,
44		.token_ids = token_ids.items,
45		.token_locs = token_locs.items,
	42	.token_tags = tokens.items(.tag),
	43	.token_starts = tokens.items(.start),
46	44	.errors = .{},
	45	.nodes = .{},
	46	.extra_data = .{},
47	47	.tok_i = 0,
48	48	};
49	49	defer parser.errors.deinit(gpa);
50		errdefer parser.arena.deinit();
51
52		while (token_ids.items[parser.tok_i] == .LineComment) parser.tok_i += 1;
53
54		const root_node = try parser.parseRoot();
	50	defer parser.nodes.deinit(gpa);
	51	defer parser.extra_data.deinit(gpa);
	52
	53	// Empirically, Zig source code has a 2:1 ratio of tokens to AST nodes.
	54	// Make sure at least 1 so we can use appendAssumeCapacity on the root node below.
	55	const estimated_node_count = (tokens.len + 2) / 2;
	56	try parser.nodes.ensureCapacity(gpa, estimated_node_count);
	57
	58	// Root node must be index 0.
	59	// Root <- skip ContainerMembers eof
	60	parser.nodes.appendAssumeCapacity(.{
	61	.tag = .Root,
	62	.main_token = 0,
	63	.data = .{
	64	.lhs = undefined,
	65	.rhs = undefined,
	66	},
	67	});
	68	const root_decls = try parser.parseContainerMembers(true);
	69	// parseContainerMembers will try to skip as much
	70	// invalid tokens as it can, so we are now at EOF.
	71	assert(parser.token_tags[parser.tok_i] == .Eof);
	72	parser.nodes.items(.data)[0] = .{
	73	.lhs = root_decls.start,
	74	.rhs = root_decls.end,
	75	};
55	76
56		const tree = try parser.arena.allocator.create(Tree);
57		tree.* = .{
58		.gpa = gpa,
	77	// TODO experiment with compacting the MultiArrayList slices here
	78	return Tree{
59	79	.source = source,
60		.token_ids = token_ids.toOwnedSlice(),
61		.token_locs = token_locs.toOwnedSlice(),
	80	.tokens = tokens.toOwnedSlice(),
	81	.nodes = parser.nodes.toOwnedSlice(),
	82	.extra_data = parser.extra_data.toOwnedSlice(gpa),
62	83	.errors = parser.errors.toOwnedSlice(gpa),
63		.root_node = root_node,
64		.arena = parser.arena.state,
65	84	};
66		return tree;
67	85	}
68	86
	87	const null_node: Node.Index = 0;
	88
69	89	/// Represents in-progress parsing, will be converted to an ast.Tree after completion.
70	90	const Parser = struct {
71		arena: std.heap.ArenaAllocator,
72	91	gpa: *Allocator,
73	92	source: []const u8,
74		token_ids: []const Token.Id,
75		token_locs: []const Token.Loc,
	93	token_tags: []const Token.Tag,
	94	token_starts: []const ast.ByteOffset,
76	95	tok_i: TokenIndex,
77	96	errors: std.ArrayListUnmanaged(AstError),
	97	nodes: ast.NodeList,
	98	extra_data: std.ArrayListUnmanaged(Node.Index),
	99
	100	const SmallSpan = union(enum) {
	101	zero_or_one: Node.Index,
	102	multi: []Node.Index,
78	103
79		/// Root <- skip ContainerMembers eof
80		fn parseRoot(p: Parser) Allocator.Error!Node.Root {
81		const decls = try parseContainerMembers(p, true);
82		defer p.gpa.free(decls);
	104	fn deinit(self: SmallSpan, gpa: *Allocator) void {
	105	switch (self) {
	106	.zero_or_one => {},
	107	.multi => \|list\| gpa.free(list),
	108	}
	109	}
	110	};
83	111
84		// parseContainerMembers will try to skip as much
85		// invalid tokens as it can so this can only be the EOF
86		const eof_token = p.eatToken(.Eof).?;
	112	fn listToSpan(p: *Parser, list: []const Node.Index) !Node.SubRange {
	113	try p.extra_data.appendSlice(p.gpa, list);
	114	return Node.SubRange{
	115	.start = @intCast(Node.Index, p.extra_data.items.len - list.len),
	116	.end = @intCast(Node.Index, p.extra_data.items.len),
	117	};
	118	}
87	119
88		const decls_len = @intCast(NodeIndex, decls.len);
89		const node = try Node.Root.create(&p.arena.allocator, decls_len, eof_token);
90		std.mem.copy(*Node, node.decls(), decls);
	120	fn addNode(p: *Parser, elem: ast.NodeList.Elem) Allocator.Error!Node.Index {
	121	const result = @intCast(Node.Index, p.nodes.len);
	122	try p.nodes.append(p.gpa, elem);
	123	return result;
	124	}
91	125
92		return node;
	126	fn addExtra(p: *Parser, extra: anytype) Allocator.Error!Node.Index {
	127	const fields = std.meta.fields(@TypeOf(extra));
	128	try p.extra_data.ensureCapacity(p.gpa, p.extra_data.items.len + fields.len);
	129	const result = @intCast(u32, p.extra_data.items.len);
	130	inline for (fields) \|field\| {
	131	comptime assert(field.field_type == Node.Index);
	132	p.extra_data.appendAssumeCapacity(@field(extra, field.name));
	133	}
	134	return result;
	135	}
	136
	137	fn warn(p: *Parser, msg: ast.Error) error{OutOfMemory}!void {
	138	@setCold(true);
	139	try p.errors.append(p.gpa, msg);
	140	}
	141
	142	fn fail(p: *Parser, msg: ast.Error) error{ ParseError, OutOfMemory } {
	143	@setCold(true);
	144	try p.warn(msg);
	145	return error.ParseError;
93	146	}
94	147
95	148	/// ContainerMembers
...	...	@@ -99,8 +152,8 @@ const Parser = struct {
99	152	/// / ContainerField COMMA ContainerMembers
100	153	/// / ContainerField
101	154	/// /
102		fn parseContainerMembers(p: Parser, top_level: bool) ![]Node {
103		var list = std.ArrayList(*Node).init(p.gpa);
	155	fn parseContainerMembers(p: *Parser, top_level: bool) !Node.SubRange {
	156	var list = std.ArrayList(Node.Index).init(p.gpa);
104	157	defer list.deinit();
105	158
106	159	var field_state: union(enum) {
...	...	@@ -115,103 +168,98 @@ const Parser = struct {
115	168	err,
116	169	} = .none;
117	170
118		while (true) {
119		if (try p.parseContainerDocComments()) \|node\| {
120		try list.append(node);
121		continue;
122		}
	171	// Skip container doc comments.
	172	while (p.eatToken(.ContainerDocComment)) \|_\| {}
123	173
124		const doc_comments = try p.parseDocComment();
	174	while (true) {
	175	const doc_comment = p.eatDocComments();
125	176
126		if (p.parseTestDecl() catch \|err\| switch (err) {
	177	const test_decl_node = p.parseTestDecl() catch \|err\| switch (err) {
127	178	error.OutOfMemory => return error.OutOfMemory,
128	179	error.ParseError => {
129	180	p.findNextContainerMember();
130	181	continue;
131	182	},
132		}) \|node\| {
	183	};
	184	if (test_decl_node != 0) {
133	185	if (field_state == .seen) {
134		field_state = .{ .end = node.firstToken() };
	186	field_state = .{ .end = p.nodes.items(.main_token)[test_decl_node] };
135	187	}
136		node.cast(Node.TestDecl).?.doc_comments = doc_comments;
137		try list.append(node);
	188	try list.append(test_decl_node);
138	189	continue;
139	190	}
140	191
141		if (p.parseTopLevelComptime() catch \|err\| switch (err) {
	192	const comptime_node = p.parseTopLevelComptime() catch \|err\| switch (err) {
142	193	error.OutOfMemory => return error.OutOfMemory,
143	194	error.ParseError => {
144	195	p.findNextContainerMember();
145	196	continue;
146	197	},
147		}) \|node\| {
	198	};
	199	if (comptime_node != 0) {
148	200	if (field_state == .seen) {
149		field_state = .{ .end = node.firstToken() };
	201	field_state = .{ .end = p.nodes.items(.main_token)[comptime_node] };
150	202	}
151		node.cast(Node.Comptime).?.doc_comments = doc_comments;
152		try list.append(node);
	203	try list.append(comptime_node);
153	204	continue;
154	205	}
155	206
156	207	const visib_token = p.eatToken(.Keyword_pub);
157	208
158		if (p.parseTopLevelDecl(doc_comments, visib_token) catch \|err\| switch (err) {
	209	const top_level_decl = p.parseTopLevelDecl() catch \|err\| switch (err) {
159	210	error.OutOfMemory => return error.OutOfMemory,
160	211	error.ParseError => {
161	212	p.findNextContainerMember();
162	213	continue;
163	214	},
164		}) \|node\| {
	215	};
	216	if (top_level_decl != 0) {
165	217	if (field_state == .seen) {
166		field_state = .{ .end = visib_token orelse node.firstToken() };
	218	field_state = .{
	219	.end = visib_token orelse p.nodes.items(.main_token)[top_level_decl],
	220	};
167	221	}
168		try list.append(node);
	222	try list.append(top_level_decl);
169	223	continue;
170	224	}
171	225
172	226	if (visib_token != null) {
173		try p.errors.append(p.gpa, .{
174		.ExpectedPubItem = .{ .token = p.tok_i },
175		});
	227	try p.warn(.{ .ExpectedPubItem = .{ .token = p.tok_i } });
176	228	// ignore this pub
177	229	continue;
178	230	}
179	231
180		if (p.parseContainerField() catch \|err\| switch (err) {
	232	const container_field = p.parseContainerField() catch \|err\| switch (err) {
181	233	error.OutOfMemory => return error.OutOfMemory,
182	234	error.ParseError => {
183	235	// attempt to recover
184	236	p.findNextContainerMember();
185	237	continue;
186	238	},
187		}) \|node\| {
	239	};
	240	if (container_field != 0) {
188	241	switch (field_state) {
189	242	.none => field_state = .seen,
190	243	.err, .seen => {},
191	244	.end => \|tok\| {
192		try p.errors.append(p.gpa, .{
193		.DeclBetweenFields = .{ .token = tok },
194		});
	245	try p.warn(.{ .DeclBetweenFields = .{ .token = tok } });
195	246	// continue parsing, error will be reported later
196	247	field_state = .err;
197	248	},
198	249	}
199
200		const field = node.cast(Node.ContainerField).?;
201		field.doc_comments = doc_comments;
202		try list.append(node);
	250	try list.append(container_field);
203	251	const comma = p.eatToken(.Comma) orelse {
204	252	// try to continue parsing
205	253	const index = p.tok_i;
206	254	p.findNextContainerMember();
207		const next = p.token_ids[p.tok_i];
	255	const next = p.token_tags[p.tok_i];
208	256	switch (next) {
209	257	.Eof => {
210	258	// no invalid tokens were found
211	259	if (index == p.tok_i) break;
212	260
213	261	// Invalid tokens, add error and exit
214		try p.errors.append(p.gpa, .{
	262	try p.warn(.{
215	263	.ExpectedToken = .{ .token = index, .expected_id = .Comma },
216	264	});
217	265	break;
...	...	@@ -219,35 +267,33 @@ const Parser = struct {
219	267	else => {
220	268	if (next == .RBrace) {
221	269	if (!top_level) break;
222		_ = p.nextToken();
	270	p.tok_i += 1;
223	271	}
224	272
225	273	// add error and continue
226		try p.errors.append(p.gpa, .{
	274	try p.warn(.{
227	275	.ExpectedToken = .{ .token = index, .expected_id = .Comma },
228	276	});
229	277	continue;
230	278	},
231	279	}
232	280	};
233		if (try p.parseAppendedDocComment(comma)) \|appended_comment\|
234		field.doc_comments = appended_comment;
235	281	continue;
236	282	}
237	283
238	284	// Dangling doc comment
239		if (doc_comments != null) {
240		try p.errors.append(p.gpa, .{
241		.UnattachedDocComment = .{ .token = doc_comments.?.firstToken() },
	285	if (doc_comment) \|tok\| {
	286	try p.warn(.{
	287	.UnattachedDocComment = .{ .token = tok },
242	288	});
243	289	}
244	290
245		const next = p.token_ids[p.tok_i];
	291	const next = p.token_tags[p.tok_i];
246	292	switch (next) {
247	293	.Eof => break,
248	294	.Keyword_comptime => {
249		_ = p.nextToken();
250		try p.errors.append(p.gpa, .{
	295	p.tok_i += 1;
	296	try p.warn(.{
251	297	.ExpectedBlockOrField = .{ .token = p.tok_i },
252	298	});
253	299	},
...	...	@@ -255,20 +301,20 @@ const Parser = struct {
255	301	const index = p.tok_i;
256	302	if (next == .RBrace) {
257	303	if (!top_level) break;
258		_ = p.nextToken();
	304	p.tok_i += 1;
259	305	}
260	306
261	307	// this was likely not supposed to end yet,
262	308	// try to find the next declaration
263	309	p.findNextContainerMember();
264		try p.errors.append(p.gpa, .{
	310	try p.warn(.{
265	311	.ExpectedContainerMembers = .{ .token = index },
266	312	});
267	313	},
268	314	}
269	315	}
270	316
271		return list.toOwnedSlice();
	317	return p.listToSpan(list.items);
272	318	}
273	319
274	320	/// Attempts to find next container member by searching for certain tokens
...	...	@@ -276,7 +322,7 @@ const Parser = struct {
276	322	var level: u32 = 0;
277	323	while (true) {
278	324	const tok = p.nextToken();
279		switch (p.token_ids[tok]) {
	325	switch (p.token_tags[tok]) {
280	326	// any of these can start a new top level declaration
281	327	.Keyword_test,
282	328	.Keyword_comptime,
...	...	@@ -293,7 +339,7 @@ const Parser = struct {
293	339	.Identifier,
294	340	=> {
295	341	if (level == 0) {
296		p.putBackToken(tok);
	342	p.tok_i -= 1;
297	343	return;
298	344	}
299	345	},
...	...	@@ -310,13 +356,13 @@ const Parser = struct {
310	356	.RBrace => {
311	357	if (level == 0) {
312	358	// end of container, exit
313		p.putBackToken(tok);
	359	p.tok_i -= 1;
314	360	return;
315	361	}
316	362	level -= 1;
317	363	},
318	364	.Eof => {
319		p.putBackToken(tok);
	365	p.tok_i -= 1;
320	366	return;
321	367	},
322	368	else => {},
...	...	@@ -329,11 +375,11 @@ const Parser = struct {
329	375	var level: u32 = 0;
330	376	while (true) {
331	377	const tok = p.nextToken();
332		switch (p.token_ids[tok]) {
	378	switch (p.token_tags[tok]) {
333	379	.LBrace => level += 1,
334	380	.RBrace => {
335	381	if (level == 0) {
336		p.putBackToken(tok);
	382	p.tok_i -= 1;
337	383	return;
338	384	}
339	385	level -= 1;
...	...	@@ -344,7 +390,7 @@ const Parser = struct {
344	390	}
345	391	},
346	392	.Eof => {
347		p.putBackToken(tok);
	393	p.tok_i -= 1;
348	394	return;
349	395	},
350	396	else => {},
...	...	@@ -352,328 +398,315 @@ const Parser = struct {
352	398	}
353	399	}
354	400
355		/// Eat a multiline container doc comment
356		fn parseContainerDocComments(p: Parser) !?Node {
357		if (p.eatToken(.ContainerDocComment)) \|first_line\| {
358		while (p.eatToken(.ContainerDocComment)) \|_\| {}
359		const node = try p.arena.allocator.create(Node.DocComment);
360		node.* = .{ .first_line = first_line };
361		return &node.base;
362		}
363		return null;
364		}
365
366		/// TestDecl <- KEYWORD_test STRINGLITERALSINGLE Block
367		fn parseTestDecl(p: Parser) !?Node {
368		const test_token = p.eatToken(.Keyword_test) orelse return null;
369		const name_node = try p.parseStringLiteralSingle();
370		const block_node = (try p.parseBlock(null)) orelse {
371		try p.errors.append(p.gpa, .{ .ExpectedLBrace = .{ .token = p.tok_i } });
372		return error.ParseError;
373		};
374
375		const test_node = try p.arena.allocator.create(Node.TestDecl);
376		test_node.* = .{
377		.doc_comments = null,
378		.test_token = test_token,
379		.name = name_node,
380		.body_node = block_node,
381		};
382		return &test_node.base;
	401	/// TestDecl <- KEYWORD_test STRINGLITERALSINGLE? Block
	402	fn parseTestDecl(p: *Parser) !Node.Index {
	403	const test_token = p.eatToken(.Keyword_test) orelse return null_node;
	404	const name_token = try p.expectToken(.StringLiteral);
	405	const block_node = try p.parseBlock();
	406	if (block_node == 0) return p.fail(.{ .ExpectedLBrace = .{ .token = p.tok_i } });
	407	return p.addNode(.{
	408	.tag = .TestDecl,
	409	.main_token = test_token,
	410	.data = .{
	411	.lhs = name_token,
	412	.rhs = block_node,
	413	},
	414	});
383	415	}
384	416
385	417	/// TopLevelComptime <- KEYWORD_comptime BlockExpr
386		fn parseTopLevelComptime(p: Parser) !?Node {
387		const tok = p.eatToken(.Keyword_comptime) orelse return null;
388		const lbrace = p.eatToken(.LBrace) orelse {
389		p.putBackToken(tok);
390		return null;
391		};
392		p.putBackToken(lbrace);
393		const block_node = try p.expectNode(parseBlockExpr, .{
394		.ExpectedLabelOrLBrace = .{ .token = p.tok_i },
395		});
396
397		const comptime_node = try p.arena.allocator.create(Node.Comptime);
398		comptime_node.* = .{
399		.doc_comments = null,
400		.comptime_token = tok,
401		.expr = block_node,
402		};
403		return &comptime_node.base;
	418	fn parseTopLevelComptime(p: *Parser) !Node.Index {
	419	if (p.token_tags[p.tok_i] == .Keyword_comptime and
	420	p.token_tags[p.tok_i + 1] == .LBrace)
	421	{
	422	return p.addNode(.{
	423	.tag = .Comptime,
	424	.main_token = p.nextToken(),
	425	.data = .{
	426	.lhs = try p.parseBlock(),
	427	.rhs = undefined,
	428	},
	429	});
	430	} else {
	431	return null_node;
	432	}
404	433	}
405	434
406	435	/// TopLevelDecl
407	436	/// <- (KEYWORD_export / KEYWORD_extern STRINGLITERALSINGLE? / (KEYWORD_inline / KEYWORD_noinline))? FnProto (SEMICOLON / Block)
408	437	/// / (KEYWORD_export / KEYWORD_extern STRINGLITERALSINGLE?)? KEYWORD_threadlocal? VarDecl
409	438	/// / KEYWORD_usingnamespace Expr SEMICOLON
410		fn parseTopLevelDecl(p: Parser, doc_comments: ?Node.DocComment, visib_token: ?TokenIndex) !?*Node {
411		var lib_name: ?*Node = null;
412		const extern_export_inline_token = blk: {
413		if (p.eatToken(.Keyword_export)) \|token\| break :blk token;
414		if (p.eatToken(.Keyword_extern)) \|token\| {
415		lib_name = try p.parseStringLiteralSingle();
416		break :blk token;
	439	fn parseTopLevelDecl(p: *Parser) !Node.Index {
	440	const extern_export_inline_token = p.nextToken();
	441	var expect_fn: bool = false;
	442	var exported: bool = false;
	443	switch (p.token_tags[extern_export_inline_token]) {
	444	.Keyword_extern => _ = p.eatToken(.StringLiteral),
	445	.Keyword_export => exported = true,
	446	.Keyword_inline, .Keyword_noinline => expect_fn = true,
	447	else => p.tok_i -= 1,
	448	}
	449	const fn_proto = try p.parseFnProto();
	450	if (fn_proto != 0) {
	451	switch (p.token_tags[p.tok_i]) {
	452	.Semicolon => {
	453	const semicolon_token = p.nextToken();
	454	try p.parseAppendedDocComment(semicolon_token);
	455	return fn_proto;
	456	},
	457	.LBrace => {
	458	const body_block = try p.parseBlock();
	459	assert(body_block != 0);
	460	return p.addNode(.{
	461	.tag = .FnDecl,
	462	.main_token = p.nodes.items(.main_token)[fn_proto],
	463	.data = .{
	464	.lhs = fn_proto,
	465	.rhs = body_block,
	466	},
	467	});
	468	},
	469	else => {
	470	// Since parseBlock only return error.ParseError on
	471	// a missing '}' we can assume this function was
	472	// supposed to end here.
	473	try p.warn(.{ .ExpectedSemiOrLBrace = .{ .token = p.tok_i } });
	474	return null_node;
	475	},
417	476	}
418		if (p.eatToken(.Keyword_inline)) \|token\| break :blk token;
419		if (p.eatToken(.Keyword_noinline)) \|token\| break :blk token;
420		break :blk null;
421		};
422
423		if (try p.parseFnProto(.top_level, .{
424		.doc_comments = doc_comments,
425		.visib_token = visib_token,
426		.extern_export_inline_token = extern_export_inline_token,
427		.lib_name = lib_name,
428		})) \|node\| {
429		return node;
430	477	}
431
432		if (extern_export_inline_token) \|token\| {
433		if (p.token_ids[token] == .Keyword_inline or
434		p.token_ids[token] == .Keyword_noinline)
435		{
436		try p.errors.append(p.gpa, .{
437		.ExpectedFn = .{ .token = p.tok_i },
438		});
439		return error.ParseError;
440		}
	478	if (expect_fn) {
	479	try p.warn(.{
	480	.ExpectedFn = .{ .token = p.tok_i },
	481	});
	482	return error.ParseError;
441	483	}
442	484
443	485	const thread_local_token = p.eatToken(.Keyword_threadlocal);
444
445		if (try p.parseVarDecl(.{
446		.doc_comments = doc_comments,
447		.visib_token = visib_token,
448		.thread_local_token = thread_local_token,
449		.extern_export_token = extern_export_inline_token,
450		.lib_name = lib_name,
451		})) \|node\| {
452		return node;
	486	const var_decl = try p.parseVarDecl();
	487	if (var_decl != 0) {
	488	const semicolon_token = try p.expectToken(.Semicolon);
	489	try p.parseAppendedDocComment(semicolon_token);
	490	return var_decl;
453	491	}
454
455	492	if (thread_local_token != null) {
456		try p.errors.append(p.gpa, .{
457		.ExpectedVarDecl = .{ .token = p.tok_i },
458		});
459		// ignore this and try again;
460		return error.ParseError;
	493	return p.fail(.{ .ExpectedVarDecl = .{ .token = p.tok_i } });
461	494	}
462	495
463		if (extern_export_inline_token) \|token\| {
464		try p.errors.append(p.gpa, .{
465		.ExpectedVarDeclOrFn = .{ .token = p.tok_i },
466		});
467		// ignore this and try again;
468		return error.ParseError;
	496	if (exported) {
	497	return p.fail(.{ .ExpectedVarDeclOrFn = .{ .token = p.tok_i } });
469	498	}
470	499
471		const use_token = p.eatToken(.Keyword_usingnamespace) orelse return null;
472		const expr = try p.expectNode(parseExpr, .{
473		.ExpectedExpr = .{ .token = p.tok_i },
474		});
	500	const usingnamespace_token = p.eatToken(.Keyword_usingnamespace) orelse return null_node;
	501	const expr = try p.expectExpr();
475	502	const semicolon_token = try p.expectToken(.Semicolon);
476
477		const node = try p.arena.allocator.create(Node.Use);
478		node.* = .{
479		.doc_comments = doc_comments orelse try p.parseAppendedDocComment(semicolon_token),
480		.visib_token = visib_token,
481		.use_token = use_token,
482		.expr = expr,
483		.semicolon_token = semicolon_token,
484		};
485
486		return &node.base;
	503	try p.parseAppendedDocComment(semicolon_token);
	504	return p.addNode(.{
	505	.tag = .UsingNamespace,
	506	.main_token = usingnamespace_token,
	507	.data = .{
	508	.lhs = expr,
	509	.rhs = undefined,
	510	},
	511	});
487	512	}
488	513
489	514	/// FnProto <- KEYWORD_fn IDENTIFIER? LPAREN ParamDeclList RPAREN ByteAlign? LinkSection? CallConv? EXCLAMATIONMARK? (Keyword_anytype / TypeExpr)
490		fn parseFnProto(p: *Parser, level: enum { top_level, as_type }, fields: struct {
491		doc_comments: ?*Node.DocComment = null,
492		visib_token: ?TokenIndex = null,
493		extern_export_inline_token: ?TokenIndex = null,
494		lib_name: ?*Node = null,
495		}) !?*Node {
496		// TODO: Remove once extern/async fn rewriting is
497		var is_async: ?void = null;
498		var is_extern_prototype: ?void = null;
499		const cc_token: ?TokenIndex = blk: {
500		if (p.eatToken(.Keyword_extern)) \|token\| {
501		is_extern_prototype = {};
502		break :blk token;
503		}
504		if (p.eatToken(.Keyword_async)) \|token\| {
505		is_async = {};
506		break :blk token;
507		}
508		break :blk null;
509		};
510		const fn_token = p.eatToken(.Keyword_fn) orelse {
511		if (cc_token) \|token\|
512		p.putBackToken(token);
513		return null;
514		};
515		const name_token = p.eatToken(.Identifier);
516		const lparen = try p.expectToken(.LParen);
	515	fn parseFnProto(p: *Parser) !Node.Index {
	516	const fn_token = p.eatToken(.Keyword_fn) orelse return null_node;
	517	_ = p.eatToken(.Identifier);
517	518	const params = try p.parseParamDeclList();
518		defer p.gpa.free(params);
519		const var_args_token = p.eatToken(.Ellipsis3);
520		const rparen = try p.expectToken(.RParen);
	519	defer params.deinit(p.gpa);
521	520	const align_expr = try p.parseByteAlign();
522	521	const section_expr = try p.parseLinkSection();
523	522	const callconv_expr = try p.parseCallconv();
524		const exclamation_token = p.eatToken(.Bang);
	523	const bang_token = p.eatToken(.Bang);
525	524
526		const return_type_expr = (try p.parseAnyType()) orelse
527		try p.expectNodeRecoverable(parseTypeExpr, .{
	525	const return_type_expr = try p.parseTypeExpr();
	526	if (return_type_expr == 0) {
528	527	// most likely the user forgot to specify the return type.
529	528	// Mark return type as invalid and try to continue.
530		.ExpectedReturnType = .{ .token = p.tok_i },
531		});
532
533		// TODO https://github.com/ziglang/zig/issues/3750
534		const R = Node.FnProto.ReturnType;
535		const return_type = if (return_type_expr == null)
536		R{ .Invalid = rparen }
537		else if (exclamation_token != null)
538		R{ .InferErrorSet = return_type_expr.? }
539		else
540		R{ .Explicit = return_type_expr.? };
	529	try p.warn(.{ .ExpectedReturnType = .{ .token = p.tok_i } });
	530	}
541	531
542		const body_node: ?*Node = switch (level) {
543		.top_level => blk: {
544		if (p.eatToken(.Semicolon)) \|_\| {
545		break :blk null;
546		}
547		const body_block = (try p.parseBlock(null)) orelse {
548		// Since parseBlock only return error.ParseError on
549		// a missing '}' we can assume this function was
550		// supposed to end here.
551		try p.errors.append(p.gpa, .{ .ExpectedSemiOrLBrace = .{ .token = p.tok_i } });
552		break :blk null;
553		};
554		break :blk body_block;
	532	if (align_expr == 0 and section_expr == 0 and callconv_expr == 0) {
	533	switch (params) {
	534	.zero_or_one => \|param\| return p.addNode(.{
	535	.tag = .FnProtoSimple,
	536	.main_token = fn_token,
	537	.data = .{
	538	.lhs = param,
	539	.rhs = return_type_expr,
	540	},
	541	}),
	542	.multi => \|list\| {
	543	const span = try p.listToSpan(list);
	544	return p.addNode(.{
	545	.tag = .FnProtoSimpleMulti,
	546	.main_token = fn_token,
	547	.data = .{
	548	.lhs = try p.addExtra(Node.SubRange{
	549	.start = span.start,
	550	.end = span.end,
	551	}),
	552	.rhs = return_type_expr,
	553	},
	554	});
	555	},
	556	}
	557	}
	558	switch (params) {
	559	.zero_or_one => \|param\| return p.addNode(.{
	560	.tag = .FnProtoOne,
	561	.main_token = fn_token,
	562	.data = .{
	563	.lhs = try p.addExtra(Node.FnProtoOne{
	564	.param = param,
	565	.align_expr = align_expr,
	566	.section_expr = section_expr,
	567	.callconv_expr = callconv_expr,
	568	}),
	569	.rhs = return_type_expr,
	570	},
	571	}),
	572	.multi => \|list\| {
	573	const span = try p.listToSpan(list);
	574	return p.addNode(.{
	575	.tag = .FnProto,
	576	.main_token = fn_token,
	577	.data = .{
	578	.lhs = try p.addExtra(Node.FnProto{
	579	.params_start = span.start,
	580	.params_end = span.end,
	581	.align_expr = align_expr,
	582	.section_expr = section_expr,
	583	.callconv_expr = callconv_expr,
	584	}),
	585	.rhs = return_type_expr,
	586	},
	587	});
555	588	},
556		.as_type => null,
557		};
558
559		const fn_proto_node = try Node.FnProto.create(&p.arena.allocator, .{
560		.params_len = params.len,
561		.fn_token = fn_token,
562		.return_type = return_type,
563		}, .{
564		.doc_comments = fields.doc_comments,
565		.visib_token = fields.visib_token,
566		.name_token = name_token,
567		.var_args_token = var_args_token,
568		.extern_export_inline_token = fields.extern_export_inline_token,
569		.body_node = body_node,
570		.lib_name = fields.lib_name,
571		.align_expr = align_expr,
572		.section_expr = section_expr,
573		.callconv_expr = callconv_expr,
574		.is_extern_prototype = is_extern_prototype,
575		.is_async = is_async,
576		});
577		std.mem.copy(Node.FnProto.ParamDecl, fn_proto_node.params(), params);
578
579		return &fn_proto_node.base;
	589	}
580	590	}
581	591
582	592	/// VarDecl <- (KEYWORD_const / KEYWORD_var) IDENTIFIER (COLON TypeExpr)? ByteAlign? LinkSection? (EQUAL Expr)? SEMICOLON
583		fn parseVarDecl(p: *Parser, fields: struct {
584		doc_comments: ?*Node.DocComment = null,
585		visib_token: ?TokenIndex = null,
586		thread_local_token: ?TokenIndex = null,
587		extern_export_token: ?TokenIndex = null,
588		lib_name: ?*Node = null,
589		comptime_token: ?TokenIndex = null,
590		}) !?*Node {
	593	fn parseVarDecl(p: *Parser) !Node.Index {
591	594	const mut_token = p.eatToken(.Keyword_const) orelse
592	595	p.eatToken(.Keyword_var) orelse
593		return null;
	596	return null_node;
594	597
595	598	const name_token = try p.expectToken(.Identifier);
596		const type_node = if (p.eatToken(.Colon) != null)
597		try p.expectNode(parseTypeExpr, .{
598		.ExpectedTypeExpr = .{ .token = p.tok_i },
599		})
600		else
601		null;
	599	const type_node: Node.Index = if (p.eatToken(.Colon) == null) 0 else try p.expectTypeExpr();
602	600	const align_node = try p.parseByteAlign();
603	601	const section_node = try p.parseLinkSection();
604		const eq_token = p.eatToken(.Equal);
605		const init_node = if (eq_token != null) blk: {
606		break :blk try p.expectNode(parseExpr, .{
607		.ExpectedExpr = .{ .token = p.tok_i },
	602	const init_node: Node.Index = if (p.eatToken(.Equal) == null) 0 else try p.expectExpr();
	603	if (section_node == 0) {
	604	if (align_node == 0) {
	605	return p.addNode(.{
	606	.tag = .SimpleVarDecl,
	607	.main_token = mut_token,
	608	.data = .{
	609	.lhs = type_node,
	610	.rhs = init_node,
	611	},
	612	});
	613	} else if (type_node == 0) {
	614	return p.addNode(.{
	615	.tag = .AlignedVarDecl,
	616	.main_token = mut_token,
	617	.data = .{
	618	.lhs = align_node,
	619	.rhs = init_node,
	620	},
	621	});
	622	} else {
	623	return p.addNode(.{
	624	.tag = .LocalVarDecl,
	625	.main_token = mut_token,
	626	.data = .{
	627	.lhs = try p.addExtra(Node.LocalVarDecl{
	628	.type_node = type_node,
	629	.align_node = align_node,
	630	}),
	631	.rhs = init_node,
	632	},
	633	});
	634	}
	635	} else {
	636	return p.addNode(.{
	637	.tag = .GlobalVarDecl,
	638	.main_token = mut_token,
	639	.data = .{
	640	.lhs = try p.addExtra(Node.GlobalVarDecl{
	641	.type_node = type_node,
	642	.align_node = align_node,
	643	.section_node = section_node,
	644	}),
	645	.rhs = init_node,
	646	},
608	647	});
609		} else null;
610		const semicolon_token = try p.expectToken(.Semicolon);
611
612		const doc_comments = fields.doc_comments orelse try p.parseAppendedDocComment(semicolon_token);
613
614		const node = try Node.VarDecl.create(&p.arena.allocator, .{
615		.mut_token = mut_token,
616		.name_token = name_token,
617		.semicolon_token = semicolon_token,
618		}, .{
619		.doc_comments = doc_comments,
620		.visib_token = fields.visib_token,
621		.thread_local_token = fields.thread_local_token,
622		.eq_token = eq_token,
623		.comptime_token = fields.comptime_token,
624		.extern_export_token = fields.extern_export_token,
625		.lib_name = fields.lib_name,
626		.type_node = type_node,
627		.align_node = align_node,
628		.section_node = section_node,
629		.init_node = init_node,
630		});
631		return &node.base;
	648	}
632	649	}
633	650
634	651	/// ContainerField <- KEYWORD_comptime? IDENTIFIER (COLON TypeExpr ByteAlign?)? (EQUAL Expr)?
635		fn parseContainerField(p: Parser) !?Node {
	652	fn parseContainerField(p: *Parser) !Node.Index {
636	653	const comptime_token = p.eatToken(.Keyword_comptime);
637	654	const name_token = p.eatToken(.Identifier) orelse {
638		if (comptime_token) \|t\| p.putBackToken(t);
639		return null;
	655	if (comptime_token) \|_\| p.tok_i -= 1;
	656	return null_node;
640	657	};
641	658
642		var align_expr: ?*Node = null;
643		var type_expr: ?*Node = null;
	659	var align_expr: Node.Index = 0;
	660	var type_expr: Node.Index = 0;
644	661	if (p.eatToken(.Colon)) \|_\| {
645		if (p.eatToken(.Keyword_anytype) orelse p.eatToken(.Keyword_var)) \|anytype_tok\| {
646		const node = try p.arena.allocator.create(Node.OneToken);
647		node.* = .{
648		.base = .{ .tag = .AnyType },
649		.token = anytype_tok,
650		};
651		type_expr = &node.base;
652		} else {
653		type_expr = try p.expectNode(parseTypeExpr, .{
654		.ExpectedTypeExpr = .{ .token = p.tok_i },
	662	if (p.eatToken(.Keyword_anytype)) \|anytype_tok\| {
	663	type_expr = try p.addNode(.{
	664	.tag = .AnyType,
	665	.main_token = anytype_tok,
	666	.data = .{
	667	.lhs = undefined,
	668	.rhs = undefined,
	669	},
655	670	});
	671	} else {
	672	type_expr = try p.expectTypeExpr();
656	673	align_expr = try p.parseByteAlign();
657	674	}
658	675	}
659	676
660		const value_expr = if (p.eatToken(.Equal)) \|_\|
661		try p.expectNode(parseExpr, .{
662		.ExpectedExpr = .{ .token = p.tok_i },
663		})
664		else
665		null;
666
667		const node = try p.arena.allocator.create(Node.ContainerField);
668		node.* = .{
669		.doc_comments = null,
670		.comptime_token = comptime_token,
671		.name_token = name_token,
672		.type_expr = type_expr,
673		.value_expr = value_expr,
674		.align_expr = align_expr,
675		};
676		return &node.base;
	677	const value_expr: Node.Index = if (p.eatToken(.Equal) == null) 0 else try p.expectExpr();
	678
	679	if (align_expr == 0) {
	680	return p.addNode(.{
	681	.tag = .ContainerFieldInit,
	682	.main_token = name_token,
	683	.data = .{
	684	.lhs = type_expr,
	685	.rhs = value_expr,
	686	},
	687	});
	688	} else if (value_expr == 0) {
	689	return p.addNode(.{
	690	.tag = .ContainerFieldAlign,
	691	.main_token = name_token,
	692	.data = .{
	693	.lhs = type_expr,
	694	.rhs = align_expr,
	695	},
	696	});
	697	} else {
	698	return p.addNode(.{
	699	.tag = .ContainerField,
	700	.main_token = name_token,
	701	.data = .{
	702	.lhs = type_expr,
	703	.rhs = try p.addExtra(Node.ContainerField{
	704	.value_expr = value_expr,
	705	.align_expr = align_expr,
	706	}),
	707	},
	708	});
	709	}
677	710	}
678	711
679	712	/// Statement
...	...	@@ -687,833 +720,1475 @@ const Parser = struct {
687	720	/// / LabeledStatement
688	721	/// / SwitchExpr
689	722	/// / AssignExpr SEMICOLON
690		fn parseStatement(p: Parser) Error!?Node {
	723	fn parseStatement(p: *Parser) Error!Node.Index {
691	724	const comptime_token = p.eatToken(.Keyword_comptime);
692	725
693		if (try p.parseVarDecl(.{
694		.comptime_token = comptime_token,
695		})) \|node\| {
696		return node;
	726	const var_decl = try p.parseVarDecl();
	727	if (var_decl != 0) {
	728	_ = try p.expectTokenRecoverable(.Semicolon);
	729	return var_decl;
697	730	}
698	731
699	732	if (comptime_token) \|token\| {
700		const block_expr = try p.expectNode(parseBlockExprStatement, .{
701		.ExpectedBlockOrAssignment = .{ .token = p.tok_i },
	733	return p.addNode(.{
	734	.tag = .Comptime,
	735	.main_token = token,
	736	.data = .{
	737	.lhs = try p.expectBlockExprStatement(),
	738	.rhs = undefined,
	739	},
702	740	});
703
704		const node = try p.arena.allocator.create(Node.Comptime);
705		node.* = .{
706		.doc_comments = null,
707		.comptime_token = token,
708		.expr = block_expr,
709		};
710		return &node.base;
711	741	}
712	742
713		if (p.eatToken(.Keyword_nosuspend)) \|nosuspend_token\| {
714		const block_expr = try p.expectNode(parseBlockExprStatement, .{
715		.ExpectedBlockOrAssignment = .{ .token = p.tok_i },
716		});
717
718		const node = try p.arena.allocator.create(Node.Nosuspend);
719		node.* = .{
720		.nosuspend_token = nosuspend_token,
721		.expr = block_expr,
722		};
723		return &node.base;
	743	const token = p.nextToken();
	744	switch (p.token_tags[token]) {
	745	.Keyword_nosuspend => {
	746	return p.addNode(.{
	747	.tag = .Nosuspend,
	748	.main_token = token,
	749	.data = .{
	750	.lhs = try p.expectBlockExprStatement(),
	751	.rhs = undefined,
	752	},
	753	});
	754	},
	755	.Keyword_suspend => {
	756	const block_expr: Node.Index = if (p.eatToken(.Semicolon) != null)
	757	0
	758	else
	759	try p.expectBlockExprStatement();
	760	return p.addNode(.{
	761	.tag = .Suspend,
	762	.main_token = token,
	763	.data = .{
	764	.lhs = block_expr,
	765	.rhs = undefined,
	766	},
	767	});
	768	},
	769	.Keyword_defer => return p.addNode(.{
	770	.tag = .Defer,
	771	.main_token = token,
	772	.data = .{
	773	.lhs = undefined,
	774	.rhs = try p.expectBlockExprStatement(),
	775	},
	776	}),
	777	.Keyword_errdefer => return p.addNode(.{
	778	.tag = .ErrDefer,
	779	.main_token = token,
	780	.data = .{
	781	.lhs = try p.parsePayload(),
	782	.rhs = try p.expectBlockExprStatement(),
	783	},
	784	}),
	785	else => p.tok_i -= 1,
724	786	}
725	787
726		if (p.eatToken(.Keyword_suspend)) \|suspend_token\| {
727		const semicolon = p.eatToken(.Semicolon);
728
729		const body_node = if (semicolon == null) blk: {
730		break :blk try p.expectNode(parseBlockExprStatement, .{
731		.ExpectedBlockOrExpression = .{ .token = p.tok_i },
732		});
733		} else null;
	788	const if_statement = try p.parseIfStatement();
	789	if (if_statement != 0) return if_statement;
734	790
735		const node = try p.arena.allocator.create(Node.Suspend);
736		node.* = .{
737		.suspend_token = suspend_token,
738		.body = body_node,
739		};
740		return &node.base;
741		}
	791	const labeled_statement = try p.parseLabeledStatement();
	792	if (labeled_statement != 0) return labeled_statement;
742	793
743		const defer_token = p.eatToken(.Keyword_defer) orelse p.eatToken(.Keyword_errdefer);
744		if (defer_token) \|token\| {
745		const payload = if (p.token_ids[token] == .Keyword_errdefer)
746		try p.parsePayload()
747		else
748		null;
749		const expr_node = try p.expectNode(parseBlockExprStatement, .{
750		.ExpectedBlockOrExpression = .{ .token = p.tok_i },
751		});
752		const node = try p.arena.allocator.create(Node.Defer);
753		node.* = .{
754		.defer_token = token,
755		.expr = expr_node,
756		.payload = payload,
757		};
758		return &node.base;
759		}
	794	const switch_expr = try p.parseSwitchExpr();
	795	if (switch_expr != 0) return switch_expr;
760	796
761		if (try p.parseIfStatement()) \|node\| return node;
762		if (try p.parseLabeledStatement()) \|node\| return node;
763		if (try p.parseSwitchExpr()) \|node\| return node;
764		if (try p.parseAssignExpr()) \|node\| {
	797	const assign_expr = try p.parseAssignExpr();
	798	if (assign_expr != 0) {
765	799	_ = try p.expectTokenRecoverable(.Semicolon);
766		return node;
	800	return assign_expr;
767	801	}
768	802
769		return null;
	803	return null_node;
	804	}
	805
	806	fn expectStatement(p: *Parser) !Node.Index {
	807	const statement = try p.parseStatement();
	808	if (statement == 0) {
	809	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	810	}
	811	return statement;
770	812	}
771	813
772	814	/// IfStatement
773	815	/// <- IfPrefix BlockExpr ( KEYWORD_else Payload? Statement )?
774	816	/// / IfPrefix AssignExpr ( SEMICOLON / KEYWORD_else Payload? Statement )
775		fn parseIfStatement(p: Parser) !?Node {
776		const if_node = (try p.parseIfPrefix()) orelse return null;
777		const if_prefix = if_node.cast(Node.If).?;
778
779		const block_expr = (try p.parseBlockExpr());
780		const assign_expr = if (block_expr == null)
781		try p.expectNode(parseAssignExpr, .{
782		.ExpectedBlockOrAssignment = .{ .token = p.tok_i },
783		})
784		else
785		null;
786
787		const semicolon = if (assign_expr != null) p.eatToken(.Semicolon) else null;
788
789		const else_node = if (semicolon == null) blk: {
790		const else_token = p.eatToken(.Keyword_else) orelse break :blk null;
791		const payload = try p.parsePayload();
792		const else_body = try p.expectNode(parseStatement, .{
793		.InvalidToken = .{ .token = p.tok_i },
794		});
795
796		const node = try p.arena.allocator.create(Node.Else);
797		node.* = .{
798		.else_token = else_token,
799		.payload = payload,
800		.body = else_body,
801		};
802
803		break :blk node;
804		} else null;
805
806		if (block_expr) \|body\| {
807		if_prefix.body = body;
808		if_prefix.@"else" = else_node;
809		return if_node;
810		}
811
812		if (assign_expr) \|body\| {
813		if_prefix.body = body;
814		if (semicolon != null) return if_node;
815		if (else_node != null) {
816		if_prefix.@"else" = else_node;
817		return if_node;
	817	fn parseIfStatement(p: *Parser) !Node.Index {
	818	const if_token = p.eatToken(.Keyword_if) orelse return null_node;
	819	_ = try p.expectToken(.LParen);
	820	const condition = try p.expectExpr();
	821	_ = try p.expectToken(.RParen);
	822	const then_payload = try p.parsePtrPayload();
	823
	824	// TODO propose to change the syntax so that semicolons are always required
	825	// inside if statements, even if there is an `else`.
	826	var else_required = false;
	827	const then_expr = blk: {
	828	const block_expr = try p.parseBlockExpr();
	829	if (block_expr != 0) break :blk block_expr;
	830	const assign_expr = try p.parseAssignExpr();
	831	if (assign_expr == 0) {
	832	return p.fail(.{ .ExpectedBlockOrAssignment = .{ .token = p.tok_i } });
	833	}
	834	if (p.eatToken(.Semicolon)) \|_\| {
	835	return p.addNode(.{
	836	.tag = if (then_payload == 0) .IfSimple else .IfSimpleOptional,
	837	.main_token = if_token,
	838	.data = .{
	839	.lhs = condition,
	840	.rhs = assign_expr,
	841	},
	842	});
	843	}
	844	else_required = true;
	845	break :blk assign_expr;
	846	};
	847	const else_token = p.eatToken(.Keyword_else) orelse {
	848	if (else_required) {
	849	return p.fail(.{ .ExpectedSemiOrElse = .{ .token = p.tok_i } });
818	850	}
819		try p.errors.append(p.gpa, .{
820		.ExpectedSemiOrElse = .{ .token = p.tok_i },
	851	return p.addNode(.{
	852	.tag = if (then_payload == 0) .IfSimple else .IfSimpleOptional,
	853	.main_token = if_token,
	854	.data = .{
	855	.lhs = condition,
	856	.rhs = then_expr,
	857	},
821	858	});
822		}
823
824		return if_node;
	859	};
	860	const else_payload = try p.parsePayload();
	861	const else_expr = try p.expectStatement();
	862	const tag = if (else_payload != 0)
	863	Node.Tag.IfError
	864	else if (then_payload != 0)
	865	Node.Tag.IfOptional
	866	else
	867	Node.Tag.If;
	868	return p.addNode(.{
	869	.tag = tag,
	870	.main_token = if_token,
	871	.data = .{
	872	.lhs = condition,
	873	.rhs = try p.addExtra(Node.If{
	874	.then_expr = then_expr,
	875	.else_expr = else_expr,
	876	}),
	877	},
	878	});
825	879	}
826	880
827	881	/// LabeledStatement <- BlockLabel? (Block / LoopStatement)
828		fn parseLabeledStatement(p: Parser) !?Node {
829		var colon: TokenIndex = undefined;
830		const label_token = p.parseBlockLabel(&colon);
831
832		if (try p.parseBlock(label_token)) \|node\| return node;
833
834		if (try p.parseLoopStatement()) \|node\| {
835		if (node.cast(Node.For)) \|for_node\| {
836		for_node.label = label_token;
837		} else if (node.cast(Node.While)) \|while_node\| {
838		while_node.label = label_token;
839		} else unreachable;
840		return node;
841		}
	882	fn parseLabeledStatement(p: *Parser) !Node.Index {
	883	const label_token = p.parseBlockLabel();
	884	const block = try p.parseBlock();
	885	if (block != 0) return block;
842	886
843		if (label_token != null) {
844		try p.errors.append(p.gpa, .{
845		.ExpectedLabelable = .{ .token = p.tok_i },
846		});
847		return error.ParseError;
	887	const loop_stmt = try p.parseLoopStatement();
	888	if (loop_stmt != 0) return loop_stmt;
	889
	890	if (label_token != 0) {
	891	return p.fail(.{ .ExpectedLabelable = .{ .token = p.tok_i } });
848	892	}
849	893
850		return null;
	894	return null_node;
851	895	}
852	896
853	897	/// LoopStatement <- KEYWORD_inline? (ForStatement / WhileStatement)
854		fn parseLoopStatement(p: Parser) !?Node {
	898	fn parseLoopStatement(p: *Parser) !Node.Index {
855	899	const inline_token = p.eatToken(.Keyword_inline);
856	900
857		if (try p.parseForStatement()) \|node\| {
858		node.cast(Node.For).?.inline_token = inline_token;
859		return node;
860		}
	901	const for_statement = try p.parseForStatement();
	902	if (for_statement != 0) return for_statement;
861	903
862		if (try p.parseWhileStatement()) \|node\| {
863		node.cast(Node.While).?.inline_token = inline_token;
864		return node;
865		}
866		if (inline_token == null) return null;
	904	const while_statement = try p.parseWhileStatement();
	905	if (while_statement != 0) return while_statement;
	906
	907	if (inline_token == null) return null_node;
867	908
868	909	// If we've seen "inline", there should have been a "for" or "while"
869		try p.errors.append(p.gpa, .{
870		.ExpectedInlinable = .{ .token = p.tok_i },
871		});
872		return error.ParseError;
	910	return p.fail(.{ .ExpectedInlinable = .{ .token = p.tok_i } });
873	911	}
874	912
	913	/// ForPrefix <- KEYWORD_for LPAREN Expr RPAREN PtrIndexPayload
875	914	/// ForStatement
876	915	/// <- ForPrefix BlockExpr ( KEYWORD_else Statement )?
877	916	/// / ForPrefix AssignExpr ( SEMICOLON / KEYWORD_else Statement )
878		fn parseForStatement(p: Parser) !?Node {
879		const node = (try p.parseForPrefix()) orelse return null;
880		const for_prefix = node.cast(Node.For).?;
881
882		if (try p.parseBlockExpr()) \|block_expr_node\| {
883		for_prefix.body = block_expr_node;
884
885		if (p.eatToken(.Keyword_else)) \|else_token\| {
886		const statement_node = try p.expectNode(parseStatement, .{
887		.InvalidToken = .{ .token = p.tok_i },
	917	fn parseForStatement(p: *Parser) !Node.Index {
	918	const for_token = p.eatToken(.Keyword_for) orelse return null_node;
	919	_ = try p.expectToken(.LParen);
	920	const array_expr = try p.expectExpr();
	921	_ = try p.expectToken(.RParen);
	922	_ = try p.parsePtrIndexPayload();
	923
	924	// TODO propose to change the syntax so that semicolons are always required
	925	// inside while statements, even if there is an `else`.
	926	var else_required = false;
	927	const then_expr = blk: {
	928	const block_expr = try p.parseBlockExpr();
	929	if (block_expr != 0) break :blk block_expr;
	930	const assign_expr = try p.parseAssignExpr();
	931	if (assign_expr == 0) {
	932	return p.fail(.{ .ExpectedBlockOrAssignment = .{ .token = p.tok_i } });
	933	}
	934	if (p.eatToken(.Semicolon)) \|_\| {
	935	return p.addNode(.{
	936	.tag = .ForSimple,
	937	.main_token = for_token,
	938	.data = .{
	939	.lhs = array_expr,
	940	.rhs = assign_expr,
	941	},
888	942	});
889
890		const else_node = try p.arena.allocator.create(Node.Else);
891		else_node.* = .{
892		.else_token = else_token,
893		.payload = null,
894		.body = statement_node,
895		};
896		for_prefix.@"else" = else_node;
897
898		return node;
899	943	}
900
901		return node;
902		}
903
904		for_prefix.body = try p.expectNode(parseAssignExpr, .{
905		.ExpectedBlockOrAssignment = .{ .token = p.tok_i },
906		});
907
908		if (p.eatToken(.Semicolon) != null) return node;
909
910		if (p.eatToken(.Keyword_else)) \|else_token\| {
911		const statement_node = try p.expectNode(parseStatement, .{
912		.ExpectedStatement = .{ .token = p.tok_i },
	944	else_required = true;
	945	break :blk assign_expr;
	946	};
	947	const else_token = p.eatToken(.Keyword_else) orelse {
	948	if (else_required) {
	949	return p.fail(.{ .ExpectedSemiOrElse = .{ .token = p.tok_i } });
	950	}
	951	return p.addNode(.{
	952	.tag = .ForSimple,
	953	.main_token = for_token,
	954	.data = .{
	955	.lhs = array_expr,
	956	.rhs = then_expr,
	957	},
913	958	});
914
915		const else_node = try p.arena.allocator.create(Node.Else);
916		else_node.* = .{
917		.else_token = else_token,
918		.payload = null,
919		.body = statement_node,
920		};
921		for_prefix.@"else" = else_node;
922		return node;
923		}
924
925		try p.errors.append(p.gpa, .{
926		.ExpectedSemiOrElse = .{ .token = p.tok_i },
	959	};
	960	return p.addNode(.{
	961	.tag = .For,
	962	.main_token = for_token,
	963	.data = .{
	964	.lhs = array_expr,
	965	.rhs = try p.addExtra(Node.If{
	966	.then_expr = then_expr,
	967	.else_expr = try p.expectStatement(),
	968	}),
	969	},
927	970	});
928
929		return node;
930	971	}
931	972
	973	/// WhilePrefix <- KEYWORD_while LPAREN Expr RPAREN PtrPayload? WhileContinueExpr?
932	974	/// WhileStatement
933	975	/// <- WhilePrefix BlockExpr ( KEYWORD_else Payload? Statement )?
934	976	/// / WhilePrefix AssignExpr ( SEMICOLON / KEYWORD_else Payload? Statement )
935		fn parseWhileStatement(p: Parser) !?Node {
936		const node = (try p.parseWhilePrefix()) orelse return null;
937		const while_prefix = node.cast(Node.While).?;
938
939		if (try p.parseBlockExpr()) \|block_expr_node\| {
940		while_prefix.body = block_expr_node;
941
942		if (p.eatToken(.Keyword_else)) \|else_token\| {
943		const payload = try p.parsePayload();
944
945		const statement_node = try p.expectNode(parseStatement, .{
946		.InvalidToken = .{ .token = p.tok_i },
947		});
948
949		const else_node = try p.arena.allocator.create(Node.Else);
950		else_node.* = .{
951		.else_token = else_token,
952		.payload = payload,
953		.body = statement_node,
954		};
955		while_prefix.@"else" = else_node;
	977	fn parseWhileStatement(p: *Parser) !Node.Index {
	978	const while_token = p.eatToken(.Keyword_while) orelse return null_node;
	979	_ = try p.expectToken(.LParen);
	980	const condition = try p.expectExpr();
	981	_ = try p.expectToken(.RParen);
	982	const then_payload = try p.parsePtrPayload();
	983	const continue_expr = try p.parseWhileContinueExpr();
956	984
957		return node;
	985	// TODO propose to change the syntax so that semicolons are always required
	986	// inside while statements, even if there is an `else`.
	987	var else_required = false;
	988	const then_expr = blk: {
	989	const block_expr = try p.parseBlockExpr();
	990	if (block_expr != 0) break :blk block_expr;
	991	const assign_expr = try p.parseAssignExpr();
	992	if (assign_expr == 0) {
	993	return p.fail(.{ .ExpectedBlockOrAssignment = .{ .token = p.tok_i } });
958	994	}
959
960		return node;
961		}
962
963		while_prefix.body = try p.expectNode(parseAssignExpr, .{
964		.ExpectedBlockOrAssignment = .{ .token = p.tok_i },
965		});
966
967		if (p.eatToken(.Semicolon) != null) return node;
968
969		if (p.eatToken(.Keyword_else)) \|else_token\| {
970		const payload = try p.parsePayload();
971
972		const statement_node = try p.expectNode(parseStatement, .{
973		.ExpectedStatement = .{ .token = p.tok_i },
974		});
975
976		const else_node = try p.arena.allocator.create(Node.Else);
977		else_node.* = .{
978		.else_token = else_token,
979		.payload = payload,
980		.body = statement_node,
981		};
982		while_prefix.@"else" = else_node;
983		return node;
984		}
985
986		try p.errors.append(p.gpa, .{
987		.ExpectedSemiOrElse = .{ .token = p.tok_i },
	995	if (p.eatToken(.Semicolon)) \|_\| {
	996	if (continue_expr == 0) {
	997	return p.addNode(.{
	998	.tag = if (then_payload == 0) .WhileSimple else .WhileSimpleOptional,
	999	.main_token = while_token,
	1000	.data = .{
	1001	.lhs = condition,
	1002	.rhs = assign_expr,
	1003	},
	1004	});
	1005	} else {
	1006	return p.addNode(.{
	1007	.tag = if (then_payload == 0) .WhileCont else .WhileContOptional,
	1008	.main_token = while_token,
	1009	.data = .{
	1010	.lhs = condition,
	1011	.rhs = try p.addExtra(Node.WhileCont{
	1012	.continue_expr = continue_expr,
	1013	.then_expr = assign_expr,
	1014	}),
	1015	},
	1016	});
	1017	}
	1018	}
	1019	else_required = true;
	1020	break :blk assign_expr;
	1021	};
	1022	const else_token = p.eatToken(.Keyword_else) orelse {
	1023	if (else_required) {
	1024	return p.fail(.{ .ExpectedSemiOrElse = .{ .token = p.tok_i } });
	1025	}
	1026	if (continue_expr == 0) {
	1027	return p.addNode(.{
	1028	.tag = if (then_payload == 0) .WhileSimple else .WhileSimpleOptional,
	1029	.main_token = while_token,
	1030	.data = .{
	1031	.lhs = condition,
	1032	.rhs = then_expr,
	1033	},
	1034	});
	1035	} else {
	1036	return p.addNode(.{
	1037	.tag = if (then_payload == 0) .WhileCont else .WhileContOptional,
	1038	.main_token = while_token,
	1039	.data = .{
	1040	.lhs = condition,
	1041	.rhs = try p.addExtra(Node.WhileCont{
	1042	.continue_expr = continue_expr,
	1043	.then_expr = then_expr,
	1044	}),
	1045	},
	1046	});
	1047	}
	1048	};
	1049	const else_payload = try p.parsePayload();
	1050	const else_expr = try p.expectStatement();
	1051	const tag = if (else_payload != 0)
	1052	Node.Tag.WhileError
	1053	else if (then_payload != 0)
	1054	Node.Tag.WhileOptional
	1055	else
	1056	Node.Tag.While;
	1057	return p.addNode(.{
	1058	.tag = tag,
	1059	.main_token = while_token,
	1060	.data = .{
	1061	.lhs = condition,
	1062	.rhs = try p.addExtra(Node.While{
	1063	.continue_expr = continue_expr,
	1064	.then_expr = then_expr,
	1065	.else_expr = else_expr,
	1066	}),
	1067	},
988	1068	});
989
990		return node;
991	1069	}
992	1070
993	1071	/// BlockExprStatement
994	1072	/// <- BlockExpr
995	1073	/// / AssignExpr SEMICOLON
996		fn parseBlockExprStatement(p: Parser) !?Node {
997		if (try p.parseBlockExpr()) \|node\| return node;
998		if (try p.parseAssignExpr()) \|node\| {
	1074	fn parseBlockExprStatement(p: *Parser) !Node.Index {
	1075	const block_expr = try p.parseBlockExpr();
	1076	if (block_expr != 0) {
	1077	return block_expr;
	1078	}
	1079	const assign_expr = try p.parseAssignExpr();
	1080	if (assign_expr != 0) {
999	1081	_ = try p.expectTokenRecoverable(.Semicolon);
1000		return node;
	1082	return assign_expr;
1001	1083	}
1002		return null;
	1084	return null_node;
	1085	}
	1086
	1087	fn expectBlockExprStatement(p: *Parser) !Node.Index {
	1088	const node = try p.parseBlockExprStatement();
	1089	if (node == 0) {
	1090	return p.fail(.{ .ExpectedBlockOrExpression = .{ .token = p.tok_i } });
	1091	}
	1092	return node;
1003	1093	}
1004	1094
1005	1095	/// BlockExpr <- BlockLabel? Block
1006		fn parseBlockExpr(p: Parser) Error!?Node {
1007		var colon: TokenIndex = undefined;
1008		const label_token = p.parseBlockLabel(&colon);
1009		const block_node = (try p.parseBlock(label_token)) orelse {
1010		if (label_token) \|label\| {
1011		p.putBackToken(label + 1); // ":"
1012		p.putBackToken(label); // IDENTIFIER
1013		}
1014		return null;
1015		};
1016		return block_node;
	1096	fn parseBlockExpr(p: *Parser) Error!Node.Index {
	1097	switch (p.token_tags[p.tok_i]) {
	1098	.Identifier => {
	1099	if (p.token_tags[p.tok_i + 1] == .Colon and
	1100	p.token_tags[p.tok_i + 2] == .LBrace)
	1101	{
	1102	p.tok_i += 2;
	1103	return p.parseBlock();
	1104	} else {
	1105	return null_node;
	1106	}
	1107	},
	1108	.LBrace => return p.parseBlock(),
	1109	else => return null_node,
	1110	}
1017	1111	}
1018	1112
1019	1113	/// AssignExpr <- Expr (AssignOp Expr)?
1020		fn parseAssignExpr(p: Parser) !?Node {
1021		return p.parseBinOpExpr(parseAssignOp, parseExpr, .Once);
1022		}
	1114	/// AssignOp
	1115	/// <- ASTERISKEQUAL
	1116	/// / SLASHEQUAL
	1117	/// / PERCENTEQUAL
	1118	/// / PLUSEQUAL
	1119	/// / MINUSEQUAL
	1120	/// / LARROW2EQUAL
	1121	/// / RARROW2EQUAL
	1122	/// / AMPERSANDEQUAL
	1123	/// / CARETEQUAL
	1124	/// / PIPEEQUAL
	1125	/// / ASTERISKPERCENTEQUAL
	1126	/// / PLUSPERCENTEQUAL
	1127	/// / MINUSPERCENTEQUAL
	1128	/// / EQUAL
	1129	fn parseAssignExpr(p: *Parser) !Node.Index {
	1130	const expr = try p.parseExpr();
	1131	if (expr == 0) return null_node;
1023	1132
1024		/// Expr <- BoolOrExpr
1025		fn parseExpr(p: Parser) Error!?Node {
1026		return p.parsePrefixOpExpr(parseTry, parseBoolOrExpr);
	1133	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1134	.AsteriskEqual => .AssignMul,
	1135	.SlashEqual => .AssignDiv,
	1136	.PercentEqual => .AssignMod,
	1137	.PlusEqual => .AssignAdd,
	1138	.MinusEqual => .AssignSub,
	1139	.AngleBracketAngleBracketLeftEqual => .AssignBitShiftLeft,
	1140	.AngleBracketAngleBracketRightEqual => .AssignBitShiftRight,
	1141	.AmpersandEqual => .AssignBitAnd,
	1142	.CaretEqual => .AssignBitXor,
	1143	.PipeEqual => .AssignBitOr,
	1144	.AsteriskPercentEqual => .AssignMulWrap,
	1145	.PlusPercentEqual => .AssignAddWrap,
	1146	.MinusPercentEqual => .AssignSubWrap,
	1147	.Equal => .Assign,
	1148	else => return expr,
	1149	};
	1150	return p.addNode(.{
	1151	.tag = tag,
	1152	.main_token = p.nextToken(),
	1153	.data = .{
	1154	.lhs = expr,
	1155	.rhs = try p.expectExpr(),
	1156	},
	1157	});
1027	1158	}
1028	1159
1029		/// BoolOrExpr <- BoolAndExpr (KEYWORD_or BoolAndExpr)*
1030		fn parseBoolOrExpr(p: Parser) !?Node {
1031		return p.parseBinOpExpr(
1032		SimpleBinOpParseFn(.Keyword_or, .BoolOr),
1033		parseBoolAndExpr,
1034		.Infinitely,
1035		);
	1160	fn expectAssignExpr(p: *Parser) !Node.Index {
	1161	const expr = try p.parseAssignExpr();
	1162	if (expr == 0) {
	1163	return p.fail(.{ .ExpectedExprOrAssignment = .{ .token = p.tok_i } });
	1164	}
	1165	return expr;
1036	1166	}
1037	1167
1038		/// BoolAndExpr <- CompareExpr (KEYWORD_and CompareExpr)*
1039		fn parseBoolAndExpr(p: Parser) !?Node {
1040		return p.parseBinOpExpr(
1041		SimpleBinOpParseFn(.Keyword_and, .BoolAnd),
1042		parseCompareExpr,
1043		.Infinitely,
1044		);
	1168	/// Expr <- BoolOrExpr
	1169	fn parseExpr(p: *Parser) Error!Node.Index {
	1170	return p.parseBoolOrExpr();
1045	1171	}
1046	1172
1047		/// CompareExpr <- BitwiseExpr (CompareOp BitwiseExpr)?
1048		fn parseCompareExpr(p: Parser) !?Node {
1049		return p.parseBinOpExpr(parseCompareOp, parseBitwiseExpr, .Once);
	1173	fn expectExpr(p: *Parser) Error!Node.Index {
	1174	const node = try p.parseExpr();
	1175	if (node == 0) {
	1176	return p.fail(.{ .ExpectedExpr = .{ .token = p.tok_i } });
	1177	} else {
	1178	return node;
	1179	}
1050	1180	}
1051	1181
1052		/// BitwiseExpr <- BitShiftExpr (BitwiseOp BitShiftExpr)*
1053		fn parseBitwiseExpr(p: Parser) !?Node {
1054		return p.parseBinOpExpr(parseBitwiseOp, parseBitShiftExpr, .Infinitely);
1055		}
	1182	/// BoolOrExpr <- BoolAndExpr (KEYWORD_or BoolAndExpr)*
	1183	fn parseBoolOrExpr(p: *Parser) Error!Node.Index {
	1184	var res = try p.parseBoolAndExpr();
	1185	if (res == 0) return null_node;
1056	1186
1057		/// BitShiftExpr <- AdditionExpr (BitShiftOp AdditionExpr)*
1058		fn parseBitShiftExpr(p: Parser) !?Node {
1059		return p.parseBinOpExpr(parseBitShiftOp, parseAdditionExpr, .Infinitely);
	1187	while (true) {
	1188	switch (p.token_tags[p.tok_i]) {
	1189	.Keyword_or => {
	1190	const or_token = p.nextToken();
	1191	const rhs = try p.parseBoolAndExpr();
	1192	if (rhs == 0) {
	1193	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	1194	}
	1195	res = try p.addNode(.{
	1196	.tag = .BoolOr,
	1197	.main_token = or_token,
	1198	.data = .{
	1199	.lhs = res,
	1200	.rhs = rhs,
	1201	},
	1202	});
	1203	},
	1204	else => return res,
	1205	}
	1206	}
1060	1207	}
1061	1208
1062		/// AdditionExpr <- MultiplyExpr (AdditionOp MultiplyExpr)*
1063		fn parseAdditionExpr(p: Parser) !?Node {
1064		return p.parseBinOpExpr(parseAdditionOp, parseMultiplyExpr, .Infinitely);
1065		}
	1209	/// BoolAndExpr <- CompareExpr (KEYWORD_and CompareExpr)*
	1210	fn parseBoolAndExpr(p: *Parser) !Node.Index {
	1211	var res = try p.parseCompareExpr();
	1212	if (res == 0) return null_node;
1066	1213
1067		/// MultiplyExpr <- PrefixExpr (MultiplyOp PrefixExpr)*
1068		fn parseMultiplyExpr(p: Parser) !?Node {
1069		return p.parseBinOpExpr(parseMultiplyOp, parsePrefixExpr, .Infinitely);
	1214	while (true) {
	1215	switch (p.token_tags[p.tok_i]) {
	1216	.Keyword_and => {
	1217	const and_token = p.nextToken();
	1218	const rhs = try p.parseCompareExpr();
	1219	if (rhs == 0) {
	1220	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	1221	}
	1222	res = try p.addNode(.{
	1223	.tag = .BoolAnd,
	1224	.main_token = and_token,
	1225	.data = .{
	1226	.lhs = res,
	1227	.rhs = rhs,
	1228	},
	1229	});
	1230	},
	1231	else => return res,
	1232	}
	1233	}
1070	1234	}
1071	1235
1072		/// PrefixExpr <- PrefixOp* PrimaryExpr
1073		fn parsePrefixExpr(p: Parser) !?Node {
1074		return p.parsePrefixOpExpr(parsePrefixOp, parsePrimaryExpr);
	1236	/// CompareExpr <- BitwiseExpr (CompareOp BitwiseExpr)?
	1237	/// CompareOp
	1238	/// <- EQUALEQUAL
	1239	/// / EXCLAMATIONMARKEQUAL
	1240	/// / LARROW
	1241	/// / RARROW
	1242	/// / LARROWEQUAL
	1243	/// / RARROWEQUAL
	1244	fn parseCompareExpr(p: *Parser) !Node.Index {
	1245	const expr = try p.parseBitwiseExpr();
	1246	if (expr == 0) return null_node;
	1247
	1248	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1249	.EqualEqual => .EqualEqual,
	1250	.BangEqual => .BangEqual,
	1251	.AngleBracketLeft => .LessThan,
	1252	.AngleBracketRight => .GreaterThan,
	1253	.AngleBracketLeftEqual => .LessOrEqual,
	1254	.AngleBracketRightEqual => .GreaterOrEqual,
	1255	else => return expr,
	1256	};
	1257	return p.addNode(.{
	1258	.tag = tag,
	1259	.main_token = p.nextToken(),
	1260	.data = .{
	1261	.lhs = expr,
	1262	.rhs = try p.expectBitwiseExpr(),
	1263	},
	1264	});
1075	1265	}
1076	1266
1077		/// PrimaryExpr
1078		/// <- AsmExpr
1079		/// / IfExpr
1080		/// / KEYWORD_break BreakLabel? Expr?
1081		/// / KEYWORD_comptime Expr
1082		/// / KEYWORD_nosuspend Expr
1083		/// / KEYWORD_continue BreakLabel?
1084		/// / KEYWORD_resume Expr
1085		/// / KEYWORD_return Expr?
1086		/// / BlockLabel? LoopExpr
1087		/// / Block
1088		/// / CurlySuffixExpr
1089		fn parsePrimaryExpr(p: Parser) !?Node {
1090		if (try p.parseAsmExpr()) \|node\| return node;
1091		if (try p.parseIfExpr()) \|node\| return node;
1092
1093		if (p.eatToken(.Keyword_break)) \|token\| {
1094		const label = try p.parseBreakLabel();
1095		const expr_node = try p.parseExpr();
1096		const node = try Node.ControlFlowExpression.create(&p.arena.allocator, .{
1097		.tag = .Break,
1098		.ltoken = token,
1099		}, .{
1100		.label = label,
1101		.rhs = expr_node,
1102		});
1103		return &node.base;
1104		}
	1267	/// BitwiseExpr <- BitShiftExpr (BitwiseOp BitShiftExpr)*
	1268	/// BitwiseOp
	1269	/// <- AMPERSAND
	1270	/// / CARET
	1271	/// / PIPE
	1272	/// / KEYWORD_orelse
	1273	/// / KEYWORD_catch Payload?
	1274	fn parseBitwiseExpr(p: *Parser) !Node.Index {
	1275	var res = try p.parseBitShiftExpr();
	1276	if (res == 0) return null_node;
1105	1277
1106		if (p.eatToken(.Keyword_comptime)) \|token\| {
1107		const expr_node = try p.expectNode(parseExpr, .{
1108		.ExpectedExpr = .{ .token = p.tok_i },
1109		});
1110		const node = try p.arena.allocator.create(Node.Comptime);
1111		node.* = .{
1112		.doc_comments = null,
1113		.comptime_token = token,
1114		.expr = expr_node,
	1278	while (true) {
	1279	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1280	.Ampersand => .BitAnd,
	1281	.Caret => .BitXor,
	1282	.Pipe => .BitOr,
	1283	.Keyword_orelse => .OrElse,
	1284	.Keyword_catch => {
	1285	const catch_token = p.nextToken();
	1286	_ = try p.parsePayload();
	1287	const rhs = try p.parseBitShiftExpr();
	1288	if (rhs == 0) {
	1289	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	1290	}
	1291	res = try p.addNode(.{
	1292	.tag = .Catch,
	1293	.main_token = catch_token,
	1294	.data = .{
	1295	.lhs = res,
	1296	.rhs = rhs,
	1297	},
	1298	});
	1299	continue;
	1300	},
	1301	else => return res,
1115	1302	};
1116		return &node.base;
1117		}
1118
1119		if (p.eatToken(.Keyword_nosuspend)) \|token\| {
1120		const expr_node = try p.expectNode(parseExpr, .{
1121		.ExpectedExpr = .{ .token = p.tok_i },
	1303	res = try p.addNode(.{
	1304	.tag = tag,
	1305	.main_token = p.nextToken(),
	1306	.data = .{
	1307	.lhs = res,
	1308	.rhs = try p.expectBitShiftExpr(),
	1309	},
1122	1310	});
1123		const node = try p.arena.allocator.create(Node.Nosuspend);
1124		node.* = .{
1125		.nosuspend_token = token,
1126		.expr = expr_node,
1127		};
1128		return &node.base;
1129	1311	}
	1312	}
1130	1313
1131		if (p.eatToken(.Keyword_continue)) \|token\| {
1132		const label = try p.parseBreakLabel();
1133		const node = try Node.ControlFlowExpression.create(&p.arena.allocator, .{
1134		.tag = .Continue,
1135		.ltoken = token,
1136		}, .{
1137		.label = label,
1138		.rhs = null,
1139		});
1140		return &node.base;
	1314	fn expectBitwiseExpr(p: *Parser) Error!Node.Index {
	1315	const node = try p.parseBitwiseExpr();
	1316	if (node == 0) {
	1317	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	1318	} else {
	1319	return node;
1141	1320	}
	1321	}
1142	1322
1143		if (p.eatToken(.Keyword_resume)) \|token\| {
1144		const expr_node = try p.expectNode(parseExpr, .{
1145		.ExpectedExpr = .{ .token = p.tok_i },
1146		});
1147		const node = try p.arena.allocator.create(Node.SimplePrefixOp);
1148		node.* = .{
1149		.base = .{ .tag = .Resume },
1150		.op_token = token,
1151		.rhs = expr_node,
1152		};
1153		return &node.base;
1154		}
	1323	/// BitShiftExpr <- AdditionExpr (BitShiftOp AdditionExpr)*
	1324	/// BitShiftOp
	1325	/// <- LARROW2
	1326	/// / RARROW2
	1327	fn parseBitShiftExpr(p: *Parser) Error!Node.Index {
	1328	var res = try p.parseAdditionExpr();
	1329	if (res == 0) return null_node;
1155	1330
1156		if (p.eatToken(.Keyword_return)) \|token\| {
1157		const expr_node = try p.parseExpr();
1158		const node = try Node.ControlFlowExpression.create(&p.arena.allocator, .{
1159		.tag = .Return,
1160		.ltoken = token,
1161		}, .{
1162		.rhs = expr_node,
	1331	while (true) {
	1332	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1333	.AngleBracketAngleBracketLeft => .BitShiftLeft,
	1334	.AngleBracketAngleBracketRight => .BitShiftRight,
	1335	else => return res,
	1336	};
	1337	res = try p.addNode(.{
	1338	.tag = tag,
	1339	.main_token = p.nextToken(),
	1340	.data = .{
	1341	.lhs = res,
	1342	.rhs = try p.expectAdditionExpr(),
	1343	},
1163	1344	});
1164		return &node.base;
1165	1345	}
	1346	}
1166	1347
1167		var colon: TokenIndex = undefined;
1168		const label = p.parseBlockLabel(&colon);
1169		if (try p.parseLoopExpr()) \|node\| {
1170		if (node.cast(Node.For)) \|for_node\| {
1171		for_node.label = label;
1172		} else if (node.cast(Node.While)) \|while_node\| {
1173		while_node.label = label;
1174		} else unreachable;
	1348	fn expectBitShiftExpr(p: *Parser) Error!Node.Index {
	1349	const node = try p.parseBitShiftExpr();
	1350	if (node == 0) {
	1351	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	1352	} else {
1175	1353	return node;
1176	1354	}
1177		if (label) \|token\| {
1178		p.putBackToken(token + 1); // ":"
1179		p.putBackToken(token); // IDENTIFIER
1180		}
1181
1182		if (try p.parseBlock(null)) \|node\| return node;
1183		if (try p.parseCurlySuffixExpr()) \|node\| return node;
1184
1185		return null;
1186		}
1187
1188		/// IfExpr <- IfPrefix Expr (KEYWORD_else Payload? Expr)?
1189		fn parseIfExpr(p: Parser) !?Node {
1190		return p.parseIf(parseExpr);
1191	1355	}
1192	1356
1193		/// Block <- LBRACE Statement* RBRACE
1194		fn parseBlock(p: Parser, label_token: ?TokenIndex) !?Node {
1195		const lbrace = p.eatToken(.LBrace) orelse return null;
1196
1197		var statements = std.ArrayList(*Node).init(p.gpa);
1198		defer statements.deinit();
	1357	/// AdditionExpr <- MultiplyExpr (AdditionOp MultiplyExpr)*
	1358	/// AdditionOp
	1359	/// <- PLUS
	1360	/// / MINUS
	1361	/// / PLUS2
	1362	/// / PLUSPERCENT
	1363	/// / MINUSPERCENT
	1364	fn parseAdditionExpr(p: *Parser) Error!Node.Index {
	1365	var res = try p.parseMultiplyExpr();
	1366	if (res == 0) return null_node;
1199	1367
1200	1368	while (true) {
1201		const statement = (p.parseStatement() catch \|err\| switch (err) {
1202		error.OutOfMemory => return error.OutOfMemory,
1203		error.ParseError => {
1204		// try to skip to the next statement
1205		p.findNextStmt();
1206		continue;
	1369	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1370	.Plus => .Add,
	1371	.Minus => .Sub,
	1372	.PlusPlus => .ArrayCat,
	1373	.PlusPercent => .AddWrap,
	1374	.MinusPercent => .SubWrap,
	1375	else => return res,
	1376	};
	1377	res = try p.addNode(.{
	1378	.tag = tag,
	1379	.main_token = p.nextToken(),
	1380	.data = .{
	1381	.lhs = res,
	1382	.rhs = try p.expectMultiplyExpr(),
1207	1383	},
1208		}) orelse break;
1209		try statements.append(statement);
	1384	});
1210	1385	}
	1386	}
1211	1387
1212		const rbrace = try p.expectToken(.RBrace);
1213
1214		const statements_len = @intCast(NodeIndex, statements.items.len);
1215
1216		if (label_token) \|label\| {
1217		const block_node = try Node.LabeledBlock.alloc(&p.arena.allocator, statements_len);
1218		block_node.* = .{
1219		.label = label,
1220		.lbrace = lbrace,
1221		.statements_len = statements_len,
1222		.rbrace = rbrace,
1223		};
1224		std.mem.copy(*Node, block_node.statements(), statements.items);
1225		return &block_node.base;
1226		} else {
1227		const block_node = try Node.Block.alloc(&p.arena.allocator, statements_len);
1228		block_node.* = .{
1229		.lbrace = lbrace,
1230		.statements_len = statements_len,
1231		.rbrace = rbrace,
1232		};
1233		std.mem.copy(*Node, block_node.statements(), statements.items);
1234		return &block_node.base;
	1388	fn expectAdditionExpr(p: *Parser) Error!Node.Index {
	1389	const node = try p.parseAdditionExpr();
	1390	if (node == 0) {
	1391	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
1235	1392	}
	1393	return node;
1236	1394	}
1237	1395
1238		/// LoopExpr <- KEYWORD_inline? (ForExpr / WhileExpr)
1239		fn parseLoopExpr(p: Parser) !?Node {
1240		const inline_token = p.eatToken(.Keyword_inline);
	1396	/// MultiplyExpr <- PrefixExpr (MultiplyOp PrefixExpr)*
	1397	/// MultiplyOp
	1398	/// <- PIPE2
	1399	/// / ASTERISK
	1400	/// / SLASH
	1401	/// / PERCENT
	1402	/// / ASTERISK2
	1403	/// / ASTERISKPERCENT
	1404	fn parseMultiplyExpr(p: *Parser) Error!Node.Index {
	1405	var res = try p.parsePrefixExpr();
	1406	if (res == 0) return null_node;
1241	1407
1242		if (try p.parseForExpr()) \|node\| {
1243		node.cast(Node.For).?.inline_token = inline_token;
1244		return node;
	1408	while (true) {
	1409	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1410	.PipePipe => .MergeErrorSets,
	1411	.Asterisk => .Mul,
	1412	.Slash => .Div,
	1413	.Percent => .Mod,
	1414	.AsteriskAsterisk => .ArrayMult,
	1415	.AsteriskPercent => .MulWrap,
	1416	else => return res,
	1417	};
	1418	res = try p.addNode(.{
	1419	.tag = tag,
	1420	.main_token = p.nextToken(),
	1421	.data = .{
	1422	.lhs = res,
	1423	.rhs = try p.expectPrefixExpr(),
	1424	},
	1425	});
1245	1426	}
	1427	}
1246	1428
1247		if (try p.parseWhileExpr()) \|node\| {
1248		node.cast(Node.While).?.inline_token = inline_token;
1249		return node;
	1429	fn expectMultiplyExpr(p: *Parser) Error!Node.Index {
	1430	const node = try p.parseMultiplyExpr();
	1431	if (node == 0) {
	1432	return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
1250	1433	}
1251
1252		if (inline_token == null) return null;
1253
1254		// If we've seen "inline", there should have been a "for" or "while"
1255		try p.errors.append(p.gpa, .{
1256		.ExpectedInlinable = .{ .token = p.tok_i },
1257		});
1258		return error.ParseError;
	1434	return node;
1259	1435	}
1260	1436
1261		/// ForExpr <- ForPrefix Expr (KEYWORD_else Expr)?
1262		fn parseForExpr(p: Parser) !?Node {
1263		const node = (try p.parseForPrefix()) orelse return null;
1264		const for_prefix = node.cast(Node.For).?;
1265
1266		const body_node = try p.expectNode(parseExpr, .{
1267		.ExpectedExpr = .{ .token = p.tok_i },
	1437	/// PrefixExpr <- PrefixOp* PrimaryExpr
	1438	/// PrefixOp
	1439	/// <- EXCLAMATIONMARK
	1440	/// / MINUS
	1441	/// / TILDE
	1442	/// / MINUSPERCENT
	1443	/// / AMPERSAND
	1444	/// / KEYWORD_try
	1445	/// / KEYWORD_await
	1446	fn parsePrefixExpr(p: *Parser) Error!Node.Index {
	1447	const tag: Node.Tag = switch (p.token_tags[p.tok_i]) {
	1448	.Bang => .BoolNot,
	1449	.Minus => .Negation,
	1450	.Tilde => .BitNot,
	1451	.MinusPercent => .NegationWrap,
	1452	.Ampersand => .AddressOf,
	1453	.Keyword_try => .Try,
	1454	.Keyword_await => .Await,
	1455	else => return p.parsePrimaryExpr(),
	1456	};
	1457	return p.addNode(.{
	1458	.tag = tag,
	1459	.main_token = p.nextToken(),
	1460	.data = .{
	1461	.lhs = try p.expectPrefixExpr(),
	1462	.rhs = undefined,
	1463	},
1268	1464	});
1269		for_prefix.body = body_node;
1270
1271		if (p.eatToken(.Keyword_else)) \|else_token\| {
1272		const body = try p.expectNode(parseExpr, .{
1273		.ExpectedExpr = .{ .token = p.tok_i },
1274		});
1275
1276		const else_node = try p.arena.allocator.create(Node.Else);
1277		else_node.* = .{
1278		.else_token = else_token,
1279		.payload = null,
1280		.body = body,
1281		};
	1465	}
1282	1466
1283		for_prefix.@"else" = else_node;
	1467	fn expectPrefixExpr(p: *Parser) Error!Node.Index {
	1468	const node = try p.parsePrefixExpr();
	1469	if (node == 0) {
	1470	return p.fail(.{ .ExpectedPrefixExpr = .{ .token = p.tok_i } });
1284	1471	}
1285
1286	1472	return node;
1287	1473	}
1288	1474
1289		/// WhileExpr <- WhilePrefix Expr (KEYWORD_else Payload? Expr)?
1290		fn parseWhileExpr(p: Parser) !?Node {
1291		const node = (try p.parseWhilePrefix()) orelse return null;
1292		const while_prefix = node.cast(Node.While).?;
1293
1294		const body_node = try p.expectNode(parseExpr, .{
1295		.ExpectedExpr = .{ .token = p.tok_i },
1296		});
1297		while_prefix.body = body_node;
1298
1299		if (p.eatToken(.Keyword_else)) \|else_token\| {
1300		const payload = try p.parsePayload();
1301		const body = try p.expectNode(parseExpr, .{
1302		.ExpectedExpr = .{ .token = p.tok_i },
1303		});
1304
1305		const else_node = try p.arena.allocator.create(Node.Else);
1306		else_node.* = .{
1307		.else_token = else_token,
1308		.payload = payload,
1309		.body = body,
1310		};
1311
1312		while_prefix.@"else" = else_node;
	1475	/// TypeExpr <- PrefixTypeOp* ErrorUnionExpr
	1476	/// PrefixTypeOp
	1477	/// <- QUESTIONMARK
	1478	/// / KEYWORD_anyframe MINUSRARROW
	1479	/// / ArrayTypeStart (ByteAlign / KEYWORD_const / KEYWORD_volatile / KEYWORD_allowzero)*
	1480	/// / PtrTypeStart (KEYWORD_align LPAREN Expr (COLON INTEGER COLON INTEGER)? RPAREN / KEYWORD_const / KEYWORD_volatile / KEYWORD_allowzero)*
	1481	/// PtrTypeStart
	1482	/// <- ASTERISK
	1483	/// / ASTERISK2
	1484	/// / LBRACKET ASTERISK (LETTERC / COLON Expr)? RBRACKET
	1485	/// ArrayTypeStart <- LBRACKET Expr? (COLON Expr)? RBRACKET
	1486	fn parseTypeExpr(p: *Parser) Error!Node.Index {
	1487	switch (p.token_tags[p.tok_i]) {
	1488	.QuestionMark => return p.addNode(.{
	1489	.tag = .OptionalType,
	1490	.main_token = p.nextToken(),
	1491	.data = .{
	1492	.lhs = try p.expectTypeExpr(),
	1493	.rhs = undefined,
	1494	},
	1495	}),
	1496	.Keyword_anyframe => switch (p.token_tags[p.tok_i + 1]) {
	1497	.Arrow => return p.addNode(.{
	1498	.tag = .AnyFrameType,
	1499	.main_token = p.nextToken(),
	1500	.data = .{
	1501	.lhs = p.nextToken(),
	1502	.rhs = try p.expectTypeExpr(),
	1503	},
	1504	}),
	1505	else => return p.parseErrorUnionExpr(),
	1506	},
	1507	.Asterisk => {
	1508	const asterisk = p.nextToken();
	1509	const mods = try p.parsePtrModifiers();
	1510	const elem_type = try p.expectTypeExpr();
	1511	if (mods.bit_range_start == 0) {
	1512	return p.addNode(.{
	1513	.tag = .PtrTypeAligned,
	1514	.main_token = asterisk,
	1515	.data = .{
	1516	.lhs = mods.align_node,
	1517	.rhs = elem_type,
	1518	},
	1519	});
	1520	} else {
	1521	return p.addNode(.{
	1522	.tag = .PtrType,
	1523	.main_token = asterisk,
	1524	.data = .{
	1525	.lhs = try p.addExtra(Node.PtrType{
	1526	.sentinel = 0,
	1527	.align_node = mods.align_node,
	1528	.bit_range_start = mods.bit_range_start,
	1529	.bit_range_end = mods.bit_range_end,
	1530	}),
	1531	.rhs = elem_type,
	1532	},
	1533	});
	1534	}
	1535	},
	1536	.AsteriskAsterisk => {
	1537	const asterisk = p.nextToken();
	1538	const mods = try p.parsePtrModifiers();
	1539	const elem_type = try p.expectTypeExpr();
	1540	const inner: Node.Index = inner: {
	1541	if (mods.bit_range_start == 0) {
	1542	break :inner try p.addNode(.{
	1543	.tag = .PtrTypeAligned,
	1544	.main_token = asterisk,
	1545	.data = .{
	1546	.lhs = mods.align_node,
	1547	.rhs = elem_type,
	1548	},
	1549	});
	1550	} else {
	1551	break :inner try p.addNode(.{
	1552	.tag = .PtrType,
	1553	.main_token = asterisk,
	1554	.data = .{
	1555	.lhs = try p.addExtra(Node.PtrType{
	1556	.sentinel = 0,
	1557	.align_node = mods.align_node,
	1558	.bit_range_start = mods.bit_range_start,
	1559	.bit_range_end = mods.bit_range_end,
	1560	}),
	1561	.rhs = elem_type,
	1562	},
	1563	});
	1564	}
	1565	};
	1566	return p.addNode(.{
	1567	.tag = .PtrTypeAligned,
	1568	.main_token = asterisk,
	1569	.data = .{
	1570	.lhs = 0,
	1571	.rhs = inner,
	1572	},
	1573	});
	1574	},
	1575	.LBracket => switch (p.token_tags[p.tok_i + 1]) {
	1576	.Asterisk => {
	1577	const lbracket = p.nextToken();
	1578	const asterisk = p.nextToken();
	1579	var sentinel: Node.Index = 0;
	1580	prefix: {
	1581	if (p.eatToken(.Identifier)) \|ident\| {
	1582	const token_slice = p.source[p.token_starts[ident]..][0..2];
	1583	if (!std.mem.eql(u8, token_slice, "c]")) {
	1584	p.tok_i -= 1;
	1585	} else {
	1586	break :prefix;
	1587	}
	1588	}
	1589	if (p.eatToken(.Colon)) \|_\| {
	1590	sentinel = try p.expectExpr();
	1591	}
	1592	}
	1593	_ = try p.expectToken(.RBracket);
	1594	const mods = try p.parsePtrModifiers();
	1595	const elem_type = try p.expectTypeExpr();
	1596	if (mods.bit_range_start == 0) {
	1597	if (sentinel == 0) {
	1598	return p.addNode(.{
	1599	.tag = .PtrTypeAligned,
	1600	.main_token = asterisk,
	1601	.data = .{
	1602	.lhs = mods.align_node,
	1603	.rhs = elem_type,
	1604	},
	1605	});
	1606	} else if (mods.align_node == 0) {
	1607	return p.addNode(.{
	1608	.tag = .PtrTypeSentinel,
	1609	.main_token = asterisk,
	1610	.data = .{
	1611	.lhs = sentinel,
	1612	.rhs = elem_type,
	1613	},
	1614	});
	1615	} else {
	1616	return p.addNode(.{
	1617	.tag = .SliceType,
	1618	.main_token = asterisk,
	1619	.data = .{
	1620	.lhs = try p.addExtra(.{
	1621	.sentinel = sentinel,
	1622	.align_node = mods.align_node,
	1623	}),
	1624	.rhs = elem_type,
	1625	},
	1626	});
	1627	}
	1628	} else {
	1629	return p.addNode(.{
	1630	.tag = .PtrType,
	1631	.main_token = asterisk,
	1632	.data = .{
	1633	.lhs = try p.addExtra(.{
	1634	.sentinel = sentinel,
	1635	.align_node = mods.align_node,
	1636	.bit_range_start = mods.bit_range_start,
	1637	.bit_range_end = mods.bit_range_end,
	1638	}),
	1639	.rhs = elem_type,
	1640	},
	1641	});
	1642	}
	1643	},
	1644	else => {
	1645	const lbracket = p.nextToken();
	1646	const len_expr = try p.parseExpr();
	1647	const sentinel: Node.Index = if (p.eatToken(.Colon)) \|_\|
	1648	try p.expectExpr()
	1649	else
	1650	0;
	1651	_ = try p.expectToken(.RBracket);
	1652	const mods = try p.parsePtrModifiers();
	1653	const elem_type = try p.expectTypeExpr();
	1654	if (mods.bit_range_start != 0) {
	1655	@panic("TODO implement this error");
	1656	//try p.warn(.{
	1657	// .BitRangeInvalid = .{ .node = mods.bit_range_start },
	1658	//});
	1659	}
	1660	if (len_expr == 0) {
	1661	if (sentinel == 0) {
	1662	return p.addNode(.{
	1663	.tag = .PtrTypeAligned,
	1664	.main_token = lbracket,
	1665	.data = .{
	1666	.lhs = mods.align_node,
	1667	.rhs = elem_type,
	1668	},
	1669	});
	1670	} else if (mods.align_node == 0) {
	1671	return p.addNode(.{
	1672	.tag = .PtrTypeSentinel,
	1673	.main_token = lbracket,
	1674	.data = .{
	1675	.lhs = sentinel,
	1676	.rhs = elem_type,
	1677	},
	1678	});
	1679	} else {
	1680	return p.addNode(.{
	1681	.tag = .SliceType,
	1682	.main_token = lbracket,
	1683	.data = .{
	1684	.lhs = try p.addExtra(.{
	1685	.sentinel = sentinel,
	1686	.align_node = mods.align_node,
	1687	}),
	1688	.rhs = elem_type,
	1689	},
	1690	});
	1691	}
	1692	} else {
	1693	if (mods.align_node != 0) {
	1694	@panic("TODO implement this error");
	1695	//try p.warn(.{
	1696	// .AlignInvalid = .{ .node = mods.align_node },
	1697	//});
	1698	}
	1699	if (sentinel == 0) {
	1700	return p.addNode(.{
	1701	.tag = .ArrayType,
	1702	.main_token = lbracket,
	1703	.data = .{
	1704	.lhs = len_expr,
	1705	.rhs = elem_type,
	1706	},
	1707	});
	1708	} else {
	1709	return p.addNode(.{
	1710	.tag = .ArrayTypeSentinel,
	1711	.main_token = lbracket,
	1712	.data = .{
	1713	.lhs = len_expr,
	1714	.rhs = try p.addExtra(.{
	1715	.elem_type = elem_type,
	1716	.sentinel = sentinel,
	1717	}),
	1718	},
	1719	});
	1720	}
	1721	}
	1722	},
	1723	},
	1724	else => return p.parseErrorUnionExpr(),
1313	1725	}
	1726	}
1314	1727
	1728	fn expectTypeExpr(p: *Parser) Error!Node.Index {
	1729	const node = try p.parseTypeExpr();
	1730	if (node == 0) {
	1731	return p.fail(.{ .ExpectedTypeExpr = .{ .token = p.tok_i } });
	1732	}
1315	1733	return node;
1316	1734	}
1317	1735
1318		/// CurlySuffixExpr <- TypeExpr InitList?
1319		fn parseCurlySuffixExpr(p: Parser) !?Node {
1320		const lhs = (try p.parseTypeExpr()) orelse return null;
1321		const suffix_op = (try p.parseInitList(lhs)) orelse return lhs;
1322		return suffix_op;
	1736	/// PrimaryExpr
	1737	/// <- AsmExpr
	1738	/// / IfExpr
	1739	/// / KEYWORD_break BreakLabel? Expr?
	1740	/// / KEYWORD_comptime Expr
	1741	/// / KEYWORD_nosuspend Expr
	1742	/// / KEYWORD_continue BreakLabel?
	1743	/// / KEYWORD_resume Expr
	1744	/// / KEYWORD_return Expr?
	1745	/// / BlockLabel? LoopExpr
	1746	/// / Block
	1747	/// / CurlySuffixExpr
	1748	fn parsePrimaryExpr(p: *Parser) !Node.Index {
	1749	switch (p.token_tags[p.tok_i]) {
	1750	.Keyword_asm => return p.parseAsmExpr(),
	1751	.Keyword_if => return p.parseIfExpr(),
	1752	.Keyword_break => {
	1753	p.tok_i += 1;
	1754	return p.addNode(.{
	1755	.tag = .Break,
	1756	.main_token = p.tok_i - 1,
	1757	.data = .{
	1758	.lhs = try p.parseBreakLabel(),
	1759	.rhs = try p.parseExpr(),
	1760	},
	1761	});
	1762	},
	1763	.Keyword_continue => {
	1764	p.tok_i += 1;
	1765	return p.addNode(.{
	1766	.tag = .Continue,
	1767	.main_token = p.tok_i - 1,
	1768	.data = .{
	1769	.lhs = try p.parseBreakLabel(),
	1770	.rhs = undefined,
	1771	},
	1772	});
	1773	},
	1774	.Keyword_comptime => {
	1775	p.tok_i += 1;
	1776	return p.addNode(.{
	1777	.tag = .Comptime,
	1778	.main_token = p.tok_i - 1,
	1779	.data = .{
	1780	.lhs = try p.expectExpr(),
	1781	.rhs = undefined,
	1782	},
	1783	});
	1784	},
	1785	.Keyword_nosuspend => {
	1786	p.tok_i += 1;
	1787	return p.addNode(.{
	1788	.tag = .Nosuspend,
	1789	.main_token = p.tok_i - 1,
	1790	.data = .{
	1791	.lhs = try p.expectExpr(),
	1792	.rhs = undefined,
	1793	},
	1794	});
	1795	},
	1796	.Keyword_resume => {
	1797	p.tok_i += 1;
	1798	return p.addNode(.{
	1799	.tag = .Resume,
	1800	.main_token = p.tok_i - 1,
	1801	.data = .{
	1802	.lhs = try p.expectExpr(),
	1803	.rhs = undefined,
	1804	},
	1805	});
	1806	},
	1807	.Keyword_return => {
	1808	p.tok_i += 1;
	1809	return p.addNode(.{
	1810	.tag = .Return,
	1811	.main_token = p.tok_i - 1,
	1812	.data = .{
	1813	.lhs = try p.parseExpr(),
	1814	.rhs = undefined,
	1815	},
	1816	});
	1817	},
	1818	.Identifier => {
	1819	if (p.token_tags[p.tok_i + 1] == .Colon) {
	1820	switch (p.token_tags[p.tok_i + 2]) {
	1821	.Keyword_inline => {
	1822	p.tok_i += 3;
	1823	switch (p.token_tags[p.tok_i]) {
	1824	.Keyword_for => return p.parseForExpr(),
	1825	.Keyword_while => return p.parseWhileExpr(),
	1826	else => return p.fail(.{
	1827	.ExpectedInlinable = .{ .token = p.tok_i },
	1828	}),
	1829	}
	1830	},
	1831	.Keyword_for => {
	1832	p.tok_i += 2;
	1833	return p.parseForExpr();
	1834	},
	1835	.Keyword_while => {
	1836	p.tok_i += 2;
	1837	return p.parseWhileExpr();
	1838	},
	1839	.LBrace => {
	1840	p.tok_i += 2;
	1841	return p.parseBlock();
	1842	},
	1843	else => return p.parseCurlySuffixExpr(),
	1844	}
	1845	} else {
	1846	return p.parseCurlySuffixExpr();
	1847	}
	1848	},
	1849	.Keyword_inline => {
	1850	p.tok_i += 2;
	1851	switch (p.token_tags[p.tok_i]) {
	1852	.Keyword_for => return p.parseForExpr(),
	1853	.Keyword_while => return p.parseWhileExpr(),
	1854	else => return p.fail(.{
	1855	.ExpectedInlinable = .{ .token = p.tok_i },
	1856	}),
	1857	}
	1858	},
	1859	.Keyword_for => return p.parseForExpr(),
	1860	.Keyword_while => return p.parseWhileExpr(),
	1861	.LBrace => return p.parseBlock(),
	1862	else => return p.parseCurlySuffixExpr(),
	1863	}
1323	1864	}
1324	1865
1325		/// InitList
1326		/// <- LBRACE FieldInit (COMMA FieldInit)* COMMA? RBRACE
1327		/// / LBRACE Expr (COMMA Expr)* COMMA? RBRACE
1328		/// / LBRACE RBRACE
1329		fn parseInitList(p: Parser, lhs: Node) !?*Node {
1330		const lbrace = p.eatToken(.LBrace) orelse return null;
1331		var init_list = std.ArrayList(*Node).init(p.gpa);
1332		defer init_list.deinit();
	1866	/// IfExpr <- IfPrefix Expr (KEYWORD_else Payload? Expr)?
	1867	fn parseIfExpr(p: *Parser) !Node.Index {
	1868	return p.parseIf(parseExpr);
	1869	}
1333	1870
1334		if (try p.parseFieldInit()) \|field_init\| {
1335		try init_list.append(field_init);
1336		while (p.eatToken(.Comma)) \|_\| {
1337		const next = (try p.parseFieldInit()) orelse break;
1338		try init_list.append(next);
1339		}
1340		const node = try Node.StructInitializer.alloc(&p.arena.allocator, init_list.items.len);
1341		node.* = .{
1342		.lhs = lhs,
1343		.rtoken = try p.expectToken(.RBrace),
1344		.list_len = init_list.items.len,
1345		};
1346		std.mem.copy(*Node, node.list(), init_list.items);
1347		return &node.base;
1348		}
	1871	/// Block <- LBRACE Statement* RBRACE
	1872	fn parseBlock(p: *Parser) !Node.Index {
	1873	const lbrace = p.eatToken(.LBrace) orelse return null_node;
1349	1874
1350		if (try p.parseExpr()) \|expr\| {
1351		try init_list.append(expr);
1352		while (p.eatToken(.Comma)) \|_\| {
1353		const next = (try p.parseExpr()) orelse break;
1354		try init_list.append(next);
1355		}
1356		const node = try Node.ArrayInitializer.alloc(&p.arena.allocator, init_list.items.len);
1357		node.* = .{
1358		.lhs = lhs,
1359		.rtoken = try p.expectToken(.RBrace),
1360		.list_len = init_list.items.len,
1361		};
1362		std.mem.copy(*Node, node.list(), init_list.items);
1363		return &node.base;
	1875	var statements = std.ArrayList(Node.Index).init(p.gpa);
	1876	defer statements.deinit();
	1877
	1878	while (true) {
	1879	const statement = (p.parseStatement() catch \|err\| switch (err) {
	1880	error.OutOfMemory => return error.OutOfMemory,
	1881	error.ParseError => {
	1882	// try to skip to the next statement
	1883	p.findNextStmt();
	1884	continue;
	1885	},
	1886	});
	1887	if (statement == 0) break;
	1888	try statements.append(statement);
1364	1889	}
1365	1890
1366		const node = try p.arena.allocator.create(Node.StructInitializer);
1367		node.* = .{
1368		.lhs = lhs,
1369		.rtoken = try p.expectToken(.RBrace),
1370		.list_len = 0,
	1891	const rbrace = try p.expectToken(.RBrace);
	1892	const statements_span = try p.listToSpan(statements.items);
	1893
	1894	return p.addNode(.{
	1895	.tag = .Block,
	1896	.main_token = lbrace,
	1897	.data = .{
	1898	.lhs = statements_span.start,
	1899	.rhs = statements_span.end,
	1900	},
	1901	});
	1902	}
	1903
	1904	/// ForPrefix <- KEYWORD_for LPAREN Expr RPAREN PtrIndexPayload
	1905	/// ForExpr <- ForPrefix Expr (KEYWORD_else Expr)?
	1906	fn parseForExpr(p: *Parser) !Node.Index {
	1907	const for_token = p.eatToken(.Keyword_for) orelse return null_node;
	1908	_ = try p.expectToken(.LParen);
	1909	const array_expr = try p.expectExpr();
	1910	_ = try p.expectToken(.RParen);
	1911	_ = try p.parsePtrIndexPayload();
	1912
	1913	const then_expr = try p.expectExpr();
	1914	const else_token = p.eatToken(.Keyword_else) orelse {
	1915	return p.addNode(.{
	1916	.tag = .ForSimple,
	1917	.main_token = for_token,
	1918	.data = .{
	1919	.lhs = array_expr,
	1920	.rhs = then_expr,
	1921	},
	1922	});
	1923	};
	1924	const else_expr = try p.expectExpr();
	1925	return p.addNode(.{
	1926	.tag = .For,
	1927	.main_token = for_token,
	1928	.data = .{
	1929	.lhs = array_expr,
	1930	.rhs = try p.addExtra(Node.If{
	1931	.then_expr = then_expr,
	1932	.else_expr = else_expr,
	1933	}),
	1934	},
	1935	});
	1936	}
	1937
	1938	/// WhilePrefix <- KEYWORD_while LPAREN Expr RPAREN PtrPayload? WhileContinueExpr?
	1939	/// WhileExpr <- WhilePrefix Expr (KEYWORD_else Payload? Expr)?
	1940	fn parseWhileExpr(p: *Parser) !Node.Index {
	1941	const while_token = p.eatToken(.Keyword_while) orelse return null_node;
	1942	_ = try p.expectToken(.LParen);
	1943	const condition = try p.expectExpr();
	1944	_ = try p.expectToken(.RParen);
	1945	const then_payload = try p.parsePtrPayload();
	1946	const continue_expr = try p.parseWhileContinueExpr();
	1947
	1948	const then_expr = try p.expectExpr();
	1949	const else_token = p.eatToken(.Keyword_else) orelse {
	1950	if (continue_expr == 0) {
	1951	return p.addNode(.{
	1952	.tag = if (then_payload == 0) .WhileSimple else .WhileSimpleOptional,
	1953	.main_token = while_token,
	1954	.data = .{
	1955	.lhs = condition,
	1956	.rhs = then_expr,
	1957	},
	1958	});
	1959	} else {
	1960	return p.addNode(.{
	1961	.tag = if (then_payload == 0) .WhileCont else .WhileContOptional,
	1962	.main_token = while_token,
	1963	.data = .{
	1964	.lhs = condition,
	1965	.rhs = try p.addExtra(Node.WhileCont{
	1966	.continue_expr = continue_expr,
	1967	.then_expr = then_expr,
	1968	}),
	1969	},
	1970	});
	1971	}
1371	1972	};
1372		return &node.base;
	1973	const else_payload = try p.parsePayload();
	1974	const else_expr = try p.expectExpr();
	1975	const tag = if (else_payload != 0)
	1976	Node.Tag.WhileError
	1977	else if (then_payload != 0)
	1978	Node.Tag.WhileOptional
	1979	else
	1980	Node.Tag.While;
	1981	return p.addNode(.{
	1982	.tag = tag,
	1983	.main_token = while_token,
	1984	.data = .{
	1985	.lhs = condition,
	1986	.rhs = try p.addExtra(Node.While{
	1987	.continue_expr = continue_expr,
	1988	.then_expr = then_expr,
	1989	.else_expr = else_expr,
	1990	}),
	1991	},
	1992	});
1373	1993	}
1374	1994
	1995	/// CurlySuffixExpr <- TypeExpr InitList?
1375	1996	/// InitList
1376	1997	/// <- LBRACE FieldInit (COMMA FieldInit)* COMMA? RBRACE
1377	1998	/// / LBRACE Expr (COMMA Expr)* COMMA? RBRACE
1378	1999	/// / LBRACE RBRACE
1379		fn parseAnonInitList(p: Parser, dot: TokenIndex) !?Node {
1380		const lbrace = p.eatToken(.LBrace) orelse return null;
1381		var init_list = std.ArrayList(*Node).init(p.gpa);
1382		defer init_list.deinit();
	2000	fn parseCurlySuffixExpr(p: *Parser) !Node.Index {
	2001	const lhs = try p.parseTypeExpr();
	2002	if (lhs == 0) return null_node;
	2003	const lbrace = p.eatToken(.LBrace) orelse return lhs;
	2004
	2005	// If there are 0 or 1 items, we can use ArrayInitOne/StructInitOne;
	2006	// otherwise we use the full ArrayInit/StructInit.
	2007
	2008	if (p.eatToken(.RBrace)) \|_\| {
	2009	return p.addNode(.{
	2010	.tag = .StructInitOne,
	2011	.main_token = lbrace,
	2012	.data = .{
	2013	.lhs = lhs,
	2014	.rhs = 0,
	2015	},
	2016	});
	2017	}
	2018	const field_init = try p.parseFieldInit();
	2019	if (field_init != 0) {
	2020	const comma_one = p.eatToken(.Comma);
	2021	if (p.eatToken(.RBrace)) \|_\| {
	2022	return p.addNode(.{
	2023	.tag = .StructInitOne,
	2024	.main_token = lbrace,
	2025	.data = .{
	2026	.lhs = lhs,
	2027	.rhs = field_init,
	2028	},
	2029	});
	2030	}
	2031
	2032	var init_list = std.ArrayList(Node.Index).init(p.gpa);
	2033	defer init_list.deinit();
1383	2034
1384		if (try p.parseFieldInit()) \|field_init\| {
1385	2035	try init_list.append(field_init);
1386		while (p.eatToken(.Comma)) \|_\| {
1387		const next = (try p.parseFieldInit()) orelse break;
1388		try init_list.append(next);
1389		}
1390		const node = try Node.StructInitializerDot.alloc(&p.arena.allocator, init_list.items.len);
1391		node.* = .{
1392		.dot = dot,
1393		.rtoken = try p.expectToken(.RBrace),
1394		.list_len = init_list.items.len,
1395		};
1396		std.mem.copy(*Node, node.list(), init_list.items);
1397		return &node.base;
1398		}
1399	2036
1400		if (try p.parseExpr()) \|expr\| {
1401		try init_list.append(expr);
1402		while (p.eatToken(.Comma)) \|_\| {
1403		const next = (try p.parseExpr()) orelse break;
	2037	while (true) {
	2038	const next = try p.expectFieldInit();
1404	2039	try init_list.append(next);
	2040
	2041	switch (p.token_tags[p.nextToken()]) {
	2042	.Comma => {
	2043	if (p.eatToken(.RBrace)) \|_\| break;
	2044	continue;
	2045	},
	2046	.RBrace => break,
	2047	.Colon, .RParen, .RBracket => {
	2048	p.tok_i -= 1;
	2049	return p.fail(.{
	2050	.ExpectedToken = .{
	2051	.token = p.tok_i,
	2052	.expected_id = .RBrace,
	2053	},
	2054	});
	2055	},
	2056	else => {
	2057	// This is likely just a missing comma;
	2058	// give an error but continue parsing this list.
	2059	p.tok_i -= 1;
	2060	try p.warn(.{
	2061	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2062	});
	2063	},
	2064	}
1405	2065	}
1406		const node = try Node.ArrayInitializerDot.alloc(&p.arena.allocator, init_list.items.len);
1407		node.* = .{
1408		.dot = dot,
1409		.rtoken = try p.expectToken(.RBrace),
1410		.list_len = init_list.items.len,
1411		};
1412		std.mem.copy(*Node, node.list(), init_list.items);
1413		return &node.base;
	2066	const span = try p.listToSpan(init_list.items);
	2067	return p.addNode(.{
	2068	.tag = .StructInit,
	2069	.main_token = lbrace,
	2070	.data = .{
	2071	.lhs = lhs,
	2072	.rhs = try p.addExtra(Node.SubRange{
	2073	.start = span.start,
	2074	.end = span.end,
	2075	}),
	2076	},
	2077	});
1414	2078	}
1415	2079
1416		const node = try p.arena.allocator.create(Node.StructInitializerDot);
1417		node.* = .{
1418		.dot = dot,
1419		.rtoken = try p.expectToken(.RBrace),
1420		.list_len = 0,
1421		};
1422		return &node.base;
1423		}
	2080	const elem_init = try p.expectExpr();
	2081	if (p.eatToken(.RBrace)) \|_\| {
	2082	return p.addNode(.{
	2083	.tag = .ArrayInitOne,
	2084	.main_token = lbrace,
	2085	.data = .{
	2086	.lhs = lhs,
	2087	.rhs = elem_init,
	2088	},
	2089	});
	2090	}
1424	2091
1425		/// TypeExpr <- PrefixTypeOp* ErrorUnionExpr
1426		fn parseTypeExpr(p: Parser) Error!?Node {
1427		return p.parsePrefixOpExpr(parsePrefixTypeOp, parseErrorUnionExpr);
1428		}
	2092	var init_list = std.ArrayList(Node.Index).init(p.gpa);
	2093	defer init_list.deinit();
1429	2094
1430		/// ErrorUnionExpr <- SuffixExpr (EXCLAMATIONMARK TypeExpr)?
1431		fn parseErrorUnionExpr(p: Parser) !?Node {
1432		const suffix_expr = (try p.parseSuffixExpr()) orelse return null;
	2095	try init_list.append(elem_init);
1433	2096
1434		if (try SimpleBinOpParseFn(.Bang, .ErrorUnion)(p)) \|node\| {
1435		const error_union = node.castTag(.ErrorUnion).?;
1436		const type_expr = try p.expectNode(parseTypeExpr, .{
1437		.ExpectedTypeExpr = .{ .token = p.tok_i },
1438		});
1439		error_union.lhs = suffix_expr;
1440		error_union.rhs = type_expr;
1441		return node;
	2097	while (p.eatToken(.Comma)) \|_\| {
	2098	const next = try p.parseExpr();
	2099	if (next == 0) break;
	2100	try init_list.append(next);
1442	2101	}
	2102	_ = try p.expectToken(.RBrace);
	2103	const span = try p.listToSpan(init_list.items);
	2104	return p.addNode(.{
	2105	.tag = .ArrayInit,
	2106	.main_token = lbrace,
	2107	.data = .{
	2108	.lhs = lhs,
	2109	.rhs = try p.addExtra(Node.SubRange{
	2110	.start = span.start,
	2111	.end = span.end,
	2112	}),
	2113	},
	2114	});
	2115	}
1443	2116
1444		return suffix_expr;
	2117	/// ErrorUnionExpr <- SuffixExpr (EXCLAMATIONMARK TypeExpr)?
	2118	fn parseErrorUnionExpr(p: *Parser) !Node.Index {
	2119	const suffix_expr = try p.parseSuffixExpr();
	2120	if (suffix_expr == 0) return null_node;
	2121	const bang = p.eatToken(.Bang) orelse return suffix_expr;
	2122	return p.addNode(.{
	2123	.tag = .ErrorUnion,
	2124	.main_token = bang,
	2125	.data = .{
	2126	.lhs = suffix_expr,
	2127	.rhs = try p.expectTypeExpr(),
	2128	},
	2129	});
1445	2130	}
1446	2131
1447	2132	/// SuffixExpr
1448	2133	/// <- KEYWORD_async PrimaryTypeExpr SuffixOp* FnCallArguments
1449	2134	/// / PrimaryTypeExpr (SuffixOp / FnCallArguments)*
1450		fn parseSuffixExpr(p: Parser) !?Node {
1451		const maybe_async = p.eatToken(.Keyword_async);
1452		if (maybe_async) \|async_token\| {
1453		const token_fn = p.eatToken(.Keyword_fn);
1454		if (token_fn != null) {
1455		// TODO: remove this hack when async fn rewriting is
1456		// HACK: If we see the keyword `fn`, then we assume that
1457		// we are parsing an async fn proto, and not a call.
1458		// We therefore put back all tokens consumed by the async
1459		// prefix...
1460		p.putBackToken(token_fn.?);
1461		p.putBackToken(async_token);
1462		return p.parsePrimaryTypeExpr();
1463		}
1464		var res = try p.expectNode(parsePrimaryTypeExpr, .{
1465		.ExpectedPrimaryTypeExpr = .{ .token = p.tok_i },
1466		});
	2135	/// FnCallArguments <- LPAREN ExprList RPAREN
	2136	/// ExprList <- (Expr COMMA)* Expr?
	2137	/// TODO detect when there is 1 or less parameter to the call and emit
	2138	/// CallOne instead of Call.
	2139	fn parseSuffixExpr(p: *Parser) !Node.Index {
	2140	if (p.eatToken(.Keyword_async)) \|async_token\| {
	2141	var res = try p.expectPrimaryTypeExpr();
1467	2142
1468		while (try p.parseSuffixOp(res)) \|node\| {
	2143	while (true) {
	2144	const node = try p.parseSuffixOp(res);
	2145	if (node == 0) break;
1469	2146	res = node;
1470	2147	}
1471
1472		const params = (try p.parseFnCallArguments()) orelse {
1473		try p.errors.append(p.gpa, .{
1474		.ExpectedParamList = .{ .token = p.tok_i },
1475		});
1476		// ignore this, continue parsing
	2148	const lparen = (try p.expectTokenRecoverable(.LParen)) orelse {
	2149	try p.warn(.{ .ExpectedParamList = .{ .token = p.tok_i } });
1477	2150	return res;
1478	2151	};
1479		defer p.gpa.free(params.list);
1480		const node = try Node.Call.alloc(&p.arena.allocator, params.list.len);
1481		node.* = .{
1482		.lhs = res,
1483		.params_len = params.list.len,
1484		.async_token = async_token,
1485		.rtoken = params.rparen,
1486		};
1487		std.mem.copy(*Node, node.params(), params.list);
1488		return &node.base;
	2152	const params = try ListParseFn(parseExpr)(p);
	2153	_ = try p.expectToken(.RParen);
	2154
	2155	return p.addNode(.{
	2156	.tag = .Call,
	2157	.main_token = lparen,
	2158	.data = .{
	2159	.lhs = res,
	2160	.rhs = try p.addExtra(Node.SubRange{
	2161	.start = params.start,
	2162	.end = params.end,
	2163	}),
	2164	},
	2165	});
1489	2166	}
1490		if (try p.parsePrimaryTypeExpr()) \|expr\| {
1491		var res = expr;
	2167	var res = try p.parsePrimaryTypeExpr();
	2168	if (res == 0) return res;
1492	2169
1493		while (true) {
1494		if (try p.parseSuffixOp(res)) \|node\| {
1495		res = node;
1496		continue;
1497		}
1498		if (try p.parseFnCallArguments()) \|params\| {
1499		defer p.gpa.free(params.list);
1500		const call = try Node.Call.alloc(&p.arena.allocator, params.list.len);
1501		call.* = .{
1502		.lhs = res,
1503		.params_len = params.list.len,
1504		.async_token = null,
1505		.rtoken = params.rparen,
1506		};
1507		std.mem.copy(*Node, call.params(), params.list);
1508		res = &call.base;
1509		continue;
1510		}
1511		break;
	2170	while (true) {
	2171	const suffix_op = try p.parseSuffixOp(res);
	2172	if (suffix_op != 0) {
	2173	res = suffix_op;
	2174	continue;
1512	2175	}
1513		return res;
	2176	const lparen = p.eatToken(.LParen) orelse return res;
	2177	const params = try ListParseFn(parseExpr)(p);
	2178	_ = try p.expectToken(.RParen);
	2179
	2180	res = try p.addNode(.{
	2181	.tag = .Call,
	2182	.main_token = lparen,
	2183	.data = .{
	2184	.lhs = res,
	2185	.rhs = try p.addExtra(Node.SubRange{
	2186	.start = params.start,
	2187	.end = params.end,
	2188	}),
	2189	},
	2190	});
1514	2191	}
1515
1516		return null;
1517	2192	}
1518	2193
1519	2194	/// PrimaryTypeExpr
...	...	@@ -1521,6 +2196,7 @@ const Parser = struct {
1521	2196	/// / CHAR_LITERAL
1522	2197	/// / ContainerDecl
1523	2198	/// / DOT IDENTIFIER
	2199	/// / DOT InitList
1524	2200	/// / ErrorSetDecl
1525	2201	/// / FLOAT
1526	2202	/// / FnProto
...	...	@@ -1539,260 +2215,497 @@ const Parser = struct {
1539	2215	/// / KEYWORD_unreachable
1540	2216	/// / STRINGLITERAL
1541	2217	/// / SwitchExpr
1542		fn parsePrimaryTypeExpr(p: Parser) !?Node {
1543		if (try p.parseBuiltinCall()) \|node\| return node;
1544		if (p.eatToken(.CharLiteral)) \|token\| {
1545		const node = try p.arena.allocator.create(Node.OneToken);
1546		node.* = .{
1547		.base = .{ .tag = .CharLiteral },
1548		.token = token,
1549		};
1550		return &node.base;
1551		}
1552		if (try p.parseContainerDecl()) \|node\| return node;
1553		if (try p.parseAnonLiteral()) \|node\| return node;
1554		if (try p.parseErrorSetDecl()) \|node\| return node;
1555		if (try p.parseFloatLiteral()) \|node\| return node;
1556		if (try p.parseFnProto(.as_type, .{})) \|node\| return node;
1557		if (try p.parseGroupedExpr()) \|node\| return node;
1558		if (try p.parseLabeledTypeExpr()) \|node\| return node;
1559		if (try p.parseIdentifier()) \|node\| return node;
1560		if (try p.parseIfTypeExpr()) \|node\| return node;
1561		if (try p.parseIntegerLiteral()) \|node\| return node;
1562		if (p.eatToken(.Keyword_comptime)) \|token\| {
1563		const expr = (try p.parseTypeExpr()) orelse return null;
1564		const node = try p.arena.allocator.create(Node.Comptime);
1565		node.* = .{
1566		.doc_comments = null,
1567		.comptime_token = token,
1568		.expr = expr,
1569		};
1570		return &node.base;
1571		}
1572		if (p.eatToken(.Keyword_error)) \|token\| {
1573		const period = try p.expectTokenRecoverable(.Period);
1574		const identifier = try p.expectNodeRecoverable(parseIdentifier, .{
1575		.ExpectedIdentifier = .{ .token = p.tok_i },
1576		});
1577		const global_error_set = try p.createLiteral(.ErrorType, token);
1578		if (period == null or identifier == null) return global_error_set;
1579
1580		const node = try p.arena.allocator.create(Node.SimpleInfixOp);
1581		node.* = .{
1582		.base = Node{ .tag = .Period },
1583		.op_token = period.?,
1584		.lhs = global_error_set,
1585		.rhs = identifier.?,
1586		};
1587		return &node.base;
1588		}
1589		if (p.eatToken(.Keyword_false)) \|token\| return p.createLiteral(.BoolLiteral, token);
1590		if (p.eatToken(.Keyword_null)) \|token\| return p.createLiteral(.NullLiteral, token);
1591		if (p.eatToken(.Keyword_anyframe)) \|token\| {
1592		const node = try p.arena.allocator.create(Node.AnyFrameType);
1593		node.* = .{
1594		.anyframe_token = token,
1595		.result = null,
1596		};
1597		return &node.base;
1598		}
1599		if (p.eatToken(.Keyword_true)) \|token\| return p.createLiteral(.BoolLiteral, token);
1600		if (p.eatToken(.Keyword_undefined)) \|token\| return p.createLiteral(.UndefinedLiteral, token);
1601		if (p.eatToken(.Keyword_unreachable)) \|token\| return p.createLiteral(.Unreachable, token);
1602		if (try p.parseStringLiteral()) \|node\| return node;
1603		if (try p.parseSwitchExpr()) \|node\| return node;
1604
1605		return null;
1606		}
1607
1608	2218	/// ContainerDecl <- (KEYWORD_extern / KEYWORD_packed)? ContainerDeclAuto
1609		fn parseContainerDecl(p: Parser) !?Node {
1610		const layout_token = p.eatToken(.Keyword_extern) orelse
1611		p.eatToken(.Keyword_packed);
1612
1613		const node = (try p.parseContainerDeclAuto()) orelse {
1614		if (layout_token) \|token\|
1615		p.putBackToken(token);
1616		return null;
1617		};
1618		node.cast(Node.ContainerDecl).?.*.layout_token = layout_token;
1619		return node;
1620		}
1621
1622		/// ErrorSetDecl <- KEYWORD_error LBRACE IdentifierList RBRACE
1623		fn parseErrorSetDecl(p: Parser) !?Node {
1624		const error_token = p.eatToken(.Keyword_error) orelse return null;
1625		if (p.eatToken(.LBrace) == null) {
1626		// Might parse as `KEYWORD_error DOT IDENTIFIER` later in PrimaryTypeExpr, so don't error
1627		p.putBackToken(error_token);
1628		return null;
1629		}
1630		const decls = try p.parseErrorTagList();
1631		defer p.gpa.free(decls);
1632		const rbrace = try p.expectToken(.RBrace);
1633
1634		const node = try Node.ErrorSetDecl.alloc(&p.arena.allocator, decls.len);
1635		node.* = .{
1636		.error_token = error_token,
1637		.decls_len = decls.len,
1638		.rbrace_token = rbrace,
1639		};
1640		std.mem.copy(*Node, node.decls(), decls);
1641		return &node.base;
1642		}
1643
	2219	/// ContainerDeclAuto <- ContainerDeclType LBRACE ContainerMembers RBRACE
	2220	/// InitList
	2221	/// <- LBRACE FieldInit (COMMA FieldInit)* COMMA? RBRACE
	2222	/// / LBRACE Expr (COMMA Expr)* COMMA? RBRACE
	2223	/// / LBRACE RBRACE
	2224	/// ErrorSetDecl <- KEYWORD_error LBRACE IdentifierList RBRACE
1644	2225	/// GroupedExpr <- LPAREN Expr RPAREN
1645		fn parseGroupedExpr(p: Parser) !?Node {
1646		const lparen = p.eatToken(.LParen) orelse return null;
1647		const expr = try p.expectNode(parseExpr, .{
1648		.ExpectedExpr = .{ .token = p.tok_i },
1649		});
1650		const rparen = try p.expectToken(.RParen);
1651
1652		const node = try p.arena.allocator.create(Node.GroupedExpression);
1653		node.* = .{
1654		.lparen = lparen,
1655		.expr = expr,
1656		.rparen = rparen,
1657		};
1658		return &node.base;
1659		}
1660
1661	2226	/// IfTypeExpr <- IfPrefix TypeExpr (KEYWORD_else Payload? TypeExpr)?
1662		fn parseIfTypeExpr(p: Parser) !?Node {
1663		return p.parseIf(parseTypeExpr);
1664		}
1665
1666	2227	/// LabeledTypeExpr
1667	2228	/// <- BlockLabel Block
1668	2229	/// / BlockLabel? LoopTypeExpr
1669		fn parseLabeledTypeExpr(p: Parser) !?Node {
1670		var colon: TokenIndex = undefined;
1671		const label = p.parseBlockLabel(&colon);
	2230	/// LoopTypeExpr <- KEYWORD_inline? (ForTypeExpr / WhileTypeExpr)
	2231	fn parsePrimaryTypeExpr(p: *Parser) !Node.Index {
	2232	switch (p.token_tags[p.tok_i]) {
	2233	.CharLiteral,
	2234	.IntegerLiteral,
	2235	.FloatLiteral,
	2236	.StringLiteral,
	2237	.Keyword_false,
	2238	.Keyword_true,
	2239	.Keyword_null,
	2240	.Keyword_undefined,
	2241	.Keyword_unreachable,
	2242	.Keyword_anyframe,
	2243	=> return p.addNode(.{
	2244	.tag = .OneToken,
	2245	.main_token = p.nextToken(),
	2246	.data = .{
	2247	.lhs = undefined,
	2248	.rhs = undefined,
	2249	},
	2250	}),
	2251
	2252	.Builtin => return p.parseBuiltinCall(),
	2253	.Keyword_fn => return p.parseFnProto(),
	2254	.Keyword_if => return p.parseIf(parseTypeExpr),
	2255	.Keyword_switch => return p.parseSwitchExpr(),
	2256
	2257	.Keyword_extern,
	2258	.Keyword_packed,
	2259	=> {
	2260	p.tok_i += 1;
	2261	return p.parseContainerDeclAuto();
	2262	},
1672	2263
1673		if (label) \|label_token\| {
1674		if (try p.parseBlock(label_token)) \|node\| return node;
1675		}
	2264	.Keyword_struct,
	2265	.Keyword_opaque,
	2266	.Keyword_enum,
	2267	.Keyword_union,
	2268	=> return p.parseContainerDeclAuto(),
	2269
	2270	.Keyword_comptime => return p.addNode(.{
	2271	.tag = .Comptime,
	2272	.main_token = p.nextToken(),
	2273	.data = .{
	2274	.lhs = try p.expectTypeExpr(),
	2275	.rhs = undefined,
	2276	},
	2277	}),
	2278	.MultilineStringLiteralLine => {
	2279	const first_line = p.nextToken();
	2280	while (p.token_tags[p.tok_i] == .MultilineStringLiteralLine) {
	2281	p.tok_i += 1;
	2282	}
	2283	return p.addNode(.{
	2284	.tag = .OneToken,
	2285	.main_token = first_line,
	2286	.data = .{
	2287	.lhs = undefined,
	2288	.rhs = undefined,
	2289	},
	2290	});
	2291	},
	2292	.Identifier => switch (p.token_tags[p.tok_i + 1]) {
	2293	.Colon => switch (p.token_tags[p.tok_i + 2]) {
	2294	.Keyword_inline => {
	2295	p.tok_i += 3;
	2296	switch (p.token_tags[p.tok_i]) {
	2297	.Keyword_for => return p.parseForTypeExpr(),
	2298	.Keyword_while => return p.parseWhileTypeExpr(),
	2299	else => return p.fail(.{
	2300	.ExpectedInlinable = .{ .token = p.tok_i },
	2301	}),
	2302	}
	2303	},
	2304	.Keyword_for => {
	2305	p.tok_i += 2;
	2306	return p.parseForTypeExpr();
	2307	},
	2308	.Keyword_while => {
	2309	p.tok_i += 2;
	2310	return p.parseWhileTypeExpr();
	2311	},
	2312	else => return p.addNode(.{
	2313	.tag = .Identifier,
	2314	.main_token = p.nextToken(),
	2315	.data = .{
	2316	.lhs = undefined,
	2317	.rhs = undefined,
	2318	},
	2319	}),
	2320	},
	2321	else => return p.addNode(.{
	2322	.tag = .Identifier,
	2323	.main_token = p.nextToken(),
	2324	.data = .{
	2325	.lhs = undefined,
	2326	.rhs = undefined,
	2327	},
	2328	}),
	2329	},
	2330	.Period => switch (p.token_tags[p.tok_i + 1]) {
	2331	.Identifier => return p.addNode(.{
	2332	.tag = .EnumLiteral,
	2333	.data = .{
	2334	.lhs = p.nextToken(), // dot
	2335	.rhs = undefined,
	2336	},
	2337	.main_token = p.nextToken(), // identifier
	2338	}),
	2339	.LBrace => {
	2340	const lbrace = p.tok_i + 1;
	2341	p.tok_i = lbrace + 1;
	2342
	2343	// If there are 0, 1, or 2 items, we can use ArrayInitDotTwo/StructInitDotTwo;
	2344	// otherwise we use the full ArrayInitDot/StructInitDot.
	2345
	2346	if (p.eatToken(.RBrace)) \|_\| {
	2347	return p.addNode(.{
	2348	.tag = .StructInitDotTwo,
	2349	.main_token = lbrace,
	2350	.data = .{
	2351	.lhs = 0,
	2352	.rhs = 0,
	2353	},
	2354	});
	2355	}
	2356	const field_init_one = try p.parseFieldInit();
	2357	if (field_init_one != 0) {
	2358	const comma_one = p.eatToken(.Comma);
	2359	if (p.eatToken(.RBrace)) \|_\| {
	2360	return p.addNode(.{
	2361	.tag = .StructInitDotTwo,
	2362	.main_token = lbrace,
	2363	.data = .{
	2364	.lhs = field_init_one,
	2365	.rhs = 0,
	2366	},
	2367	});
	2368	}
	2369	if (comma_one == null) {
	2370	try p.warn(.{
	2371	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2372	});
	2373	}
	2374	const field_init_two = try p.expectFieldInit();
	2375	const comma_two = p.eatToken(.Comma);
	2376	if (p.eatToken(.RBrace)) \|_\| {
	2377	return p.addNode(.{
	2378	.tag = .StructInitDotTwo,
	2379	.main_token = lbrace,
	2380	.data = .{
	2381	.lhs = field_init_one,
	2382	.rhs = field_init_two,
	2383	},
	2384	});
	2385	}
	2386	if (comma_two == null) {
	2387	try p.warn(.{
	2388	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2389	});
	2390	}
	2391	var init_list = std.ArrayList(Node.Index).init(p.gpa);
	2392	defer init_list.deinit();
	2393
	2394	try init_list.appendSlice(&[_]Node.Index{ field_init_one, field_init_two });
	2395
	2396	while (true) {
	2397	const next = try p.expectFieldInit();
	2398	if (next == 0) break;
	2399	try init_list.append(next);
	2400	switch (p.token_tags[p.nextToken()]) {
	2401	.Comma => {
	2402	if (p.eatToken(.RBrace)) \|_\| break;
	2403	continue;
	2404	},
	2405	.RBrace => break,
	2406	.Colon, .RParen, .RBracket => {
	2407	p.tok_i -= 1;
	2408	return p.fail(.{
	2409	.ExpectedToken = .{
	2410	.token = p.tok_i,
	2411	.expected_id = .RBrace,
	2412	},
	2413	});
	2414	},
	2415	else => {
	2416	p.tok_i -= 1;
	2417	try p.warn(.{
	2418	.ExpectedToken = .{
	2419	.token = p.tok_i,
	2420	.expected_id = .Comma,
	2421	},
	2422	});
	2423	},
	2424	}
	2425	}
	2426	const span = try p.listToSpan(init_list.items);
	2427	return p.addNode(.{
	2428	.tag = .StructInitDot,
	2429	.main_token = lbrace,
	2430	.data = .{
	2431	.lhs = span.start,
	2432	.rhs = span.end,
	2433	},
	2434	});
	2435	}
1676	2436
1677		if (try p.parseLoopTypeExpr()) \|node\| {
1678		switch (node.tag) {
1679		.For => node.cast(Node.For).?.label = label,
1680		.While => node.cast(Node.While).?.label = label,
1681		else => unreachable,
1682		}
1683		return node;
1684		}
	2437	const elem_init_one = try p.expectExpr();
	2438	const comma_one = p.eatToken(.Comma);
	2439	if (p.eatToken(.RBrace)) \|_\| {
	2440	return p.addNode(.{
	2441	.tag = .ArrayInitDotTwo,
	2442	.main_token = lbrace,
	2443	.data = .{
	2444	.lhs = elem_init_one,
	2445	.rhs = 0,
	2446	},
	2447	});
	2448	}
	2449	if (comma_one == null) {
	2450	try p.warn(.{
	2451	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2452	});
	2453	}
	2454	const elem_init_two = try p.expectExpr();
	2455	const comma_two = p.eatToken(.Comma);
	2456	if (p.eatToken(.RBrace)) \|_\| {
	2457	return p.addNode(.{
	2458	.tag = .ArrayInitDotTwo,
	2459	.main_token = lbrace,
	2460	.data = .{
	2461	.lhs = elem_init_one,
	2462	.rhs = elem_init_two,
	2463	},
	2464	});
	2465	}
	2466	if (comma_two == null) {
	2467	try p.warn(.{
	2468	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2469	});
	2470	}
	2471	var init_list = std.ArrayList(Node.Index).init(p.gpa);
	2472	defer init_list.deinit();
	2473
	2474	try init_list.appendSlice(&[_]Node.Index{ elem_init_one, elem_init_two });
	2475
	2476	while (true) {
	2477	const next = try p.expectExpr();
	2478	if (next == 0) break;
	2479	try init_list.append(next);
	2480	switch (p.token_tags[p.nextToken()]) {
	2481	.Comma => continue,
	2482	.RBrace => break,
	2483	.Colon, .RParen, .RBracket => {
	2484	p.tok_i -= 1;
	2485	return p.fail(.{
	2486	.ExpectedToken = .{
	2487	.token = p.tok_i,
	2488	.expected_id = .RBrace,
	2489	},
	2490	});
	2491	},
	2492	else => {
	2493	p.tok_i -= 1;
	2494	try p.warn(.{
	2495	.ExpectedToken = .{
	2496	.token = p.tok_i,
	2497	.expected_id = .Comma,
	2498	},
	2499	});
	2500	},
	2501	}
	2502	}
	2503	const span = try p.listToSpan(init_list.items);
	2504	return p.addNode(.{
	2505	.tag = .ArrayInitDot,
	2506	.main_token = lbrace,
	2507	.data = .{
	2508	.lhs = span.start,
	2509	.rhs = span.end,
	2510	},
	2511	});
	2512	},
	2513	else => return null_node,
	2514	},
	2515	.Keyword_error => switch (p.token_tags[p.tok_i + 1]) {
	2516	.LBrace => {
	2517	const error_token = p.tok_i;
	2518	p.tok_i += 2;
	2519
	2520	if (p.eatToken(.RBrace)) \|_\| {
	2521	return p.addNode(.{
	2522	.tag = .ErrorSetDecl,
	2523	.main_token = error_token,
	2524	.data = .{
	2525	.lhs = undefined,
	2526	.rhs = undefined,
	2527	},
	2528	});
	2529	}
1685	2530
1686		if (label) \|token\| {
1687		p.putBackToken(colon);
1688		p.putBackToken(token);
	2531	while (true) {
	2532	const doc_comment = p.eatDocComments();
	2533	const identifier = try p.expectToken(.Identifier);
	2534	switch (p.token_tags[p.nextToken()]) {
	2535	.Comma => {
	2536	if (p.eatToken(.RBrace)) \|_\| break;
	2537	continue;
	2538	},
	2539	.RBrace => break,
	2540	.Colon, .RParen, .RBracket => {
	2541	p.tok_i -= 1;
	2542	return p.fail(.{
	2543	.ExpectedToken = .{
	2544	.token = p.tok_i,
	2545	.expected_id = .RBrace,
	2546	},
	2547	});
	2548	},
	2549	else => {
	2550	// This is likely just a missing comma;
	2551	// give an error but continue parsing this list.
	2552	p.tok_i -= 1;
	2553	try p.warn(.{
	2554	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2555	});
	2556	},
	2557	}
	2558	}
	2559	return p.addNode(.{
	2560	.tag = .ErrorSetDecl,
	2561	.main_token = error_token,
	2562	.data = .{
	2563	.lhs = undefined,
	2564	.rhs = undefined,
	2565	},
	2566	});
	2567	},
	2568	else => return p.addNode(.{
	2569	.tag = .ErrorValue,
	2570	.main_token = p.nextToken(),
	2571	.data = .{
	2572	.lhs = try p.expectToken(.Period),
	2573	.rhs = try p.expectToken(.Identifier),
	2574	},
	2575	}),
	2576	},
	2577	.LParen => return p.addNode(.{
	2578	.tag = .GroupedExpression,
	2579	.main_token = p.nextToken(),
	2580	.data = .{
	2581	.lhs = try p.expectExpr(),
	2582	.rhs = try p.expectToken(.RParen),
	2583	},
	2584	}),
	2585	else => return null_node,
1689	2586	}
1690		return null;
1691	2587	}
1692	2588
1693		/// LoopTypeExpr <- KEYWORD_inline? (ForTypeExpr / WhileTypeExpr)
1694		fn parseLoopTypeExpr(p: Parser) !?Node {
1695		const inline_token = p.eatToken(.Keyword_inline);
1696
1697		if (try p.parseForTypeExpr()) \|node\| {
1698		node.cast(Node.For).?.inline_token = inline_token;
1699		return node;
1700		}
1701
1702		if (try p.parseWhileTypeExpr()) \|node\| {
1703		node.cast(Node.While).?.inline_token = inline_token;
1704		return node;
	2589	fn expectPrimaryTypeExpr(p: *Parser) !Node.Index {
	2590	const node = try p.parsePrimaryTypeExpr();
	2591	if (node == 0) {
	2592	return p.fail(.{ .ExpectedPrimaryTypeExpr = .{ .token = p.tok_i } });
1705	2593	}
1706
1707		if (inline_token == null) return null;
1708
1709		// If we've seen "inline", there should have been a "for" or "while"
1710		try p.errors.append(p.gpa, .{
1711		.ExpectedInlinable = .{ .token = p.tok_i },
1712		});
1713		return error.ParseError;
	2594	return node;
1714	2595	}
1715	2596
	2597	/// ForPrefix <- KEYWORD_for LPAREN Expr RPAREN PtrIndexPayload
1716	2598	/// ForTypeExpr <- ForPrefix TypeExpr (KEYWORD_else TypeExpr)?
1717		fn parseForTypeExpr(p: Parser) !?Node {
1718		const node = (try p.parseForPrefix()) orelse return null;
1719		const for_prefix = node.cast(Node.For).?;
1720
1721		const type_expr = try p.expectNode(parseTypeExpr, .{
1722		.ExpectedTypeExpr = .{ .token = p.tok_i },
1723		});
1724		for_prefix.body = type_expr;
1725
1726		if (p.eatToken(.Keyword_else)) \|else_token\| {
1727		const else_expr = try p.expectNode(parseTypeExpr, .{
1728		.ExpectedTypeExpr = .{ .token = p.tok_i },
	2599	fn parseForTypeExpr(p: *Parser) !Node.Index {
	2600	const for_token = p.eatToken(.Keyword_for) orelse return null_node;
	2601	_ = try p.expectToken(.LParen);
	2602	const array_expr = try p.expectTypeExpr();
	2603	_ = try p.expectToken(.RParen);
	2604	_ = try p.parsePtrIndexPayload();
	2605
	2606	const then_expr = try p.expectExpr();
	2607	const else_token = p.eatToken(.Keyword_else) orelse {
	2608	return p.addNode(.{
	2609	.tag = .ForSimple,
	2610	.main_token = for_token,
	2611	.data = .{
	2612	.lhs = array_expr,
	2613	.rhs = then_expr,
	2614	},
1729	2615	});
1730
1731		const else_node = try p.arena.allocator.create(Node.Else);
1732		else_node.* = .{
1733		.else_token = else_token,
1734		.payload = null,
1735		.body = else_expr,
1736		};
1737
1738		for_prefix.@"else" = else_node;
1739		}
1740
1741		return node;
	2616	};
	2617	const else_expr = try p.expectTypeExpr();
	2618	return p.addNode(.{
	2619	.tag = .For,
	2620	.main_token = for_token,
	2621	.data = .{
	2622	.lhs = array_expr,
	2623	.rhs = try p.addExtra(Node.If{
	2624	.then_expr = then_expr,
	2625	.else_expr = else_expr,
	2626	}),
	2627	},
	2628	});
1742	2629	}
1743	2630
	2631	/// WhilePrefix <- KEYWORD_while LPAREN Expr RPAREN PtrPayload? WhileContinueExpr?
1744	2632	/// WhileTypeExpr <- WhilePrefix TypeExpr (KEYWORD_else Payload? TypeExpr)?
1745		fn parseWhileTypeExpr(p: Parser) !?Node {
1746		const node = (try p.parseWhilePrefix()) orelse return null;
1747		const while_prefix = node.cast(Node.While).?;
	2633	fn parseWhileTypeExpr(p: *Parser) !Node.Index {
	2634	const while_token = p.eatToken(.Keyword_while) orelse return null_node;
	2635	_ = try p.expectToken(.LParen);
	2636	const condition = try p.expectExpr();
	2637	_ = try p.expectToken(.RParen);
	2638	const then_payload = try p.parsePtrPayload();
	2639	const continue_expr = try p.parseWhileContinueExpr();
1748	2640
1749		const type_expr = try p.expectNode(parseTypeExpr, .{
1750		.ExpectedTypeExpr = .{ .token = p.tok_i },
	2641	const then_expr = try p.expectTypeExpr();
	2642	const else_token = p.eatToken(.Keyword_else) orelse {
	2643	if (continue_expr == 0) {
	2644	return p.addNode(.{
	2645	.tag = if (then_payload == 0) .WhileSimple else .WhileSimpleOptional,
	2646	.main_token = while_token,
	2647	.data = .{
	2648	.lhs = condition,
	2649	.rhs = then_expr,
	2650	},
	2651	});
	2652	} else {
	2653	return p.addNode(.{
	2654	.tag = if (then_payload == 0) .WhileCont else .WhileContOptional,
	2655	.main_token = while_token,
	2656	.data = .{
	2657	.lhs = condition,
	2658	.rhs = try p.addExtra(Node.WhileCont{
	2659	.continue_expr = continue_expr,
	2660	.then_expr = then_expr,
	2661	}),
	2662	},
	2663	});
	2664	}
	2665	};
	2666	const else_payload = try p.parsePayload();
	2667	const else_expr = try p.expectTypeExpr();
	2668	const tag = if (else_payload != 0)
	2669	Node.Tag.WhileError
	2670	else if (then_payload != 0)
	2671	Node.Tag.WhileOptional
	2672	else
	2673	Node.Tag.While;
	2674	return p.addNode(.{
	2675	.tag = tag,
	2676	.main_token = while_token,
	2677	.data = .{
	2678	.lhs = condition,
	2679	.rhs = try p.addExtra(Node.While{
	2680	.continue_expr = continue_expr,
	2681	.then_expr = then_expr,
	2682	.else_expr = else_expr,
	2683	}),
	2684	},
1751	2685	});
1752		while_prefix.body = type_expr;
1753
1754		if (p.eatToken(.Keyword_else)) \|else_token\| {
1755		const payload = try p.parsePayload();
1756
1757		const else_expr = try p.expectNode(parseTypeExpr, .{
1758		.ExpectedTypeExpr = .{ .token = p.tok_i },
1759		});
1760
1761		const else_node = try p.arena.allocator.create(Node.Else);
1762		else_node.* = .{
1763		.else_token = else_token,
1764		.payload = null,
1765		.body = else_expr,
1766		};
1767
1768		while_prefix.@"else" = else_node;
1769		}
1770
1771		return node;
1772	2686	}
1773	2687
1774	2688	/// SwitchExpr <- KEYWORD_switch LPAREN Expr RPAREN LBRACE SwitchProngList RBRACE
1775		fn parseSwitchExpr(p: Parser) !?Node {
1776		const switch_token = p.eatToken(.Keyword_switch) orelse return null;
	2689	fn parseSwitchExpr(p: *Parser) !Node.Index {
	2690	const switch_token = p.eatToken(.Keyword_switch) orelse return null_node;
1777	2691	_ = try p.expectToken(.LParen);
1778		const expr_node = try p.expectNode(parseExpr, .{
1779		.ExpectedExpr = .{ .token = p.tok_i },
1780		});
	2692	const expr_node = try p.expectExpr();
1781	2693	_ = try p.expectToken(.RParen);
1782	2694	_ = try p.expectToken(.LBrace);
1783	2695	const cases = try p.parseSwitchProngList();
1784		defer p.gpa.free(cases);
1785		const rbrace = try p.expectToken(.RBrace);
1786
1787		const node = try Node.Switch.alloc(&p.arena.allocator, cases.len);
1788		node.* = .{
1789		.switch_token = switch_token,
1790		.expr = expr_node,
1791		.cases_len = cases.len,
1792		.rbrace = rbrace,
1793		};
1794		std.mem.copy(*Node, node.cases(), cases);
1795		return &node.base;
	2696	_ = try p.expectToken(.RBrace);
	2697
	2698	return p.addNode(.{
	2699	.tag = .Switch,
	2700	.main_token = switch_token,
	2701	.data = .{
	2702	.lhs = expr_node,
	2703	.rhs = try p.addExtra(Node.SubRange{
	2704	.start = cases.start,
	2705	.end = cases.end,
	2706	}),
	2707	},
	2708	});
1796	2709	}
1797	2710
1798	2711	/// AsmExpr <- KEYWORD_asm KEYWORD_volatile? LPAREN Expr AsmOutput? RPAREN
...	...	@@ -1800,1696 +2713,939 @@ const Parser = struct {
1800	2713	/// AsmInput <- COLON AsmInputList AsmClobbers?
1801	2714	/// AsmClobbers <- COLON StringList
1802	2715	/// StringList <- (STRINGLITERAL COMMA)* STRINGLITERAL?
1803		fn parseAsmExpr(p: Parser) !?Node {
1804		const asm_token = p.eatToken(.Keyword_asm) orelse return null;
1805		const volatile_token = p.eatToken(.Keyword_volatile);
	2716	/// AsmOutputList <- (AsmOutputItem COMMA)* AsmOutputItem?
	2717	/// AsmInputList <- (AsmInputItem COMMA)* AsmInputItem?
	2718	fn parseAsmExpr(p: *Parser) !Node.Index {
	2719	const asm_token = p.assertToken(.Keyword_asm);
	2720	_ = p.eatToken(.Keyword_volatile);
1806	2721	_ = try p.expectToken(.LParen);
1807		const template = try p.expectNode(parseExpr, .{
1808		.ExpectedExpr = .{ .token = p.tok_i },
1809		});
1810
1811		var arena_outputs: []Node.Asm.Output = &[0]Node.Asm.Output{};
1812		var arena_inputs: []Node.Asm.Input = &[0]Node.Asm.Input{};
1813		var arena_clobbers: []Node = &[0]Node{};
1814
1815		if (p.eatToken(.Colon) != null) {
1816		const outputs = try p.parseAsmOutputList();
1817		defer p.gpa.free(outputs);
1818		arena_outputs = try p.arena.allocator.dupe(Node.Asm.Output, outputs);
1819
1820		if (p.eatToken(.Colon) != null) {
1821		const inputs = try p.parseAsmInputList();
1822		defer p.gpa.free(inputs);
1823		arena_inputs = try p.arena.allocator.dupe(Node.Asm.Input, inputs);
1824
1825		if (p.eatToken(.Colon) != null) {
1826		const clobbers = try ListParseFn(*Node, parseStringLiteral)(p);
1827		defer p.gpa.free(clobbers);
1828		arena_clobbers = try p.arena.allocator.dupe(*Node, clobbers);
1829		}
1830		}
	2722	const template = try p.expectExpr();
	2723
	2724	if (p.eatToken(.RParen)) \|_\| {
	2725	return p.addNode(.{
	2726	.tag = .AsmSimple,
	2727	.main_token = asm_token,
	2728	.data = .{
	2729	.lhs = template,
	2730	.rhs = undefined,
	2731	},
	2732	});
1831	2733	}
1832	2734
1833		const node = try p.arena.allocator.create(Node.Asm);
1834		node.* = .{
1835		.asm_token = asm_token,
1836		.volatile_token = volatile_token,
1837		.template = template,
1838		.outputs = arena_outputs,
1839		.inputs = arena_inputs,
1840		.clobbers = arena_clobbers,
1841		.rparen = try p.expectToken(.RParen),
1842		};
1843
1844		return &node.base;
1845		}
	2735	_ = try p.expectToken(.Colon);
1846	2736
1847		/// DOT IDENTIFIER
1848		fn parseAnonLiteral(p: Parser) !?Node {
1849		const dot = p.eatToken(.Period) orelse return null;
	2737	var list = std.ArrayList(Node.Index).init(p.gpa);
	2738	defer list.deinit();
1850	2739
1851		// anon enum literal
1852		if (p.eatToken(.Identifier)) \|name\| {
1853		const node = try p.arena.allocator.create(Node.EnumLiteral);
1854		node.* = .{
1855		.dot = dot,
1856		.name = name,
1857		};
1858		return &node.base;
	2740	while (true) {
	2741	const output_item = try p.parseAsmOutputItem();
	2742	if (output_item == 0) break;
	2743	try list.append(output_item);
	2744	switch (p.token_tags[p.tok_i]) {
	2745	.Comma => p.tok_i += 1,
	2746	.Colon, .RParen, .RBrace, .RBracket => break, // All possible delimiters.
	2747	else => {
	2748	// This is likely just a missing comma;
	2749	// give an error but continue parsing this list.
	2750	try p.warn(.{
	2751	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2752	});
	2753	},
	2754	}
1859	2755	}
1860
1861		if (try p.parseAnonInitList(dot)) \|node\| {
1862		return node;
	2756	if (p.eatToken(.Colon)) \|_\| {
	2757	while (true) {
	2758	const input_item = try p.parseAsmInputItem();
	2759	if (input_item == 0) break;
	2760	try list.append(input_item);
	2761	switch (p.token_tags[p.tok_i]) {
	2762	.Comma => p.tok_i += 1,
	2763	.Colon, .RParen, .RBrace, .RBracket => break, // All possible delimiters.
	2764	else => {
	2765	// This is likely just a missing comma;
	2766	// give an error but continue parsing this list.
	2767	try p.warn(.{
	2768	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2769	});
	2770	},
	2771	}
	2772	}
	2773	if (p.eatToken(.Colon)) \|_\| {
	2774	while (p.eatToken(.StringLiteral)) \|_\| {
	2775	switch (p.token_tags[p.tok_i]) {
	2776	.Comma => p.tok_i += 1,
	2777	.Colon, .RParen, .RBrace, .RBracket => break,
	2778	else => {
	2779	// This is likely just a missing comma;
	2780	// give an error but continue parsing this list.
	2781	try p.warn(.{
	2782	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	2783	});
	2784	},
	2785	}
	2786	}
	2787	}
1863	2788	}
1864
1865		p.putBackToken(dot);
1866		return null;
	2789	_ = try p.expectToken(.RParen);
	2790	const span = try p.listToSpan(list.items);
	2791	return p.addNode(.{
	2792	.tag = .Asm,
	2793	.main_token = asm_token,
	2794	.data = .{
	2795	.lhs = template,
	2796	.rhs = try p.addExtra(Node.SubRange{
	2797	.start = span.start,
	2798	.end = span.end,
	2799	}),
	2800	},
	2801	});
1867	2802	}
1868	2803
1869	2804	/// AsmOutputItem <- LBRACKET IDENTIFIER RBRACKET STRINGLITERAL LPAREN (MINUSRARROW TypeExpr / IDENTIFIER) RPAREN
1870		fn parseAsmOutputItem(p: *Parser) !?Node.Asm.Output {
1871		const lbracket = p.eatToken(.LBracket) orelse return null;
1872		const name = try p.expectNode(parseIdentifier, .{
1873		.ExpectedIdentifier = .{ .token = p.tok_i },
1874		});
	2805	fn parseAsmOutputItem(p: *Parser) !Node.Index {
	2806	_ = p.eatToken(.LBracket) orelse return null_node;
	2807	const identifier = try p.expectToken(.Identifier);
1875	2808	_ = try p.expectToken(.RBracket);
1876
1877		const constraint = try p.expectNode(parseStringLiteral, .{
1878		.ExpectedStringLiteral = .{ .token = p.tok_i },
1879		});
1880
	2809	const constraint = try p.expectToken(.StringLiteral);
1881	2810	_ = try p.expectToken(.LParen);
1882		const kind: Node.Asm.Output.Kind = blk: {
1883		if (p.eatToken(.Arrow) != null) {
1884		const return_ident = try p.expectNode(parseTypeExpr, .{
1885		.ExpectedTypeExpr = .{ .token = p.tok_i },
1886		});
1887		break :blk .{ .Return = return_ident };
1888		}
1889		const variable = try p.expectNode(parseIdentifier, .{
1890		.ExpectedIdentifier = .{ .token = p.tok_i },
1891		});
1892		break :blk .{ .Variable = variable.castTag(.Identifier).? };
1893		};
1894		const rparen = try p.expectToken(.RParen);
1895
1896		return Node.Asm.Output{
1897		.lbracket = lbracket,
1898		.symbolic_name = name,
1899		.constraint = constraint,
1900		.kind = kind,
1901		.rparen = rparen,
1902		};
	2811	const rhs: Node.Index = if (p.eatToken(.Arrow)) \|_\| try p.expectTypeExpr() else null_node;
	2812	_ = try p.expectToken(.RParen);
	2813	return p.addNode(.{
	2814	.tag = .AsmOutput,
	2815	.main_token = identifier,
	2816	.data = .{
	2817	.lhs = constraint,
	2818	.rhs = rhs,
	2819	},
	2820	});
1903	2821	}
1904	2822
1905	2823	/// AsmInputItem <- LBRACKET IDENTIFIER RBRACKET STRINGLITERAL LPAREN Expr RPAREN
1906		fn parseAsmInputItem(p: *Parser) !?Node.Asm.Input {
1907		const lbracket = p.eatToken(.LBracket) orelse return null;
1908		const name = try p.expectNode(parseIdentifier, .{
1909		.ExpectedIdentifier = .{ .token = p.tok_i },
1910		});
	2824	fn parseAsmInputItem(p: *Parser) !Node.Index {
	2825	_ = p.eatToken(.LBracket) orelse return null_node;
	2826	const identifier = try p.expectToken(.Identifier);
1911	2827	_ = try p.expectToken(.RBracket);
1912
1913		const constraint = try p.expectNode(parseStringLiteral, .{
1914		.ExpectedStringLiteral = .{ .token = p.tok_i },
1915		});
1916
	2828	const constraint = try p.expectToken(.StringLiteral);
1917	2829	_ = try p.expectToken(.LParen);
1918		const expr = try p.expectNode(parseExpr, .{
1919		.ExpectedExpr = .{ .token = p.tok_i },
	2830	const expr = try p.expectExpr();
	2831	_ = try p.expectToken(.RParen);
	2832	return p.addNode(.{
	2833	.tag = .AsmInput,
	2834	.main_token = identifier,
	2835	.data = .{
	2836	.lhs = constraint,
	2837	.rhs = expr,
	2838	},
1920	2839	});
1921		const rparen = try p.expectToken(.RParen);
1922
1923		return Node.Asm.Input{
1924		.lbracket = lbracket,
1925		.symbolic_name = name,
1926		.constraint = constraint,
1927		.expr = expr,
1928		.rparen = rparen,
1929		};
1930	2840	}
1931	2841
1932	2842	/// BreakLabel <- COLON IDENTIFIER
1933		fn parseBreakLabel(p: *Parser) !?TokenIndex {
1934		_ = p.eatToken(.Colon) orelse return null;
1935		const ident = try p.expectToken(.Identifier);
1936		return ident;
	2843	fn parseBreakLabel(p: *Parser) !TokenIndex {
	2844	_ = p.eatToken(.Colon) orelse return @as(TokenIndex, 0);
	2845	return p.expectToken(.Identifier);
1937	2846	}
1938	2847
1939	2848	/// BlockLabel <- IDENTIFIER COLON
1940		fn parseBlockLabel(p: Parser, colon_token: TokenIndex) ?TokenIndex {
1941		const identifier = p.eatToken(.Identifier) orelse return null;
1942		if (p.eatToken(.Colon)) \|colon\| {
1943		colon_token.* = colon;
	2849	fn parseBlockLabel(p: *Parser) TokenIndex {
	2850	if (p.token_tags[p.tok_i] == .Identifier and
	2851	p.token_tags[p.tok_i + 1] == .Colon)
	2852	{
	2853	const identifier = p.tok_i;
	2854	p.tok_i += 2;
1944	2855	return identifier;
1945	2856	}
1946		p.putBackToken(identifier);
1947		return null;
	2857	return 0;
1948	2858	}
1949	2859
1950	2860	/// FieldInit <- DOT IDENTIFIER EQUAL Expr
1951		fn parseFieldInit(p: Parser) !?Node {
1952		const period_token = p.eatToken(.Period) orelse return null;
1953		const name_token = p.eatToken(.Identifier) orelse {
1954		// Because of anon literals `.{` is also valid.
1955		p.putBackToken(period_token);
1956		return null;
1957		};
1958		const eq_token = p.eatToken(.Equal) orelse {
1959		// `.Name` may also be an enum literal, which is a later rule.
1960		p.putBackToken(name_token);
1961		p.putBackToken(period_token);
1962		return null;
1963		};
1964		const expr_node = try p.expectNode(parseExpr, .{
1965		.ExpectedExpr = .{ .token = p.tok_i },
1966		});
	2861	fn parseFieldInit(p: *Parser) !Node.Index {
	2862	if (p.token_tags[p.tok_i + 0] == .Period and
	2863	p.token_tags[p.tok_i + 1] == .Identifier and
	2864	p.token_tags[p.tok_i + 2] == .Equal)
	2865	{
	2866	p.tok_i += 3;
	2867	return p.expectExpr();
	2868	} else {
	2869	return null_node;
	2870	}
	2871	}
1967	2872
1968		const node = try p.arena.allocator.create(Node.FieldInitializer);
1969		node.* = .{
1970		.period_token = period_token,
1971		.name_token = name_token,
1972		.expr = expr_node,
1973		};
1974		return &node.base;
	2873	fn expectFieldInit(p: *Parser) !Node.Index {
	2874	_ = try p.expectToken(.Period);
	2875	_ = try p.expectToken(.Identifier);
	2876	_ = try p.expectToken(.Equal);
	2877	return p.expectExpr();
1975	2878	}
1976	2879
1977	2880	/// WhileContinueExpr <- COLON LPAREN AssignExpr RPAREN
1978		fn parseWhileContinueExpr(p: Parser) !?Node {
1979		_ = p.eatToken(.Colon) orelse return null;
	2881	fn parseWhileContinueExpr(p: *Parser) !Node.Index {
	2882	_ = p.eatToken(.Colon) orelse return null_node;
1980	2883	_ = try p.expectToken(.LParen);
1981		const node = try p.expectNode(parseAssignExpr, .{
1982		.ExpectedExprOrAssignment = .{ .token = p.tok_i },
1983		});
	2884	const node = try p.parseAssignExpr();
	2885	if (node == 0) return p.fail(.{ .ExpectedExprOrAssignment = .{ .token = p.tok_i } });
1984	2886	_ = try p.expectToken(.RParen);
1985	2887	return node;
1986	2888	}
1987	2889
1988	2890	/// LinkSection <- KEYWORD_linksection LPAREN Expr RPAREN
1989		fn parseLinkSection(p: Parser) !?Node {
1990		_ = p.eatToken(.Keyword_linksection) orelse return null;
	2891	fn parseLinkSection(p: *Parser) !Node.Index {
	2892	_ = p.eatToken(.Keyword_linksection) orelse return null_node;
1991	2893	_ = try p.expectToken(.LParen);
1992		const expr_node = try p.expectNode(parseExpr, .{
1993		.ExpectedExpr = .{ .token = p.tok_i },
1994		});
	2894	const expr_node = try p.expectExpr();
1995	2895	_ = try p.expectToken(.RParen);
1996	2896	return expr_node;
1997	2897	}
1998	2898
1999	2899	/// CallConv <- KEYWORD_callconv LPAREN Expr RPAREN
2000		fn parseCallconv(p: Parser) !?Node {
2001		_ = p.eatToken(.Keyword_callconv) orelse return null;
	2900	fn parseCallconv(p: *Parser) !Node.Index {
	2901	_ = p.eatToken(.Keyword_callconv) orelse return null_node;
2002	2902	_ = try p.expectToken(.LParen);
2003		const expr_node = try p.expectNode(parseExpr, .{
2004		.ExpectedExpr = .{ .token = p.tok_i },
2005		});
	2903	const expr_node = try p.expectExpr();
2006	2904	_ = try p.expectToken(.RParen);
2007	2905	return expr_node;
2008	2906	}
2009	2907
2010		/// ParamDecl <- (KEYWORD_noalias / KEYWORD_comptime)? (IDENTIFIER COLON)? ParamType
2011		fn parseParamDecl(p: *Parser) !?Node.FnProto.ParamDecl {
2012		const doc_comments = try p.parseDocComment();
2013		const noalias_token = p.eatToken(.Keyword_noalias);
2014		const comptime_token = if (noalias_token == null) p.eatToken(.Keyword_comptime) else null;
2015		const name_token = blk: {
2016		const identifier = p.eatToken(.Identifier) orelse break :blk null;
2017		if (p.eatToken(.Colon) != null) break :blk identifier;
2018		p.putBackToken(identifier); // ParamType may also be an identifier
2019		break :blk null;
2020		};
2021		const param_type = (try p.parseParamType()) orelse {
2022		// Only return cleanly if no keyword, identifier, or doc comment was found
2023		if (noalias_token == null and
2024		comptime_token == null and
2025		name_token == null and
2026		doc_comments == null)
2027		{
2028		return null;
2029		}
2030		try p.errors.append(p.gpa, .{
2031		.ExpectedParamType = .{ .token = p.tok_i },
2032		});
2033		return error.ParseError;
2034		};
2035
2036		return Node.FnProto.ParamDecl{
2037		.doc_comments = doc_comments,
2038		.comptime_token = comptime_token,
2039		.noalias_token = noalias_token,
2040		.name_token = name_token,
2041		.param_type = param_type,
2042		};
2043		}
2044
	2908	/// ParamDecl
	2909	/// <- (KEYWORD_noalias / KEYWORD_comptime)? (IDENTIFIER COLON)? ParamType
	2910	/// / DOT3
2045	2911	/// ParamType
2046	2912	/// <- Keyword_anytype
2047		/// / DOT3
2048	2913	/// / TypeExpr
2049		fn parseParamType(p: *Parser) !?Node.FnProto.ParamDecl.ParamType {
2050		// TODO cast from tuple to error union is broken
2051		const P = Node.FnProto.ParamDecl.ParamType;
2052		if (try p.parseAnyType()) \|node\| return P{ .any_type = node };
2053		if (try p.parseTypeExpr()) \|node\| return P{ .type_expr = node };
2054		return null;
	2914	/// This function can return null nodes and then still return nodes afterwards,
	2915	/// such as in the case of anytype and `...`. Caller must look for rparen to find
	2916	/// out when there are no more param decls left.
	2917	fn expectParamDecl(p: *Parser) !Node.Index {
	2918	_ = p.eatDocComments();
	2919	switch (p.token_tags[p.tok_i]) {
	2920	.Keyword_noalias, .Keyword_comptime => p.tok_i += 1,
	2921	.Ellipsis3 => {
	2922	p.tok_i += 1;
	2923	return null_node;
	2924	},
	2925	else => {},
	2926	}
	2927	if (p.token_tags[p.tok_i] == .Identifier and
	2928	p.token_tags[p.tok_i + 1] == .Colon)
	2929	{
	2930	p.tok_i += 2;
	2931	}
	2932	switch (p.token_tags[p.tok_i]) {
	2933	.Keyword_anytype => {
	2934	p.tok_i += 1;
	2935	return null_node;
	2936	},
	2937	else => return p.expectTypeExpr(),
	2938	}
2055	2939	}
2056	2940
2057		/// IfPrefix <- KEYWORD_if LPAREN Expr RPAREN PtrPayload?
2058		fn parseIfPrefix(p: Parser) !?Node {
2059		const if_token = p.eatToken(.Keyword_if) orelse return null;
2060		_ = try p.expectToken(.LParen);
2061		const condition = try p.expectNode(parseExpr, .{
2062		.ExpectedExpr = .{ .token = p.tok_i },
2063		});
2064		_ = try p.expectToken(.RParen);
2065		const payload = try p.parsePtrPayload();
2066
2067		const node = try p.arena.allocator.create(Node.If);
2068		node.* = .{
2069		.if_token = if_token,
2070		.condition = condition,
2071		.payload = payload,
2072		.body = undefined, // set by caller
2073		.@"else" = null,
2074		};
2075		return &node.base;
	2941	/// Payload <- PIPE IDENTIFIER PIPE
	2942	fn parsePayload(p: *Parser) !TokenIndex {
	2943	_ = p.eatToken(.Pipe) orelse return @as(TokenIndex, 0);
	2944	const identifier = try p.expectToken(.Identifier);
	2945	_ = try p.expectToken(.Pipe);
	2946	return identifier;
2076	2947	}
2077	2948
2078		/// WhilePrefix <- KEYWORD_while LPAREN Expr RPAREN PtrPayload? WhileContinueExpr?
2079		fn parseWhilePrefix(p: Parser) !?Node {
2080		const while_token = p.eatToken(.Keyword_while) orelse return null;
	2949	/// PtrPayload <- PIPE ASTERISK? IDENTIFIER PIPE
	2950	fn parsePtrPayload(p: *Parser) !TokenIndex {
	2951	_ = p.eatToken(.Pipe) orelse return @as(TokenIndex, 0);
	2952	_ = p.eatToken(.Asterisk);
	2953	const identifier = try p.expectToken(.Identifier);
	2954	_ = try p.expectToken(.Pipe);
	2955	return identifier;
	2956	}
2081	2957
2082		_ = try p.expectToken(.LParen);
2083		const condition = try p.expectNode(parseExpr, .{
2084		.ExpectedExpr = .{ .token = p.tok_i },
2085		});
2086		_ = try p.expectToken(.RParen);
	2958	/// PtrIndexPayload <- PIPE ASTERISK? IDENTIFIER (COMMA IDENTIFIER)? PIPE
	2959	/// Returns the first identifier token, if any.
	2960	fn parsePtrIndexPayload(p: *Parser) !TokenIndex {
	2961	_ = p.eatToken(.Pipe) orelse return @as(TokenIndex, 0);
	2962	_ = p.eatToken(.Asterisk);
	2963	const identifier = try p.expectToken(.Identifier);
	2964	if (p.eatToken(.Comma) != null) {
	2965	_ = try p.expectToken(.Identifier);
	2966	}
	2967	_ = try p.expectToken(.Pipe);
	2968	return identifier;
	2969	}
2087	2970
2088		const payload = try p.parsePtrPayload();
2089		const continue_expr = try p.parseWhileContinueExpr();
	2971	/// SwitchProng <- SwitchCase EQUALRARROW PtrPayload? AssignExpr
	2972	/// SwitchCase
	2973	/// <- SwitchItem (COMMA SwitchItem)* COMMA?
	2974	/// / KEYWORD_else
	2975	fn parseSwitchProng(p: *Parser) !Node.Index {
	2976	if (p.eatToken(.Keyword_else)) \|_\| {
	2977	const arrow_token = try p.expectToken(.EqualAngleBracketRight);
	2978	_ = try p.parsePtrPayload();
	2979	return p.addNode(.{
	2980	.tag = .SwitchCaseOne,
	2981	.main_token = arrow_token,
	2982	.data = .{
	2983	.lhs = 0,
	2984	.rhs = try p.expectAssignExpr(),
	2985	},
	2986	});
	2987	}
	2988	const first_item = try p.parseSwitchItem();
	2989	if (first_item == 0) return null_node;
	2990
	2991	if (p.token_tags[p.tok_i] == .RBrace) {
	2992	const arrow_token = try p.expectToken(.EqualAngleBracketRight);
	2993	_ = try p.parsePtrPayload();
	2994	return p.addNode(.{
	2995	.tag = .SwitchCaseOne,
	2996	.main_token = arrow_token,
	2997	.data = .{
	2998	.lhs = first_item,
	2999	.rhs = try p.expectAssignExpr(),
	3000	},
	3001	});
	3002	}
2090	3003
2091		const node = try p.arena.allocator.create(Node.While);
2092		node.* = .{
2093		.label = null,
2094		.inline_token = null,
2095		.while_token = while_token,
2096		.condition = condition,
2097		.payload = payload,
2098		.continue_expr = continue_expr,
2099		.body = undefined, // set by caller
2100		.@"else" = null,
2101		};
2102		return &node.base;
2103		}
2104
2105		/// ForPrefix <- KEYWORD_for LPAREN Expr RPAREN PtrIndexPayload
2106		fn parseForPrefix(p: Parser) !?Node {
2107		const for_token = p.eatToken(.Keyword_for) orelse return null;
2108
2109		_ = try p.expectToken(.LParen);
2110		const array_expr = try p.expectNode(parseExpr, .{
2111		.ExpectedExpr = .{ .token = p.tok_i },
2112		});
2113		_ = try p.expectToken(.RParen);
2114
2115		const payload = try p.expectNode(parsePtrIndexPayload, .{
2116		.ExpectedPayload = .{ .token = p.tok_i },
2117		});
2118
2119		const node = try p.arena.allocator.create(Node.For);
2120		node.* = .{
2121		.label = null,
2122		.inline_token = null,
2123		.for_token = for_token,
2124		.array_expr = array_expr,
2125		.payload = payload,
2126		.body = undefined, // set by caller
2127		.@"else" = null,
2128		};
2129		return &node.base;
2130		}
2131
2132		/// Payload <- PIPE IDENTIFIER PIPE
2133		fn parsePayload(p: Parser) !?Node {
2134		const lpipe = p.eatToken(.Pipe) orelse return null;
2135		const identifier = try p.expectNode(parseIdentifier, .{
2136		.ExpectedIdentifier = .{ .token = p.tok_i },
2137		});
2138		const rpipe = try p.expectToken(.Pipe);
2139
2140		const node = try p.arena.allocator.create(Node.Payload);
2141		node.* = .{
2142		.lpipe = lpipe,
2143		.error_symbol = identifier,
2144		.rpipe = rpipe,
2145		};
2146		return &node.base;
2147		}
2148
2149		/// PtrPayload <- PIPE ASTERISK? IDENTIFIER PIPE
2150		fn parsePtrPayload(p: Parser) !?Node {
2151		const lpipe = p.eatToken(.Pipe) orelse return null;
2152		const asterisk = p.eatToken(.Asterisk);
2153		const identifier = try p.expectNode(parseIdentifier, .{
2154		.ExpectedIdentifier = .{ .token = p.tok_i },
2155		});
2156		const rpipe = try p.expectToken(.Pipe);
2157
2158		const node = try p.arena.allocator.create(Node.PointerPayload);
2159		node.* = .{
2160		.lpipe = lpipe,
2161		.ptr_token = asterisk,
2162		.value_symbol = identifier,
2163		.rpipe = rpipe,
2164		};
2165		return &node.base;
2166		}
2167
2168		/// PtrIndexPayload <- PIPE ASTERISK? IDENTIFIER (COMMA IDENTIFIER)? PIPE
2169		fn parsePtrIndexPayload(p: Parser) !?Node {
2170		const lpipe = p.eatToken(.Pipe) orelse return null;
2171		const asterisk = p.eatToken(.Asterisk);
2172		const identifier = try p.expectNode(parseIdentifier, .{
2173		.ExpectedIdentifier = .{ .token = p.tok_i },
2174		});
2175
2176		const index = if (p.eatToken(.Comma) == null)
2177		null
2178		else
2179		try p.expectNode(parseIdentifier, .{
2180		.ExpectedIdentifier = .{ .token = p.tok_i },
2181		});
2182
2183		const rpipe = try p.expectToken(.Pipe);
2184
2185		const node = try p.arena.allocator.create(Node.PointerIndexPayload);
2186		node.* = .{
2187		.lpipe = lpipe,
2188		.ptr_token = asterisk,
2189		.value_symbol = identifier,
2190		.index_symbol = index,
2191		.rpipe = rpipe,
2192		};
2193		return &node.base;
2194		}
2195
2196		/// SwitchProng <- SwitchCase EQUALRARROW PtrPayload? AssignExpr
2197		fn parseSwitchProng(p: Parser) !?Node {
2198		const node = (try p.parseSwitchCase()) orelse return null;
2199		const arrow = try p.expectToken(.EqualAngleBracketRight);
2200		const payload = try p.parsePtrPayload();
2201		const expr = try p.expectNode(parseAssignExpr, .{
2202		.ExpectedExprOrAssignment = .{ .token = p.tok_i },
2203		});
2204
2205		const switch_case = node.cast(Node.SwitchCase).?;
2206		switch_case.arrow_token = arrow;
2207		switch_case.payload = payload;
2208		switch_case.expr = expr;
2209
2210		return node;
2211		}
2212
2213		/// SwitchCase
2214		/// <- SwitchItem (COMMA SwitchItem)* COMMA?
2215		/// / KEYWORD_else
2216		fn parseSwitchCase(p: Parser) !?Node {
2217		var list = std.ArrayList(*Node).init(p.gpa);
2218		defer list.deinit();
2219
2220		if (try p.parseSwitchItem()) \|first_item\| {
2221		try list.append(first_item);
2222		while (p.eatToken(.Comma) != null) {
2223		const next_item = (try p.parseSwitchItem()) orelse break;
2224		try list.append(next_item);
2225		}
2226		} else if (p.eatToken(.Keyword_else)) \|else_token\| {
2227		const else_node = try p.arena.allocator.create(Node.SwitchElse);
2228		else_node.* = .{
2229		.token = else_token,
2230		};
2231		try list.append(&else_node.base);
2232		} else return null;
2233
2234		const node = try Node.SwitchCase.alloc(&p.arena.allocator, list.items.len);
2235		node.* = .{
2236		.items_len = list.items.len,
2237		.arrow_token = undefined, // set by caller
2238		.payload = null,
2239		.expr = undefined, // set by caller
2240		};
2241		std.mem.copy(*Node, node.items(), list.items);
2242		return &node.base;
	3004	var list = std.ArrayList(Node.Index).init(p.gpa);
	3005	defer list.deinit();
	3006
	3007	try list.append(first_item);
	3008	while (p.eatToken(.Comma)) \|_\| {
	3009	const next_item = try p.parseSwitchItem();
	3010	if (next_item == 0) break;
	3011	try list.append(next_item);
	3012	}
	3013	const span = try p.listToSpan(list.items);
	3014	const arrow_token = try p.expectToken(.EqualAngleBracketRight);
	3015	_ = try p.parsePtrPayload();
	3016	return p.addNode(.{
	3017	.tag = .SwitchCaseMulti,
	3018	.main_token = arrow_token,
	3019	.data = .{
	3020	.lhs = try p.addExtra(Node.SubRange{
	3021	.start = span.start,
	3022	.end = span.end,
	3023	}),
	3024	.rhs = try p.expectAssignExpr(),
	3025	},
	3026	});
2243	3027	}
2244	3028
2245	3029	/// SwitchItem <- Expr (DOT3 Expr)?
2246		fn parseSwitchItem(p: Parser) !?Node {
2247		const expr = (try p.parseExpr()) orelse return null;
2248		if (p.eatToken(.Ellipsis3)) \|token\| {
2249		const range_end = try p.expectNode(parseExpr, .{
2250		.ExpectedExpr = .{ .token = p.tok_i },
2251		});
2252
2253		const node = try p.arena.allocator.create(Node.SimpleInfixOp);
2254		node.* = .{
2255		.base = Node{ .tag = .Range },
2256		.op_token = token,
2257		.lhs = expr,
2258		.rhs = range_end,
2259		};
2260		return &node.base;
2261		}
2262		return expr;
2263		}
2264
2265		/// AssignOp
2266		/// <- ASTERISKEQUAL
2267		/// / SLASHEQUAL
2268		/// / PERCENTEQUAL
2269		/// / PLUSEQUAL
2270		/// / MINUSEQUAL
2271		/// / LARROW2EQUAL
2272		/// / RARROW2EQUAL
2273		/// / AMPERSANDEQUAL
2274		/// / CARETEQUAL
2275		/// / PIPEEQUAL
2276		/// / ASTERISKPERCENTEQUAL
2277		/// / PLUSPERCENTEQUAL
2278		/// / MINUSPERCENTEQUAL
2279		/// / EQUAL
2280		fn parseAssignOp(p: Parser) !?Node {
2281		const token = p.nextToken();
2282		const op: Node.Tag = switch (p.token_ids[token]) {
2283		.AsteriskEqual => .AssignMul,
2284		.SlashEqual => .AssignDiv,
2285		.PercentEqual => .AssignMod,
2286		.PlusEqual => .AssignAdd,
2287		.MinusEqual => .AssignSub,
2288		.AngleBracketAngleBracketLeftEqual => .AssignBitShiftLeft,
2289		.AngleBracketAngleBracketRightEqual => .AssignBitShiftRight,
2290		.AmpersandEqual => .AssignBitAnd,
2291		.CaretEqual => .AssignBitXor,
2292		.PipeEqual => .AssignBitOr,
2293		.AsteriskPercentEqual => .AssignMulWrap,
2294		.PlusPercentEqual => .AssignAddWrap,
2295		.MinusPercentEqual => .AssignSubWrap,
2296		.Equal => .Assign,
2297		else => {
2298		p.putBackToken(token);
2299		return null;
2300		},
2301		};
2302
2303		const node = try p.arena.allocator.create(Node.SimpleInfixOp);
2304		node.* = .{
2305		.base = .{ .tag = op },
2306		.op_token = token,
2307		.lhs = undefined, // set by caller
2308		.rhs = undefined, // set by caller
2309		};
2310		return &node.base;
2311		}
2312
2313		/// CompareOp
2314		/// <- EQUALEQUAL
2315		/// / EXCLAMATIONMARKEQUAL
2316		/// / LARROW
2317		/// / RARROW
2318		/// / LARROWEQUAL
2319		/// / RARROWEQUAL
2320		fn parseCompareOp(p: Parser) !?Node {
2321		const token = p.nextToken();
2322		const op: Node.Tag = switch (p.token_ids[token]) {
2323		.EqualEqual => .EqualEqual,
2324		.BangEqual => .BangEqual,
2325		.AngleBracketLeft => .LessThan,
2326		.AngleBracketRight => .GreaterThan,
2327		.AngleBracketLeftEqual => .LessOrEqual,
2328		.AngleBracketRightEqual => .GreaterOrEqual,
2329		else => {
2330		p.putBackToken(token);
2331		return null;
2332		},
2333		};
2334
2335		return p.createInfixOp(token, op);
2336		}
2337
2338		/// BitwiseOp
2339		/// <- AMPERSAND
2340		/// / CARET
2341		/// / PIPE
2342		/// / KEYWORD_orelse
2343		/// / KEYWORD_catch Payload?
2344		fn parseBitwiseOp(p: Parser) !?Node {
2345		const token = p.nextToken();
2346		const op: Node.Tag = switch (p.token_ids[token]) {
2347		.Ampersand => .BitAnd,
2348		.Caret => .BitXor,
2349		.Pipe => .BitOr,
2350		.Keyword_orelse => .OrElse,
2351		.Keyword_catch => {
2352		const payload = try p.parsePayload();
2353		const node = try p.arena.allocator.create(Node.Catch);
2354		node.* = .{
2355		.op_token = token,
2356		.lhs = undefined, // set by caller
2357		.rhs = undefined, // set by caller
2358		.payload = payload,
2359		};
2360		return &node.base;
2361		},
2362		else => {
2363		p.putBackToken(token);
2364		return null;
2365		},
2366		};
2367
2368		return p.createInfixOp(token, op);
2369		}
2370
2371		/// BitShiftOp
2372		/// <- LARROW2
2373		/// / RARROW2
2374		fn parseBitShiftOp(p: Parser) !?Node {
2375		const token = p.nextToken();
2376		const op: Node.Tag = switch (p.token_ids[token]) {
2377		.AngleBracketAngleBracketLeft => .BitShiftLeft,
2378		.AngleBracketAngleBracketRight => .BitShiftRight,
2379		else => {
2380		p.putBackToken(token);
2381		return null;
2382		},
2383		};
2384
2385		return p.createInfixOp(token, op);
2386		}
2387
2388		/// AdditionOp
2389		/// <- PLUS
2390		/// / MINUS
2391		/// / PLUS2
2392		/// / PLUSPERCENT
2393		/// / MINUSPERCENT
2394		fn parseAdditionOp(p: Parser) !?Node {
2395		const token = p.nextToken();
2396		const op: Node.Tag = switch (p.token_ids[token]) {
2397		.Plus => .Add,
2398		.Minus => .Sub,
2399		.PlusPlus => .ArrayCat,
2400		.PlusPercent => .AddWrap,
2401		.MinusPercent => .SubWrap,
2402		else => {
2403		p.putBackToken(token);
2404		return null;
2405		},
2406		};
2407
2408		return p.createInfixOp(token, op);
2409		}
2410
2411		/// MultiplyOp
2412		/// <- PIPE2
2413		/// / ASTERISK
2414		/// / SLASH
2415		/// / PERCENT
2416		/// / ASTERISK2
2417		/// / ASTERISKPERCENT
2418		fn parseMultiplyOp(p: Parser) !?Node {
2419		const token = p.nextToken();
2420		const op: Node.Tag = switch (p.token_ids[token]) {
2421		.PipePipe => .MergeErrorSets,
2422		.Asterisk => .Mul,
2423		.Slash => .Div,
2424		.Percent => .Mod,
2425		.AsteriskAsterisk => .ArrayMult,
2426		.AsteriskPercent => .MulWrap,
2427		else => {
2428		p.putBackToken(token);
2429		return null;
2430		},
2431		};
2432
2433		return p.createInfixOp(token, op);
2434		}
2435
2436		/// PrefixOp
2437		/// <- EXCLAMATIONMARK
2438		/// / MINUS
2439		/// / TILDE
2440		/// / MINUSPERCENT
2441		/// / AMPERSAND
2442		/// / KEYWORD_try
2443		/// / KEYWORD_await
2444		fn parsePrefixOp(p: Parser) !?Node {
2445		const token = p.nextToken();
2446		switch (p.token_ids[token]) {
2447		.Bang => return p.allocSimplePrefixOp(.BoolNot, token),
2448		.Minus => return p.allocSimplePrefixOp(.Negation, token),
2449		.Tilde => return p.allocSimplePrefixOp(.BitNot, token),
2450		.MinusPercent => return p.allocSimplePrefixOp(.NegationWrap, token),
2451		.Ampersand => return p.allocSimplePrefixOp(.AddressOf, token),
2452		.Keyword_try => return p.allocSimplePrefixOp(.Try, token),
2453		.Keyword_await => return p.allocSimplePrefixOp(.Await, token),
2454		else => {
2455		p.putBackToken(token);
2456		return null;
2457		},
2458		}
2459		}
2460
2461		fn allocSimplePrefixOp(p: Parser, comptime tag: Node.Tag, token: TokenIndex) !?Node {
2462		const node = try p.arena.allocator.create(Node.SimplePrefixOp);
2463		node.* = .{
2464		.base = .{ .tag = tag },
2465		.op_token = token,
2466		.rhs = undefined, // set by caller
2467		};
2468		return &node.base;
2469		}
2470
2471		// TODO: ArrayTypeStart is either an array or a slice, but const/allowzero only work on
2472		// pointers. Consider updating this rule:
2473		// ...
2474		// / ArrayTypeStart
2475		// / SliceTypeStart (ByteAlign / KEYWORD_const / KEYWORD_volatile / KEYWORD_allowzero)*
2476		// / PtrTypeStart ...
2477
2478		/// PrefixTypeOp
2479		/// <- QUESTIONMARK
2480		/// / KEYWORD_anyframe MINUSRARROW
2481		/// / ArrayTypeStart (ByteAlign / KEYWORD_const / KEYWORD_volatile / KEYWORD_allowzero)*
2482		/// / PtrTypeStart (KEYWORD_align LPAREN Expr (COLON INTEGER COLON INTEGER)? RPAREN / KEYWORD_const / KEYWORD_volatile / KEYWORD_allowzero)*
2483		fn parsePrefixTypeOp(p: Parser) !?Node {
2484		if (p.eatToken(.QuestionMark)) \|token\| {
2485		const node = try p.arena.allocator.create(Node.SimplePrefixOp);
2486		node.* = .{
2487		.base = .{ .tag = .OptionalType },
2488		.op_token = token,
2489		.rhs = undefined, // set by caller
2490		};
2491		return &node.base;
2492		}
	3030	fn parseSwitchItem(p: *Parser) !Node.Index {
	3031	const expr = try p.parseExpr();
	3032	if (expr == 0) return null_node;
2493	3033
2494		if (p.eatToken(.Keyword_anyframe)) \|token\| {
2495		const arrow = p.eatToken(.Arrow) orelse {
2496		p.putBackToken(token);
2497		return null;
2498		};
2499		const node = try p.arena.allocator.create(Node.AnyFrameType);
2500		node.* = .{
2501		.anyframe_token = token,
2502		.result = .{
2503		.arrow_token = arrow,
2504		.return_type = undefined, // set by caller
	3034	if (p.eatToken(.Ellipsis3)) \|token\| {
	3035	return p.addNode(.{
	3036	.tag = .SwitchRange,
	3037	.main_token = token,
	3038	.data = .{
	3039	.lhs = expr,
	3040	.rhs = try p.expectExpr(),
2505	3041	},
2506		};
2507		return &node.base;
2508		}
2509
2510		if (try p.parsePtrTypeStart()) \|node\| {
2511		// If the token encountered was **, there will be two nodes instead of one.
2512		// The attributes should be applied to the rightmost operator.
2513		var ptr_info = if (node.cast(Node.PtrType)) \|ptr_type\|
2514		if (p.token_ids[ptr_type.op_token] == .AsteriskAsterisk)
2515		&ptr_type.rhs.cast(Node.PtrType).?.ptr_info
2516		else
2517		&ptr_type.ptr_info
2518		else if (node.cast(Node.SliceType)) \|slice_type\|
2519		&slice_type.ptr_info
2520		else
2521		unreachable;
2522
2523		while (true) {
2524		if (p.eatToken(.Keyword_align)) \|align_token\| {
2525		const lparen = try p.expectToken(.LParen);
2526		const expr_node = try p.expectNode(parseExpr, .{
2527		.ExpectedExpr = .{ .token = p.tok_i },
2528		});
2529
2530		// Optional bit range
2531		const bit_range = if (p.eatToken(.Colon)) \|_\| bit_range_value: {
2532		const range_start = try p.expectNode(parseIntegerLiteral, .{
2533		.ExpectedIntegerLiteral = .{ .token = p.tok_i },
2534		});
2535		_ = try p.expectToken(.Colon);
2536		const range_end = try p.expectNode(parseIntegerLiteral, .{
2537		.ExpectedIntegerLiteral = .{ .token = p.tok_i },
2538		});
2539
2540		break :bit_range_value ast.PtrInfo.Align.BitRange{
2541		.start = range_start,
2542		.end = range_end,
2543		};
2544		} else null;
2545		_ = try p.expectToken(.RParen);
2546
2547		if (ptr_info.align_info != null) {
2548		try p.errors.append(p.gpa, .{
2549		.ExtraAlignQualifier = .{ .token = p.tok_i - 1 },
2550		});
2551		continue;
2552		}
2553
2554		ptr_info.align_info = ast.PtrInfo.Align{
2555		.node = expr_node,
2556		.bit_range = bit_range,
2557		};
2558
2559		continue;
2560		}
2561		if (p.eatToken(.Keyword_const)) \|const_token\| {
2562		if (ptr_info.const_token != null) {
2563		try p.errors.append(p.gpa, .{
2564		.ExtraConstQualifier = .{ .token = p.tok_i - 1 },
2565		});
2566		continue;
2567		}
2568		ptr_info.const_token = const_token;
2569		continue;
2570		}
2571		if (p.eatToken(.Keyword_volatile)) \|volatile_token\| {
2572		if (ptr_info.volatile_token != null) {
2573		try p.errors.append(p.gpa, .{
2574		.ExtraVolatileQualifier = .{ .token = p.tok_i - 1 },
2575		});
2576		continue;
2577		}
2578		ptr_info.volatile_token = volatile_token;
2579		continue;
2580		}
2581		if (p.eatToken(.Keyword_allowzero)) \|allowzero_token\| {
2582		if (ptr_info.allowzero_token != null) {
2583		try p.errors.append(p.gpa, .{
2584		.ExtraAllowZeroQualifier = .{ .token = p.tok_i - 1 },
2585		});
2586		continue;
2587		}
2588		ptr_info.allowzero_token = allowzero_token;
2589		continue;
2590		}
2591		break;
2592		}
2593
2594		return node;
2595		}
2596
2597		if (try p.parseArrayTypeStart()) \|node\| {
2598		if (node.cast(Node.SliceType)) \|slice_type\| {
2599		// Collect pointer qualifiers in any order, but disallow duplicates
2600		while (true) {
2601		if (try p.parseByteAlign()) \|align_expr\| {
2602		if (slice_type.ptr_info.align_info != null) {
2603		try p.errors.append(p.gpa, .{
2604		.ExtraAlignQualifier = .{ .token = p.tok_i - 1 },
2605		});
2606		continue;
2607		}
2608		slice_type.ptr_info.align_info = ast.PtrInfo.Align{
2609		.node = align_expr,
2610		.bit_range = null,
2611		};
2612		continue;
2613		}
2614		if (p.eatToken(.Keyword_const)) \|const_token\| {
2615		if (slice_type.ptr_info.const_token != null) {
2616		try p.errors.append(p.gpa, .{
2617		.ExtraConstQualifier = .{ .token = p.tok_i - 1 },
2618		});
2619		continue;
2620		}
2621		slice_type.ptr_info.const_token = const_token;
2622		continue;
2623		}
2624		if (p.eatToken(.Keyword_volatile)) \|volatile_token\| {
2625		if (slice_type.ptr_info.volatile_token != null) {
2626		try p.errors.append(p.gpa, .{
2627		.ExtraVolatileQualifier = .{ .token = p.tok_i - 1 },
2628		});
2629		continue;
2630		}
2631		slice_type.ptr_info.volatile_token = volatile_token;
2632		continue;
2633		}
2634		if (p.eatToken(.Keyword_allowzero)) \|allowzero_token\| {
2635		if (slice_type.ptr_info.allowzero_token != null) {
2636		try p.errors.append(p.gpa, .{
2637		.ExtraAllowZeroQualifier = .{ .token = p.tok_i - 1 },
2638		});
2639		continue;
2640		}
2641		slice_type.ptr_info.allowzero_token = allowzero_token;
2642		continue;
2643		}
2644		break;
2645		}
2646		}
2647		return node;
2648		}
2649
2650		return null;
2651		}
2652
2653		/// SuffixOp
2654		/// <- LBRACKET Expr (DOT2 (Expr (COLON Expr)?)?)? RBRACKET
2655		/// / DOT IDENTIFIER
2656		/// / DOTASTERISK
2657		/// / DOTQUESTIONMARK
2658		fn parseSuffixOp(p: Parser, lhs: Node) !?*Node {
2659		if (p.eatToken(.LBracket)) \|_\| {
2660		const index_expr = try p.expectNode(parseExpr, .{
2661		.ExpectedExpr = .{ .token = p.tok_i },
2662		});
2663
2664		if (p.eatToken(.Ellipsis2) != null) {
2665		const end_expr = try p.parseExpr();
2666		const sentinel: ?*Node = if (p.eatToken(.Colon) != null)
2667		try p.parseExpr()
2668		else
2669		null;
2670		const rtoken = try p.expectToken(.RBracket);
2671		const node = try p.arena.allocator.create(Node.Slice);
2672		node.* = .{
2673		.lhs = lhs,
2674		.rtoken = rtoken,
2675		.start = index_expr,
2676		.end = end_expr,
2677		.sentinel = sentinel,
2678		};
2679		return &node.base;
2680		}
2681
2682		const rtoken = try p.expectToken(.RBracket);
2683		const node = try p.arena.allocator.create(Node.ArrayAccess);
2684		node.* = .{
2685		.lhs = lhs,
2686		.rtoken = rtoken,
2687		.index_expr = index_expr,
2688		};
2689		return &node.base;
2690		}
2691
2692		if (p.eatToken(.PeriodAsterisk)) \|period_asterisk\| {
2693		const node = try p.arena.allocator.create(Node.SimpleSuffixOp);
2694		node.* = .{
2695		.base = .{ .tag = .Deref },
2696		.lhs = lhs,
2697		.rtoken = period_asterisk,
2698		};
2699		return &node.base;
2700		}
2701
2702		if (p.eatToken(.Invalid_periodasterisks)) \|period_asterisk\| {
2703		try p.errors.append(p.gpa, .{
2704		.AsteriskAfterPointerDereference = .{ .token = period_asterisk },
2705		});
2706		const node = try p.arena.allocator.create(Node.SimpleSuffixOp);
2707		node.* = .{
2708		.base = .{ .tag = .Deref },
2709		.lhs = lhs,
2710		.rtoken = period_asterisk,
2711		};
2712		return &node.base;
2713		}
2714
2715		if (p.eatToken(.Period)) \|period\| {
2716		if (try p.parseIdentifier()) \|identifier\| {
2717		const node = try p.arena.allocator.create(Node.SimpleInfixOp);
2718		node.* = .{
2719		.base = Node{ .tag = .Period },
2720		.op_token = period,
2721		.lhs = lhs,
2722		.rhs = identifier,
2723		};
2724		return &node.base;
2725		}
2726		if (p.eatToken(.QuestionMark)) \|question_mark\| {
2727		const node = try p.arena.allocator.create(Node.SimpleSuffixOp);
2728		node.* = .{
2729		.base = .{ .tag = .UnwrapOptional },
2730		.lhs = lhs,
2731		.rtoken = question_mark,
2732		};
2733		return &node.base;
2734		}
2735		try p.errors.append(p.gpa, .{
2736		.ExpectedSuffixOp = .{ .token = p.tok_i },
2737	3042	});
2738		return null;
2739	3043	}
2740
2741		return null;
2742		}
2743
2744		/// FnCallArguments <- LPAREN ExprList RPAREN
2745		/// ExprList <- (Expr COMMA)* Expr?
2746		fn parseFnCallArguments(p: *Parser) !?AnnotatedParamList {
2747		if (p.eatToken(.LParen) == null) return null;
2748		const list = try ListParseFn(*Node, parseExpr)(p);
2749		errdefer p.gpa.free(list);
2750		const rparen = try p.expectToken(.RParen);
2751		return AnnotatedParamList{ .list = list, .rparen = rparen };
2752		}
2753
2754		const AnnotatedParamList = struct {
2755		list: []*Node,
2756		rparen: TokenIndex,
2757		};
2758
2759		/// ArrayTypeStart <- LBRACKET Expr? (COLON Expr)? RBRACKET
2760		fn parseArrayTypeStart(p: Parser) !?Node {
2761		const lbracket = p.eatToken(.LBracket) orelse return null;
2762		const expr = try p.parseExpr();
2763		const sentinel = if (p.eatToken(.Colon)) \|_\|
2764		try p.expectNode(parseExpr, .{
2765		.ExpectedExpr = .{ .token = p.tok_i },
2766		})
2767		else
2768		null;
2769		const rbracket = try p.expectToken(.RBracket);
2770
2771		if (expr) \|len_expr\| {
2772		if (sentinel) \|s\| {
2773		const node = try p.arena.allocator.create(Node.ArrayTypeSentinel);
2774		node.* = .{
2775		.op_token = lbracket,
2776		.rhs = undefined, // set by caller
2777		.len_expr = len_expr,
2778		.sentinel = s,
2779		};
2780		return &node.base;
2781		} else {
2782		const node = try p.arena.allocator.create(Node.ArrayType);
2783		node.* = .{
2784		.op_token = lbracket,
2785		.rhs = undefined, // set by caller
2786		.len_expr = len_expr,
2787		};
2788		return &node.base;
2789		}
2790		}
2791
2792		const node = try p.arena.allocator.create(Node.SliceType);
2793		node.* = .{
2794		.op_token = lbracket,
2795		.rhs = undefined, // set by caller
2796		.ptr_info = .{ .sentinel = sentinel },
2797		};
2798		return &node.base;
	3044	return expr;
2799	3045	}
2800	3046
2801		/// PtrTypeStart
2802		/// <- ASTERISK
2803		/// / ASTERISK2
2804		/// / LBRACKET ASTERISK (LETTERC / COLON Expr)? RBRACKET
2805		fn parsePtrTypeStart(p: Parser) !?Node {
2806		if (p.eatToken(.Asterisk)) \|asterisk\| {
2807		const sentinel = if (p.eatToken(.Colon)) \|_\|
2808		try p.expectNode(parseExpr, .{
2809		.ExpectedExpr = .{ .token = p.tok_i },
2810		})
2811		else
2812		null;
2813		const node = try p.arena.allocator.create(Node.PtrType);
2814		node.* = .{
2815		.op_token = asterisk,
2816		.rhs = undefined, // set by caller
2817		.ptr_info = .{ .sentinel = sentinel },
2818		};
2819		return &node.base;
2820		}
	3047	const PtrModifiers = struct {
	3048	align_node: Node.Index,
	3049	bit_range_start: Node.Index,
	3050	bit_range_end: Node.Index,
	3051	};
2821	3052
2822		if (p.eatToken(.AsteriskAsterisk)) \|double_asterisk\| {
2823		const node = try p.arena.allocator.create(Node.PtrType);
2824		node.* = .{
2825		.op_token = double_asterisk,
2826		.rhs = undefined, // set by caller
2827		};
	3053	fn parsePtrModifiers(p: *Parser) !PtrModifiers {
	3054	var result: PtrModifiers = .{
	3055	.align_node = 0,
	3056	.bit_range_start = 0,
	3057	.bit_range_end = 0,
	3058	};
	3059	var saw_const = false;
	3060	var saw_volatile = false;
	3061	var saw_allowzero = false;
	3062	while (true) {
	3063	switch (p.token_tags[p.tok_i]) {
	3064	.Keyword_align => {
	3065	if (result.align_node != 0) {
	3066	try p.warn(.{
	3067	.ExtraAlignQualifier = .{ .token = p.tok_i },
	3068	});
	3069	}
	3070	p.tok_i += 1;
	3071	_ = try p.expectToken(.LParen);
	3072	result.align_node = try p.expectExpr();
2828	3073
2829		// Special case for **, which is its own token
2830		const child = try p.arena.allocator.create(Node.PtrType);
2831		child.* = .{
2832		.op_token = double_asterisk,
2833		.rhs = undefined, // set by caller
2834		};
2835		node.rhs = &child.base;
	3074	if (p.eatToken(.Colon)) \|_\| {
	3075	result.bit_range_start = try p.expectExpr();
	3076	_ = try p.expectToken(.Colon);
	3077	result.bit_range_end = try p.expectExpr();
	3078	}
2836	3079
2837		return &node.base;
2838		}
2839		if (p.eatToken(.LBracket)) \|lbracket\| {
2840		const asterisk = p.eatToken(.Asterisk) orelse {
2841		p.putBackToken(lbracket);
2842		return null;
2843		};
2844		if (p.eatToken(.Identifier)) \|ident\| {
2845		const token_loc = p.token_locs[ident];
2846		const token_slice = p.source[token_loc.start..token_loc.end];
2847		if (!std.mem.eql(u8, token_slice, "c")) {
2848		p.putBackToken(ident);
2849		} else {
2850		_ = try p.expectToken(.RBracket);
2851		const node = try p.arena.allocator.create(Node.PtrType);
2852		node.* = .{
2853		.op_token = lbracket,
2854		.rhs = undefined, // set by caller
2855		};
2856		return &node.base;
2857		}
	3080	_ = try p.expectToken(.RParen);
	3081	},
	3082	.Keyword_const => {
	3083	if (saw_const) {
	3084	try p.warn(.{
	3085	.ExtraConstQualifier = .{ .token = p.tok_i },
	3086	});
	3087	}
	3088	p.tok_i += 1;
	3089	saw_const = true;
	3090	},
	3091	.Keyword_volatile => {
	3092	if (saw_volatile) {
	3093	try p.warn(.{
	3094	.ExtraVolatileQualifier = .{ .token = p.tok_i },
	3095	});
	3096	}
	3097	p.tok_i += 1;
	3098	saw_volatile = true;
	3099	},
	3100	.Keyword_allowzero => {
	3101	if (saw_allowzero) {
	3102	try p.warn(.{
	3103	.ExtraAllowZeroQualifier = .{ .token = p.tok_i },
	3104	});
	3105	}
	3106	p.tok_i += 1;
	3107	saw_allowzero = true;
	3108	},
	3109	else => return result,
2858	3110	}
2859		const sentinel = if (p.eatToken(.Colon)) \|_\|
2860		try p.expectNode(parseExpr, .{
2861		.ExpectedExpr = .{ .token = p.tok_i },
2862		})
2863		else
2864		null;
2865		_ = try p.expectToken(.RBracket);
2866		const node = try p.arena.allocator.create(Node.PtrType);
2867		node.* = .{
2868		.op_token = lbracket,
2869		.rhs = undefined, // set by caller
2870		.ptr_info = .{ .sentinel = sentinel },
2871		};
2872		return &node.base;
2873	3111	}
2874		return null;
2875	3112	}
2876	3113
2877		/// ContainerDeclAuto <- ContainerDeclType LBRACE ContainerMembers RBRACE
2878		fn parseContainerDeclAuto(p: Parser) !?Node {
2879		const container_decl_type = (try p.parseContainerDeclType()) orelse return null;
2880		const lbrace = try p.expectToken(.LBrace);
2881		const members = try p.parseContainerMembers(false);
2882		defer p.gpa.free(members);
2883		const rbrace = try p.expectToken(.RBrace);
2884
2885		const members_len = @intCast(NodeIndex, members.len);
2886		const node = try Node.ContainerDecl.alloc(&p.arena.allocator, members_len);
2887		node.* = .{
2888		.layout_token = null,
2889		.kind_token = container_decl_type.kind_token,
2890		.init_arg_expr = container_decl_type.init_arg_expr,
2891		.fields_and_decls_len = members_len,
2892		.lbrace_token = lbrace,
2893		.rbrace_token = rbrace,
2894		};
2895		std.mem.copy(*Node, node.fieldsAndDecls(), members);
2896		return &node.base;
	3114	/// SuffixOp
	3115	/// <- LBRACKET Expr (DOT2 (Expr (COLON Expr)?)?)? RBRACKET
	3116	/// / DOT IDENTIFIER
	3117	/// / DOTASTERISK
	3118	/// / DOTQUESTIONMARK
	3119	fn parseSuffixOp(p: *Parser, lhs: Node.Index) !Node.Index {
	3120	switch (p.token_tags[p.tok_i]) {
	3121	.LBracket => {
	3122	const lbracket = p.nextToken();
	3123	const index_expr = try p.expectExpr();
	3124
	3125	if (p.eatToken(.Ellipsis2)) \|_\| {
	3126	const end_expr = try p.parseExpr();
	3127	if (end_expr == 0) {
	3128	_ = try p.expectToken(.RBracket);
	3129	return p.addNode(.{
	3130	.tag = .SliceOpen,
	3131	.main_token = lbracket,
	3132	.data = .{
	3133	.lhs = lhs,
	3134	.rhs = index_expr,
	3135	},
	3136	});
	3137	}
	3138	const sentinel: Node.Index = if (p.eatToken(.Colon)) \|_\|
	3139	try p.parseExpr()
	3140	else
	3141	0;
	3142	_ = try p.expectToken(.RBracket);
	3143	return p.addNode(.{
	3144	.tag = .Slice,
	3145	.main_token = lbracket,
	3146	.data = .{
	3147	.lhs = lhs,
	3148	.rhs = try p.addExtra(.{
	3149	.start = index_expr,
	3150	.end = end_expr,
	3151	.sentinel = sentinel,
	3152	}),
	3153	},
	3154	});
	3155	}
	3156	_ = try p.expectToken(.RBracket);
	3157	return p.addNode(.{
	3158	.tag = .ArrayAccess,
	3159	.main_token = lbracket,
	3160	.data = .{
	3161	.lhs = lhs,
	3162	.rhs = index_expr,
	3163	},
	3164	});
	3165	},
	3166	.PeriodAsterisk => return p.addNode(.{
	3167	.tag = .Deref,
	3168	.main_token = p.nextToken(),
	3169	.data = .{
	3170	.lhs = lhs,
	3171	.rhs = undefined,
	3172	},
	3173	}),
	3174	.Invalid_periodasterisks => {
	3175	const period_asterisk = p.nextToken();
	3176	try p.warn(.{ .AsteriskAfterPointerDereference = .{ .token = period_asterisk } });
	3177	return p.addNode(.{
	3178	.tag = .Deref,
	3179	.main_token = period_asterisk,
	3180	.data = .{
	3181	.lhs = lhs,
	3182	.rhs = undefined,
	3183	},
	3184	});
	3185	},
	3186	.Period => switch (p.token_tags[p.tok_i + 1]) {
	3187	.Identifier => return p.addNode(.{
	3188	.tag = .FieldAccess,
	3189	.main_token = p.nextToken(),
	3190	.data = .{
	3191	.lhs = lhs,
	3192	.rhs = p.nextToken(),
	3193	},
	3194	}),
	3195	.QuestionMark => return p.addNode(.{
	3196	.tag = .UnwrapOptional,
	3197	.main_token = p.nextToken(),
	3198	.data = .{
	3199	.lhs = lhs,
	3200	.rhs = p.nextToken(),
	3201	},
	3202	}),
	3203	else => {
	3204	p.tok_i += 1;
	3205	try p.warn(.{ .ExpectedSuffixOp = .{ .token = p.tok_i } });
	3206	return null_node;
	3207	},
	3208	},
	3209	else => return null_node,
	3210	}
2897	3211	}
2898	3212
2899		/// Holds temporary data until we are ready to construct the full ContainerDecl AST node.
2900		const ContainerDeclType = struct {
2901		kind_token: TokenIndex,
2902		init_arg_expr: Node.ContainerDecl.InitArg,
2903		};
2904
	3213	/// Caller must have already verified the first token.
2905	3214	/// ContainerDeclType
2906	3215	/// <- KEYWORD_struct
2907	3216	/// / KEYWORD_enum (LPAREN Expr RPAREN)?
2908	3217	/// / KEYWORD_union (LPAREN (KEYWORD_enum (LPAREN Expr RPAREN)? / Expr) RPAREN)?
2909	3218	/// / KEYWORD_opaque
2910		fn parseContainerDeclType(p: *Parser) !?ContainerDeclType {
2911		const kind_token = p.nextToken();
2912
2913		const init_arg_expr = switch (p.token_ids[kind_token]) {
2914		.Keyword_struct, .Keyword_opaque => Node.ContainerDecl.InitArg{ .None = {} },
	3219	fn parseContainerDeclAuto(p: *Parser) !Node.Index {
	3220	const main_token = p.nextToken();
	3221	const arg_expr = switch (p.token_tags[main_token]) {
	3222	.Keyword_struct, .Keyword_opaque => null_node,
2915	3223	.Keyword_enum => blk: {
2916		if (p.eatToken(.LParen) != null) {
2917		const expr = try p.expectNode(parseExpr, .{
2918		.ExpectedExpr = .{ .token = p.tok_i },
2919		});
	3224	if (p.eatToken(.LParen)) \|_\| {
	3225	const expr = try p.expectExpr();
2920	3226	_ = try p.expectToken(.RParen);
2921		break :blk Node.ContainerDecl.InitArg{ .Type = expr };
	3227	break :blk expr;
	3228	} else {
	3229	break :blk null_node;
2922	3230	}
2923		break :blk Node.ContainerDecl.InitArg{ .None = {} };
2924	3231	},
2925	3232	.Keyword_union => blk: {
2926		if (p.eatToken(.LParen) != null) {
2927		if (p.eatToken(.Keyword_enum) != null) {
2928		if (p.eatToken(.LParen) != null) {
2929		const expr = try p.expectNode(parseExpr, .{
2930		.ExpectedExpr = .{ .token = p.tok_i },
2931		});
	3233	if (p.eatToken(.LParen)) \|_\| {
	3234	if (p.eatToken(.Keyword_enum)) \|_\| {
	3235	if (p.eatToken(.LParen)) \|_\| {
	3236	const enum_tag_expr = try p.expectExpr();
	3237	_ = try p.expectToken(.RParen);
2932	3238	_ = try p.expectToken(.RParen);
	3239
	3240	_ = try p.expectToken(.LBrace);
	3241	const members = try p.parseContainerMembers(false);
	3242	_ = try p.expectToken(.RBrace);
	3243	return p.addNode(.{
	3244	.tag = .TaggedUnionEnumTag,
	3245	.main_token = main_token,
	3246	.data = .{
	3247	.lhs = enum_tag_expr,
	3248	.rhs = try p.addExtra(Node.SubRange{
	3249	.start = members.start,
	3250	.end = members.end,
	3251	}),
	3252	},
	3253	});
	3254	} else {
2933	3255	_ = try p.expectToken(.RParen);
2934		break :blk Node.ContainerDecl.InitArg{ .Enum = expr };
	3256
	3257	_ = try p.expectToken(.LBrace);
	3258	const members = try p.parseContainerMembers(false);
	3259	_ = try p.expectToken(.RBrace);
	3260	return p.addNode(.{
	3261	.tag = .TaggedUnion,
	3262	.main_token = main_token,
	3263	.data = .{
	3264	.lhs = members.start,
	3265	.rhs = members.end,
	3266	},
	3267	});
2935	3268	}
	3269	} else {
	3270	const expr = try p.expectExpr();
2936	3271	_ = try p.expectToken(.RParen);
2937		break :blk Node.ContainerDecl.InitArg{ .Enum = null };
	3272	break :blk expr;
2938	3273	}
2939		const expr = try p.expectNode(parseExpr, .{
2940		.ExpectedExpr = .{ .token = p.tok_i },
2941		});
2942		_ = try p.expectToken(.RParen);
2943		break :blk Node.ContainerDecl.InitArg{ .Type = expr };
	3274	} else {
	3275	break :blk null_node;
2944	3276	}
2945		break :blk Node.ContainerDecl.InitArg{ .None = {} };
2946		},
2947		else => {
2948		p.putBackToken(kind_token);
2949		return null;
2950	3277	},
	3278	else => unreachable,
2951	3279	};
2952
2953		return ContainerDeclType{
2954		.kind_token = kind_token,
2955		.init_arg_expr = init_arg_expr,
2956		};
	3280	_ = try p.expectToken(.LBrace);
	3281	const members = try p.parseContainerMembers(false);
	3282	_ = try p.expectToken(.RBrace);
	3283	if (arg_expr == 0) {
	3284	return p.addNode(.{
	3285	.tag = .ContainerDecl,
	3286	.main_token = main_token,
	3287	.data = .{
	3288	.lhs = members.start,
	3289	.rhs = members.end,
	3290	},
	3291	});
	3292	} else {
	3293	return p.addNode(.{
	3294	.tag = .ContainerDeclArg,
	3295	.main_token = main_token,
	3296	.data = .{
	3297	.lhs = arg_expr,
	3298	.rhs = try p.addExtra(Node.SubRange{
	3299	.start = members.start,
	3300	.end = members.end,
	3301	}),
	3302	},
	3303	});
	3304	}
2957	3305	}
2958	3306
	3307	/// Holds temporary data until we are ready to construct the full ContainerDecl AST node.
2959	3308	/// ByteAlign <- KEYWORD_align LPAREN Expr RPAREN
2960		fn parseByteAlign(p: Parser) !?Node {
2961		_ = p.eatToken(.Keyword_align) orelse return null;
	3309	fn parseByteAlign(p: *Parser) !Node.Index {
	3310	_ = p.eatToken(.Keyword_align) orelse return null_node;
2962	3311	_ = try p.expectToken(.LParen);
2963		const expr = try p.expectNode(parseExpr, .{
2964		.ExpectedExpr = .{ .token = p.tok_i },
2965		});
	3312	const expr = try p.expectExpr();
2966	3313	_ = try p.expectToken(.RParen);
2967	3314	return expr;
2968	3315	}
2969	3316
2970		/// IdentifierList <- (IDENTIFIER COMMA)* IDENTIFIER?
2971		/// Only ErrorSetDecl parses an IdentifierList
2972		fn parseErrorTagList(p: Parser) ![]Node {
2973		return ListParseFn(*Node, parseErrorTag)(p);
2974		}
2975
2976	3317	/// SwitchProngList <- (SwitchProng COMMA)* SwitchProng?
2977		fn parseSwitchProngList(p: Parser) ![]Node {
2978		return ListParseFn(*Node, parseSwitchProng)(p);
	3318	fn parseSwitchProngList(p: *Parser) !Node.SubRange {
	3319	return ListParseFn(parseSwitchProng)(p);
2979	3320	}
2980	3321
2981		/// AsmOutputList <- (AsmOutputItem COMMA)* AsmOutputItem?
2982		fn parseAsmOutputList(p: *Parser) Error![]Node.Asm.Output {
2983		return ListParseFn(Node.Asm.Output, parseAsmOutputItem)(p);
2984		}
	3322	/// ParamDeclList <- (ParamDecl COMMA)* ParamDecl?
	3323	fn parseParamDeclList(p: *Parser) !SmallSpan {
	3324	_ = try p.expectToken(.LParen);
	3325	if (p.eatToken(.RParen)) \|_\| {
	3326	return SmallSpan{ .zero_or_one = 0 };
	3327	}
	3328	const param_one = while (true) {
	3329	const param = try p.expectParamDecl();
	3330	if (param != 0) break param;
	3331	switch (p.token_tags[p.nextToken()]) {
	3332	.Comma => continue,
	3333	.RParen => return SmallSpan{ .zero_or_one = 0 },
	3334	else => {
	3335	// This is likely just a missing comma;
	3336	// give an error but continue parsing this list.
	3337	p.tok_i -= 1;
	3338	try p.warn(.{
	3339	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	3340	});
	3341	},
	3342	}
	3343	} else unreachable;
2985	3344
2986		/// AsmInputList <- (AsmInputItem COMMA)* AsmInputItem?
2987		fn parseAsmInputList(p: *Parser) Error![]Node.Asm.Input {
2988		return ListParseFn(Node.Asm.Input, parseAsmInputItem)(p);
2989		}
	3345	const param_two = while (true) {
	3346	switch (p.token_tags[p.nextToken()]) {
	3347	.Comma => {
	3348	if (p.eatToken(.RParen)) \|_\| {
	3349	return SmallSpan{ .zero_or_one = param_one };
	3350	}
	3351	const param = try p.expectParamDecl();
	3352	if (param != 0) break param;
	3353	continue;
	3354	},
	3355	.RParen => return SmallSpan{ .zero_or_one = param_one },
	3356	.Colon, .RBrace, .RBracket => {
	3357	p.tok_i -= 1;
	3358	return p.fail(.{
	3359	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .RParen },
	3360	});
	3361	},
	3362	else => {
	3363	// This is likely just a missing comma;
	3364	// give an error but continue parsing this list.
	3365	p.tok_i -= 1;
	3366	try p.warn(.{
	3367	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	3368	});
	3369	},
	3370	}
	3371	} else unreachable;
2990	3372
2991		/// ParamDeclList <- (ParamDecl COMMA)* ParamDecl?
2992		fn parseParamDeclList(p: *Parser) ![]Node.FnProto.ParamDecl {
2993		return ListParseFn(Node.FnProto.ParamDecl, parseParamDecl)(p);
	3373	var list = std.ArrayList(Node.Index).init(p.gpa);
	3374	defer list.deinit();
	3375
	3376	try list.appendSlice(&[_]Node.Index{ param_one, param_two });
	3377
	3378	while (true) {
	3379	switch (p.token_tags[p.nextToken()]) {
	3380	.Comma => {
	3381	if (p.token_tags[p.tok_i] == .RParen) {
	3382	p.tok_i += 1;
	3383	return SmallSpan{ .multi = list.toOwnedSlice() };
	3384	}
	3385	const param = try p.expectParamDecl();
	3386	if (param != 0) {
	3387	try list.append(param);
	3388	}
	3389	continue;
	3390	},
	3391	.RParen => return SmallSpan{ .multi = list.toOwnedSlice() },
	3392	.Colon, .RBrace, .RBracket => {
	3393	p.tok_i -= 1;
	3394	return p.fail(.{
	3395	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .RParen },
	3396	});
	3397	},
	3398	else => {
	3399	// This is likely just a missing comma;
	3400	// give an error but continue parsing this list.
	3401	p.tok_i -= 1;
	3402	try p.warn(.{
	3403	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
	3404	});
	3405	},
	3406	}
	3407	}
2994	3408	}
2995	3409
2996		const NodeParseFn = fn (p: Parser) Error!?Node;
	3410	const NodeParseFn = fn (p: *Parser) Error!Node.Index;
2997	3411
2998		fn ListParseFn(comptime E: type, comptime nodeParseFn: anytype) ParseFn([]E) {
	3412	fn ListParseFn(comptime nodeParseFn: anytype) (fn (p: *Parser) Error!Node.SubRange) {
2999	3413	return struct {
3000		pub fn parse(p: *Parser) ![]E {
3001		var list = std.ArrayList(E).init(p.gpa);
	3414	pub fn parse(p: *Parser) Error!Node.SubRange {
	3415	var list = std.ArrayList(Node.Index).init(p.gpa);
3002	3416	defer list.deinit();
3003	3417
3004		while (try nodeParseFn(p)) \|item\| {
	3418	while (true) {
	3419	const item = try nodeParseFn(p);
	3420	if (item == 0) break;
	3421
3005	3422	try list.append(item);
3006	3423
3007		switch (p.token_ids[p.tok_i]) {
3008		.Comma => _ = p.nextToken(),
	3424	switch (p.token_tags[p.tok_i]) {
	3425	.Comma => p.tok_i += 1,
3009	3426	// all possible delimiters
3010	3427	.Colon, .RParen, .RBrace, .RBracket => break,
3011	3428	else => {
3012		// this is likely just a missing comma,
3013		// continue parsing this list and give an error
3014		try p.errors.append(p.gpa, .{
	3429	// This is likely just a missing comma;
	3430	// give an error but continue parsing this list.
	3431	try p.warn(.{
3015	3432	.ExpectedToken = .{ .token = p.tok_i, .expected_id = .Comma },
3016	3433	});
3017	3434	},
3018	3435	}
3019	3436	}
3020		return list.toOwnedSlice();
3021		}
3022		}.parse;
3023		}
3024
3025		fn SimpleBinOpParseFn(comptime token: Token.Id, comptime op: Node.Tag) NodeParseFn {
3026		return struct {
3027		pub fn parse(p: Parser) Error!?Node {
3028		const op_token = if (token == .Keyword_and) switch (p.token_ids[p.tok_i]) {
3029		.Keyword_and => p.nextToken(),
3030		.Invalid_ampersands => blk: {
3031		try p.errors.append(p.gpa, .{
3032		.InvalidAnd = .{ .token = p.tok_i },
3033		});
3034		break :blk p.nextToken();
3035		},
3036		else => return null,
3037		} else p.eatToken(token) orelse return null;
3038
3039		const node = try p.arena.allocator.create(Node.SimpleInfixOp);
3040		node.* = .{
3041		.base = .{ .tag = op },
3042		.op_token = op_token,
3043		.lhs = undefined, // set by caller
3044		.rhs = undefined, // set by caller
3045		};
3046		return &node.base;
	3437	return p.listToSpan(list.items);
3047	3438	}
3048	3439	}.parse;
3049	3440	}
3050	3441
3051		// Helper parsers not included in the grammar
	3442	/// FnCallArguments <- LPAREN ExprList RPAREN
	3443	/// ExprList <- (Expr COMMA)* Expr?
	3444	/// TODO detect when we can emit BuiltinCallTwo instead of BuiltinCall.
	3445	fn parseBuiltinCall(p: *Parser) !Node.Index {
	3446	const builtin_token = p.eatToken(.Builtin) orelse return null_node;
3052	3447
3053		fn parseBuiltinCall(p: Parser) !?Node {
3054		const token = p.eatToken(.Builtin) orelse return null;
3055		const params = (try p.parseFnCallArguments()) orelse {
3056		try p.errors.append(p.gpa, .{
	3448	const lparen = (try p.expectTokenRecoverable(.LParen)) orelse {
	3449	try p.warn(.{
3057	3450	.ExpectedParamList = .{ .token = p.tok_i },
3058	3451	});
3059
3060		// lets pretend this was an identifier so we can continue parsing
3061		const node = try p.arena.allocator.create(Node.OneToken);
3062		node.* = .{
3063		.base = .{ .tag = .Identifier },
3064		.token = token,
3065		};
3066		return &node.base;
3067		};
3068		defer p.gpa.free(params.list);
3069
3070		const node = try Node.BuiltinCall.alloc(&p.arena.allocator, params.list.len);
3071		node.* = .{
3072		.builtin_token = token,
3073		.params_len = params.list.len,
3074		.rparen_token = params.rparen,
3075		};
3076		std.mem.copy(*Node, node.params(), params.list);
3077		return &node.base;
3078		}
3079
3080		fn parseErrorTag(p: Parser) !?Node {
3081		const doc_comments = try p.parseDocComment(); // no need to rewind on failure
3082		const token = p.eatToken(.Identifier) orelse return null;
3083
3084		const node = try p.arena.allocator.create(Node.ErrorTag);
3085		node.* = .{
3086		.doc_comments = doc_comments,
3087		.name_token = token,
3088		};
3089		return &node.base;
3090		}
3091
3092		fn parseIdentifier(p: Parser) !?Node {
3093		const token = p.eatToken(.Identifier) orelse return null;
3094		const node = try p.arena.allocator.create(Node.OneToken);
3095		node.* = .{
3096		.base = .{ .tag = .Identifier },
3097		.token = token,
	3452	// Pretend this was an identifier so we can continue parsing.
	3453	return p.addNode(.{
	3454	.tag = .OneToken,
	3455	.main_token = builtin_token,
	3456	.data = .{
	3457	.lhs = undefined,
	3458	.rhs = undefined,
	3459	},
	3460	});
3098	3461	};
3099		return &node.base;
	3462	const params = try ListParseFn(parseExpr)(p);
	3463	_ = try p.expectToken(.RParen);
	3464	return p.addNode(.{
	3465	.tag = .BuiltinCall,
	3466	.main_token = builtin_token,
	3467	.data = .{
	3468	.lhs = params.start,
	3469	.rhs = params.end,
	3470	},
	3471	});
3100	3472	}
3101	3473
3102		fn parseAnyType(p: Parser) !?Node {
3103		const token = p.eatToken(.Keyword_anytype) orelse
3104		p.eatToken(.Keyword_var) orelse return null; // TODO remove in next release cycle
3105		const node = try p.arena.allocator.create(Node.OneToken);
3106		node.* = .{
3107		.base = .{ .tag = .AnyType },
3108		.token = token,
3109		};
3110		return &node.base;
	3474	fn parseOneToken(p: *Parser, token_tag: Token.Tag) !Node.Index {
	3475	const token = p.eatToken(token_tag) orelse return null_node;
	3476	return p.addNode(.{
	3477	.tag = .OneToken,
	3478	.main_token = token,
	3479	.data = .{
	3480	.lhs = undefined,
	3481	.rhs = undefined,
	3482	},
	3483	});
3111	3484	}
3112	3485
3113		fn createLiteral(p: Parser, tag: ast.Node.Tag, token: TokenIndex) !Node {
3114		const result = try p.arena.allocator.create(Node.OneToken);
3115		result.* = .{
3116		.base = .{ .tag = tag },
3117		.token = token,
3118		};
3119		return &result.base;
	3486	fn expectOneToken(p: *Parser, token_tag: Token.Tag) !Node.Index {
	3487	const node = try p.expectOneTokenRecoverable(token_tag);
	3488	if (node == 0) return error.ParseError;
	3489	return node;
3120	3490	}
3121	3491
3122		fn parseStringLiteralSingle(p: Parser) !?Node {
3123		if (p.eatToken(.StringLiteral)) \|token\| {
3124		const node = try p.arena.allocator.create(Node.OneToken);
3125		node.* = .{
3126		.base = .{ .tag = .StringLiteral },
3127		.token = token,
3128		};
3129		return &node.base;
	3492	fn expectOneTokenRecoverable(p: *Parser, token_tag: Token.Tag) !Node.Index {
	3493	const node = p.parseOneToken(token_tag);
	3494	if (node == 0) {
	3495	try p.warn(.{
	3496	.ExpectedToken = .{
	3497	.token = p.tok_i,
	3498	.expected_id = token_tag,
	3499	},
	3500	});
3130	3501	}
3131		return null;
	3502	return node;
3132	3503	}
3133	3504
3134	3505	// string literal or multiline string literal
3135		fn parseStringLiteral(p: Parser) !?Node {
3136		if (try p.parseStringLiteralSingle()) \|node\| return node;
3137
3138		if (p.eatToken(.MultilineStringLiteralLine)) \|first_line\| {
3139		const start_tok_i = p.tok_i;
3140		var tok_i = start_tok_i;
3141		var count: usize = 1; // including first_line
3142		while (true) : (tok_i += 1) {
3143		switch (p.token_ids[tok_i]) {
3144		.LineComment => continue,
3145		.MultilineStringLiteralLine => count += 1,
3146		else => break,
	3506	fn parseStringLiteral(p: *Parser) !Node.Index {
	3507	switch (p.token_tags[p.tok_i]) {
	3508	.StringLiteral => return p.addNode(.{
	3509	.tag = .OneToken,
	3510	.main_token = p.nextToken(),
	3511	.data = .{
	3512	.lhs = undefined,
	3513	.rhs = undefined,
	3514	},
	3515	}),
	3516	.MultilineStringLiteralLine => {
	3517	const first_line = p.nextToken();
	3518	while (p.token_tags[p.tok_i] == .MultilineStringLiteralLine) {
	3519	p.tok_i += 1;
3147	3520	}
3148		}
3149
3150		const node = try Node.MultilineStringLiteral.alloc(&p.arena.allocator, count);
3151		node.* = .{ .lines_len = count };
3152		const lines = node.lines();
3153		tok_i = start_tok_i;
3154		lines[0] = first_line;
3155		count = 1;
3156		while (true) : (tok_i += 1) {
3157		switch (p.token_ids[tok_i]) {
3158		.LineComment => continue,
3159		.MultilineStringLiteralLine => {
3160		lines[count] = tok_i;
3161		count += 1;
	3521	return p.addNode(.{
	3522	.tag = .OneToken,
	3523	.main_token = first_line,
	3524	.data = .{
	3525	.lhs = undefined,
	3526	.rhs = undefined,
3162	3527	},
3163		else => break,
3164		}
3165		}
3166		p.tok_i = tok_i;
3167		return &node.base;
	3528	});
	3529	},
	3530	else => return null_node,
3168	3531	}
3169
3170		return null;
3171	3532	}
3172	3533
3173		fn parseIntegerLiteral(p: Parser) !?Node {
3174		const token = p.eatToken(.IntegerLiteral) orelse return null;
3175		const node = try p.arena.allocator.create(Node.OneToken);
3176		node.* = .{
3177		.base = .{ .tag = .IntegerLiteral },
3178		.token = token,
3179		};
3180		return &node.base;
	3534	fn expectStringLiteral(p: *Parser) !Node.Index {
	3535	const node = try p.parseStringLiteral();
	3536	if (node == 0) {
	3537	return p.fail(.{ .ExpectedStringLiteral = .{ .token = p.tok_i } });
	3538	}
	3539	return node;
3181	3540	}
3182	3541
3183		fn parseFloatLiteral(p: Parser) !?Node {
3184		const token = p.eatToken(.FloatLiteral) orelse return null;
3185		const node = try p.arena.allocator.create(Node.OneToken);
3186		node.* = .{
3187		.base = .{ .tag = .FloatLiteral },
3188		.token = token,
3189		};
3190		return &node.base;
	3542	fn expectIntegerLiteral(p: *Parser) !Node.Index {
	3543	const node = p.parseOneToken(.IntegerLiteral);
	3544	if (node != 0) {
	3545	return p.fail(.{ .ExpectedIntegerLiteral = .{ .token = p.tok_i } });
	3546	}
	3547	return node;
3191	3548	}
3192	3549
3193		fn parseTry(p: Parser) !?Node {
3194		const token = p.eatToken(.Keyword_try) orelse return null;
3195		const node = try p.arena.allocator.create(Node.SimplePrefixOp);
3196		node.* = .{
3197		.base = .{ .tag = .Try },
3198		.op_token = token,
3199		.rhs = undefined, // set by caller
3200		};
3201		return &node.base;
3202		}
	3550	/// KEYWORD_if LPAREN Expr RPAREN PtrPayload? Body (KEYWORD_else Payload? Body)?
	3551	fn parseIf(p: *Parser, bodyParseFn: NodeParseFn) !Node.Index {
	3552	const if_token = p.eatToken(.Keyword_if) orelse return null_node;
	3553	_ = try p.expectToken(.LParen);
	3554	const condition = try p.expectExpr();
	3555	_ = try p.expectToken(.RParen);
	3556	const then_payload = try p.parsePtrPayload();
3203	3557
3204		/// IfPrefix Body (KEYWORD_else Payload? Body)?
3205		fn parseIf(p: Parser, bodyParseFn: NodeParseFn) !?Node {
3206		const node = (try p.parseIfPrefix()) orelse return null;
3207		const if_prefix = node.cast(Node.If).?;
	3558	const then_expr = try bodyParseFn(p);
	3559	if (then_expr == 0) return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
3208	3560
3209		if_prefix.body = try p.expectNode(bodyParseFn, .{
3210		.InvalidToken = .{ .token = p.tok_i },
	3561	const else_token = p.eatToken(.Keyword_else) orelse return p.addNode(.{
	3562	.tag = if (then_payload == 0) .IfSimple else .IfSimpleOptional,
	3563	.main_token = if_token,
	3564	.data = .{
	3565	.lhs = condition,
	3566	.rhs = then_expr,
	3567	},
3211	3568	});
3212
3213		const else_token = p.eatToken(.Keyword_else) orelse return node;
3214		const payload = try p.parsePayload();
3215		const else_expr = try p.expectNode(bodyParseFn, .{
3216		.InvalidToken = .{ .token = p.tok_i },
	3569	const else_payload = try p.parsePayload();
	3570	const else_expr = try bodyParseFn(p);
	3571	if (else_expr == 0) return p.fail(.{ .InvalidToken = .{ .token = p.tok_i } });
	3572
	3573	const tag = if (else_payload != 0)
	3574	Node.Tag.IfError
	3575	else if (then_payload != 0)
	3576	Node.Tag.IfOptional
	3577	else
	3578	Node.Tag.If;
	3579	return p.addNode(.{
	3580	.tag = tag,
	3581	.main_token = if_token,
	3582	.data = .{
	3583	.lhs = condition,
	3584	.rhs = try p.addExtra(Node.If{
	3585	.then_expr = then_expr,
	3586	.else_expr = else_expr,
	3587	}),
	3588	},
3217	3589	});
3218		const else_node = try p.arena.allocator.create(Node.Else);
3219		else_node.* = .{
3220		.else_token = else_token,
3221		.payload = payload,
3222		.body = else_expr,
3223		};
3224		if_prefix.@"else" = else_node;
3225
3226		return node;
3227	3590	}
3228	3591
3229		/// Eat a multiline doc comment
3230		fn parseDocComment(p: Parser) !?Node.DocComment {
	3592	/// Skips over doc comment tokens. Returns the first one, if any.
	3593	fn eatDocComments(p: *Parser) ?TokenIndex {
3231	3594	if (p.eatToken(.DocComment)) \|first_line\| {
3232	3595	while (p.eatToken(.DocComment)) \|_\| {}
3233		const node = try p.arena.allocator.create(Node.DocComment);
3234		node.* = .{ .first_line = first_line };
3235		return node;
	3596	return first_line;
3236	3597	}
3237	3598	return null;
3238	3599	}
3239	3600
3240	3601	fn tokensOnSameLine(p: *Parser, token1: TokenIndex, token2: TokenIndex) bool {
3241		return std.mem.indexOfScalar(u8, p.source[p.token_locs[token1].end..p.token_locs[token2].start], '\n') == null;
	3602	return std.mem.indexOfScalar(u8, p.source[p.token_starts[token1]..p.token_starts[token2]], '\n') == null;
3242	3603	}
3243	3604
3244	3605	/// Eat a single-line doc comment on the same line as another node
3245		fn parseAppendedDocComment(p: Parser, after_token: TokenIndex) !?Node.DocComment {
3246		const comment_token = p.eatToken(.DocComment) orelse return null;
3247		if (p.tokensOnSameLine(after_token, comment_token)) {
3248		const node = try p.arena.allocator.create(Node.DocComment);
3249		node.* = .{ .first_line = comment_token };
3250		return node;
3251		}
3252		p.putBackToken(comment_token);
3253		return null;
3254		}
3255
3256		/// Op* Child
3257		fn parsePrefixOpExpr(p: Parser, comptime opParseFn: NodeParseFn, comptime childParseFn: NodeParseFn) Error!?Node {
3258		if (try opParseFn(p)) \|first_op\| {
3259		var rightmost_op = first_op;
3260		while (true) {
3261		switch (rightmost_op.tag) {
3262		.AddressOf,
3263		.Await,
3264		.BitNot,
3265		.BoolNot,
3266		.OptionalType,
3267		.Negation,
3268		.NegationWrap,
3269		.Resume,
3270		.Try,
3271		=> {
3272		if (try opParseFn(p)) \|rhs\| {
3273		rightmost_op.cast(Node.SimplePrefixOp).?.rhs = rhs;
3274		rightmost_op = rhs;
3275		} else break;
3276		},
3277		.ArrayType => {
3278		if (try opParseFn(p)) \|rhs\| {
3279		rightmost_op.cast(Node.ArrayType).?.rhs = rhs;
3280		rightmost_op = rhs;
3281		} else break;
3282		},
3283		.ArrayTypeSentinel => {
3284		if (try opParseFn(p)) \|rhs\| {
3285		rightmost_op.cast(Node.ArrayTypeSentinel).?.rhs = rhs;
3286		rightmost_op = rhs;
3287		} else break;
3288		},
3289		.SliceType => {
3290		if (try opParseFn(p)) \|rhs\| {
3291		rightmost_op.cast(Node.SliceType).?.rhs = rhs;
3292		rightmost_op = rhs;
3293		} else break;
3294		},
3295		.PtrType => {
3296		var ptr_type = rightmost_op.cast(Node.PtrType).?;
3297		// If the token encountered was **, there will be two nodes
3298		if (p.token_ids[ptr_type.op_token] == .AsteriskAsterisk) {
3299		rightmost_op = ptr_type.rhs;
3300		ptr_type = rightmost_op.cast(Node.PtrType).?;
3301		}
3302		if (try opParseFn(p)) \|rhs\| {
3303		ptr_type.rhs = rhs;
3304		rightmost_op = rhs;
3305		} else break;
3306		},
3307		.AnyFrameType => {
3308		const prom = rightmost_op.cast(Node.AnyFrameType).?;
3309		if (try opParseFn(p)) \|rhs\| {
3310		prom.result.?.return_type = rhs;
3311		rightmost_op = rhs;
3312		} else break;
3313		},
3314		else => unreachable,
3315		}
3316		}
3317
3318		// If any prefix op existed, a child node on the RHS is required
3319		switch (rightmost_op.tag) {
3320		.AddressOf,
3321		.Await,
3322		.BitNot,
3323		.BoolNot,
3324		.OptionalType,
3325		.Negation,
3326		.NegationWrap,
3327		.Resume,
3328		.Try,
3329		=> {
3330		const prefix_op = rightmost_op.cast(Node.SimplePrefixOp).?;
3331		prefix_op.rhs = try p.expectNode(childParseFn, .{
3332		.InvalidToken = .{ .token = p.tok_i },
3333		});
3334		},
3335		.ArrayType => {
3336		const prefix_op = rightmost_op.cast(Node.ArrayType).?;
3337		prefix_op.rhs = try p.expectNode(childParseFn, .{
3338		.InvalidToken = .{ .token = p.tok_i },
3339		});
3340		},
3341		.ArrayTypeSentinel => {
3342		const prefix_op = rightmost_op.cast(Node.ArrayTypeSentinel).?;
3343		prefix_op.rhs = try p.expectNode(childParseFn, .{
3344		.InvalidToken = .{ .token = p.tok_i },
3345		});
3346		},
3347		.PtrType => {
3348		const prefix_op = rightmost_op.cast(Node.PtrType).?;
3349		prefix_op.rhs = try p.expectNode(childParseFn, .{
3350		.InvalidToken = .{ .token = p.tok_i },
3351		});
3352		},
3353		.SliceType => {
3354		const prefix_op = rightmost_op.cast(Node.SliceType).?;
3355		prefix_op.rhs = try p.expectNode(childParseFn, .{
3356		.InvalidToken = .{ .token = p.tok_i },
3357		});
3358		},
3359		.AnyFrameType => {
3360		const prom = rightmost_op.cast(Node.AnyFrameType).?;
3361		prom.result.?.return_type = try p.expectNode(childParseFn, .{
3362		.InvalidToken = .{ .token = p.tok_i },
3363		});
3364		},
3365		else => unreachable,
3366		}
3367
3368		return first_op;
3369		}
3370
3371		// Otherwise, the child node is optional
3372		return childParseFn(p);
3373		}
3374
3375		/// Child (Op Child)*
3376		/// Child (Op Child)?
3377		fn parseBinOpExpr(
3378		p: *Parser,
3379		opParseFn: NodeParseFn,
3380		childParseFn: NodeParseFn,
3381		chain: enum {
3382		Once,
3383		Infinitely,
3384		},
3385		) Error!?*Node {
3386		var res = (try childParseFn(p)) orelse return null;
3387
3388		while (try opParseFn(p)) \|node\| {
3389		const right = try p.expectNode(childParseFn, .{
3390		.InvalidToken = .{ .token = p.tok_i },
3391		});
3392		const left = res;
3393		res = node;
3394
3395		if (node.castTag(.Catch)) \|op\| {
3396		op.lhs = left;
3397		op.rhs = right;
3398		} else if (node.cast(Node.SimpleInfixOp)) \|op\| {
3399		op.lhs = left;
3400		op.rhs = right;
3401		}
3402
3403		switch (chain) {
3404		.Once => break,
3405		.Infinitely => continue,
3406		}
	3606	fn parseAppendedDocComment(p: *Parser, after_token: TokenIndex) !void {
	3607	const comment_token = p.eatToken(.DocComment) orelse return;
	3608	if (!p.tokensOnSameLine(after_token, comment_token)) {
	3609	p.tok_i -= 1;
3407	3610	}
3408
3409		return res;
3410	3611	}
3411	3612
3412		fn createInfixOp(p: Parser, op_token: TokenIndex, tag: Node.Tag) !Node {
3413		const node = try p.arena.allocator.create(Node.SimpleInfixOp);
3414		node.* = .{
3415		.base = Node{ .tag = tag },
3416		.op_token = op_token,
3417		.lhs = undefined, // set by caller
3418		.rhs = undefined, // set by caller
3419		};
3420		return &node.base;
	3613	fn eatToken(p: *Parser, tag: Token.Tag) ?TokenIndex {
	3614	return if (p.token_tags[p.tok_i] == tag) p.nextToken() else null;
3421	3615	}
3422	3616
3423		fn eatToken(p: *Parser, id: Token.Id) ?TokenIndex {
3424		return if (p.token_ids[p.tok_i] == id) p.nextToken() else null;
	3617	fn assertToken(p: *Parser, tag: Token.Tag) TokenIndex {
	3618	const token = p.nextToken();
	3619	assert(p.token_tags[token] == tag);
	3620	return token;
3425	3621	}
3426	3622
3427		fn expectToken(p: *Parser, id: Token.Id) Error!TokenIndex {
3428		return (try p.expectTokenRecoverable(id)) orelse error.ParseError;
	3623	fn expectToken(p: *Parser, tag: Token.Tag) Error!TokenIndex {
	3624	const token = p.nextToken();
	3625	if (p.token_tags[token] != tag) {
	3626	return p.fail(.{ .ExpectedToken = .{ .token = token, .expected_id = tag } });
	3627	}
	3628	return token;
3429	3629	}
3430	3630
3431		fn expectTokenRecoverable(p: *Parser, id: Token.Id) !?TokenIndex {
3432		const token = p.nextToken();
3433		if (p.token_ids[token] != id) {
3434		try p.errors.append(p.gpa, .{
3435		.ExpectedToken = .{ .token = token, .expected_id = id },
	3631	fn expectTokenRecoverable(p: *Parser, tag: Token.Tag) !?TokenIndex {
	3632	if (p.token_tags[p.tok_i] != tag) {
	3633	try p.warn(.{
	3634	.ExpectedToken = .{ .token = p.tok_i, .expected_id = tag },
3436	3635	});
3437		// go back so that we can recover properly
3438		p.putBackToken(token);
3439	3636	return null;
	3637	} else {
	3638	return p.nextToken();
3440	3639	}
3441		return token;
3442	3640	}
3443	3641
3444	3642	fn nextToken(p: *Parser) TokenIndex {
3445	3643	const result = p.tok_i;
3446	3644	p.tok_i += 1;
3447		assert(p.token_ids[result] != .LineComment);
3448		if (p.tok_i >= p.token_ids.len) return result;
3449
3450		while (true) {
3451		if (p.token_ids[p.tok_i] != .LineComment) return result;
3452		p.tok_i += 1;
3453		}
3454		}
3455
3456		fn putBackToken(p: *Parser, putting_back: TokenIndex) void {
3457		while (p.tok_i > 0) {
3458		p.tok_i -= 1;
3459		if (p.token_ids[p.tok_i] == .LineComment) continue;
3460		assert(putting_back == p.tok_i);
3461		return;
3462		}
3463		}
3464
3465		/// TODO Delete this function. I don't like the inversion of control.
3466		fn expectNode(
3467		p: *Parser,
3468		parseFn: NodeParseFn,
3469		/// if parsing fails
3470		err: AstError,
3471		) Error!*Node {
3472		return (try p.expectNodeRecoverable(parseFn, err)) orelse return error.ParseError;
3473		}
3474
3475		/// TODO Delete this function. I don't like the inversion of control.
3476		fn expectNodeRecoverable(
3477		p: *Parser,
3478		parseFn: NodeParseFn,
3479		/// if parsing fails
3480		err: AstError,
3481		) !?*Node {
3482		return (try parseFn(p)) orelse {
3483		try p.errors.append(p.gpa, err);
3484		return null;
3485		};
	3645	return result;
3486	3646	}
3487	3647	};
3488	3648
3489		fn ParseFn(comptime T: type) type {
3490		return fn (p: *Parser) Error!T;
3491		}
3492
3493		test "std.zig.parser" {
	3649	test {
3494	3650	_ = @import("parser_test.zig");
3495	3651	}

lib/std/zig/parser_test.zig+9-14

...	...	@@ -3736,12 +3736,13 @@ var fixed_buffer_mem: [100 * 1024]u8 = undefined;
3736	3736	fn testParse(source: []const u8, allocator: mem.Allocator, anything_changed: bool) ![]u8 {
3737	3737	const stderr = io.getStdErr().writer();
3738	3738
3739		const tree = try std.zig.parse(allocator, source);
3740		defer tree.deinit();
	3739	var tree = try std.zig.parse(allocator, source);
	3740	defer tree.deinit(allocator);
3741	3741
3742		for (tree.errors) \|*parse_error\| {
3743		const token = tree.token_locs[parse_error.loc()];
3744		const loc = tree.tokenLocation(0, parse_error.loc());
	3742	for (tree.errors) \|parse_error\| {
	3743	const error_token = tree.errorToken(parse_error);
	3744	const token_start = tree.tokens.items(.start)[error_token];
	3745	const loc = tree.tokenLocation(0, error_token);
3745	3746	try stderr.print("(memory buffer):{d}:{d}: error: ", .{ loc.line + 1, loc.column + 1 });
3746	3747	try tree.renderError(parse_error, stderr);
3747	3748	try stderr.print("\n{s}\n", .{source[loc.line_start..loc.line_end]});
...	...	@@ -3750,13 +3751,7 @@ fn testParse(source: []const u8, allocator: mem.Allocator, anything_changed: b
3750	3751	while (i < loc.column) : (i += 1) {
3751	3752	try stderr.writeAll(" ");
3752	3753	}
3753		}
3754		{
3755		const caret_count = token.end - token.start;
3756		var i: usize = 0;
3757		while (i < caret_count) : (i += 1) {
3758		try stderr.writeAll("~");
3759		}
	3754	try stderr.writeAll("^");
3760	3755	}
3761	3756	try stderr.writeAll("\n");
3762	3757	}
...	...	@@ -3825,8 +3820,8 @@ fn testCanonical(source: []const u8) !void {
3825	3820	const Error = @TagType(std.zig.ast.Error);
3826	3821
3827	3822	fn testError(source: []const u8, expected_errors: []const Error) !void {
3828		const tree = try std.zig.parse(std.testing.allocator, source);
3829		defer tree.deinit();
	3823	var tree = try std.zig.parse(std.testing.allocator, source);
	3824	defer tree.deinit(std.testing.allocator);
3830	3825
3831	3826	std.testing.expect(tree.errors.len == expected_errors.len);
3832	3827	for (expected_errors) \|expected, i\| {

lib/std/zig/tokenizer.zig+435-435

...	...	@@ -7,7 +7,7 @@ const std = @import("../std.zig");
7	7	const mem = std.mem;
8	8
9	9	pub const Token = struct {
10		id: Id,
	10	tag: Tag,
11	11	loc: Loc,
12	12
13	13	pub const Loc = struct {
...	...	@@ -15,7 +15,7 @@ pub const Token = struct {
15	15	end: usize,
16	16	};
17	17
18		pub const keywords = std.ComptimeStringMap(Id, .{
	18	pub const keywords = std.ComptimeStringMap(Tag, .{
19	19	.{ "align", .Keyword_align },
20	20	.{ "allowzero", .Keyword_allowzero },
21	21	.{ "and", .Keyword_and },
...	...	@@ -71,11 +71,11 @@ pub const Token = struct {
71	71	.{ "while", .Keyword_while },
72	72	});
73	73
74		pub fn getKeyword(bytes: []const u8) ?Id {
	74	pub fn getKeyword(bytes: []const u8) ?Tag {
75	75	return keywords.get(bytes);
76	76	}
77	77
78		pub const Id = enum {
	78	pub const Tag = enum {
79	79	Invalid,
80	80	Invalid_ampersands,
81	81	Invalid_periodasterisks,
...	...	@@ -198,8 +198,8 @@ pub const Token = struct {
198	198	Keyword_volatile,
199	199	Keyword_while,
200	200
201		pub fn symbol(id: Id) []const u8 {
202		return switch (id) {
	201	pub fn symbol(tag: Tag) []const u8 {
	202	return switch (tag) {
203	203	.Invalid => "Invalid",
204	204	.Invalid_ampersands => "&&",
205	205	.Invalid_periodasterisks => ".**",
...	...	@@ -334,7 +334,7 @@ pub const Tokenizer = struct {
334	334
335	335	/// For debugging purposes
336	336	pub fn dump(self: Tokenizer, token: const Token) void {
337		std.debug.warn("{s} \"{s}\"\n", .{ @tagName(token.id), self.buffer[token.start..token.end] });
	337	std.debug.warn("{s} \"{s}\"\n", .{ @tagName(token.tag), self.buffer[token.start..token.end] });
338	338	}
339	339
340	340	pub fn init(buffer: []const u8) Tokenizer {
...	...	@@ -421,7 +421,7 @@ pub const Tokenizer = struct {
421	421	const start_index = self.index;
422	422	var state: State = .start;
423	423	var result = Token{
424		.id = .Eof,
	424	.tag = .Eof,
425	425	.loc = .{
426	426	.start = self.index,
427	427	.end = undefined,
...	...	@@ -438,14 +438,14 @@ pub const Tokenizer = struct {
438	438	},
439	439	'"' => {
440	440	state = .string_literal;
441		result.id = .StringLiteral;
	441	result.tag = .StringLiteral;
442	442	},
443	443	'\'' => {
444	444	state = .char_literal;
445	445	},
446	446	'a'...'z', 'A'...'Z', '_' => {
447	447	state = .identifier;
448		result.id = .Identifier;
	448	result.tag = .Identifier;
449	449	},
450	450	'@' => {
451	451	state = .saw_at_sign;
...	...	@@ -460,42 +460,42 @@ pub const Tokenizer = struct {
460	460	state = .pipe;
461	461	},
462	462	'(' => {
463		result.id = .LParen;
	463	result.tag = .LParen;
464	464	self.index += 1;
465	465	break;
466	466	},
467	467	')' => {
468		result.id = .RParen;
	468	result.tag = .RParen;
469	469	self.index += 1;
470	470	break;
471	471	},
472	472	'[' => {
473		result.id = .LBracket;
	473	result.tag = .LBracket;
474	474	self.index += 1;
475	475	break;
476	476	},
477	477	']' => {
478		result.id = .RBracket;
	478	result.tag = .RBracket;
479	479	self.index += 1;
480	480	break;
481	481	},
482	482	';' => {
483		result.id = .Semicolon;
	483	result.tag = .Semicolon;
484	484	self.index += 1;
485	485	break;
486	486	},
487	487	',' => {
488		result.id = .Comma;
	488	result.tag = .Comma;
489	489	self.index += 1;
490	490	break;
491	491	},
492	492	'?' => {
493		result.id = .QuestionMark;
	493	result.tag = .QuestionMark;
494	494	self.index += 1;
495	495	break;
496	496	},
497	497	':' => {
498		result.id = .Colon;
	498	result.tag = .Colon;
499	499	self.index += 1;
500	500	break;
501	501	},
...	...	@@ -519,20 +519,20 @@ pub const Tokenizer = struct {
519	519	},
520	520	'\\' => {
521	521	state = .backslash;
522		result.id = .MultilineStringLiteralLine;
	522	result.tag = .MultilineStringLiteralLine;
523	523	},
524	524	'{' => {
525		result.id = .LBrace;
	525	result.tag = .LBrace;
526	526	self.index += 1;
527	527	break;
528	528	},
529	529	'}' => {
530		result.id = .RBrace;
	530	result.tag = .RBrace;
531	531	self.index += 1;
532	532	break;
533	533	},
534	534	'~' => {
535		result.id = .Tilde;
	535	result.tag = .Tilde;
536	536	self.index += 1;
537	537	break;
538	538	},
...	...	@@ -550,14 +550,14 @@ pub const Tokenizer = struct {
550	550	},
551	551	'0' => {
552	552	state = .zero;
553		result.id = .IntegerLiteral;
	553	result.tag = .IntegerLiteral;
554	554	},
555	555	'1'...'9' => {
556	556	state = .int_literal_dec;
557		result.id = .IntegerLiteral;
	557	result.tag = .IntegerLiteral;
558	558	},
559	559	else => {
560		result.id = .Invalid;
	560	result.tag = .Invalid;
561	561	self.index += 1;
562	562	break;
563	563	},
...	...	@@ -565,42 +565,42 @@ pub const Tokenizer = struct {
565	565
566	566	.saw_at_sign => switch (c) {
567	567	'"' => {
568		result.id = .Identifier;
	568	result.tag = .Identifier;
569	569	state = .string_literal;
570	570	},
571	571	else => {
572	572	// reinterpret as a builtin
573	573	self.index -= 1;
574	574	state = .builtin;
575		result.id = .Builtin;
	575	result.tag = .Builtin;
576	576	},
577	577	},
578	578
579	579	.ampersand => switch (c) {
580	580	'&' => {
581		result.id = .Invalid_ampersands;
	581	result.tag = .Invalid_ampersands;
582	582	self.index += 1;
583	583	break;
584	584	},
585	585	'=' => {
586		result.id = .AmpersandEqual;
	586	result.tag = .AmpersandEqual;
587	587	self.index += 1;
588	588	break;
589	589	},
590	590	else => {
591		result.id = .Ampersand;
	591	result.tag = .Ampersand;
592	592	break;
593	593	},
594	594	},
595	595
596	596	.asterisk => switch (c) {
597	597	'=' => {
598		result.id = .AsteriskEqual;
	598	result.tag = .AsteriskEqual;
599	599	self.index += 1;
600	600	break;
601	601	},
602	602	'*' => {
603		result.id = .AsteriskAsterisk;
	603	result.tag = .AsteriskAsterisk;
604	604	self.index += 1;
605	605	break;
606	606	},
...	...	@@ -608,43 +608,43 @@ pub const Tokenizer = struct {
608	608	state = .asterisk_percent;
609	609	},
610	610	else => {
611		result.id = .Asterisk;
	611	result.tag = .Asterisk;
612	612	break;
613	613	},
614	614	},
615	615
616	616	.asterisk_percent => switch (c) {
617	617	'=' => {
618		result.id = .AsteriskPercentEqual;
	618	result.tag = .AsteriskPercentEqual;
619	619	self.index += 1;
620	620	break;
621	621	},
622	622	else => {
623		result.id = .AsteriskPercent;
	623	result.tag = .AsteriskPercent;
624	624	break;
625	625	},
626	626	},
627	627
628	628	.percent => switch (c) {
629	629	'=' => {
630		result.id = .PercentEqual;
	630	result.tag = .PercentEqual;
631	631	self.index += 1;
632	632	break;
633	633	},
634	634	else => {
635		result.id = .Percent;
	635	result.tag = .Percent;
636	636	break;
637	637	},
638	638	},
639	639
640	640	.plus => switch (c) {
641	641	'=' => {
642		result.id = .PlusEqual;
	642	result.tag = .PlusEqual;
643	643	self.index += 1;
644	644	break;
645	645	},
646	646	'+' => {
647		result.id = .PlusPlus;
	647	result.tag = .PlusPlus;
648	648	self.index += 1;
649	649	break;
650	650	},
...	...	@@ -652,31 +652,31 @@ pub const Tokenizer = struct {
652	652	state = .plus_percent;
653	653	},
654	654	else => {
655		result.id = .Plus;
	655	result.tag = .Plus;
656	656	break;
657	657	},
658	658	},
659	659
660	660	.plus_percent => switch (c) {
661	661	'=' => {
662		result.id = .PlusPercentEqual;
	662	result.tag = .PlusPercentEqual;
663	663	self.index += 1;
664	664	break;
665	665	},
666	666	else => {
667		result.id = .PlusPercent;
	667	result.tag = .PlusPercent;
668	668	break;
669	669	},
670	670	},
671	671
672	672	.caret => switch (c) {
673	673	'=' => {
674		result.id = .CaretEqual;
	674	result.tag = .CaretEqual;
675	675	self.index += 1;
676	676	break;
677	677	},
678	678	else => {
679		result.id = .Caret;
	679	result.tag = .Caret;
680	680	break;
681	681	},
682	682	},
...	...	@@ -684,8 +684,8 @@ pub const Tokenizer = struct {
684	684	.identifier => switch (c) {
685	685	'a'...'z', 'A'...'Z', '_', '0'...'9' => {},
686	686	else => {
687		if (Token.getKeyword(self.buffer[result.loc.start..self.index])) \|id\| {
688		result.id = id;
	687	if (Token.getKeyword(self.buffer[result.loc.start..self.index])) \|tag\| {
	688	result.tag = tag;
689	689	}
690	690	break;
691	691	},
...	...	@@ -724,7 +724,7 @@ pub const Tokenizer = struct {
724	724	state = .char_literal_backslash;
725	725	},
726	726	'\'', 0x80...0xbf, 0xf8...0xff => {
727		result.id = .Invalid;
	727	result.tag = .Invalid;
728	728	break;
729	729	},
730	730	0xc0...0xdf => { // 110xxxxx
...	...	@@ -746,7 +746,7 @@ pub const Tokenizer = struct {
746	746
747	747	.char_literal_backslash => switch (c) {
748	748	'\n' => {
749		result.id = .Invalid;
	749	result.tag = .Invalid;
750	750	break;
751	751	},
752	752	'x' => {
...	...	@@ -769,7 +769,7 @@ pub const Tokenizer = struct {
769	769	}
770	770	},
771	771	else => {
772		result.id = .Invalid;
	772	result.tag = .Invalid;
773	773	break;
774	774	},
775	775	},
...	...	@@ -780,7 +780,7 @@ pub const Tokenizer = struct {
780	780	seen_escape_digits = 0;
781	781	},
782	782	else => {
783		result.id = .Invalid;
	783	result.tag = .Invalid;
784	784	state = .char_literal_unicode_invalid;
785	785	},
786	786	},
...	...	@@ -791,14 +791,14 @@ pub const Tokenizer = struct {
791	791	},
792	792	'}' => {
793	793	if (seen_escape_digits == 0) {
794		result.id = .Invalid;
	794	result.tag = .Invalid;
795	795	state = .char_literal_unicode_invalid;
796	796	} else {
797	797	state = .char_literal_end;
798	798	}
799	799	},
800	800	else => {
801		result.id = .Invalid;
	801	result.tag = .Invalid;
802	802	state = .char_literal_unicode_invalid;
803	803	},
804	804	},
...	...	@@ -813,12 +813,12 @@ pub const Tokenizer = struct {
813	813
814	814	.char_literal_end => switch (c) {
815	815	'\'' => {
816		result.id = .CharLiteral;
	816	result.tag = .CharLiteral;
817	817	self.index += 1;
818	818	break;
819	819	},
820	820	else => {
821		result.id = .Invalid;
	821	result.tag = .Invalid;
822	822	break;
823	823	},
824	824	},
...	...	@@ -831,7 +831,7 @@ pub const Tokenizer = struct {
831	831	}
832	832	},
833	833	else => {
834		result.id = .Invalid;
	834	result.tag = .Invalid;
835	835	break;
836	836	},
837	837	},
...	...	@@ -847,58 +847,58 @@ pub const Tokenizer = struct {
847	847
848	848	.bang => switch (c) {
849	849	'=' => {
850		result.id = .BangEqual;
	850	result.tag = .BangEqual;
851	851	self.index += 1;
852	852	break;
853	853	},
854	854	else => {
855		result.id = .Bang;
	855	result.tag = .Bang;
856	856	break;
857	857	},
858	858	},
859	859
860	860	.pipe => switch (c) {
861	861	'=' => {
862		result.id = .PipeEqual;
	862	result.tag = .PipeEqual;
863	863	self.index += 1;
864	864	break;
865	865	},
866	866	'\|' => {
867		result.id = .PipePipe;
	867	result.tag = .PipePipe;
868	868	self.index += 1;
869	869	break;
870	870	},
871	871	else => {
872		result.id = .Pipe;
	872	result.tag = .Pipe;
873	873	break;
874	874	},
875	875	},
876	876
877	877	.equal => switch (c) {
878	878	'=' => {
879		result.id = .EqualEqual;
	879	result.tag = .EqualEqual;
880	880	self.index += 1;
881	881	break;
882	882	},
883	883	'>' => {
884		result.id = .EqualAngleBracketRight;
	884	result.tag = .EqualAngleBracketRight;
885	885	self.index += 1;
886	886	break;
887	887	},
888	888	else => {
889		result.id = .Equal;
	889	result.tag = .Equal;
890	890	break;
891	891	},
892	892	},
893	893
894	894	.minus => switch (c) {
895	895	'>' => {
896		result.id = .Arrow;
	896	result.tag = .Arrow;
897	897	self.index += 1;
898	898	break;
899	899	},
900	900	'=' => {
901		result.id = .MinusEqual;
	901	result.tag = .MinusEqual;
902	902	self.index += 1;
903	903	break;
904	904	},
...	...	@@ -906,19 +906,19 @@ pub const Tokenizer = struct {
906	906	state = .minus_percent;
907	907	},
908	908	else => {
909		result.id = .Minus;
	909	result.tag = .Minus;
910	910	break;
911	911	},
912	912	},
913	913
914	914	.minus_percent => switch (c) {
915	915	'=' => {
916		result.id = .MinusPercentEqual;
	916	result.tag = .MinusPercentEqual;
917	917	self.index += 1;
918	918	break;
919	919	},
920	920	else => {
921		result.id = .MinusPercent;
	921	result.tag = .MinusPercent;
922	922	break;
923	923	},
924	924	},
...	...	@@ -928,24 +928,24 @@ pub const Tokenizer = struct {
928	928	state = .angle_bracket_angle_bracket_left;
929	929	},
930	930	'=' => {
931		result.id = .AngleBracketLeftEqual;
	931	result.tag = .AngleBracketLeftEqual;
932	932	self.index += 1;
933	933	break;
934	934	},
935	935	else => {
936		result.id = .AngleBracketLeft;
	936	result.tag = .AngleBracketLeft;
937	937	break;
938	938	},
939	939	},
940	940
941	941	.angle_bracket_angle_bracket_left => switch (c) {
942	942	'=' => {
943		result.id = .AngleBracketAngleBracketLeftEqual;
	943	result.tag = .AngleBracketAngleBracketLeftEqual;
944	944	self.index += 1;
945	945	break;
946	946	},
947	947	else => {
948		result.id = .AngleBracketAngleBracketLeft;
	948	result.tag = .AngleBracketAngleBracketLeft;
949	949	break;
950	950	},
951	951	},
...	...	@@ -955,24 +955,24 @@ pub const Tokenizer = struct {
955	955	state = .angle_bracket_angle_bracket_right;
956	956	},
957	957	'=' => {
958		result.id = .AngleBracketRightEqual;
	958	result.tag = .AngleBracketRightEqual;
959	959	self.index += 1;
960	960	break;
961	961	},
962	962	else => {
963		result.id = .AngleBracketRight;
	963	result.tag = .AngleBracketRight;
964	964	break;
965	965	},
966	966	},
967	967
968	968	.angle_bracket_angle_bracket_right => switch (c) {
969	969	'=' => {
970		result.id = .AngleBracketAngleBracketRightEqual;
	970	result.tag = .AngleBracketAngleBracketRightEqual;
971	971	self.index += 1;
972	972	break;
973	973	},
974	974	else => {
975		result.id = .AngleBracketAngleBracketRight;
	975	result.tag = .AngleBracketAngleBracketRight;
976	976	break;
977	977	},
978	978	},
...	...	@@ -985,30 +985,30 @@ pub const Tokenizer = struct {
985	985	state = .period_asterisk;
986	986	},
987	987	else => {
988		result.id = .Period;
	988	result.tag = .Period;
989	989	break;
990	990	},
991	991	},
992	992
993	993	.period_2 => switch (c) {
994	994	'.' => {
995		result.id = .Ellipsis3;
	995	result.tag = .Ellipsis3;
996	996	self.index += 1;
997	997	break;
998	998	},
999	999	else => {
1000		result.id = .Ellipsis2;
	1000	result.tag = .Ellipsis2;
1001	1001	break;
1002	1002	},
1003	1003	},
1004	1004
1005	1005	.period_asterisk => switch (c) {
1006	1006	'*' => {
1007		result.id = .Invalid_periodasterisks;
	1007	result.tag = .Invalid_periodasterisks;
1008	1008	break;
1009	1009	},
1010	1010	else => {
1011		result.id = .PeriodAsterisk;
	1011	result.tag = .PeriodAsterisk;
1012	1012	break;
1013	1013	},
1014	1014	},
...	...	@@ -1016,15 +1016,15 @@ pub const Tokenizer = struct {
1016	1016	.slash => switch (c) {
1017	1017	'/' => {
1018	1018	state = .line_comment_start;
1019		result.id = .LineComment;
	1019	result.tag = .LineComment;
1020	1020	},
1021	1021	'=' => {
1022		result.id = .SlashEqual;
	1022	result.tag = .SlashEqual;
1023	1023	self.index += 1;
1024	1024	break;
1025	1025	},
1026	1026	else => {
1027		result.id = .Slash;
	1027	result.tag = .Slash;
1028	1028	break;
1029	1029	},
1030	1030	},
...	...	@@ -1033,7 +1033,7 @@ pub const Tokenizer = struct {
1033	1033	state = .doc_comment_start;
1034	1034	},
1035	1035	'!' => {
1036		result.id = .ContainerDocComment;
	1036	result.tag = .ContainerDocComment;
1037	1037	state = .container_doc_comment;
1038	1038	},
1039	1039	'\n' => break,
...	...	@@ -1048,16 +1048,16 @@ pub const Tokenizer = struct {
1048	1048	state = .line_comment;
1049	1049	},
1050	1050	'\n' => {
1051		result.id = .DocComment;
	1051	result.tag = .DocComment;
1052	1052	break;
1053	1053	},
1054	1054	'\t', '\r' => {
1055	1055	state = .doc_comment;
1056		result.id = .DocComment;
	1056	result.tag = .DocComment;
1057	1057	},
1058	1058	else => {
1059	1059	state = .doc_comment;
1060		result.id = .DocComment;
	1060	result.tag = .DocComment;
1061	1061	self.checkLiteralCharacter();
1062	1062	},
1063	1063	},
...	...	@@ -1083,7 +1083,7 @@ pub const Tokenizer = struct {
1083	1083	},
1084	1084	else => {
1085	1085	if (isIdentifierChar(c)) {
1086		result.id = .Invalid;
	1086	result.tag = .Invalid;
1087	1087	}
1088	1088	break;
1089	1089	},
...	...	@@ -1093,7 +1093,7 @@ pub const Tokenizer = struct {
1093	1093	state = .int_literal_bin;
1094	1094	},
1095	1095	else => {
1096		result.id = .Invalid;
	1096	result.tag = .Invalid;
1097	1097	break;
1098	1098	},
1099	1099	},
...	...	@@ -1104,7 +1104,7 @@ pub const Tokenizer = struct {
1104	1104	'0'...'1' => {},
1105	1105	else => {
1106	1106	if (isIdentifierChar(c)) {
1107		result.id = .Invalid;
	1107	result.tag = .Invalid;
1108	1108	}
1109	1109	break;
1110	1110	},
...	...	@@ -1114,7 +1114,7 @@ pub const Tokenizer = struct {
1114	1114	state = .int_literal_oct;
1115	1115	},
1116	1116	else => {
1117		result.id = .Invalid;
	1117	result.tag = .Invalid;
1118	1118	break;
1119	1119	},
1120	1120	},
...	...	@@ -1125,7 +1125,7 @@ pub const Tokenizer = struct {
1125	1125	'0'...'7' => {},
1126	1126	else => {
1127	1127	if (isIdentifierChar(c)) {
1128		result.id = .Invalid;
	1128	result.tag = .Invalid;
1129	1129	}
1130	1130	break;
1131	1131	},
...	...	@@ -1135,7 +1135,7 @@ pub const Tokenizer = struct {
1135	1135	state = .int_literal_dec;
1136	1136	},
1137	1137	else => {
1138		result.id = .Invalid;
	1138	result.tag = .Invalid;
1139	1139	break;
1140	1140	},
1141	1141	},
...	...	@@ -1145,16 +1145,16 @@ pub const Tokenizer = struct {
1145	1145	},
1146	1146	'.' => {
1147	1147	state = .num_dot_dec;
1148		result.id = .FloatLiteral;
	1148	result.tag = .FloatLiteral;
1149	1149	},
1150	1150	'e', 'E' => {
1151	1151	state = .float_exponent_unsigned;
1152		result.id = .FloatLiteral;
	1152	result.tag = .FloatLiteral;
1153	1153	},
1154	1154	'0'...'9' => {},
1155	1155	else => {
1156	1156	if (isIdentifierChar(c)) {
1157		result.id = .Invalid;
	1157	result.tag = .Invalid;
1158	1158	}
1159	1159	break;
1160	1160	},
...	...	@@ -1164,7 +1164,7 @@ pub const Tokenizer = struct {
1164	1164	state = .int_literal_hex;
1165	1165	},
1166	1166	else => {
1167		result.id = .Invalid;
	1167	result.tag = .Invalid;
1168	1168	break;
1169	1169	},
1170	1170	},
...	...	@@ -1174,23 +1174,23 @@ pub const Tokenizer = struct {
1174	1174	},
1175	1175	'.' => {
1176	1176	state = .num_dot_hex;
1177		result.id = .FloatLiteral;
	1177	result.tag = .FloatLiteral;
1178	1178	},
1179	1179	'p', 'P' => {
1180	1180	state = .float_exponent_unsigned;
1181		result.id = .FloatLiteral;
	1181	result.tag = .FloatLiteral;
1182	1182	},
1183	1183	'0'...'9', 'a'...'f', 'A'...'F' => {},
1184	1184	else => {
1185	1185	if (isIdentifierChar(c)) {
1186		result.id = .Invalid;
	1186	result.tag = .Invalid;
1187	1187	}
1188	1188	break;
1189	1189	},
1190	1190	},
1191	1191	.num_dot_dec => switch (c) {
1192	1192	'.' => {
1193		result.id = .IntegerLiteral;
	1193	result.tag = .IntegerLiteral;
1194	1194	self.index -= 1;
1195	1195	state = .start;
1196	1196	break;
...	...	@@ -1203,14 +1203,14 @@ pub const Tokenizer = struct {
1203	1203	},
1204	1204	else => {
1205	1205	if (isIdentifierChar(c)) {
1206		result.id = .Invalid;
	1206	result.tag = .Invalid;
1207	1207	}
1208	1208	break;
1209	1209	},
1210	1210	},
1211	1211	.num_dot_hex => switch (c) {
1212	1212	'.' => {
1213		result.id = .IntegerLiteral;
	1213	result.tag = .IntegerLiteral;
1214	1214	self.index -= 1;
1215	1215	state = .start;
1216	1216	break;
...	...	@@ -1219,12 +1219,12 @@ pub const Tokenizer = struct {
1219	1219	state = .float_exponent_unsigned;
1220	1220	},
1221	1221	'0'...'9', 'a'...'f', 'A'...'F' => {
1222		result.id = .FloatLiteral;
	1222	result.tag = .FloatLiteral;
1223	1223	state = .float_fraction_hex;
1224	1224	},
1225	1225	else => {
1226	1226	if (isIdentifierChar(c)) {
1227		result.id = .Invalid;
	1227	result.tag = .Invalid;
1228	1228	}
1229	1229	break;
1230	1230	},
...	...	@@ -1234,7 +1234,7 @@ pub const Tokenizer = struct {
1234	1234	state = .float_fraction_dec;
1235	1235	},
1236	1236	else => {
1237		result.id = .Invalid;
	1237	result.tag = .Invalid;
1238	1238	break;
1239	1239	},
1240	1240	},
...	...	@@ -1248,7 +1248,7 @@ pub const Tokenizer = struct {
1248	1248	'0'...'9' => {},
1249	1249	else => {
1250	1250	if (isIdentifierChar(c)) {
1251		result.id = .Invalid;
	1251	result.tag = .Invalid;
1252	1252	}
1253	1253	break;
1254	1254	},
...	...	@@ -1258,7 +1258,7 @@ pub const Tokenizer = struct {
1258	1258	state = .float_fraction_hex;
1259	1259	},
1260	1260	else => {
1261		result.id = .Invalid;
	1261	result.tag = .Invalid;
1262	1262	break;
1263	1263	},
1264	1264	},
...	...	@@ -1272,7 +1272,7 @@ pub const Tokenizer = struct {
1272	1272	'0'...'9', 'a'...'f', 'A'...'F' => {},
1273	1273	else => {
1274	1274	if (isIdentifierChar(c)) {
1275		result.id = .Invalid;
	1275	result.tag = .Invalid;
1276	1276	}
1277	1277	break;
1278	1278	},
...	...	@@ -1292,7 +1292,7 @@ pub const Tokenizer = struct {
1292	1292	state = .float_exponent_num;
1293	1293	},
1294	1294	else => {
1295		result.id = .Invalid;
	1295	result.tag = .Invalid;
1296	1296	break;
1297	1297	},
1298	1298	},
...	...	@@ -1303,7 +1303,7 @@ pub const Tokenizer = struct {
1303	1303	'0'...'9' => {},
1304	1304	else => {
1305	1305	if (isIdentifierChar(c)) {
1306		result.id = .Invalid;
	1306	result.tag = .Invalid;
1307	1307	}
1308	1308	break;
1309	1309	},
...	...	@@ -1327,18 +1327,18 @@ pub const Tokenizer = struct {
1327	1327	=> {},
1328	1328
1329	1329	.identifier => {
1330		if (Token.getKeyword(self.buffer[result.loc.start..self.index])) \|id\| {
1331		result.id = id;
	1330	if (Token.getKeyword(self.buffer[result.loc.start..self.index])) \|tag\| {
	1331	result.tag = tag;
1332	1332	}
1333	1333	},
1334	1334	.line_comment, .line_comment_start => {
1335		result.id = .LineComment;
	1335	result.tag = .LineComment;
1336	1336	},
1337	1337	.doc_comment, .doc_comment_start => {
1338		result.id = .DocComment;
	1338	result.tag = .DocComment;
1339	1339	},
1340	1340	.container_doc_comment => {
1341		result.id = .ContainerDocComment;
	1341	result.tag = .ContainerDocComment;
1342	1342	},
1343	1343
1344	1344	.int_literal_dec_no_underscore,
...	...	@@ -1361,76 +1361,76 @@ pub const Tokenizer = struct {
1361	1361	.char_literal_unicode,
1362	1362	.string_literal_backslash,
1363	1363	=> {
1364		result.id = .Invalid;
	1364	result.tag = .Invalid;
1365	1365	},
1366	1366
1367	1367	.equal => {
1368		result.id = .Equal;
	1368	result.tag = .Equal;
1369	1369	},
1370	1370	.bang => {
1371		result.id = .Bang;
	1371	result.tag = .Bang;
1372	1372	},
1373	1373	.minus => {
1374		result.id = .Minus;
	1374	result.tag = .Minus;
1375	1375	},
1376	1376	.slash => {
1377		result.id = .Slash;
	1377	result.tag = .Slash;
1378	1378	},
1379	1379	.zero => {
1380		result.id = .IntegerLiteral;
	1380	result.tag = .IntegerLiteral;
1381	1381	},
1382	1382	.ampersand => {
1383		result.id = .Ampersand;
	1383	result.tag = .Ampersand;
1384	1384	},
1385	1385	.period => {
1386		result.id = .Period;
	1386	result.tag = .Period;
1387	1387	},
1388	1388	.period_2 => {
1389		result.id = .Ellipsis2;
	1389	result.tag = .Ellipsis2;
1390	1390	},
1391	1391	.period_asterisk => {
1392		result.id = .PeriodAsterisk;
	1392	result.tag = .PeriodAsterisk;
1393	1393	},
1394	1394	.pipe => {
1395		result.id = .Pipe;
	1395	result.tag = .Pipe;
1396	1396	},
1397	1397	.angle_bracket_angle_bracket_right => {
1398		result.id = .AngleBracketAngleBracketRight;
	1398	result.tag = .AngleBracketAngleBracketRight;
1399	1399	},
1400	1400	.angle_bracket_right => {
1401		result.id = .AngleBracketRight;
	1401	result.tag = .AngleBracketRight;
1402	1402	},
1403	1403	.angle_bracket_angle_bracket_left => {
1404		result.id = .AngleBracketAngleBracketLeft;
	1404	result.tag = .AngleBracketAngleBracketLeft;
1405	1405	},
1406	1406	.angle_bracket_left => {
1407		result.id = .AngleBracketLeft;
	1407	result.tag = .AngleBracketLeft;
1408	1408	},
1409	1409	.plus_percent => {
1410		result.id = .PlusPercent;
	1410	result.tag = .PlusPercent;
1411	1411	},
1412	1412	.plus => {
1413		result.id = .Plus;
	1413	result.tag = .Plus;
1414	1414	},
1415	1415	.percent => {
1416		result.id = .Percent;
	1416	result.tag = .Percent;
1417	1417	},
1418	1418	.caret => {
1419		result.id = .Caret;
	1419	result.tag = .Caret;
1420	1420	},
1421	1421	.asterisk_percent => {
1422		result.id = .AsteriskPercent;
	1422	result.tag = .AsteriskPercent;
1423	1423	},
1424	1424	.asterisk => {
1425		result.id = .Asterisk;
	1425	result.tag = .Asterisk;
1426	1426	},
1427	1427	.minus_percent => {
1428		result.id = .MinusPercent;
	1428	result.tag = .MinusPercent;
1429	1429	},
1430	1430	}
1431	1431	}
1432	1432
1433		if (result.id == .Eof) {
	1433	if (result.tag == .Eof) {
1434	1434	if (self.pending_invalid_token) \|token\| {
1435	1435	self.pending_invalid_token = null;
1436	1436	return token;
...	...	@@ -1446,7 +1446,7 @@ pub const Tokenizer = struct {
1446	1446	const invalid_length = self.getInvalidCharacterLength();
1447	1447	if (invalid_length == 0) return;
1448	1448	self.pending_invalid_token = .{
1449		.id = .Invalid,
	1449	.tag = .Invalid,
1450	1450	.loc = .{
1451	1451	.start = self.index,
1452	1452	.end = self.index + invalid_length,
...	...	@@ -1493,14 +1493,14 @@ pub const Tokenizer = struct {
1493	1493	};
1494	1494
1495	1495	test "tokenizer" {
1496		testTokenize("test", &[_]Token.Id{.Keyword_test});
	1496	testTokenize("test", &[_]Token.Tag{.Keyword_test});
1497	1497	}
1498	1498
1499	1499	test "tokenizer - unknown length pointer and then c pointer" {
1500	1500	testTokenize(
1501	1501	\\[*]u8
1502	1502	\\[*c]u8
1503		, &[_]Token.Id{
	1503	, &[_]Token.Tag{
1504	1504	.LBracket,
1505	1505	.Asterisk,
1506	1506	.RBracket,
...	...	@@ -1516,70 +1516,70 @@ test "tokenizer - unknown length pointer and then c pointer" {
1516	1516	test "tokenizer - char literal with hex escape" {
1517	1517	testTokenize(
1518	1518	\\'\x1b'
1519		, &[_]Token.Id{.CharLiteral});
	1519	, &[_]Token.Tag{.CharLiteral});
1520	1520	testTokenize(
1521	1521	\\'\x1'
1522		, &[_]Token.Id{ .Invalid, .Invalid });
	1522	, &[_]Token.Tag{ .Invalid, .Invalid });
1523	1523	}
1524	1524
1525	1525	test "tokenizer - char literal with unicode escapes" {
1526	1526	// Valid unicode escapes
1527	1527	testTokenize(
1528	1528	\\'\u{3}'
1529		, &[_]Token.Id{.CharLiteral});
	1529	, &[_]Token.Tag{.CharLiteral});
1530	1530	testTokenize(
1531	1531	\\'\u{01}'
1532		, &[_]Token.Id{.CharLiteral});
	1532	, &[_]Token.Tag{.CharLiteral});
1533	1533	testTokenize(
1534	1534	\\'\u{2a}'
1535		, &[_]Token.Id{.CharLiteral});
	1535	, &[_]Token.Tag{.CharLiteral});
1536	1536	testTokenize(
1537	1537	\\'\u{3f9}'
1538		, &[_]Token.Id{.CharLiteral});
	1538	, &[_]Token.Tag{.CharLiteral});
1539	1539	testTokenize(
1540	1540	\\'\u{6E09aBc1523}'
1541		, &[_]Token.Id{.CharLiteral});
	1541	, &[_]Token.Tag{.CharLiteral});
1542	1542	testTokenize(
1543	1543	\\"\u{440}"
1544		, &[_]Token.Id{.StringLiteral});
	1544	, &[_]Token.Tag{.StringLiteral});
1545	1545
1546	1546	// Invalid unicode escapes
1547	1547	testTokenize(
1548	1548	\\'\u'
1549		, &[_]Token.Id{.Invalid});
	1549	, &[_]Token.Tag{.Invalid});
1550	1550	testTokenize(
1551	1551	\\'\u{{'
1552		, &[_]Token.Id{ .Invalid, .Invalid });
	1552	, &[_]Token.Tag{ .Invalid, .Invalid });
1553	1553	testTokenize(
1554	1554	\\'\u{}'
1555		, &[_]Token.Id{ .Invalid, .Invalid });
	1555	, &[_]Token.Tag{ .Invalid, .Invalid });
1556	1556	testTokenize(
1557	1557	\\'\u{s}'
1558		, &[_]Token.Id{ .Invalid, .Invalid });
	1558	, &[_]Token.Tag{ .Invalid, .Invalid });
1559	1559	testTokenize(
1560	1560	\\'\u{2z}'
1561		, &[_]Token.Id{ .Invalid, .Invalid });
	1561	, &[_]Token.Tag{ .Invalid, .Invalid });
1562	1562	testTokenize(
1563	1563	\\'\u{4a'
1564		, &[_]Token.Id{.Invalid});
	1564	, &[_]Token.Tag{.Invalid});
1565	1565
1566	1566	// Test old-style unicode literals
1567	1567	testTokenize(
1568	1568	\\'\u0333'
1569		, &[_]Token.Id{ .Invalid, .Invalid });
	1569	, &[_]Token.Tag{ .Invalid, .Invalid });
1570	1570	testTokenize(
1571	1571	\\'\U0333'
1572		, &[_]Token.Id{ .Invalid, .IntegerLiteral, .Invalid });
	1572	, &[_]Token.Tag{ .Invalid, .IntegerLiteral, .Invalid });
1573	1573	}
1574	1574
1575	1575	test "tokenizer - char literal with unicode code point" {
1576	1576	testTokenize(
1577	1577	\\'💩'
1578		, &[_]Token.Id{.CharLiteral});
	1578	, &[_]Token.Tag{.CharLiteral});
1579	1579	}
1580	1580
1581	1581	test "tokenizer - float literal e exponent" {
1582		testTokenize("a = 4.94065645841246544177e-324;\n", &[_]Token.Id{
	1582	testTokenize("a = 4.94065645841246544177e-324;\n", &[_]Token.Tag{
1583	1583	.Identifier,
1584	1584	.Equal,
1585	1585	.FloatLiteral,
...	...	@@ -1588,7 +1588,7 @@ test "tokenizer - float literal e exponent" {
1588	1588	}
1589	1589
1590	1590	test "tokenizer - float literal p exponent" {
1591		testTokenize("a = 0x1.a827999fcef32p+1022;\n", &[_]Token.Id{
	1591	testTokenize("a = 0x1.a827999fcef32p+1022;\n", &[_]Token.Tag{
1592	1592	.Identifier,
1593	1593	.Equal,
1594	1594	.FloatLiteral,
...	...	@@ -1597,71 +1597,71 @@ test "tokenizer - float literal p exponent" {
1597	1597	}
1598	1598
1599	1599	test "tokenizer - chars" {
1600		testTokenize("'c'", &[_]Token.Id{.CharLiteral});
	1600	testTokenize("'c'", &[_]Token.Tag{.CharLiteral});
1601	1601	}
1602	1602
1603	1603	test "tokenizer - invalid token characters" {
1604		testTokenize("#", &[_]Token.Id{.Invalid});
1605		testTokenize("`", &[_]Token.Id{.Invalid});
1606		testTokenize("'c", &[_]Token.Id{.Invalid});
1607		testTokenize("'", &[_]Token.Id{.Invalid});
1608		testTokenize("''", &[_]Token.Id{ .Invalid, .Invalid });
	1604	testTokenize("#", &[_]Token.Tag{.Invalid});
	1605	testTokenize("`", &[_]Token.Tag{.Invalid});
	1606	testTokenize("'c", &[_]Token.Tag{.Invalid});
	1607	testTokenize("'", &[_]Token.Tag{.Invalid});
	1608	testTokenize("''", &[_]Token.Tag{ .Invalid, .Invalid });
1609	1609	}
1610	1610
1611	1611	test "tokenizer - invalid literal/comment characters" {
1612		testTokenize("\"\x00\"", &[_]Token.Id{
	1612	testTokenize("\"\x00\"", &[_]Token.Tag{
1613	1613	.StringLiteral,
1614	1614	.Invalid,
1615	1615	});
1616		testTokenize("//\x00", &[_]Token.Id{
	1616	testTokenize("//\x00", &[_]Token.Tag{
1617	1617	.LineComment,
1618	1618	.Invalid,
1619	1619	});
1620		testTokenize("//\x1f", &[_]Token.Id{
	1620	testTokenize("//\x1f", &[_]Token.Tag{
1621	1621	.LineComment,
1622	1622	.Invalid,
1623	1623	});
1624		testTokenize("//\x7f", &[_]Token.Id{
	1624	testTokenize("//\x7f", &[_]Token.Tag{
1625	1625	.LineComment,
1626	1626	.Invalid,
1627	1627	});
1628	1628	}
1629	1629
1630	1630	test "tokenizer - utf8" {
1631		testTokenize("//\xc2\x80", &[_]Token.Id{.LineComment});
1632		testTokenize("//\xf4\x8f\xbf\xbf", &[_]Token.Id{.LineComment});
	1631	testTokenize("//\xc2\x80", &[_]Token.Tag{.LineComment});
	1632	testTokenize("//\xf4\x8f\xbf\xbf", &[_]Token.Tag{.LineComment});
1633	1633	}
1634	1634
1635	1635	test "tokenizer - invalid utf8" {
1636		testTokenize("//\x80", &[_]Token.Id{
	1636	testTokenize("//\x80", &[_]Token.Tag{
1637	1637	.LineComment,
1638	1638	.Invalid,
1639	1639	});
1640		testTokenize("//\xbf", &[_]Token.Id{
	1640	testTokenize("//\xbf", &[_]Token.Tag{
1641	1641	.LineComment,
1642	1642	.Invalid,
1643	1643	});
1644		testTokenize("//\xf8", &[_]Token.Id{
	1644	testTokenize("//\xf8", &[_]Token.Tag{
1645	1645	.LineComment,
1646	1646	.Invalid,
1647	1647	});
1648		testTokenize("//\xff", &[_]Token.Id{
	1648	testTokenize("//\xff", &[_]Token.Tag{
1649	1649	.LineComment,
1650	1650	.Invalid,
1651	1651	});
1652		testTokenize("//\xc2\xc0", &[_]Token.Id{
	1652	testTokenize("//\xc2\xc0", &[_]Token.Tag{
1653	1653	.LineComment,
1654	1654	.Invalid,
1655	1655	});
1656		testTokenize("//\xe0", &[_]Token.Id{
	1656	testTokenize("//\xe0", &[_]Token.Tag{
1657	1657	.LineComment,
1658	1658	.Invalid,
1659	1659	});
1660		testTokenize("//\xf0", &[_]Token.Id{
	1660	testTokenize("//\xf0", &[_]Token.Tag{
1661	1661	.LineComment,
1662	1662	.Invalid,
1663	1663	});
1664		testTokenize("//\xf0\x90\x80\xc0", &[_]Token.Id{
	1664	testTokenize("//\xf0\x90\x80\xc0", &[_]Token.Tag{
1665	1665	.LineComment,
1666	1666	.Invalid,
1667	1667	});
...	...	@@ -1669,28 +1669,28 @@ test "tokenizer - invalid utf8" {
1669	1669
1670	1670	test "tokenizer - illegal unicode codepoints" {
1671	1671	// unicode newline characters.U+0085, U+2028, U+2029
1672		testTokenize("//\xc2\x84", &[_]Token.Id{.LineComment});
1673		testTokenize("//\xc2\x85", &[_]Token.Id{
	1672	testTokenize("//\xc2\x84", &[_]Token.Tag{.LineComment});
	1673	testTokenize("//\xc2\x85", &[_]Token.Tag{
1674	1674	.LineComment,
1675	1675	.Invalid,
1676	1676	});
1677		testTokenize("//\xc2\x86", &[_]Token.Id{.LineComment});
1678		testTokenize("//\xe2\x80\xa7", &[_]Token.Id{.LineComment});
1679		testTokenize("//\xe2\x80\xa8", &[_]Token.Id{
	1677	testTokenize("//\xc2\x86", &[_]Token.Tag{.LineComment});
	1678	testTokenize("//\xe2\x80\xa7", &[_]Token.Tag{.LineComment});
	1679	testTokenize("//\xe2\x80\xa8", &[_]Token.Tag{
1680	1680	.LineComment,
1681	1681	.Invalid,
1682	1682	});
1683		testTokenize("//\xe2\x80\xa9", &[_]Token.Id{
	1683	testTokenize("//\xe2\x80\xa9", &[_]Token.Tag{
1684	1684	.LineComment,
1685	1685	.Invalid,
1686	1686	});
1687		testTokenize("//\xe2\x80\xaa", &[_]Token.Id{.LineComment});
	1687	testTokenize("//\xe2\x80\xaa", &[_]Token.Tag{.LineComment});
1688	1688	}
1689	1689
1690	1690	test "tokenizer - string identifier and builtin fns" {
1691	1691	testTokenize(
1692	1692	\\const @"if" = @import("std");
1693		, &[_]Token.Id{
	1693	, &[_]Token.Tag{
1694	1694	.Keyword_const,
1695	1695	.Identifier,
1696	1696	.Equal,
...	...	@@ -1705,7 +1705,7 @@ test "tokenizer - string identifier and builtin fns" {
1705	1705	test "tokenizer - multiline string literal with literal tab" {
1706	1706	testTokenize(
1707	1707	\\\\foo bar
1708		, &[_]Token.Id{
	1708	, &[_]Token.Tag{
1709	1709	.MultilineStringLiteralLine,
1710	1710	});
1711	1711	}
...	...	@@ -1718,7 +1718,7 @@ test "tokenizer - comments with literal tab" {
1718	1718	\\// foo
1719	1719	\\/// foo
1720	1720	\\/// /foo
1721		, &[_]Token.Id{
	1721	, &[_]Token.Tag{
1722	1722	.LineComment,
1723	1723	.ContainerDocComment,
1724	1724	.DocComment,
...	...	@@ -1729,21 +1729,21 @@ test "tokenizer - comments with literal tab" {
1729	1729	}
1730	1730
1731	1731	test "tokenizer - pipe and then invalid" {
1732		testTokenize("\|\|=", &[_]Token.Id{
	1732	testTokenize("\|\|=", &[_]Token.Tag{
1733	1733	.PipePipe,
1734	1734	.Equal,
1735	1735	});
1736	1736	}
1737	1737
1738	1738	test "tokenizer - line comment and doc comment" {
1739		testTokenize("//", &[_]Token.Id{.LineComment});
1740		testTokenize("// a / b", &[_]Token.Id{.LineComment});
1741		testTokenize("// /", &[_]Token.Id{.LineComment});
1742		testTokenize("/// a", &[_]Token.Id{.DocComment});
1743		testTokenize("///", &[_]Token.Id{.DocComment});
1744		testTokenize("////", &[_]Token.Id{.LineComment});
1745		testTokenize("//!", &[_]Token.Id{.ContainerDocComment});
1746		testTokenize("//!!", &[_]Token.Id{.ContainerDocComment});
	1739	testTokenize("//", &[_]Token.Tag{.LineComment});
	1740	testTokenize("// a / b", &[_]Token.Tag{.LineComment});
	1741	testTokenize("// /", &[_]Token.Tag{.LineComment});
	1742	testTokenize("/// a", &[_]Token.Tag{.DocComment});
	1743	testTokenize("///", &[_]Token.Tag{.DocComment});
	1744	testTokenize("////", &[_]Token.Tag{.LineComment});
	1745	testTokenize("//!", &[_]Token.Tag{.ContainerDocComment});
	1746	testTokenize("//!!", &[_]Token.Tag{.ContainerDocComment});
1747	1747	}
1748	1748
1749	1749	test "tokenizer - line comment followed by identifier" {
...	...	@@ -1751,7 +1751,7 @@ test "tokenizer - line comment followed by identifier" {
1751	1751	\\ Unexpected,
1752	1752	\\ // another
1753	1753	\\ Another,
1754		, &[_]Token.Id{
	1754	, &[_]Token.Tag{
1755	1755	.Identifier,
1756	1756	.Comma,
1757	1757	.LineComment,
...	...	@@ -1761,14 +1761,14 @@ test "tokenizer - line comment followed by identifier" {
1761	1761	}
1762	1762
1763	1763	test "tokenizer - UTF-8 BOM is recognized and skipped" {
1764		testTokenize("\xEF\xBB\xBFa;\n", &[_]Token.Id{
	1764	testTokenize("\xEF\xBB\xBFa;\n", &[_]Token.Tag{
1765	1765	.Identifier,
1766	1766	.Semicolon,
1767	1767	});
1768	1768	}
1769	1769
1770	1770	test "correctly parse pointer assignment" {
1771		testTokenize("b.*=3;\n", &[_]Token.Id{
	1771	testTokenize("b.*=3;\n", &[_]Token.Tag{
1772	1772	.Identifier,
1773	1773	.PeriodAsterisk,
1774	1774	.Equal,
...	...	@@ -1778,14 +1778,14 @@ test "correctly parse pointer assignment" {
1778	1778	}
1779	1779
1780	1780	test "correctly parse pointer dereference followed by asterisk" {
1781		testTokenize("\"b\".* ** 10", &[_]Token.Id{
	1781	testTokenize("\"b\".* ** 10", &[_]Token.Tag{
1782	1782	.StringLiteral,
1783	1783	.PeriodAsterisk,
1784	1784	.AsteriskAsterisk,
1785	1785	.IntegerLiteral,
1786	1786	});
1787	1787
1788		testTokenize("(\"b\".)* 10", &[_]Token.Id{
	1788	testTokenize("(\"b\".)* 10", &[_]Token.Tag{
1789	1789	.LParen,
1790	1790	.StringLiteral,
1791	1791	.PeriodAsterisk,
...	...	@@ -1794,7 +1794,7 @@ test "correctly parse pointer dereference followed by asterisk" {
1794	1794	.IntegerLiteral,
1795	1795	});
1796	1796
1797		testTokenize("\"b\".*** 10", &[_]Token.Id{
	1797	testTokenize("\"b\".*** 10", &[_]Token.Tag{
1798	1798	.StringLiteral,
1799	1799	.Invalid_periodasterisks,
1800	1800	.AsteriskAsterisk,
...	...	@@ -1803,252 +1803,252 @@ test "correctly parse pointer dereference followed by asterisk" {
1803	1803	}
1804	1804
1805	1805	test "tokenizer - range literals" {
1806		testTokenize("0...9", &[_]Token.Id{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
1807		testTokenize("'0'...'9'", &[_]Token.Id{ .CharLiteral, .Ellipsis3, .CharLiteral });
1808		testTokenize("0x00...0x09", &[_]Token.Id{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
1809		testTokenize("0b00...0b11", &[_]Token.Id{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
1810		testTokenize("0o00...0o11", &[_]Token.Id{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
	1806	testTokenize("0...9", &[_]Token.Tag{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
	1807	testTokenize("'0'...'9'", &[_]Token.Tag{ .CharLiteral, .Ellipsis3, .CharLiteral });
	1808	testTokenize("0x00...0x09", &[_]Token.Tag{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
	1809	testTokenize("0b00...0b11", &[_]Token.Tag{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
	1810	testTokenize("0o00...0o11", &[_]Token.Tag{ .IntegerLiteral, .Ellipsis3, .IntegerLiteral });
1811	1811	}
1812	1812
1813	1813	test "tokenizer - number literals decimal" {
1814		testTokenize("0", &[_]Token.Id{.IntegerLiteral});
1815		testTokenize("1", &[_]Token.Id{.IntegerLiteral});
1816		testTokenize("2", &[_]Token.Id{.IntegerLiteral});
1817		testTokenize("3", &[_]Token.Id{.IntegerLiteral});
1818		testTokenize("4", &[_]Token.Id{.IntegerLiteral});
1819		testTokenize("5", &[_]Token.Id{.IntegerLiteral});
1820		testTokenize("6", &[_]Token.Id{.IntegerLiteral});
1821		testTokenize("7", &[_]Token.Id{.IntegerLiteral});
1822		testTokenize("8", &[_]Token.Id{.IntegerLiteral});
1823		testTokenize("9", &[_]Token.Id{.IntegerLiteral});
1824		testTokenize("1..", &[_]Token.Id{ .IntegerLiteral, .Ellipsis2 });
1825		testTokenize("0a", &[_]Token.Id{ .Invalid, .Identifier });
1826		testTokenize("9b", &[_]Token.Id{ .Invalid, .Identifier });
1827		testTokenize("1z", &[_]Token.Id{ .Invalid, .Identifier });
1828		testTokenize("1z_1", &[_]Token.Id{ .Invalid, .Identifier });
1829		testTokenize("9z3", &[_]Token.Id{ .Invalid, .Identifier });
1830
1831		testTokenize("0_0", &[_]Token.Id{.IntegerLiteral});
1832		testTokenize("0001", &[_]Token.Id{.IntegerLiteral});
1833		testTokenize("01234567890", &[_]Token.Id{.IntegerLiteral});
1834		testTokenize("012_345_6789_0", &[_]Token.Id{.IntegerLiteral});
1835		testTokenize("0_1_2_3_4_5_6_7_8_9_0", &[_]Token.Id{.IntegerLiteral});
1836
1837		testTokenize("00_", &[_]Token.Id{.Invalid});
1838		testTokenize("0_0_", &[_]Token.Id{.Invalid});
1839		testTokenize("0__0", &[_]Token.Id{ .Invalid, .Identifier });
1840		testTokenize("0_0f", &[_]Token.Id{ .Invalid, .Identifier });
1841		testTokenize("0_0_f", &[_]Token.Id{ .Invalid, .Identifier });
1842		testTokenize("0_0_f_00", &[_]Token.Id{ .Invalid, .Identifier });
1843		testTokenize("1_,", &[_]Token.Id{ .Invalid, .Comma });
1844
1845		testTokenize("1.", &[_]Token.Id{.FloatLiteral});
1846		testTokenize("0.0", &[_]Token.Id{.FloatLiteral});
1847		testTokenize("1.0", &[_]Token.Id{.FloatLiteral});
1848		testTokenize("10.0", &[_]Token.Id{.FloatLiteral});
1849		testTokenize("0e0", &[_]Token.Id{.FloatLiteral});
1850		testTokenize("1e0", &[_]Token.Id{.FloatLiteral});
1851		testTokenize("1e100", &[_]Token.Id{.FloatLiteral});
1852		testTokenize("1.e100", &[_]Token.Id{.FloatLiteral});
1853		testTokenize("1.0e100", &[_]Token.Id{.FloatLiteral});
1854		testTokenize("1.0e+100", &[_]Token.Id{.FloatLiteral});
1855		testTokenize("1.0e-100", &[_]Token.Id{.FloatLiteral});
1856		testTokenize("1_0_0_0.0_0_0_0_0_1e1_0_0_0", &[_]Token.Id{.FloatLiteral});
1857		testTokenize("1.+", &[_]Token.Id{ .FloatLiteral, .Plus });
1858
1859		testTokenize("1e", &[_]Token.Id{.Invalid});
1860		testTokenize("1.0e1f0", &[_]Token.Id{ .Invalid, .Identifier });
1861		testTokenize("1.0p100", &[_]Token.Id{ .Invalid, .Identifier });
1862		testTokenize("1.0p-100", &[_]Token.Id{ .Invalid, .Identifier, .Minus, .IntegerLiteral });
1863		testTokenize("1.0p1f0", &[_]Token.Id{ .Invalid, .Identifier });
1864		testTokenize("1.0_,", &[_]Token.Id{ .Invalid, .Comma });
1865		testTokenize("1_.0", &[_]Token.Id{ .Invalid, .Period, .IntegerLiteral });
1866		testTokenize("1._", &[_]Token.Id{ .Invalid, .Identifier });
1867		testTokenize("1.a", &[_]Token.Id{ .Invalid, .Identifier });
1868		testTokenize("1.z", &[_]Token.Id{ .Invalid, .Identifier });
1869		testTokenize("1._0", &[_]Token.Id{ .Invalid, .Identifier });
1870		testTokenize("1._+", &[_]Token.Id{ .Invalid, .Identifier, .Plus });
1871		testTokenize("1._e", &[_]Token.Id{ .Invalid, .Identifier });
1872		testTokenize("1.0e", &[_]Token.Id{.Invalid});
1873		testTokenize("1.0e,", &[_]Token.Id{ .Invalid, .Comma });
1874		testTokenize("1.0e_", &[_]Token.Id{ .Invalid, .Identifier });
1875		testTokenize("1.0e+_", &[_]Token.Id{ .Invalid, .Identifier });
1876		testTokenize("1.0e-_", &[_]Token.Id{ .Invalid, .Identifier });
1877		testTokenize("1.0e0_+", &[_]Token.Id{ .Invalid, .Plus });
	1814	testTokenize("0", &[_]Token.Tag{.IntegerLiteral});
	1815	testTokenize("1", &[_]Token.Tag{.IntegerLiteral});
	1816	testTokenize("2", &[_]Token.Tag{.IntegerLiteral});
	1817	testTokenize("3", &[_]Token.Tag{.IntegerLiteral});
	1818	testTokenize("4", &[_]Token.Tag{.IntegerLiteral});
	1819	testTokenize("5", &[_]Token.Tag{.IntegerLiteral});
	1820	testTokenize("6", &[_]Token.Tag{.IntegerLiteral});
	1821	testTokenize("7", &[_]Token.Tag{.IntegerLiteral});
	1822	testTokenize("8", &[_]Token.Tag{.IntegerLiteral});
	1823	testTokenize("9", &[_]Token.Tag{.IntegerLiteral});
	1824	testTokenize("1..", &[_]Token.Tag{ .IntegerLiteral, .Ellipsis2 });
	1825	testTokenize("0a", &[_]Token.Tag{ .Invalid, .Identifier });
	1826	testTokenize("9b", &[_]Token.Tag{ .Invalid, .Identifier });
	1827	testTokenize("1z", &[_]Token.Tag{ .Invalid, .Identifier });
	1828	testTokenize("1z_1", &[_]Token.Tag{ .Invalid, .Identifier });
	1829	testTokenize("9z3", &[_]Token.Tag{ .Invalid, .Identifier });
	1830
	1831	testTokenize("0_0", &[_]Token.Tag{.IntegerLiteral});
	1832	testTokenize("0001", &[_]Token.Tag{.IntegerLiteral});
	1833	testTokenize("01234567890", &[_]Token.Tag{.IntegerLiteral});
	1834	testTokenize("012_345_6789_0", &[_]Token.Tag{.IntegerLiteral});
	1835	testTokenize("0_1_2_3_4_5_6_7_8_9_0", &[_]Token.Tag{.IntegerLiteral});
	1836
	1837	testTokenize("00_", &[_]Token.Tag{.Invalid});
	1838	testTokenize("0_0_", &[_]Token.Tag{.Invalid});
	1839	testTokenize("0__0", &[_]Token.Tag{ .Invalid, .Identifier });
	1840	testTokenize("0_0f", &[_]Token.Tag{ .Invalid, .Identifier });
	1841	testTokenize("0_0_f", &[_]Token.Tag{ .Invalid, .Identifier });
	1842	testTokenize("0_0_f_00", &[_]Token.Tag{ .Invalid, .Identifier });
	1843	testTokenize("1_,", &[_]Token.Tag{ .Invalid, .Comma });
	1844
	1845	testTokenize("1.", &[_]Token.Tag{.FloatLiteral});
	1846	testTokenize("0.0", &[_]Token.Tag{.FloatLiteral});
	1847	testTokenize("1.0", &[_]Token.Tag{.FloatLiteral});
	1848	testTokenize("10.0", &[_]Token.Tag{.FloatLiteral});
	1849	testTokenize("0e0", &[_]Token.Tag{.FloatLiteral});
	1850	testTokenize("1e0", &[_]Token.Tag{.FloatLiteral});
	1851	testTokenize("1e100", &[_]Token.Tag{.FloatLiteral});
	1852	testTokenize("1.e100", &[_]Token.Tag{.FloatLiteral});
	1853	testTokenize("1.0e100", &[_]Token.Tag{.FloatLiteral});
	1854	testTokenize("1.0e+100", &[_]Token.Tag{.FloatLiteral});
	1855	testTokenize("1.0e-100", &[_]Token.Tag{.FloatLiteral});
	1856	testTokenize("1_0_0_0.0_0_0_0_0_1e1_0_0_0", &[_]Token.Tag{.FloatLiteral});
	1857	testTokenize("1.+", &[_]Token.Tag{ .FloatLiteral, .Plus });
	1858
	1859	testTokenize("1e", &[_]Token.Tag{.Invalid});
	1860	testTokenize("1.0e1f0", &[_]Token.Tag{ .Invalid, .Identifier });
	1861	testTokenize("1.0p100", &[_]Token.Tag{ .Invalid, .Identifier });
	1862	testTokenize("1.0p-100", &[_]Token.Tag{ .Invalid, .Identifier, .Minus, .IntegerLiteral });
	1863	testTokenize("1.0p1f0", &[_]Token.Tag{ .Invalid, .Identifier });
	1864	testTokenize("1.0_,", &[_]Token.Tag{ .Invalid, .Comma });
	1865	testTokenize("1_.0", &[_]Token.Tag{ .Invalid, .Period, .IntegerLiteral });
	1866	testTokenize("1._", &[_]Token.Tag{ .Invalid, .Identifier });
	1867	testTokenize("1.a", &[_]Token.Tag{ .Invalid, .Identifier });
	1868	testTokenize("1.z", &[_]Token.Tag{ .Invalid, .Identifier });
	1869	testTokenize("1._0", &[_]Token.Tag{ .Invalid, .Identifier });
	1870	testTokenize("1._+", &[_]Token.Tag{ .Invalid, .Identifier, .Plus });
	1871	testTokenize("1._e", &[_]Token.Tag{ .Invalid, .Identifier });
	1872	testTokenize("1.0e", &[_]Token.Tag{.Invalid});
	1873	testTokenize("1.0e,", &[_]Token.Tag{ .Invalid, .Comma });
	1874	testTokenize("1.0e_", &[_]Token.Tag{ .Invalid, .Identifier });
	1875	testTokenize("1.0e+_", &[_]Token.Tag{ .Invalid, .Identifier });
	1876	testTokenize("1.0e-_", &[_]Token.Tag{ .Invalid, .Identifier });
	1877	testTokenize("1.0e0_+", &[_]Token.Tag{ .Invalid, .Plus });
1878	1878	}
1879	1879
1880	1880	test "tokenizer - number literals binary" {
1881		testTokenize("0b0", &[_]Token.Id{.IntegerLiteral});
1882		testTokenize("0b1", &[_]Token.Id{.IntegerLiteral});
1883		testTokenize("0b2", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1884		testTokenize("0b3", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1885		testTokenize("0b4", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1886		testTokenize("0b5", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1887		testTokenize("0b6", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1888		testTokenize("0b7", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1889		testTokenize("0b8", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1890		testTokenize("0b9", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1891		testTokenize("0ba", &[_]Token.Id{ .Invalid, .Identifier });
1892		testTokenize("0bb", &[_]Token.Id{ .Invalid, .Identifier });
1893		testTokenize("0bc", &[_]Token.Id{ .Invalid, .Identifier });
1894		testTokenize("0bd", &[_]Token.Id{ .Invalid, .Identifier });
1895		testTokenize("0be", &[_]Token.Id{ .Invalid, .Identifier });
1896		testTokenize("0bf", &[_]Token.Id{ .Invalid, .Identifier });
1897		testTokenize("0bz", &[_]Token.Id{ .Invalid, .Identifier });
1898
1899		testTokenize("0b0000_0000", &[_]Token.Id{.IntegerLiteral});
1900		testTokenize("0b1111_1111", &[_]Token.Id{.IntegerLiteral});
1901		testTokenize("0b10_10_10_10", &[_]Token.Id{.IntegerLiteral});
1902		testTokenize("0b0_1_0_1_0_1_0_1", &[_]Token.Id{.IntegerLiteral});
1903		testTokenize("0b1.", &[_]Token.Id{ .IntegerLiteral, .Period });
1904		testTokenize("0b1.0", &[_]Token.Id{ .IntegerLiteral, .Period, .IntegerLiteral });
1905
1906		testTokenize("0B0", &[_]Token.Id{ .Invalid, .Identifier });
1907		testTokenize("0b_", &[_]Token.Id{ .Invalid, .Identifier });
1908		testTokenize("0b_0", &[_]Token.Id{ .Invalid, .Identifier });
1909		testTokenize("0b1_", &[_]Token.Id{.Invalid});
1910		testTokenize("0b0__1", &[_]Token.Id{ .Invalid, .Identifier });
1911		testTokenize("0b0_1_", &[_]Token.Id{.Invalid});
1912		testTokenize("0b1e", &[_]Token.Id{ .Invalid, .Identifier });
1913		testTokenize("0b1p", &[_]Token.Id{ .Invalid, .Identifier });
1914		testTokenize("0b1e0", &[_]Token.Id{ .Invalid, .Identifier });
1915		testTokenize("0b1p0", &[_]Token.Id{ .Invalid, .Identifier });
1916		testTokenize("0b1_,", &[_]Token.Id{ .Invalid, .Comma });
	1881	testTokenize("0b0", &[_]Token.Tag{.IntegerLiteral});
	1882	testTokenize("0b1", &[_]Token.Tag{.IntegerLiteral});
	1883	testTokenize("0b2", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1884	testTokenize("0b3", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1885	testTokenize("0b4", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1886	testTokenize("0b5", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1887	testTokenize("0b6", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1888	testTokenize("0b7", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1889	testTokenize("0b8", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1890	testTokenize("0b9", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1891	testTokenize("0ba", &[_]Token.Tag{ .Invalid, .Identifier });
	1892	testTokenize("0bb", &[_]Token.Tag{ .Invalid, .Identifier });
	1893	testTokenize("0bc", &[_]Token.Tag{ .Invalid, .Identifier });
	1894	testTokenize("0bd", &[_]Token.Tag{ .Invalid, .Identifier });
	1895	testTokenize("0be", &[_]Token.Tag{ .Invalid, .Identifier });
	1896	testTokenize("0bf", &[_]Token.Tag{ .Invalid, .Identifier });
	1897	testTokenize("0bz", &[_]Token.Tag{ .Invalid, .Identifier });
	1898
	1899	testTokenize("0b0000_0000", &[_]Token.Tag{.IntegerLiteral});
	1900	testTokenize("0b1111_1111", &[_]Token.Tag{.IntegerLiteral});
	1901	testTokenize("0b10_10_10_10", &[_]Token.Tag{.IntegerLiteral});
	1902	testTokenize("0b0_1_0_1_0_1_0_1", &[_]Token.Tag{.IntegerLiteral});
	1903	testTokenize("0b1.", &[_]Token.Tag{ .IntegerLiteral, .Period });
	1904	testTokenize("0b1.0", &[_]Token.Tag{ .IntegerLiteral, .Period, .IntegerLiteral });
	1905
	1906	testTokenize("0B0", &[_]Token.Tag{ .Invalid, .Identifier });
	1907	testTokenize("0b_", &[_]Token.Tag{ .Invalid, .Identifier });
	1908	testTokenize("0b_0", &[_]Token.Tag{ .Invalid, .Identifier });
	1909	testTokenize("0b1_", &[_]Token.Tag{.Invalid});
	1910	testTokenize("0b0__1", &[_]Token.Tag{ .Invalid, .Identifier });
	1911	testTokenize("0b0_1_", &[_]Token.Tag{.Invalid});
	1912	testTokenize("0b1e", &[_]Token.Tag{ .Invalid, .Identifier });
	1913	testTokenize("0b1p", &[_]Token.Tag{ .Invalid, .Identifier });
	1914	testTokenize("0b1e0", &[_]Token.Tag{ .Invalid, .Identifier });
	1915	testTokenize("0b1p0", &[_]Token.Tag{ .Invalid, .Identifier });
	1916	testTokenize("0b1_,", &[_]Token.Tag{ .Invalid, .Comma });
1917	1917	}
1918	1918
1919	1919	test "tokenizer - number literals octal" {
1920		testTokenize("0o0", &[_]Token.Id{.IntegerLiteral});
1921		testTokenize("0o1", &[_]Token.Id{.IntegerLiteral});
1922		testTokenize("0o2", &[_]Token.Id{.IntegerLiteral});
1923		testTokenize("0o3", &[_]Token.Id{.IntegerLiteral});
1924		testTokenize("0o4", &[_]Token.Id{.IntegerLiteral});
1925		testTokenize("0o5", &[_]Token.Id{.IntegerLiteral});
1926		testTokenize("0o6", &[_]Token.Id{.IntegerLiteral});
1927		testTokenize("0o7", &[_]Token.Id{.IntegerLiteral});
1928		testTokenize("0o8", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1929		testTokenize("0o9", &[_]Token.Id{ .Invalid, .IntegerLiteral });
1930		testTokenize("0oa", &[_]Token.Id{ .Invalid, .Identifier });
1931		testTokenize("0ob", &[_]Token.Id{ .Invalid, .Identifier });
1932		testTokenize("0oc", &[_]Token.Id{ .Invalid, .Identifier });
1933		testTokenize("0od", &[_]Token.Id{ .Invalid, .Identifier });
1934		testTokenize("0oe", &[_]Token.Id{ .Invalid, .Identifier });
1935		testTokenize("0of", &[_]Token.Id{ .Invalid, .Identifier });
1936		testTokenize("0oz", &[_]Token.Id{ .Invalid, .Identifier });
1937
1938		testTokenize("0o01234567", &[_]Token.Id{.IntegerLiteral});
1939		testTokenize("0o0123_4567", &[_]Token.Id{.IntegerLiteral});
1940		testTokenize("0o01_23_45_67", &[_]Token.Id{.IntegerLiteral});
1941		testTokenize("0o0_1_2_3_4_5_6_7", &[_]Token.Id{.IntegerLiteral});
1942		testTokenize("0o7.", &[_]Token.Id{ .IntegerLiteral, .Period });
1943		testTokenize("0o7.0", &[_]Token.Id{ .IntegerLiteral, .Period, .IntegerLiteral });
1944
1945		testTokenize("0O0", &[_]Token.Id{ .Invalid, .Identifier });
1946		testTokenize("0o_", &[_]Token.Id{ .Invalid, .Identifier });
1947		testTokenize("0o_0", &[_]Token.Id{ .Invalid, .Identifier });
1948		testTokenize("0o1_", &[_]Token.Id{.Invalid});
1949		testTokenize("0o0__1", &[_]Token.Id{ .Invalid, .Identifier });
1950		testTokenize("0o0_1_", &[_]Token.Id{.Invalid});
1951		testTokenize("0o1e", &[_]Token.Id{ .Invalid, .Identifier });
1952		testTokenize("0o1p", &[_]Token.Id{ .Invalid, .Identifier });
1953		testTokenize("0o1e0", &[_]Token.Id{ .Invalid, .Identifier });
1954		testTokenize("0o1p0", &[_]Token.Id{ .Invalid, .Identifier });
1955		testTokenize("0o_,", &[_]Token.Id{ .Invalid, .Identifier, .Comma });
	1920	testTokenize("0o0", &[_]Token.Tag{.IntegerLiteral});
	1921	testTokenize("0o1", &[_]Token.Tag{.IntegerLiteral});
	1922	testTokenize("0o2", &[_]Token.Tag{.IntegerLiteral});
	1923	testTokenize("0o3", &[_]Token.Tag{.IntegerLiteral});
	1924	testTokenize("0o4", &[_]Token.Tag{.IntegerLiteral});
	1925	testTokenize("0o5", &[_]Token.Tag{.IntegerLiteral});
	1926	testTokenize("0o6", &[_]Token.Tag{.IntegerLiteral});
	1927	testTokenize("0o7", &[_]Token.Tag{.IntegerLiteral});
	1928	testTokenize("0o8", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1929	testTokenize("0o9", &[_]Token.Tag{ .Invalid, .IntegerLiteral });
	1930	testTokenize("0oa", &[_]Token.Tag{ .Invalid, .Identifier });
	1931	testTokenize("0ob", &[_]Token.Tag{ .Invalid, .Identifier });
	1932	testTokenize("0oc", &[_]Token.Tag{ .Invalid, .Identifier });
	1933	testTokenize("0od", &[_]Token.Tag{ .Invalid, .Identifier });
	1934	testTokenize("0oe", &[_]Token.Tag{ .Invalid, .Identifier });
	1935	testTokenize("0of", &[_]Token.Tag{ .Invalid, .Identifier });
	1936	testTokenize("0oz", &[_]Token.Tag{ .Invalid, .Identifier });
	1937
	1938	testTokenize("0o01234567", &[_]Token.Tag{.IntegerLiteral});
	1939	testTokenize("0o0123_4567", &[_]Token.Tag{.IntegerLiteral});
	1940	testTokenize("0o01_23_45_67", &[_]Token.Tag{.IntegerLiteral});
	1941	testTokenize("0o0_1_2_3_4_5_6_7", &[_]Token.Tag{.IntegerLiteral});
	1942	testTokenize("0o7.", &[_]Token.Tag{ .IntegerLiteral, .Period });
	1943	testTokenize("0o7.0", &[_]Token.Tag{ .IntegerLiteral, .Period, .IntegerLiteral });
	1944
	1945	testTokenize("0O0", &[_]Token.Tag{ .Invalid, .Identifier });
	1946	testTokenize("0o_", &[_]Token.Tag{ .Invalid, .Identifier });
	1947	testTokenize("0o_0", &[_]Token.Tag{ .Invalid, .Identifier });
	1948	testTokenize("0o1_", &[_]Token.Tag{.Invalid});
	1949	testTokenize("0o0__1", &[_]Token.Tag{ .Invalid, .Identifier });
	1950	testTokenize("0o0_1_", &[_]Token.Tag{.Invalid});
	1951	testTokenize("0o1e", &[_]Token.Tag{ .Invalid, .Identifier });
	1952	testTokenize("0o1p", &[_]Token.Tag{ .Invalid, .Identifier });
	1953	testTokenize("0o1e0", &[_]Token.Tag{ .Invalid, .Identifier });
	1954	testTokenize("0o1p0", &[_]Token.Tag{ .Invalid, .Identifier });
	1955	testTokenize("0o_,", &[_]Token.Tag{ .Invalid, .Identifier, .Comma });
1956	1956	}
1957	1957
1958	1958	test "tokenizer - number literals hexadeciaml" {
1959		testTokenize("0x0", &[_]Token.Id{.IntegerLiteral});
1960		testTokenize("0x1", &[_]Token.Id{.IntegerLiteral});
1961		testTokenize("0x2", &[_]Token.Id{.IntegerLiteral});
1962		testTokenize("0x3", &[_]Token.Id{.IntegerLiteral});
1963		testTokenize("0x4", &[_]Token.Id{.IntegerLiteral});
1964		testTokenize("0x5", &[_]Token.Id{.IntegerLiteral});
1965		testTokenize("0x6", &[_]Token.Id{.IntegerLiteral});
1966		testTokenize("0x7", &[_]Token.Id{.IntegerLiteral});
1967		testTokenize("0x8", &[_]Token.Id{.IntegerLiteral});
1968		testTokenize("0x9", &[_]Token.Id{.IntegerLiteral});
1969		testTokenize("0xa", &[_]Token.Id{.IntegerLiteral});
1970		testTokenize("0xb", &[_]Token.Id{.IntegerLiteral});
1971		testTokenize("0xc", &[_]Token.Id{.IntegerLiteral});
1972		testTokenize("0xd", &[_]Token.Id{.IntegerLiteral});
1973		testTokenize("0xe", &[_]Token.Id{.IntegerLiteral});
1974		testTokenize("0xf", &[_]Token.Id{.IntegerLiteral});
1975		testTokenize("0xA", &[_]Token.Id{.IntegerLiteral});
1976		testTokenize("0xB", &[_]Token.Id{.IntegerLiteral});
1977		testTokenize("0xC", &[_]Token.Id{.IntegerLiteral});
1978		testTokenize("0xD", &[_]Token.Id{.IntegerLiteral});
1979		testTokenize("0xE", &[_]Token.Id{.IntegerLiteral});
1980		testTokenize("0xF", &[_]Token.Id{.IntegerLiteral});
1981		testTokenize("0x0z", &[_]Token.Id{ .Invalid, .Identifier });
1982		testTokenize("0xz", &[_]Token.Id{ .Invalid, .Identifier });
1983
1984		testTokenize("0x0123456789ABCDEF", &[_]Token.Id{.IntegerLiteral});
1985		testTokenize("0x0123_4567_89AB_CDEF", &[_]Token.Id{.IntegerLiteral});
1986		testTokenize("0x01_23_45_67_89AB_CDE_F", &[_]Token.Id{.IntegerLiteral});
1987		testTokenize("0x0_1_2_3_4_5_6_7_8_9_A_B_C_D_E_F", &[_]Token.Id{.IntegerLiteral});
1988
1989		testTokenize("0X0", &[_]Token.Id{ .Invalid, .Identifier });
1990		testTokenize("0x_", &[_]Token.Id{ .Invalid, .Identifier });
1991		testTokenize("0x_1", &[_]Token.Id{ .Invalid, .Identifier });
1992		testTokenize("0x1_", &[_]Token.Id{.Invalid});
1993		testTokenize("0x0__1", &[_]Token.Id{ .Invalid, .Identifier });
1994		testTokenize("0x0_1_", &[_]Token.Id{.Invalid});
1995		testTokenize("0x_,", &[_]Token.Id{ .Invalid, .Identifier, .Comma });
1996
1997		testTokenize("0x1.", &[_]Token.Id{.FloatLiteral});
1998		testTokenize("0x1.0", &[_]Token.Id{.FloatLiteral});
1999		testTokenize("0xF.", &[_]Token.Id{.FloatLiteral});
2000		testTokenize("0xF.0", &[_]Token.Id{.FloatLiteral});
2001		testTokenize("0xF.F", &[_]Token.Id{.FloatLiteral});
2002		testTokenize("0xF.Fp0", &[_]Token.Id{.FloatLiteral});
2003		testTokenize("0xF.FP0", &[_]Token.Id{.FloatLiteral});
2004		testTokenize("0x1p0", &[_]Token.Id{.FloatLiteral});
2005		testTokenize("0xfp0", &[_]Token.Id{.FloatLiteral});
2006		testTokenize("0x1.+0xF.", &[_]Token.Id{ .FloatLiteral, .Plus, .FloatLiteral });
2007
2008		testTokenize("0x0123456.789ABCDEF", &[_]Token.Id{.FloatLiteral});
2009		testTokenize("0x0_123_456.789_ABC_DEF", &[_]Token.Id{.FloatLiteral});
2010		testTokenize("0x0_1_2_3_4_5_6.7_8_9_A_B_C_D_E_F", &[_]Token.Id{.FloatLiteral});
2011		testTokenize("0x0p0", &[_]Token.Id{.FloatLiteral});
2012		testTokenize("0x0.0p0", &[_]Token.Id{.FloatLiteral});
2013		testTokenize("0xff.ffp10", &[_]Token.Id{.FloatLiteral});
2014		testTokenize("0xff.ffP10", &[_]Token.Id{.FloatLiteral});
2015		testTokenize("0xff.p10", &[_]Token.Id{.FloatLiteral});
2016		testTokenize("0xffp10", &[_]Token.Id{.FloatLiteral});
2017		testTokenize("0xff_ff.ff_ffp1_0_0_0", &[_]Token.Id{.FloatLiteral});
2018		testTokenize("0xf_f_f_f.f_f_f_fp+1_000", &[_]Token.Id{.FloatLiteral});
2019		testTokenize("0xf_f_f_f.f_f_f_fp-1_00_0", &[_]Token.Id{.FloatLiteral});
2020
2021		testTokenize("0x1e", &[_]Token.Id{.IntegerLiteral});
2022		testTokenize("0x1e0", &[_]Token.Id{.IntegerLiteral});
2023		testTokenize("0x1p", &[_]Token.Id{.Invalid});
2024		testTokenize("0xfp0z1", &[_]Token.Id{ .Invalid, .Identifier });
2025		testTokenize("0xff.ffpff", &[_]Token.Id{ .Invalid, .Identifier });
2026		testTokenize("0x0.p", &[_]Token.Id{.Invalid});
2027		testTokenize("0x0.z", &[_]Token.Id{ .Invalid, .Identifier });
2028		testTokenize("0x0._", &[_]Token.Id{ .Invalid, .Identifier });
2029		testTokenize("0x0_.0", &[_]Token.Id{ .Invalid, .Period, .IntegerLiteral });
2030		testTokenize("0x0_.0.0", &[_]Token.Id{ .Invalid, .Period, .FloatLiteral });
2031		testTokenize("0x0._0", &[_]Token.Id{ .Invalid, .Identifier });
2032		testTokenize("0x0.0_", &[_]Token.Id{.Invalid});
2033		testTokenize("0x0_p0", &[_]Token.Id{ .Invalid, .Identifier });
2034		testTokenize("0x0_.p0", &[_]Token.Id{ .Invalid, .Period, .Identifier });
2035		testTokenize("0x0._p0", &[_]Token.Id{ .Invalid, .Identifier });
2036		testTokenize("0x0.0_p0", &[_]Token.Id{ .Invalid, .Identifier });
2037		testTokenize("0x0._0p0", &[_]Token.Id{ .Invalid, .Identifier });
2038		testTokenize("0x0.0p_0", &[_]Token.Id{ .Invalid, .Identifier });
2039		testTokenize("0x0.0p+_0", &[_]Token.Id{ .Invalid, .Identifier });
2040		testTokenize("0x0.0p-_0", &[_]Token.Id{ .Invalid, .Identifier });
2041		testTokenize("0x0.0p0_", &[_]Token.Id{ .Invalid, .Eof });
	1959	testTokenize("0x0", &[_]Token.Tag{.IntegerLiteral});
	1960	testTokenize("0x1", &[_]Token.Tag{.IntegerLiteral});
	1961	testTokenize("0x2", &[_]Token.Tag{.IntegerLiteral});
	1962	testTokenize("0x3", &[_]Token.Tag{.IntegerLiteral});
	1963	testTokenize("0x4", &[_]Token.Tag{.IntegerLiteral});
	1964	testTokenize("0x5", &[_]Token.Tag{.IntegerLiteral});
	1965	testTokenize("0x6", &[_]Token.Tag{.IntegerLiteral});
	1966	testTokenize("0x7", &[_]Token.Tag{.IntegerLiteral});
	1967	testTokenize("0x8", &[_]Token.Tag{.IntegerLiteral});
	1968	testTokenize("0x9", &[_]Token.Tag{.IntegerLiteral});
	1969	testTokenize("0xa", &[_]Token.Tag{.IntegerLiteral});
	1970	testTokenize("0xb", &[_]Token.Tag{.IntegerLiteral});
	1971	testTokenize("0xc", &[_]Token.Tag{.IntegerLiteral});
	1972	testTokenize("0xd", &[_]Token.Tag{.IntegerLiteral});
	1973	testTokenize("0xe", &[_]Token.Tag{.IntegerLiteral});
	1974	testTokenize("0xf", &[_]Token.Tag{.IntegerLiteral});
	1975	testTokenize("0xA", &[_]Token.Tag{.IntegerLiteral});
	1976	testTokenize("0xB", &[_]Token.Tag{.IntegerLiteral});
	1977	testTokenize("0xC", &[_]Token.Tag{.IntegerLiteral});
	1978	testTokenize("0xD", &[_]Token.Tag{.IntegerLiteral});
	1979	testTokenize("0xE", &[_]Token.Tag{.IntegerLiteral});
	1980	testTokenize("0xF", &[_]Token.Tag{.IntegerLiteral});
	1981	testTokenize("0x0z", &[_]Token.Tag{ .Invalid, .Identifier });
	1982	testTokenize("0xz", &[_]Token.Tag{ .Invalid, .Identifier });
	1983
	1984	testTokenize("0x0123456789ABCDEF", &[_]Token.Tag{.IntegerLiteral});
	1985	testTokenize("0x0123_4567_89AB_CDEF", &[_]Token.Tag{.IntegerLiteral});
	1986	testTokenize("0x01_23_45_67_89AB_CDE_F", &[_]Token.Tag{.IntegerLiteral});
	1987	testTokenize("0x0_1_2_3_4_5_6_7_8_9_A_B_C_D_E_F", &[_]Token.Tag{.IntegerLiteral});
	1988
	1989	testTokenize("0X0", &[_]Token.Tag{ .Invalid, .Identifier });
	1990	testTokenize("0x_", &[_]Token.Tag{ .Invalid, .Identifier });
	1991	testTokenize("0x_1", &[_]Token.Tag{ .Invalid, .Identifier });
	1992	testTokenize("0x1_", &[_]Token.Tag{.Invalid});
	1993	testTokenize("0x0__1", &[_]Token.Tag{ .Invalid, .Identifier });
	1994	testTokenize("0x0_1_", &[_]Token.Tag{.Invalid});
	1995	testTokenize("0x_,", &[_]Token.Tag{ .Invalid, .Identifier, .Comma });
	1996
	1997	testTokenize("0x1.", &[_]Token.Tag{.FloatLiteral});
	1998	testTokenize("0x1.0", &[_]Token.Tag{.FloatLiteral});
	1999	testTokenize("0xF.", &[_]Token.Tag{.FloatLiteral});
	2000	testTokenize("0xF.0", &[_]Token.Tag{.FloatLiteral});
	2001	testTokenize("0xF.F", &[_]Token.Tag{.FloatLiteral});
	2002	testTokenize("0xF.Fp0", &[_]Token.Tag{.FloatLiteral});
	2003	testTokenize("0xF.FP0", &[_]Token.Tag{.FloatLiteral});
	2004	testTokenize("0x1p0", &[_]Token.Tag{.FloatLiteral});
	2005	testTokenize("0xfp0", &[_]Token.Tag{.FloatLiteral});
	2006	testTokenize("0x1.+0xF.", &[_]Token.Tag{ .FloatLiteral, .Plus, .FloatLiteral });
	2007
	2008	testTokenize("0x0123456.789ABCDEF", &[_]Token.Tag{.FloatLiteral});
	2009	testTokenize("0x0_123_456.789_ABC_DEF", &[_]Token.Tag{.FloatLiteral});
	2010	testTokenize("0x0_1_2_3_4_5_6.7_8_9_A_B_C_D_E_F", &[_]Token.Tag{.FloatLiteral});
	2011	testTokenize("0x0p0", &[_]Token.Tag{.FloatLiteral});
	2012	testTokenize("0x0.0p0", &[_]Token.Tag{.FloatLiteral});
	2013	testTokenize("0xff.ffp10", &[_]Token.Tag{.FloatLiteral});
	2014	testTokenize("0xff.ffP10", &[_]Token.Tag{.FloatLiteral});
	2015	testTokenize("0xff.p10", &[_]Token.Tag{.FloatLiteral});
	2016	testTokenize("0xffp10", &[_]Token.Tag{.FloatLiteral});
	2017	testTokenize("0xff_ff.ff_ffp1_0_0_0", &[_]Token.Tag{.FloatLiteral});
	2018	testTokenize("0xf_f_f_f.f_f_f_fp+1_000", &[_]Token.Tag{.FloatLiteral});
	2019	testTokenize("0xf_f_f_f.f_f_f_fp-1_00_0", &[_]Token.Tag{.FloatLiteral});
	2020
	2021	testTokenize("0x1e", &[_]Token.Tag{.IntegerLiteral});
	2022	testTokenize("0x1e0", &[_]Token.Tag{.IntegerLiteral});
	2023	testTokenize("0x1p", &[_]Token.Tag{.Invalid});
	2024	testTokenize("0xfp0z1", &[_]Token.Tag{ .Invalid, .Identifier });
	2025	testTokenize("0xff.ffpff", &[_]Token.Tag{ .Invalid, .Identifier });
	2026	testTokenize("0x0.p", &[_]Token.Tag{.Invalid});
	2027	testTokenize("0x0.z", &[_]Token.Tag{ .Invalid, .Identifier });
	2028	testTokenize("0x0._", &[_]Token.Tag{ .Invalid, .Identifier });
	2029	testTokenize("0x0_.0", &[_]Token.Tag{ .Invalid, .Period, .IntegerLiteral });
	2030	testTokenize("0x0_.0.0", &[_]Token.Tag{ .Invalid, .Period, .FloatLiteral });
	2031	testTokenize("0x0._0", &[_]Token.Tag{ .Invalid, .Identifier });
	2032	testTokenize("0x0.0_", &[_]Token.Tag{.Invalid});
	2033	testTokenize("0x0_p0", &[_]Token.Tag{ .Invalid, .Identifier });
	2034	testTokenize("0x0_.p0", &[_]Token.Tag{ .Invalid, .Period, .Identifier });
	2035	testTokenize("0x0._p0", &[_]Token.Tag{ .Invalid, .Identifier });
	2036	testTokenize("0x0.0_p0", &[_]Token.Tag{ .Invalid, .Identifier });
	2037	testTokenize("0x0._0p0", &[_]Token.Tag{ .Invalid, .Identifier });
	2038	testTokenize("0x0.0p_0", &[_]Token.Tag{ .Invalid, .Identifier });
	2039	testTokenize("0x0.0p+_0", &[_]Token.Tag{ .Invalid, .Identifier });
	2040	testTokenize("0x0.0p-_0", &[_]Token.Tag{ .Invalid, .Identifier });
	2041	testTokenize("0x0.0p0_", &[_]Token.Tag{ .Invalid, .Eof });
2042	2042	}
2043	2043
2044		fn testTokenize(source: []const u8, expected_tokens: []const Token.Id) void {
	2044	fn testTokenize(source: []const u8, expected_tokens: []const Token.Tag) void {
2045	2045	var tokenizer = Tokenizer.init(source);
2046	2046	for (expected_tokens) \|expected_token_id\| {
2047	2047	const token = tokenizer.next();
2048		if (token.id != expected_token_id) {
2049		std.debug.panic("expected {s}, found {s}\n", .{ @tagName(expected_token_id), @tagName(token.id) });
	2048	if (token.tag != expected_token_id) {
	2049	std.debug.panic("expected {s}, found {s}\n", .{ @tagName(expected_token_id), @tagName(token.tag) });
2050	2050	}
2051	2051	}
2052	2052	const last_token = tokenizer.next();
2053		std.testing.expect(last_token.id == .Eof);
	2053	std.testing.expect(last_token.tag == .Eof);
2054	2054	}