authorgravatar for thejoshwolfe@gmail.comJosh Wolfe <thejoshwolfe@gmail.com> 2021-03-03 14:30:46-05:00
committergravatar for noreply@github.comGitHub <noreply@github.com> 2021-03-03 14:30:46-05:00
log7fbe9e7d60fa17e49177f52df922bbf295ecc619
tree2c0b79b5a1659d3700c43298bcfb5bb76ff5d67d
parent5bd9a6451685ad056628e312aadaf80e66a4c004
signaturebadge-question-mark Signed by PGP key 4AEE18F83AFDEB23

update docs and grammar to allow CRLF line endings (#8063)

1 files changed, 51 insertions(+), 23 deletions(-)

doc/langref.html.in+51-23
...@@ -10447,13 +10447,40 @@ fn readU32Be() u32 {}...@@ -10447,13 +10447,40 @@ fn readU32Be() u32 {}
10447 {#header_close#}10447 {#header_close#}
10448 {#header_open|Source Encoding#}10448 {#header_open|Source Encoding#}
10449 <p>Zig source code is encoded in UTF-8. An invalid UTF-8 byte sequence results in a compile error.</p>10449 <p>Zig source code is encoded in UTF-8. An invalid UTF-8 byte sequence results in a compile error.</p>
10450 <p>Throughout all zig source code (including in comments), some codepoints are never allowed:</p>10450 <p>Throughout all zig source code (including in comments), some code points are never allowed:</p>
10451 <ul>10451 <ul>
10452 <li>Ascii control characters, except for U+000a (LF): U+0000 - U+0009, U+000b - U+0001f, U+007f. (Note that Windows line endings (CRLF) are not allowed, and hard tabs are not allowed.)</li>10452 <li>Ascii control characters, except for U+000a (LF), U+000d (CR), and U+0009 (HT): U+0000 - U+0008, U+000b - U+000c, U+000e - U+0001f, U+007f.</li>
10453 <li>Non-Ascii Unicode line endings: U+0085 (NEL), U+2028 (LS), U+2029 (PS).</li>10453 <li>Non-Ascii Unicode line endings: U+0085 (NEL), U+2028 (LS), U+2029 (PS).</li>
10454 </ul>10454 </ul>
10455 <p>The codepoint U+000a (LF) (which is encoded as the single-byte value 0x0a) is the line terminator character. This character always terminates a line of zig source code (except possibly the last line of the file).</p>10455 <p>
10456 <p>For some discussion on the rationale behind these design decisions, see <a href="https://github.com/ziglang/zig/issues/663">issue #663</a></p>10456 LF (byte value 0x0a, code point U+000a, {#syntax#}'\n'{#endsyntax#}) is the line terminator in Zig source code.
10457 This byte value terminates every line of zig source code except the last line of the file.
10458 It is recommended that non-empty source files end with an empty line, which means the last byte would be 0x0a (LF).
10459 </p>
10460 <p>
10461 Each LF may be immediately preceded by a single CR (byte value 0x0d, code point U+000d, {#syntax#}'\r'{#endsyntax#})
10462 to form a Windows style line ending, but this is discouraged.
10463 A CR in any other context is not allowed.
10464 </p>
10465 <p>
10466 HT hard tabs (byte value 0x09, code point U+0009, {#syntax#}'\t'{#endsyntax#}) are interchangeable with
10467 SP spaces (byte value 0x20, code point U+0020, {#syntax#}' '{#endsyntax#}) as a token separator,
10468 but use of hard tabs is discouraged. See {#link|Grammar#}.
10469 </p>
10470 <p>
10471 Note that running <code>zig fmt</code> on a source file will implement all recommendations mentioned here.
10472 Note also that the stage1 compiler does <a href="https://github.com/ziglang/zig/wiki/FAQ#why-does-zig-force-me-to-use-spaces-instead-of-tabs">not yet support CR or HT</a> control characters.
10473 </p>
10474 <p>
10475 Note that a tool reading Zig source code can make assumptions if the source code is assumed to be correct Zig code.
10476 For example, when identifying the ends of lines, a tool can use a naive search such as <code>/\n/</code>,
10477 or an <a href="https://msdn.microsoft.com/en-us/library/dd409797.aspx">advanced</a>
10478 search such as <code>/\r\n?|[\n\u0085\u2028\u2029]/</code>, and in either case line endings will be correctly identified.
10479 For another example, when identifying the whitespace before the first token on a line,
10480 a tool can either use a naive search such as <code>/[ \t]/</code>,
10481 or an <a href="https://tc39.es/ecma262/#sec-characterclassescape">advanced</a> search such as <code>/\s/</code>,
10482 and in either case whitespace will be correctly identified.
10483 </p>
10457 {#header_close#}10484 {#header_close#}
1045810485
10459 {#header_open|Keyword Reference#}10486 {#header_open|Keyword Reference#}
...@@ -11373,6 +11400,7 @@ ExprList &lt;- (Expr COMMA)* Expr?...@@ -11373,6 +11400,7 @@ ExprList &lt;- (Expr COMMA)* Expr?
1137311400
11374# *** Tokens ***11401# *** Tokens ***
11375eof &lt;- !.11402eof &lt;- !.
11403eol &lt;- ('\r'? '\n') | eof
11376hex &lt;- [0-9a-fA-F]11404hex &lt;- [0-9a-fA-F]
11377hex_ &lt;- ('_'/hex)11405hex_ &lt;- ('_'/hex)
11378dec &lt;- [0-9]11406dec &lt;- [0-9]
...@@ -11382,39 +11410,39 @@ dec_int &lt;- dec (dec_* dec)?...@@ -11382,39 +11410,39 @@ dec_int &lt;- dec (dec_* dec)?
11382hex_int &lt;- hex (hex_* dec)?11410hex_int &lt;- hex (hex_* dec)?
1138311411
11384char_escape11412char_escape
11385 &lt;- &quot;\\x&quot; hex hex11413 &lt;- '\\x' hex hex
11386 / &quot;\\u{&quot; hex+ &quot;}&quot;11414 / '\\u{' hex+ '}'
11387 / &quot;\\&quot; [nr\\t'&quot;]11415 / '\\' [nr\\t'&quot;]
11388char_char11416char_char
11389 &lt;- char_escape11417 &lt;- char_escape
11390 / [^\\'\n]11418 / [^\\'\r\n]
11391string_char11419string_char
11392 &lt;- char_escape11420 &lt;- char_escape
11393 / [^\\&quot;\n]11421 / [^\\&quot;\r\n]
1139411422
11395line_comment &lt;- '//'[^\n]*11423line_comment &lt;- '//'[^\r\n]* eol
11396line_string &lt;- (&quot;\\\\&quot; [^\n]* [ \n]*)+11424line_string &lt;- ('\\\\' [^\r\n]* eol skip)+
11397skip &lt;- ([ \n] / line_comment)*11425skip &lt;- ([ \t] / eol / line_comment)*
1139811426
11399CHAR_LITERAL &lt;- &quot;'&quot; char_char &quot;'&quot; skip11427CHAR_LITERAL &lt;- &quot;'&quot; char_char &quot;'&quot; skip
11400FLOAT11428FLOAT
11401 &lt;- &quot;0x&quot; hex_* hex &quot;.&quot; hex_int ([pP] [-+]? hex_int)? skip11429 &lt;- '0x' hex_* hex '.' hex_int ([pP] [-+]? hex_int)? skip
11402 / dec_int &quot;.&quot; dec_int ([eE] [-+]? dec_int)? skip11430 / dec_int '.' dec_int ([eE] [-+]? dec_int)? skip
11403 / &quot;0x&quot; hex_* hex &quot;.&quot;? [pP] [-+]? hex_int skip11431 / '0x' hex_* hex '.'? [pP] [-+]? hex_int skip
11404 / dec_int &quot;.&quot;? [eE] [-+]? dec_int skip11432 / dec_int '.'? [eE] [-+]? dec_int skip
11405INTEGER11433INTEGER
11406 &lt;- &quot;0b&quot; [_01]* [01] skip11434 &lt;- '0b' [_01]* [01] skip
11407 / &quot;0o&quot; [_0-7]* [0-7] skip11435 / '0o' [_0-7]* [0-7] skip
11408 / &quot;0x&quot; hex_* hex skip11436 / '0x' hex_* hex skip
11409 / dec_int skip11437 / dec_int skip
11410STRINGLITERALSINGLE &lt;- &quot;\&quot;&quot; string_char* &quot;\&quot;&quot; skip11438STRINGLITERALSINGLE &lt;- '&quot;' string_char* '&quot;' skip
11411STRINGLITERAL11439STRINGLITERAL
11412 &lt;- STRINGLITERALSINGLE11440 &lt;- STRINGLITERALSINGLE
11413 / line_string skip11441 / line_string skip
11414IDENTIFIER11442IDENTIFIER
11415 &lt;- !keyword [A-Za-z_] [A-Za-z0-9_]* skip11443 &lt;- !keyword [A-Za-z_] [A-Za-z0-9_]* skip
11416 / &quot;@\&quot;&quot; string_char* &quot;\&quot;&quot; skip11444 / '@&quot;' string_char* '&quot;' skip
11417BUILTINIDENTIFIER &lt;- &quot;@&quot;[A-Za-z_][A-Za-z0-9_]* skip11445BUILTINIDENTIFIER &lt;- '@'[A-Za-z_][A-Za-z0-9_]* skip
1141811446
1141911447
11420AMPERSAND &lt;- '&amp;' ![=] skip11448AMPERSAND &lt;- '&amp;' ![=] skip