Whitespace in enclosed tokens

[slevithan]

Whitespace in character classes is mentioned in the proposal, but how should flag x handle whitespace in other enclosed tokens: {…}, \p{…}, \P{…}, \u{…}, [\q{…}], (?<…>), \k<…>.

Prior art is generally the following:

• Whitespace in an interval quantifier ({ …}, {… ,}, etc.) leads to { matching a literal {. Presumably this wouldn't be the case in ECMAScript with flag u or v, and I'm assuming whitespace with flag x would not be allowed.
• \p{…} and \P{…} allow insignificant whitespace (per Unicode recommendations) even without flag x. Not so in ECMAScript, which is stricter, and I'm assuming this would not be relaxed with flag x.
• \u{…} is not common in other regex flavors, but some have equivalent \x{…}, and I believe they make it an error to add whitespace even with flag x.
• [\q{…}] is unique to ECMAScript AFAIK, but I assume it would allow insignificant whitespace with flag x.
• (?<…>) and \k<…> do not allow whitespace, even with flag x.

I'm assuming things like \u {…}, ( ?:…) and (? :…) would be errors even with flag x, and that \0 0 would be equivalent to \0(?:)0.

[rbuckton]

x is generally intended to serve two purposes:

1. Ignoring whitespace that would otherwise be treated as a pattern character.
2. Allowing single-line comments in a multi-line RegExp string that start with #.

• Whitespace in an interval quantifier ({ …}, {… ,}, etc.) leads to { matching a literal {. Presumably this wouldn't be the case in ECMAScript with flag u or v, and I'm assuming whitespace with flag x would not be allowed.

For quantifiers, I think I would prefer that x has no effect on whitespace inside of the quantifier since whitespace in a quantifier isn't interpreted as a pattern character under u and v modes. I think that best aligns with the intended purpose of x.

That does mean /A{ 2}/x in Annex B would match "A{2}", though, since we would interpret the invalid quantifier as pattern characters and the whitespace would then be ignored. We could choose to issue a Syntax Error for this under x mode, even under Annex B, due to the ambiguity.

• \p{…} and \P{…} allow insignificant whitespace (per Unicode recommendations) even without flag x. Not so in ECMAScript, which is stricter, and I'm assuming this would not be relaxed with flag x.

I don't think there would be any harm in allowing whitespace in \p/\P under x, but we may want to forbid it since it doesn't really align with the intended purpose of x.

• \u{…} is not common in other regex flavors, but some have equivalent \x{…}, and I believe they make it an error to add whitespace even with flag x.

We should not allow whitespace here so that \u{…} in RegExp remains consistent with string literals and identifiers.

• [\q{…}] is unique to ECMAScript AFAIK, but I assume it would allow insignificant whitespace with flag x.

Given that whitespace inside of \q{…} is interpreted as a literal pattern character, I expect we would want to ignore it under x just as we would for other whitespace in a character class.

• (?<…>) and \k<…> do not allow whitespace, even with flag x.

Since whitespace in these cases is not interpreted as a literal pattern character, I would agree it should be disallowed.

I'm assuming things like \u {…}, ( ?:…) and (? :…) would be errors even with flag x, [...]

Following the same principals, I would expect each of these would be disallowed. ( ?:) may be a special case, though since it is allowed in Annex B. In Annex B under x mode we would interpret this as a capturing group starting with ?, which is illegal as there is no Term to quantify.

[...] and that \0 0 would be equivalent to \0(?:)0.

Yes, this is correct. Under x mode, whitespace is not stripped from the RegExp before parsing, but rather is treated as an independent token that does have any matching semantics and is not included in a CharSet for the purpose of evaluating character classes.