Infinite loop when compiling nested macros

[bnoordhuis]

Apologies if this has been reported before. A search didn't turn up anything but you know how GH's search is.

$ cat t.rs
fn main() { print!(format!("BAM")); }

$ rustc t.rs 2>&1 | head -20
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }
                          ^~~~~~
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }
                          ^~~~~~
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }
                          ^~~~~~
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }
                          ^~~~~~
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }
                          ^~~~~~
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }
                          ^~~~~~
t.rs:1:20: 1:26 error: macro undefined: 'format'
t.rs:1 fn main() { print!(format!("BAM")); }

Tested with today's x86_64 lucid nightly from https://launchpad.net/~hansjorg/+archive/rust

[alexcrichton]

That sounds pretty bad!

Nominating.

[pnkfelix]

Accepted for 1.0, P-high

[edwardw]

The code path seems to be:
https://github.com/mozilla/rust/blob/master/src/libsyntax/ext/format.rs#L799
https://github.com/mozilla/rust/blob/master/src/libsyntax/ext/base.rs#L302

The syntax::ext::base::ExtCtxt::expand_expr only looks for the built-in macros returned by syntax_expander_table, so it fails to recognize the second macro format! which is not a built-in one.

But how to fix it?

[sfackler]

You'd probably want to move the SyntaxEnv from MacroExpander to inside the ExtCtxt.

[edwardw]

Moving SyntaxEnv from MacroExpander to inside ExtCtxt is a great idea! I tried it but there's a practical problem. There are several important match expressions in which a instance of SyntaxEnv is borrowed immutably and ExtCtxt mutably. Moving SyntaxEnv to inside ExtCtxt would make that illegal.

In the same vein, changing MacroExpanderFn from
fn(ecx: &mut ExtCtxt, ...) -> MacResult
to
fn(ecx: &mut ExtCtxt, env: &SyntaxEnv, ...) -> MacResult
or
fn(expander: &mut MacroExpander, ...) -> MacResult
will have the same effect. But that approach involves about 170+ changes to the libsyntax.

Is there any other way to do it?

[sfackler]

I think the best thing is to probably restructure the matches to avoid the double borrow. I'm not sure how painful that'll be though. Changing the signature of MacroExpanderFn seems a bit unfortunate.

[edwardw]

I couldn't find a way to untangle the double borrow. Here's what I found so far for whoever may pick up this issue in the future.

The issue reported in this ticket is twofold. One is that unless it is one of the hardcoded macros defined in syntax::ext::base::syntax_expander_table, a nested macro will be deemed as undefined; the other is that a nested undefined macro will cause a infinite loop in the phase 2 of crate compilation.

The reason that a nested legit macro deemed as undefined is that the inner macro gets expanded in the expander function of outer macro. But the expander function has no syntax context information right now, in that context there are macros being injected from std and syntax extensions. Moving SyntaxEnv to inside ExtCtxt will do with minimum modification to the current macro API.

The #11803 seems to be remotely related.

[edwardw]

Do we plan to support nested macro invocations? #9323 is also relevant.