Custom importer remains blocked if two imports are run in parallel

[saper]

This is from #1672 (comment) by @MartijnHols

Reproduction code is in https://github.com/EmicoEcommerce/node-sass-segfault-repro

I have traced this with a debug-enabled version of node-sass binding on FreeBSD.

[saper]

When running only one render process at a time (process.env.MAX_SIMULTANEOUS_BUILDS == 1) we are getting the following execution paths:

(full log - https://gist.github.com/saper/75c8d334ada304005c9a26f3528c9ec9)

• Thread 10 (LWP 102048 of process 76788) sets a custom importer with a context structure (cookie) at 0x803193700 and wants to compile complex.scss
• Thread 8 (LWP 102046 of process 76788) sets a custom importer with a context structure (cookie) at 0x8031940a0 and wants to compile faulty.scss
• Thread 8 invokes the custom importer (0x8031940a0)
• Thread 8 exists [LWP 102046 of process 76788 exited]
• Thread 10 exists [LWP 102048 of process 76788 exited]

The process completes correctly.

Thread #	source	LWP	dctx	importer cookie
10	complex.scss	102048	0x803192a00	0x803193700
8	faulty.scss	102046	0x803192500	0x8031940a0

[saper]

When running two builds at a time (process.env.MAX_SIMULTANEOUS_BUILDS == 2) we get the following executions paths:

(full log - https://gist.github.com/saper/7c77ad6407cb24dde66dad2c39553de8)

Thread #	source	LWP	dctx	importer cookie
11	complex.scss	102063	0x803192b00	0x803193d90
9	faulty.scss	102057	0x803192900	0x803193d20
10	complex.scss	102059	0x803192a00	0x803193a10
8	faulty.scss	102054	0x803192d00	0x8031939a0

• Thread 11 starts rendering complex.scss
• Thread 9 starts rendering faulty.scss
• Thread 10 starts rendering complex.scss
• Thread 9 tries to import breakpoints.scss
• JavaScript notices failure: The expected error occurred. Exiting now which may trigger the segmentation fault error in the still running process(es).
• Thread LWP 102042 exists
• Thread 8 starts rendering faulty.scss
• Thread 8 tries to import breakpoints.scss
• Thread 9 exits
• Thread 10 exits
• Thread 11 exits
• Thread 8 remains hanging, waiting for the callback from the JavaScript importer

[saper]

Can we simplify the custom importer to some small function? How does it behave in case of an error?

[MartijnHols]

Great, thanks! Regarding hanging processes - I think this might be the architectural problem we are facing documented in #857. Can you try a workaround as documented there with environment variable and see if that helps?

Running for i in {1..100}; do echo $i; env UV_THREADPOOL_SIZE=5 yarn start; done in the reproduction repository seems to take a lot more iterations before it freezes than without the env var, but eventually it still freezes. Regular call freezes within less than 10 iterations 95% of the time, with an increased threadpool size it doesn't freeze before 15 iterations about 95% of the time, and usually goes quite far. It completed all iterations once without freeze.

Unfortunately for my actual closed source project it makes no difference and still freezes every single time apart from the occasional segmentation fault.