Scan for test helpers causes error when non-accessible folders present

[maritz]

Description

When there is a directory in your project that is unaccessible by the user runnig the tests, ava runs into an error during startup and exits.

In my case this is a volume created by the postgresql docker image and it resets the directory permissions on every start.

The problem is that the defaultHelperPatterns are set in a way that tries to read all directories and cannot be changed afaict.

Error Message & Stack Trace

glob error { Error: EACCES: permission denied, scandir '/home/maritz/{redacted}/db-volume/test'
  errno: -13,
  code: 'EACCES',
  syscall: 'scandir',
  path: '/home/maritz/{redacted}/db-volume/test' }
/home/maritz/{redacted}/node_modules/ava/lib/cli.js:206
					throw err;
					^

Error: EACCES: permission denied, scandir '/home/maritz/{redacted}/db-volume/__tests__/helpers'
    at handlePaths (/home/maritz/{redacted}node_modules/ava/lib/ava-files.js:14:18)
    at AvaFiles.findTestHelpers (/home/maritz/{redacted}/node_modules/ava/lib/ava-files.js:140:10)
    at Api._precompileHelpers (/home/maritz/{redacted}node_modules/ava/api.js:140:5)
    at _setupPrecompiler.then (/home/maritz/{redacted}/node_modules/ava/api.js:167:21)
    at <anonymous>

Making it work locally

When going manually into

ava/lib/ava-files.js

Line 90 in a84c96f

const defaultHelperPatterns = () => [

and changing the defaultHelperPatterns to only scan my src folder it works fine.

const defaultHelperPatterns = () => [
  'src/**/__tests__/helpers/**/*.js',
  'src/**/__tests__/**/_*.js',
  'src/**/test/helpers/**/*.js',
  'src/**/test/**/_*.js',
];

Fix options?

I see two ways to properly fix this (aside from removing inaccessible directories):

1. Add an option to change the defaultHelperPattern (related to Too slow in large projects #1418 (comment))
   or
2. Catch EACCES errors and print out a warning and continue

[novemberborn]

Sorry you're running into this @maritz. We do need to change how we select files but it's a larger change. I haven't been able to focus on that part of the project unfortunately.

Ignoring EACCESS errors isn't great either, as it means one day helpers work, and then the next they won't, and you wouldn't know why.

I've done a lot of thinking on this previously, but then didn't get round to implementing anything. Regardless of where we end up eventually, I think it should be possible to disable helper compilation altogether. That is something we could add now. What do you think?

Setting "helpers": false in AVA's configuration could do it. We can later expand that to take custom patterns.

[maritz]

It would fix my specific issue, for sure. Not sure it's the best long-term solution for everyone though.

I am curious though: Why would printing an error and ignoring potentially cause breakage in your opinion?

[novemberborn]

Not sure it's the best long-term solution for everyone though.

It's not, but it's a feature we'll end up having anyway, and it fixes some problems now.

Why would printing an error and ignoring potentially cause breakage in your opinion?

I assume the error will mean we don't find any helper files. So one day AVA compiles your helpers, and the next it doesn't, because there's a directory it can't traverse. Know you're left wondering why helper compilation stopped working.

[maritz]

you're left wondering why helper compilation stopped working.

Well, it would log out a warning about the inaccessible directories. But if the user makes the directory that contains his helpers inaccessible (or something else does), it's not gonna work anymore anyways, right?!

[novemberborn]

We only get all glob results or an error, so we can't warn. Long term we'd want to exit with an error and have users exclude those directories from the helpers. For now let's ship a way to disable helper compilation.

[maritz]

Oh. that makes sense. 👍