WasmFS: Minimal runtime + closure support

[kripken]

[kripken]

This fails on closure erroring on things like addUniqueRunDependency. This problem is not unique to WasmFS, as it can be shown without WasmFS like this:

./emcc tests/hello_libcxx.cpp -O3 --closure 1 -s MINIMAL_RUNTIME

Including enough filesystem code in JS is enough to show the problem, it seems, with or without WasmFS. But with WasmFS we include more JS filesystem code by default atm, which is why I seem to be seeing it more locally.

[Kagami]

Not sure if I need to create separate issue. --closure 1 fails with WASMFS for me without anything special like MINIMAL_RUNTIME. Consider this example:

#include <stdio.h>

#ifdef __EMSCRIPTEN__
#include <emscripten.h>
#else
#define EM_ASM(...)
#endif

int main(void) {
  EM_ASM(
    FS_writeFile("test.txt", "123");
  );
  return 0;
}

emcc -o main.js main.c -s WASMFS=1 -s 'EXPORTED_RUNTIME_METHODS=["FS"]' --closure 1

ERROR - [JSC_UNDEFINED_VARIABLE] variable wasmFS$backends is undeclared
  1324|       assert(wasmFS$backends[backend]);
                     ^^^^^^^^^^^^^^^

1 error(s), 0 warning(s)

It doesn't like the undeclared use of wasmFS$backends in JS code. This can be fixed by adding to pre.js:

var wasmFS$backends;

[kripken]

@Kagami Hmm, that doesn't work even without -sWASMFS. The issue I think is that you export FS but call FS_writeFile which is unrelated (even if it starts with FS).

Try to use FS.writeFile instead, the method on the FS object. Also do -sFORCE_FILESYSTEM which includes the JS FS API, including FS.writeFile. With those two changes that compiles for me.

We should have better errors for this, however. In particular perhaps we should handle -sEXPORTED_RUNTIME_METHODS=FS as the same as -sFORCE_FILESYSTEM perhaps?

[Kagami]

that doesn't work even without -sWASMFS

For some reason it's FS.writeFile in default FS. But you don't need to force the full filesystem, few methods are exported when you just require FS export.

-sFORCE_FILESYSTEM which includes the JS FS API, including FS.writeFile

Oh yeah, that works.

-sEXPORTED_RUNTIME_METHODS=FS as the same as -sFORCE_FILESYSTEM perhaps

The problem is that I just need the FS_createDataFile, FS_writeFile functions in pre.js.

If you force filesystem, it drastically inflates the size of the runtime:

emcc -O3 -o main.js main.c -s WASMFS=1 -s 'EXPORTED_RUNTIME_METHODS=["FS"]' --closure 1 --pre-js pre.js
# 9.7K
emcc -O3 -o main.js main.c -s WASMFS=1 -s 'EXPORTED_RUNTIME_METHODS=["FS"]' -s FORCE_FILESYSTEM=1 --closure 1
# 15K

So I'm better with var wasmFS$backends; workaround than with the increased build size 😄

However, when I omit -s 'EXPORTED_RUNTIME_METHODS=["FS"]', but keep -s FORCE_FILESYSTEM=1, the build size is 10K, about the same as without forcing FS, and FS.writeFile still works. Strange.

What I really need is -s 'EXPORTED_RUNTIME_METHODS=["FS_writeFile"]', but right now it also produces many closure errors.

Test program I'm using to check the output:

#include <stdio.h>

#ifdef __EMSCRIPTEN__
#include <emscripten.h>
#else
#define EM_ASM(...)
#endif

int main(void) {
  EM_ASM(
    FS.writeFile("test.txt", "123\n");
  );

  FILE *file = fopen("test.txt", "r");
  char buffer[100];
  size_t bytesRead = fread(buffer, 1, sizeof(buffer), file);
  fwrite(buffer, 1, bytesRead, stdout);
  fclose(file);

  return 0;
}

[Kagami]

What I really need is -s 'EXPORTED_RUNTIME_METHODS=["FS_writeFile"]'

One correction, I don't need those methods to be exported on Module object, but only inside the pre.js where I have the same namespace as the rest of the runtime.

So perhaps just -s FORCE_FILESYSTEM=1 is the correct way? Just not sure it will always eliminate functions which I don't use.

Seems like you can't just ask for functions like FS_writeFile inside runtime, without Module exports.

[kripken]

In general, if you want small code size then using the FS from C/C++ is best. Then no JS support code at all may end up in the final build, and wasm's DCE is very effective. For that reason we haven't tried to optimize JS code size in WasmFS much, and so FORCE_FILESYSTEM includes most of the JS support code all at once.

Some methods like FS_writeFile are used by the file packager, so they are considered part of the runtime, and have special handling. But those aren't publicly documented APIs and might change, so I wouldn't recommend using them. Instead, either use FORCE_FILESYSTEM if you want JS convenience and are ok with the JS size, or use the FS from wasm otherwise, those are the best-supported paths atm. If someone spent time to optimize JS code size that would be welcome, however, but I'm not aware of anyone doing so now.

[Kagami]

So if I need to have some file written on WebAssembly side, you would recommend to create C function like int my_write_file() with fwrite calls, and use it in pre.js via -s 'EXPORTED_RUNTIME_METHODS=["my_write_file"]?

[sbc100]

So if I need to have some file written on WebAssembly side, you would recommend to create C function like int my_write_file() with fwrite calls, and use it in pre.js via -s 'EXPORTED_RUNTIME_METHODS=["my_write_file"]?

If the function is native you would do -sEXPORTED_FUNCTIONS=_my_write_file (note the extra leading underscore needed on the command line, and when called from JS

[Kagami]

-sEXPORTED_FUNCTIONS=_my_write_file

Got it, thanks.

One advantage of JS API is that you have FS_createDataFile which allows to preload files into WASM memory before the runtime is started though.

In my tests it made first call of the WASM function with access to that file slightly faster, compared to FS_writeFile after the runtime was started.