Corruption of MicroPython file system on micro:bit V2 (with custom MicroPython build)

[kevinjwalters]

I compiled up a version of MicroPython based on the master branch a few months back with some of my bug fixes and with the file system relocated and enlarged (0x70000 to 0x78000). This version appears to work fine and I've actually done some testing with it running for almost 2 weeks. But if I do more hand a handful of extra file operations like you'd do for testing and developing new code in Thonny (4.1.7) then the file system becomes corrupt visible as errors when running the python code. After things have got borked using os.stat shows many files with incorrect file system length and use of open/readline shows the files are corrupt with a mish-mash of other files. Thonny reports no errors while copying files. If the micro:bit is power cycled then a connection can be made to the serial console but it (REPL) will not respond. I've got about 10+ files in there.

Are there any normal file operations that can corrupt the MicroPython file system in flash? How likely is it that I've picked up a bug that corrupts the flash where the file system is?

Copying the MicroPython hex file and copying the .py files back is my cumbersome workaround.

[scruss]

This sounds like the sort of thing that happens when you make a filesystem too large for the flash that contains it. Are you sure that you can make it that large safely?

[kevinjwalters]

I had a look at the source code and this jumps out, comment is useful here for highlighting limit.

/** Maximum number of chunks allowed in filesystem. 252 chunks is 31.5kB */
#define MAX_CHUNKS_IN_FILE_SYSTEM 252

I'll go throught it in more detail to understand the limitation. I've spent most of my time with CircuitPython where it's common to have "large" file systems for CIRCUITPY.

I added a note to microbit-foundation/micropython-microbit-v2#226 (comment)

[kevinjwalters]

That could be it, I'm not sure. Here are my current guesses.

1. I didn't enlarge it properly. This would normally be top of my list but based on a previous change it looks like only a filesystem.ld change is required.
2. My new code has bugs in which is trashing the flash. I'm not sure if this is possible and the code can run for 10+ days without problems so this seems unlikely. I will give it a review in near future.
3. Something else uses that region of flash or there's some special case for that region on the nrf52/nRF52833.
4. Latent MicroPython bug with large filenames / number of files / filesizes.
5. I picked up a bug that's not in current production version by using master branch.
6. I do normally copy a zero byte (length) file across - I'm starting to become suspicious of this and will do some testing without it. (Without it the problem still occurs).
7. Bug in Thonny (seems very unlikely and Thonny normally reports errors). Thonny is the only thing writing to MicroPython filesystem, the application does not do any writes.
8. I compiled it "incorrectly" (seems unlikely, almost all of it controlled by build files).

It was 32k briefly in the past but I don't know if that version ever got used significantly.

I've just noticed microbit-foundation/micropython-microbit-v2#105 which I need to read through more carefully.

[kevinjwalters]

This is an example of what happens. The working application with filesystem view provided by running some code in REPL.

>>> for filename in os.listdir():
    print("{:23s} {:d}".format(filename, os.stat(filename)[6]))

zc_utils.py             1558
config.py               93
zhh-D.txt               0
datecalc.py             1304
main.py                 5931
mcp7940_tiny.py         1735
tzinfo.py               3045
zc_bg.py                1810
zc_bg_blank.py          118
zc_bg_pendulum.py       750
zc_bg_temperature.py    753
zc_clockcomms.py        1511
zc_comboclock.py        3284

good

I updated zc_bg_temperature.py a few times. Then updated main.py once and it now errors where it should not.

MPY: soft reboot
Traceback (most recent call last):
  File "main.py", line 13
IndentationError: unexpected indent

>>> import os
>>> for filename in os.listdir():
    print("{:23s} {:d}".format(filename, os.stat(filename)[6]))

zc_utils.py             172
zc_bg_temperature.py    304
main.py                 498
config.py               93
zhh-D.txt               0
datecalc.py             422
mcp7940_tiny.py         349
tzinfo.py               273
zc_bg.py                550
zc_bg_blank.py          370
zc_bg_pendulum.py       624

The files have

1. all got smaller,
2. there's corruption causing errors like the exception shown above and
3. and two have disappeared!

The config.py and zhh-D.txt files are the only ones that appear unaffected, inspecting the contents confirms the configuration file is okay.

>>> idx = 1 ; fh = open("config.py")
>>> while line:=fh.readline():
    print(idx, line)
    idx += 1
    
1 THIS_MICROBIT_CLOCK_PPM=+16.60

2 THIS_ZHH_RTC_PPM=-0.41

3 THIS_CLOCK_NUMBER=1

4 TZ="Europe/London"

[kevinjwalters]

My latest guess on this is it's caused by leftover bits of a previous file system in flash and that's possible because the MICROBIT.hex files produced when you compile MicroPython don't have a file system image in, unlike what comes out of https://python.microbit.org/

I created a ticket for this: microbit-foundation/micropython-microbit-v2#231