Add Thread Local Storage (TLS) support

[andrewboie]

Is your enhancement proposal related to a problem? Please describe.
Right now, it is not possible for a user thread to access thread-local data without making a system call. The thread-local data itself is stored in a read/write area in the thread's stack object, but we don't have a way know its address without reading _current->userspace_local_data, which only supervisor mode can do (both knowing the value of _current and reading one of its members)

This is all a gross hack anyway. We want real TLS as GCC understand it.

Solution requirement

support for GCC thread-local storage aka __thread. We know everything at build time, so it's a matter of reserving enough memory in the stack object at thread creation to hold it all, likely using linker variables created directly or indirectly by the compiler.

Implementing this will be architecture-specific, and depend on the type/capabilities of the memory protection hardware, and/or features of the CPU. But it's generally known how to do this, on x86 a special data segment is used, and on other arches a register is reserved for the base of the TLS area.

once in place, get rid of hacks for errno on supported platforms; arch will select some Kconfig if real TLS is supported. At minimum need all userspace platforms to support this.

Further reading:
https://chao-tic.github.io/blog/2018/12/25/tls
https://www.akkadia.org/drepper/tls.pdf
https://www.linuxjournal.com/content/creating-vdso-colonels-other-chicken

[andrewboie]

A lot of the work for this was already done in #26545 which we could separate out and implement for x86 and ARC

[keith-packard]

Ok, I spent a bit of time today researching how this could work on x86. Of course, x86 is the oddest of odd-ball chips...

TLS support on x86 uses the GS segment register; TLS variables are referenced as small offsets from the base of the GS segment. That means we need to program the GS register correctly. And that means creating a suitable GDT entry, because the GS register references a descriptor table.

An approach that I think will work is to define CONFIG_MP_NUM_CPUS additional GDT entries and then select one of these for each CPU in the system, setting the GS in each CPU appropriately when the CPU is initialized. When switching threads, the TLS base address (stored in the thread structure) would then be used to compute a new GDT entry value which would be stored in the relevant GDT value using the current GS value.

The rest of the TLS code as provided in the picolibc patch should "just work", with the TLS segment getting allocated out of the thread stack and initialized by using data out of the executable.

all of this depends on using -ftls-model=local-exec, which is the simplest TLS model.

[andrewboie]

Some reading material for x86 here https://wiki.osdev.org/Thread_Local_Storage

There's some slight differences between 32-bit and 64-bit.

32-bit we create all our GDT entries at build time with the gen_gdt.py script. We can make TLS depend on CONFIG_GDT_DYNAMIC. It ends up in the data section and we should be able to update it at runtime on context switch. There is no SMP support for 32-bit so no need to worry about that.

64-bit this instead uses the %fs segment. The GDT is hard-coded in locore.S, and you can see where we create the TSS descriptors for each CPU, probably just need to augment each one with TLS descriptors (and maybe use a macro instead of repeating ourselves up to 4 times). Same deal should be fairly straightforward to update on context switch, probably where we hook into other C code at the top of __resume.

[keith-packard]

We can just re-use the DS descriptor for the TLS segment register and then adjust the base address using the MSR (for 32-bit mode) or maybe the new wrfsbase instruction in 64-bit mode. We could create a separate GDT entry in 32-bit mode to set the length field to match the TLS size if desired; that would catch stray references through gs. There's no need to mess with the GDT itself at runtime; the only thing we want to adjust is the base address.