Add FAQ on memory fragmentation #1053
Conversation
Added FAQ section addressing memory leak concerns, including explanations and recommendations.
jlizen
left a comment
jlizen
left a comment
There was a problem hiding this comment.
Great addition, thanks for writing this!
Looks like it didn't actually. merge, FYI, since our permissions were scrambled in the github org move. Might be worth reopening to not lose track.
|
|
||
| <img width="2036" height="1177" alt="image" src="https://github.com/user-attachments/assets/778e2857-8c1a-49f4-968b-47739f445858" /> | ||
|
|
||
| **What can I do about it?** |
There was a problem hiding this comment.
Is it worth talking about optimizing serde to reduce allocations via eg #[serde(borrow)] and Cow<'json, str> and other micro optimizations? This is a known issue with serde that you can work around somewhat.
An article is actually currently making the rounds about similar workflows: https://sander.saares.eu/2025/10/15/memory-allocation-is-the-root-of-all-evil-part-2-rust/
|
|
||
| **What can I do about it?** | ||
|
|
||
| You can experiment with different allocators like `jemalloc`, along with different environment flags. For example setting a single arena for `glibc` via `MALLOC_ARENA_MAX=1`. In `jemalloc` enabling a background thread with a short decay for dirty memory sweeps will induce more frequent sweeps. |
There was a problem hiding this comment.
I generally recommend mimalloc instead these days, since jemalloc is unmaintained and mimalloc has been showing better results in many scenarios anyway. Though we could mention both too.
|
|
||
| We recommend that you don’t mess with your allocator settings until you start seeing degradation / getting closer to max memory for your function. | ||
|
|
||
| Finally, we note that sandboxes have a finite lifespan, and will recycle over time and as needed, thus ameliorating the accumulated fragmentation. |
There was a problem hiding this comment.
nit: ameliorating -> garbage collecting maybe?
|
|
||
| The metric `used_memory_max` increases because the allocator keeps requesting more memory pages from the operating system because memory gets fragmented and we end up with lots of small objects allocated in non-contiguous blocks. This means that we may have sufficient free memory to handle the request, but we don’t necessarily have enough contiguous memory marked as “safely deallocated” to be able to reuse it. | ||
|
|
||
| The culprit of these allocations according to our investigation is serde::deserialize |
There was a problem hiding this comment.
nit: needs a period or colon
|
As a side concern - this might be a bit much detail to go inline in the README? What do you think about a standalone .md for it and instead just a link from the main README? |
3 participants
Added FAQ section addressing memory leak concerns, including explanations and recommendations.
🔏 By submitting this pull request