Image source caching strategy

[wouterlucas]

Should we cache image sources?

Related discussion:
#497 (comment)

Lightning 3 prior versions to 2.9.x will cache image source data in JS and only clear GPU Textures when a memory limit is reached. The thinking process is we could clear image source data knowing the browser would cache the image in its network layers. However there is no reliable way to determine whether this is the case or not.

The risk is that if this is not properly cached in the lower browser layers it would introduce additional delays in loading images as we would need to fetch the entire image source again.

By keeping this in JS this guaranteed the image source is always readily available at the cost of JS HEAP usage.

Options:

• Make the option to cache image source data configurable, so we can turn it on for devices that might have issues with caching images at the lower layers
• Introduce a Image Source cache mechanism that will cache images up to a predefined Mb limit, before it starts pruning sources to more granular control the HEAP allocations for image source data

Also I've half-assumed this is what L2 does as well, based on a vague recollection of how the source data processing works in L2. If someone knows please speak up or otherwise I'll go down a trip of peeking at L2 code again.

Suggestions/ideas are welcome 😃

cc: @erikhaandrikman @jfboeve @elsassph

[elsassph]

As far as I can tell L2 doesn't keep images in heap memory (goes direct to GPU), fully unloads and reloads from network.

[elsassph]

I don't necessarily mean that it's better, but for instance we can have 200+ tiles on a page, not counting a hero carousel with full-screen images. Naively keeping all the images in heap would definitely kill some devices.

[wouterlucas]

Yeah I agree - but you could take this further. Why keep the source data at all once you've uploaded the image to the GPU at all? No need to keep it once the texture is created and wait for the texture to be cleaned up because it is no longer renderable. We could delete it right after successful GPU Texture creation.

Alternatively it would save a few milliseconds when reloading images, especially if we have a pool, by keeping it on the HEAP.

Few options:

A) Delete the source data straight after creation of the Texture on the GPU - though we need to be careful with creating a large GC trail.
B) Delete the source data batched, to offload but still save on HEAP
C) Only delete the source data if the texture itself is removed from the GPU
D) Keep a "pool" of source data on deck, with a monitor and limits. Having image source data on the HEAP when they need to be reloaded again.

Personally I think D is a little overkill? and might introduce extra delays, more housekeeping, complexity, etc.
Leaning towards option B, just for the sake of keeping the processing low at the most crucial moments. We could clean out source data during the processSome in between frames.

[michielvandergeest]

I agree with your reasoning and option B sounds like a good idea. We can still expand on the functionality in the future if needed with something like option D. But unless there proves to be an issue it seems like unneeded complexity at this point.

[elsassph]

I think L2 does A.

I don't think B is useful. By design, browsers will batch their memory GC runs. In fact it's often the opposite problem that, as CPU usage is high, they wait too long, until heap goes too high, and they have to run a big GC in one go - a problem webOS devices are suffering very notably.

On our end, we counter GPU textures unloading by having our own MRU texture pinning logic, which prevents Lightning to GC unused textures within a certain limit and based on how recently they were used.

[wouterlucas]

Yeah you're absolutely right - it wont matter to when it will be GC'ed. I'm just looking for ways to balance out our processing and generate an as distributed as possible load over time.

I.e. I'd rather start cleaning up stuff during idle time then during frame generation time, assuming there will be moments where we have downtime. This should tie nicely into the GC as that too will wait for idle time for a non-critical cleanup.

But agree, that builds upon the idea that there will be downtime. Even with high CPU though I'd assume there will be moments when the renderer is idle?

[elsassph]

I don't imagine that the JS work to dereference heap images would have much impact. The real work happens when the VM does its mark&sweep.

In terms of CPU you can easily overload the machines by intensively interacting (like scrolling long pages) or simply having an HTML5 player at work.

[wouterlucas]

Currently merged in #507 with option A.