Skip to content

Conversation

dunkeroni
Copy link
Contributor

Summary

Client-side support for making image adjustments in the canvas raster layers. These effects are applied using konva filters. They can be kept on to automatically apply to any new edits of the layer or finalized to bake the results in with the Finish button. Can be collapsed to take up less space or temporarily disabled to compare against the original.

Generating, creating from bounding box, and saving the canvas will use the current visible state of the layer (no need to finalize). If a saved canvas is recalled with the Remix option, the original layer with any pending adjustments will appear again. Duplicating a layer creates a duplicate with identical pending adjustments if they exist. Select Object will persist any pending adjustments if the user selects Apply, but not Save As (select object uses the underlying layer data directly instead of the view).

Has two modes:
Simple: Brightness, Contrast, Saturation, Temperature, Tint, Sharpness
"Brightness" is an absolute shift applied to all pixels and causes clipping on the ends. Some users might expect that to be called "Exposure" depending on the software they are coming from. Open to suggestions if we want to swap terms.
image

Curves: Master, Red, Green, Blue
image

Right click menu to access:
image

Related Issues / Discussions

Similar to the filters exposed in #7594 but in the frontend. Backend filters can still be useful for specific colorspace actions, but most of the common use cases should be solved with these settings.

A note on performance: For large layers, the filters can take noticeable time to complete. Konva filters re-apply any time the settings are changed or when the layer has its data updated. If a layer has pending adjustments, they will recalculate when:

  • User finishes drawing a line or erasing on that layer
  • User clicks Apply on a transformation edit

Moving layers, generating new layers, or making edits to other layers does not trigger recalculation.

QA Instructions

Demo video of interactions (uses old right-click menu)
Screencast From 2025-08-13 02-22-58.webm

Merge Plan

Ready to merge

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added the frontend PRs that change frontend files label Aug 13, 2025
Copy link
Collaborator

@psychedelicious psychedelicious left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nice to have this in the app, thanks for working on it.

  • As you noted, perf is not great as layer size increases. It seems easy to accidentally end up with a super big layer and stack of adjustments that bogs the system down. I think this is OK for a v1 implementation but we should do our best to make it faster ASAP. I imagine the only real way to do that is native code. Maybe https://wasm-vips.kleisauke.nl/ or port to a diff lang and compile to wasm.
  • I ended up with a full UI crash at some point. Somethign was trying to access a property of undefined. IIRC I was adjusting sharpness. I refreshed the page and it started working again. Sorry but I didn't capture the error; I am unable to reproduce it now.

Some UI/UX feedback:

  • It's not clear if you have selected simple or curves. Suggest setting the colorScheme on the button to invokeBlue to indicate active mode. Added code suggestion
  • When disabled, you can still fiddle w/ the sliders and curves editor but they don't do anything. Suggest disabling the knobs n dials when disabled
  • There's no way to dismiss the adjustment without applying/finishing it. And finishing always rasterizes the layer. Suggest comparing selected adjust mode and its settings and if unchagned from defaults, when finishing, do not rasterize
  • Layout is a bit wonky when panel is at min width - also needs some padding:
    image
  • Curves editor canvas does not scale when resizing panel, resulting in stretched appearance:
    image
  • The font size for channel labels in curve editor is too small
  • Curves editor needs padding around it
  • Some wonkiness when dragging a point across the x coord of another point:
    Screen.Recording.2025-08-15.at.1.09.56.pm.mov
  • Some awkward behaviour w/ doubleclicking, I see we are calculating the nearest point and remvoing it on double click but it's awkward when double clicking away from an existing point:
    Screen.Recording.2025-08-15.at.1.12.21.pm.mov
  • Some wonkiness when clicking on the top-right point:
    Screen.Recording.2025-08-15.at.1.13.41.pm.mov

@dunkeroni
Copy link
Contributor Author

Thanks for the thorough feedback while I bumble my way through frontend code like a hog in a chandelier showroom. It'll take me some time to work through it, and I might ping you for thoughts as I go.

Some points for right now:

  • There's no way to dismiss the adjustment without applying/finishing it. And finishing always rasterizes the layer. Suggest comparing selected adjust mode and its settings and if unchanged from defaults, when finishing, do not rasterize

It can be removed without finishing with the Right Click menu. I had wanted a Cancel button, but it's already quite cluttered when the panel is shrunk, and Reset felt more important to have there. Another trashcan/X button on the layer is small but might trip users up.
Maybe the Simple/Curves selector should be moved down into the collapsible area to keep Enable/Reset/Finish/Cancel at the top? Not super happy with the layout as it is. Also it's more colorful than other layers are, but that kinda helps because it nags my attention to either finish or remove it instead of leaving them on.
image

  • Layout is a bit wonky when panel is at min width - also needs some padding:

This one confused me because I was sure I checked that, but it turns out to be browser-specific.
Fine on Chrome:
Screenshot From 2025-08-15 01-57-58

Wonky on Firefox:
Screenshot From 2025-08-15 01-57-47
I'm guessing there is some better more general way that I should be defining those to fit correctly in all cases. I'll play with it and check on both for the future.

@psychedelicious
Copy link
Collaborator

It can be removed without finishing with the Right Click menu. I had wanted a Cancel button, but it's already quite cluttered when the panel is shrunk, and Reset felt more important to have there. Another trashcan/X button on the layer is small but might trip users up.

Maybe the Simple/Curves selector should be moved down into the collapsible area to keep Enable/Reset/Finish/Cancel at the top? Not super happy with the layout as it is. Also it's more colorful than other layers are, but that kinda helps because it nags my attention to either finish or remove it instead of leaving them on.

Could use IconButton for reset (counter-clockwise circle arrow) cancel (trashcan) and finish (check) buttons to get a bit more room.

Nitpick - I think "apply" is a better word than "finish" for Invoke - we use "apply" all over the place for this kind of action.

@psychedelicious
Copy link
Collaborator

I teamed up with Claude to implement the filters (and the LightnessToAlpha filter, used by Control layers) in a few different non-JS ways: Notes:

wasm-vips:

  • No exact mapping of the functionality you implemented - which I like! - to what this library provides.
  • Pain to set up.
  • In the end it wasn't substantially faster than JS, I suspect bc I had to do some conversion of the array buffer before passing off to teh library

C++ compiled to WASM:

  • Much faster. Fastest option for LightnessToAlpha
  • Can port the JS logic directly and it's very simple logic.
  • Surely can parallelize but I didn't get that far.

WebGL:

  • Substantially faster than C++ for simple adjustments. Not faster for LightnessToAlpha
  • Claude couldn't figure out the curves filter, I didn't push it.

The code for each implementation was super messy and hacky and I think I discarded it all last week. But just wanted to throw these ideas out.

@dunkeroni
Copy link
Contributor Author

Got delayed a couple of weeks with work.
I went through and addressed most (but not yet all) comments. I replied to a bunch to keep track, sorry for all the pings.

Main/Simple editor changes:

  1. Mode is now highlighted blue
  2. Icon buttons for Cancel, Reset, Apply (TODO: Tooltips)
  3. Slider now in the middle to match the layout we use everywhere else for slider/number combos (TODO: number box is too wide)
  4. Fixed the flex resizing issue on non-chrome browsers
  5. Sharpness can go negative to soften. Also limited to -0.5 to 0.5 range
image

Curves editor changes:

  1. Redraws instead of stretching now so the lines and dots don't go fat and wonky
  2. Continues to follow the mouse direction at the edge of the graph if the pointer goes out of bounds, and correctly places the control point when the user releases the click in that case.
  3. Now limits the X motion to remain between the two neighboring control points, which prevents dragging past an existing point from absorbing/shoving it, and makes it possible to set up thresholds where two points are aligned vertically.
  4. I tried to reduce the moire pattern that the histogram makes on resize. It's still there but less noticeable.
image

Copy link
Collaborator

@psychedelicious psychedelicious left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I have cleaned up some stuff to simplify/conventionalize React-y stuff and improve UI rendering performance. As a high-level metric, this improved FPS from ~11 to ~19.

There are some more gains to be had in the filter calculations themselves without moving to a native code implementation.

I teamed up with an LLM to investigate optimizing the algorithms. We got pretty good results. While the improvements were highly variable, they averaged to ~30% faster than the current implementation for simple and more like ~50% faster for curves.

I added a check to the benchmarks to compare the outputs of the original and optimized implementations and found they were not identical. One or two channel values were often +/-1 different from the current implementation.

I suppose this difference was related to precision, and while it is a very small diff and not noticeable to the user, I decided to not change the algorithms.

I also fixed a small bug in the curves editor points sorting that could incorrectly sorted points where x=255. The bug essentially converted the real "last" point into a "user" point. So you could move it away from 255,255.

@psychedelicious psychedelicious enabled auto-merge (rebase) September 11, 2025 07:21
@psychedelicious psychedelicious merged commit 48c8a9c into invoke-ai:main Sep 11, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
frontend PRs that change frontend files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants