feat(base,sdk): Support for dirty cross-process lock in the EventCache
#5856
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 tasks
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
4 times, most recently
from
d6bf9c0 to
b13c1fa
Compare
CodSpeed Performance ReportMerging #5856 will not alter performanceComparing Summary
|
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
7 times, most recently
from
08d2859 to
2305356
Compare
This patch renames the `CrossProcessLockKind` type to `CrossProcessLockState`.
This patch adds a `#[must_use]` attribute on `CrossProcessLockGuard` and `CrossProcessLockState` to avoid a misuse.
This patch adds the `CrossProcessLockState::map` method along with its companion `MappedCrossProcessLockState` type. The idea is to facilitate the creation of custom `CrossProcessLockState`-like type in various usage of the cross-process lock.
This patch updates `EventCacheStoreLock::lock()` to return an `EventCacheStoreLockState` instead of an `EventCacheStoreLockGuard`, so that the caller has to handle dirty locks.
This patch replicates the `is_dirty` and `clear_dirty` methods from `CrossProcessLock` to `CrossProcessLockGuard`. It allows to get an access to this API from a guard when one doesn't have the cross-process lock at hand.
…store. This patch changes `EventCacheStoreLockState` to own a clone of the inner store. It helps to remove the `'a` lifetime, and so it “disconnects” from the lifetime of the store.
This patch implements `Clone` for `CrossProcessLockGuard`.
This patch implements `Clone` for `EventCacheStoreLockGuard`.
This patch extracts fields from `RoomEventCacheState` and move them into `RoomEventCacheStateLock`. This lock provides 2 methods: `read` and `write`, respectively to acquire a read-only lock, and a write-only lock, represented by the `RoomEventCacheStateLockReadGuard` and the `RoomEventCacheStateLockWriteGuard` types. All “public” methods on `RoomEventCacheState` now are facade to the read and write guards. This refactoring makes the code to compile with the last change in `EventCacheStore::lock`, which now returns a `EventCacheStoreLockState`. The next step is to re-load `RoomEventCacheStateLock` when the lock is dirty! But before doing that, we need this new mechanism to centralise the management of the store lock.
… dirty. This patch updates the `RoomEventCacheStateLock::read` and `write` methods to reset the state when the cross-process lock is dirty.
Hywan
force-pushed
the
feat-cross-process-lock-event-cache
branch
from
2305356 to
8ee41f4
Compare
This patch replaces `sleep` by `yield_now`, which has the same effect in this case, and makes the tests run faster.
This patch implements `RoomEventCacheStateLockWriteGuard::downgrade` to simplify the code a little bit.
This patch fixes a problem found in a test (not commited yet) where it was impossible to do multiple calls to `read` if the first guard was still alive. See the comments to learn more.
This patch adds the new `test_reset_when_dirty` test, which ensures the state is correctly reset when the cross-process lock over the store becomes dirty.
Hywan
commented
Nov 19, 2025
| // Lock is dirty, not a problem, it's the first time we are creating this state, no | ||
| // need to refresh. | ||
| EventCacheStoreLockState::Dirty(guard) => { | ||
| EventCacheStoreLockGuard::clear_dirty(&guard); |
Member
Author
There was a problem hiding this comment.
Note: I must add a test for this case, and see the dirty marker is reset.
| }); | ||
| } | ||
| // All good now, mark the cross-process lock as non-dirty. | ||
| EventCacheStoreLockGuard::clear_dirty(&guard.store); |
Member
Author
There was a problem hiding this comment.
Note: I must add an assertion to see if the dirty marker is reset.
| current = *pred_meta; | ||
| } | ||
| // All good now, mark the cross-process lock as non-dirty. | ||
| EventCacheStoreLockGuard::clear_dirty(&guard.store); |
Member
Author
There was a problem hiding this comment.
Note: I must add an assertion to see if the dirty marker is reset.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
EventCachestores and organises all events. ARoomEventCacherepresents a subset of theEventCache, be all events associated to a particular room. That's the type used by eachRoomto store and organise its events. TheEventCacheStoreis the type representing the store for theEventCache. It is used byRoomEventCacheto represent the stored data.RoomEventCachealso has in-memory data. In-memory data are always in-sync with data in the store. TheEventCacheStoreis behind aCrossProcessLock, a special lock that protects data from being acquired across several processes. So far, so good.We've recently introduced the possibility for a cross-process to be invalidated/to be dirty, see #4874. When a
CrossProcessLockis obtained, it is now possible to know if another process obtained it before, thus invalidating the loaded data from the store we might have in-memory. It is a sign we need to refresh the in-memory data.This is what this set of patches is doing: we are reacting to a dirty cross-process lock and refreshing the in-memory state of
RoomEventCacheaccordingly.This set of patches must be reviewed one-by-one to understand the modifications and their impact. The tl;dr is the following:
RoomEventCacheStatebecomesRoomEventCacheStateLock, representing 2 locks at once: the per-thread lock over the state, and the cross-process lock over the store,RoomEventCacheStateInnerbecomesRoomEventCacheStateLockInner,RoomEventCacheInner::stateholds aRoomEventCacheLockinstead of aRwLock<RoomEventCacheState>,RoomEventCacheLockimplements areadand awritemethods, to respectively obtain a read and a write lock. These operations are no longer infallible. It impacts a couple of callers, but nothing fancy.The impacted code is mostly internal code.
Todo
VectorDiffs are broadcasted to the subcribers. This isn't done byshrink_to_last_chunkbecause it's supposed to run when no more subscribers are listening. We must handle this, I forgot!