[virtio-pmem] Implementation #5463
Open
+2,322
−133
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5463 +/- ##
==========================================
- Coverage 82.79% 82.64% -0.16%
==========================================
Files 263 269 +6
Lines 27218 27731 +513
==========================================
+ Hits 22535 22917 +382
- Misses 4683 4814 +131
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ShadowCurse
force-pushed
the
virtio_pmem
branch
2 times, most recently
from
d8f9547 to
5970613
Compare
ShadowCurse
added
Status: Awaiting review
ShadowCurse
force-pushed
the
virtio_pmem
branch
2 times, most recently
from
a8bedbb to
1d2aeb2
Compare
ShadowCurse
force-pushed
the
virtio_pmem
branch
from
1d2aeb2 to
7d83503
Compare
Manciukic
reviewed
Oct 7, 2025
ShadowCurse
force-pushed
the
virtio_pmem
branch
from
7d83503 to
efd93ea
Compare
ShadowCurse
force-pushed
the
virtio_pmem
branch
2 times, most recently
from
9a554b4 to
4b4779b
Compare
bchalios
requested changes
Oct 7, 2025
ShadowCurse
force-pushed
the
virtio_pmem
branch
10 times, most recently
from
7eff29f to
4a19190
Compare
|
I took a look at the codecov report. It looks like that there are a few things we're missing that we could (meaningfully) cover:
|
Manciukic
reviewed
Oct 9, 2025
| pub struct PmemState { | ||
| pub virtio_state: VirtioDeviceState, | ||
| pub config_space: ConfigSpace, | ||
| pub config: PmemConfig, |
There was a problem hiding this comment.
do we need to do anything about the backing file to ensure writes to it are visible to other processes? For example, we need to ensure that the process taking the snapshot can read the correct contents.
Are we doing anything special for the existing block devices to flush them to disk?
There was a problem hiding this comment.
Currently we do not flush block devices (block and pmem) on snapshot creation. If VM is killed right after snapshot is created, pmem will be synced by the kernel.
Comment on lines
+197
to
+204
| In the case where multiple VMs have `virtio-pmem` devices that point to the same | ||
| underlying file the memory overhead can be amortized since total maximum memory | ||
| usage will only include a single instance of `virtio-pmem` memory. |
There was a problem hiding this comment.
I would not mention this as we do not recommend this usecase
There was a problem hiding this comment.
I think it is valid to leave this explanation even if we don't recomment this since it is still a valid usage of pmem.
docs/pmem.md
Outdated
Comment on lines
187
to
191
| Since `virtio-pmem` resides in host memory it does increase the maximum possible | ||
| memory usage of a VM since now VM can use all of its RAM and access all of the | ||
| `virtio-pmem` memory. In order to minimize the overhead, it is highly | ||
| recommended to use `DAX` mode to avoid unnecessary duplication of data in guest | ||
| page cache. |
There was a problem hiding this comment.
I think we should make clear that the resident memory used to back the virito-pmem does not count towards the VM memory limit, but that can be reclaimed (paged out) by the host.
There was a problem hiding this comment.
added a section about reclamation of memory.
What do you mean by does not count towards the VM memory limit?
ShadowCurse
force-pushed
the
virtio_pmem
branch
12 times, most recently
from
5bac831 to
552e3a7
Compare
msync is used by virtio-pmem device to trigger sync of mmaped file content to the underlying file. Signed-off-by: Egor Lazarchuk <[email protected]>
Add implementations of device, event handling, metrics. Add device config and builder types for API use. Signed-off-by: Egor Lazarchuk <[email protected]>
Update VmResources type with virtio-pmem configuration field to allow virtio-pmem devices be configured through config files and later through API calls. Signed-off-by: Egor Lazarchuk <[email protected]>
Both virtio-block and virtio-pmem can act as root devices for a VM. Add a check to prevent specifing more than 1 root device for a VM. Signed-off-by: Egor Lazarchuk <[email protected]>
Add /pmem/id PUT request for virtio-pmem configuration. Add corresponding metrics. Signed-off-by: Egor Lazarchuk <[email protected]>
Virtio-pmem devices need to allocate a memory region in guest physical memory. The safe place to do this is past 64bit MMIO region. Signed-off-by: Egor Lazarchuk <[email protected]>
Add a counter for KVM slot ids into VmCommon struct. This is done because virtio-pmem device needs to obtain it's KVM slot id independently from number of slots in GuestMemoryMmap. Signed-off-by: Egor Lazarchuk <[email protected]>
Add methods to attach virtio-pmem devices to Vmm. Add methods to create KVM memory slot for virtio-pmem devices. Signed-off-by: Egor Lazarchuk <[email protected]>
Add logic to store and restore virtio-pmem device information in a snapshot. Signed-off-by: Egor Lazarchuk <[email protected]>
Add functional and API tests for virtio-pmem device and its configuration fields Signed-off-by: Egor Lazarchuk <[email protected]>
Expose virtio-pmem metrics in the logger, so they are exported in metrics.json. Update integration tests to expect new metrics. Signed-off-by: Egor Lazarchuk <[email protected]>
Add description of pmem APIs in swagger file and device-api.md Signed-off-by: Egor Lazarchuk <[email protected]>
Add new document about virtio-pmem configuration and usage. Signed-off-by: Egor Lazarchuk <[email protected]>
Add a note about addition of virtio-pmem device. Signed-off-by: Egor Lazarchuk <[email protected]>
ShadowCurse
force-pushed
the
virtio_pmem
branch
from
552e3a7 to
e527e0c
Compare
bchalios
approved these changes
Oct 13, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Add
virtio-pmemdevice support.Closes #5448
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyleto verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.