Skip to content

Preloaded disposables #1512

New issue
New issue
Open
Feature
15 of 19 issues completed
Open
Preloaded disposables#1512
Feature
15 of 19 issues completed
Assignees
ben-grande
Labels
C: coreThis issue pertains to a Qubes core component.P: majorPriority: major. Between "default" and "critical" in severity.community devThis is being developed by a member of the community rather than a core Qubes developer.meta-issueThis issue serves to collect and organize a group of other issues.release notesThis issue should be mentioned in the release notes.targets-4.3Feature planned for Qubes 4.3. Remove label if not implemented by release; leave if implemented.
@qubesuser

Description

@qubesuser
qubesuser
opened on Dec 13, 2015

The problem you're addressing (if any)
Disposable vms are very useful for the intent they're made, however one drawback they have is that usage is not instant as other appvms, and one need to wait for it to load before using it (varying from 7-20s depending on hardware)

Describe the solution you'd like
It would be great if there was an option to "preload" the dispvm (quantity to preload would be defined by the user and limited by hardware specs) so whenever you need to use a dispvm the target program launches automatically.

Where is the value to a user, and who might that user be?
It would be a great benefit in terms of speed and convenience depending on how much the user relies on dispvms

Additional context
Currently behaviour would be preserved, if you launch a program for a dispvm using the qubes menu, each call would use a different preloaded dispvm, no reuse would be made. Also when one dispvm is "used", another one would be preloaded to keep the defined amount always ready.

Original description

Starting disposable VMs is faster than normal VMs, but it can often still take several seconds and be a noticeable delay in the user experience.

This proposes to solve this issue by keeping one or more disposable VMs always around runnning, but without qubes-guid started and thus "invisible".

When the user requests a disposable VMs, the system takes one of those cached disposable VMs, adjusts them if necessary and starts qubes-guid, and then starts another cached disposable VMs for the next request.

This allows instantaneously started DispVMs at the cost of losing 1.5-6 GB of RAM, which can be a good tradeoff at least for machines with >= 16GB RAM.

There are two ways of doing this: the most flexible way would be to support any DispVM usage by starting the appropriate service on the cached DVM, and there is an inflexible but faster way that pre-starts the application as well, but only supports a limited number of DispVM applications started from dom0 (typically a web browser and a terminal).

My code implements the "inflexible" way and offers two modes: a faster "separate" mode that keeps around a DispVM for each configured application, and a slower but less RAM hungry "unified" mode that keeps a DispVM with all the applications running, and kills the ones not needed at user request.

You can find the implementation at: https://github.com/qubesuser/qubes-core-admin/tree/insta_dvm

You'll need to create a configuration file in /etc/qubes/dvms like the one provided in the branch. The mode is chosen automatically depending on available RAM, but can be configured in /etc/qubes/cached-dvm-mode

The branch is missing packaging for qubes-start-cached-dvm and the dvms config file, systemd integration for starting it at boot, and making dom0 start menu entries use it.

It's also somewhat hackish overall and might need a rewrite in Python and adjustment to the new core code if shipped after that.

Initial implementation details

So, this task appeared much simpler than it really it, here is a breakdown of what must be discussed/defined (see QA), and what must be done to complete this task. I mixed some comments made by @marmarek and myself to attempt to create the scope of this issue, which is not yet finished and I will keep updating this comment.

Describe the solution you'd like

It would be great if there was an option to "preload" the dispvm (quantity to preload would be defined by the user and limited by hardware specs) so whenever you need to use a dispvm the target program launches automatically.

Relevant functions:

  • API method: qubes.api.admin.create_disposable()
  • Validation: qubes.vm.dispvm.DispVM.from_appvm()
  • Storage: qubes.vm.qubesvm.QubesVM.create_on_disk()
  • Autostart: qubes.vm.qubesvm.QubesVM.on_create_on_disk()

qubes.vm.dispvm.DispVM.from_appvm() seems to be the place to handle checks for the preloaded DispVM preference, in case it is bigger than 0, it should return one of the already preloaded DispVMs.

Preloaded DispVM management:

  1. Preload DispVM per DispVM template:

    1. A process to control the creation of preloaded DispVMs (QA)
    2. On qubes.vm.dispvm.DispVM.from_appvm() create the logic for returning the preloaded DispVM or let it return a new name.
    3. Add the preloaded qubes to a list, to not query every running DispVM that was created by that template that is already in use by the user for anything they want.
      1. Qubesd can be restarted during system runtime and it needs to rebuild the list.
      2. Qubes in the list should be marked somehow to be included in the list.
    4. Set the feature internal=1
    5. The process controlling the preloaded DispVMs then calls admin.vm.Start.
    6. Handle the event domain-started with admin.vm.Pause.

  2. When a Qrexec call targets a DispVM template or a preloaded domain has the event domain-unpaused emitted:

    1. Return one of the preloaded DispVMs from an available list.
    2. Remove the returned qube from the available list.
    3. Preload another DispVM for later use.
      1. Define when the nest DispVM should be preloaded, should it have a delay to not interfere with starting of the application of the preloaded DispVM user just got? Configurable delay is interesting, a value from 0-10 seconds seems appropriate. Check if this is a concern, maybe check available memory, total CPU cores or per qube vcpu (QA)
    4. Cleanup with dispvm.cleanup() after the user closes the first application they started.

Trigger start of preloaded DispVMs:

  • First batch should be triggered on boot, after qubes that have the autostart=True preference (normal qubes should have higher precedence) to avoid memory hiccups of starting preloaded DispVMs but not being able to assign memory to sys-usb.
  • Systemd service that doesn't start when kernel command-line has qubes.skip_autostart (see systemctl cat [email protected] and /proc/cmdline)
  • Internal API (see internal. in qubes-core-admin/qubes/api/internal.py)
  • Some qubes that require preloaded DispVMs only when powered on such as qubes-builder can have startup/shutdown scripts to set the property on its DispVM template with admin.vm.property.Set. It is the simplest solution that doesn't involve creating dependencies on client qubes being running or not.

Resource management:

  • If there is not enough memory to start the qube (qmemman would fail starting the qube), retry at a later time when more memory is available.
  • For a later version, dynamically adjust number of pre-started disposables, which can also be 0, considering a system that has many qubes set to autostart.

Prevent incorrect user interaction with preloaded DispVM:

User needs to be prevented from interaction with "preloaded" disposables, so they really remain clean. This includes NOT showing them:

  • In the menu
  • As target in policy prompts (for example when user copies files)
  • Possibly also some actions in domains widget should be inactive (hiding completely is probably a bad idea, as it skews view of the system memory)
  • Not sure if qvm-run should also be prevented, I'm okay with not blocking it. And then, when it gets used, those things need to be undone.

For this, the feature internal=1 will be set. The domain will also be paused and when used, it will trigger domain-unpaused.

Tasks

Tasks:

With the information above, the following tasks seems appropriate:

  • Add qube preference (qvm-prefs) dispvm-preload and dispvm-preload-max
    • Discover how a new preference is added (QA)
    • Discover how to preload on autostart (QA)
    • In qubes.vm.dispvm.DispVM.from_appvm(), reply with a domain name that is preloaded when necessary.
    • Discover how to manage the lifetime of the domains (Events was the answer)
    • Autostart preloaded DispVMs on boot unless kernel command-line asks to skip it.
    • Prevent user from interacting with preloaded DispVM to keep it clean (Set internal and pause qube)
    • Add parameter to stop preloading DispVMs on shutdown (qvm-shutdown kills preloaded qubes, solving this)
    • Hide preloaded qube from target in policy prompts (for example when user copies files)
  • Testing
    • With kernel command-line to skip autostart
    • When a triggered qube starts and then shuts down
    • Open application from the app menu
    • Open application from qui-domains
    • Open application from another qube targeting the DispVM template
    • Remove the internal feature and check if app-menus update
    • Remove the internal feature and check if qui-domains update
    • Try to stop preloading every DispVM when shutting down all qubes
    • Try to run an application in the preloaded DispVM when it is starting before it is paused (doesn't show until after preloading, as expected).
    • Check if pause is not too early, see user journal for services still starting, top for services consuming some bootstrap resources
    • Set an application to autostart in the disposable template and check if it opens before it is paused (race condition confirmed): Avoid showing windows from disposables when they are being preloaded #9907
    • Preload before GUI session starts: GUI daemon should handle connections to paused and suspended qubes #9940
    • Increase and decrease, decrease and decrease, increase and increase the preload-dispvm-max fast and see if problems occurs, such as orphans (not in the feature list)
    • Run multiple calls to get dispvm and return different disposables: 
      qvm-features default-dvm preload-dispvm-max 5
# wait some time till all qubes are preloaded to have the fastest return
for x in 1 2 3 4 5; do DISPLAY=:0.0 qvm-run --dispvm -- 'xterm -e echo $x' & : ; done
    • Decrease the maximum from 1 to 0 and attempt to get preloaded qube (should not delete the qube in this case) 
      qvm-features default-dvm preload-dispvm-max 1 && sleep 5
qvm-features default-dvm preload-dispvm-max 0 & qvm-run --dispvm -- 'xterm -e echo hey' &
    • Cloning a qube must only copy the preload-dispvm-max feature but not the other preload features.
    • Deal with effects on system resource and clock time skews on extended pause or suspend (on suspend, this is dealt with autostart event)
    • If there is not enough memory to preload qubes, it should be retried later on whenever possible (reequesting a disposable or using one)
    • Run all integration tests and unit tests
  • Document:
    • Docs strings (not many done)
    • qvm-features manual page

Additional context

Currently behaviour would be preserved, if you launch a program for a dispvm using the qubes menu, each call would use a different preloaded dispvm, no reuse would be made. Also when one dispvm is "used", another one would be preloaded to keep the defined amount always ready.

In other words, every call to a DispVM template that would use it as a base to create a DispVM should consider if the DispVM template has the preference to preload set and use the preloaded DispVMs.

Pull requests

Core:

GUI:

Global feature:

Late GUID:

Qmemman:

Memory threshold:

Installer:

Deferred netvm:

Dependency switch:

Backup:

Benchmark:

Site documentation:

Misc:

Issues

Done:

Important and sorted by priority:

Less important:

Sub-issues

Metadata

Metadata

Assignees

Labels

C: coreThis issue pertains to a Qubes core component.P: majorPriority: major. Between "default" and "critical" in severity.community devThis is being developed by a member of the community rather than a core Qubes developer.meta-issueThis issue serves to collect and organize a group of other issues.release notesThis issue should be mentioned in the release notes.targets-4.3Feature planned for Qubes 4.3. Remove label if not implemented by release; leave if implemented.

Type

Feature

Projects

Current team tasks

Status

In review
Qubes OS 4.3

Status

In review

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions