qvm-open-in-dvm/qvm-open-in-vm doesn't work with KDE-based templates

[rysiekpl]

Qubes OS release

R4.3, but might affect older versions

Brief summary

qvm-open-in-dvm and qvm-open-in-vm do not work with DispVM based on KDE-based templates.

Steps to reproduce

1. Create a KDE-based template (for example, following this guide),
2. Create a DisposableVM Template based on that template. Set it as default dispvm (either globally, or for a VM you will use in the next step)
3. Start any VM, create a non-empty txt file there.
4. Call qvm-open-in-dvm on this file

Expected behavior

File opens in a new DispVM.
Changes to the file are transferred back to the source VM (after closing the editor).

Actual behavior

The DispVM starts and then immediately shuts down.

Additional information

Similar to #6884

What's happening here is that qubes-open is using xdg-open, which in turn uses kde-open or kde-open5 in the KDE-based cube. These (kde-open/kde-open5) start the application that handles the file in a way that it is not their child process.

This is how pstree looks when opening a file in a working DispVM (where firefox-esr happens to be the application handling the file type):

        ├─qrexec-fork-ser───qrexec-fork-ser─┬─logger
        │                                   └─vm-file-editor───qubes-open─┬─cat
        │                                                                 └─xdg-open───firefox-esr

And this is how it looks in a cube where kde-open/kde-open5 was used to launch the application (in this case, gwenview):

        ├─systemd─┬─(sd-pam)
        │         ├─(...)
        │         ├─gwenview───13*[{gwenview}]

qubes-open is expected to wait until all its child processes are done; after it exits, the parent proces -- vm-file-editor -- cleans up the temporary directory with the file being opened/edited, and exits, allowing the cube to shut down.

Since kde-open/kde-open5 immediately return, and since the application process is not a child of qubes-open, the DispVM immediately shuts down.

In fact, this also affects qvm-open-in-vm when the target cube uses a KDE-based template. In that case, the application is started, but by the time it actually fully starts, the file is gone, as vm-file-editor already managed to delete it after qubes-open exited.

Proposed solution

Currently qubes-open uses a combination of exec and cat to keep stdout/stderr connected even if the handler (say, exo-open):

# The "| cat" is rather silly hack, to wait for all the children processes too
# (unless they close their stdout). This is needed on Xfce template, because
# exo-open starts the application in the background, but keeps the original
# stdout/err connected.
export DE=generic
exec xdg-open "$@" | cat

This does not work for kde-open/kde-open5. Instead, I tested a quick-fix based on pidwait:

export DE=generic
xdg-open "$@"
pidwait -f "$1"

This works well in KDE-based templates, for viewing as well as editing files. Testing in GNOME-based and Xfce-based templates is my next step.

I'm sure this is not a perfect solution, there is possibly a race condition here where some other application could open that same temporary file and get picked up by pidwait. This seems unlikely to me, but flagging just as well. That said, the | cat method used currently is also described as a "silly hack".

[marmarek]

Thanks for great analysis of the issue!
Do you know how kde-open starts the editor? Some dbus activation? Or maybe plain systemd-run? There might be a way to wait for editor to finish using the same API it was started with.

[rysiekpl]

I do not know how kde-open/kde-open5 start the relevant applications. I tried to establish that, but frankly KDE codebase is byzantine in its complexity, and I could not find my way around it to pin-point where that even happens.

My understanding is that different handlers (kde-open/kde-open5, exo-open, gvfs-open, etc) can be expected to use different mechanisms for starting applications, with very different side-effects. So, we would probably need to either:

1. be aware which handler is going to be used and deploy a relevant specific technique of waiting on that process; or
2. somehow cycle through all possible such techniques.

Option 1. gets uncomfortably close to re-implementing xdg-open from scratch. Option 2. would mean a lot of complex code run each time a file is opened using qvm-open-in-vm/qvm-open-in-dvm.

So I went with a much simpler, if perhaps not perfect, pidwait-for-process-that-got-started-with-the-filename-as-argument approach. I think it should work regardless of what specific handler is executed by xdg-open down the line.

[rysiekpl]

I have now tested it on Xfce- and GNOME-based templates. The solution required some fixes, specifically:

• the - A flag for pidwait to ignore parent processes, otherwise qubes-open got stuck waiting for itself to exit;
• making sure the exit code of the script is the exit code of xdg-open, so that any issues can correctly trickle back to parent proxcess.

All implemented in this pull request: QubesOS/qubes-core-agent-linux#642

[rysiekpl]

Let me be more specific – this has been tested on:

• Debian 13 and Fedora 43 based templates (GNOME and Xfce) flavours; the only difference from clean templates was replacing /usr/bin/qubes-open with my patched version.
• Debian 13 based template with KDE installed according to this guide and /usr/bin/qubes-open replaced with my patched version.
• DispVMs based on the Debian 13 KDE template mentioned above.

I have not tested it in DispVMs based on GNOME and Xfce templates, but I don't think there is any reason to expect it won't work – I can see the qvm-open-in-vm process running until the handler application is closed in the test cube in all tested configurations (Debian/Fedora, GNOME/Xfce/KDE).

[rysiekpl]

What are the next steps here? Is there anything else I should do or test to help with merging the PR? Is there any issue with the PR?