Extend functionality to deserialize rootfs upper layer of an arbitrary container to a tar archive.

[benldrmn]

#12119 implemented rootfs snapshotting deserialization (restore) for single-container sandboxes only.
This PR extends this functionality to any container, including containers that are part of a multi-container sandbox.
builds on top of: #12411

[benldrmn]

Hi @ayushr2 - just making sure this doesn't fall between the cracks (it is the 2nd part of the already merged container rootfs snapshotting MR you merged).

[ayushr2]

@benldrmn I think rootfs snapshot deserialization is already support for multi-container. 3eb1b76 (the PR you linked) added functionality in tmpfs to deserialize from TAR but did not add any runsc API to utilize that.

In a730d6c, we added the necessary plumbing.

In multi-container setup, like k8s, each container within the Pod has its own container spec. So each container can set its own dev.gvisor.tar.rootfs.upper annotation. It is respected in the codebase for sub-containers:

gvisor/runsc/sandbox/sandbox.go

Lines 459 to 476 in 428bb2a

	// StartSubcontainer starts running a sub-container inside the sandbox.
	func (s *Sandbox) StartSubcontainer(spec *specs.Spec, conf *config.Config, cid string, stdios, goferFiles, goferFilestores []*os.File, devIOFile *os.File, goferConfs []boot.GoferMountConf) error {
	log.Debugf("Start sub-container %q in sandbox %q, PID: %d", cid, s.ID, s.Pid.Load())
	if err := s.configureStdios(conf, stdios); err != nil {
	return err
	}
	s.fixPidns(spec)
	var rootfsUpperTarFile *os.File
	if path := specutils.RootfsTarUpperPath(spec); path != "" {
	var err error
	rootfsUpperTarFile, err = os.OpenFile(path, os.O_RDONLY, 0644)
	if err != nil {
	return fmt.Errorf("opening rootfs upper tar file: %v", err)
	}
	}
	defer rootfsUpperTarFile.Close()

This feature already exists.

[benldrmn]

@benldrmn I think rootfs snapshot deserialization is already support for multi-container. 3eb1b76 (the PR you linked) added functionality in tmpfs to deserialize from TAR but did not add any runsc API to utilize that.

In a730d6c, we added the necessary plumbing.

In multi-container setup, like k8s, each container within the Pod has its own container spec. So each container can set its own dev.gvisor.tar.rootfs.upper annotation. It is respected in the codebase for sub-containers:

gvisor/runsc/sandbox/sandbox.go

Lines 459 to 476 in 428bb2a

	// StartSubcontainer starts running a sub-container inside the sandbox.
	func (s *Sandbox) StartSubcontainer(spec *specs.Spec, conf *config.Config, cid string, stdios, goferFiles, goferFilestores []*os.File, devIOFile *os.File, goferConfs []boot.GoferMountConf) error {
	log.Debugf("Start sub-container %q in sandbox %q, PID: %d", cid, s.ID, s.Pid.Load())
	if err := s.configureStdios(conf, stdios); err != nil {
	return err
	}
	s.fixPidns(spec)
	var rootfsUpperTarFile *os.File
	if path := specutils.RootfsTarUpperPath(spec); path != "" {
	var err error
	rootfsUpperTarFile, err = os.OpenFile(path, os.O_RDONLY, 0644)
	if err != nil {
	return fmt.Errorf("opening rootfs upper tar file: %v", err)
	}
	}
	defer rootfsUpperTarFile.Close()

This feature already exists.

Right, I might have "oversold" the deserialize functionality in this PR, I think the main contribution is allowing to specify different rootfs paths for different containers in the pod with the newly added annotationRootfsUpperTarContainer annotation. Without that, when creating a multi-container pod in k8s, it is not possible to pick different snapshot tars for different containers.

[ayushr2]

Ah you are probably using Pod annotations, which are copied across all container's OCI specs.

But theoretically, different containers in a Pod can have different annotations. So you can set dev.gvisor.tar.rootfs.upper on each container in a Pod with different values. We do that sometimes in our shim. But I assume that's not an option for you?

[benldrmn]

AFAIK k8s only allows setting Pod annotations, there’s no way to annotate specific containers

…

On Wed, 14 Jan 2026 at 0:56 Ayush Ranjan ***@***.***> wrote: *ayushr2* left a comment (google/gvisor#12415) <#12415 (comment)> Ah you are probably using Pod annotations, which are copied across all container's OCI specs. But theoretically, different containers in a Pod can have different annotations. So you can set dev.gvisor.tar.rootfs.upper on each container in a Pod with different values. We do that sometimes in our shim. But I assume that's not an option for you? — Reply to this email directly, view it on GitHub <#12415 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AELDOAIMDQDOPMTON6KZ2CT4GVZ2NAVCNFSM6AAAAACQGD7436VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTONBWHEZTINBQGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[benldrmn]

@ayushr2 thanks - addressed all of your comments in my latest commit. I haven't squashed it yet to ease the review / clearer diff - let me know if you prefer me always squashing to a single commit, even during review.

[ayushr2]

Thanks!

[ayushr2]

Also please squash your commits:

gvisor/CONTRIBUTING.md

Lines 95 to 99 in e0a2f60

	When iterating on a pull request, please amend your commit rather than creating
	new commits on the same branch. Our merge bot applies all commits from the
	branch to master and does not yet support squash merging. If you'd like to
	submit a pull request with multiple commits, please ensure they are independent,
	logical units of work that can be applied separately.