Batch TreeArtifact metadata through BazelOutputService

[mzhaom]

Description

Keep Skyframe as the central place that discovers and constructs final output
metadata, while allowing ActionOutputMetadataStore to query an
OutputService for all TreeArtifact child statuses in one BatchStat call.

When no batch statter is available, the existing per-child filesystem path is
unchanged. There is no metadata injection from SpawnRunner or
RemoteExecutionService.

Motivation

BazelOutputService already exposes a multi-path BatchStat, but
TreeArtifact construction requested every child's fast digest separately.
For a lazy output filesystem, every request can cross the output-service RPC
boundary even though the service already owns the staged file and directory
metadata.

In a representative remote-output benchmark producing 4,098 files and 528 MiB
of logical output, the profile contained approximately 4,100 one-path
BazelOutputService.BatchStat calls. An equivalent batching patch on Bazel
8.5.1 produced:

Configuration	Elapsed time
Eager output download	4.100 s
Lazy output service before batching	1.484 s
Lazy output service with batched metadata	0.896 s
Inaccessible minimal-download roofline	0.549 s

This removes the RPC fanout while deliberately retaining Skyframe's final
filesystem traversal. That distinction is important for actions containing
multiple spawns or a mix of remote and local outputs: metadata is collected
from the final output-service state, rather than copied from an earlier remote
ActionResult.

Build API Changes

No.

Checklist

• I have added tests for the new use case.
• Documentation is not applicable because this is an internal remote
  execution optimization without user-facing configuration changes.

Validation

• bazel test //src/test/java/com/google/devtools/build/lib/skyframe:ActionOutputMetadataStoreTest --test_timeout=60
• bazel test //src/test/java/com/google/devtools/build/lib/remote:RemoteTests --test_timeout=60
• bazel build //src:bazel-dev

The regression test stages a nested TreeArtifact, modifies one child before
Skyframe collects metadata, and verifies both the final digest and exactly one
batch request containing all children.

Release Notes

RELNOTES: None

[coeuvre]

We intentionally moved away from injecting metadata within the SpawnRunner because there are too many edge cases and we wanted skyframe to be the central place to collect metadata. For this optimization, is it possible to "inject" file/dir metadata into BazelOutputService instead?

[mzhaom]

Thanks — yes. I traced this back to the mixed TreeArtifact issue fixed in #16812 and removed the SpawnRunner/RemoteExecutionService metadata injection entirely.

The revised design keeps Skyframe responsible for traversing the final TreeArtifact and constructing its TreeArtifactValue. After traversal, Skyframe sends all child paths in one multi-path BazelOutputService.BatchStat request. The output service is therefore the authoritative source of file/directory status, but Skyframe still decides what metadata belongs to the completed action.

This avoids stale ActionResult metadata when a later/local spawn modifies a mixed tree, while reducing roughly 4,100 one-path RPCs to one request. The new regression test overwrites a staged child before collection and verifies the final digest plus exactly one batch containing all children.

[coeuvre]

Thanks, it now looks better. I will import the code.