Skip to content

feat: add TarUtilities tar creation methods#354

Open
wangxiaoxuan273 wants to merge 4 commits into
oras-project:mainfrom
wangxiaoxuan273:feat/tar-creation
Open

feat: add TarUtilities tar creation methods#354
wangxiaoxuan273 wants to merge 4 commits into
oras-project:mainfrom
wangxiaoxuan273:feat/tar-creation

Conversation

@wangxiaoxuan273

@wangxiaoxuan273 wangxiaoxuan273 commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

Description

Part of #37
Add tar archive creation utilities as part of the FileStore implementation breakdown.

Changes

  • TarUtilities.TarDirectoryAsync — creates a tar archive from a directory, writing entries to an output stream. Supports:
    • Recursive directory traversal (subdirectories and files)
    • Reproducible tarballs (Unix epoch timestamps when reproducible: true)
    • Symbolic link entries
    • Portable entries (uid/gid set to zero)
  • TarUtilities.CreateDirectoryEntry — creates a PaxTarEntry for a directory with correct mode and optional epoch timestamp
  • TarUtilities.GetUnixFileMode — returns standard Unix file modes: 755 for directories, 644 for files

Tests (13 test methods)

  • TarDirectoryAsync: basic files, subdirectories, reproducible timestamps, non-reproducible timestamps, empty directory, file content preservation, uid/gid zero, prefix normalization
  • CreateDirectoryEntry: slash appending, preserving trailing slash, reproducible timestamp
  • GetUnixFileMode: directory returns 755, file returns 644

Context

Part 4 of the FileStore PR breakdown (#328), per sajayantony's suggested split. This PR has no dependencies and can be reviewed independently.

Copilot AI review requested due to automatic review settings March 2, 2026 06:09
@codecov

codecov Bot commented Mar 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.82353% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.17%. Comparing base (5953fb1) to head (9583222).

Files with missing lines Patch % Lines
src/OrasProject.Oras/Content/File/TarUtilities.cs 98.82% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #354      +/-   ##
==========================================
+ Coverage   91.76%   92.17%   +0.41%     
==========================================
  Files          64       65       +1     
  Lines        2755     2925     +170     
  Branches      364      386      +22     
==========================================
+ Hits         2528     2696     +168     
  Misses        138      138              
- Partials       89       91       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds internal tar-creation helpers to support the ongoing FileStore implementation breakdown, plus a focused xUnit test suite validating basic tar entry construction behavior.

Changes:

  • Introduces TarUtilities with TarDirectoryAsync, directory entry creation, and default Unix mode helpers.
  • Adds unit tests covering directory/file entries, timestamps (reproducible vs non-reproducible), uid/gid portability, and prefix normalization.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 9 comments.

File Description
src/OrasProject.Oras/Content/File/TarUtilities.cs Implements tar writing for a directory tree, including reproducible timestamp option and basic symlink handling.
tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs Adds unit tests validating tar contents and metadata produced by TarUtilities.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs Outdated
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs Outdated
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs Outdated
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs
Comment thread tests/OrasProject.Oras.Tests/Content/File/TarUtilitiesTest.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Signed-off-by: Xiaoxuan Wang <wangxiaoxuan119@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs

@akashsinghal akashsinghal left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

This review was generated by Copilot using Claude Opus 4.6, mimicking the review style of shizhMSFT. This is not the real shizhMSFT.

The implementation is solid — prefix validation, symlink-aware traversal, OS-aware separator normalization, and deterministic ordering are all well done. Good test coverage across the major paths. A few observations below.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs Outdated
Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs
@akashsinghal

Copy link
Copy Markdown
Collaborator

Many lines in both files are wrapped at ~55-60 columns, well below the repo's target of ~100 (hard limit 120). This makes the code harder to scan. A few examples from TarUtilities.cs:

// Currently (wrapped at ~55 cols):
var dirs =
    Directory.GetDirectories(currentDir);         // 61 chars unwrapped

var relativePath = Path.GetRelativePath(
    sourceDirectory, dir);                        // 74 chars unwrapped

symEntry.ModificationTime =
    DateTimeOffset.UnixEpoch;                     // 72 chars unwrapped

using var tarWriter = new TarWriter(
    outputStream, leaveOpen: true);               // 71 chars unwrapped

await tarWriter.WriteEntryAsync(
    rootEntry, cancellationToken)
    .ConfigureAwait(false);                       // 90 chars unwrapped

All of these fit comfortably on a single line under 100 columns. Same pattern throughout the test file. Consider unwrapping lines that are under ~100 columns to improve readability.

@akashsinghal

Copy link
Copy Markdown
Collaborator

Test consolidation opportunity — several tests share identical setups and could be merged without losing coverage:

  • WithSubdirectories + DeterministicOrder + Reproducible_SetsEpoch: DeterministicOrder already creates subdirs with reproducible: true and verifies entry names in order. Adding epoch timestamp assertions and entry type checks covers all three tests in one.
  • BasicFiles + SetsUidGidToZero + PrefixWithoutSlash: All create simple files. A single test asserting entry names, types, uid/gid=0, and trailing-slash normalization covers all three.
  • 3 CreateDirectoryEntry_* tests: Could be a single test or [Theory] — each is ~10 lines testing a trivial helper.
  • 2 GetUnixFileMode_* tests: Could be one [Theory] with [InlineData].

Tests that should stay separate (unique coverage): EmptyDirectory, FileContent_IsPreserved, NonReproducible_HasRecentTime, InvalidPrefix_Throws, MissingSource_Throws, both symlink tests.

This would bring the count from ~18 to ~11-12 tests with equivalent coverage and less duplicated setup code.

- Add exception docs to TarDirectoryAsync for
  ArgumentException and DirectoryNotFoundException
- Document recursion depth assumption and symlink target
  preservation behavior in WalkDirectoryAsync remarks
- Clarify GetUnixFileMode returns hardcoded portable modes
  (755/644), not actual filesystem permissions

Signed-off-by: Xiaoxuan Wang <wangxiaoxuan119@gmail.com>
- Unwrap lines in TarUtilities.cs to target ~100 columns
- Consolidate related tests in TarUtilitiesTest.cs:
  merge BasicFiles+SetsUidGidToZero+PrefixWithoutSlash,
  merge WithSubdirectories+DeterministicOrder+Reproducible_SetsEpoch,
  merge CreateDirectoryEntry tests into Theory,
  merge GetUnixFileMode tests into Theory

Signed-off-by: Xiaoxuan Wang <wangxiaoxuan119@gmail.com>
Copilot AI review requested due to automatic review settings March 4, 2026 06:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.


You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/OrasProject.Oras/Content/File/TarUtilities.cs
- Add Path.IsPathRooted check plus cross-platform HasDrivePrefix
  guard (e.g. C:/abs, C:\abs) since Path.IsPathRooted does not
  recognise drive letters on non-Windows platforms
- Add test cases for C:/abs and C:\abs prefixes

Signed-off-by: Xiaoxuan Wang <wangxiaoxuan119@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.


You can also share your feedback on Copilot code review. Take the survey.

@akashsinghal

Copy link
Copy Markdown
Collaborator

Caution

This review was generated by Copilot using Claude Opus 4.6, mimicking the review style of shizhMSFT. This is not the real shizhMSFT.

Second-round review of PR #354

All 30 prior threads addressed. Test consolidation (18 → 11 methods), lazy enumeration for non-reproducible mode, symlink handling for directories, prefix validation improvements, and better docs — clean iteration. Two observations:

1. ReadAllTarEntriesAsync uses default copyData: false (TarUtilitiesTest.cs:761)

TarReader.GetNextEntryAsync() defaults to copyData: false, which means DataStream on each entry references the reader's internal stream and becomes invalid after advancing. The tests only access metadata (Name, EntryType, Uid, Gid, ModificationTime, LinkName) so this works today, but it's a latent fragility — a future test author adding DataStream assertions to an entry from this list will get silently wrong results. Consider passing copyData: true in the helper, or adding a comment explaining the constraint.

2. System.IO.File.OpenRead fully qualified (TarUtilities.cs:208)

nit: System.IO.File.OpenRead(file) is fully qualified to avoid collision with the File namespace OrasProject.Oras.Content.File. This is correct but a bit noisy. A using IOFile = System.IO.File; alias at the top would clean up this and any future calls. Same pattern exists in the test file with System.IO.File.WriteAllTextAsync.

Otherwise SGTM. The symlink handling (both file and directory), reproducible mode with deterministic ordering, and prefix validation are well-implemented.

@akashsinghal

Copy link
Copy Markdown
Collaborator

@wangxiaoxuan273 could you address the final comments on this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants