Skip to content

draft articles, providing details about using subprocess run and it's more common inputs and patterns - #347

Merged
heckj merged 15 commits into
mainfrom
articles
Jul 24, 2026
Merged

draft articles, providing details about using subprocess run and it's more common inputs and patterns#347
heckj merged 15 commits into
mainfrom
articles

Conversation

@heckj

@heckj heckj commented Jul 8, 2026

Copy link
Copy Markdown
Member

I wanted to wrap up the great conversations I saw during the development work for this library to provide some of the concrete "how to use it" advice touching on collecting result content, searching for executables vs. providing a path, and the just edging on the stream processing for results from a long running process - or even the input streaming.

I put together these two articles, pulling heavily in influence from the conversations in Swift Forums.

Please look over the example code blocks with a high degree of skepticism, and verify that I'm not suggesting poor code patterns or encouraging use in a way that would be harmful.

@heckj
heckj requested a review from FranzBusch July 8, 2026 00:29
@heckj heckj self-assigned this Jul 8, 2026
@heckj heckj added the documentation Improvements or additions to documentation label Jul 8, 2026
@heckj
heckj requested a review from invalidname July 8, 2026 00:30
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
}
```

The preceding example uses ``run(_:arguments:environment:workingDirectory:platformOptions:input:output:error:)-(_,_,_,_,_,Input,_,_)``:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is supposed to end after error:)

@heckj heckj Jul 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, need the disambiguating extension on that one. The only one that doesn’t need it (or the run) is the one with body: closure

@dempseyatgithub dempseyatgithub Jul 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize now this is DocC-specific markup, so it won't appear exactly like this once processed by DocC. Feel free to mark this as resolved.

Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md
Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
@broken-circle

Copy link
Copy Markdown
Member

Heads-up: #349 removes the deprecated standardOutput/standardError aliases on FileDescriptorOutput, including their DocC curation. This PR touches those same two lines in OutputProtocol.md, so depending on merge order there may be a collision. Mentioning it in case it's easiest to just drop those two entries here; I haven't looked at the rest of this PR's changes yet, so those properties may also be referenced elsewhere.

Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
The function `run` returns only after the body returns and the process exits.
Because of that, the `execution` value, its streams, and the input writer are valid only inside the body.

## Drain every stream, concurrently

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!! This section is very useful. I was struggling to find the right place to emphasize the concurrency requirement.

Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
@heckj

heckj commented Jul 9, 2026

Copy link
Copy Markdown
Member Author

thanks @broken-circle - no worries, it's already merged - so I'll rebase and I need to update based on it (and other's) changes anyway based on the feedback. Bits I missed when assembling ;-)

@heckj
heckj marked this pull request as ready for review July 9, 2026 19:27

@invalidname invalidname left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really useful. I've left a few suggestions to consider.

Comment thread Sources/Subprocess/Documentation.docc/Subprocess.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/Subprocess.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/Subprocess.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/Subprocess.md
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated

> Note: A non-zero exit never surfaces as a Swift error, and neither does output on standard error.
> To run a `catch` block when a command “fails,” check ``TerminationStatus/isSuccess`` yourself and throw from your own code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a nice section that clears up some potential misunderstandings.

Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md Outdated
}
```

The same structure can chain two subprocesses, sending the stream of output from one subprocess to the input of another.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think creating the equivalent to a shell pipe is important enough that this should be its own section.
Possibly 'Chaining Subprocesses' or even 'Chaining Subprocesses To Pipe Commands' or 'Pipe Commands By Chaining Subprocesses'.

When I first scanned the article, I was hoping to find code on how to pipe using Subprocess and missed this section completely.

I also think using the term 'pipe' or 'piping' either in a heading title or body text would make this section easier to find in searches.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be an excellent additional article to add, referenced from here. I don't think it's a great addition for this article though, since it's trying to be more introductory and common use case, and doing so would expand the scope and narrative of the this article, which is really meant to provide the basics and rules behind handling streaming. The piping and chaining concepts are something I'd describe as more advanced topics. Not invalid, just not something I want to try and cover in this article.

``FileDescriptorOutput/fileDescriptor(_:closeAfterSpawningProcess:)``,
which is efficient for large amounts of data.
When you use a file descriptor, the bytes never pass through your code.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly add a code example showing writing the output of a command that generates a lot of text directly to a file via file descriptor?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to keep this article focused on the basics and most common usage. I agree that an example showing how to use the file descriptors effectively - where they shine - would be a great add, but I don't want to expand the scope of this article to include that. I think that would be better as it's own, focused thing - I'm seeing that as "advanced use" of this library.

@invalidname invalidname left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thanks for all your work on this.

* The *collecting* form waits for the subprocess to finish and provides the result.
* The *streaming* form lets you read output while the subprocess runs.

You choose between them depending on whether you pass a trailing closure.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence isn't entirely accurate. You choose whether you stream or collect by passing the appropriate options to output and error parameters. Technically, you could choose to collect (via .string, .data, etc.) even when passing a trailing closure. In other words, you are not forced to stream if you pass in a trailing closure.

@heckj heckj Jul 24, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you! I'm going to use almost your sentence description directly!

Comment thread Sources/Subprocess/Documentation.docc/GettingStarted.md
Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated
let writer = sort.standardInputWriter
try await withThrowingTaskGroup(of: Void.self) { group in
group.addTask {
_ = try await run(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure if we should highlight this example. Subprocess 1.0 deliberately lacks process piping because it's scheduled for 1.1: #151

While this example works, it's inefficient (parent output shouldn't need to be "read out" to a sequence and then write into child process via writer.write; they should just have the pipes connected directly), and it's not the shape we want to encourage developers to copy.

On the other hand, this example is how you can emulate process piping with the API today, so I do see it has value. @heckj, what do you think? Should we leave this out if we know we will be introducing an official API later?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was on the fence about this, given how awkward the API is, but it was a frequent ask in the conversations I followed, not even also tracking #151. I'm inclined to leave it in, primarily because of the asks about it specifically, and it's what we have with the current release of this API. I'd also suggest that as we pin down better API for this down the road, we make a note to explicitly come back and change up this article, or break it out into it's own article (if it's sufficiently deep to warrant it) when updated API is available.

More than happy to pop back when we get there to revise and update, or draft a new article along these lines!

Comment thread Sources/Subprocess/Documentation.docc/StreamingAndInput.md Outdated

@iCharlesHu iCharlesHu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much!!

heckj and others added 15 commits July 24, 2026 08:12
  - Add `- Throws:` clauses to all six `run` overloads: they throw
    `SubprocessError` on launch/limit failures (or rethrow the body
    closure's error), and a non-zero exit is a normal result to inspect
    via `terminationStatus`, not a thrown error.
  - Fix imprecise "The subprocess throws an error…" comments on `.string`,
    `.bytes`, `.data`, and `maxLineLength` to attribute the throw to
    `run`/iteration and name `SubprocessError`.
  - Correct the landing-page example: `standardOutput` is non-optional, so
    it no longer prints `Optional(...)`.
Co-authored-by: Chris Adamson <invalidname@gmail.com>
Co-authored-by: Chris Adamson <invalidname@gmail.com>
Co-authored-by: Chris Adamson <invalidname@gmail.com>
Co-authored-by: Chris Adamson <invalidname@gmail.com>
- GettingStarted.md: qualify terminationStatus link as ExecutionResult/terminationStatus
- OutputProtocol.md: remove standardOutput/standardError (they live on Execution, not OutputProtocol)
@heckj
heckj merged commit ac91dd2 into main Jul 24, 2026
45 checks passed
@heckj
heckj deleted the articles branch July 24, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants