New project: opencontainers/oras

[jdolitsky]

Summary

Adopt the ORAS project located at deislabs/oras.

Overview

ORAS is a CLI that can publish arbitrary content to an OCI registry, with special features for setting mediatypes on manifest configs and on content.

Note: the manifest mediatype itself is always application/vnd.oci.image.manifest.v1+json.

Example - uploading rockets, a brand new type of package:

# Create a thing
printf '🚀' > rocket.txt

# Create a manifest config
printf '{"RocketVersion":"v0.1.0"}' > rocket-config.json

# Upload your thing with a custom mediatype
oras push localhost:5000/mystuff/myrocket:v0.1.0 rocket.txt:text/plain \
  --manifest-config rocket-config.json:application/vnd.acme.rocket.config.v1+json

See manifest created:

$ curl -s -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
    http://localhost:5000/v2/mystuff/myrocket/manifests/v0.1.0 | jq
{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.acme.rocket.config.v1+json",
    "digest": "sha256:310175f34d2d4d5cba3418be06ddd1ef948147d729516d78318ec7f5c2d83d49",
    "size": 26
  },
  "layers": [
    {
      "mediaType": "text/plain",
      "digest": "sha256:ebbc0b2870eb323f2b6cffa5c493ceef81ae7eb36afc73d4e0367301631daec5",
      "size": 4,
      "annotations": {
        "org.opencontainers.image.title": "rocket.txt"
      }
    }
  ]
}

Get that thing:

$ curl -s http://localhost:5000/v2/mystuff/myrocket/blobs/sha256:ebbc0b2870eb323f2b6cffa5c493ceef81ae7eb36afc73d4e0367301631daec5
🚀

Additional Usage

ORAS is built primarily on top of Go packages provided by containerd, but it also imports packages from the docker/cli, which enables "docker-style" auth login:

oras login -u username -p password localhost:5000 -c rocket-creds.json

There are also public Go packages available to build on top of ORAS. The following is the equivalent of the rocket example with the CLI above, but in Go:

package main

import (
	"context"
	"fmt"

	"github.com/containerd/containerd/remotes/docker"
	"github.com/deislabs/oras/pkg/content"
	"github.com/deislabs/oras/pkg/oras"
	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)

func main() {
	ctx := context.Background()
	resolver := docker.NewResolver(docker.ResolverOptions{})
	store := content.NewMemoryStore()

	registryRootURL := "localhost:5000"
	registryNamespace := "mystuff/myrocket"

	rocketVersion := "v0.1.0"
	rocketFileName := "rocket.txt"
	rocketMediaType := "text/plain"
	rocketContent := []byte("🚀")
	rocketDescriptor := store.Add(rocketFileName, rocketMediaType, rocketContent)

	rocketConfigMediaType := "application/vnd.acme.rocket.config.v1+json"
	rocketConfigContent := []byte(fmt.Sprintf("{\"RocketVersion\":\"%s\"}", rocketVersion))
	rocketConfigDescriptor := store.Add("", rocketConfigMediaType, rocketConfigContent)

	ref := fmt.Sprintf("%s/%s:%s", registryRootURL, registryNamespace, rocketVersion)
	_, err := oras.Push(ctx, resolver, ref, store, []ocispec.Descriptor{rocketDescriptor},
		oras.WithConfig(rocketConfigDescriptor))
	if err != nil {
		panic(err)
	}

	fmt.Println("Pushed to", ref)
	fmt.Printf("\nTry:\n\ncurl -s -H 'Accept: application/vnd.oci.image.manifest.v1+json' \\\n" +
		"    %s/v2/%s/manifests/%s | jq\n", registryRootURL, registryNamespace, rocketVersion)
}

You can see all features in the project README.

Adoption

The following projects are already successfully using ORAS to work with custom artifacts:

• Helm
• Conftest
• Singularity

Why move it into opencontainers?

For a few reasons:

• Provide end users a method to publish and retrieve any type of content to/from an OCI registry
• Provide a reference implementation for the in-progress artifacts spec
• Expand awareness of the project to a broader audience
• Encourage more community contributions

[caniszczyk]

I'm open to it, @opencontainers/tob has the final say, thoughts?

[ArangoGutierrez]

++ I love this!

[vbatts]

right on. I'm indifferent to this yet. I think I saw the tool as more of a conversation starter for the opencontainers/artifacts, rather than a tool that folks would incorporate.
I'll have to read more sides of the story.

[jdolitsky]

I think it is definitely tied to the artifacts work, and if it makes more sense we could rebrand it to the "artifacts" CLI and place it into that repo.

And of course, if the decision is made to adopt it, we can discuss a redesign/rewrite of the tool's API to better meet the needs of OCI as it relates to artifacts and more generalized use of distribution-spec.

In any case, I'm under the opinion that it's better to have the code somewhere in this org (vs. deislabs) in order to continue to prove out the spec(s) as they evolve.

[SteveLasker]

@jdolitsky and @shizhMSFT initially built ORAS to support Helm in a registry, which led to the Artifacts project. ORAS can push and pull any OCI thing to/from an OCI distribution spec compliant registry.

What I really liked about their design was it can be a cli, or a go library. It supports Helm, Singularity, OPA and a few others as a library today, and a few we likely don't know about, But, we've also discussed how can we we push/pull OCI Images, which it also supports, as you just specify the manifest.config.mediaType.

Whether it's named ORAS (OCI Registry As Storage) or something else is something to discuss. I really like Josh's logo, but I suppose it could also be something like ORaA (OCI Registry as Artifacts).

As a project moves into a foundation, the question is whether it will be maintained? Is it just one person, one company? ORAS has multiple contributors, with @jdolitsky and @shizhMSFT as the most active. I do suspect we'll have others as we finalize the artifacts spec and other registries complete their support.

I would keep it as a separate repo as it has builds that are triggered that should be separate from the artifacts spec.

So, LGTM

[ArangoGutierrez]

My 2 cents: I haven't really look at ORAS code in depth, but I have played with the functionality and is feature set. and I like the overall idea, I mean, an object store that acts as a container registry can open the door for OCI 2.0 image and other ideas. like not using TAR files to store layers, but a new packaging paradigm as is being explored for OCIv2

[garethr]

As a user of ORAS, for Conftest, I'm supportive of this. I found it easy to integrate and not to have caused any issues so far. It's a good way for Go folks to quickly see the potential of the artifact work.

Some quick background, as it would be good to document some patterns for future users. This was the thread where we settled on OPA media types. open-policy-agent/opa#1413. Advice and guidance here would be useful for future implementers. ORAS makes that easy to implement once you determine the media types you want.

[caniszczyk]

@jdolitsky can you turn this into a PR like #67 ?

we can move the discussion there

[jdolitsky]

will do, yes

[jdolitsky]

This has been created as a PR: #68

Closing this issue - please continue discussion there