Getting Started

📖 Examples: For step-by-step examples with real outputs, see the Examples README.

Testsuite filenames
Command Examples
Testsuite examples
Output Examples
Integration Examples
- With CI/CD Pipeline

Testsuite filenames

Test suite files are discovered when they match either xprin.yaml or the *_xprin.yaml pattern.

Command Examples

How to Run Tests

xprin supports running tests in several ways. You can run a single file, multiple files, all files in a directory, recursively, or any combination of these:

# Single file
xprin test tests/basic_xprin.yaml

# Multiple files
xprin test tests/test1_xprin.yaml tests/test2_xprin.yaml

# Directory (non-recursive) - finds all *_xprin.yaml files in the directory
xprin test tests/

# Recursive - finds all *_xprin.yaml files in directory and subdirectories
xprin test tests/...

# Combination - mix any of the above in a single command
xprin test tests/test1_xprin.yaml tests/... tests/test2_xprin.yaml

Usual Command Options

-v/--verbose and -q/--quiet are independent: the first controls how much detail is shown for test runs, the second only suppresses the "[no testsuite files]" and "[no test cases found]" messages. You can use both together (e.g. -q -v for verbose test output without those skip messages).

# Verbose output (RUN, PASS/FAIL, phase details). Does not affect -q.
xprin test tests/basic_xprin.yaml -v

# Quiet: suppress "[no testsuite files]" and "[no test cases found]" (e.g. when running tests/... over dirs with testdata). Does not affect -v.
xprin test tests/... -q

# Both: verbose test output, no skip messages
xprin test tests/... -q -v

# Show rendered resources (requires -v)
xprin test tests/basic_xprin.yaml -v --show-render

# Show validation results (requires -v)
xprin test tests/basic_xprin.yaml -v --show-validate

# Show hooks execution (requires -v)
xprin test tests/advanced_xprin.yaml -v --show-hooks

# Show assertion results (requires -v)
xprin test tests/advanced_xprin.yaml -v --show-assertions

# Debug mode (shows detailed execution information)
xprin test tests/basic_xprin.yaml --debug

# Export artifacts (inputs and outputs of every test case) to a timestamped directory.
# Each run creates <dir>/xprin-artifacts-<YYYYMMDDHHMMSS>/ containing inputs/ and outputs/ per test case.
# After each suite, the path is printed to stderr:  artifacts: /path/to/xprin-artifacts-.../suite
xprin test tests/... --artifacts-dir /tmp/xprin-debug
xprin test tests/... -a /tmp/xprin-debug

Updating Golden Files

xprin update-goldens runs crossplane render on each test case that has assertions.diff or assertions.dyff entries, and writes the render output to the expected: paths. Use it to bootstrap or regenerate golden files after your Composition changes.

Test cases without assertions.diff/assertions.dyff are reported as SKIP immediately. For test cases that do have golden assertions, pre-test hooks run normally (they may affect render output), but validate, assertions.xprin, and post-test hooks are skipped.

For Claim-input test cases, a full-render golden file (an expected: path with no resource: set) is also reused to keep the Claim-derived XR's name stable across runs, instead of a new random suffix every time - see convert-claim-to-xr.md.

# Regenerate golden files for all tests in a directory
xprin update-goldens tests/

# Regenerate for a specific file
xprin update-goldens tests/my_test_xprin.yaml

Example output:

--- PASS: My test case (1.23s)
    written: tests/golden_full_render.yaml
    written: tests/golden_single_resource.yaml
ok      tests/my_test_xprin.yaml        1.234s

After running, commit the generated files so they become the baseline for xprin test.

Configuration Management

# Show current configuration
xprin config

# Check configuration and dependencies
xprin check

# Check with no output on success (exit code only; useful in scripts or CI)
xprin check -q

# Or use the config command (equivalent)
xprin config --check

# Use custom global config file
xprin -c /path/to/config.yaml config --check
xprin -c /path/to/config.yaml test tests/

Dependency display: xprin config prints dependency values exactly as written in your config file. If no config file is found, it reports "No configuration file provided." and exits. xprin check and xprin config --check resolve dependency paths (e.g. command names like crossplane are resolved to their absolute path via PATH) and append (from PATH) when the binary was found in PATH, so you can see which executable is used.

Testsuite examples

Simple Test Suite

A simple testsuite consists of an array of test cases, each one with its own inputs.

# tests/simple_xprin.yaml
tests:
- name: "Database Setup"
  inputs:
    xr: database-xr.yaml
    composition: database-composition.yaml
    functions: /path/to/functions
    crds:
    - /path/to/crds
- name: "Web Application"
  inputs:
    claim: webapp-claim.yaml
    composition: webapp-composition.yaml
    functions: /path/to/functions

It supports both XR and Claim as inputs. The Claim will be converted to XR using the xprin-helpers convert-claim-to-xr tool:

If the test case also sets patches.xrd (see Patching below), the converted XR's kind is resolved from the XRD instead of guessed as "X" prefixed to the Claim's kind.
The converted XR normally gets a new random name suffix on every run. If the test case also has a full-render assertions.diff/assertions.dyff entry, that golden file's XR name is reused instead, keeping it (and anything derived from it) stable across runs — see Updating Golden Files.

In this example:

the "Database Setup" test will run both render and validate
the "Web Application" test will run only render. The validate command will be skipped, because no CRDs are defined.

Common Inputs

It might be possible that the test cases in a testsuite file might have common inputs. To avoid duplication, we can define them in the common block:

# tests/common_inputs_xprin.yaml
common:
  inputs:
    functions: /path/to/functions
    crds:
    - /path/to/crds

tests:
- name: "Database Setup"
  inputs:
    xr: database-xr.yaml
    composition: database-composition.yaml
- name: "Web Application"
  inputs:
    claim: webapp-claim.yaml
    composition: webapp-composition.yaml

If an input is defined both in the common and the testcase level, the testcase prevails.

Patching

We can do simple patching via the xprin-helpers patch-xr tool.

# tests/patch_inputs_xprin.yaml
common:
  inputs:
    composition: database-composition.yaml
    functions: /path/to/functions
    crds:
    - /path/to/crds

tests:
- name: "EU Database Setup"
  patches:
    xrd: /path/to/crds/xrd.yaml
  inputs:
    xr: eu-database-xr.yaml
- name: "US Database Setup"
  patches:
    connection-secret: true
  inputs:
    claim: us-database-claim.yaml

Similarly to inputs, the patches can be defined either in the common or on the testcase level. In case they are defined in both, the testcase level prevails.

Hooks

Hooks can run arbitrary shell commands before and after each test.

Pre-test hooks

Use pre-test hooks for:

test environment setup (creating temp dirs, copying fixtures, exporting env)
pre-test validations (e.g. ensure .spec.k8s_version matches 1.3[0-9])
advanced patching of any input (see input template variables)

Example:

# tests/pre_post_hooks_xprin.yaml
common:
  hooks:
    pre-test:
    - name: "setup environment"
      run: ./scripts/setup_test_env.sh
    - name: "validate k8s version"
      run: ./scripts/validate_k8s_version.sh {{ .Inputs.XR }}
    - name: "patch XR size to small if unset"
      run: ./scripts/patch_xr_size.sh {{ .Inputs.XR }}

For more complex operations, you can use inline commands. Here's an example with a one-liner:

hooks:
  pre-test:
  - name: "validate and patch in one step"
    run: v=$(yq -r '.spec.k8s_version' {{ .Inputs.XR }}) && echo "$v" | grep -Eq '^1\.3[0-9]$' || { echo "Invalid version: $v"; exit 1; } && yq -i '.spec.parameters.size //= "small"' {{ .Inputs.XR }}

Notes:

{{ .Inputs.XR }} and other input variables are available in pre-test hooks.
See Template Variables for the full list.
Scripts can be stored in your repository and referenced by path, making hooks cleaner and reusable.

Post-test hooks

Use post-test hooks for:

cleaning up the test environment
printing outputs (see output template variables)
comparing inputs and outputs (e.g. diff -u {{ .Inputs.XR }} {{ .Outputs.XR }})
assertions via scripts or external tools (e.g. Kyverno Chainsaw, UpTest) using the render output

Example:

tests:
- name: "XR sanity"
  inputs:
    xr: xr.yaml
    composition: comp.yaml
  hooks:
    post-test:
    # 1) Print paths to outputs
    - name: "print outputs"
      run: ./scripts/print_outputs.sh {{ .Outputs.XR }} {{ .Outputs.Render }}

    # 2) Compare original XR vs rendered XR
    - name: "diff inputs vs outputs"
      run: ./scripts/diff_xr.sh {{ .Inputs.XR }} {{ .Outputs.XR }}

    # 3) Cleanup
    - name: "cleanup"
      run: ./scripts/cleanup_test_env.sh

Assertions

Assertions provide declarative validation of rendered resources without writing custom scripts. They are grouped by engine:

xprin: In-process assertions: count, existence, field type/value checks on rendered resources.
diff: Golden-file comparison using a unified diff
dyff: Golden-file comparison using dyff (structural YAML diff, document-aware).

Quick Example:

tests:
- name: "AWS Infra"
  inputs:
    xr: xr.yaml
    composition: composition.yaml
    functions: /path/to/functions
  assertions:
    xprin:
    - name: "Number of resources"
      type: "Count"
      value: 3
    - name: "SecurityGroup should exist"
      type: "Exists"
      resource: "SecurityGroup/platform-aws-sg"
    - name: "RDS backupRetentionPeriod should equal 1"
      type: "FieldValue"
      resource: "Cluster/platform-aws-rds"
      field: "spec.forProvider.backupRetentionPeriod"
      operator: "=="
      value: 1
    diff:
    - name: "Full render matches golden (using diff)"
      expected: golden_full_render.yaml
    - name: "Cluster resource matches golden (using diff)"
      expected: golden_single_resource.yaml
      resource: "Cluster/platform-aws-rds"
    dyff:
    - name: "Full render matches golden (using dyff)"
      expected: golden_full_render.yaml

For complete documentation on all assertion types, examples, and usage, see Assertions.

Golden files (diff/dyff expected: paths) can be bootstrapped or regenerated automatically with xprin update-goldens.

When to use assertions vs hooks:

Use assertions for declarative validation: xprin (count, existence, field checks), diff/dyff (golden-file comparison).
Use hooks for complex operations, external tool integration, or custom validation logic.

Test Chaining

Tests can depend on each other: a test case lists the IDs of the test cases it depends on in needs, and it runs after them, and only if they passed (otherwise it is skipped, with the reason printed under it). Only a test case with an id can be needed, and only if it is listed before.

tests:
- name: "Initial Database Setup"
  id: "db_setup"
  inputs:
    xr: database-xr.yaml
    composition: database-composition.yaml
    functions: /path/to/functions

- name: "Application Deployment"
  needs:
  - db_setup   # runs after db_setup, and only if it passed
  inputs:
    xr: app-xr.yaml
    composition: app-composition.yaml
    functions: /path/to/functions

A test case that needs another one can also use its outputs: tests with IDs have their outputs stored in a shared artifacts directory, and the test cases that list them in needs can reference them using the .Tests.{test-id} template variable.

This is useful for:

Sequential workflows: Using outputs from one test as inputs for the next
Incremental validation: Building up complexity across multiple test cases
Cross-test comparisons: Comparing outputs between different test scenarios

Example:

tests:
- name: "Initial Database Setup"
  id: "db_setup"
  inputs:
    xr: database-xr.yaml
    composition: database-composition.yaml
    functions: /path/to/functions
  hooks:
    post-test:
    - name: "save database outputs"
      run: ./scripts/save_test_outputs.sh {{ .Outputs.XR }} {{ .Outputs.RenderCount }}

- name: "Application Deployment"
  needs:
  - db_setup
  inputs:
    xr: app-xr.yaml
    composition: app-composition.yaml
    functions: /path/to/functions
  hooks:
    pre-test:
    - name: "verify database setup"
      run: ./scripts/verify_database_setup.sh {{ .Tests.db_setup.Outputs.Render }} {{ .Tests.db_setup.Outputs.XR }}

    post-test:
    - name: "compare outputs"
      run: ./scripts/compare_test_outputs.sh {{ .Tests.db_setup.Outputs.RenderCount }} {{ .Outputs.RenderCount }} {{ .Tests.db_setup.Outputs.Render }}

For detailed information about how test chaining works, see How It Works.

A test case can only use the outputs of the test cases it lists in needs. xprin used to find them automatically, which became deprecated in 0.6.0 and will stop working in 0.7.0: see Migrating from Automatic Detection.

Output Examples

Successful Test Run

$ xprin test tests/basic_xprin.yaml
ok      tests/basic_xprin.yaml     4.896s

Verbose Output

$ xprin test tests/basic_xprin.yaml -v
=== RUN   Database_Setup
--- PASS: Database_Setup (2.33s)
=== RUN   Web_Application
--- PASS: Web_Application (2.21s)
PASS
ok      tests/basic_xprin.yaml     4.546s

Failed Test Run

Failures are shown with section headers (Render:, Validate:, Assertions:, Pre-test Hooks:, Post-test Hooks:) and symbols [✓] pass, [x] fail, [!] error. Example:

$ xprin test tests/broken_xprin.yaml
--- FAIL: gcp (0.00s)
    [!] failed to expand or verify paths:
    [!] composition file not found: stat /path/to/comp.yaml: no such file or directory
--- FAIL: gcp_claim_aws_composition (0.22s)
    Render:
        [!] crossplane: error: composition's compositeTypeRef.kind (XAWSAccount) does not match XR's kind (XGCPAccount)
--- FAIL: EKS_clustername (2.15s)
    Validate:
        [!] could not find CRD/XRD for: kubernetes.crossplane.io/v1alpha2, Kind=Object
        Total 2 resources: 1 missing schemas, 1 success cases, 0 failure cases
        crossplane: error: cannot validate resources: could not validate all resources, schema(s) missing
FAIL
FAIL    tests/broken_xprin.yaml  4.863s
FAIL

Show Rendered Resources

$ xprin test -v --show-render tests/hello_xprin.yaml
=== RUN   onecluster
    Render:
        ├── XCluster/mycluster-72nd5
        └── Object/mycluster
--- PASS: onecluster (2.15s)
PASS
ok      tests/hello_xprin.yaml     4.422s

Show Validation Results

$ xprin test -v --show-validate tests/hello_xprin.yaml
=== RUN   onecluster
    Validate:
        [✓] mycluster.myorg.com/v1alpha1, Kind=XCluster, mycluster-qngkb validated successfully
        [✓] kubernetes.crossplane.io/v1alpha2, Kind=Object, mycluster validated successfully
        Total 2 resources: 0 missing schemas, 2 success cases, 0 failure cases
--- PASS: onecluster (3.03s)
PASS
ok      tests/hello_xprin.yaml     5.193s

Integration Examples

With CI/CD Pipeline

# .github/workflows/test.yml
name: Test Compositions
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-go@v3
        with:
          go-version: '1.24'
      - name: Install xprin
        run: |
          go install github.com/crossplane-contrib/xprin/cmd/xprin@latest
          go install github.com/crossplane-contrib/xprin/cmd/xprin-helpers@latest
      - name: Check xprin dependencies
        run: xprin check
      - name: Run tests
        run: xprin test tests/

Next Steps:

Learn by doing with step-by-step Examples that build complexity progressively
Or jump to the Test Suite Specification for a complete reference of all available options

Navigation Menu

Repository navigation

getting-started.md

getting-started.md

Getting Started

Table of Contents

Testsuite filenames

Command Examples

How to Run Tests

Usual Command Options

Updating Golden Files

Configuration Management

Testsuite examples

Simple Test Suite

Common Inputs

Patching

Hooks

Pre-test hooks

Post-test hooks

Assertions

Test Chaining

Output Examples

Successful Test Run

Verbose Output

Failed Test Run

Show Rendered Resources

Show Validation Results

Integration Examples

With CI/CD Pipeline

FilesExpand file tree

getting-started.md

Latest commit

History

getting-started.md

File metadata and controls

Getting Started

Table of Contents

Testsuite filenames

Command Examples

How to Run Tests

Usual Command Options

Updating Golden Files

Configuration Management

Testsuite examples

Simple Test Suite

Common Inputs

Patching

Hooks

Pre-test hooks

Post-test hooks

Assertions

Test Chaining

Output Examples

Successful Test Run

Verbose Output

Failed Test Run

Show Rendered Resources

Show Validation Results

Integration Examples

With CI/CD Pipeline