Skip to content

add Kubernetes binary classifier and update download tool - #5147

Open
witchcraze wants to merge 3 commits into
anchore:mainfrom
witchcraze:5045_add-k8s-binary-classifier
Open

add Kubernetes binary classifier and update download tool#5147
witchcraze wants to merge 3 commits into
anchore:mainfrom
witchcraze:5045_add-k8s-binary-classifier

Conversation

@witchcraze

Copy link
Copy Markdown
Contributor

Description

This PR added Kubernetes binary classifier.
And this PR also updated the development testing tool (binary/internal/manager) accordingly.

Many older Kubernetes binaries are not available as public Docker images, which meant that test data could not be collected using the existing configuration. To address this issue, I added a from-urls feature that directly downloads file from URLs.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (please discuss with the team first; Syft is 1.0 software and we won't accept breaking changes without going to 2.0)
  • Documentation (updates the documentation)
  • Chore (improve the developer experience, fix a test flake, etc, without changing the visible behavior of Syft)
  • Performance (make Syft run faster or use less memory, without changing visible behavior much)

Checklist

  • I have added unit tests that cover changed behavior
  • I have tested my code in common scenarios and confirmed there are no regressions
  • I have added comments to my code, particularly in hard-to-understand sections

Issue references

Fixes #5045


Details
$ go test -v ./syft/pkg/cataloger/binary -run Test_Cataloger_PositiveCases/kube
=== RUN   Test_Cataloger_PositiveCases
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-alpha.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.36.0-alpha.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-beta.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.36.0-beta.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-rc.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.36.0-rc.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.36.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.28.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.28.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.25.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.25.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.6.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.6.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.5.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kube-apiserver/1.5.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.36.0-alpha.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.36.0-alpha.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.36.0-beta.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.36.0-beta.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.36.0-rc.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.36.0-rc.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.35.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.35.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.28.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.28.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.25.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.25.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.6.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.6.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.5.0/linux-amd64
    snippet_or_binary.go:50: using snippet for "kubelet/1.5.0/linux-amd64"
--- PASS: Test_Cataloger_PositiveCases (1.74s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-alpha.0/linux-amd64 (0.11s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-beta.0/linux-amd64 (0.11s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-rc.0/linux-amd64 (0.11s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0/linux-amd64 (0.11s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.28.0/linux-amd64 (0.12s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.25.0/linux-amd64 (0.13s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.6.0/linux-amd64 (0.15s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.5.0/linux-amd64 (0.13s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.36.0-alpha.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.36.0-beta.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.36.0-rc.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.35.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.28.0/linux-amd64 (0.10s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.25.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.6.0/linux-amd64 (0.13s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.5.0/linux-amd64 (0.12s)
PASS
ok      github.com/anchore/syft/syft/pkg/cataloger/binary       1.766s

$ go test -v ./syft/pkg/cataloger/binary -run Test_Cataloger_PositiveCases/kube -must-use-original-binaries
=== RUN   Test_Cataloger_PositiveCases
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-alpha.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.36.0-alpha.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-beta.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.36.0-beta.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-rc.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.36.0-rc.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.36.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.36.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.28.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.28.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.25.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.25.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.6.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.6.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kube-apiserver/1.5.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kube-apiserver/1.5.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.36.0-alpha.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.36.0-alpha.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.36.0-beta.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.36.0-beta.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.36.0-rc.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.36.0-rc.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.35.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.35.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.28.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.28.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.25.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.25.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.6.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.6.0/linux-amd64"
=== RUN   Test_Cataloger_PositiveCases/kubelet/1.5.0/linux-amd64
    snippet_or_binary.go:62: forcing the use of the original binary for "kubelet/1.5.0/linux-amd64"
--- PASS: Test_Cataloger_PositiveCases (10.28s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-alpha.0/linux-amd64 (0.10s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-beta.0/linux-amd64 (0.11s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0-rc.0/linux-amd64 (0.10s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.36.0/linux-amd64 (0.29s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.28.0/linux-amd64 (0.29s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.25.0/linux-amd64 (0.73s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.6.0/linux-amd64 (2.33s)
    --- PASS: Test_Cataloger_PositiveCases/kube-apiserver/1.5.0/linux-amd64 (2.18s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.36.0-alpha.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.36.0-beta.0/linux-amd64 (0.10s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.36.0-rc.0/linux-amd64 (0.09s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.35.0/linux-amd64 (0.24s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.28.0/linux-amd64 (0.25s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.25.0/linux-amd64 (0.52s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.6.0/linux-amd64 (1.59s)
    --- PASS: Test_Cataloger_PositiveCases/kubelet/1.5.0/linux-amd64 (1.25s)
PASS
ok      github.com/anchore/syft/syft/pkg/cataloger/binary       10.297s

I can not find general pattern of v1.10.0

$ for i in $(seq 5 36); do  echo "===== v1.$i.0 =====";  go run ./cmd/syft -q "registry.k8s.io/kube-apiserver:v1.$i.0" | grep -E 'apiserver |k8s.io/kubernetes ';  done
===== v1.5.0 =====
kube-apiserver  1.5.0    binary
===== v1.6.0 =====
kube-apiserver  1.6.0    binary
===== v1.7.0 =====
kube-apiserver  1.7.0    binary
===== v1.8.0 =====
kube-apiserver  1.8.0    binary
===== v1.9.0 =====
kube-apiserver  1.9.0    binary
===== v1.10.0 =====
===== v1.11.0 =====
kube-apiserver  1.11.0   binary
===== v1.12.0 =====
kube-apiserver  1.12.0   binary
===== v1.13.0 =====
kube-apiserver          1.13.0                 binary
===== v1.14.0 =====
kube-apiserver          1.14.0                 binary
===== v1.15.0 =====
kube-apiserver          1.15.0                 binary
===== v1.16.0 =====
kube-apiserver          1.16.0                 binary
===== v1.17.0 =====
kube-apiserver          1.17.0                binary
===== v1.18.0 =====
kube-apiserver          1.18.0                binary
===== v1.19.0 =====
kube-apiserver                         1.19.0           binary
===== v1.20.0 =====
kube-apiserver                         1.20.0           binary
===== v1.21.0 =====
kube-apiserver                         1.21.0           binary
===== v1.22.0 =====
kube-apiserver                         1.22.0           binary
===== v1.23.0 =====
kube-apiserver                         1.23.0           binary
===== v1.24.0 =====
k8s.io/kubernetes/cmd/kube-apiserver   UNKNOWN          go-module
kube-apiserver                         1.24.0           binary
===== v1.25.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v1.25.0                                go-module
kube-apiserver                                                               1.25.0                                 binary
===== v1.26.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v1.26.0                                go-module
kube-apiserver                                                               1.26.0                                 binary
===== v1.27.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v1.27.0                                go-module
kube-apiserver                                                               1.27.0                                 binary
===== v1.28.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v1.28.0                                go-module
kube-apiserver                                                               1.28.0                                 binary
===== v1.29.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v1.29.0                                go-module
kube-apiserver                                                               1.29.0                                 binary
===== v1.30.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v0.0.0-20240417172702-7c48c2bd72b9     go-module
kube-apiserver                                                               1.30.0                                 binary
===== v1.31.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                go-module
k8s.io/apiserver                                                             UNKNOWN                                go-module
k8s.io/kubernetes                                                            v0.0.0-20240813072848-9edcffcde559     go-module
kube-apiserver                                                               1.31.0                                 binary
===== v1.32.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                               go-module
k8s.io/apiserver                                                             UNKNOWN                               go-module
k8s.io/kubernetes                                                            v0.0.0-20241211175915-70d3cc986aa8    go-module
kube-apiserver                                                               1.32.0                                binary
===== v1.33.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                               go-module
k8s.io/apiserver                                                             UNKNOWN                               go-module
k8s.io/kubernetes                                                            v1.33.0                               go-module
kube-apiserver                                                               1.33.0                                binary
===== v1.34.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                               go-module
k8s.io/apiserver                                                             UNKNOWN                               go-module
k8s.io/kubernetes                                                            v1.34.0                               go-module
kube-apiserver                                                               1.34.0                                binary
===== v1.35.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                               go-module
k8s.io/apiserver                                                             UNKNOWN                               go-module
k8s.io/kubernetes                                                            v1.35.0                               go-module
kube-apiserver                                                               1.35.0                                binary
===== v1.36.0 =====
k8s.io/apiextensions-apiserver                                               UNKNOWN                                 go-module
k8s.io/apiserver                                                             UNKNOWN                                 go-module
k8s.io/kubernetes                                                            v1.36.0                                 go-module
kube-apiserver                                                               1.36.0                                  binary


$ for i in $(seq 17 35); do echo "===== v1.$i.0 ====="; go run ./cmd/syft -q "kindest/node:v1.$i.0" | grep -E 'kubelet |k8s.io/kubernetes '; done
===== v1.17.0 =====
k8s.io/kubernetes                                  v1.17.0                               go-module
kubelet                                            1.17.0                                binary
===== v1.18.0 =====
k8s.io/kubernetes                                  v1.17.0                               go-module
kubelet                                            1.18.0                                binary
===== v1.19.0 =====
k8s.io/kubernetes                                  v1.18.0                                go-module
kubelet                                            1.19.0                                 binary
===== v1.20.0 =====
k8s.io/kubernetes                                  v1.18.0                                go-module
kubelet                                            1.20.0                                 binary
===== v1.21.0 =====
exit status 1
===== v1.22.0 =====
k8s.io/kubernetes                                    v1.21.0                                            go-module
kubelet                                              1.22.0                                             binary
===== v1.23.0 =====
k8s.io/kubernetes                                                            v1.22.0                                            go-module
kubelet                                                                      1.23.0                                             binary
===== v1.24.0 =====
k8s.io/kubernetes                                                            v1.24.0                                            go-module
k8s.io/kubernetes/cmd/kubelet                                                UNKNOWN                                            go-module
kubelet                                                                      1.24.0                                             binary
===== v1.25.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v1.24.0                                            go-module
k8s.io/kubernetes                                                                       v1.25.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.25.0                                             binary
===== v1.26.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v1.26.0                                            go-module  (+3 duplicates)
kubelet                                                                                 1.26.0                                             binary
===== v1.27.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v1.26.0                                            go-module
k8s.io/kubernetes                                                                       v1.27.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.27.0                                             binary
===== v1.28.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v1.27.0                                            go-module
k8s.io/kubernetes                                                                       v1.28.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.28.0                                             binary
===== v1.29.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v1.27.0                                            go-module
k8s.io/kubernetes                                                                       v1.29.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.29.0                                             binary
===== v1.30.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v0.0.0-20240417172702-7c48c2bd72b9                 go-module  (+2 duplicates)
k8s.io/kubernetes                                                                       v1.29.0-rc.2                                       go-module
k8s.io/kubernetes/staging/src/k8s.io/kubelet                                            v0.0.0-20231207112802-e4636d05bc32                 go-module
kubelet                                                                                 1.30.0                                             binary
===== v1.31.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubernetes                                                                       v0.0.0-20240813072848-9edcffcde559                 go-module  (+2 duplicates)
k8s.io/kubernetes                                                                       v1.30.0                                            go-module
k8s.io/kubernetes/staging/src/k8s.io/kubelet                                            v0.0.0-20240417172702-7c48c2bd72b9                 go-module
kubelet                                                                                 1.31.0                                             binary
===== v1.32.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubelet                                                                          v0.32.0-rc.2                                       go-module
k8s.io/kubernetes                                                                       v0.0.0-20241211175915-70d3cc986aa8                 go-module  (+2 duplicates)
kubelet                                                                                 1.32.0                                             binary
===== v1.33.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubelet                                                                          v0.31.2                                            go-module
k8s.io/kubelet                                                                          v0.32.0-rc.2                                       go-module
k8s.io/kubernetes                                                                       v1.33.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.33.0                                             binary
===== v1.34.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubelet                                                                          v0.32.3                                            go-module
k8s.io/kubelet                                                                          v0.33.0-rc.0                                       go-module
k8s.io/kubernetes                                                                       v1.34.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.34.0                                             binary
===== v1.35.0 =====
k8s.io/kubelet                                                                          UNKNOWN                                            go-module  (+1 duplicate)
k8s.io/kubelet                                                                          v0.33.0-rc.0                                       go-module
k8s.io/kubernetes                                                                       v1.35.0                                            go-module  (+2 duplicates)
kubelet                                                                                 1.35.0                                             binary

Signed-off-by: witchcraze <witchcraze@gmail.com>
Signed-off-by: witchcraze <witchcraze@gmail.com>
Comment thread syft/pkg/cataloger/binary/testdata/config.yaml
Signed-off-by: witchcraze <witchcraze@gmail.com>
@witchcraze
witchcraze requested a review from kzantow August 9, 2026 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Catalog Kubernetes binary

2 participants