-
Notifications
You must be signed in to change notification settings - Fork 58
fix(dashmate)!: give Debian packages versions apt can order #4282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shumkov
wants to merge
10
commits into
v4.2-dev
Choose a base branch
from
feat/dashmate/deb-version-ordering
base: v4.2-dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
c140da5
fix(dashmate)!: give Debian packages versions apt can order
shumkov 33e4e84
ci(dashmate): make releases verifiable and safe to hold a signing key
shumkov c8ed516
ci: pin the remaining mutable third-party action refs to commit shas
shumkov 2c047e8
ci(dashmate): bind published hashes to names, and order the baseline …
shumkov 9d89691
ci(dashmate): re-check for a signature immediately before replacing c…
shumkov a269069
docs(dashmate): install the package that was downloaded, not every match
shumkov 868640c
ci(dashmate): stop the release token outliving the step that needs it
shumkov 59fd78e
refactor(dashmate): move the release packaging scripts next to the pa…
shumkov 609f597
build: point every publishable package at the repository it lives in
shumkov 68aed13
fix(dashmate): report a missing dpkg before downloading anything to c…
shumkov File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| cmd_usage="Usage: check-built-deb-version.sh VALIDATED_VERSION [DIST_PATH] [WORK_PATH] | ||
|
|
||
| Checks that every deb in DIST_PATH carries VALIDATED_VERSION in its control field. | ||
|
|
||
| The version gate validates a version derived from the tag; nothing else proves the deb | ||
| that actually gets built carries it. Without this check the gate's verdict applies to a | ||
| prediction rather than to the bytes that ship. | ||
|
|
||
| DIST_PATH directory the packages were built into, default packages/dashmate/dist | ||
| WORK_PATH directory for intermediate files, default \$RUNNER_TEMP or \$TMPDIR | ||
|
|
||
| EXIT CODES: | ||
| 0 every built deb carries the validated version | ||
| 1 a deb carries another version, or there was nothing to check | ||
| 2 wrong arguments | ||
| " | ||
|
|
||
| VALIDATED_VERSION="${1:-}" | ||
|
|
||
| DIR_PATH=$(dirname "$(realpath "$0")") | ||
|
|
||
| DIST_PATH="${2:-${DIR_PATH}/../dist}" | ||
| WORK_PATH="${3:-${RUNNER_TEMP:-${TMPDIR:-/tmp}}}" | ||
|
|
||
| if [ -z "${VALIDATED_VERSION}" ]; then | ||
| echo "::error::The version gate did not publish a validated version" | ||
| echo "$cmd_usage" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| debs="${WORK_PATH}/built-debs" | ||
| find "${DIST_PATH}" -type f -name '*.deb' > "${debs}" | ||
| if [ ! -s "${debs}" ]; then | ||
| echo "::error::No deb was produced to check" | ||
| exit 1 | ||
| fi | ||
| while IFS= read -r deb; do | ||
| built_version="$(dpkg-deb -f "${deb}" Version)" | ||
| if [ "${built_version}" != "${VALIDATED_VERSION}" ]; then | ||
| echo "::error::${deb} carries version ${built_version}, but the gate validated ${VALIDATED_VERSION}; the packaging scheme and the gate disagree" | ||
| exit 1 | ||
| fi | ||
| echo "${deb} carries the validated version ${built_version}" | ||
| done < "${debs}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -e | ||
|
|
||
| cmd_usage="Usage: check-deb-version.sh NEW_VERSION PREVIOUS_VERSION | ||
|
|
||
| Exits successfully only when NEW_VERSION sorts strictly above PREVIOUS_VERSION under | ||
| dpkg's version comparison, which is what decides whether apt offers a release as an | ||
| upgrade at all. | ||
|
|
||
| Both arguments are Debian package versions ([EPOCH:]UPSTREAM[-REVISION]), not semver | ||
| tags. Translate a semver version first: | ||
|
|
||
| packages/dashmate/scripts/check-deb-version.sh \\ | ||
| \"\$(node packages/dashmate/scripts/deb-version.js 4.1.0-rc.4)\" \\ | ||
| \"\$(node packages/dashmate/scripts/deb-version.js 4.1.0-rc.3)\" | ||
|
|
||
| EXIT CODES: | ||
| 0 new version sorts above the previous one | ||
| 1 new version is equal to or below the previous one | ||
| 2 wrong arguments | ||
| 3 dpkg is unavailable, so the comparison could not be made | ||
| " | ||
|
|
||
| NEW_VERSION="$1" | ||
| PREVIOUS_VERSION="$2" | ||
|
|
||
| if [ -z "$NEW_VERSION" ] || [ -z "$PREVIOUS_VERSION" ] | ||
| then | ||
| echo "$cmd_usage" >&2 | ||
| exit 2 | ||
| fi | ||
|
|
||
| # dpkg is the only authority on its own ordering rules, and the rules are subtle enough | ||
| # (`~` below the empty string, digits and letters ordered differently) that guessing here | ||
| # would defeat the point of the check. Refuse to answer instead of answering wrongly. | ||
| if ! command -v dpkg > /dev/null 2>&1 | ||
| then | ||
| echo "check-deb-version.sh: dpkg not found, cannot compare Debian versions." >&2 | ||
| echo "Run this on a Debian based host or inside a container that has dpkg." >&2 | ||
| exit 3 | ||
| fi | ||
|
|
||
| if dpkg --compare-versions "$NEW_VERSION" gt "$PREVIOUS_VERSION" | ||
| then | ||
| echo "$NEW_VERSION sorts above $PREVIOUS_VERSION" | ||
| exit 0 | ||
| fi | ||
|
|
||
| if dpkg --compare-versions "$NEW_VERSION" eq "$PREVIOUS_VERSION" | ||
| then | ||
| echo "$NEW_VERSION is the version that is already published." >&2 | ||
| echo "Set DASHMATE_DEB_REVISION to the next Debian revision to rebuild it." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "$NEW_VERSION sorts below $PREVIOUS_VERSION, so apt would refuse it as a downgrade." >&2 | ||
| echo "Set DASHMATE_DEB_EPOCH to overtake a version published under a different scheme." >&2 | ||
| exit 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| cmd_usage="Usage: check-release-deb-version.sh CURRENT_TAG | ||
|
|
||
| Checks that the deb built for CURRENT_TAG sorts above the deb of the last release | ||
| operators were offered, which is what decides whether apt takes it as an upgrade. | ||
|
|
||
| The baseline release is chosen by deb-release-baseline.js, its version is read from | ||
| the published package's own control field, and the two are compared by | ||
| check-deb-version.sh. | ||
|
|
||
| ENVIRONMENT: | ||
| GITHUB_REPOSITORY owner/name of the repository to read releases from | ||
| GH_TOKEN token the GitHub CLI authenticates with | ||
| GITHUB_OUTPUT step output file; the validated version is appended to it as | ||
| validated_version when set | ||
|
|
||
| EXIT CODES: | ||
| 0 the new version sorts above the baseline, or there is no baseline | ||
| 1 the new version is equal to or below the baseline, or the check could not run | ||
| 2 wrong arguments | ||
| 3 dpkg is unavailable, so the comparison could not be made | ||
| " | ||
|
|
||
| CURRENT_TAG="${1:-}" | ||
|
|
||
| if [ -z "${CURRENT_TAG}" ] | ||
| then | ||
| echo "$cmd_usage" >&2 | ||
| exit 2 | ||
| fi | ||
|
|
||
| if [ -z "${GITHUB_REPOSITORY:-}" ] | ||
| then | ||
| echo "check-release-deb-version.sh: GITHUB_REPOSITORY is not set." >&2 | ||
| echo "$cmd_usage" >&2 | ||
| exit 2 | ||
| fi | ||
|
|
||
| # Both tools are needed before anything is downloaded: the baseline version is | ||
| # read with dpkg-deb and compared with dpkg. Checking here keeps a host without | ||
| # them reporting the documented "could not compare" status rather than dying on | ||
| # a missing command part way through. | ||
| for tool in dpkg dpkg-deb | ||
| do | ||
| if ! command -v "$tool" > /dev/null 2>&1 | ||
| then | ||
| echo "check-release-deb-version.sh: $tool not found, cannot compare Debian versions." >&2 | ||
| echo "Run this on a Debian based host or inside a container that has dpkg." >&2 | ||
| exit 3 | ||
| fi | ||
| done | ||
|
|
||
| DIR_PATH=$(dirname "$(realpath "$0")") | ||
|
|
||
| compare="${DIR_PATH}/check-deb-version.sh" | ||
| translate="${DIR_PATH}/deb-version.js" | ||
| select_baseline="${DIR_PATH}/deb-release-baseline.js" | ||
| if [ ! -x "${compare}" ] || [ ! -f "${translate}" ] || [ ! -f "${select_baseline}" ]; then | ||
| echo "::error::${compare}, ${translate} or ${select_baseline} is missing" | ||
| exit 1 | ||
| fi | ||
| # The comparison runs on Debian versions, never on the semver tags: | ||
| # "4.1.0-1" is valid as both and means something different in each. | ||
| new_version="$(node "${translate}" "${CURRENT_TAG}")" | ||
| # Published so the packaging job can prove the deb it actually builds | ||
| # carries the version validated here, rather than both jobs | ||
| # independently predicting it from the tag. | ||
| if [ -n "${GITHUB_OUTPUT:-}" ]; then | ||
| echo "validated_version=${new_version}" >> "${GITHUB_OUTPUT}" | ||
| fi | ||
|
|
||
| # Every release is fetched, not just the first page: the repository has | ||
| # several hundred of them, and a single page silently hides older | ||
| # lines. Which one becomes the baseline is decided by the script rather | ||
| # than by the API's default ordering, so the choice of predecessor is | ||
| # not an undocumented implementation detail. | ||
| baseline="$(gh api --paginate --slurp "repos/${GITHUB_REPOSITORY}/releases?per_page=100" \ | ||
| | node "${select_baseline}" "${CURRENT_TAG}")" | ||
|
|
||
| if [ -z "${baseline}" ]; then | ||
| echo "::notice::No published deb to compare ${CURRENT_TAG} against" | ||
| exit 0 | ||
| fi | ||
|
|
||
| baseline_tag="${baseline%%$'\t'*}" | ||
| baseline_asset="${baseline#*$'\t'}" | ||
|
|
||
| # Read the baseline from the deb's own control field: the only version | ||
| # apt looks at, the only one a server-side rename of the asset cannot | ||
| # alter, and what operators actually installed rather than what the tag | ||
| # would produce today. | ||
| mkdir -p baseline-deb | ||
| gh release download "${baseline_tag}" --repo "${GITHUB_REPOSITORY}" \ | ||
| --pattern "${baseline_asset}" --dir baseline-deb --clobber | ||
| baseline_version="$(dpkg-deb -f "baseline-deb/${baseline_asset}" Version)" | ||
|
shumkov marked this conversation as resolved.
|
||
| if [ -z "${baseline_version}" ]; then | ||
| echo "::error::Could not read the Version field from ${baseline_asset} in ${baseline_tag}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Comparing ${CURRENT_TAG} (${new_version}) against ${baseline_asset} from ${baseline_tag} (${baseline_version})" | ||
| status=0 | ||
| "${compare}" "${new_version}" "${baseline_version}" || status=$? | ||
| if [ "${status}" -eq 1 ]; then | ||
| echo "::error::${new_version} does not sort above ${baseline_version}, so apt would refuse this release as a downgrade or report it as already the newest version. Rebuilding an already published version needs DASHMATE_DEB_REVISION; re-releasing a version whose predecessor carried a git sha in its upstream part needs DASHMATE_DEB_EPOCH=1. Both are read by packages/dashmate/scripts/deb-version.js and must be set for the packaging job as well." | ||
| fi | ||
| exit "${status}" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: dashpay/platform
Length of output: 797
Add
repository.directoryto the Dashmate package metadata.packages/dashmate/package.jsondoes not setrepository.directorytopackages/dashmate. Add this field so npm identifies the package subdirectory in the monorepo.🤖 Prompt for AI Agents