Release process

Shipping is a single pipeline: release-train.yml runs on every push to main, builds the runtime once, and promotes that identical wasm through the networks. A second workflow, watch-mainnet-release.yml, watches the chain and publishes release artifacts once the mainnet upgrade actually executes.

The release train

push to main
  └─ srtool build (once per train)
       └─ deploy devnet ───── smoke-check devnet
            └─ deploy testnet ── smoke-check testnet ── publish SDK rc to PyPI
                 └─ propose mainnet upgrade (multisig)

Build once (srtool)

The train builds the runtime with srtool — a pinned Docker build environment that makes the wasm byte-reproducible. The artifact (subtensor.wasm + digest) is uploaded once and reused by every deploy job; promotion never rebuilds. Anyone can verify the hash independently by running scripts/srtool/build-srtool-image.sh followed by scripts/srtool/run-srtool.sh locally (see Repository scripts).

The ship lever: spec_version

Every deploy job first compares the built runtime's spec_version (runtime/src/lib.rs) with what the target chain is running, and no-ops unless the build is newer. Consequences:

• A merge without a spec_version bump builds and then does nothing — this is deliberate, and it's why the Spec Version Check on PRs exists (label no-spec-version-bump to opt out knowingly).
• Stale queued trains are harmless: they no-op at each guard.
• Rollback is not a concept here — you ship a newer fixed version.

Promotion gates

Human approval lives in GitHub environment settings, not workflow code:

Deploys to devnet and testnet are direct setCode transactions via the CI deploy key. After the on-chain spec_version is verified, the train moves the corresponding network mirror branch. That branch push triggers both Docker publishers from the exact deployed commit, updating the matching :devnet or :testnet tag in ghcr.io/raofoundation/subtensor and ghcr.io/raofoundation/subtensor-localnet. It does not move either package's :latest tag. The smoke suite then validates the live network while the independent image builds run.

The mainnet multisig ceremony

CI never upgrades mainnet directly. Before any on-chain action, the train reserves the immutable v<spec_version> tag at the exact source commit. Its final job then submits a multisig proposal: the CI key is one half of a 2-of-2 deployment multisig that holds a SudoUncheckedSetCode proxy on the sudo key. The triumvirate then approves the proposal 2-of-3 out-of-band — no GitHub credential can unilaterally change the mainnet runtime. Until they sign, nothing happens on chain.

For rehearsal, the mainnet-clone PR label spins up a live clone of mainnet running your runtime with a public endpoint (mainnet clone testing).

The release watcher

watch-mainnet-release.yml polls mainnet every 10 minutes. When an upgrade executes, it resolves the release-train artifact whose commit matches the immutable tag and whose runtime hash matches the finalized on-chain :code. The triumvirate's signatures already approved that exact runtime, so the watcher publishes the release without waiting for anyone:

1. mainnet mirror moved to the release commit. The push publishes the production Docker images :mainnet, :v<spec_version>, and :latest and the localnet images :mainnet and :v<spec_version>.
2. GitHub release v<spec_version>: the release train's proposal pre-release becomes the final, latest release with the verified runtime assets attached.

This job runs in the mainnet-release environment, which admits only main, has no reviewers, and holds only the mirror deploy key. It rechecks the finalized runtime before touching the mirror and never rewrites a final release.

The same run then requests two publications that still wait for a mainnet environment approval, because the PyPI trusted publisher and the production Vercel token are bound to that environment:

3. Python SDK + bittensor-core wheels to PyPI (trusted publishing, with PEP 740 provenance attestations)
4. Production website/docs to Vercel

They are requested once per release and never block steps 1 and 2. Each rechecks the finalized runtime after approval, so a late approval cannot publish packages or docs for a runtime mainnet has already replaced.

This ordering means the release always reflects what is actually running on mainnet, not what was merged.

PyPI is the terminal state for stable Python publication. The watcher requires the SDK wheel and source distribution plus the full bittensor-core Linux and macOS wheel matrix and source distribution. Every file must carry a PEP 740 attestation from this repository's mainnet release workflow. Dispatch the watcher to retry a failed, partial, or unapproved upload, even if the GitHub release already exists or the release-train artifact has expired. Retries require the release target, immutable tag, protected mainnet mirror, and finalized runtime bytes to agree. The SDK's committed X.Y.Z.dev0 version is stamped to X.Y.Z for this build. The release train rejects a base SDK or core version that already exists on PyPI before publishing release candidates. Releases through v432 predate automated stable Python publication and are handled as historical state. A missed website deployment can be redeployed with deploy-docs.yml (production).

Docker images

Every mirror branch publishes its own moving tag from the exact commit it points at: :devnet, :testnet, and :mainnet. Only the mainnet build also publishes the immutable :v<spec_version> tag, after checking that the commit is still the mirror head and carries that release tag. Production :latest moves only with it, so :latest is always the node running on mainnet. Every merge to main publishes the moving :main tag, so developers can prepare against merged features before they reach mainnet. Localnet :latest remains an alias for its :main image. To republish the current mainnet images, dispatch docker.yml with tag=mainnet and docker-localnet.yml with branch-or-tag=mainnet.

Hotfixes

The same pipeline applies: land the fix on main with a spec_version bump and let the train promote it. There is no side channel that skips devnet and testnet validation.