Skip to content

cuda.bindings: configure independent CUDA 12 and CUDA 13 packages - #3174

Draft
rwgk wants to merge 11 commits into
mainfrom
rwgk/cb12-on-main-03-package-selection
Draft

rwgk wants to merge 11 commits into
mainfrom
rwgk/cb12-on-main-03-package-selection

Conversation

@rwgk

@rwgk rwgk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make the two bindings source roots explicit without changing the distribution name or Python namespace. Both build cuda-bindings / cuda.bindings, in separate environments.

  • Record exactly one current and one maintenance bindings root, their toolkit pins, and the shared Python matrix in ci/versions.yml. Currently these are CUDA 13.4 and CUDA 12.9; this is a two-line policy, not support for arbitrary numbers of release lines.
  • Use standard setuptools-scm parsing with one tag selector per release line. The standalone metapackage carries its selectors in cuda_python/pyproject.toml; the metadata hook checks them against the bindings roots and registry.
  • Preserve ordinary source installs: cuda-python defaults to CUDA 13; CUDA_PYTHON_BUILD_MAJOR=12 selects CUDA 12. Dependencies follow the resolved package version, so a CUDA 12 sdist rebuilt outside Git keeps CUDA 12 dependencies without requiring that variable. The CUDA 12 metapackage directly requires the matching bindings; CUDA 13 additionally declares Core and Pathfinder dependencies.
  • Require Python 3.11 or newer consistently, including CUDA 12 metadata and current support documentation. Add a CUDA 12 Pixi workspace and lockfile alongside the existing per-package environments.
  • Keep the checks that catch disagreement between packaging, toolkit pins, Pixi, and Core floors. Toolkit versions use structured numeric data internally. The registry metadata hook checks toolkit pins against Core floors; the separate Core hook checks its extras and documentation.
  • Update contributor and agent guidance for two maintained roots. Implementation and review must inspect the counterpart root in either direction, then apply matching changes or explain why they do not apply.
  • Retain backport_branch temporarily for existing release and backport consumers. Toolkit selection comes exclusively from the package-root registry; cuda.bindings: document shared maintenance and release branches #3178 removes the transitional field with its last consumer.

Toolkit pins select build inputs and the release family; setuptools-scm derives the package version from Git history. A package release patch version need not equal its toolkit patch pin.

The repository-only CI helper package shares parsing and declares its dependencies. It is not shipped in a CUDA Python runtime wheel. The temporary maintenance-version bootstrap is introduced with its actual CI callers in #3175.

Stack / Review Boundary

Current main already contains cuda_bindings_12/ and cuda_bindings_13/. This remaining stack makes both roots participate in packaging, CI, documentation, and independent releases from the same integration branch, replacing routine CUDA 12 artifact sourcing from 12.9.x. Each root still builds the same cuda-bindings distribution and cuda.bindings namespace; use separate environments for the two majors.

The six open review layers are:

PR Scope
#3174 Package configuration, version selection, developer environments, and cross-major review guidance
#3175 Change planning, wheel/sdist builds, and artifact reuse
#3176 GPU/source tests, coverage, documentation, and CI workflow integration
#3177 Releases selected from tagged source and exact-tag CI artifacts
#3178 Maintenance/release-branch policy and manual backport automation
#3182 Three CUDA 12 runtime helper fixes, with their regression coverage

#3174 through #3176 are intended to land together. They remain separate review scopes; #3174 and #3175 are incomplete CI states. Evaluate the combined integration at #3176. The remaining three PRs are subsequent layers, with the complete proposed source tree at #3182.

The stack baseline is main 2e6aa4e9ef89. The first two source-import/integration PRs are already merged into that baseline.

For the complete workflow, start with CONTRIBUTING.md, ci/README.md, .github/RELEASE-bindings.md, and the root/package AGENTS.md files in the top source tree. These documents are part of this stack and should describe the resulting development and release procedures.

Suggested Review Order

  1. ci/versions.yml - the two package roots and shared Python matrix.
  2. ci/tools/bindings_config.py - registry reading and cross-file consistency checks.
  3. cuda_python/pyproject.toml and setup.py - standalone selectors, default selection, and version-derived dependencies.
  4. CONTRIBUTING.md and AGENTS.md - source-build requirements and bidirectional maintenance/review guidance.

Validation

At the exact published heads, 328 CI-tool tests pass at #3176 and 341 at #3182. Full local pre-commit run --all-files --show-diff-on-failure passes at both, including actionlint and Lychee.

Focused packaging regressions and standalone sdist-to-wheel rehearsals cover default CUDA 13 selection, explicit CUDA 12 selection, and rebuilding a CUDA 12 sdist outside Git without a selector. The latter must preserve CUDA 12 dependencies.

The inherited CI workflows are not yet migrated at this review boundary. Full hosted integration is evaluated at #3176: full CI for the combined #3174–#3176 state.

Checklist

  • Focused tests protect selection, packaging, and release outcomes that could fail silently.
  • This description identifies the layer's scope and validation boundary.

@rwgk rwgk added this to the cuda.bindings next milestone Oct 21, 2026
@rwgk rwgk added enhancement Any code-related improvements CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module labels Oct 21, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk rwgk self-assigned this Oct 21, 2026
@github-actions github-actions Bot added the cuda.core Everything related to the cuda.core module label Oct 21, 2026
@rwgk
rwgk added this pull request to stack #3179 October 21, 2026 04:48
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch from 631efa9 to 051c1ef Compare October 21, 2026 05:27
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch 6 times, most recently from 631b63c to ecb50fa Compare October 21, 2026 19:33
@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@mdboom, below is a codex assessment, based on a static code analysis, of which PRs in the stack might be independently mergable.

I'm not sure we should invest time into building temporary compatibility bridges. Would it work for you to aim for merging 3174, 3175, 3176 as one group? The idea would be to get them reviewed in this stack, but then combine the final reviewed code into one normal PR, to be merged as one unit (so that we don't have non-CI-testable intermediate states on main).


#3174 is not independently mergeable as written, but making it so is practical. Two changes currently require later layers:

These layers were originally and intentionally divided for review; they do not all represent working intermediate states.

Current Boundaries

Cumulative stack through Assessment
#3174 Existing CI breaks on the configuration and packaging changes.
#3175 Build integration arrives, but matching test-workflow changes are still missing.
#3176 First sensible full-CI target for both majors. Release and backport workflows still need later changes.
#3177 Release routing is updated; the old backport workflow remains incompatible.
#3178 First existing boundary that also completes those operational workflow updates.

These are code-based assessments; I have not run new CI.

Recommendation

Make #3174 independently mergeable with a small temporary compatibility bridge:

  1. Retain the old configuration fields alongside the new registry, validating that their values agree.
  2. Explicitly select CUDA 13 in the existing metapackage build callers.
  3. Remove the transitional fields as later layers migrate their consumers.

That means several files and focused regression checks, without redesigning the stack or pulling substantial CI/release work into #3174.

This would let us merge the packaging foundation separately while keeping main operational. #3176 would then be the next useful cumulative full-CI target. Making #3175 independently functional would require substantially more boundary reshuffling, so I would defer that decision.

@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

I'll work ahead a bit, to see where we land:

codex response to my previous comment:

Yes—reviewing the three layers separately, then landing their combined result in one normal PR, looks practical. I read your comment; that approach avoids putting the incomplete intermediate states on main.

At the combined #3176 state, the toolkit configuration consumers and metapackage build callers have already been migrated. The remaining adjustment is considerably smaller than making #3174 independently mergeable.

Required Adjustment

Postpone removing backport_branch: "12.9.x" until #3178 retires its consumers.

That means:

This preserves the existing backport workflow and current-major release-note checking. Both still require that setting; the release-note checker currently errors without it.

There is no need to restore the old toolkit configuration fields or add temporary metapackage defaults: the combined three-layer state already handles those.

Release Boundary

The intended intermediate state would be:

One existing policy change also needs explicit documentation: #3176’s release documentation resolver requires a registry in the tagged source. Dry-runs against older tags therefore require historical tooling. That policy is already intended for the finished stack; it takes effect here.

Keep Review Straightforward

I would organize the work as follows:

  1. Add one focused commit to cuda.bindings: configure independent CUDA 12 and CUDA 13 packages #3174 preserving the still-used backport setting, with its validator checks.
  2. Adjust cuda.bindings: document shared maintenance and release branches #3178 to remove it when the final consumer disappears.
  3. Describe cuda.bindings: configure independent CUDA 12 and CUDA 13 packages #3174–CI: test and document both bindings majors on main #3176 as one merge group, with the release boundary stated clearly.
  4. After review, create the combined PR and verify that its Git tree exactly matches the reviewed cumulative state.

Validation should include focused release/backport checks, plus one full CI run on the combined landing PR, rather than full runs on all three review layers.

This keeps the substantial review boundaries intact: packaging, builds, then tests/documentation. The additional change is a small correction to migration order. Nothing changed or triggered yet.

@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@mdboom, #3174–#3176 now form a validated landing point, with separate review scopes for package configuration, builds, and tests/docs.

Full CI is green at both the cumulative #3176 head and the stack top, #3182. The #3176 state preserves the existing CUDA 13 release and backport flows; the later release-routing and maintenance-policy changes remain separate.

Could you please review #3174–#3176 with a view to landing them together? The intent is to retain the three review scopes, then combine them for merging (as explained in my previous comment).

Base automatically changed from rwgk/cb12-on-main-02-source-integration to main October 23, 2026 15:04
@mdboom
mdboom force-pushed the rwgk/cb12-on-main-03-package-selection branch from b087aaf to c8571e0 Compare October 23, 2026 15:04
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch from c8571e0 to 96c2a38 Compare October 25, 2026 23:56

@mdboom mdboom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found with my agent:

The 12.9.x branch builds cp310 wheels. Its build-wheel.yml matrix starts at 3.10, and I checked that on upstream/12.9.x. The shared python.supported list in [ci/versions.yml](vscode-webview://1lq1g48ng0sclg835m7ufv5pq987ujci9cjh6dj4ho9llgobp0ub/ci/versions.yml) starts at 3.11, and #3175 feeds it to every wheel build. After this stack, 12.9.x wheels for 3.10 stop being built. Meanwhile cuda_bindings_12 declares requires-python = ">=3.10", keeps the 3.10 classifier, and its pixi build variants include 3.10. Python 3.10 users would silently fall back to the old 12.9.10 or a source build. The change also drops the 3.9 classifier, and the 12.9.x pyproject has no requires-python. If dropping 3.10 is intended, raise requires-python to match. Otherwise the registry needs a per-package Python list.

I think maybe the bring-over of 12.9.x predates dropping of Python 3.10 support there. It is fine to drop Python 3.10 support across the board (even for 12.9.x cuda-bindings) since it is no longer supported. Let's just make all the updates necessary here to be consistent about that.

Comment thread ci/tools/bindings_config.py
Comment thread cuda_bindings_12/pyproject.toml Outdated
Comment thread ci/tools/bindings_config.py Outdated
Comment thread ci/tools/bindings_scm_version.py Outdated
Comment thread ci/tools/bindings_config.py Outdated
Comment thread ci/tools/bindings_config.py Outdated
Comment thread ci/tools/bindings_config.py Outdated
Comment thread cuda_python/setup.py Outdated
Comment thread ci/versions.yml
Comment thread ci/tools/bindings_config.py Outdated
Comment thread cuda_python/setup.py Outdated
rwgk added 11 commits October 26, 2026 14:32
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Read repository-owned configuration directly, retain release-routing and package consistency checks, and use packaging.Version for canonical release tags. Store toolkit components once as integers while preserving JSON and shell interfaces. Identify the hooks responsible for Core floor checks.

Defer the pre-milestone SCM bootstrap helper and its tests to the following CI-build layer, where its first workflow callers appear.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Align package metadata, Pixi build variants, the generated lockfile, and current installation/support documentation with the shared CI Python matrix. Preserve historical release notes.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Preserve ordinary external source installs without an environment selector. Explicit CUDA 12 and CUDA 13 builds retain their matching SCM selectors and dependencies; reject unsupported explicit selections.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch from 96c2a38 to 4d06d29 Compare October 26, 2026 21:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants