Skip to content

Faster AUTO_LOWPP_ARRAY implementation - #3186

Open
mdboom wants to merge 1 commit into
NVIDIA:mainfrom
mdboom:fast-array
Open

mdboom wants to merge 1 commit into
NVIDIA:mainfrom
mdboom:fast-array

Conversation

@mdboom

@mdboom mdboom commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

This brings in generator changes from !597.

The premise here is to stop using numpy.recarray (which are implemented in Python, not C) for these arrays of structs. To avoid changing the API, certain operations lazily still return recarrays, but the most important operations avoid that for a /significantly/ faster implementation.

Details

  • Generated wrappers store _data as a plain numpy.ndarray with the same numpy.record dtype, so NumPy integer keys still return numpy.record.
  • Results that used to be recarray are still recarray: struct-typed fields and lists of field names are converted with .view(numpy.recarray) before they are returned.
  • Integer keys and slices still return wrappers, built without going through recarray.
  • Generated field properties read and write _data by key (_data["field"]) instead of by attribute.
  • ptr / int() get the address through the buffer protocol instead of .ctypes.data. The address is not cached, so it always reflects the current _data.
  • Wrappers for nested struct members are created on first access instead of at construction.
image

@copy-pr-bot

copy-pr-bot Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.bindings Everything related to the cuda.bindings module label Oct 22, 2026
@mdboom

mdboom commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@github-actions

Copy link
Copy Markdown
Contributor

@mdboom
mdboom marked this pull request as ready for review October 26, 2026 14:01
@coderabbitai

coderabbitai Bot commented Oct 26, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between a2fb804 and 58c773a.

Files selected for processing: 21
  • cuda_bindings_13/cuda/bindings/_internal/cudla_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/cudla_windows.pyx
  • cuda_bindings_13/cuda/bindings/_internal/cufile_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/driver_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/driver_windows.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvfatbin_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvfatbin_windows.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvjitlink_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvjitlink_windows.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvml_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvml_windows.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvrtc_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvrtc_windows.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvvm_linux.pyx
  • cuda_bindings_13/cuda/bindings/_internal/nvvm_windows.pyx
  • cuda_bindings_13/cuda/bindings/_v2/driver.pyx
  • cuda_bindings_13/cuda/bindings/cufile.pyx
  • cuda_bindings_13/cuda/bindings/cycufile.pxd
  • cuda_bindings_13/cuda/bindings/cydriver.pxd
  • cuda_bindings_13/cuda/bindings/nvml.pyx
  • cuda_bindings_13/docs/source/release/13.5.0-notes.rst

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Summary

Summary by CodeRabbit

  • Performance
    • Improved common operations on arrays of CUDA structures, with reported speedups of 5–30× in several binding areas.
  • Compatibility
    • Array wrappers now use structured NumPy arrays while retaining field access and support for indexed views.

The pull request updates generated Cython import placement and changes cuFile and CUDA driver struct-array wrappers to use structured-record arrays. It also adds a release-note entry reporting faster operations for these wrappers.

Changes: CUDA struct-array wrappers

Layer / File(s) Summary
Generated preamble imports
cuda_bindings_13/cuda/bindings/_internal/*, cuda_bindings_13/cuda/bindings/cycufile.pxd, cuda_bindings_13/cuda/bindings/cydriver.pxd
Generated bindings move existing Cython imports into preambles or reorder them. Generated content hashes change; the declarations and runtime behavior described in these files remain unchanged.
cuFile structured-array wrappers
cuda_bindings_13/cuda/bindings/cufile.pyx
IOEvents, PerGpuStats, IOVec, Descr, and IOParams use structured-record arrays, named-field access, and buffer-based pointer access. Indexing returns owner-retaining views.
Driver structured-array wrappers
cuda_bindings_13/cuda/bindings/_v2/driver.pyx, cuda_bindings_13/docs/source/release/13.5.0-notes.rst
Eight wrappers use structured-record arrays and owner-retaining views. Nested wrappers are created on first property access where applicable. The release note reports 5–30× faster struct-array operations in cufile, nvml, and _v2.driver.

Suggested labels: enhancement

Suggested reviewers: rwgk

Priority: Normal

Change: Feature

Merge Risk: Minimal · up to 58c77

This change speeds up wrappers for arrays of CUDA structs while preserving the existing API. No concrete merge-blocking issue remains after review.

Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@mdboom mdboom self-assigned this Oct 26, 2026
@mdboom mdboom added the enhancement Any code-related improvements label Oct 26, 2026
@mdboom mdboom added this to the cuda.bindings 13.5.0 & 12.9.10 milestone Oct 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.bindings Everything related to the cuda.bindings module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant