[reidmorrison]

Summary

This is a docs-only PR (no code changes). It adds maintainer and contributor documentation from a full review of main (55dc3de), plus a prioritized TODO and a phased roadmap for getting Ortex onto Nx 1.0 and the latest ONNX Runtime. We would like to help implement the roadmap if it is useful.

• AGENTS.md: setup, commands, gotchas, design rules, and recipes for contributors and AI coding agents.
• docs/: architecture, the Elixir and native layers, build and packaging, execution providers, testing and CI, and an upgrade guide for Nx 1.0, Rustler 0.39, and ort 2.0.0-rc.15 (ONNX Runtime 1.31).
• TODO.md: findings, a triage of all open issues and PRs, and the roadmap.

Main findings

Verified on macOS arm64 with Elixir 1.20.4, OTP 29.1.2, and Rust 1.99.1.

• main compiles and all 117 tests pass. They also pass after bumping to Nx 1.0 and Hex Rustler 0.38 with no code changes. Two things block users today:
  • Hex 0.1.10 predates the Elixir 1.19 Inspect fix.
  • It requires {:nx, "~> 0.6"}, which excludes Nx 1.0.
• Time-sensitive: Rust 1.100 (about 2026-11-12) breaks every dynamically linked build on ort rc.8. That includes CUDA/TensorRT and this repo's Linux CI. Reproduced on Rust 1.100.0-beta.7: ort-sys's copy-dylibs fails with the new cargo build-dir layout. ort rc.14+ fixes it.
• Moving from ort rc.8 to rc.15 took about 172 changed lines in a compile probe. Modernise, update to Ort 2.0.0-rc.11 #47 already covers most of it. The real work is design:
  • ONNX Runtime allows only one environment per process.
  • Session::run now needs &mut self.
  • Optimization level 3 changed meaning.
  • The platform floor rises: glibc 2.39, Apple Silicon only, CUDA 13.

Proposed order

The details are in TODO.md section 2.

1. A 0.1.11 patch release on the current ort:
   • widen Nx to allow 1.0;
   • ship the Elixir 1.19 fix;
   • fix crashes and the GPU library copy;
   • keep CI green on Rust 1.100.
2. The modernization half of Modernise, update to Ort 2.0.0-rc.11 #47.
3. ort rc.15.
4. One session-options API, consolidating Add ability to set ONNX session options #50, Add an intra_threads option to Ortex.load #51, and Add per-session ONNX Runtime threading options #52.
5. Precompiled NIFs (Add precompiled NIF artifacts via rustler_precompiled (#27) #49).

Fork PRs were reviewed statically only.

🤖 Generated with Claude Code

[reidmorrison]

Hi @mortont and @josevalim,

Thanks for Ortex. We use it, and we'd like to help get it current. This PR is documentation only. Before the larger work, we'd like to propose a small 0.1.11 patch release on the current ort rc.8. It would unblock most users quickly, and we're happy to do the work.

Why a patch release first

• Nx 1.0 can't be used with Ortex. The published 0.1.10 requires {:nx, "~> 0.6"}, which excludes Nx 1.0, so any app on Nx, EXLA or Bumblebee 1.0 can't install Ortex. Widening this to "~> 0.6 or ~> 1.0" works without code changes; all 117 tests pass on Nx 1.0.
• Inspecting a model is broken for Hex users on Elixir 1.19+. The fix (Uses Inspec.Algebra.to_doc syntax to support Elixir 1.19 #48) is on main but hasn't been released.
• GPU builds will stop compiling on Rust 1.100, due around November 12. Cargo's new build-dir layout breaks ort-sys rc.8's copy-dylibs. I reproduced this on Rust 1.100.0-beta.7. It hits every dynamically linked build (CUDA/TensorRT), including this repo's Linux CI, which enables those features in config/config.exs. The real fix is ort rc.14+, so 0.1.11 would keep CI CPU-only and document the Rust pin for GPU users in the meantime.
• GPU libraries are no longer copied next to the NIF. New installs resolve Hex Rustler 0.38, which moved the cargo build directory, so Ortex.Util.copy_ort_libs/0 finds nothing.

Proposed scope for 0.1.11 (each item maps to an ID in TODO.md)

1. Widen the Nx requirement and add Nx 1.0 plus Elixir 1.20 / OTP 29 to CI (COMPAT-01, CI-01, CI-02).
2. Fix copy_ort_libs/0 for current Rustler (BUILD-01).
3. Implement stack/3, which Nx 0.8+ made a backend callback (BUG-14).
4. Accept multiple partitions in Ortex.Serving.init/3 (BUG-08).
5. Fix three crash and memory-safety bugs:
   • add a length check to the raw-pointer read in from_binary (SAFE-01);
   • return errors instead of panicking on unsupported tensor types (SAFE-02);
   • stop large models from overflowing the dirty-scheduler stack during load, from Elixir crash quitely when loading ONNX model #42 (SAFE-05).
6. Fill the smaller Nx contract gaps:
   • honor the to_binary limit (BUG-04);
   • implement backend_copy and backend_deallocate (BUG-05);
   • handle NIF errors in the shape ops (BUG-07);
   • validate the input count (BUG-02).
7. Fix the concat test that only exercises 2 of 12 dtypes, and correct the README (TEST-01, DOC-01).

Most of these fixes already exist in #47, so we'd pull them out into a small PR, which also starts the split you asked for there. After that, the roadmap continues with the rest of #47's modernization, then ort rc.15 / ONNX Runtime 1.31, one session-options API combining #50, #51 and #52, and finally precompiled NIFs (#49).

Section 1 of TODO.md summarizes the findings and section 2 has the roadmap. Section 3 has every item with file references, and sections 4 and 5 triage the open issues and PRs. docs/upgrading.md covers the ort upgrade in detail.

If this direction works for you, we can open the 0.1.11 PR right away. Because of the Rust 1.100 date, it would be good to release before mid-November. Happy to adjust the scope to whatever you prefer.