Skip to content

About

Fine-tuning Particle Physics Foundation Models on MINERvA Open Data

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

minerva-ml

This repository contains the data processing and model training code used for ML studies on MINERvA events.

Evaluation plots — browse the latest model-comparison figures (classification, regression, training curves) by configuration: minerva-ml plots · Event viewer (work in progress)

Dataset

The processed dataset used by this project is available on Hugging Face:

  • gregorkrzmanc/minerva-ml
  • It is a preprocessed version of the MINERvA open data release for ML/physics tasks such as available-energy estimation and event tagging.
  • Source data comes from MINERvA open data: MINERvA Open Data.
  • This is a derived dataset and is not an official MINERvA collaboration product.

For detailed data fields and semantics, see DATASET.md. For model architecture details, see MODELS.md. For NERSC-specific paths and SLURM commands, see INTERNAL.md.

Repository workflow

The typical workflow is:

  1. Download raw playlists
  2. Preprocess ROOT files into ML-ready tensors
  3. Split into train/val/test
  4. Train models locally or submit SLURM jobs
  5. Run test evaluation (eval) on checkpoints, then produce figures with src.eval (or notebooks)

0) Environment set-up

Use the gkrz/minerva_ml:v1 container (see the provided Dockerfile).

1) Get the data (two options)

Choose one of the following:

  • Option A (from scratch): Download raw MINERvA playlists, then preprocess locally.
  • Option B (quick start): Download the already preprocessed dataset from Hugging Face and skip local preprocessing.

Option A: Download raw playlists (from scratch)

Set SCRATCH first, then run:

# Monte Carlo playlists
python -m src.scripts.download_data

# Recorded data playlists
python -m src.scripts.download_data --prefix MediumEnergy_FHC_Data_Playlist

Option B: Download preprocessed dataset from Hugging Face

If you want to skip raw playlist processing, download the preprocessed dataset snapshot:

pip install -U "huggingface_hub[cli]"
huggingface-cli download gregorkrzmanc/minerva-ml \
  --repo-type dataset \
  --local-dir <HF_DATA_DIR>

After download, point your training/splitting commands to the downloaded folder structure.

2) Preprocess dataset

Skip this section if you used Option B and already have the preprocessed files you need.

Minimal invocation (creates .pb files with event-wise particle tensors and labels):

python -m src.scripts.preprocess_dataset --output-dir <OUTPUT_DIR>

For a full pipeline on this project’s layout—preprocess, split playlists 1A / 1B, and extract baselines—edit paths in the script if needed, then run:

bash src/scripts/preprocess.sh

src/scripts/preprocess.sh sets DATA_DIR, runs preprocess_dataset with blob/prong limits and playlist selection, runs split_dataset per playlist (with different val/test ratios for 1B vs 1A), and runs extract_baselines against the raw playlist directories under scratch.

3) Split into train / val / test

python -m src.scripts.split_dataset \
  --input-dir <PREPROCESSED_DIR> \
  --output-dir <SPLIT_OUTPUT_DIR>

To inspect created features quickly, see notebooks/stats.ipynb.

Dataset distribution plots

After splitting, summarize the ML-ready .pb tensors with src.scripts.plot_dataset_distributions. Run from the repository root with your project Python environment activated (see section 0).

Full run (reads the split, writes all PDFs and a JSON summary):

python -m src.scripts.plot_dataset_distributions \
  --data-path <SPLIT_OUTPUT_DIR> \
  --playlist 1A \
  --split train \
  --out-dir plots/dataset_distributions

Quick check on a subsample (faster; good for testing):

python -m src.scripts.plot_dataset_distributions \
  --data-path <SPLIT_OUTPUT_DIR> \
  --playlist 1A --split train \
  --max-events 50000 \
  --out-dir plots/dataset_distributions

What gets produced

Outputs live under plots/dataset_distributions/ (per playlist + split, tag like 1A_train):

Path Contents
particle_features_<tag>.pdf Per-particle columns (η, φ, log pT, log E, PID, …)
logE_by_PID_<tag>.pdf log(E) histogram per reconstruction PID type
global_features_<tag>.pdf 16 event-level conditioning features
truth_labels_<tag>.pdf All 15 MC truth columns
n_particles_per_event_<tag>.pdf Token multiplicity
summary_<tag>.json min/max/mean/std/percentiles per variable
poster/ Publication-style step histograms (see below)
poster_v2/ Same figures, two-red palette + Matter font

Poster figures (poster/ and poster_v2/):

File Description
tokens_per_event_<tag>.pdf Tokens per MINERvA event
energy_comparison_<tag>.pdf True neutrino energy vs sum of token energies
e_available_spectrum_<tag>.pdf MC E_available and E_available + E_μ
e_available_by_class_<tag>.pdf E_ν plus E_available per Pi_labels_v2 class (CC 1π±, CC Nπ±, CC 1π⁰, CC other, NC)
joint/tokens_per_event_<tag>.pdf MINERvA vs JetClass-II token multiplicity overlay
joint/energy_comparison_<tag>.pdf MINERvA neutrino energy vs JetClass-II jet energy (log–log)

Poster plots use density-normalized step histograms. The first full run also writes pickle caches under poster/poster_bins_<tag>.pkl (and poster_bins_jetclass2_<tag>.pkl when JetClass-II is enabled).

Fast replot (poster only, no dataset read)

After a full run has built the poster caches, restyle or tweak poster code without reloading .pb files:

python -m src.scripts.plot_dataset_distributions \
  --playlist 1A --split train \
  --out-dir plots/dataset_distributions \
  --poster-from-cache

This regenerates both poster/ and poster_v2/. If a cache predates newer plots (e.g. E_available spectra), the script reloads truth labels from the dataset only for those missing sections.

JetClass-II overlays (optional)

Joint MINERvA/JetClass-II figures need preprocessed JetClass-II h5 shards. By default one file is sampled from the OmniLearned path; skip with --no-jetclass2:

python -m src.scripts.plot_dataset_distributions \
  --data-path <SPLIT_OUTPUT_DIR> \
  --playlist 1A --split train \
  --jetclass2-path /path/to/jetclass2/train \
  --jetclass2-n-files 1 \
  --max-events 50000 \
  --out-dir plots/dataset_distributions

Useful flags

Flag Purpose
--playlist Playlist subfolder (1A, 1B, …)
--split train, val, or test
--max-events Subsample events (deterministic with --seed)
--bins Histogram bins for overview grids (default 80)
--log-y Log y-axis on overview continuous histograms
--poster-from-cache Regenerate poster PDFs from cached bins only
--no-jetclass2 Skip JetClass-II and joint overlays

4) Train models

Models are launched via src/jobs/submit_new_dataset_jobs.py, which submits the standard comparison model set (one SLURM job per (model, task, seed)) or, for MLP, prints a local training command.

Supported models (plot keys): HyperScale-small, HyperScale-small-rw, OmniLearned-small, OmniLearned-small-rw, OmniLearned-medium, HyperScale-medium, HyperScale-medium-rw, BERT-tiny, BERT-tiny-rw, BERT-small, BERT-small-rw, plus MLP.

# List available model keys
python src/jobs/submit_new_dataset_jobs.py --list-models

# Submit all models, both tasks, for the given seeds
python src/jobs/submit_new_dataset_jobs.py --seeds 55 56 57 58

# Submit a subset of models/tasks
python src/jobs/submit_new_dataset_jobs.py --seeds 55 --models HyperScale-small --tasks regression

# Ablation: zero muon-token kinematics (tags run names with _noMuonKin)
python src/jobs/submit_new_dataset_jobs.py --seeds 55 --models HyperScale-small --remove-muon-kinematics

# Write .slurm files without calling sbatch
python src/jobs/submit_new_dataset_jobs.py --seeds 55 --dry-run

Dataset/output paths default to this project's NERSC layout; pass --data-path to point at a different preprocessed dataset root. See INTERNAL.md for the exact hardcoded paths, SLURM account, and cluster layout.

5) Analysis (test eval, evaluation plots, notebooks)

After training, group the runs you want to compare in Weights & Biases by assigning the same tag to each run (in the run’s overview or via the API). The tools below use that tag against the minerva-models project under your W&B entity (set WANDB_ENTITY and use wandb login as needed).

Test evaluation (eval)

Run or preview eval for checkpoints that still need test_results (skipped if an .npz for that dataset already exists). Activate your project Python environment first; child evals use the same interpreter (sys.executable).

# Preview commands
python -m src.scripts.evaluate_single_gpu --wandb-flag <TAG> --dry-run

# Run evals sequentially on the current node (login or single-GPU session)
python -m src.scripts.evaluate_single_gpu --wandb-flag <TAG>

--wandb-flag only considers runs whose checkpoint folder name matches a wandb run name with that tag; omit it to evaluate every run under --ckpt-dir (default: see --help). Evaluation is very small and fast—it is fine to run on login nodes when a GPU is available locally.

Multi-GPU evaluation (--num-gpus)

On a multi-GPU node, pass --num-gpus N (alias -n N) to spread the eval jobs across N GPUs. One job runs per GPU concurrently, each pinned via CUDA_VISIBLE_DEVICES, and idle GPUs immediately pick up the next pending job. Eval jobs are deduplicated the same way as the single-GPU path (a (folder, dataset) is skipped if its .npz already exists), so the GPUs never repeat work.

python -m src.scripts.evaluate_single_gpu --wandb-flag <TAG> --num-gpus 4

When CUDA_VISIBLE_DEVICES is already set (as SLURM does on an allocation), the script only schedules onto those allocated GPUs and caps --num-gpus to the number visible (warning if you ask for more). --num-gpus 1 (the default) keeps the original sequential behavior. See INTERNAL.md for the SLURM allocation command used on NERSC.

Evaluation plots (src.eval)

Offline plotting reads cached pickles and writes PDFs under plots/. Run from the repository root with your project Python environment activated (see section 0).

W&B runs are selected by the wandb tag passed as --flag (see fetch_runs_from_wandb in src/utils/utils.py). Model keys in plot configs must match the grouped model names in those pickles (e.g. OmniLearned-small, BERT-tiny).

1. Cache eval inputs (once per W&B tag)

export FLAG=Run_2703
export OUT=<CACHE_DIR>

python -m src.eval.collect_eval_data \
    --flag $FLAG --out-dir $OUT

Writes classification_<TAG>.pkl and regression_<TAG>.pkl under $OUT. Alternatively, download pre-built pickles from gregorkrzmanc/minerva-ml-eval and point $OUT at that directory.

2. Build plot caches (once; expensive step)

Use scripts/rebuild_plot_caches.sh to rebuild all caches after new eval data:

bash scripts/rebuild_plot_caches.sh --force --sync-canonical

Caches are written outside the repo, next to $OUT (see INTERNAL.md for the exact NERSC path).

Cache file Built by Used for
classification_metrics.pkl build_classification_cache q3, W, pions (--plots-only)
classification_light.pkl plot_classification_light light appendix + q3 light panels
steps.pkl build_steps_cache training-curve plots (plot_steps --plots-only)
regression.pkl plot_regression / collect_eval_data energy-regression plots

3. Generate PDFs from cache (fast; config-driven)

One-shot (rebuild caches + render + publish):

bash scripts/rebuild_plot_caches.sh --force --config default --publish

PDFs only (caches already fresh):

bash scripts/generate_comparison_plots.sh --config default
bash scripts/publish_plots.sh

A per-tag wrapper script that chains eval → collect → rebuild → replot for a single W&B tag also exists (scripts/rebuild_plots_Run_0707.sh); see INTERNAL.md for its usage and SLURM invocation.

Figures for a LaTeX paper

Copies selected PDFs from plots/<config>/ into figures_latex/<config>/ with stable names. Pass the same config name used for generate_comparison_plots.sh:

python -m src.scripts.copy_figures_for_paper --config V1Paper
# -> figures_latex/V1Paper/
# or: python -m src.scripts.copy_figures_for_paper -c V1Paper --dry-run
# optional: --plots-root plots/V1Paper --figures-root figures/V1Paper

6) Small per-bin demo datasets (event viewer + scoring)

Moved to EVENT_VIEWER.md: a lightweight demo pipeline that carves the ML-ready dataset into small per-kinematic-bin samples, renders an interactive HTML event viewer, and scores those samples with trained models.

Citation

If you use this code or dataset, please cite:

@misc{krzmanc2026,
      title={Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions},
      author={Gregor Krzmanc and Vinicius Mikuni and Benjamin Nachman and Callum Wilkinson},
      year={2026},
      eprint={2604.12364},
      archivePrefix={arXiv},
      primaryClass={hep-ex},
      url={https://arxiv.org/abs/2604.12364},
}

See also CITATION.bib.

License

This project is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0) — see LICENSE.

About

Fine-tuning Particle Physics Foundation Models on MINERvA Open Data

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages