Grasp-Oriented Non-Prehensile Manipulation via Learning a Graspability Field

Licheng Zhong · Gim Hee Lee

ECCV 2026

Project Page | arXiv

🚀 Dependencies

1. Create conda env with Python 3.11

conda create -n gomp python=3.11

2. Install IsaacLab, we are using IsaacSim 5.0 and IsaacLab with commit number e4d644f2b1ffccd184b909e8d65a83d86847a6ea.

3. Install additional dependencies

git clone https://github.com/Colmar-zlicheng/GOMP.git
cd GOMP

pip install -r requirements.txt

# install PyTorch3D (https://github.com/facebookresearch/pytorch3d)
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

# install flash-attn
pip install flash-attn --no-build-isolation

💿 Datasets

Our datasets is available at GOMP_USD.

To use our datasets in this repo, download into data folder as:

mkdir data
cd data
hf download zlicheng/GOMP_USD --repo-type dataset --local-dir ./GOMP_USD

🚂 Train

Pre-computed reference poses

Pre-computed reference poses will be automatic generated and saved when first run the process. We also provide generated files at HERE as generation takes lots of time.

You can download our generated files by:

hf download zlicheng/GOMP --repo-type model --include assets/grasp_cache/* --local-dir ./

Alternatively, run the process directly to generate at the first time. But you need to download the weights for SE(3)-Dif at HERE (download grasp_diffusion folder under ./assets).

If you try to generate by yourself, be noted that the process will be intentionally stopped after generation. Just re-run to continue. Make sure use_dumped_grasp is True. This is to avoid an unknown issue that: if we do train/eval together with this pose generation in the same process, the performance will be obvious low.

Train teacher model

# running without wandb logger
CUDA_VISIBLE_DEVICES=0 python scripts/train_ppo.py --task Isaac-Franka-GOMP-Tea --num_envs 4096 --headless
# running with wandb logger
CUDA_VISIBLE_DEVICES=0 python scripts/train_ppo.py --task Isaac-Franka-GOMP-Tea --num_envs 4096 --headless --logger wandb --log_project_name GOMP --run_name GOMP-Tea

Train student model

1. Reduce action space of teacher model

CUDA_VISIBLE_DEVICES=0 python scripts/train_ppo.py --task Isaac-Franka-GOMP-Tea-Action --num_envs 4096 --headless --logger wandb --log_project_name GOMP --run_name GOMP-Tea-Action --load_model_weights <path-to-ckpt> --reset_std

2. Distill into the student model

CUDA_VISIBLE_DEVICES=0,1 python scripts/train_ppo.py --task Isaac-Franka-GOMP-Stu --num_envs 2048 --headless --logger wandb --log_project_name GOMP --run_name GOMP-Stu --nvdr_device cuda:1 --resume_ckpt <path-to-ckpt>

Checkpoint

All the training checkpoints will be saved under logs.

Our checkpoints are available at HERE, which can be downloaded by:

hf download zlicheng/GOMP --repo-type model --include checkpoints/* --local-dir ./

🧪 Test

Test teacher model

CUDA_VISIBLE_DEVICES=0 python scripts/play_ppo.py --task Isaac-Franka-GOMP-Tea-Unseen --num_envs 1024 --headless --checkpoint <path-to-ckpt>

Test student model

If render pointcloud with same gpu:

CUDA_VISIBLE_DEVICES=0 python scripts/play_ppo.py --task Isaac-Franka-GOMP-Stu-Unseen --num_envs 1024 --headless --checkpoint <path-to-ckpt> --nvdr_device cuda:0

If render pointcloud with different gpu:

CUDA_VISIBLE_DEVICES=0,1 python scripts/play_ppo.py --task Isaac-Franka-GOMP-Stu-Unseen --num_envs 1024 --headless --checkpoint <path-to-ckpt> --nvdr_device cuda:1

🔑 License

The code and model provided herein are available for usage as specified in the LICENSE file. By downloading and using the code you agree to the terms in the LICENSE.

🌏 Citation

@inproceedings{zhong2026gomp,
    title       = {Grasp-Oriented Non-Prehensile Manipulation via Learning a Graspability Field},
    author      = {Zhong, Licheng and Lee, Gim Hee},
    booktitle   = {European Conference on Computer Vision (ECCV)},
    year        = {2026}
}

For more questions, please create an GitHub Issue or contact Licheng Zhong: zlicheng@u.nus.edu