Skip to content
aviralsaxena16Public

About

Mobile image editing app for depth-aware interactive relighting and tap-to-lift object segmentation, powered by React Native, FastAPI, Depth Anything V2, MobileSAM, and LaMa.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

27 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gemini_Generated_Image_s7xca2s7xca2s7xc

Halo

Relight any photo, lift objects out with a tap, and place them back like layers in a design tool — all on your phone.

Python React Native Expo FastAPI PyTorch OpenCV

Demo · Features · Architecture · Setup · Run the App · Roadmap · Contributing

If Halo is useful to you, consider starring the repo — it genuinely helps.


Demo

demo iit

Relight a photo, tap an object to lift it out, drag it into place — all on-device touch, no desktop tools.


Why Halo

Most AI photo editing tools are either desktop-only, buried behind a full Photoshop-style workflow, or a single-trick app that only does one thing well. Halo does two things people actually want on mobile, and does them fast:

  • Relight a flat photo with a draggable, depth-aware light source
  • Lift out and move objects with a tap — smart segmentation + inpainting, no manual masking

No tutorials needed. Tap, drag, done.


Features

Feature What it does Why it matters
Relighting Drag a virtual light source across the image and adjust intensity Studio-quality lighting in seconds, no rig required
Smart segmentation Point-based object selection powered by MobileSAM Skip manual masking entirely
Inpainting Reconstructs the background after object removal Clean, natural results — no visible holes
Layered object placement Extracted objects become movable, resizable layers Reposition and resize like a real design tool
Mobile-first UI Large touch targets, gesture-driven controls Feels native, not like a ported desktop tool
FastAPI backend Handles uploads, jobs, segmentation, relighting, downloads AI logic stays decoupled from the app UI

Core workflow

  1. Upload a photo from the mobile app.
  2. Choose relight or segment mode.
  3. For segmentation, tap the object you want to isolate.
  4. Run the model.
  5. The backend returns the extracted object and inpainted background.
  6. The object becomes a draggable, resizable layer.
  7. Accept the result and save, or keep editing.

Architecture

Halo is split into a frontend app and a backend AI engine.

flowchart LR
    A[Mobile / Expo App] --> B[Frontend UI]
    B --> C[Image Upload]
    C --> D[FastAPI Backend]
    D --> E[DepthAnything V2]
    D --> F[MobileSAM]
    D --> G[LaMa Inpainting]
    D --> H[Inpainted result + cropped object]
    H --> B
    B --> I[Final edited image / download]
Loading

Frontend

  • Built with React Native + Expo
  • Handles image selection, touch gestures, relight controls, and object placement
  • Makes requests to the backend API
  • Displays the edited image and layered object overlays

Backend

  • FastAPI server
  • Receives uploaded images
  • Runs segmentation and inpainting jobs
  • Returns processed image output and cropped segments
  • Exposes status and download routes for each job

AI pipeline

  • Depth estimation: Depth Anything V2
  • Segmentation: MobileSAM
  • Background completion: LaMa inpainting
  • Relighting: custom depth-based light adjustment pipeline

This separation keeps the app simple and lets the heavy AI work live in one place.


Project structure

Halo/
├── assets/
│   ├── logo.png
│   ├── banner.png
│   └── demo.gif
├── Backend/
│   ├── jobs/
│   ├── Masking/
│   │   ├── MobileSAM/
│   │   └── mobile_sam.pt
│   ├── main.py
│   ├── requirements.txt
│   └── server.py
├── Depth-Anything-V2/
│   ├── checkpoints/
│   ├── depth_anything_v2/
│   └── ...
├── Frontend/
│   ├── assets/
│   ├── src/
│   ├── App.js
│   ├── package.json
│   └── ...
├── README.md
└── .gitignore

Tech stack

Mobile frontend

  • React Native
  • Expo
  • Expo Image Picker
  • React Native Gesture Handling
  • React Native Safe Area Context
  • Expo Linear Gradient

Backend

  • Python
  • FastAPI
  • Uvicorn
  • OpenCV
  • Pillow
  • PyTorch
  • TorchVision
  • transformers / diffusers / timm

AI models

  • MobileSAM for segmentation
  • Depth Anything V2 for depth-aware editing
  • LaMa for inpainting
  • Custom relighting logic for interactive light placement

Setup

Prerequisites

  • Python 3.10 or 3.11
  • Node.js 18+ and npm
  • Expo CLI
  • Git
  • Android Studio or a physical phone with Expo Go
  • A working camera/photos permission flow for mobile

1) Clone the repository

git clone https://github.com/YOUR_USERNAME/halo.git
cd halo

2) Install Python dependencies

cd Backend
python -m venv .venv
source .venv/bin/activate
# Windows PowerShell:
# .\.venv\Scripts\Activate.ps1

pip install -r requirements.txt

3) Install frontend dependencies

cd ../Frontend
npm install

4) Prepare AI model files

This project expects model assets to exist in the workspace.

Depth Anything V2

The app expects the checkpoint at:

Depth-Anything-V2/checkpoints/depth_anything_v2_vitl.pth

If the folder is present but the checkpoint is missing, download the required Depth Anything V2 weights and place them there.

MobileSAM

The backend loads:

Backend/Masking/mobile_sam.pt

It also expects the MobileSAM source repository under:

Backend/Masking/MobileSAM/

Make sure both are present before starting the backend.

If the model files are missing, the server may still start, but object segmentation and inpainting features will not work correctly.


Run the app

Start the backend

From the project root:

cd Backend
source .venv/bin/activate
# Windows:
# .\.venv\Scripts\Activate.ps1

uvicorn server:app --host 0.0.0.0 --port 8000 --reload

Health check:

http://localhost:8000/health

Start the frontend

cd Frontend
npx expo start

Then scan the QR code with Expo Go, or run it in an Android Studio emulator / iOS simulator.

Mobile connection note

The app already includes backend host probing logic for common local setups, including:

  • Android emulator: http://10.0.2.2:8000
  • Genymotion: http://10.0.3.2:8000
  • Local LAN IP fallback

If you're testing from a phone on the same Wi-Fi network, the app should usually auto-detect the backend host.


How it works

Relight mode

Upload a photo, then drag the light source around the canvas. Halo estimates scene depth and adjusts the lighting effect so it feels realistic instead of flat.

Segment mode

Tap the object you want to isolate. MobileSAM understands that region and cuts it out; LaMa inpainting fills the gap it leaves behind so the result looks intentional, not edited.

Layered output

Once processed, the object stays as a separate layer above the image — move, resize, and reposition it freely, just like a cut-out sticker over a background. It's never fused into the base image, so it stays fully editable.


API overview

Endpoint Description
POST /upload Upload an image file
POST /upload_base64 Upload a base64-encoded image
GET /status/{job_id} Check job status
POST /relight Apply relighting to a job
POST /segment Start segmentation for selected points
GET /download/{job_id}/{filename} Fetch processed images
GET /health Backend health check

Troubleshooting

Backend fails to start
  • Check your Python version is compatible
  • Confirm all pip packages installed successfully
  • Verify the model checkpoints are present
  • Make sure the correct virtual environment is active
Mobile app can't connect to the backend
  • Confirm your phone and PC are on the same Wi-Fi network
  • Confirm the backend is running on port 8000
  • Check your firewall isn't blocking requests
  • Confirm Expo is serving your app correctly
Segmentation gives poor results
  • Choose a clear object boundary
  • Make sure the object is clearly visible in the image
  • Avoid tiny or fuzzy selections
  • Use a higher-quality source image
Missing model files

If you see warnings around MobileSAM or Depth Anything on startup, verify the model directories and checkpoint paths exactly match the expected locations above.


Roadmap

  • More selection tools (box select, lasso)
  • Rotate handles for object placement
  • Undo/redo support
  • Export quality presets
  • Cloud deployment for AI processing
  • More advanced object-aware editing tools

Contributing

Contributions are welcome — whether it's a bug fix, a new feature, or just improving the docs.

  1. Fork the repo
  2. Create a branch: git checkout -b feature/your-feature
  3. Commit your changes: git commit -m "Add your feature"
  4. Push and open a pull request

If you're planning something larger, open an issue first so we can talk through the approach.


License

This project is under the repository's license. See the included license file for full terms.


Halo logo

Built for real-world image editing, designed for touch.

⭐ Star this repo if Halo saved you a trip to Photoshop.

About

Mobile image editing app for depth-aware interactive relighting and tap-to-lift object segmentation, powered by React Native, FastAPI, Depth Anything V2, MobileSAM, and LaMa.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages