sattwik21/anytraverse-studio

🚜 AnyTraverse Studio — Live Evaluation & HITL Dashboard

A live Gradio dashboard for evaluating the AnyTraverse zero-shot off-road traversability framework (paper, PyPI).

Try it online: huggingface.co/spaces/sattwik21/anytraverse-studio (runs on ZeroGPU), or run it locally on your own machine (see below).

Features

• Live per-frame evaluation of any uploaded off-road video: raw image + ROI box · traversability map · uncertainty map · ROI crop
• Attention maps for all prompts live in the UI (grid of up to 5 columns, aspect-ratio preserved) — never baked into the exported video
• Human-in-the-Loop: the run halts whenever the traversal state is not ok; provide a τ update (e.g. mud: -0.7; gravel: 0.6, or just ok) and press Resume
• Editable live: thresholds and ROI bounds update the running pipeline without restarting
• Live dual-metric chart + 0–1 gauge bars + telemetry table
• Composed analysis video exported as browser-playable H.264 (bundled imageio-ffmpeg, no system ffmpeg required)
• Hard frame skip: pass every k-th frame to the VLM; skipped frames are neither shown in the UI nor written to the video

Run locally

1. Prerequisites

• Linux / macOS / Windows with Python 3.12+
• A GPU with CUDA is strongly recommended (CLIPSeg + CLIP inference). CPU works but is slow.
• uv (fast, optional) or pip + a virtualenv

2. Clone

git clone https://huggingface.co/sattwik21/anytraverse-studio
cd anytraverse-studio

3. Install dependencies

With uv (recommended):

uv sync

Or with plain pip:

python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt

If Torch pulls the wrong build, install the CUDA build explicitly, e.g.:

pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install torchvision --index-url https://download.pytorch.org/whl/cu128

4. Run

python app.py

The dashboard opens at http://localhost:7860. Since share=True is enabled outside Hugging Face Spaces, Gradio will also print a temporary public share link at startup — you can ignore it for local use.

5. Usage

1. Upload an off-road video (.mp4, .mov, .avi).
2. Leave τ as {} for default preferences, or enter e.g. {"road": 1.0, "grass": 0.0, "bush": -0.8}.
3. Set the Ref Scene Sim. Threshold, ROI Uncertainty Threshold and Frame Skip.
4. Press ▶️ Go / Reset.
5. When the run halts (unknown_scene / unknown_object), type an operator update in the box and press ✅ Apply & Resume.

Notes

• The first run downloads the VLM weights (CLIPSeg + CLIP, ~1 GB) and caches them locally.
• data/weights/ and the generated *.mp4 files are runtime artifacts; the dashboard writes raw_opencv_temp.mp4 and anytraverse_h264_output.mp4 in the working directory.

Deployment

The interactive Space lives at huggingface.co/spaces/sattwik21/anytraverse-studio. This repo is the model card / source for the same dashboard.