Skip to content
ayuxy027Public

About

Off-Road Semantic Segmentation Neural Network

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

Off-Road Semantic Segmentation

Semantic segmentation model that labels every pixel of desert images into 10 terrain classes. Built for the Duality AI Falcon Hackathon.

Results

Version Epochs Val IoU Test IoU Status
V1 Baseline 10 0.2972 0.2347 Reference
V2 Optimized 10 0.2943 0.2347 Faster training
V3 Full Train 30 0.4712 0.4700 Best

V3 doubles the test IoU compared to V1. Rare classes improved 6 to 48 times.

Quick Start

1. Setup

python -m venv .venv
source .venv/bin/activate
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install numpy pillow matplotlib opencv-python tqdm

2. Train

python scripts/trainSegmentation.py

Outputs: model weights in models/, training curves in trainStats/.

3. Evaluate

python scripts/testSegmentation.py --dataDir /path/to/dataset --outputDir predictions

Outputs: evaluation metrics and comparison images in predictions/.

Architecture

Component Details
Backbone DINOv2 ViT-B/14-reg, frozen, self-supervised
Head ConvNeXt-style, 3 blocks, 256 channels
Input 960x540 resized to 546x280
Classes 11 (background + 10 terrain types)
Loss 0.5 weighted CE + 0.5 Dice
Optimizer AdamW, lr=1e-3, cosine schedule, weight decay 1e-4
Batch Size 4
Augmentation Flip, ColorJitter, GaussianBlur
Precision AMP (CUDA autocast)

Per-Class IoU

Class V1 V2 V3
Sky 0.9449 0.9412 0.9570
Trees 0.0907 0.0864 0.5836
Lush Bushes 0.0112 0.0105 0.4771
Dry Grass 0.1514 0.1577 0.5372
Dry Bushes 0.0600 0.0675 0.3612
Ground Clutter 0.0491 0.0598 0.2454
Flowers - - 0.5008
Logs 0.0356 0.0343 0.2299
Rocks 0.2646 0.2053 0.2431
Landscape 0.5031 0.5485 0.4359

Project Structure

Offroad_Segmentation_Scripts/
├── docs/              Task description, workflow, hackathon briefs
├── scripts/           trainSegmentation.py, testSegmentation.py, visualize.py
├── models/            Trained model weights (.pth files)
├── experiments/       Baseline and exp1-exp4 outputs
│   ├── baseline/      Original 10-epoch run
│   ├── exp1/          3 epochs, AdamW
│   ├── exp2/          5 epochs, cleaner eval
│   ├── exp3/          10 epochs, SGD
│   └── exp4/          10 epochs, batch=2, AMP
├── trainStats/        V3 training curves and metrics
├── predictionsBatch1/ V3 batch 1 predictions
├── predictionsBatch2/ V3 batch 2 predictions
├── predictionsBatch3/ V3 batch 3 predictions (best: 0.4700)
├── frontend/          Interactive React report
│   ├── src/pages/     Overview, V1, V2, V3 tabs
│   └── public/        Charts and comparison images
├── envSetup/          Environment setup scripts
└── README.md          This file

Interactive Report

React + TypeScript dashboard with 4 tabs:

Tab Content
Overview V1 vs V2 vs V3 comparison table
V1 Baseline results, training curves, prediction samples
V2 Optimized results, training curves, prediction samples
V3 30-epoch full training results, per-class breakdown

Run with:

cd frontend
npm install
npm run dev

PDF Report

View the full report: SNN_Duality.pdf

Hardware

  • GPU: NVIDIA GeForce RTX 4050 Laptop, 6GB VRAM
  • CUDA: 12.1
  • PyTorch: 2.5.1

About

Off-Road Semantic Segmentation Neural Network

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages