cd /home/$USER/workspace/
git clone git@github.com:behnamasadi/robotic_notes.git
vcpkg is configured as a git submodule. Initialize it:
cd /home/$USER/workspace/robotic_notes
git submodule update --init --recursive
set the path:
export VCPKG_ROOT=$PWD/vcpkg
export PATH=$VCPKG_ROOT:$PATH
Setting VCPKG_ROOT tells vcpkg where your vcpkg instance is located.
Install required system dependencies for vcpkg (on Linux):
sudo apt-get install -y bison flex build-essential cmake autoconf autoconf-archive automake libtool libltdl-dev libx11-dev libxft-dev libxext-dev libxtst-dev libxrandr-dev ninja-build pkg-config
Now you can run:
cmake -S . -B build \
-DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DVCPKG_TARGET_TRIPLET=x64-linux-release
The VCPKG_TARGET_TRIPLET=x64-linux-release option ensures vcpkg only builds release packages, which significantly reduces build time (especially for large packages like OpenCV) and disk space usage. This is already configured in CMakeLists.txt, but you can explicitly set it as shown above.
The Rerun C++ SDK (and its Arrow dependency) is disabled by default because the SDK is downloaded and built via FetchContent, which is heavy. The Python rerun-sdk package covers the same use cases for the notebooks. To opt in and build the rerun-dependent C++ targets, add the following flags:
cmake -S . -B build \
-DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DVCPKG_TARGET_TRIPLET=x64-linux-release \
-DBUILD_RERUN_EXAMPLES=ON \
-DVCPKG_MANIFEST_FEATURES=rerun
cmake --build build --parallel
conda create -n robotic_notes python=3.12 -y
conda activate robotic_notes
cd /home/$USER/anaconda3/envs/robotic_notes/
Create this soft link.
ln -s /home/$USER/workspace/robotic_notes /home/$USER/anaconda3/envs/robotic_notes/src
Install the python packages:
conda install -c conda-forge opencv
pip install graphslam
conda install conda-forge::gtsam
conda install conda-forge::matplotlib
conda install conda-forge::plotly
conda install -c conda-forge jupyterlab
pip install ahrs
pip install pyceres
pip install liegroups
pip install rerun-sdk[notebook]==0.29.2
pip install "gradio_rerun==0.29.2"
pip install "gradio==6.5.1"
pip install ipykernel
pip install jupyterlab
- Robotic System Design & Design Patterns
- 1. High-Level Architecture of a Humanoid Robot
- 2. Perception Pipeline: Consistency, Robustness, Synchronization
- 3. High-Frequency Sensor Pipelines in C++
- 4. Concurrency & Real-Time: SPMC Sensor Distribution
- 5. Task Execution: State Machine vs Behavior Tree
- 6. Debugging a Regressing Perception Model in Production
- 7. Time Synchronization
- 8. Safety & Fault Tolerance
- 9. Logging, Observability & Replay
- Configuration of Robot
- Configuration Space - (C-space )
- Degrees of freedom.
- Task Space
- Work Space
- Dexterous space:
- dof
- Topology
- Algebraic topology (playlist)
- Non-Holonomic Constraints, Pfaffian Constraints and Holonomic Constraints
- Kinematics of Differential Drive Robots and Wheel odometry
- Velocity-based (dead reckoning)
- Nonlinear uncertainty model associated with a robot's position over time (The Banana Distribution is Gaussian)
- Coordinate Frame Conventions — ENU, NED, FRD, FLU, RDF, RUB
- 1. Global References
- 2. Accelerometer Model
- 3. Gyroscope Model
- 4. Attitude from gravity (Tilt)
- Expressing IMU reading with Quaternion
- 5. Quaternion from Accelerometer
- 6. Quaternion Integration
- 7.1 Quaternion Derivative
- Relationship Between Euler-Angle Rates and Body-Axis Rates
- Complementary Filter
- Quaternion-Based Complementary Filter
- Accelerometer-Based Correction
- Attitude from angular rate (Attitude propagation)
- IMU Integration
- Noise Spectral Density
- Signal-to-noise Ratio
- Allan Variance curve
- Autoregressive model
- Madgwick Orientation Filter
- Mahony Orientation Filter
- Simulating IMU Measurements
- IMU Propagation Derivations
- IMU Noise Model
- The standard deviation of the discrete-time noise process
- Square binary fiducials (ArUco, AprilTag
- Composite calibration boards (ChArUco, AprilGrid)
- Which marker should I use?
- AprilTag — coordinate frame and measurements
- Generating AprilTags
- Detecting AprilTags (C++, Python, ROS)
- Common pitfalls
- AprilTag vs ArUco (and ChArUco / AprilGrid)
- Choosing a calibration target
- Calibration target files (AprilGrid PDF, YAMLs)
- Supported camera models and distortion
- Building the Kalibr Docker image
- Recording the calibration bag
- Camera-only calibration
- Camera–IMU calibration
- Common pitfalls
- 1. Names
- 2. Remapping Arguments
- NodeHandles
- Roslaunch
- URDF
- Publishing the State
- ROS best practices
- move_base
- ROS Odometery Model
- ROS State Estimation
- EKF Implementations
- Differential Drive Wheel Systems
- Choosing a distro
- Pull the image
- Run a persistent container
- Extending the image with a Dockerfile
- Docker Compose for multi-container setups
- ROS 2 ↔ ROS 1 bridge
- ROS 2 inside the container
- Colcon
- Packages
- Xacro
- Launch files
- Nav2
- teleop_twist_keyboard
- Useful container shortcuts
- Gazebo Versions
- Installation
- Building a model
- Building world
- Moving the robot
- Sensors
- Spawn URDF
- ROS 2 integration
- Distro / Gazebo mapping
- Dockerfile (Jazzy + Harmonic + ros_gz)
- Running the container
- GPU access (NVIDIA)
- Launching Gazebo
ros_gz_bridgeessentials- Demo recipes
- Troubleshooting
- Probability Review (Marginals/conditionals, probability vs likelihood, conditional Gaussians ) (foundational primer)
- Bayes Filter
- Maximum A-Posteriori (MAP) Estimation
- Kalman Filter
- Extended Kalman Filter
- Extended Kalman Filter for Differential Drive Robot
- Error State Extended Kalman Filter
- Error State Extended Kalman Filter(IMU, a GNSS, and a LiDAR)
- Multi-State Constraint Kalman Filter (MSCKF)
- Quaternion kinematics for the error-state Kalman filter
- Active Exposure Control for Robust Visual Odometry in HDR Environments
- Pose Graph SLAM from Scratch
- nano-pgo
- General Graph Optimization g2o explained
- Factor Graphs, Bayes Trees, and iSAM2 (with GTSAM)
- Resilient Autonomy in Perceptually-degraded Environments
- HBA Large-Scale LiDAR Mapping Module
- Hierarchical, multi-resolution volumetric mapping (wavemap)
- kiss-icp
- TagSLAM SLAM with tags
- OpenDroneMap
- Interactive SLAM
- Volumetric TSDF Fusion of Multiple Depth Maps
- Euclidean Signed Distance Field (ESDF)
- Lidar odometry smoothing using ES EKF and KissICP for Ouster sensors with IMUs
- Multisensor-aided Inertial Navigation System (MINS)
- GLOMAP explained
- Zero-Shot Point Cloud Registration
- Add Apriltag to loop closure
- Navtech Radar SLAM
- Procrustes Analysis
- Wahba's Problem
- Quaternion Estimator Algorithm (QUEST)
- Kabsch Algorithm
- Umeyama Algorithm
- Iterative Closest Point (ICP)
- Rotation Averaging — the same SVD/Procrustes trick applied to N estimates of one rotation
The vio_benchmark/ subproject is a self-contained
framework for comparing visual-inertial odometry estimators against
public datasets (EuRoC MAV, TUM-VIO, and others). It bundles OpenVINS
and VINS-Fusion as submodules, ships per-dataset calibration files
and ground-truth references, and includes pipeline scripts plus a
rerun.io visualisation that overlays multiple trajectories with the
live stereo camera projected inside each estimator's frustum. First
head-to-head result: OpenVINS 0.295 m APE vs VINS-Fusion 0.248 m on
EuRoC MH_01_easy.
- Trajectory analysis methodology and case studies
- Estimator consistency — NEES, NIS, and auditing the covariance
- Benchmark methodology — run-to-run variance and paired significance
- Runtime evaluation — latency, throughput, and tail costs
- Diagnostic procedure for broken VIO
- Estimator parameter reference
- Visual-Inertial Navigation Systems: An Introduction
- IMU Propagation Derivations (OpenVINS)
- Error State Kalman Filter VIO (ESKF-VIO)
- Overview
- Sensor setup
- Calibration files and projection matrices
- Ground-truth poses
- Displaying ground-truth poses in Rerun
- Stereo vision
- Sparse/dense reconstruction from known poses with COLMAP
- Understanding KITTI poses
- Dataset recipes — EuRoC, TUM-VIO, M2DGR, 4Seasons
- M3DGR — Multi-sensor, Multi-scenario, Massive-baseline SLAM Dataset for Ground Robots
- rvp group SLAM datasets
- DSEC — Stereo Event Camera Dataset for Driving Scenarios
- Hilti x Trimble SLAM Challenge 2026
- Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
- A Comparison of Modern General-Purpose Visual SLAM Approaches
- ETH3D
The lio_benchmark/ subproject is a self-contained
Gazebo simulator for LiDAR-inertial odometry development. It ships a
SubT-style rover with a 16-beam LiDAR + IMU + cameras, three
cave/tunnel worlds, and FAST-LIO wired into the rover by default.
One-line bring-up via docker compose. Unlike VIO, LIO works
correctly on this sim (FAST-LIO consistently produces 0.3–1.5 %
end-point error on rover recordings) — the architectural reason is in
the VIO diagnostic guide §3.
- lio_benchmark README and quick start
- Simulator GPU/rendering performance notes
- FAST-LIO + LiDAR tuning reference
- Why LIO is robust to bad IMU but VIO isn't
- Robust Real-time LiDAR-inertial Initialization
- Lidar SLAM for Automated Driving (MATLAB)
- FAST-LIO
- iG-LIO — incremental GICP tightly-coupled LIO
- DLIO — Direct LiDAR-Inertial Odometry with Continuous-Time Motion Correction
- CT-LIO — Continuous-Time LiDAR-Inertial Odometry
- LIO-SAM
- GLIM — versatile and extensible range-based 3D mapping framework
- LIMO — Lidar-Monocular Visual Odometry
- Structure from Motion from Scratch
- Rotation Averaging
- Single rotation averaging — chordal/SVD mean, quaternion (Markley) mean, Karcher mean, Weiszfeld median
- Multiple rotation averaging (rotation synchronization) — the global-SfM problem, Govindu → Martinec–Pajdla → Chatterjee–Govindu IRLS → Shonan
- Why outlier relative rotations, not the optimizer, are the hard part
- Robust Rotation Averaging (talk)
- Bundler
- Noah Snavely Reprojection Error
- Global Structure-from-Motion Revisited
- LightGlue
- XRefine: Attention-Guided Keypoint Match Refinement
- DenseSFM
- Pixel-Perfect Structure-from-Motion
- image-matching-webui
- Implementation tutorial
- Depth + pose training objective
- Supervised depth add-on
- Shapes, sizes and VRAM knobs
- Minimal architectures
- Training recipe
- Evaluation
- Baselines for a 4 GB GPU
- Multi-scale output
- Implementation tips
- Overview of the main families
- Pose regression heads — CNN vs ViT
- ViT-based VO architectures
- Temporal modeling
- Input and output representations
- Self-supervised VO
- DUSt3R & MASt3R notebook
- The paper landscape
- The 2026 role inversion
- VGGT-SLAM loop closure
- Persistent state as a learned filter
- The pointmap
- Pointmap head
- The canonical frame
- Global alignment, not bundle adjustment
- Matching — MASt3R's descriptor head
- Recovering intrinsics
- Recovering the relative pose
- Sparse global alignment (MASt3R-SfM)
- Speedy MASt3R
- MASt3R-SLAM with Rerun
- Gaussian Splatting
- instant-ngp
- Hierarchical 3D Gaussians for large datasets
- EvDeblurNeRF
- MegaScenes
- Morpheus
- Trajectory analysis
- Why naive comparison fails — the alignment problem
- Absolute Pose Error (APE / ATE)
- Relative Pose Error (RPE)
- Alignment — Umeyama Sim(3), SE(3), yaw-only, posyaw
- Sub-trajectory drift
- Which statistics to report
- Tools —
evoandrpg_trajectory_evaluation - Reporting recipe
- Evaluation metrics for poses and trajectories
- The one residual behind ATE and mAA
- Same Umeyama, different robustness
- The convention trap: $C = -R^\top t$
- Two different metrics are both called mAA
- image-matching-benchmark
- Estimator consistency
- Accuracy and consistency are different failure modes
- NEES — Normalised Estimation Error Squared
- The gauge trap — why absolute NEES is usually meaningless
- ANEES — and why one run tests almost nothing
- NIS — consistency without ground truth
- Diagnosis — FEJ, double-counting, sparsification
- Where P comes from — and the convention trap
- Reporting recipe
- Benchmark methodology
- "Deterministic" algorithms are not
- How many runs, and what to do with them
- Compare paired, not unpaired
- Bootstrap confidence intervals
- Aggregation traps — failures, weighting, normalisation
- Cherry-picking and leaderboard overfitting
- Reporting checklist
- Runtime evaluation
- Three numbers that get called "speed"
- The mean is the wrong statistic — report the tail
- Per-thread budgets
- Measurement traps — build type, playback rate, throttling
- Memory and energy
- Tooling — perf, Tracy, ros2 topic delay
- Reporting template
- Types of loss functions used in VO
- Rotation loss overview
- Quaternion loss
- Geodesic loss
- Full transformation loss $SE(3)$
- How to pick the weights
- Modern Robotics Mechanics, Planning, and Control (Kevin M. Lynch, Frank C. Park)
- Modern Robotics Mechanics, Planning, and Control (Instructor Solution Manual, Solutions )
- MODERN ROBOTICS MECHANICS, PLANNING, AND CONTROL (Practice Exercises)
- Basic Knowledge on Visual SLAM: From Theory to Practice, by Xiang Gao, Tao Zhang, Qinrui Yan and Yi Liu
- STATE ESTIMATION FOR ROBOTICS (Timothy D. Barfoot)
- SLAM for Dummies
- VSLAM Handbook
- SLAM Handbook
- Matrix Calculus (for Machine Learning and Beyond)
- Reinforcement Learning: A Comprehensive Overview
- Linear Algebra for Computer Vision, Robotics, and Machine Learning