WebARKitLib-rs 🦀

WebARKitLib-rs is a high-performance, memory-safe Rust port of the WebARKitLib (originally C/C++).

This project aims to provide a pure-Rust implementation of the core ARToolKit algorithms, targeting both native systems and WebAssembly (WASM) for high-performance augmented reality in the browser.

Note

This project is currently a Work in Progress. Core marker detection and pose estimation are functional. KPM/NFT (Natural Feature Tracking) is in an early stage with partial Rust porting -- a fully idiomatic Rust KPM pipeline with a working example is the primary goal for v1.0.0.

🌟 Key Features

• Pure Rust: Built for safety, speed, and modern concurrency. (crates.io)
• Dual WASM Strategy: Automated release of both Standard and SIMD binaries to maximize performance and compatibility.
• SIMD-Accelerated Pattern Matching: Uses i16 fixed-point arithmetic and WASM i32x4_dot_i16x8 instructions for ultra-fast correlation matching.
• Barcode Marker Support: Robust decoding for square matrix markers (3x3 to 6x6) with ECC.
• WASM Ready: High-performance tracking in the browser via WebAssembly. (@webarkit/webarkitlib-wasm)
• Side-Effect Free: Pure mathematical engine, easy to test and integrate.
• CI-Integrated Benchmarking: Performance parity with original C implementation.

⚡ Performance & Accuracy

• SIMD-Accelerated Pattern Matching: Uses i16 fixed-point arithmetic for the core correlation loops, leveraging WASM i32x4_dot_i16x8 instructions for significant speedups.
• Numerical Parity: Our SIMD and Scalar paths are calibrated to produce bit-identical results (including rounding logic in grayscale and thresholding), ensuring deterministic behavior across all devices.
• Sub-millisecond Tracking: Core detection loop is optimized for sub-millisecond execution on standard ARM/x86 hardware.

🚀 Getting Started

Add to Your Project

Add webarkitlib-rs to your Cargo.toml:

[dependencies]
webarkitlib-rs = "0.3"

To enable the C++ FFI backend for KPM (Natural Feature Tracking):

[dependencies]
webarkitlib-rs = { version = "0.3", features = ["ffi-backend"] }

Native Rust Example

To see the marker detection in action on your local machine, run the provided simple example:

cargo run --example simple

This example loads a camera parameter file, a marker (pattern or barcode), and a sample image, performing detection and outputting the 3D pose extrinsics.

Barcode Detection Example

A unified, parameterized barcode example is available for testing all supported matrix code types:

# Default: 3x3 matrix code type, auto-sweeps threshold 60–180
cargo run --example barcode

# Specify a matrix code type (e.g. 4x4) and supply the matching marker image
cargo run --example barcode -- -m 4x4 -i crates/core/examples/Data/marker_07_4x4.jpg

# Use a fixed threshold instead of auto-sweeping
cargo run --example barcode -- -m 3x3 -t 100

# Full example: type, threshold, and custom image path
cargo run --example barcode -- -m 5x5 -t 120 -i path/to/marker.jpg

Options:

Flag	Long form	Description	Default
-m	--matrix-code-type	Matrix code type (3x3, 3x3parity65, 3x3hamming63, 4x4, 4x4bch1393, 4x4bch1355, 5x5, 5x5bch22125, 5x5bch2277, 6x6)	3x3
-t	--threshold	Fixed labeling threshold (0–255). When omitted, sweeps 60–180 in steps of 20	(auto-sweep)
-i	--image	Path to the input image	bundled 3x3 marker image

WebAssembly (WASM) Demo

The WASM port allows you to run the AR engine directly in most modern browsers.

1. Build the modules: Use the npm script to generate both Standard and SIMD bundles (works on all platforms):

   npm run build:wasm

   Alternatively, download the pre-built wasm-package artifact from the latest CI run and extract its contents into crates/wasm/pkg/.

2. Run the demo: The www folder contains two web demos. Serve it with any local HTTP server:

   cd crates/wasm/www
   # Serve using any local HTTP server, e.g.:
   npx serve .

   • simple.html – static image demo with engine selector and threshold visualization.
   • simple_video_marker_example.html – live webcam demo with engine selector, marker type (pattern/barcode) selector, and threshold slider.

📊 Benchmarking

We maintain a strict performance comparison with the original C library to ensure our Rust port remains competitive.

Detailed SIMD performance results and reproduction steps can be found in the BENCHMARKS.md file.

Running the Comparison

1. Bootstrap the C library:

   cd benchmarks/c_benchmark
   python ../bootstrap.py --bootstrap-file libraries.json

2. Execute the Suite:

   # Rust Benchmark
   cargo bench -p webarkitlib-rs --bench marker_bench
   
   # C Benchmark (Manual build required once)
   cd benchmarks/c_benchmark/build
   cmake --build . --config Release
   ./c_benchmark ../../data/camera_para.dat ../../data/patt.hiro ../../data/hiro.raw 429 317

Tracking Performance

We use criterion to track performance over time. You can save specific snapshots as "baselines":

# Save a milestone baseline
cargo bench -- --save-baseline milestone-20260307

🏗️ Project Structure

The workspace contains two crates:

• crates/core (webarkitlib-rs): The unified core AR engine (pure Rust), including:
  • ar2 module: NFT tracking algorithms, .iset / .fset marker I/O.
  • kpm module: Keypoint Matching with pluggable backends (Rust + C++ FFI).
  • Core modules: image processing, pattern matching, labeling, ICP, pose estimation.
• crates/wasm (webarkitlib-wasm): WASM bindings, dual-build scripts, and diagnostic web demo.
• benchmarks: C vs Rust performance comparison suite.
• ARCHITECTURE.md: Detailed technical overview of the library's design and SIMD optimizations.

🗺️ Roadmap

Completed Milestones

• M1 -- KPM/NFT Core (partial): Ported the initial KPM (Keypoint Matching) scaffolding -- binary feature types, matching orchestration, ICP-based pose refinement, and .fset3 / .iset / .fset I/O. The C++ FreakMatcher FFI backend works but the full pipeline is not yet wired end-to-end.
• M2 -- AR2 I/O: Ported AR2 image set (.iset) and feature set (.fset) binary I/O from C to Rust.
• M3 -- Architectural Consolidation: Unified the workspace from 4 crates down to 2 (webarkitlib-rs + webarkitlib-wasm), with KPM and AR2 as submodules of the core crate.

🎯 Short-term Goals (toward v1.0.0)

• Complete KPM in idiomatic Rust: Port the remaining KPM feature extraction and matching logic to pure Rust, removing the C++ FFI dependency, and ship a working end-to-end NFT example.
• Enhanced Documentation: Expand API reference with complete module-level docs, integration walkthroughs for JS/TS, and detailed usage examples.
• WASM Memory Management: Improve resource cleanup when switching engines or markers in long-running browser sessions.

🔭 Long-term Vision (post v1.0.0)

• Multi-Marker Support: Port arMulti logic to enable tracking of multiple markers simultaneously.
• Advanced Video Abstraction: Develop a cross-platform video handling layer to simplify integration with various input sources.

📜 Credits

This project is a port of the excellent WebARKitLib project. Special thanks to the original ARToolKit contributors.

📄 License

This project is licensed under the LGPLv3 License - see the LICENSE file for details.