feat(vision): glass and mirror depth filter via SAM3 confidence map

[pfmephisto]

Problem

Glass surfaces and mirrors produce unreliable depth readings from RGB-D sensors — they are either transparent (depth comes from the far side of the glass) or perfectly reflective (depth comes from a mirrored reflection). Both artefacts contaminate the fused cloud, produce floating geometry in the cell-complex, and can cause the MIP solve to choose wrong faces.

Goal

Use SAM3's open-vocabulary prompting to detect glass and mirror pixels, build a per-frame confidence map that down-weights those pixels, and apply the map before depth is fused into the point cloud.

Architecture

Option A — additional prompt pass (recommended)

Run a second SAM3 inference pass per frame with prompts ["glass", "mirror", "window pane", "transparent surface"], using a higher confidence threshold (e.g. 0.5 vs 0.25 for structural classes). The glass-detection pass does not contribute class labels to the label image — its output is only the confidence map.

This keeps glass artefacts out of segmentation labels while still allowing the depth filter to act.

Option B — extra prompts in the existing pass

Add glass and mirror to the default prompt list in libs/reusex/include/vision/tensor_rt/Data.hpp:36–60, mark those class IDs as "transparent material" in label_semantics.hpp, and exclude them from painting into the semantic label image while still extracting their masks for the confidence map.

Building the confidence map

After SAM3 forward (libs/reusex/src/vision/tensor_rt/Sam3.cpp:129–303), each DetectionBox carries:

• segmentation->mask: binary CV_8U mask in original image coordinates
• score: float in [0, 1] (object.hpp:196)

Confidence map construction:

cv::Mat glass_confidence(frame_size, CV_32F, cv::Scalar(1.0f));
for (auto& box : glass_detections) {
    // Where the mask is set, suppress confidence proportional to detection score
    cv::Mat roi = glass_confidence(box.bounding_box());
    roi.setTo(cv::Scalar(1.0f - box.score), box.segmentation->mask);
}

1.0 = trust depth; 0.0 = ignore depth. The existing depth filter API already accepts optional confidence maps (libs/reusex/include/segmentation/depth_filters.hpp:10–34).

Integration point in the annotation pipeline

libs/reusex/src/vision/annotate.cpp:96–194 — after model->forward(batch) (line 179):

model->forward(batch)
    ↓
extract_glass_masks(results)    ← new: filter DetectionBoxArray to glass classes
    ↓
build_glass_confidence_map()    ← new: CV_32F, same size as depth image
    ↓
filter_non_glass(results)       ← new: remove glass boxes from label image pass
    ↓
make_labeled_image(label_img, filtered_results)   ← unchanged
    ↓
dataset->save(results, confidence_map)  ← pass map through to reconstruction

Depth reconstruction consumer

rux create clouds calls depth back-projection in libs/reusex/include/segmentation/reconstruct.hpp. The confidence map should be applied there to zero (or NaN) pixels where confidence < threshold before the depth is projected into 3D.

Storing the confidence map

Two options:

1. Apply it immediately during create clouds and discard (simpler, zero storage cost)
2. Store it in ProjectDB alongside the label image for later re-use

Start with option 1 unless re-use is needed.

CLI exposure

rux create annotate --glass-filter [--glass-threshold 0.5]

The flag defaults to off to avoid changing existing behaviour. When enabled, glass/mirror pixels are zeroed from depth before cloud creation.

Acceptance criteria

• A frame known to contain a glass door/window produces a depth mask that excludes the glass area
• Glass pixels do not appear as semantic labels in the label image
• rux create clouds honours the confidence map when --glass-filter was used during annotation
• Existing annotation behaviour is unchanged when --glass-filter is not passed
• At least one unit test covers the confidence-map construction logic

Key files

File	Lines	Purpose
libs/reusex/src/vision/annotate.cpp	96–194	Main annotation loop — insert glass extraction after forward()
libs/reusex/include/vision/annotate.hpp	14–31	AnnotationConfig — add glass_filter bool + glass_threshold float
libs/reusex/include/vision/common/object.hpp	193–206	DetectionBox — score field and segmentation mask
libs/reusex/src/vision/tensor_rt/Sam3.cpp	831–957	Postprocess — where masks and scores are finalised
libs/reusex/src/vision/osd/osd.cpp	352–390	make_labeled_image — skip glass class IDs here
libs/reusex/include/segmentation/depth_filters.hpp	10–34	Existing confidence-map aware depth filter API
libs/reusex/include/segmentation/reconstruct.hpp	—	Back-projection — add confidence_map parameter
libs/reusex/include/vision/tensor_rt/Data.hpp	36–60	Default prompt list — add glass/mirror if using Option B