Some inquiry questions

[1713876447]

Hello, during my process of reproducing the code, when extracting 768-dimensional hierarchical features, I used
CLIP ViT-L-14-336 model
instead of openseg. However, when performing hierarchical queries, significant errors occurred.
So,could you provide the query words or the file sai_nag_openseg.pt to me identify the root cause of the problem?
Thank for your help!

[EryiXie]

Hi, CLIP ViT-L-14-336 and OpenSeg are two different models, though the final token are the same 768-dims. Therefore, the performance gap is expected. In fact in our testing, for object classification (open vocabulary feature <-> a closed set of text labels), OpenSeg performans much better than CLIP.
For the openseg feature per scene of 3RScan dataset (RIO-10 more specific), I need to check the license and term of use of the oringinal datset to see if I can redistribute (likely yes, since GaussianWorld did it).
And I will come back to you when the data is ready uploaded.

[1713876447]

Thank you very much for your response and for making this project publicly available.

I eventually used OpenSeg to extract the language-aligned features. I would like to kindly ask whether you would also consider releasing the evaluation scripts, as I am currently working on the evaluation of both “open-vocabulary semantic segmentation on ScanNet” and “relationship-guided 3D instance segmentation on ScanNet++”.

For the ScanNet open-vocabulary semantic segmentation task, I obtained an mIoU of approximately 0.560 on a single scene when averaging only over the categories that appear in the ground-truth annotations. Therefore, I would like to clarify how the overall evaluation over the ten ScanNet scenes is computed: should the mIoU be averaged over all categories that appear in either the ground truth or the predictions, only over the categories present in the ground truth, or over the fixed set of 19 classes?

For the ScanNet++ relationship-guided 3D instance segmentation evaluation, I am still in the process of constructing the evaluation pipeline, as I have not found a standard reference implementation. If you could release the evaluation code, it would be extremely helpful for reproducing and further studying your work.

Thank you again for your excellent work and for open-sourcing the project.