Gradio GUI

.gitignore

@@ -0,0 +1,9 @@
+data
+imgs
+my_pipeline
+segmentation.ipynb
+gaussiansplatting/submodules/*
+gaussiansplatting/dependencies/*
+**/__pycache__
+outputs
+**.pth

README.md

@@ -1,6 +1,53 @@
 # SAGS (Segment Anything in 3D Gaussians)
 > [**Segment Anything in 3D Gaussians**](https://arxiv.org/abs/2401.17857)
 
+## SAGS with a Gradio Interface
+
+SAGS with a Gradio Interface.
+Multiview Annotation and Merge.
+
+Gradio版本的SAGS，支持多视角标注，支持多次标注并合并输出为一个Gaussian Splatting结果
+需要的文件目录：Gaussian训练的数据集文件夹和结果文件夹。结果文件夹中args文件保存了数据集文件夹的目录，因此只需要打开结果文件夹的目录即可。
+
+### 25.05.03 app2.py
+app2.py：修改用户流程，解决遮挡物体分割不佳的问题
+1. 使用单视角标注，得到单视角的GS结果
+2. 融合多个单视角GS的结果，加入最终的GS分割结果
+
+New Pipeline in app2.py: (to solve poor segmentation of occluded objects)
+1. Using single perspective annotation to obtain GS results from a single perspective
+2. Continuously merging new single perspective results into the final result
+
+compare to app.py:
+1. Directly using multi view annotation to obtain multi view masks
+2. Generate the final GS result directly using multi perspective annotation
+
+### 25.05.02
+No Cached Result of SAM. Now the GPU Memory Usage is stable.
+不使用计算结果的缓存了。使用缓存的版本是app_cached_result(deprecated).py
+
+### 25.05.01
+Speedup x10 (in multi view segment) by change some numpy operator to torch, see app.py
+every single click need 1800MB GPU Memory(with 90pic dataset). I will fix it.
+分割确实快了，还需要去除保存计算结果缓存的模块
+
+### 25.04.30
+Use it to segment any gaussian easily, capable with multiview annotation ability.
+Text prompt would be release when I'm available.
+Undo and Redo might incur additional GPU Memory cost.
+Requires gradio==5
+
+Gradio版本的SAGS，支持多视角标注，背面也可以加约束了。
+发现背面加约束这个事情有点鸡肋，猜测是3D投票过程并不能很好支持复杂的遮挡关系
+每次的撤消和重做的结果都保存了，不会重复存，但是反复折腾可能会占点显存。
+
+计划：
+- ✅改为进行多次单视角GS分割，将多个视角得到的GS结果直接组合得到最终结果
+- ✅尝试加快分割，不使用计算结果的缓存，减少GPU内存占用
+- ⭕️有空想整一个文本提示版本
+
+
+
 ## Introduction
 3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis, benefiting from its high-quality rendering results and real-time rendering speed. However, the 3D Gaussians learned by 3D-GS have ambiguous structures without any geometry constraints. This inherent issue in 3D-GS leads to a rough boundary when segmenting individual objects. To remedy these problems, we propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS to improve segmentation accuracy while preserving segmentation speed. Specifically, we introduce a Gaussian Decomposition scheme, which ingeniously utilizes the special structure of 3D Gaussian, finds out, and then decomposes the boundary Gaussians. Moreover, to achieve fast interactive 3D segmentation, we introduce a novel training-free pipeline by lifting a 2D foundation model to 3D-GS. Extensive experiments demonstrate that our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.