Skip to content

3D Reconstruction for Scene Understanding using Semantic/Language knowledge

Notifications You must be signed in to change notification settings

SPIResarchGroup/3DSceneUnderstanding

ย 
ย 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

23 Commits
ย 
ย 
ย 
ย 

Repository files navigation

3D Scene Understanding

Overall

This repository focuses on Large Scene Reconstruction in Autonomous Driving. Our group belongs to Pseudo3DV.
This curated collection of resources is related to tackling real-world challenges in autonomous driving perception using NeRF (Neural Radiance Fields) and 3D Gaussian Splatting (3D-GS).

Furthermore, we are currently operating a paper and code review study in PsuedoLab. Go check it out! (Every Tuesday at 7pm, PseudoLab Discord Room YL!) ๐Ÿš—๐Ÿ’ก

์Šคํ„ฐ๋”” ์†Œ๊ฐœ

"๋ฆฌ์ฒ˜๋“œ ํŒŒ์ธ๋งŒ(Feynman)" What I cannot create, I do not understand.

์•ˆ๋…•ํ•˜์„ธ์š”. ์ €ํฌ๊ฐ€ ์ง„ํ–‰ํ•  ์Šคํ„ฐ๋””๋Š” ๋กœ๋ด‡์ด ์„ธ์ƒ์„ ์ดํ•ดํ•˜๋Š” ๋ฐฉ์‹์„ ๋‹ค๋ฃจ๋Š” 3D Scene Understanding ๋…ผ๋ฌธ๋“ค์„ ์‚ดํŽด๋ณด๊ณ ์ž ํ•ฉ๋‹ˆ๋‹ค. ๊ธฐ๋ณธ์ ์œผ๋กœ 3์ฐจ์› ์žฌ๊ตฌ์„ฑ ๋ชจ๋ธ์„ ๊ธฐ๋ฐ˜์œผ๋กœ (Semantic, Language ๋“ฑ ์ „๋ฐ˜์— ๊ฑธ์ณ์„œ) ์„ธ์ƒ์„ ์ดํ•ดํ•˜๋Š” ๋ฐฉ์‹์„ Surveyํ•จ์œผ๋กœ์จ 3D Reconstruction์˜ ํ™œ์šฉ ๊ฐ€๋Šฅ์„ฑ์„ ๋ชจ์ƒ‰ํ•˜๊ณ ์ž ํ•ฉ๋‹ˆ๋‹ค. ๊ธฐ๋ณธ์ ์ธ 3D Reconstruction์„ ๋ฉ”์ธ์œผ๋กœ ๋‹ค๋ฃจ๊ธฐ๋Š” ํ•˜๋‚˜, VLA, Scene Graph Generation ๋“ฑ ์ž์‹ ์ด ๊ณต๋ถ€ํ•˜๊ณ  ์‹ถ์€ ๋„๋ฉ”์ธ์— ๋งž์ถฐ ์ง„ํ–‰ํ•˜์…”๋„ ๋ฌด๋ฐฉํ•ฉ๋‹ˆ๋‹ค.
๋…ผ๋ฌธ์€ ๋…„๋„๋ณ„๋กœ ์ˆœ์ฐจ์ ์œผ๋กœ ์ฝ์œผ๋ฉฐ, ์ฃผ์ œ์— ๋ฒ—์–ด๋‚˜์ง€ ์•Š์œผ๋ฉฐ ์ž์‹ ์˜ ๋„๋ฉ”์ธ์— ๋งž๋Š” ๋…ผ๋ฌธ์„ ์ •ํ•˜์—ฌ ๋ฆฌ๋ทฐํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋ฆฌ๊ณ  ํ•˜๊ธฐ์˜ Schedule์„ ์ฐธ๊ณ ํ•ด์ฃผ์‹œ๊ณ , ๋ชจ๋“  ๋Ÿฌ๋„ˆ๋ถ„๋“ค์ด ์ž์‹ ์˜ ๋„๋ฉ”์ธ์˜ ๋…ผ๋ฌธ ๋ฐ ๊ด€๋ จ ์ค‘์š”ํ•œ ์ฝ”๋“œ๋ฅผ ๋ฆฌ๋ทฐํ•˜๋Š” ํ˜•์‹์œผ๋กœ ์ง„ํ–‰ํ•˜๋ฉฐ, ๊ฒฉ์ฃผ๋กœ ๋ฐœํ‘œ๋ฅผ ์ง„ํ–‰ํ•˜๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.** (์Šคํ„ฐ๋””์›์ด ๊ด€์‹ฌ์žˆ์œผ์‹  ๋…ผ๋ฌธ๊ณผ ๋ฐœํ‘œํ•˜๊ณ  ์‹ถ์€ ๋…ผ๋ฌธ์ด ์žˆ์œผ์‹œ๋ฉด ๋ฏธ๋ฆฌ ๋ง์”€์ฃผ์‹œ๋ฉด ๋ฐ˜์˜ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.) ์ด์ „ ์Šคํ„ฐ๋”” NeRFwithRealWorld, UrbanSceneRecon๋ฅผ ์ฐธ๊ณ ํ•˜์‹œ๋ฉด ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

์ €ํฌ ์Šคํ„ฐ๋””๋Š” ๋ฐ€๋„ ์žˆ๊ณ , ์‹ฌ๋„ ์žˆ๋Š” ์ดํ•ด๋ฅผ ๋ชฉํ‘œ๋กœ ์ง„ํ–‰ํ•  ์˜ˆ์ •์ž…๋‹ˆ๋‹ค. ๊ทธ๋ฆฌํ•˜์—ฌ ๋‹ค์Œ๊ณผ ๊ฐ™์€ LINK๋ฅผ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค.

๋ชจ์ง‘ ๋Œ€์ƒ

  • ์ ๊ทน์ ์€ ์˜์‚ฌ์†Œํ†ต ๋ฐ ๋„์ „์ •์‹ ์„ ๊ฐ€์ง€์‹  ๋ถ„
  • NeRF์— ๋Œ€ํ•ด์„œ ์ฝ์–ด๋ณด๊ณ , ํ…Œ์ŠคํŠธ ํ•ด๋ณด์‹  ๋ถ„
  • ๊ฑด์„ค์ ์ธ ์˜์‚ฌ์†Œํ†ต์„ ์ง€ํ–ฅํ•˜๋Š” ๋ถ„
  • ์ฝ”๋“œ ์ฝ๋Š”๋ฐ ํฐ ๋ถ€๋‹ด ์—†์œผ์‹  ๋ถ„
  • ํ–ฅํ›„ ๋ชจ์ž„ ์ง„ํ–‰ ์‹œ ์ฐธ์—ฌ ๊ฐ€๋Šฅํ•˜์‹  ๋ถ„

${\textsf{\color{red}* NeRF, 3D-GS์— ๋Œ€ํ•œ ๊ธฐ๋ณธ์ ์ธ ์ดํ•ด๊ฐ€ ํ•„์š”ํ•˜๋ฉฐ, ์ ๊ทน์ ์œผ๋กœ ๊ฑด์„ค์ ์ธ ์˜์‚ฌ์†Œํ†ต ํ•˜์‹ค ๋ถ„์„ ์„ ํ˜ธํ•ฉ๋‹ˆ๋‹ค.}}$ ${\textsf{\color{red}* ์บ (CAM)์„ ํ•„์ˆ˜๋กœ ์ผœ์•ผ ํ•ฉ๋‹ˆ๋‹ค.}}$

๊ฒฉ์ฃผ ๋…ผ๋ฌธ๊ณผ ์ฝ”๋“œ๋ฅผ PPT์— ๋ฐ˜๋“œ์‹œ ์ฒจ๋ถ€ํ•˜์—ฌ ๋ฐœํ‘œ๋ฅผ ์ง„ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

๋ชจ์ง‘ ์ธ์›

  • 4-5๋ช…

์ฃผ์˜์‚ฌํ•ญ

  • ๋ฐœํ‘œ ์‹œ PPT๋ฅผ ํ™œ์šฉํ•ด์•ผํ•ฉ๋‹ˆ๋‹ค. (์ฒญ์ค‘์ด ์ดํ•ดํ•  ์ˆ˜ ์žˆ๋„๋ก ์ค€๋น„ํ•ด์ฃผ์…”์•ผ ํ•ฉ๋‹ˆ๋‹ค.) (๊ฒฝ๊ณ  2ํšŒ ์‹œ ์ œ๋ช…)
  • ์Šคํ„ฐ๋”” ์ฐธ๊ฐ€๊ฐ€ ๋ถˆ๊ฐ€๋Šฅํ•œ ๊ฒฝ์šฐ ๋ฏธ๋ฆฌ 5์ผ ์ „์—๋Š” ๋ง์”€์ฃผ์…”์•ผ ํ•ฉ๋‹ˆ๋‹ค. (๊ฒฝ๊ณ  2ํšŒ ์‹œ ์ œ๋ช…)

ํ™œ๋™ ๊ธฐ๊ฐ„

์ฒซ ๋ชจ์ž„ ๋‚ ์งœ & ์‹œ๊ฐ„ & ์žฅ์†Œ

  • ๋‚ ์งœ: 9์›” 9์ผ (ํ™”์š”์ผ)
  • ์‹œ๊ฐ„: ์˜คํ›„ 7์‹œ
  • ์žฅ์†Œ: #ROOM-YL

Team member

Schedule

Date Paper Task Speaker
25/09/09 OT ---- Seongjun Choi
25/09/16 Survey 3DGS, Survey 3DGS in Robotics basic study All Members
25/09/23 Survey 3DGS, Survey 3DGS in Robotics Survey on each domain All Members
25/10/14 TBD...
25/10/28 TBD...
25/11/11 TBD...
25/11/25 TBD...
25/12/09 TBD...
25/12/23 TBD...

About

3D Reconstruction for Scene Understanding using Semantic/Language knowledge

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published