Skip to content

Latest commit

ย 

History

History
38 lines (30 loc) ยท 1.19 KB

File metadata and controls

38 lines (30 loc) ยท 1.19 KB

๋Œ€ํ•œ๋ฏผ๊ตญ ๋Œ€์„  ํŠธ๋ Œ๋“œ ๋ถ„์„ ์‹œ๊ฐํ™”

๋ณธ ํ”„๋กœ์ ํŠธ๋Š” 2025๋…„ ์ œ21๋Œ€ ๋Œ€ํ†ต๋ น ์„ ๊ฑฐ๋ฅผ ์•ž๋‘๊ณ , ๋Œ€์ค‘์˜ ๊ด€์‹ฌ์‚ฌ ๋ณ€ํ™”๋ฅผ ๋‹ค์–‘ํ•œ ๋ฐ์ดํ„ฐ ์†Œ์Šค๋ฅผ ํ†ตํ•ด ๋ถ„์„ํ•˜๊ณ  ์‹œ๊ฐํ™”ํ•˜๋Š” ๊ฒƒ์„ ๋ชฉํ‘œ๋กœ ํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ“Œ ํ”„๋กœ์ ํŠธ ๋ชฉํ‘œ

  • ์„ ๊ฑฐ ๊ธฐ๊ฐ„ ๋Œ€์ค‘์˜ ๊ด€์‹ฌ์‚ฌ๋ฅผ ์‹œ๊ฐํ™”ํ•œ ๋Œ€์‹œ๋ณด๋“œ ์ œ๊ณต
  • End-to-end ๋ฐ์ดํ„ฐ ํŒŒ์ดํ”„๋ผ์ธ ๊ตฌ์ถ•
  • Docker, Airflow, Snowflake ๋“ฑ ๋ฐ์ดํ„ฐ ์—”์ง€๋‹ˆ์–ด๋ง ์—ญ๋Ÿ‰ ๊ฐ•ํ™”

๐Ÿ—ƒ๏ธ ์‚ฌ์šฉ ๋ฐ์ดํ„ฐ

  • Naver DataLab API
  • Google Trends (Pytrends)
  • ๋„ค์ด๋ฒ„ ๋‰ด์Šค

๐Ÿ› ๏ธ ์‚ฌ์šฉ ๊ธฐ์ˆ 

  • Language: Python, SQL
  • Container: Docker
  • Workflow: Apache Airflow
  • Storage: Amazon S3, Snowflake
  • Visualization: Apache Superset
  • Collaboration: Slack, ZEP, Notion, Git
  • IDE: Visual Studio Code

๐Ÿงฉ DAG ๊ตฌ์„ฑ

  • Naver Datalab DAG
  • Google Trends DAG
  • Naver News DAG

๐Ÿง  ๊ธฐ๋Œ€ํšจ๊ณผ

  • ๋‹ค์ฐจ์›์  ๋Œ€์„  ํŠธ๋ Œ๋“œ ํŒŒ์•…
  • ์ •์ฑ… ๋ฐ ์ „๋žต ์ˆ˜๋ฆฝ์— ์œ ์˜๋ฏธํ•œ ์ธ์‚ฌ์ดํŠธ ์ œ๊ณต
  • ์ž๋™ํ™”๋œ ๋ฐ์ดํ„ฐ ์ฒ˜๋ฆฌ๋กœ ์šด์˜ ํšจ์œจ์„ฑ ๊ฐ•ํ™”

๐Ÿ”ง ํ–ฅํ›„ ๊ฐœ์„  ๋ฐฉํ–ฅ

  • CI/CD ํŒŒ์ดํ”„๋ผ์ธ ๊ตฌ์ถ•
  • dbt๋ฅผ ํ†ตํ•œ ๋ฐ์ดํ„ฐ ๋ชจ๋ธ๋ง ์ฒด๊ณ„ํ™”
  • ์˜ˆ์ธก ๊ธฐ๋ฐ˜ ํŠธ๋ Œ๋“œ ๋ถ„์„ ๋„์ž