PSET 8 SAM + VLM Question

book/segmentation.html

@@ -490,6 +490,21 @@ <h1><a href="index.html" style="text-decoration:none;">Robotic Manipulation</a><
       </ol>
 
     </exercise>    
+
+    <exercise><h1>Vision-Language Segmentation</h1>
+
+      <p> For this exercise, you will explore how Vision-Language Models (VLMs) and the Segment Anything Model (SAM) can be combined to achieve
+         language-driven object segmentation. You will work exclusively in 
+         <script>document.write(notebook_link('segmentation', notebook='segmentation_sam', link_text='this notebook'))</script>. You will be asked to complete the
+      following steps: </p>
+
+      <ol type="a">
+        <li> Analyze SAM's segmentation capabilities and understand its limitations in object identification</li>
+        <li> Use a Vision-Language Model to generate bounding boxes from natural language prompts</li>
+        <li> Combine VLM-generated bounding boxes with SAM to produce precise segmentation masks for specified objects</li>
+      </ol>
+
+    </exercise>    
   </section>
 
 </chapter>