DroneAgent

DroneAgent - drone autonomy powered by Generative AI

Features

• Flying drone fully controlled via Gemini LLM-powered AI agent built using smolagents library (0 human control)
• As input agent receives a text description of the mission and then tries to execute it
• Agent can do basic actions (take off, land, basic movement around) as well as more complex tasks (navigating in the environment, finding things in the surrounding environment - see demo below)
• Agent receives photos from the drone camera at each step of the mission
• Basic agent observability (via LangFuse) and terminal logs

YouTube Demo

Drone mission: Find something I can eat and get closer to it

After exploring the room around, the drone lands in front of a table with a basket of fruits.

Full video with DroneAgent "thinking" process.

Prerequisites

1. DJI Tello drone
2. Cable internet connection (since WiFi will be used by the drone)
3. Google Gemini Developer account
4. uv python package and project manager

Installation

1. Clone the repository:

   git clone https://github.com/VladKha/drone-agent.git
   cd drone-agent

2. Install uv

3. Set up the Python venv (using Python 3.12):

   make create_venv

4. Install the required packages:

   make install_requirements

Environment Variables

Create a .env file in the root directory from .env.example file and add all required environment variables:

• GOOGLE_API_KEY from AI Studio
• LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, LANGFUSE_HOST from LangFuse

How to Run

1. Turn on and connect to your DJI Tello drone via WiFi
2. Connect to the internet via cable
3. Run the agent: make run_app

Technologies Used

• Python 3.12
• smolagents: agent framework
• Google Gemini: LLM for agents "brains"
• DJITelloPy: DJI Tello drone python interface
• LangFuse: LLM observability
• dotenv: managing environment variables
• tenacity: retrying failed requests