🎭 Cartoon Animator

Transform any photo into an animated talking cartoon with synchronized audio!

✨ Features

• 🎨 Cartoonification - Convert photos to cartoon style using advanced OpenCV filters
• 🎬 Animation - Animate cartoons using First Order Motion Model
• 🔊 Audio Sync - Automatically sync audio from reference video
• 🌐 Web Interface - Easy-to-use browser interface
• ⚡ GPU Accelerated - Fast processing with CUDA support

🚀 Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Download Pre-trained Model

Download the VoxCeleb checkpoint:

• URL: https://drive.google.com/open?id=1PyQJmkdCsAkOYwUyaj_l-l0as-iLDgeH
• Save as: checkpoints/vox-cpk.pth.tar

Or use the download script:

python download_checkpoint.py

3. Run!

Web Interface (Easiest):

python app.py
# Open http://localhost:5000

Command Line:

python cartoon_animator.py photo.jpg talking_video.mp4 output.mp4

🎯 How It Works

1. Cartoonify - Your uploaded image is converted to cartoon style
2. Detect Keypoints - AI detects facial/body keypoints in both cartoon and video
3. Generate Motion - Motion from video is transferred to cartoon
4. Sync Audio - Audio from reference video is added to final output

📖 Usage

Web Interface

1. Start the server: python app.py
2. Open http://localhost:5000
3. Upload your image and reference video
4. Choose cartoon style
5. Click "Create Animated Cartoon"
6. Download your result!

Command Line

python cartoon_animator.py <source_image> <driving_video> <output_video>

Example:

python cartoon_animator.py selfie.jpg talking.mp4 animated_selfie.mp4

Cartoon Styles

• Advanced (Recommended) - Best quality with color quantization
• Bilateral - Smooth colors with sharp edges
• Pencil - Pencil sketch effect
• Stylization - Artistic stylization

🎨 Examples

Example 1: Talking Portrait

python cartoon_animator.py portrait.jpg speech_video.mp4 talking_portrait.mp4

Input: Portrait photo + Video of someone talking Output: Cartoon portrait that talks with synced audio

Example 2: Singing Character

python cartoon_animator.py character.png singing_video.mp4 singing_character.mp4

Input: Character image + Singing video Output: Animated singing cartoon

🛠️ Technical Details

Cartoonification

• Bilateral filtering for color smoothing
• Adaptive thresholding for edge detection
• Color quantization (K-means clustering)
• HSV enhancement for vibrant colors

Animation

• First Order Motion Model
• 10 keypoint detection
• Dense motion field generation
• Occlusion-aware rendering

Audio Processing

• Automatic audio extraction from reference video
• Frame-perfect synchronization
• AAC audio codec

📊 Performance

Hardware	Speed	Time for 30s video
RTX 3080	~30 FPS	~1 minute
RTX 2060	~20 FPS	~1.5 minutes
CPU (i7)	~3 FPS	~10 minutes

🔧 Requirements

• Python 3.7+
• PyTorch 1.8+
• OpenCV 4.5+
• 4-6 GB RAM
• 2-3 GB GPU memory (optional)

📁 Project Structure

midnightproject/
├── cartoonify.py           # Cartoonification module
├── cartoon_animator.py     # Main pipeline
├── app.py                  # Web interface
├── modules/                # Animation models
│   ├── util.py
│   ├── keypoint_detector.py
│   ├── dense_motion.py
│   └── generator.py
├── config/                 # Model configuration
│   └── vox-256.yaml
├── checkpoints/            # Pre-trained models
└── requirements.txt        # Dependencies

🎓 Credits

Cartoonification

Based on OpenCV techniques from:

• Data Flair: https://data-flair.training/blogs/cartoonify-image-opencv-python/

Animation

Based on First Order Motion Model:

• Paper: "First Order Motion Model for Image Animation" (NeurIPS 2019)
• Authors: Aliaksandr Siarohin et al.
• Repository: https://github.com/AliaksandrSiarohin/first-order-model

🆘 Troubleshooting

"No module named 'torch'"

pip install torch torchvision

"Checkpoint not found"

Download the checkpoint file and place it in checkpoints/vox-cpk.pth.tar

"CUDA out of memory"

The system will automatically fall back to CPU if GPU memory is insufficient

Poor cartoon quality

Try different cartoon styles (advanced, bilateral, pencil, stylization)

Audio not syncing

Ensure the reference video has audio and is in a supported format (MP4, AVI)

📄 License

For educational and research purposes. See original repositories for licensing details.

🎉 Have Fun!

Create amazing animated cartoons and share them with the world! 🎭✨