Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
294 changes: 291 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ For projects requiring multi-machine collaboration, Qlean provides a simple API
- 🛡️ **RAII-style Interface**: Automatic resource management ensures VMs are properly cleaned up
- 📦 **Out-of-the-Box**: Automated image downloading and extraction, no manual configuration needed
- 🐧 **Linux Native**: Native support for Linux hosts with multiple Linux distributions
- 🌐 **Multi-Distro Support**: Built-in support for Debian, Ubuntu, Fedora, and Arch Linux
- 🎯 **Custom Images**: Use any Linux distribution with URL/local path + checksum verification
- ⚡ **Performance Optimized****: Streaming hash computation with 5-30% performance improvement

## Usage

Expand Down Expand Up @@ -143,6 +146,246 @@ async fn test_ping() -> Result<()> {

For more examples, please refer to the [tests](tests) directory.

## Multi-Distribution Support

Qlean provides built-in support for multiple mainstream Linux distributions. Each distribution is optimized for ease of use and compatibility.

### Supported Distributions

| Distribution | Status | WSL Compatible | Boot File Extraction | Version |
|--------------|--------|----------------|---------------------|---------|
| **Debian** | ✅ Stable | ⚠️ Requires guestfish | Auto (guestfish) | Debian 13 (Trixie) |
| **Ubuntu** | ✅ Stable | ✅ **Fully Compatible** | Pre-extracted | Ubuntu 24.04 LTS (Noble) |
| **Fedora** | ✅ Stable | ⚠️ Requires guestfish | Auto (guestfish) | Fedora 41 |
| **Arch** | ✅ Stable | ⚠️ Requires guestfish | Auto (guestfish) | Latest |
| **Custom** | ✅ Stable | ✅ Optional | Flexible | Any Linux distro |

### Quick Start with Different Distributions

**Ubuntu (Recommended for WSL users)**
```rust
use anyhow::Result;
use qlean::{Distro, create_image, MachineConfig, with_machine};

#[tokio::test]
async fn test_ubuntu_vm() -> Result<()> {
// Ubuntu - WSL friendly, no guestfish required
let image = create_image(Distro::Ubuntu, "ubuntu-noble-cloudimg").await?;
let config = MachineConfig::default();

with_machine(&image, &config, |vm| {
Box::pin(async {
let result = vm.exec("lsb_release -a").await?;
assert!(result.status.success());
Ok(())
})
})
.await?;

Ok(())
}
```
Comment on lines +166 to +187
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you bother to run this example before submitting?


**Fedora**
```rust
#[tokio::test]
async fn test_fedora_vm() -> Result<()> {
// Fedora - requires guestfish on host
let image = create_image(Distro::Fedora, "fedora-41-cloud").await?;
let config = MachineConfig::default();

with_machine(&image, &config, |vm| {
Box::pin(async {
let result = vm.exec("cat /etc/fedora-release").await?;
assert!(result.status.success());
Ok(())
})
})
.await?;

Ok(())
}
```

**Arch Linux**
```rust
#[tokio::test]
async fn test_arch_vm() -> Result<()> {
// Arch - requires guestfish on host
let image = create_image(Distro::Arch, "arch-cloud").await?;
let config = MachineConfig::default();

with_machine(&image, &config, |vm| {
Box::pin(async {
let result = vm.exec("uname -r").await?;
assert!(result.status.success());
Ok(())
})
})
.await?;

Ok(())
}
```

> **💡 Tip**: Ubuntu is the recommended distribution for WSL users as it doesn't require guestfish for boot file extraction.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What motivated the focus on WSL compatibility here? Did you encounter a specific runtime issue under WSL? If so, please share details.


## Custom Images

Qlean supports using custom Linux distributions through URL downloads or local file paths, with **mandatory checksum verification** for security.

### Features

- 🔗 **Flexible Sources**: Download from URL or use local qcow2 files
- 🔒 **Security First**: Mandatory SHA-256/SHA-512 checksum verification
- 🪟 **WSL Compatible**: Optional pre-extracted kernel/initrd mode
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly were you planning to put here—an emoji?

- 🎯 **Any Distribution**: Support for any Linux distribution with qcow2 images

### Two Modes

#### Mode 1: Pre-extracted Boot Files (Recommended for WSL)

Provide the image, kernel, and initrd files separately with checksums. **This mode works on WSL** and doesn't require guestfish.
```rust
use anyhow::Result;
use qlean::{create_custom_image, CustomImageConfig, ImageSource, ShaType};

#[tokio::test]
async fn test_custom_ubuntu() -> Result<()> {
let config = CustomImageConfig {
// Main qcow2 image
image_source: ImageSource::Url(
"https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img".into()
),
image_hash: "abc123...".into(), // Get from SHA256SUMS file
image_hash_type: ShaType::Sha256,

// Pre-extracted kernel (WSL-friendly)
kernel_source: Some(ImageSource::Url(
"https://cloud-images.ubuntu.com/noble/current/unpacked/noble-server-cloudimg-amd64-vmlinuz-generic".into()
)),
kernel_hash: Some("def456...".into()),

// Pre-extracted initrd
initrd_source: Some(ImageSource::Url(
"https://cloud-images.ubuntu.com/noble/current/unpacked/noble-server-cloudimg-amd64-initrd-generic".into()
)),
initrd_hash: Some("ghi789...".into()),
};

let image = create_custom_image("my-ubuntu", config).await?;

// Use the image...
Ok(())
}
```

#### Mode 2: Auto-extract Boot Files (Native Linux only)

Provide only the image file with its checksum. Qlean will automatically extract kernel and initrd using guestfish.
```rust
#[tokio::test]
async fn test_custom_auto_extract() -> Result<()> {
let config = CustomImageConfig {
image_source: ImageSource::Url(
"https://example.com/my-distro.qcow2".into()
),
image_hash: "your-sha256-hash".into(),
image_hash_type: ShaType::Sha256,

// No kernel/initrd - will auto-extract
kernel_source: None,
kernel_hash: None,
initrd_source: None,
initrd_hash: None,
};

let image = create_custom_image("my-distro", config).await?;
Ok(())
}
```

### Using Local Files

You can also use local qcow2 images:
```rust
use std::path::PathBuf;

#[tokio::test]
async fn test_local_custom_image() -> Result<()> {
let config = CustomImageConfig {
image_source: ImageSource::LocalPath(
PathBuf::from("/path/to/my-image.qcow2")
),
image_hash: "your-hash".into(),
image_hash_type: ShaType::Sha256,

kernel_source: Some(ImageSource::LocalPath(
PathBuf::from("/path/to/vmlinuz")
)),
kernel_hash: Some("kernel-hash".into()),

initrd_source: Some(ImageSource::LocalPath(
PathBuf::from("/path/to/initrd.img")
)),
initrd_hash: Some("initrd-hash".into()),
};

let image = create_custom_image("local-distro", config).await?;
Ok(())
}
```

### How to Get Checksums

**For Ubuntu cloud images:**
```bash
# 1. Visit Ubuntu cloud images
# https://cloud-images.ubuntu.com/noble/current/

# 2. Download SHA256SUMS file
wget https://cloud-images.ubuntu.com/noble/current/SHA256SUMS

# 3. Find checksums for your files
grep "noble-server-cloudimg-amd64.img" SHA256SUMS
grep "vmlinuz-generic" SHA256SUMS
grep "initrd-generic" SHA256SUMS
```

**For other distributions:**
- **Fedora**: Check the CHECKSUM file in the release directory
- **Arch**: Look for `.SHA256` files alongside the image
- **Custom images**: Compute using `sha256sum your-file.qcow2` or `sha512sum your-file.qcow2`

### Security

**All custom images require checksum verification.** This ensures:

- ✅ Protection against corrupted downloads
- ✅ Protection against man-in-the-middle attacks
- ✅ Verification of file integrity

If the checksum doesn't match, image creation will fail with an error.

### Common Errors

**Error: "guestfish not available"**

This error occurs when using auto-extraction mode (Mode 2) on WSL or without guestfish installed.

**Solution:**
- Use Mode 1 (pre-extracted boot files) for WSL compatibility, or
- Install libguestfs-tools on native Linux: `sudo apt install libguestfs-tools`

**Error: "hash mismatch"**

This indicates the file doesn't match the expected checksum.

**Solution:**
- Verify you copied the correct hash from the official source
- Re-download the file (might be corrupted)
- Check you're using the correct hash type (SHA256 vs SHA512)

## Network Configuration

Qlean uses a dedicated libvirt virtual network to provide isolated, reproducible networking for test VMs. The default network definition is stored at `~/.local/share/qlean/network.xml` as follows:
Expand All @@ -169,9 +412,54 @@ This configuration defines a **NAT-based** virtual network named `qlean` (used i

### Top-Level Interface

- `create_image(distro, name)` - Create or retrieve a VM image from the specified distribution
- `with_machine(image, config, f)` - Execute an async closure in a virtual machine with automatic resource cleanup
- `with_pool(f)` - Execute an async closure in a machine pool with automatic resource cleanup
**create_image(distro, name)** - Create or retrieve a VM image from the specified distribution

Supported distributions:
- `Distro::Debian` - Debian 13 (Trixie)
- `Distro::Ubuntu` - Ubuntu 24.04 LTS (Noble) - **WSL friendly**
- `Distro::Fedora` - Fedora 41
- `Distro::Arch` - Arch Linux (latest)
```rust
pub async fn create_image(distro: Distro, name: &str) -> Result
```

**create_custom_image(name, config)** - Create a custom image with flexible configuration
```rust
pub async fn create_custom_image(
name: &str,
config: CustomImageConfig
) -> Result
```

Configuration types:
```rust
pub struct CustomImageConfig {
// Image file (required)
pub image_source: ImageSource,
pub image_hash: String,
pub image_hash_type: ShaType,

// Optional: pre-extracted kernel and initrd (WSL-friendly)
pub kernel_source: Option,
pub kernel_hash: Option,
pub initrd_source: Option,
pub initrd_hash: Option,
}

pub enum ImageSource {
Url(String), // Download from URL
LocalPath(PathBuf), // Use local file
}

pub enum ShaType {
Sha256, // SHA-256 checksum
Sha512, // SHA-512 checksum
}
```

**with_machine(image, config, f)** - Execute an async closure in a virtual machine with automatic resource cleanup

**with_pool(f)** - Execute an async closure in a machine pool with automatic resource cleanup
- `MachineConfig` - Configuration for virtual machine resources (CPU, memory, disk)

```rust
Expand Down