Skip to content

Conversation

@bboozzoo
Copy link
Contributor

@bboozzoo bboozzoo commented Mar 27, 2024

Like LXD but with a twist, as instead of containers, it's using VMs. This is an improvement over the qemu backend, as LXD --vm generally runs with a virtio focused configuration ensuring that we're using hardware efficiently.

Another upside is that one does not need to build a special VM image anymore, LXD handles everything through cloud-init or the agent.

Comment on lines 481 to 491
for ifacename, ifaceconf := range sjson.State.Network {
if ifacename == "lo" {
// ignore loopback interface
continue
}
for _, addr := range ifaceconf.Addresses {
if addr.Family == "inet" && addr.Address != "" {
return addr.Address, nil
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one overlaps with #178, right? If this gets merged first, no need to merge #178.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, I think I mentioned somewhere that I had exact same change :) I will rebase as needed

Add a new backend named lxd-vm, which is essentially VMs orchestrated through
LXD.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
LXD VMs, especially when running in GCP take quite long to have IP addresses
assigned to them. Use a longer timeout in this case.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
… and lxd-vm backends

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
@bboozzoo bboozzoo force-pushed the bbboozzoo/lxd-vm-backend branch from 11aef7b to 77fbb21 Compare March 28, 2024 13:46
backends:
lxd-vm:
systems:
- ubuntu-20.04
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably also test 16.04, 18.04 and 22.04 as per #182 (assuming there are VM images for all old releases as well)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16.04 and 18.04 don't work in a VM, lxd-agent does not come start, so I've limited this to 20.04 and 22.04 for now. Another issue I ran into, is that tests need to be run one by one, otherwise the host runs out of disk space.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh, and 22.04 will only work when #184 or other similar fix lands.

Prepare the test to run on multiple systems and mention why Ubuntu 16.04, 18.04,
and 22.04 are not currently working.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
@cmatsuoka
Copy link
Collaborator

I'm seeing this message on execution using the lxd-vm backend, it can also be adjusted to mention virtual machine instead of container:

2024-08-27 18:07:46 Waiting for lxd container spread-8-ubuntu-22-04 to have an address...

cmatsuoka and others added 6 commits December 10, 2024 13:07
Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
Copy link
Member

@fnordahl fnordahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your work on this,I discovered this PR after authoring #221

How can we work together to bring this to life?

}
}

func LXDVM(p *Project, b *Backend, o *Options) Provider {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In light of Spread already having support for declaring multiple named backends, choosing the provider with the type keyword, what benefit does adding a new provider type give?

args = append(args, "-c", fmt.Sprintf("limits.memory=%dMiB", mem))
}
if system.Storage != Size(0) {
args = append(args, "-d", fmt.Sprintf("root,size=%d", system.Storage))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Containers could also benefit from support for the storage keyword.

if p.vm {
// VM may take considerably longer to start
// TODO: should this be configurable?
maxTimeout = 180 * time.Second
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not see this in my testing but it is probably true, are you using emulated VMs?

After the VM has an address, subsequent calls to lxc exec will fail until the lxd-agent inside the VM has started, ab8232d could be useful for this.

}
if system.Storage != Size(0) {
args = append(args, "-d", fmt.Sprintf("root,size=%d", system.Storage))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no way to specify the number of vCPUs for the VM. This is an important requirement as the default of 1 vCPU is most likely not sufficient.

In 804d1ee generic support for the LXD instance type concept is introduced driven by the already existing plan keyword.

LXD applies these constraints both to containers and VMs.

@yanksyoon
Copy link
Member

Hello! Out of curiosity, any updates on this/how we might progress on it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants