Nothing about resource (net, io, disk, cpu) isolation, limits, priorities, or guarantees. Not the same as a type 1 hypervisor. These qualities are needed to run things safely and predictably in the real world™, at scale. Also, accounting and multitenancy if it's going to be used as some sort VAR or VPS offering.
Fun fact, Incus is being used as underlying infrastructure for the NorthSec CTF, i.e. in an "as hostile as it can get" environment. If you have close to a hundred teams of hackers on your systems trying to break stuff, I think it speaks for Incus and its capabilities regarding isolation and limits.
In case you are interested, Zabbly has some interesting behind-the-scenes on Youtube (not affiliated).
Semaphor · 4h ago
So it looks like a Proxmox alternative, this [0] goes into some reasons to switch. Main selling point seems to be fully OSS and no enterprise version.
Proxmox feels like a more apt comparison, as they both act like a controlplane for KVM virtual-machines and LXC containers across one or multiple hosts.
If you are interested in running kubernetes on top of incus, that is your kubernetes cluster nodes will be made up of KVM or LXC instances - I highly recommend the cluster-api provider incus https://github.com/lxc/cluster-api-provider-incus
This provider is really well done and maintained, including ClusterClass support and array of pre-built machine images for both KVM and LXC. It also supports pivoting the mgmt cluster on to a workload cluster, enabling the mgmt cluster to upgrade itself which is really cool.
Not really, Kubernetes does a lot of different things that are out of scope for incus or lxd or docker compose for that matter or any hypervisor or …
hardwaresofton · 1h ago
like what? I'd love to hear some examples of things Kubernetes does that incus doesn't at this point
actinium226 · 1h ago
I went through the online tutorial, but I'm not really seeing how it's different from docker?
skydhash · 8m ago
Instead of ephemeral containers, you have instances that are like VM (and incus can manage VM via qemu), so pretty much everything you would use a VM for, but if you do not need the kernel separation. It's more similar to FreeBSD jails than to docker.
63stack · 4h ago
How do you handle updating the machine that Incus itself runs on? I imagine you have to be super careful not to introduce any breakage, because then all the VMs/containers go down.
What about kernel updates that require reboots? I have heard of ksplice/kexec, but I have never seen them used anywhere.
dsr_ · 3h ago
As with any such system, you need a spare box. Upgrade the spare, move the clients to it, upgrade the original.
loloquwowndueo · 2h ago
But then the clients have downtime while they’re being moved.
pezezin · 46m ago
I don't know about Incus, but on ProxMox the downtime when moving a VM is around 200 ms.
pylotlight · 1h ago
Isn't that the exact problem that k8s workloads solve by scaling onto new nodes first etc?
No downtime required.
loloquwowndueo · 1h ago
Right but incus is not k8s. You can stand up spares and switch traffic, but it’s not built in functionality and requires extra orchestration.
manosyja · 6h ago
What can this work with? It says „Containers and VMs“ -
I guess that’s LXCs and QEMU VMs?
nrabulinski · 6h ago
Yes, it uses QEMU under the hood for VMs and runs LXC containers. But also, since recently, you can run docker images in it. Very handy, especially since it has 1st class remote support, meaning you can install only the incus client and when doing `incus launch` or whatever, it will transparently start the container/vm on your remote host
oulipo · 10m ago
Is there some kind of Terraform/Pulumi integration to make it easy to deploy stuff to some VM running Incus for my deployments? Or I'm missing the point of what Incus is for?
Lightkey · 3h ago
Not to be confused with the cirrus7 incus[0], which are fanless PC models based on the ASRock DeskMini series that I'm using right now.
https://lwn.net/Articles/940684/
In case you are interested, Zabbly has some interesting behind-the-scenes on Youtube (not affiliated).
[0]: https://tadeubento.com/2024/replace-proxmox-with-incus-lxd/
If you are interested in running kubernetes on top of incus, that is your kubernetes cluster nodes will be made up of KVM or LXC instances - I highly recommend the cluster-api provider incus https://github.com/lxc/cluster-api-provider-incus
This provider is really well done and maintained, including ClusterClass support and array of pre-built machine images for both KVM and LXC. It also supports pivoting the mgmt cluster on to a workload cluster, enabling the mgmt cluster to upgrade itself which is really cool.
I was surprised to come across this provider by chance as for some reason it's not listed on the CAPI documentation provider list https://cluster-api.sigs.k8s.io/reference/providers
What about kernel updates that require reboots? I have heard of ksplice/kexec, but I have never seen them used anywhere.
[0] https://www.cirrus7.com/produkte/cirrus7-incus/