k3s fails to start when running inside a container.

[adk23333]

Checking the logs, it seems to be caused by unsupported cgroup controllers inside the container. Is there any way to enable support similar to nested virtualization in LXC containers?

[ravindu644]

. Is there any way to enable support similar to nested virtualization in LXC containers?

Well, Droidspaces does support nested containers; I even run Docker inside Droidspaces all day.

People run LXC inside Droidspaces, even on Kernel 4.9. There is no such thing as "Droidspaces doing cgroups wrong." Since Droidspaces forces the use of cgroupv2, a lot of the manual guesswork is gone. What Droidspaces containers have in /sys/fs/cgroups is simply what your kernel already supports.

If k3s failed with an error stating a cgroup controller is unavailable, it is because your kernel does not support it. That is all.

Since you created this issue under features (this is not a feature, but seems like a bug or a user error), please create a bug report instead. Closing this now, as this is the irrelevant category to post this issue.

If you want me to debug this and try your exact steps on my devices to trigger the "bug," create a proper, detailed bug report.

Thanks.

[ravindu644]

Anyways, I added a new feature called "Privileged mode" in the app. Go check it out. Use the full mode. If it didn't fix your issue, probably this is something with your kernel itself: https://github.com/ravindu644/Droidspaces-OSS/actions/runs/24399142255/artifacts/6427824817

[ravindu644]

I checked running K3S on my GKI device. To run it:

1. You have to use the forced cgroup v1 feature in Droidspaces. This is because K3S detects that your host has a hybrid cgroup setup (v1 and v2 combined), which is common in Android.
2. After that, enable the proper cgroup controllers in your kernel that K3S is complaining about. In my case, it complained about missing CPU and PID controllers.
3. K3S should work after you have satisfied its requirements.

NOTE: Make sure to comment out the hard-coded modprobe pre-execs from the K3S systemd service.

[adk23333]

I checked running K3S on my GKI device. To run it:

Thank you for your help. I’ve been tied up with work these past few days, and I didn’t explain things clearly enough earlier. I originally planned to provide more logs in a couple of days, and I’ve also noticed these missing details myself.

[adk23333]

NOTE: Make sure to comment out the hard-coded modprobe pre-execs from the K3S systemd service.

But I found that modprobe is part of the kmod package. Simply running apt install kmod can fix the issue, so there’s no need to comment out the service file.

[ravindu644]

NOTE: Make sure to comment out the hard-coded modprobe pre-execs from the K3S systemd service.

But I found that modprobe is part of the kmod package. Simply running apt install kmod can fix the issue, so there’s no need to comment out the service file.

aha true. but, from the container's perspective, modprobe cannot find any modules, right ? so, it will slap you a warn every time :)

[ravindu644]

I checked running K3S on my GKI device. To run it:

Thank you for your help. I’ve been tied up with work these past few days, and I didn’t explain things clearly enough earlier. I originally planned to provide more logs in a couple of days, and I’ve also noticed these missing details myself.

okay. I hope it works. make sure to use the full privileged mode, then cook a compatible kernel, then use cgroupv1, then try. probably should work..

[adk23333]

okay. I hope it works. make sure to use the full privileged mode, then cook a compatible kernel, then use cgroupv1, then try. probably should work..

Yes, it's working now. However, k3s check-config shows that I'm missing the cpuset and memory controllers. How can I enable them inside the Droidspaces container, or is it even possible?

My kernel already supports all controllers, as detected in Termux.

[ravindu644]

okay. I hope it works. make sure to use the full privileged mode, then cook a compatible kernel, then use cgroupv1, then try. probably should work..

Yes, it's working now. However, k3s check-config shows that I'm missing the cpuset and memory controllers. How can I enable them inside the Droidspaces container, or is it even possible?

My kernel already supports all controllers, as detected in Termux.

well. it's a false positive I guess.

inside the container, run cat /proc/cgroups

also, run cat /proc/mounts and send me the output

[ravindu644]

How can I enable them inside the Droidspaces container, or is it even possible?

I guess this happened when android mounted them in none-standard locations, or, not mounted them at all..

I wantmount | grep -i 'cgroup' from your android root shell as well

[adk23333]

inside the container, run cat /proc/cgroups

also, run cat /proc/mounts and send me the output

root@test:~# cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
cpuset  0       484     1
cpu     0       484     1
cpuacct 0       484     1
blkio   0       484     1
memory  0       484     1
freezer 22      1       1
net_cls 21      1       1
net_prio        21      1       1
misc    23      1       1
devices 6       18      1
pids    7       18      1
perf_event      20      1       1
root@test:~# cat /proc/mounts
/dev/block/loop17 / ext4 rw,seclabel,noatime,nodiratime,nodelalloc,errors=remount-ro,init_itable=0 0 0
devtmpfs /dev devtmpfs rw,seclabel,nosuid,noexec,relatime,size=7358064k,nr_inodes=1839516,mode=755 0 0
tmpfs /dev/block tmpfs rw,seclabel,nosuid,relatime,size=7610480k,nr_inodes=1902620,mode=755 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs ro,seclabel,relatime 0 0
sysfs /sys/wifi sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/kernel sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/power sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/class sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/devices sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/dev sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/hypervisor sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/powersave sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/hwconf sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/fs sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/bus sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/firmware sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/bootinfo sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/block sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/module sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/csiphy_umd_paras sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
none /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,size=4096k,nr_inodes=1024,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,name=systemd 0 0
tmpfs /sys/devices/virtual/tty/console/active tmpfs rw,seclabel,nosuid,relatime,size=7610480k,nr_inodes=1902620,mode=755 0 0
tmpfs /run tmpfs rw,seclabel,nosuid,nodev,relatime,mode=755 0 0
tmpfs /tmp tmpfs rw,seclabel,nosuid,nodev,relatime 0 0
devpts /dev/console devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/tty1 devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/tty2 devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/tty3 devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/tty4 devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/tty5 devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/tty6 devpts rw,seclabel,relatime,mode=600,ptmxmode=000 0 0
devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 0 0
devpts /dev/ptmx devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 0 0
tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,seclabel,nosuid,nodev,noexec,relatime,size=5120k 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/misc cgroup rw,nosuid,nodev,noexec,relatime,misc 0 0
debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0
configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0