Add iso2lxc: convert TurnKey ISO to Proxmox LXC template

[marcos-mendez]

Summary

Add iso2lxc utility to /usr/local/sbin/ for converting TurnKey product ISOs into Proxmox-compatible LXC container templates.

Usage

cd /turnkey/fab/products/moodle
make
iso2lxc
# Output: build/debian-13-turnkey-moodle_19.0-1_amd64.tar.gz

What it does

• Extracts squashfs from product ISO
• Removes device nodes, kernel modules, and mount points that break unprivileged LXC containers
• Creates minimal /dev structure with symlinks (no mknod)
• Masks kernel-dependent systemd services
• Packages as .tar.gz following TKL naming convention
• Auto-detects product name and version from changelog

Tested

Built and deployed Moodle v19 as unprivileged LXC on Proxmox 6.8.

[JedMeister]

Thanks for your mate. But I'm guessing that you missed https://github.com/turnkeylinux/buildtasks? Explicitly bt-container . It's probably poorly named and would be better named bt-lxc or perhaps even bt-pve-lxc - but that's the script that we use to build the existing PVE templates.

FYI when we publish all apps, we use the top level bt-* tools in that buildtasks repo to do the release builds. They don't have the flexibility that your script provides as they're intended to be completely non-interactive for app batch builds, but it does the job.

The only caveat is that it expects an existing iso with our iso naming convention. E.g. turnkey-core-19.0-trixie-amd64.iso - although by default if it doesn't find one, it should build one. I.e. clones the build code (or pulls the latest commits if it exists already), builds to product.iso, renames it, generates the iso release files and then builds the proxmox template.

TBH that repo has accumulated a lot of cruft over the years and ideally I'd like to consolidate, clean and update it all; then package it, but it works...

I'm totally open to you building on those to make them more flexible for your needs, but I'd rather keep all the build stuff together.

[marcos-mendez]

Thanks @JedMeister - and no worries at all, I appreciate you taking the time to point that out rather than just closing it.

I did know about bt-container but I should have been clearer about the use case I was solving. The two tools actually target different stages of the workflow:

bt-container is a release pipeline tool - it downloads a published ISO, verifies it, applies the headless+container patches, and optionally publishes to the Proxmox repo. It's designed for batch release builds with full BT_* infrastructure.

iso2lxc is a dev loop tool - it converts whatever make just produced inside tkldev into a testable Proxmox LXC template, with zero external dependencies. The itch I was scratching: I'm actively iterating on apps (wordpress, odoo, dns-recursor, etc.) inside a tkldev VM and I want to spin up a quick unprivileged LXC to validate the result before anything gets published upstream.

That said, you're right that long-term this belongs in buildtasks rather than scattered across tkldev. I'm thinking of a two-step improvement:

1. Extend bt-container with a --local-iso flag (or a sister script bt-container-local) that skips the download/verify steps and works directly from a path to a locally-built ISO - making it useful for the dev loop without duplicating the container-building logic.

2. Optionally, a --deploy flag that creates the LXC container directly on the local Proxmox/tkldev host after building the template, with bridge networking to the tkldev interface - so the full cycle becomes: make && bt-container --local-iso build/product.iso --deploy.

Happy to close this PR and take a stab at that as a buildtasks contribution instead. Would that direction make sense to you?

[JedMeister]

iso2lxc is a dev loop tool - it converts whatever make just produced inside tkldev into a testable Proxmox LXC template, with zero external dependencies. The itch I was scratching: I'm actively iterating on apps (wordpress, odoo, dns-recursor, etc.) inside a tkldev VM and I want to spin up a quick unprivileged LXC to validate the result before anything gets published upstream.

Ah ok, thanks for the explanation. That totally makes sense and I'm more than happy with that. Reducing dev friction for everyone is an awesome thing!

FWIW we achieve a somewhat similar ends internally using tkldev-docker. It was originally developed by Anton (one of our contract devs) because he's a Mac user and wanted to do local builds and smoke testing of builds without needing a full VM (or LXC). As hinted in the readme it's a bit of a hacky/dirty way of doing things but in my experience it works well. tkldev-docker also supports using a rootfs as a source (rather than requiring an iso) which cuts a little bit of time if you don't actually need/want an iso (yet).

bt-iso uses it too - as part of the default publishing process teamed (basic smoke testing of inithook preseeds and webapp login) with clicksnap (installed/setup in buildtasks/bin/clicksnap-setup). It leverages the support for using a rootfs as a source to test the non-interactive inithook preseeds. It builds the iso first (to ensure it's "purity") then uses build/root.sandbox as the source of the test container - so we can add the inithook preseeds plus do any other desired tweaks.

Despite the name, we all use podman rather than docker (as does bt-iso). That's mostly because it "just works" without any config tweaking to get it working in a sane way. Plus it's also a tiny install compared to docker.

Anyway, I'm rambling a bit...

That said, you're right that long-term this belongs in buildtasks rather than scattered across tkldev. I'm thinking of a two-step improvement:

1. Extend `bt-container` with a `--local-iso` flag (or a sister script `bt-container-local`) that skips the download/verify steps and works directly from a path to a locally-built ISO - making it useful for the dev loop without duplicating the container-building logic.

2. Optionally, a `--deploy` flag that creates the LXC container directly on the local Proxmox/tkldev host after building the template, with bridge networking to the tkldev interface - so the full cycle becomes: `make && bt-container --local-iso build/product.iso --deploy`.

That sounds good man! We should do something similar with that tkldev-docker script. It duplicates quite a bit of bt-docker and it would make sense to merge them.

Happy to close this PR and take a stab at that as a buildtasks contribution instead. Would that direction make sense to you?

Buldtasks is probably a better place for it - or it's own repo like our tkldev-docker (I'd be happy to pull a new repo into the turnkeylinux GH org and do a shallow clone in the tkldev/conf.d/main). OTOH now I understand your use case better I'm not totally against just including it directly in TKLDev by default - at least for now.

It's definitely not a priority right now, but I'd really like to move away from including "helper scripts" in app overlays as much as possible. It makes providing users with updates & bugfixes a bit of a PITA. Ideally I've like to (deb) package them but having them in git repos is a decent middle ground - still less mucking around to update.

TL;DR I'm open to merging this as-is but putting it in a separate repo (buildtasks or somewhere else tkldev related or it's own repo) is probably better... Let me know which way you'd like to go and we can either close this or I'll merge it.

[marcos-mendez]

Thanks @JedMeister — great context on tkldev-docker and clicksnap, I didn't know about the rootfs-as-source flow. And good to hear you're all on podman — ha, if I'd known you were all on podman I might have explored that path first! But in hindsight I think the LXC approach ended up being the right call for the reasons below.

Why LXC is the right test target for TKL

My reasoning for building iso2lxc instead of going the container (Docker/Podman) route comes down to what we're actually testing.

TKL appliances are full system images — they run systemd, real networking, real services, inithooks on firstboot, confconsole on login. An OCI container is a process isolation tool: no systemd, no real init, no actual network stack, no service management. When you smoke-test an appliance in Docker/Podman, you're testing in an environment that's fundamentally different from where it will actually run.

I've seen this pattern a lot in production: docker-compose "works" on the dev's laptop, but the person deploying it has no idea what the declared infrastructure actually means — what ports are exposed, what volumes persist, what depends on what. The abstraction hides too much. For a project like TKL whose whole point is empowering people to understand and own their infrastructure, I think the test environment should match the deployment environment as closely as possible.

An unprivileged LXC on the tkldev host is the deployment target. It runs the same init, the same networking, the same service lifecycle. If inithooks work there, they'll work on the user's Proxmox node. If they work in Docker, you still don't know.

That said — tkldev-docker for a quick smoke test of "does Apache start, does the login page render" makes total sense. I'm not arguing to remove it. I'm saying that for validating an appliance end-to-end, LXC is the right unit.

Some context on how we got here

For years our dev workflow at PopSolutions involved downloading published TKL templates from the Proxmox mirror, converting them to LXD with a [metadata.yaml + import dance](https://github.com/popsolutions/proxmox-2-lxd), testing locally, then exporting back to Proxmox for production. It worked, but it was a lot of friction and an extra moving part (LXD) that didn't need to be there.

iso2lxc is the natural simplification: build in tkldev, convert to LXC template, test on the same host — no LXD, no image downloads, no format conversions. We don't even run LXD anymore.

Where should iso2lxc live?

I think merging it into tkldev for now is the pragmatic choice. It's a single script, zero dependencies beyond what tkldev already has, and it directly serves the dev loop. If/when we extend bt-container with --local-iso and --deploy, iso2lxc can retire gracefully — but that's a larger piece of work that shouldn't block having a working dev tool today.

I'm also happy to contribute the bt-container --local-iso extension to buildtasks separately, and we can consolidate later. Whatever you prefer — let me know and I'll either close this or we merge.