From 592ebca735716d348f7b0d5f2c627606f8c11504 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Wed, 4 Mar 2026 14:56:44 +0100 Subject: [PATCH 01/11] chore: Update main page instructions --- .../magic/en/getting-started/introduction.md | 50 ++++++++++++------- src/pages/magic/[...slug].astro | 2 +- 2 files changed, 33 insertions(+), 19 deletions(-) diff --git a/src/content/magic/en/getting-started/introduction.md b/src/content/magic/en/getting-started/introduction.md index 389e039..df3d45b 100644 --- a/src/content/magic/en/getting-started/introduction.md +++ b/src/content/magic/en/getting-started/introduction.md @@ -5,11 +5,11 @@ description: "An introduction to what Magic is, how to use it, the system and ap **This project is still experimental and in early stages. Feel free to test it out, but expect major changes, bugs and of course also new features.** -Liphium Magic is a suite of tools to help build tests and provide a better developer experience when developing web services in Golang. We made it because we felt like it the barrier of making a contribution to our own projects was too high. When working on applications in a team, it's important that everyone can easily start the project and also use the same tools everyone else is using. Like with dependencies, when you need a new database for something, you shouldn't have to tell everyone in your team to complete extra steps just for their app to run again. When someone first joins your project, they can ideally set everything up with one command. That's the vision of Magic, our all-in-one developer experience toolkit. For testing your app, both automatically and manually, as well as running it on your own machine without complex setup. +Liphium Magic is a suite of tools for **Golang developers** to help build tests and provide a better developer experience, especially for complex web services with databases and multiple other dependent services. It helps you and your team easily jump from project to project without learning anything new. -The path to this goal is of course a long one, and we also know that, so for now Magic can only really help with PostgreSQL databases. It only supports this one simple database and not more. It's all we use in our apps and in the future, when also have the need for it, we will likely integrate more services into Magic or build a nice abstraction layer that makes it easy to integrate different services. +We made it because we felt like it the barrier of making a contribution to our own projects was too high. When working on applications in a team, it's important that everyone can easily start the project and also use the same tools. When someone first joins your project, they should be able to get the app running within seconds instead of reading your deployment instructions. -We hope you'll enjoy this project, feel free to open an issue in case anything annoys you or you see potential for a new feature. Be nice and maybe we'll add it. We're working on this in our freetime though, so don't expect us to answer all. Before creating a pull request for something, please consult us in the issues. We're looking forward to having a discussion with you. +That's the vision of Magic, our **all-in-one developer experience toolkit**. For testing your app, both automatically and manually (with scripts), as well as making your app runnable on your own (or any other) machine without complex setup. ## System requirements @@ -23,7 +23,11 @@ Magic only supports specific services, and while we do plan on increasing the am ### Supported databases -- PostgreSQL +- PostgreSQL v18 or above + +### Deprecated + +- PostgreSQL v14-17 Other services may be supported in the future. @@ -35,7 +39,9 @@ Other services may be supported in the future. - Test your application using integration tests (they can also call your scripts) - Test with a real database using a real connection -## Usage +## Add Magic to your project + +**Note:** This is just the quick version of this guide, you'll find a much more detailed version [on this page](/magic/documentation/integrating-magic). **1.** Add Magic to your project: @@ -52,17 +58,31 @@ func main() { magic.Start(magic.Config{ AppName: "magic-example", PlanDeployment: func(ctx *mconfig.Context) { - // Create a PostgreSQL database for the posts service - postsDB := ctx.NewPostgresDatabase("posts") + // Create a new driver for PostgreSQL databases + driver := postgres.NewDriver("postgres:18"). + // Create a PostgreSQL database for the posts service (the driver supports a builder pattern with this method) + NewDatabase("posts") + + // Make sure to register the driver in the context + ctx.Register(driver) + + // Allocate a new port for the service. This makes it possible to run multiple instances of this app + // locally, without weird configuration hell. Magic will pick a port in case the preferred one is taken. + port := ctx.ValuePort(8080) // Set up environment variables for the application ctx.WithEnvironment(mconfig.Environment{ // Database connection environment variables - "DB_HOST": postsDB.Host(ctx), - "DB_PORT": postsDB.Port(ctx), - "DB_USER": postsDB.Username(), - "DB_PASSWORD": postsDB.Password(), - "DB_DATABASE": postsDB.DatabaseName(ctx), + "DB_HOST": driver.Host(ctx), + "DB_PORT": driver.Port(ctx), + "DB_USER": driver.Username(), + "DB_PASSWORD": driver.Password(), + "DB_DATABASE": mconfig.ValueStatic("posts"), + + // Make the server listen on localhost using the port allocated by Magic + "LISTEN": mconfig.ValueWithBase([]mconfig.EnvironmentValue{port}, func(s []string) string { + return fmt.Sprintf("127.0.0.1:%s", s[0]) + }), }) }, StartFunction: Start, @@ -79,9 +99,3 @@ func Start() { **3.** You can now use `go run .` to run your app and a database will be created in a Docker container near you. Become a great wizard! If you want to be a real great one though, I would take a look at the [real project example](https://github.com/Liphium/magic/tree/main/examples/real-project) to actually see how it's done. - -## Documentation - -Because this project is new, this documentation is still just the README from the actual repository. However, you can look into the `examples` folder. It should contain about everything you need for now. - -This documentation will be expanded in the future. diff --git a/src/pages/magic/[...slug].astro b/src/pages/magic/[...slug].astro index c39a75a..c997232 100644 --- a/src/pages/magic/[...slug].astro +++ b/src/pages/magic/[...slug].astro @@ -46,7 +46,7 @@ const sections = sidebarData.default; summary={page.data.description} language={lang} linkPrefix={lang != "en" ? `/${lang}/magic` : "/magic"} - software="Magic v2.0.4" + software="Magic v3.0.0" sections={sections} >
From 099c2a1ed8793a99e726c36757b1c044d60c82a5 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Wed, 4 Mar 2026 15:12:08 +0100 Subject: [PATCH 02/11] feat: Start database documentation --- src/content/magic/en/databases/postgresql.md | 11 +++++++++++ .../magic/en/documentation/integrating-magic.md | 10 ++++++++-- src/content/magic/en/getting-started/introduction.md | 2 +- src/content/magic/en/sidebar.json | 9 +++++++++ 4 files changed, 29 insertions(+), 3 deletions(-) create mode 100644 src/content/magic/en/databases/postgresql.md diff --git a/src/content/magic/en/databases/postgresql.md b/src/content/magic/en/databases/postgresql.md new file mode 100644 index 0000000..230dd0e --- /dev/null +++ b/src/content/magic/en/databases/postgresql.md @@ -0,0 +1,11 @@ +--- +title: "Using a PostgreSQL Database" +description: "A guide on how to set up Magic so it automatically starts a PostgreSQL container on startup and how to create databases within that container." +--- + +Using a [PostgreSQL](https://www.postgresql.org/) with Magic is really simple, but there are still a few things to keep in mind. There are currently two PostgreSQL drivers available that each support different versions due to changes that happened in PostgreSQL 18: + +- **Postgres**: The normal driver that currently supports PostgreSQL 18. +- **Postgres Legacy (deprecated)**: Supports PostgreSQL 14-17. It will still be kept around until PostgreSQL 20 comes out. + +This documentation focuses on the newer driver as that one will be actively maintained by us while the old one only receives bug fixes. However, all of the things shown here should work similarly for the legacy driver, in case you need it. diff --git a/src/content/magic/en/documentation/integrating-magic.md b/src/content/magic/en/documentation/integrating-magic.md index 5047ba5..0186a26 100644 --- a/src/content/magic/en/documentation/integrating-magic.md +++ b/src/content/magic/en/documentation/integrating-magic.md @@ -26,7 +26,7 @@ Before actually getting into adding Magic to your project, make sure you have [D **1.** In your project folder, use the following command to add Magic to your `go.mod` file: ```sh -go get -u github.com/Liphium/magic/v2@latest +go get -u github.com/Liphium/magic/v3@latest ``` **2.** Create a new folder for a package (we'll go with `starter`) that will contain your current `main` function (the function that's currently the entrypoint to your program), or create a new function that will be your `main` function in there. You could do it like this: @@ -82,7 +82,7 @@ package main import ( "your-project-name/starter" - "github.com/Liphium/magic/v2" + "github.com/Liphium/magic/v3" ) func main() { @@ -152,4 +152,10 @@ To prevent you from ruining their development environment with an unfinished ver - Magic will create a `.magic` folder like normal. Tell them to clean it up after. - Magic does not [stop containers after your app has finished](/magic/getting-started/frequently-asked#why-does-magic-not-stop-containers-after-shutdown). +The way to do this, with this setup, is to just build your app without the release tag (**DO NOT DO THIS FOR PRODUCTION ENVIRONMENTS**): + +```sh +go build . # Put any other arguments here # +``` + Have fun with Magic, let's become great wizards together. diff --git a/src/content/magic/en/getting-started/introduction.md b/src/content/magic/en/getting-started/introduction.md index df3d45b..7deae0d 100644 --- a/src/content/magic/en/getting-started/introduction.md +++ b/src/content/magic/en/getting-started/introduction.md @@ -46,7 +46,7 @@ Other services may be supported in the future. **1.** Add Magic to your project: ```sh -go get -u github.com/Liphium/magic/v2@latest +go get -u github.com/Liphium/magic/v3@latest ``` **2.** Wrap your main function with `magic.Start` (please take a look at the [real project example](https://github.com/Liphium/magic/tree/main/examples/real-project) for how to really to do this, this just serves as a showcase): diff --git a/src/content/magic/en/sidebar.json b/src/content/magic/en/sidebar.json index 37a68f6..507cc40 100644 --- a/src/content/magic/en/sidebar.json +++ b/src/content/magic/en/sidebar.json @@ -25,6 +25,15 @@ } ] }, + { + "name": "Databases", + "links": [ + { + "name": "PostgreSQL", + "link": "/databases/postgresql" + } + ] + }, { "name": "Links", "links": [ From 2ac4fcc8f2d9a47b7a7e0e9e27a4bbe7c0659098 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Thu, 5 Mar 2026 15:17:52 +0100 Subject: [PATCH 03/11] feat: Integration testing guide --- .../en/documentation/configuring-magic.md | 6 ++ .../en/documentation/integrating-magic.md | 4 +- .../en/documentation/integration-tests.md | 59 +++++++++++++++++-- src/content/magic/en/sidebar.json | 4 ++ 4 files changed, 66 insertions(+), 7 deletions(-) create mode 100644 src/content/magic/en/documentation/configuring-magic.md diff --git a/src/content/magic/en/documentation/configuring-magic.md b/src/content/magic/en/documentation/configuring-magic.md new file mode 100644 index 0000000..d4c2fa8 --- /dev/null +++ b/src/content/magic/en/documentation/configuring-magic.md @@ -0,0 +1,6 @@ +--- +title: "Configure Magic to your liking" +description: "How to configure Magic and deploy different services your app may need. This will specifically teach you how to set up environment variables and more." +--- + +This part of the documentation will specifically cover everything there is to know about `mconfig.Config`, the config for Magic. If you have not integrated Magic yet, [follow the setup guide first](/magic/documentation/integrating-magic). diff --git a/src/content/magic/en/documentation/integrating-magic.md b/src/content/magic/en/documentation/integrating-magic.md index 0186a26..59b28bd 100644 --- a/src/content/magic/en/documentation/integrating-magic.md +++ b/src/content/magic/en/documentation/integrating-magic.md @@ -137,7 +137,7 @@ go run -tags release . Building actually works just like running the binary with the `release` tag. You can simply do the following: ```sh -go build -tags release . # Put any other arguments here # +go build -tags release . ``` **Note: When you build your app without the `release` tag, your app will run with Magic just like when doing `go run .`!** @@ -155,7 +155,7 @@ To prevent you from ruining their development environment with an unfinished ver The way to do this, with this setup, is to just build your app without the release tag (**DO NOT DO THIS FOR PRODUCTION ENVIRONMENTS**): ```sh -go build . # Put any other arguments here # +go build . ``` Have fun with Magic, let's become great wizards together. diff --git a/src/content/magic/en/documentation/integration-tests.md b/src/content/magic/en/documentation/integration-tests.md index 90c19f6..d6c4652 100644 --- a/src/content/magic/en/documentation/integration-tests.md +++ b/src/content/magic/en/documentation/integration-tests.md @@ -3,14 +3,63 @@ title: "Integration testing" description: "How to test with Magic: How you can create integration tests, interact with the runtime and what might be useful." --- -**This article is not complete yet, please don't follow it until this warning is gone.** - The idea of integration testing with Magic is essentially the following: -- Magic will automatically start all Docker containers and services needed for your application -- Magic will start your actual app in a separate goroutine in the same process as the testing runtime -- All of your tests are executed by the Go testing runtime (like normal, with the same environment variables set as your application) +- Magic will automatically start all Docker containers and services needed for your application. +- Magic will start your actual app in a separate goroutine in the same process as the testing runtime. +- All of your tests are executed by the Go testing runtime (like normal, with the same environment variables set as your application). ## Getting started Before you can use Magic for integration testing, please make sure you have [set up Magic properly](/magic/documentation/integrating-magic). + +Next, their is literally only one step: Magic needs to run before any of your tests, so integrate it like this: + +```go +func TestMain(m *testing.M) { + magic.PrepareTesting(m, starter.BuildMagicConfig()) +} +``` + +What you can now do: + +- All of the environment variables are shared with your app, meaning `os.Getenv` will return the same in your app as in your tests (you can use this to get the path to your API if you're creating a web service for example). +- You can call scripts you have created to simplify some of your tests (as any other function). + - If your scripts need the **Magic runner**, read below. + +## The runtime + +You may want to know more about how Magic runs your tests internally, since we of course also spin up all of your services, like when you do `go run .` normally. + +### How it works under the hood + +Magic has [profiles](/magic/getting-started/frequently-asked#can-i-run-multiple-instances-of-my-app-with-magic) that allow you to create multiple instances of your app (that can run in parallel with different database containers, etc.) and the same is done when you run your tests. For this, we use the `test` profile. So what happens before your tests run is essentially just `go run . --profile test` in a different goroutine. Your tests and your app are sharing the same process and same memory (in a way). + +**Hint:** You can use `ctx.Profile()` to check which profile is currently being used in your `PlanDeployment` function in the config. This allows you to also set different environment variables specifically for testing by simple checking `ctx.Profile() == "test"`. + +The only exception is that in this test profile all of your containers are **deleted** before your tests run, we do this to give you a predictable testing environment. **This is not done before every test, just before ALL of your tests within the same module run**. + +### What this means + +- You're essentially getting a fresh environment for every module where you use `magic.PrepareTesting`. +- Magic's test runner will make sure none of your tests write to the same containers at the same time using lock files. +- You can interact with your app's server or database however you want. +- Your app sees the same environment variables as you do (just like normal `go test`). +- Your app sees the same global state as you do (just like normal `go test`). + +## Getting the Magic runner + +For your tests, as for anything else in Magic, it's run by `mrunner.Runner`. This object provides you with a lot of useful functions and if you want to use any of its functionality in tests, you can get it like this: + +```go +magic.GetTestRunner() +``` + +### Convenient methods + +One of the things you may wanna do with it is, for example, drop all of the tables in your test database. For this reason, the runner provides you with some convenient methods: + +- `runner.ClearTables()` will clear all tables in all databases. It just clears the content of the tables, they won't be dropped. +- `runner.DropTables()` will, as the name suggests, drop all tables in all databases. + +If there is still anything you don't understand about Magic's testing runtime, let us know and we'll expand this section and, of course, hopefully explain it to you in a better way. If there is some functionality you are missing, you can always create feature requests in our repository. diff --git a/src/content/magic/en/sidebar.json b/src/content/magic/en/sidebar.json index 507cc40..4b39acc 100644 --- a/src/content/magic/en/sidebar.json +++ b/src/content/magic/en/sidebar.json @@ -19,6 +19,10 @@ "name": "Integrating Magic", "link": "/documentation/integrating-magic" }, + { + "name": "Configuring Magic", + "link": "/documentation/configuring-magic" + }, { "name": "Integration tests", "link": "/documentation/integration-tests" From 10414c1f29731225df269aa58d94635d2772fb49 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Thu, 5 Mar 2026 15:25:44 +0100 Subject: [PATCH 04/11] fix: Better explanation for test setup --- src/content/magic/en/documentation/integration-tests.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/content/magic/en/documentation/integration-tests.md b/src/content/magic/en/documentation/integration-tests.md index d6c4652..87d346c 100644 --- a/src/content/magic/en/documentation/integration-tests.md +++ b/src/content/magic/en/documentation/integration-tests.md @@ -13,16 +13,21 @@ The idea of integration testing with Magic is essentially the following: Before you can use Magic for integration testing, please make sure you have [set up Magic properly](/magic/documentation/integrating-magic). -Next, their is literally only one step: Magic needs to run before any of your tests, so integrate it like this: +From here on, it assumed that you know how to test projects in Go. If not, you may wanna research how to do that first. + +Next, there is literally only one step: Go to any of your `[something]_test.go` files and integrate Magic like this: ```go func TestMain(m *testing.M) { - magic.PrepareTesting(m, starter.BuildMagicConfig()) + magic.PrepareTesting(m, /* the build function for your Magic config */) } ``` +If you want some inspiration and see real tests written with Magic, you can take a look at the ones in our [real project example](https://github.com/Liphium/magic/blob/main/examples/real-project/starter/start_test.go). + What you can now do: +- Create tests like normal, your app is running in the same process in a different goroutine. - All of the environment variables are shared with your app, meaning `os.Getenv` will return the same in your app as in your tests (you can use this to get the path to your API if you're creating a web service for example). - You can call scripts you have created to simplify some of your tests (as any other function). - If your scripts need the **Magic runner**, read below. From 195c8f4d81693c88b498eefbe097be64d44ee670 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Fri, 6 Mar 2026 15:54:04 +0100 Subject: [PATCH 05/11] feat: Finish documentation section --- src/components/PanelWrapper.astro | 27 +++- .../en/documentation/configuring-magic.md | 118 +++++++++++++++++- .../en/documentation/integration-tests.md | 23 +++- .../magic/en/documentation/magic-scripts.md | 92 ++++++++++++++ .../magic/en/getting-started/concepts.md | 22 ++++ src/content/magic/en/sidebar.json | 8 ++ 6 files changed, 284 insertions(+), 6 deletions(-) create mode 100644 src/content/magic/en/documentation/magic-scripts.md create mode 100644 src/content/magic/en/getting-started/concepts.md diff --git a/src/components/PanelWrapper.astro b/src/components/PanelWrapper.astro index 2da1186..719df77 100644 --- a/src/components/PanelWrapper.astro +++ b/src/components/PanelWrapper.astro @@ -17,9 +17,20 @@ const sections: { links: { name: string; link: string; + href: string; + external: boolean; }[]; }[] = sectionsUnparsed; +for (const section of sections) { + for (const link of section.links) { + const external = + link.link.startsWith("https://") || link.link.startsWith("http://"); + link.href = external ? link.link : linkPrefix + link.link; + link.external = external; + } +} + const currentPath = Astro.url.pathname; --- @@ -39,10 +50,10 @@ const currentPath = Astro.url.pathname;
Planet logo -

{software}

+

{software}

@@ -58,7 +69,17 @@ const currentPath = Astro.url.pathname; {section.links.map((link) => ( {link.name} diff --git a/src/content/magic/en/documentation/configuring-magic.md b/src/content/magic/en/documentation/configuring-magic.md index d4c2fa8..fb14910 100644 --- a/src/content/magic/en/documentation/configuring-magic.md +++ b/src/content/magic/en/documentation/configuring-magic.md @@ -3,4 +3,120 @@ title: "Configure Magic to your liking" description: "How to configure Magic and deploy different services your app may need. This will specifically teach you how to set up environment variables and more." --- -This part of the documentation will specifically cover everything there is to know about `mconfig.Config`, the config for Magic. If you have not integrated Magic yet, [follow the setup guide first](/magic/documentation/integrating-magic). +This part of the documentation will specifically cover everything there is to know about `magic.Config`, the config for Magic. If you have not integrated Magic yet, [follow the setup guide first](/magic/documentation/integrating-magic). + +## Basic config values + +First, let's go over the simplest parts of the config. + +- **AppName**: This is the name Magic is going to prefix all of the Docker containers with. Make sure it's unique by project. Collisions with other projects could cause containers to overwrite each other. +- **StartFunction**: The start function of your application. Magic needs this to start up your app after all of the containers have been started. + +## Deployment planning + +Now for the more difficult part: In `PlanDeployment` you need to specify all of the containers that your app is going to need for starting up successfully. This also includes **environment variables**. + +While you can find documentation for the different services in the sidebar, here we'll go into some basics. + +### The context + +The only argument you're getting in the `PlayDeployment` function is the `mconfig.Context`. This context includes a lot of useful information you might want to use. Below it's abbreviated as `ctx`. Here are some useful functions (more is explained in the sections below as well): + +- `ctx.Profile()` gives you the current [profile](/magic/getting-started/concepts#profiles). +- `ctx.ProjectDirectory()` is the directory of your project. + +You could for example use `ctx.ProjectDirectory()` to create a new directory for some files that you might wanna use for testing. You could check specifically for the `test` profile to make sure it's the testing runtime. + +### Environment variables + +Magic handles environment variables not as strings, but as functions. We call these functions before your app starts to "generate the environment variables". + +You specify environment variables in the `PlayDeployment` function by using `ctx.WithEnvironment()`. If you want to load environment variables from a file, you can also use `ctx.LoadSecretsToEnvironment("path/to/file")`. + +Additionally, a lot of [service drivers](/magic/getting-started/concepts#service-drivers) have functions that let you retrieve some values that you can pass directly into `ctx.WithEnvironment()`. + +Because looking at code is probably easier, below is a little cheatsheet: + +```go +func PlanDeployment(ctx *mconfig.Context) { + port := mconfig.ValueStatic("6000") // For later + + ctx.WithEnvironment(mconfig.Environment{ + + // Create a static value + "HELLO": mconfig.ValueStatic("WORLD"), + + // Create a value from a function + "FROM_FUNCTION": mconfig.ValueFunction(func() string { + return "HELLO_FROM_FUNCTION" + }), + + // Additionally, you can create environment variables based on others (index in []mconfig.EnvironmentValue = index in []string) + "LISTEN": mconfig.ValueWithBase([]mconfig.EnvironmentValue{port}, func(args []string) string { + return fmt.Sprintf("127.0.0.1:%s", args[0]) + }) + }) +} +``` + +### Allocating ports + +Using environment values you can actually also allocate ports for your app to use. This is how Magic can run multiple instances of your app without them colliding on the same port: + +```go +func PlanDeployment(ctx *mconfig.Context) { + port := mconfig.ValuePort(8080) // 8080 will be used when the port is open + + ctx.WithEnvironment(mconfig.Environment{ + + // You can just set the port + "PORT": port, + + // You can use your port to generate other values + "LISTEN": mconfig.ValueWithBase([]mconfig.EnvironmentValue{port}, func(args []string) string { + return fmt.Sprintf("127.0.0.1:%s", args[0]) + }) + }) +} +``` + +### Adding services / databases + +Now, for the most interesting bit, you might wanna actually start services for your app to use: This is the main reason we made Magic. + +It's actually also pretty simple, since all you have to do is use `ctx.Register(driver)` with a [service driver](/magic/getting-started/concepts#service-drivers) like this: + +```go +func PlanDeployment(ctx *mconfig.Context) { + // Example with the PostgreSQL driver + driver := postgres.NewDriver("postgres:18").NewDatabase("posts") + + // Make sure to register the driver in the context + ctx.Register(driver) +} +``` + +**Hint:** `ctx.Register(driver)` actually gives you the driver right back, meaning you can also use it like this: + +```go +func PlanDeployment(ctx *mconfig.Context) { + // Example with the PostgreSQL driver + driver := ctx.Register(postgres.NewDriver("postgres:18").NewDatabase("posts")) +} +``` + +## Scripts + +While scripts have [their own documentation](/magic/documentation/magic-scripts) where you can more about how to actually write them, here is how you actually register them in your config. + +The `Scripts` value in your config accepts a `[]scripting.Script` which you can fill by using `scripting.CreateScript(...)`. Here is a more concrete code example: + +```go +scripting.CreateScript("your-script", "Your description.", YourFunction) +``` + +I feel like if you're using Magic you should know how to fill an array, so we'll skip that part. + +## Example config + +If you now want to look at an example config, you can find one that has basically everything you're ever going to have in the [real project example](https://github.com/Liphium/magic/blob/main/examples/real-project/starter/config.go). diff --git a/src/content/magic/en/documentation/integration-tests.md b/src/content/magic/en/documentation/integration-tests.md index 87d346c..b34ae7c 100644 --- a/src/content/magic/en/documentation/integration-tests.md +++ b/src/content/magic/en/documentation/integration-tests.md @@ -13,9 +13,28 @@ The idea of integration testing with Magic is essentially the following: Before you can use Magic for integration testing, please make sure you have [set up Magic properly](/magic/documentation/integrating-magic). +### Adding the startup hook + +After that, there is actually one more thing that's required to get your app and running with Magic. Magic has to wait for your app to start up before any of the integration tests can begin. So what you need to do is call `magic.AppStarted()` once after your app has been started. If you're using the web framework Fiber, your code for this could look like this: + +```go +// Add a startup hook to notify Magic of the app start +app.Hooks().OnListen(func(listenData fiber.ListenData) error { + if fiber.IsChild() { + return nil + } + + // Tell Magic the app has started: Makes sure tests start to run after this. + magic.AppStarted() + return nil +}) +``` + +### Setting up the testing runtime + From here on, it assumed that you know how to test projects in Go. If not, you may wanna research how to do that first. -Next, there is literally only one step: Go to any of your `[something]_test.go` files and integrate Magic like this: +There is literally only one step to integrate Magic with your tests: Go to one of your `[something]_test.go` files where you want to integrate Magic and add a `TestMain` function like this: ```go func TestMain(m *testing.M) { @@ -38,7 +57,7 @@ You may want to know more about how Magic runs your tests internally, since we o ### How it works under the hood -Magic has [profiles](/magic/getting-started/frequently-asked#can-i-run-multiple-instances-of-my-app-with-magic) that allow you to create multiple instances of your app (that can run in parallel with different database containers, etc.) and the same is done when you run your tests. For this, we use the `test` profile. So what happens before your tests run is essentially just `go run . --profile test` in a different goroutine. Your tests and your app are sharing the same process and same memory (in a way). +Magic has [profiles](/magic/getting-started/concepts#profiles) that allow you to create multiple instances of your app and we use the `test` profile for the testing runtime. So what happens before your tests run is essentially just `go run . --profile test` in a different goroutine. Your tests and your app are sharing the same process and same memory (in a way). **Hint:** You can use `ctx.Profile()` to check which profile is currently being used in your `PlanDeployment` function in the config. This allows you to also set different environment variables specifically for testing by simple checking `ctx.Profile() == "test"`. diff --git a/src/content/magic/en/documentation/magic-scripts.md b/src/content/magic/en/documentation/magic-scripts.md new file mode 100644 index 0000000..fdf91a9 --- /dev/null +++ b/src/content/magic/en/documentation/magic-scripts.md @@ -0,0 +1,92 @@ +--- +title: "Magic's scripting system" +description: "How you can add scripts to your Magic config. This allows you to share tools with contributors or call functions in your codebase through the CLI." +--- + +When working on your project, you might want to share some tools or functions you use in tests with the rest of your team. Scripts allow you to integrate those tools directly into your codebase. No more weird bash scripts needed. + +## Creating scripts + +A script is literally just any old function you may have in your codebase. There are just two requirements it needs to furfill: + +- Only have one struct (or none) and a `mrunner.Runner` (or none) as the parameters of the function. +- Have an `error` as the **last** returned value. + +Valid functions could be: + +```go +func CreateSomething(something SomeStruct) error +func SomeList() ([]string, error) +func SomeRunnerScript(runner *mrunner.Runner) error +func SomeScript([]database.Post, runner *mrunner.Runner) error +func SomeScript2(runner *mrunner.Runner, something SomeStruct) error +``` + +Lists of structs are not allowed. + +### Requirements for the struct + +Currently Magic only supports a few types for the fields in the struct, this includes: + +- Any number types (`int`, `uint`, etc.) +- Strings + +If you have a non-supported type in a struct, but you don't need it in the script, you can use the `magic:"ignore"` struct tag to make Magic skip it. + +Additionally, you can use the struct tag `validate:"some-options"` to add validations. This uses the [validator package](https://github.com/go-playground/validator), find a list of supported validations over there. + +### How your script is run + +For how to run your script, read more [below](#using-scripts). This covers how your script gets executed internally. + +Scripts are run separately from your main app process. You **DO NOT** have access to any internal state of your app in scripts. All we do is give you the same environment variables as your main app process, which should still prove plenty useful. + +Containers will also not be started, you have to start your app for that to happen. + +### Registering your script + +Now that you have a function you can register as a script, you can learn how to register it over in the [config documentation](magic/documentation/configuring-magic#scripts). + +## Using scripts + +When you want to run a script, you can use the `--run` or `-r` flag straight after `go run .` like this: + +```sh +go run . -r [script_name] +``` + +When you want to list all of the scripts available, you can just do: + +```sh +go run . --scripts +``` + +This works in any codebase with Magic allowing you to easily interact with it. + +### Skipping the form + +When you run a script, you may be prompted with a terminal form. Since that sometimes gets annoying, you can also pass the arguments in directly just after the script name. If you have this script function: + +```go +type Post struct { + Author string `prompt:"Author name" validate:"required"` + Content string `prompt:"Content of the post" validate:"required,max=256"` +} + +func CreatePost(post database.Post) error { + // ... + return nil +} +``` + +To fill a particular struct field, the index of the argument must match with it. Here is how you could call this script if it was named "create-post": + +```sh +go run . -r create-post [author] [content] +``` + +Magic will still validate all of the fields or show you an error if you don't specify one of them. + +## Script examples + +If you now want to look at example scripts, you can find some in our real project example. There are ones [using the database and runner](https://github.com/Liphium/magic/blob/main/examples/real-project/starter/scripts_database.go) as well as ones just [calling the API](https://github.com/Liphium/magic/blob/main/examples/real-project/starter/scripts_endpoints.go). diff --git a/src/content/magic/en/getting-started/concepts.md b/src/content/magic/en/getting-started/concepts.md new file mode 100644 index 0000000..8fc464f --- /dev/null +++ b/src/content/magic/en/getting-started/concepts.md @@ -0,0 +1,22 @@ +--- +title: "Magic concepts explained" +description: "A few of Magic's internal wording for systems, explained. Here you'll find what we mean by profiles and more." +--- + +Here you'll find a few common words often used in this documentation, explained in a hopefully understandable manner. Feel free to create an issue if anything is unclear. + +## Profiles + +Magic has [profiles](/magic/getting-started/frequently-asked#can-i-run-multiple-instances-of-my-app-with-magic) that allow you to create multiple instances of your app (that can run in parallel with different database containers, etc.). + +This is particularly useful when you need to test with multiple instances of your app. + +You can start your app in a profile using the `-p` or `--profile` flag after `go run .`. The default profile is named, well, `default`. So if you do `go run . -p default` it's like you didn't specify the flag at all. + +The `test` profile is reserved for the [testing runtime](/magic/documentation/integration-tests). Please don't use it for actually running your app as running tests after will delete all of the containers. + +## Service drivers + +All of the databases and other services you start using Magic are started by a **service driver**. It's what we call the thing managing the container and executing instructions on it. + +Under the hood, this is just a common interface that all of the service drivers shipping with Magic implement, so you could also create your own one. diff --git a/src/content/magic/en/sidebar.json b/src/content/magic/en/sidebar.json index 4b39acc..fe83ebc 100644 --- a/src/content/magic/en/sidebar.json +++ b/src/content/magic/en/sidebar.json @@ -9,6 +9,10 @@ { "name": "Frequently asked", "link": "/getting-started/frequently-asked" + }, + { + "name": "Concepts", + "link": "/getting-started/concepts" } ] }, @@ -23,6 +27,10 @@ "name": "Configuring Magic", "link": "/documentation/configuring-magic" }, + { + "name": "Magic scripts", + "link": "/documentation/magic-scripts" + }, { "name": "Integration tests", "link": "/documentation/integration-tests" From 87629265707dbae1b033d01a1302b5805a683f89 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Fri, 6 Mar 2026 17:44:52 +0100 Subject: [PATCH 06/11] fix: Make sure it builds again + port changes from before --- src/components/PanelWrapper.astro | 212 +++++------------------------ src/components/PanelWrapper.svelte | 49 ++++++- src/content/magic/en/sidebar.json | 4 + 3 files changed, 79 insertions(+), 186 deletions(-) diff --git a/src/components/PanelWrapper.astro b/src/components/PanelWrapper.astro index a161470..552a480 100644 --- a/src/components/PanelWrapper.astro +++ b/src/components/PanelWrapper.astro @@ -1,182 +1,36 @@ - - - - -
-
- - - - -
- {@render children()} -
-
-
- - -
{ - if (e.target === e.currentTarget) { - closeSidebar(); - } - }} - onkeydown={(e) => { - if (e.key === "Escape") { - closeSidebar(); - } - }} - class="fixed inset-0 bg-bg-800/80 backdrop-blur-sm lg:hidden transition-opacity duration-300 z-50" - class:opacity-0={!isSidebarOpen} - class:opacity-100={isSidebarOpen} - class:pointer-events-none={!isSidebarOpen} -> -
+ -
-
- -
- Planet logo -

{software}

-
- - - {#each sections as section} -
- {#if section.name} -

- {section.name} -

- {/if} - - {#each section.links as link} - - {link.name} - - {/each} -
- {/each} -
-
- - -
-
-
- - - \ No newline at end of file + + + diff --git a/src/components/PanelWrapper.svelte b/src/components/PanelWrapper.svelte index 1a7317a..91b4b82 100644 --- a/src/components/PanelWrapper.svelte +++ b/src/components/PanelWrapper.svelte @@ -8,12 +8,21 @@ link: string; } + interface ProcessedLinkItem extends LinkItem { + href: string; + external: boolean; + } + interface Section { name: string; linkPrefix: string; links: LinkItem[]; } + interface ProcessedSection extends Section { + links: ProcessedLinkItem[]; + } + interface Props { software: string; linkPrefix: string; @@ -25,6 +34,22 @@ let { software, linkPrefix, children, sections, currentPath }: Props = $props(); + const processedSections: ProcessedSection[] = $derived( + sections.map((section) => ({ + ...section, + links: section.links.map((link) => { + const external = + link.link.startsWith("https://") || + link.link.startsWith("http://"); + return { + ...link, + href: external ? link.link : linkPrefix + link.link, + external, + }; + }), + })), + ); + let isSidebarOpen = $state(false); function toggleSidebar() { @@ -63,14 +88,14 @@
Planet logo -

{software}

+

{software}

- {#each sections as section} + {#each processedSections as section}
{#if section.name}

@@ -81,7 +106,13 @@ {#each section.links as link} {link.name} @@ -139,7 +170,7 @@

- {#each sections as section} + {#each processedSections as section}
{#if section.name}

@@ -151,7 +182,11 @@ {link.name} @@ -169,7 +204,7 @@

- +
{link.name} diff --git a/src/content/magic/en/documentation/integrating-magic.md b/src/content/magic/en/documentation/integrating-magic.md index 362d280..73f6e77 100644 --- a/src/content/magic/en/documentation/integrating-magic.md +++ b/src/content/magic/en/documentation/integrating-magic.md @@ -68,7 +68,7 @@ func BuildMagicConfig() magic.Config { ``` -For filling the config with actual content, we'll soon have another guide. For now, you can look at the struct definition and the comments in the code. We've added a lot of comments and hope they help for now. +For how the config itself works and what all the values do, you can go over to [this page](/magic/documentation/configuring-magic). **5.** For your actual program entrypoint that Magic will use, let's create a new `main_magic.go` file in your root directory: From 23da24f264441fd3af22049e4768c80e22695bf8 Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Sat, 7 Mar 2026 11:39:13 +0100 Subject: [PATCH 10/11] feat: Add more nav items --- src/components/Nav.astro | 2 ++ src/components/Nav.svelte | 10 ++++++++++ src/i18n/translations.ts | 2 ++ 3 files changed, 14 insertions(+) diff --git a/src/components/Nav.astro b/src/components/Nav.astro index 8113621..1404880 100644 --- a/src/components/Nav.astro +++ b/src/components/Nav.astro @@ -8,6 +8,8 @@ const t = useTranslations(lang); const translations = { name: t("nav.name"), magic: t("nav.magic"), + main: t("nav.main"), + github: t("nav.github"), }; --- diff --git a/src/components/Nav.svelte b/src/components/Nav.svelte index 0ba29de..4b2124e 100644 --- a/src/components/Nav.svelte +++ b/src/components/Nav.svelte @@ -8,6 +8,8 @@ translations: { name: string; magic: string; + main: string; + github: string; }; } @@ -16,6 +18,10 @@ let isMenuOpen = $state(false); const links = [ + { + name: translations.main, + link: "https://liphium.com", + }, { name: translations.magic, link: getAbsoluteLocaleUrl( @@ -23,6 +29,10 @@ "/magic/getting-started/introduction", ), }, + { + name: translations.github, + link: "https://github.com/Liphium/dev", + }, ]; function toggleMenu() { diff --git a/src/i18n/translations.ts b/src/i18n/translations.ts index 43bf5b3..1a14f70 100644 --- a/src/i18n/translations.ts +++ b/src/i18n/translations.ts @@ -8,7 +8,9 @@ export const ui = { en: { // Translations for the navigation bar "nav.name": "liphium.dev", + "nav.main": "For users", "nav.magic": "Magic", + "nav.github": "GitHub", // Translations for the footer "footer.copyright": "Copyright © 2023-2026 Liphium", From f0c2b38fd9cafdf48c26cea1d493deecc0092a0e Mon Sep 17 00:00:00 2001 From: Unbreathable <70802809+Unbreathable@users.noreply.github.com> Date: Thu, 12 Mar 2026 20:18:40 +0100 Subject: [PATCH 11/11] feat(magic): CI/CD docs + FAQ cleanup --- src/content/magic/en/databases/postgresql.md | 42 ++++++++++++++++++- src/content/magic/en/documentation/ci-cd.md | 41 ++++++++++++++++++ .../magic/en/getting-started/concepts.md | 2 +- .../en/getting-started/frequently-asked.md | 18 +------- src/content/magic/en/sidebar.json | 4 ++ 5 files changed, 88 insertions(+), 19 deletions(-) create mode 100644 src/content/magic/en/documentation/ci-cd.md diff --git a/src/content/magic/en/databases/postgresql.md b/src/content/magic/en/databases/postgresql.md index 230dd0e..72c3206 100644 --- a/src/content/magic/en/databases/postgresql.md +++ b/src/content/magic/en/databases/postgresql.md @@ -3,9 +3,49 @@ title: "Using a PostgreSQL Database" description: "A guide on how to set up Magic so it automatically starts a PostgreSQL container on startup and how to create databases within that container." --- -Using a [PostgreSQL](https://www.postgresql.org/) with Magic is really simple, but there are still a few things to keep in mind. There are currently two PostgreSQL drivers available that each support different versions due to changes that happened in PostgreSQL 18: +Using [PostgreSQL](https://www.postgresql.org/) with Magic is really simple, but there are still a few things to keep in mind. There are currently two PostgreSQL drivers available that each support different versions due to changes that happened in PostgreSQL 18: - **Postgres**: The normal driver that currently supports PostgreSQL 18. - **Postgres Legacy (deprecated)**: Supports PostgreSQL 14-17. It will still be kept around until PostgreSQL 20 comes out. This documentation focuses on the newer driver as that one will be actively maintained by us while the old one only receives bug fixes. However, all of the things shown here should work similarly for the legacy driver, in case you need it. + +## Usage + +**1.** Import the `postgres` package from Magic that provides the driver using the following command: + +```sh +go get -u github.com/Liphium/magic/v3/pkg/databases/postgres@latest +``` + +**2.** You can now use the driver in your code like this: + +```go +driver := postgres.NewDriver("postgres:18") // Make sure you use a version that is supported + +// Create a new database like this: +driver.NewDatabase("test") +``` + +This driver also supports a builder pattern, meaning you can do stuff like this: + +```go +driver := postgres.NewDriver("postgres:18"). + NewDatabase("test"). + NewDatabase("test2") +``` + +For environment values this driver provides, read more below. If you want to now **register the driver** and don't know how to do that yet, [learn more here](/magic/documentation/configuring-magic). + +## Environment values + +As with every driver in Magic, the PostgreSQL driver also provides some environment variables you may want to use to connect to a database it created (`ctx` is your `mconfig.Context`): + +- `driver.Host(ctx)`: The hostname of the database (`127.0.0.1`, just use it anyway because it might change in the future). +- `driver.Port(ctx)`: The port of the database container on your local system. +- `driver.Username()`: The username for the database server (is always "postgres"). +- `driver.Password()`: The password for the database server (is always "postgres"). + +While some of the values might seem redundant as they always return the same, it's still **best practice** to use them instead of defining them yourself as the defaults might change in the future. We don't have any plans to do this kind of thing, but it _could_ happen. + +If you're wondering what the database name will be, that's just the thing you passed into `driver.NewDatabase(/* into here */)`. diff --git a/src/content/magic/en/documentation/ci-cd.md b/src/content/magic/en/documentation/ci-cd.md new file mode 100644 index 0000000..bfcb77e --- /dev/null +++ b/src/content/magic/en/documentation/ci-cd.md @@ -0,0 +1,41 @@ +--- +title: "Magic in CI/CD pipelines" +description: "How to set up a CI/CD pipeline with integration testing and everything else using Magic." +--- + +If you want to run Magic in CI/CD, the environment needs Docker, or anything Socket-compatible to Docker (like Podman), installed. + +Since explaining all of this would be stupid anyway, we provide some samples for some providers below, if you are using any other CI/CD provider and created a config for them, [feel free to contribute](https://github.com/Liphium/dev). + +Normally though, you can just use a normal pipeline for Go, as long as Docker is available in the pipeline. + +## GitHub Actions + +```yml +# This workflow tests the application and uploads the cover profile as an artifact + +name: Test (Ubuntu) + +on: + push: + pull_request: + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v4 + with: + go-version: "1.25.7" + + - name: Test + run: go test -timeout 20m -v ./... -coverpkg="./..." -coverprofile="coverage.out" + + - uses: actions/upload-artifact@v5.0.0 + with: + name: test-coverage.out + path: coverage.out +``` diff --git a/src/content/magic/en/getting-started/concepts.md b/src/content/magic/en/getting-started/concepts.md index 8fc464f..e4b65dc 100644 --- a/src/content/magic/en/getting-started/concepts.md +++ b/src/content/magic/en/getting-started/concepts.md @@ -7,7 +7,7 @@ Here you'll find a few common words often used in this documentation, explained ## Profiles -Magic has [profiles](/magic/getting-started/frequently-asked#can-i-run-multiple-instances-of-my-app-with-magic) that allow you to create multiple instances of your app (that can run in parallel with different database containers, etc.). +Magic has **profiles** that allow you to create multiple instances of your app (that can run in parallel with different database containers, etc.). This is particularly useful when you need to test with multiple instances of your app. diff --git a/src/content/magic/en/getting-started/frequently-asked.md b/src/content/magic/en/getting-started/frequently-asked.md index 14d9a07..f5f6eaf 100644 --- a/src/content/magic/en/getting-started/frequently-asked.md +++ b/src/content/magic/en/getting-started/frequently-asked.md @@ -9,15 +9,7 @@ All of your basic questions about Magic should be answered here. If they aren't, Yes, in fact, that's **one of the reasons we created Magic in the first place**. -When you run your app with Magic, just use `go run . --profile ` to create another instance. The database will also be persistent. Please just don't call it `test-default` since that's the profile we use for our test runtime (any databases may be cleared when you run tests). - -**Hint:** You can use `-p ` to set the profile as well. - -## Can I get access to the Magic runner in integration tests? - -Yes, use `magic.GetTestRunner()`. Should be as easy as that. - -But please be careful with parallel tests. +The feature you can use for it is called **profiles**, you can learn more about it [here](/magic/getting-started/concepts#profiles). ## Is there a recommended project structure? @@ -27,14 +19,6 @@ For now, you can take a look at our [real project example](https://github.com/Li Another way to learn more about our use of Magic is by literally just looking into our projects. One of the easiest to understand ones is the recently released [Hytale Matchmaking](https://github.com/Liphium/hytale-matchmaking). -## Can Magic run in CI/CD? - -**Yes**, but the environment for your CI/CD pipeline needs Docker, or anything Socket-compatible to Docker (like Podman), installed. - -We require it even when no Docker containers are started to guarantee that you'll always be able to run your tests with Magic, no matter what you add to your project and no matter what we may add in the future. - -You can take a look at how we do it on GitHub by looking at [this file](https://github.com/Liphium/hytale-matchmaking/blob/main/.github/workflows/go.yml). - ## Can Magic run tests in parallel? Unfortunately, running tests in parallel is currently **not supported**. While it is theoretically pretty easy to implement, we at Liphium haven't had any issues requiring this kind of functionality. diff --git a/src/content/magic/en/sidebar.json b/src/content/magic/en/sidebar.json index 5114a1f..e5b9ba4 100644 --- a/src/content/magic/en/sidebar.json +++ b/src/content/magic/en/sidebar.json @@ -34,6 +34,10 @@ { "name": "Integration tests", "link": "/documentation/integration-tests" + }, + { + "name": "Magic in CI/CD", + "link": "/documentation/ci-cd" } ] },