Godot threads from Lua

[bojjenclon]

Is there a proper way to use one of Godot's native threads from a Lua script? I've tried simply doing something like:

thread = Thread:new()
thread:start(self, "_save_game_in_thread", save_data)
self.thread = thread

But this just causes the game to crash with no error reported.

I know one of the stated non-goals is "Support multithreading on the Lua side" but I'm not sure if that's what you meant or if you simply meant native Lua multithreading.

[gilzoide]

The problem with multithreading and Lua is that the VMs, at least standard PUC-Rio Lua and LuaJIT, are not thread-safe.
There are some Lua-specific libraries for multithreading, like Lanes, Threads and effil that will probably work (disclaimer: I've never used any of them, just know they exist).
As far as I know, all of them create brand new Lua states for every new thread and share data between them in some way.

We could create new states for each thread in the C land before calling to Lua, which doesn't seem that bad, but then how would data sharing work? Maybe making the script instance not be a table and use only Godot data like Dictionaries? What about globals variables? Should we patch _G to a thread-safe version?
Anyway, I'm open to ideas, if you have any =]

That's why I think Lua PluginScript should probably just not deal with this, it's no easy task and there are already options out there for multithreading, although they will probably not work directly with Godot's Thread...
There's always GDScript, Visual Scripting and C#, each script can be written in the language that better suits the needs, including the need for multithreading.

[bojjenclon]

Good to know. I figured out a solution by communicating with a gdscript object, but I just wanted to make sure that's the "intended" way to do things in this case. So thanks for the explanation!

[gilzoide]

I don't know if "intended" is the best word for this, but yeah.
Maybe one day we can have a threaded version of the PluginScript, but it needs more planning/researching to be done appropriately.

[Calandiel]

Spawning new Lua states is likely the best approach for multithreading in the language. LuaJIT exploits single threaded nature of Lua extensively when dealing with the global table (or references in general).
For example, see Love2D and its approach to threading:
https://love2d.org/wiki/love.thread
It spawns a new, unique lua state for each thread and gives some degree of communication through channels.

[gilzoide]

I agree, we cannot multithread safely in LuaJIT without spawning new Lua states.

Godot's Array and Dictinaries are thread-safe if you don't change their container size, so they could be shared between threads, even across Lua states, as they're still just pointers (like godot_array * or godot_dictionary *). I guess the same is true for godot_variant *, so we could pass around about any value that is not Lua specific (no tables, Lua functions, Lua coroutines).

Do you have any thoughts on how thread support could be implemented?

[gilzoide]

@Calandiel most of the reasoning is already in this thread, but I think the problem is mostly about how to interface this threading stuff with the Godot side of things.
If this plugin were to implement just Lua-specific threading, in my opinion it is best to just use ready-made battle-tested packages from Luarocks.