Non Blocking Lua Execution

wiki

Currently, Lua code is executed like this:

1. Load Lua script.

2. Run Lua script.

(The Lua script executes until it's done. The C program is blocked.)

3. Exit.

There should be API functionality which lets a C program run a little bit of Lua code at a time. For example, there could be a function which executes the next Lua operation, then returns control to the C caller. Lua code would execute like this:

1. Load Lua script.

2. Run the next Lua operation.

3. Do any other operations which only the C program can do.

4. If the script is done, exit.

5. Else, go to step 2.

The C program is never blocked by Lua, and it has more control over the Lua VM. It should be safe to read and modify the Lua state (variables, etc.) between operations, as long as the script itself can handle it. The C program should also be able to stop the Lua script at any time.

In Lua 5.1.1, an internal function called luaV_execute runs script operations. It runs operations in an infinite loop. Between operations, it calls the debug hook. There appear to be two basic things which can cause this function to return: special script operations and a debug hook which sets the "yield" status. (I'm not an expert on the Lua source code, so I could be mistaken). Perhaps it would be best to get rid of the infinite loop and make some API changes that allow C code to set the number of times the loop may run. Ideally, it wouldn't be necessary to use any hooks or callbacks.

- It cannot be changed by the Lua script or point to a Lua function.

- The purpose of the hook is to tell luaV_execute (through a return code) whether it should yield or not, so that lua_pcall or whatever function was used to invoke Lua will return.

- When luaV_execute "yields", it yields in a way that is different from the yielding of coroutines, i.e., yielding with this hook should not interfere with coroutine use at all.

What I'm getting at is that we should be able to use the "debug hook" method described above, but without the shortcomings of an actual debug hook. The C programmer could have complete control over when Lua stops blocking. For example: every 100 operations; every operation; never; ... etc.

This approach could involve some changes to the Lua API. The C program would have to be able to resume Lua from where it "yielded", but probably not with the coroutine-centric lua_resume function. It seems like getting coroutines involved here is a bad idea, since it may preclude the Lua script using coroutines.

I think [debug.sethook] can be used for this (see also in PIL http://www.lua.org/pil/23.2.html). Line-by-line breaking in this way can be useful to implement things like Lua debuggers.--DavidManura

I don't think debug hooks are a good solution. I looked in the chat archive, and the first thing I read was the message at [4]. Mr. Sittig worries about whether Lua can be cleanly terminated from within the hook, and what would happen if the script changed the debug hook by itself. The main problem is that the debug hook is called from within Lua; the C program should drive Lua, not the other way around.

I discussed the while loop in luaV_execute some with RiciLake. Removing that while loop and then calling luaV_execute iteratively from your code is an obvious approach, but the problem area is that luaV_execute is reentrant (recursive), so you would also need to eliminate that recursion (maybe a stack approach?) to prevent a recursive call blocking too long. --DavidManura