Module Execution Proposal
 |
||
|
Module Execution Proposal |
|
This is a proposal for a new command-line switch (-p) that loads a function with the given package name via the searchers in package.loaded and then executes that function as a script, passing the command-line arguments to the function as arguments.
lua -p modname args
Example:
-- my/app.lua
print("hello", io.read"*a", ...)
local i=0; while arg[i] do print(i, arg[i]); i=i-1 end
$ echo -n 123 | lua -p my.app 3 4 5
hello 123 3 4 5
0 my.app
-1 -p
-2 lua
A typical use of this would be if you have a Lua script installed in the Lua path (e.g. LUA_PATH) rather than the system PATH, and you want to locate and execute that script. The script may have a dual use as a module and a command-line utility. Maybe this is a Lua-based profiler, debugger, code validator, or macro processor that operates on other Lua code.
lua -p yet.another.debugger mybuggyprogram.lua 3 4 5 lua -p yet.another.preprocessor myprogram.m4 4 5 5
There is precedent here--for example, the "-m" switch in Python [1]:
-m mod : run library module as a script (terminates option list)
Current alternatives for solving this are not satisfactory.
First, you can install a script into the system PATH, but this is system-dependent, and it puts a separate file in a different place outside the Lua module repository.
One solution that sort-of achieves what we want is this:
echo -n 123 | lua -e 'require "my.app" (3 4 5)'
where my.app is modified to return a function that operates on the arguments provided. The problem is that the command-line arguments are escaped inside the string. There are a number of reason why we would want to avoid that if the application is to be used a regular command-line utility. For example, we might want to define the my.app program with an alias in bash. The proposed solution allows this simply:
alias myapp="lua -p my.app" echo -n 123 | myapp 3 4 5
We might attempt this:
echo -n 123 | lua -lmy.app 3 4 5
That fails with error since Lua interprets "3" as a file name. We might work around that with this hack:
echo -n 123 | lua -lmy.app -e'os.exit(0)' 2 3 4
However, the arguments are not passed to the my.app module, neither via ... or arg. It's not passed via ... because the "-l" switch uses require, which has its own definition of what ... should contain, namely the package name. Furthermore, when the Lua interpreter processes "-l" options, the arg table is not yet constructed. Maybe it should be, but in any case we might want a more elegant solutions to the arg global variable.
Another possibility might be this:
echo -n 123 | lua -e 'require "my.app" (...)' 3 4 5
Again, my.app is modified to return a function that evaluates the arguments passed to it. This has two problems though. Again, Lua interprets "3" as a file name. Even if we eliminate that problem, the Lua interpreter does not define ... in -e statement I think it should [4], and this is a separate proposal given at the end of this document.
Allowing ... to be passed to an -e statement would be an improvement in Lua. It's still idiosyncratic, and loading a script from the package search mechanism seems fundamental enough to have a dedicated option with convenient syntax. Sure, the minimalist says that this is not strictly needed, but neither is having dedicated options for loading a module off the file system or from standard input (-) :
lua -e 'dofile("my/app.lua")'
lua -e 'assert(loadfile())()' < my/app.lua
One suggestion has been to extend require so that it would pass command-line arguments if loaded via -l. However, the essence of require(x) is that you always get the same result for the same value of x (idempotent). If it doesn't memoize, its semantics are radically different. [3]
Some other syntaxes have been proposed for the "-p" option:
lua -Lmy.app args -- showing relationship to "-l" option lua +my.app args -- related to "-" for standard input source lua @my.app args -- proposed in [8] lua -m my.app args -- Python style lua -a my.app args -- load "application" lua -p my.app args -- load from *p*ackage *p*ath.
"-L" was suggested due to the similarity with "-l", but this similarity may be misleading. "-l" goes through require, but "-L" would not. "-L" would also be the first upper-case switch.
The "+" syntax is otherwise very good, but it's possibly not POSIX complaint [2], and there is an ambiguity to be resolved in the odd case where a file name actually starts with "+".
lua ./+my.app args -- workaround using current directory "." lua -- +my.app args -- maybe treat as file name if followed by "--"?
The "-m" (as in Python) might be confused with loading a module (-l), which is -m in Perl, so -m doesn't seem convincing. We are not loading a standard module but only using the package path (-p) search mechanism to load a function that may or may not be a full blown module.
An earlier proposal of this idea for Lua was in [7][8]. That proposed a new LUA_RPATH environment variable that would be searched independently of LUA_PATH/LUA_CPATH:
LUA_RPATH=?.lua lua @my.app <parameters>
There were some questions concerning the need a new environment variable separate from LUA_PATH. Secondly, LUA_RPATH only locates Lua source files (like LUA_PATH) but does not consult other searcher functions (as occurs by going through package.loaders)--see "Related Proposal: New package.find Function" below.
The following patch implements the above proposal for the -p switch. It is possible to make this patch entirely in lua.c, but we see that the "-p" switch and the require function (from loadlib.c) share much common functionality, so it seems useful to factor out that common functionality into a new function loadmodule, and it could be useful to expose that function to Lua as well (as done here). loadmodule is somewhat analogous to the other load* functions and has this behavior if written in Lua:
local function loadmodule(name) local s = "" for i,loader in ipairs(package.loaders) do local f = loader(name) if type(f) == 'function' then return f end if f then s = s .. f end end return nil, s end
Here is the patch against Lua 5.1.2:
A related proposal is for the '-e' switch to terminate command command-line options and accept command-line options through ... and arg:
$ lua -e 'print(...)' 3 4 5 3 4 5
For example, Perl and Python implement this behavior:
$ perl -e 'print @ARGV' 3 4 5 345 python -c 'import sys; print(sys.argv)' 3 4 5 ['-c', '3', '4', '5']
Compare the Lua command-line form to the Python one:
usage: lua [options] [script [args]] usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
The Python format makes clear that there are four different types of input sources, all treated equally and all able to accept the command-line arguments. The Lua format might be rewritten as
lua [options] [(-e stat | -p mod | file | -) [arg]]
The only further change in Lua this involves is for -e to terminate command-line options and accept arguments, as suggested above.
Note that the above might imply that there can only be one -e switch. Compare Lua behavior here to Perl and Python:
$ lua -e 'print' -e '"1"' # invalid, each -e is treated
# as a separate function
$ perl -e 'print' -e '"1"' # valid, all -e's are concatenated
$ python -c 'print' -c '"1"' # valid, but second -c is interpreted
# as command-line argument
The above suggests a new function loadmodule that loads a function given a package name. Another potentially useful function would map a package name to a file system path [3]:
-- func = package.find(path, name) dofile(package.find(package.path, "foo"))
It's possible to implement this in pure Lua though--see LuaModulesLoader.
This is only applicable though to modules that do map to paths on the file system (e.g. via LUA_PATH or LUA_CPATH). One alternative for the -p switch proposal is to base it on package.find rather than loadmodule (which is essentially what Python did), but this limits the types of functions that can be loaded with this mechanism, possibly to only pure Lua files, and it is the rationale for Python's PEP 338 [1][9] which suggested doing away with this.
How does a module determine whether or not it is require'd rather than simply executed (e.g. via loadmodule or dofile)?
Ideally one would like to pass the information in as an argument (in ...), but that's incompatible with current practice where ... contains the command-line arguments if executed as a script or the module name if required. If a module is used in both ways, there's an ambiguity if the first command-line argument might happen to be a module name. Try
lua my/app.lua math
Here ... == "math", so package.loaded[...] is set. It is the case that package.loaded["my.app"] is probably not set, but you'll need to hard-code the package name "my.app" in your file.
An alternative may be to make use of a global (e.g. arg or package.loaded) or as in Python. We might even look up the stack as in local is_required = debug.getinfo(2, "f").func == require, but that not only has the disadvantage of using debug [4] but breaks if one replaces require with another function of equivalent functionality [3].
A reasonably simple solution that works well in practice (at least in 5.1) is to test for a sentinel loaded in package.loaded [5][3]. This does rely on undefined behavior and is not the cleanest. Ideally, we would want to replace this with a language feature.
A proposed change to Lua is as follows. One is a package.loading table analogous to the package.loaded table. It will contain the package names of all modules that require is currently loading. A Lua function can test package.loading[...] to determine whether it is being required. A related approach is a function package.status(name) --> loaded, loading, false. A potential problem with these methods is if you execute a module
lua my/app.lua a b c
where a just happens to be the name of a module that is currently loading (e.g. maybe in another coroutine). That would be rare, but it might happen in some situations.
A rather simple solution to all this is to just use separate files for code that is required and code that is run [4]. However, it can be nice (e.g. in unittest) to have them in the same file--for one thing, it's sort of documentation [3].
This article somewhat follows the format of a Python PEP [6].
Author: DavidManura, based on discussions with RiciLake, doub, and others.
Created: 2007-09
Lua 5.1.
(none)