Lua Build Bou
 |
||
|
Lua Build Bou |
|
Bou has grown up and been renamed lake; see [1].
I first got interested in build tools implemented as embedded DSLS (Domain Specific Languages) by Martin Fowler's article on Rake [2]. Build languages like make or nant are classic DSLs, although the term 'little language' [3] has been around a lot longer and in fact is an important part of the Unix development philosophy of specialized tools that do one thing well.
Rake is an embedded DSL because it is hosted within a big language. That offers big benefits to both developer and user; the developer doesn't have to muck around reinventing a high-level programming language, and the user has the comfort and power of a tested programming language. It's easy to do unusual things without having to extend the language. Consider make; its power comes from having all those zillions of Unix commands floating around - it is much less powerful in non-POSIX environments. And one usually gets a bizarre mixture of make and shell script which is (shall we say) less than straightforward to maintain. nant is certainly cleaner, and one can write custom tasks in CLI languages, but one still has to 'drop out of the DSL' (as Fowler puts it) to do non-trivial things. XML is excellent for specifying hierarchical data, but not so good as a syntax for programming.
I've always considered Lua to be a good language for embedded DSL applications, which leads to Bou. Bou (pronounced like 'beau') is a build system based on pure Lua - the only external dependency is on the LuaFileSystem (lfs). It is deliberately similar to make in philosophy and uses the same language of targets, rules and dependencies. However, it brings two powerful things to the party; the ability to use arbitrary Lua code and a great deal of canned knowledge about some common compile tools.
The English language is rapidly running out as a resource for good open-source project names - 'lake' would have been perfect, but alas there's already a build system called that! 'bou' is Afrikaans for 'build'; you put your targets and rules in a 'boufile'.
Consider an old friend:
#include <stdio.h>
int main(int argc, char**argv)
{
printf("Hello, World - %d parms passed\n",argc);
return 0;
}
Writing a makefile for the canonical "Hello, World!" program is overkill, but the equivalent boufile is very straightforward:
c.program 'hello'
Alternatively, you can get Bou to deduce that you have a C program, and say this:
program 'hello.c'
Executing Bou will give the following output:
> bou gcc -c -O1 -DNDEBUG hello.c gcc hello.o -o hello.exe > bou bou: up to date
Which is not in itself very impressive. But this simple boufile gives you features for free: it already knows about 'clean', knows how to use the Microsoft command-line compiler (at least on Windows), and knows how to make a debug build:
> bou clean removing 1 hello.exe 2 hello.o > bou CC=cl cl /nologo -c /O1 /DNDEBUG hello.c hello.c link /nologo hello.obj /OUT:hello.exe > bou CC=cl DEBUG=1 cl /nologo -c /Zi /DDEBUG hello.c hello.c link /nologo hello.obj /OUT:hello.exe
Observe that it was not necessary to invoke 'bou clean' before doing a debug build, because Bou is intelligent enough to know that the build has changed. This is done using a specfile:
> cat boufile.spec link /nologo $(DEPENDS) /OUT:$(TARGET) cl /nologo -c /Zi /DDEBUG $(INPUT)
By comparing the existing specfile to the commands generated, Bou can deduce that the commands have changed, and therefore require rebuilding.
For such a simple case, you can do without a boufile entirely, and let Bou deduce the tool needed to compile and run the file you specify:
> del hello.exe > bou hello.c 10 20 30 gcc hello.o -o hello.exe hello.exe 10 20 30 Hello, World - 4 parms passed
Notice that hello.o was not regenerated!
Consider a program with two files, one.c and two.c, which is called first.
c.program {'first',src='one,two'}
Running Bou, we get:
> bou gcc -c -O1 -DNDEBUG one.c gcc -c -O1 -DNDEBUG two.c gcc one.o two.o -o first.exe
This is not a very realistic situation - in practice, the source files will at least depend on some header files, and you will need to specify libraries. To specify more optional parameters, we use a common table idiom for passing 'named' parameters - note the curly braces:
c.program{'first',src='one,two',
compile_deps='common.h',libs='user32,kernel32'}
We will now link properly against the required libraries, and if common.h changes, the source files will be recompiled:
> bou gcc -c -O1 -DNDEBUG one.c gcc -c -O1 -DNDEBUG two.c gcc one.o two.o -luser32 -lkernel32 -o first.exe > bou CC=cl cl /nologo -c /O1 /DNDEBUG one.c one.c cl /nologo -c /O1 /DNDEBUG two.c two.c link /nologo one.obj two.obj user32.lib kernel32.lib /OUT:first.exe
libs gives Bou a list of libraries; it then decides how to format this list in the way appropriate for the particular tool. Other parameters include incdefs, for setting a list of include paths, and defines, for defining macros.
It is possible to say src='*.c', but this doesn't handle the fact that each source file has individual dependencies.
One very common format for expressing dependencies is the 'deps' format emitted by tools like GCC, suitable for inclusion into a makefile. Bou can process such a format explicitly. Consider the following boufile:
-- bonzo.bou cpp.defaults = {defines = 'SIMPLE',libs = 'user32'} cpp.program {'bonzo',rules=[[ cppfile.o: cppfile.cpp cpp/inc.h c/common.h cfile.o: cfile.c c/inc.h c/common.h clib.o: c/clib.c c/inc.h ]]}
The rules parameter can be set to a filename, but if the string contains newlines it's assumed to be verbatim. Bou will do three things from this specification:
* generate targets based on the implicit rules it knows * extract the include paths * construct the dependency list for each target
> bou -f bonzo.bou g++ -c -O1 -DNDEBUG -DSIMPLE -Icpp -Ic cppfile.cpp gcc -c -O1 -DNDEBUG -Ic cfile.c gcc -c -O1 -DNDEBUG -Ic c/clib.c g++ cppfile.o cfile.o clib.o -luser32 -o bonzo.exe
Notice how global library and defines settings can be set using cpp.defaults.
Consider the common task of needing to build and run a number of test programs. These may require compilation (like C/C++) or can be directly interpreted. A boufile for a directory containing both C and Lua test programs could look like this:
target('all','c,lua') target('c',forall_results('*.c',go)) target('lua',forall_results('*.lua',go))
The first target 'all' explicitly depends on the targets 'c' and 'lua'; the dependencies for 'c' are the result of generating program build-and-run targets for all C files in this directory, individually handled by go(). forall_results() is rather like map, except that it can take a wildcard instead of an explicit list and can collect multiple results from each invocation of the function.
The rule function is passed the input extension, the output extension, and a command to turn the input into the output file; the standard variables INPUT and TARGET are set for you. Rather than setting a global rule, it returns a rule set to which you add target names. In this example, progs 'one' is short for progs:add_target 'one'. rule:add_target also has a second argument which can be used to pass explicit dependencies.
progs = rule('.c','.o','gcc -c $(INPUT)') progs 'one' progs 'two'
The target function takes three arguments, a name, any dependencies, and a command to be executed. The dependencies can be a list (a string will be converted automatically) or a set of dependencies, using depends. If the dependencies argument is nil, then the target is unconditional. The command can either be a Lua function, or a string, in which case it is interpreted as a command to be run using the shell.
The depends function is useful for defered calculation of dependencies. Here is a target which depends on the results of two target sets, which are only populated later:
progs = rule('.c','.o','gcc -c $(INPUT)') files = rule('.gif','.jpg','convert $(INPUT) $(TARGET)') target('all',depends(progs,files),function() print 'yes' end) progs 'one' progs 'two' files 'pool'
At the height of the XML craze, it would have been natural to encode build rules something like this:
<program name="first" compile_deps="common.h" libs="user32,kerner32"> one, two </program>
This is also a tool-agnostic notation, but it is not a very good programming notation. By making the build language a subset of a real programming language, doing non-standard extra things doesn't require 'jumping out of the DSL'.
Bou has a way to go before it can handle all the tasks expected of a build environment, including support for installation. But hopefully it is a good starting base, and also I hope it shows that Lua is excellently suited to this kind of 'little language' application.
[4] contains bou.lua and some small example projects. Unzip this somewhere, and make sure that you have lfs on your package cpath. lua bou.lua -f hello.bou should now work; if no boufile is provided it will look for boufile in the current directory. Then create a batch file
@lua <path-to-bou.lua> %*
or script file, depending on your religion:
#!/bin/bash lua <path-to-bou.lua> "$@"
(The "$@" ensures that quoted parameters will be passed properly.)