Jim Jennings

wiki

Interest

My interest in Lua stems from a general interest in programming language design and implementation. For me, this is a diversion, a hobby, and an opportunity to stay connected with programming since I do not code at work.

The Darwin module system

During a summer vacation in June/July 2009, I wrote a first draft of the Darwin module system for Lua, which is an extension to the native Lua module system. Back in the early 1990's, I worked for a few years with [Jonathan Rees]. I had been programming in Lisp, and was fortunate enough to learn Scheme from Jonathan. The module system of [Scheme48] is tremendously useful because it provides a number of security guarantees. The Darwin module system is my own attempt to obtain some of the same guarantees in Lua. In particular:

• A module should have its own environment. Access to the objects (functions, data) of a module is granted via access to an environment.
• Therefore, you can control the privileges given to a piece of code by controlling the environments which that code can access.
• A simple declarative language for describing modules is the only tool that a user needs in order to "administer" the module system, i.e. to create the appropriate environment in which the user's code should run. (See examples below.)

Since I'm interested in Lua as an embedded language, I want strong guarantees about what capabilities are granted to user code, because the users will not be the developers of my projects; the users will be customizing my projects by injecting their own Lua code.

Relationship to the Lua module system

The Lua module system is extraordinarily flexible. Like Lua itself, the module implementation is fairly "exposed" and therefore can be extended to do all manner of strange and wonderous things. To obtain security guarantees, I wrote Darwin as a layer on top of (next to?) the Lua module system. The Lua module system is fully functioning when you use Darwin. Darwin merely adds an additional power: the power to create a run-time environment that has read-only access to an explicit set of modules and no more.

With Darwin, it is possible to do some things that are awkward (or impossible) without it, e.g.

• Ensure that user code cannot modify module code on purpose or by accident... without requiring the module author to follow any particular guidelines or rules.
• Guarantee that user code cannot obtain privileges (capabilities) that it has not been explicitly granted. Note: Darwin needs additional work in this area.
• Calculate module dependencies both statically (from declarations) and dynamically (at run-time)... without requiring the module author to follow any particular guidelines or rules.
• Expose more than one view of the same module code, e.g. a limited-privilege view for production use and a fully-exposed view for development and debugging... without modifying the module source code.
• Rename module functions and/or place them in a table whose name you can choose, regardless of whether the module source code was written to allow this... without modifying the module source code.

Due credit

The main ideas in Darwin came from Scheme48, which in turn was influenced by ML. Having just now read Jonathan Rees's [PhD thesis] after writing Darwin, it seems that I've implemented in Lua a fair portion of his security kernel design. There are a few things I have yet to do to round out the features that I think are needed in Darwin, including:

• Isolate the getfenv and setfenv functions into a separate module to make it easy to deny access to them (because they leak privileges); and
• Explore adding "type authentication" for Lua objects so that a function cannot be spoofed into accepting a forged object just because the forged object has the same structure as an authentic object. This is also a topic covered in Rees's [PhD thesis].

Code and examples

Attached below is a tar/gzip file containing the source code and some tests. I've written some documentation, which is an informal paper describing the design, plus a short reference section:

• [Darwin source code (tar/gzip format)]
• [Darwin documentation and reference (html format)] - Please excuse the lack of internal document links for navigation. This page was generated (mostly) automatically.
• [Darwin documentation and reference (rtf format)]

Darwin is (C) 2009, James S. Jennings, and will be released under the MIT open source license.

Darwin is pure Lua, and works with Lua 5.1.4. I don't know if it works with older versions.

Here is a file that I load every time I start Lua (after Darwin loads). It defines a set of structures for modules I use often. The way I have Darwin configured (which is the default), Darwin puts entries into package.preload for each declared structure. That way, my code can simply call require, e.g. require "list" or require "lanes". Darwin works fairly smoothly with the existing Lua package system.

structure.declare { name="list"; 

		    location="."; 

		    open={"_G"};

		    objects={"null"};

		    files="list.lua";

		 }



-- Just for demonstration purposes

structure.declare { name="pair"; 

		    signature={"cons", "car", "cdr", "isnull", "null"};

		    open={"_G"};

		    objects={"null"};

		    files="list.lua";

		 }



structure.declare { name="recordtype";

		    -- debug package is only needed because recordtype uses "strict"

		    open={"_G", "package"; "table", "string", "debug"};

		    environment=[[ require("recordtype"); return recordtype ]];

		 }



structure.declare { name="lprocess";

		    open={"package"};

		    environment=[[ require "lprocess"; return lprocess ]];

		 }



structure.declare { name="queuestack";

		    location=".";

		    files="queuestack.lua";

		 }



structure.declare { name="points";

		    location=".";

		    open={"_G", "math"};

		    objects={"point", "points"};

		    files="points.lua"

		 }

Note to MacOS and Windows users: You might need a small edit to _darwinpackage.lua_ in order for Darwin to load native (C/C++) libraries correctly. The change, and the reason for it, are explained in the documentation, under "Limitations". I'll create a work-around at some point.