Wiki pages
There are Lua wiki pages and projects I've been involved in.
Code and Projects:
- Formal Lua modules (core):
- Formal Lua modules (more):
- LuaToCee - convert Lua to C code using C API calls
- InlineCee - simple module that allows C code to be embedded inline in Lua code.
- LuaDist - CMake-based ports system for Lua and non-Lua packages.
- LuaFish - Lua 5.1 parser, macro system, and static type checker in LPeg [1] (related to Metalua [2])
- LuaSearch - effort to standardize Lua module documenation via POD
- ModuleReview - effort to review/classify modules in terms of quality and usefulness for Lua distributions
- SlightlyLessSimpleLuaPreprocessor - enhanced version of the enhanced version of the Lua preprocessor
- LuaProxyDll - avoid DLL dependency issue
- DetectingUndefinedVariables -
checkglobals function is an alternative to etc/strict.lua and luac -p -l - | lua test/globals.lua.
- LuaGenPlusPlus - A C++ header file to generate efficient sequences of Lua C API calls at compile-time via template metaprogramming techniques.
- LuaInterpreterInLua - reimplements the Lua 5.1 interpreter (lua.c) in Lua.
- CommandLineParsing - command line parsing (getopt)
- LuaPatch - Lua implementation of the patch utility
- ExPattern - regular-expression like support implemented in terms of Lua patterns and code generation
- SerialCommunication - serial communications in Lua/alien
- SourceOptimizer -- utility for optimizing Lua source, inlining functions
- StringQuery - string pattern matching and replacement library inspired somewhat by jQuery
- LuaAnnotate - C code comments externally stored but displayed inline.
General Topics, Patterns, Tricks, and Design:
Comments/Annotations on Other Documents:
Lua 5.3 Wishlist
Here are few things bug me, though I'm not necessarily sure what to do about all of them.
- Tracebacks might use some reform.
- Refactoring
f(...) to local ok, err = pcall(f, ...); if not ok then error(err) end or assert(pcall(f, ...)) has side-effects in traceback behavior. LuaList:2012-02/msg00858.html
- other older more general comments: LuaList:2010-08/msg00481.html
- Why truncate paths in tracebacks? LuaList:2006-03/msg00101.html
- The
level parameter used in the error function lacks maintainability since a function might be called from different levels and those levels might be refactored. Refactoring f() to (function() return f() end)() has side-effects in terms of level behavior, with no option to suppress the change in stack level. LuaCarp was a very early effort to improve this. Also see [6][7]. At least in 5.2 work, getfenv/setfenv are mostly removed and setfenv behavior in module is deprecated, so it's moving in the right direction.
- The
assert function is arguably misnamed: [8]. Also, the fail expression is always evaluated.
- I have qualms about metamethods. Despite their uses, they make semantics more complicated.
- Proliferation of raw v.s. nonraw functions and semantics. Also, where's "rawtostring"? [LuaFiveTwo].
- I don't get why the change was made in 5.2 table.* functions to partly honor metamethods. [9][10][11]
- __index (unlike the proposed __getindex) doesn't fully intercept table accesses (hence proxy tables).
- There were proposals to allow 'a:b()' not to depend on 'a.b' (e.g. __mcall or __mindex metamethod) [12]
- NamedParameters as tables seem heavy. What effect does it have on LuaJit optimizations? Converting a function from non-named to named parameters is a fundamental change to its interface, not to be taken lightly.
- Questions remain about versioning. ModuleVersioning/LuaVersionCompatibility. 5.2 is a breaking change and not fully supported in LuaJit. Other languages like Perl 6 have statements like "use v6;" to indicate the code is Perl6 rather than Perl 5.
_ENV = require '_G53'?
- Lua syntax is not always necessarily ideal as a DSL. Consider representing something like PHP, where control structures are embedded inside tree nodes, but StatementsInExpressions are not seamless in Lua. Lua commas can also clutter [13]. Compare to Lisp s-expressions or Ocaml.
- Cleanup on scope exit (ResourceAcquisitionIsInitialization). The prototypical example involves file handles, like in file_slurp readfile [3]. One of the simpler solutions is a Google Go "defer" or D language "scope(exit).
-
local ok, err = f(); if not ok then return ok, err end is a common idiom. Should it be made shorter?
- StringInterpolation for local variables has no really nice solution. Some type of compile-time preprocessing step that expands things like
printf "${x},${y},${x}" into printf("${x},${y},${x}", x, y, x) that is then evaluated at runtime might be a good compromise.
- Forwardly declared local functions look instead like global definitions. Commenting or semantic editors may help some though:
local two
local function one() two() end
function two() one() end
-
package.path and package.cpath are strings rather than arrays. LuaList:2010-06/msg00214.html . This led to the "lib" module [4].
- Debugging info lacks character positions (only has line numbers): LuaList:2010-08/msg00273.html
- luac with multiple files could be generalized: LuaList:2010-11/msg00898.html
- Lack of comments in the Lua source [14] (LuaSource) can make it more difficult to penetrate than otherwise. See LuaAnnotate though.
-
for _,v in ipairs(t) do is ugly, and it's less efficient than for i=1,#t do local v=t[i] do. The too forms do noth have the same behavior either in terms of __len.
Lua 5.2 Wishlist
Caveat: The below list may not be entirely updated to reflect the final 5.2.0 release and may need cleaned up more.
These changes I want are included in the 5.2 work releases (LuaFiveTwo):
- Module system reforms - Make the module system more consistent - see LuaModuleFunctionCritiqued. 5.2 now deprecates the module function and its ilk. Some wanted to improve rather than dispose of the module functions, but there was not consensus on precisely what that would be.
- Environment reforms - I support the _ENV proposal because of the simplifications it makes to lexical analysis (LuaInspect, LuaFish/luaanalyze) and compilation (LuaToCee). I think this is related to a more general problem though involving stack levels (see traceback points above). I agree though that _ENV is ugly and probably should not be used by most user code.
- C function improvements -- Unlike lua_cpcall, avoid allocation in C functions [15] (for performance and avoiding complications in error handling) and support multiple arguments and return values in protected calls. In LuaFiveTwo, lua_cpcall is eliminated and replaced with a lua_pushcfunction that no longer allocates, and multiple arguments and return values are supported via lua_pcall.
- Lua function caching - There is now less need to manually hoist function instantiation out of inner loops since functions are now cached. However, this feature might not be implemented to its full potential[16].
- BitwiseOperators standard library. 5.2 implements this now. I had thought the Lua community had already converged on Lua Bit
- __len metamethod for tables (as well as __pairs -- GeneralizedPairsAndIpairs). This causes me some hesitation though because of the questions it opens. Should we have a rawlen (similar to lua_rawlen in the work version)? Why should pairs() observe metamethods but not next()? Lua tables remain not truly virtualizable (LuaVirtualization). Should "table" functions recognize metamethods?--probably not [18].
- Expose all Lua operations to C API - Extend C API to support missing operators [19]. LuaFiveTwo now supports lua_compare and lua_arith. My main motivation for this is to simplify my LuaToCee output. Also it makes things orthogonal and is relatively easy for Lua to implement. Some things may remain lacking like tail calls in C functions [20].
- Hex escapes in strings are sometimes nice (avoiding conversion to decimal), particularly when reading files or communications streams whose specification includes bytes in hex format or whose individual bits have significance. Sure, you can use a helper function,
hex 'cdef' or binary '0101', but why need that.
- Some ask for ContinueProposal. I tend to find breaking out of nested loops more common. Here's my proposal that generalizes nested break+continue into a single construct: LuaList:2010-11/msg00556.html . See also GotoStatement.
- Minor odds and ends:
- closing a pipe returns exit status
- Protect against triggering undefined behavior in C libraries from Lua standard libraries [21] like
os.date. IMO, the standard library (except maybe debug) should never crash on bad input.
- Rename lua_typerror to lua_typeerror [22]. Actually, the function is now deprecated instead.
- "package.loaders" misnomer [23][24] (package.loaders contains searchers not loaders).
- Other things I don't often run into but I guess are nice: more freedom to yield (including PcallAndCoroutines), xpcall accepting arguments like pcall, non-string error messages in Lua interpreter, emphemeron tables (improves weak table garbage collection).
- `luac -l -l` is currently undocumented (done in 5.2.0)
- Treat unrecognized escape sequences as errors? LuaList:2010-10/msg00449.html
- Overcome implementation limits in number of locals [25] and concatenation AssociativityOfConcatenation is not done, but LuaFiveTwo is increasing "Maximum number of constants per chunk increased to 2^26 from 2^18" though.
Here's other older ideas not addressed in 5.2 and which I haven't decided to move them up to the 5.3 wishlist:
- Some type of macro mechanism. See MetaProgramming.
- Add efficient tests for existence of operators on objects [26][27]
- File handle virtualization - Sometimes you can pass a custom table that looks like a file handle to functions that expect a file handle, but this doesn't work all the time (e.g. if the function tests with io.type).
- Maybe add versions of lua_pushstring and lua_newuserdata that don't longjump. May be useful for mixing C++ exception code and C API longjump code.
- Small source code changes to reduce patching complexity? LuaList:2010-11/msg00899.html
- Reference manual formatting suggestions: LuaList:2010-10/msg00350.html
- Generalize Lua table constructor notation [28]. This also provides list comprehension syntax.
- Manipulations with
... are somewhat awkward unless you copy it in a table (not as efficient) and handle nils correctly. See also VarargTheSecondClassCitizen. Something like the proposed #... and ...[i] would help, although that particular syntax has problems. The dilemma is summarized by Roberto here: LuaList:2010-06/msg00434.html. Some small improvements are likely possible though.
- Abstracted liolib.c [29] - Many people embed Lua in systems without STDIO (e.g. GUI applications) and patch this file. Lua 5.2 provides some abstraction for this (e.g. luai_writestring) but it does not cover liolib.c.
- Clearer syntax errors [30][31][32]. Some improvement is possible. Really good error/warning reporting might require an external linter.
- ModuleExecutionProposal - execute Lua command-line scripts located via LUA_PATH
- On DetectingUndefinedVariables, my current opinion is that it is best handled by the text editor. I find runtime analysis (strict.lua) inferior to static bytecode analysis (globals.lua) inferior to static source analysis (LuaInspect). Furthermore globals.lua and the (hybrid) checkglobals don't work as well in 5.2 due to _ENV changes.
- ShortAnonymousFunctions - It's often said that the Lua syntax for anonymous functions can be verbose for tiny functions used in function programming (map/filter) and functions used as blocks (e.g. simulated switch or try/catch statements). The workarounds in ShortAnonymousFunctions are not very satisfying. It doesn't seem like discussions on the user group are seriously going anywhere here though.
- Allow pragmas in source to inject debugging info into the bytecode. [33] (useful for preprocessors of Lua source). Metalua currently uses its own bytecode generator because of such limitations. There is a workaround but it is not fully general: LuaList:2010-06/msg00311.html . I suspect, however, that rather than injecting debug info, it may be better to just rewrite the message handler.
- More efficiently (O(1)) preallocate or subsequently clear arrays. This may involve both Lua and C API functions like
lua_createtable [34]. See TablePreallocation for the preallocation (does not discuss clearing).
- Lua has the dichotomy of statements and expressions (StatementsInExpressions/ExpressionsAsStatements). The loophole in this are anonymous functions, but this involves a heap allocation, unless it triggers 5.2 function caching, bug that's not always triggered, and syntax can be verbose (ShortAnonymousFunctions). This can be useful in metaprogramming, and Metalua recognizes this (see "Statements in expressions" in the Metalua Manual) and extends the Lua parser/compiler to allow it while requiring no change to the Lua VM. See also the "let" statements mentioned below on this page and "Stored Expressions" in LuaDesignPatterns. The SourceOptimizer provides an automated way to inline anonymous functions into standard Lua 5.1 source code, but the transformation wasn't straightforward.
- The highly dynamic nature of Lua limits its performance. True, Lua is fast, but it could be faster. For example,
x*x is generally faster than x^2 since Lua cannot reduce the latter to the former at compile time (think: x might have a __pow metamethod). Macro processing could alleviate this (e.g. "transforming "y = SQUARE(x+y) + 1" into "y = (let t1 = x+y in t*t) + 1"). See also LuaJIT and LuaToCee. Global, or at least whole-file, flow analysis can help here.
- Some other things, not necessary related to Lua core development:
- Lua needs more standardization of quality control of libraries covering non-ANSI C functionality. ExtensionProposal and LuaRocks may help here. Related concerns involve documentation, testing, and packaging.
- Strength access to existing library frameworks in other platforms like .NET, Java, and Python. Lua acquires the vast resources of these other developer communities.
RecentChanges · preferences
edit · history
Last edited March 9, 2013 9:43 pm GMT (diff)
