Quick tour

What it looks like: Lua code

Here are a couple of lines of plain Lua, just to give a quick taste. Hopefully code is self-explanatory to most programmers:

----------------------------------------------------
-- Eratosthene's sieve
----------------------------------------------------              
function print_primes (limit)
   local sieve = { }
   local i = 2
while i < limit do while sieve[i] do i=i+1 end print (i)
for j = i*i, limit, i do sieve[j] = true end i = i+1
end end print_primes (100)

Lua syntax successfully tries not to be scary, but it offers all the power you might wish from a modern language, including real function closures, coroutines, introspection, runtime metaprogramming, error handling, sandboxing... For some serious stuff written with Lua, go here for some name-dropping, or there for open source projects

What it looks like: Metalua code

Metalua looks quite the same as Lua, except for +{...} and -{...} constructs which allow to shift meta-levels (i.e. transforming quoted code into data representing it, or conversely, evaluating a piece of code and replacing it with whatever it generated). However, since Metalua lets you extend its syntax in arbitrary ways, you can make it look just as you wish.

Metalua's primary purpose is not to simply allow a couple of macros. Most compile-time code manipulations need, in order to be readable, the support of some dedicated syntax, and Metalua helps you design and implement such syntactical support (if you feel no need for syntax support, it might mean that you macro doesn't map a major concept, and probably does more harm than good. Such macros hurt readbility, so when in doubt avoid meta-programming). Examples of useful macro extensions include:

Pattern matching

This is taken from ML. It lets you analyze the structure of some data, bind sub-parts of it to local variables, and choose a block of statements to execute depending on the analysis result, all in a single control statement. It looks like this:


----------------------------------------------------
-- Lambda-Calculus evaluator (weak head normal form)
----------------------------------------------------              
-{ extension "match" }

function replace(var, newval, term)
   match term with
   | `Var{ v } if v==var -> return newval
   | `Var{ _ } -> return term
   | `Apply{ f, x } -> 
      return `Apply{ replace(var, newval, f), replace(var, newval, x) }
   | `Lambda{ v, _ } if v==var -> return term
   | `Lambda{ v, b } ->
      return Lambda{ v, replace(var, newval, b) }
   end
end

function reduce_whnf(term)
   match term with
   | `Apply{ `Lambda { param, body }, arg } ->
      local x = replace (param, arg, body)
      return reduce_whnf(x)
   | _ -> return term
   end
end
Type annotations
Writing a syntax extension

The following piece of code adds a "let <var_list> = <expr_list> in <expr>" syntax extension, and uses it (it would be much cleaner, in the real life, to put the syntax extension and the code which uses it in separate files). The code executed at compile time is in blue, the code which is executed at runtime is in green:


---------------------------------------------------------
-- the "-{...}" means that we're going to do compile-time
-- stuff (here, syntax extension) 
---------------------------------------------------------
-{ block:
   ------------------------------------------------------
   -- Register the additional keywords in mlp.lexer
   ------------------------------------------------------
   mlp.lexer:add{ "let", "in" }

   ------------------------------------------------------
   -- Extend the expression parser; code generation is
   -- delegated to the function let_in_builder() below.
   ------------------------------------------------------
   mlp.expr:add{ 
     "let", mlp.id, "=", mlp.expr, "in", mlp.expr, 
     builder = let_in_builder }

   ------------------------------------------------------
   -- This creates the code returned by the macro.
   -- Notice the holes-in-quote-in-splice.
   ------------------------------------------------------
   local function let_in_builder (x)
     local variable, value, expr = unpack (x)
     return +{
       function (-{variable}) 
         return -{expr} 
       end (-{value}) }
   end
} -- back to "normal" code
 
a, b, c = 1, 1, -2 
roots = let sqrt_delta = (b^2-4*a*c)^0.5 in 
        { (sqrt_delta-b)/(2*a), (-sqrt_delta-b)/(2*a) } 

This program will compile "let <foo>=<bar> in <some_expr>" as if it were "(function(<foo>) return <some_expr> end) (<bar>)" (look at the quote in function let_in_builder()).

The code inside -{...} is executed during compilation, and alters the compiler itself. The bits of code in +{...}, returned by let_in_builder(), are quoted code, i.e. code intended to be spliced into regular code by a macro. -{...} inside +{...} allow to poke holes in a quote, and fill it with arbitrary data. Finally, mlp.xxx functions control mlp, the Metalua parser, and let you alter the language's syntax, typically to introduce your own macros.

In this example, you see a couple of design decisions made for Metalua: first, when you want to do advenced stuff you need to know quite a bit about what goes under the hood. This is inherited from Lua's philosophy: simple things are simple to do, but for complicated stuff, instead of hiding the scary details under leaky abstractions, we keep things as neat and simple as possible and expect you to understand them. So here you're expected to understand what should go at which meta-level, what are the main extension points in the grammar, etc. These points are chosen so that by using them, you're likely to do The Right Thing, i.e. design a syntax that is likely to be compatible with other well-written extensions. Syntax tries to guide you, by making better things look simpler. It's easy to circumvent, and that's fine: the point is to warn you when you're about to do something dangerous. If you knowingly take the risk of shooting yourself in the foot, once you're aware of that increased risk, the compiler did its job.

Finally, notice that this sample is bigger than necessary. A much terser version is provided in the standard library, although it uses slightly more advanced features.

Lua

Metalua being based on Lua, you'll need to know it. It's got the standard features you'll find in most dynamic languages such as Python, Ruby, Javascript etc. Howver, its grammar and semantics are cleaner and more consistent than most of these, thus making it an ideal platform for metaprogramming. Among not-so-mainstream features of Lua, these are noteworthy:

  • Functions are real closures. Thanks to transparent upvalues, they integrate nicely with imperative-style code.
  • Coroutines allow collaborative multithreading, easy generators, and many other interesting idioms depending on execution stack manipulations.
  • Metatables allow to deeply control the behavior of data. For instance, they allow to use tables for object oriented programming (prototype based, class based, with single or multiple inheritance, mixins, AOP, CLOS-like...). They also allow to build interesting interfaces to foreign data, e.g. proxy objects which serialize and send their data automagically.
  • Integration with C is taken very seriously: I'm not aware of any language which integrates as smoothly as Lua with C, without being itself a superset of C.

Bigger examples

There are less trivial examples in the standard library and in the manual: ML-like pattern matching, OO syntaxes, exceptions, extra operators, python-like significant indentation, lists by comprehension, lazy evaluation...

Lua resources

There's plenty of excellent stuff on the web to learn Lua, so I won't write yet another tutorial. Here's a list of my personal favorites: