More Macros in Nemerle

After a very brief introduction to Nemerle metaprogramming, what follows is some motivation and better expression macros examples.

Historically in C#, the using statement has been abused (guilty!) to provide a more convenient syntax to operations that have natural wrap up code which should be executed at the end of a logical scope. To enable a class to be usable like this, it just needs to implement the System.IDisposable interface. The problem is that these elements were designed with a very specific and important goal in mind: the deterministic disposal of unmanaged resources. That's why using wraps the call to IDisposable.Dispose() in a finally block: it has to run no matter what.

A finally block is a very strong statement. It runs in the presence of exceptions, and it can't be interrupted by a ThreadAbortException, what can delay a whole application domain shutdown. That's justified for releasing resources, but it is too strong a statement for simple wrap up code. If an exception happens, the state of the whole operation is compromised, what might mean that it doesn't make sense to run the wrap up code at all.

What's desired, and not possible in C#, is to have a new statement that has a different expansion as the using block and that integrates seamlessly and consistently into the language. Closures can be used to simulate this; but Nemerle goes one step further, and so it doesn't lock the developer with the original design of the language, enabling more expressive code that doesn't abuse the existing language constructs.

This is just one example of abuse. Other interesting examples are the mixin pattern that I've written about before and the code contracts feature, which requires unnatural syntax and an external assembly rewriter to enable design by contract in C#.

So, following the steps of IDisposable and using, I'll create the IScopable (I know, it's a bad name) interface and the scope macro in Nemerle.

IScopable will live in its own assembly, Scopable.dll:

namespace Scopable {
  public interface IScopable {
    EndScope(): void;
  }
}

It's just a simple interface where an implementing class' constructor implies the beginning of a scope and the EndScope method must be called to signal the end of the scope. It shares the simplicity of IDisposable but with different semantics, where the call to EndScope should not be wrapped in a finally block.

The scope macro is created in another assembly, Scopable.Macros.dll, which references Scopable.dll:

namespace Scopable.Macros {
  using Nemerle.Compiler;
  using Scopable;

  macro __Scope(expr, body) 
  syntax ("scope", "(", expr, ")", body) {

    def getStart(symbol) {
      <[ def $(symbol : name) = $expr ]>
    };

    def getEnd(symbol) {
      <[
        def scopable: IScopable = $(symbol : name);
        when (scopable != null) scopable.EndScope();
      ]>
    };

    def (start, end) = match (expr) {
      | <[ mutable $(symbol : name) = $_ ]>
      | <[ def $(symbol : name) = $_ ]> =>
        ( expr, getEnd(symbol) )
      | _ =>
        def symbol = Macros.NewSymbol();
        ( getStart(symbol), getEnd(symbol) )
    }

    <[ 
      $start; 
      $body; 
      $end; 
    ]>
  }

}

The scope macro introduces a new scope for an IScopable instance, where its EndScope method will be called at the end of a block. It takes two parameters: expr, which represents the expression that should result in an IScopable instance, and body, which is the block of code that must execute within the scope. Both parameters are nodes in the compiler's abstract syntax tree (AST), and are implicitly typed as Nemerle.Compiler.Parsetree.PExpr (a parsed expression).

The first two declarations inside the macro are the local functions getStart and getEnd, followed by a pattern matching expression that results in a tuple of two pieces of code to emit: one at the start of the scope and one at its end. Pattern matching is used to identify the format of the expr parameter, which then drives the format of the resulting code.

In Nemerle, there are two ways to declare a variable, with def for an immutable variable, and mutable for a mutable one. The first two clauses in the match expression check for these two forms. What's interesting about the code is that quasi-quotation is used in the pattern matching expressions, instead of the cumbersome code that would result if we were forced to use the AST expansions they generate. In effect, we write the code we want to match, and it's transformed in its AST representation. The $(symbol : name) in the quotation commands the pattern matching to capture the name of the declared variable and assign it to the autovivified symbol variable. $_ is just a shortcut to match anything and not capture any variables, since we don't really care what the rest of the expression looks like. In a successful match for either form, the tuple that results holds the input expression (it's already in the needed format) and the result of calling the getEnd function with the captured symbol name.

The third pattern captures all other forms, with the _ identifier. It then creates a new symbol with the Macros.NewSymbol() call. This new symbol is used to call both previously declared functions. getStart uses this symbol to create an assignment expression where the input expression is assigned to the symbol.

The getEnd function generates the code to end the scope. It declares a variable of type IScopable and assigns the given symbol to it. It then calls IScopable.EndScope() on it if it's not null. If the expression that that symbol represents is not of type IScopable, this assignment will fail compilation, and issue an error message back to the user. This is how the type of the input expression is implicitly enforced to be of type IScopable.

After resolving the code to start and end the scope, it's just a matter of emitting the resulting code in the right order, which is done as the last statement in the macro:

<[ 
  $start; 
  $body; 
  $end; 
]>

This last statement is also the return value of the macro, and it's the code that the compiler inserts in the program.

Now let's create a scopable class (some interesting Nemerle features are explained in the code):

namespace Scopable {
  using System.Diagnostics;
  using System.Diagnostics.Trace; // import the static members of a class

  public class Tracer : IScopable {
    private stopwatch = Stopwatch(); // creating an object doesn't use "new"
    private title: string;

    public this(title: string = "Execution") { // a constructor
      this.title = title;
      Start();
    }
    
    private Start(): void {
      WriteLine($"$title start"); // string interpolation
      stopwatch.Start();
    }

    private Finish(): void 
    implements IScopable.EndScope // aliased interface member implementation
    { 
      stopwatch.Stop();
      WriteLine($"$title end, took $(stopwatch.ElapsedMilliseconds / 1000f)s");
    }
  }
}

And use it in a program:

using System;
using System.Diagnostics;
using System.Threading;
using Scopable;
// Scopable.Macros.dll must be referenced as a macro reference, 
// its macro will be expanded in this code, 
// and it won't be part of the final assembly. 
using Scopable.Macros; 

module Program { // just like a C# static class, but without the cruft

  Main() : void {

    _ = Trace.Listeners.Add( // explicitly ignore the return value
      TextWriterTraceListener(Console.Out)); 

    scope(Tracer("The answer")) {
      Thread.Sleep(42);
    }

    _ = Console.ReadLine()

  }

}

The output was not quite the answer I was looking for:

The answer start
The answer end, took 0.044s

Tracer can also be encapsulated in its own macro, in a separate assembly from scope (macros can only be used - even by other macros - if they're referenced from an already compiled assembly):

namespace Tracer.Macros {
  using Scopable;
  using Scopable.Macros;

  macro __Tracer(title, body) 
  syntax ("trace", Optional("(", title, ")"), body) {
    def message = title ?? <[ "Execution" ]>;
    <[
      scope(Tracer($message)) $body;
    ]>
  }
}

The trace macro reuses the scope macro and just creates a scope with a Tracer instance. It also shows how to declare optional arguments in its syntax. It's usage is simple:

trace("The answer") {
  Thread.Sleep(42);
}

Nemerle also has a bunch of useful macros in its standard library. Some examples are macros for design by contract, lazy evaluation and XML literals.

There's no need to stick with what the language designers give you with such a powerful tool as macros for metaprogramming. Compiler and extension developers themselves profit a lot with such a foundation. There're certainly other ways to achieve this level of language customization, but the Nemerle compiler shows a very powerful and synergistic model, where algebraic data types (for the AST), pattern matching and quasi-quotation come together to make metaprogramming easy and declarative in a statically typed, C#-like language.