This time I'm going to talk a bit as to how to implement function calls in the Dynamic Language Runtime. Last time, I mentioned that I had initially had implemented function definitions as simply returning a raw CodeBlockExpression. This works, but it doesn't give you many options to add custom language-specific behaviors.
A more useful approach in many cases is to wrap your function into an actual object, at runtime, which can then be used for different purposes (including actually invoking the function). So instead of simply moving the CodeBlockExpression around you create a subtree that creates a new object with the CodeBlockExpression as an argument (among other things) and return that instead (with possibly other actions around it, like putting it into a variable).
This is how both IronRuby and ToyScript do it (and I imagine, IronPython). If you look at ToyScript's Def class (the language-specific AST node for a function definition), you will this in action:
Ast.Assign(
tg.GetOrMakeLocal(_name),
Ast.Call(
typeof(ToyFunction).GetMethod("Create"),
Ast.Constant(_name),
Ast.NewArray(typeof(string[]), names),
Ast.CodeBlockExpression(block, false)
)
)
As you can see, it will generate a Call expression to ToyFunction.Create(), which is a static method that returns an instance of the ToyFunction class. At runtime, ToyFunction.Create() will receive the name of the function, the names of the method parameters and a Delegate instance that can be used to actually invoke the code later on. That last part is the interesting bit.
So later on, you want to invoke your function. You already have a variable somewhere that evaluates to a Function object at runtime (in the case of ToyScript, a ToyFunction object as we mentioned). In more general terms, what you will have is an expression that will evaluate to the function object; it really could be anything (like another function call that returns another function). But how do you actually invoke it?
Turns out there are several ways you can implement this, with varying degrees of performance characteristics and needs.
Generating Calls
In the case of invoking a standalone (local or global) function you already defined in your language, the most obvious choice is to use a direct Call expression, via Ast.Call(). We already saw an example in the short snippet above, but for a different use.
Ast.Call() takes a System.Reflection.MethodInfo object (the method to execute) and a params array of expressions to use as arguments to the call (there are other overloads available). If the MethodInfo object points to a non-static method, then the first expression in the arguments list will be the instance [target] object on which to actually make the call.
This is pretty useful if you want to call methods you're aware of at compile/parse time, such as a built-in function like ToyFunction.Create() above, but it's not useful at all to do what we want: Invoking a CodeBlock we created somewhere else.
For this we need to turn to generating a Call action, using Ast.Action.Call():
public static ActionExpression Call(Type result, params Expression[] arguments);
The first parameter is the type of the return value of the call; which can be typeof(object) unless you have more advanced information about what the call will return. The second argument is the list of parameter values you want to pass to the function.
...So how do you tell Call() what to actually invoke?
Yep, that tripped me up as well. Turns out that a CallAction expression expects that the first expression in the arguments list evaluates to a callable object. At least that's how I think of it. But what's a callable object? As I understand it, it means that it references an object that either:
- Implements IDynamicObject, or
- Has a public instance method with the following signature:
object Call(CodeContext context, params object[] arguments);
The current ToyScript incarnation follows path 2. Here's the implementation of the Call() method:
[SpecialName]
public object Call(CodeContext context, params object[] arguments) {
ParameterInfo[] parameters = _target.Method.GetParameters();
if (parameters.Length > arguments.Length) {
if ((parameters.Length > 0 && parameters[0].ParameterType == typeof(CodeContext)) ||
(_target.Target != null && _target.Method.IsStatic && parameters.Length > 1 && parameters[1].ParameterType == typeof(CodeContext))) {
arguments = ArrayUtils.Insert<object>(context, arguments);
}
}
return ReflectionUtils.InvokeDelegate(_target, arguments);
}
The code can look a bit convoluted, but it's actually quite simple: It simple checks the actual arguments supplied to the call to see if the CodeContext has already been included explicitly in it; otherwise it adds it as the first argument, and then uses ReflectionUtils.InvokeDelegate() to actually invoke the ToyScript function represented by this instance of ToyFunction (_target is a field containing our Delegate instance). It's easy, but I think it doesn't have the best performance.
IDynamicObject
A slightly more complex implementation would instead implement IDynamicObject. This what IronRuby's Proc class does, and it's what I currently have working on my own language implementation. The core of the IDynamicObject idea is that, unlike with the special Call() method above, our function object is no longer responsible for actually doing the call on the target function. Instead, we're merely responsible for providing the DLR with a rule it can use to invoke it. This is exactly what you do inside your GetRule<T>() implementation.
In our case, what we really want is to respond with a new rule for the Call dynamic action, which is the only one we're interested in it. My current (simplistic) implementation works like this:
public StandardRule<T> GetRule<T>(DynamicAction action, CodeContext context, object[] args) {
switch ( action.Kind ) {
case DynamicActionKind.Call:
StandardRule<T> result = new StandardRule<T>();
ActionBinder binder = LanguageContext.Binder;
SetCallRule(result, binder, result.Parameters);
return result;
default:
return null;
}
}
private void SetCallRule(StandardRule result, ActionBinder binder, IList<Tree.Expression> args) {
// args[0] == this
result.AddTest(
Tree.Ast.Equal(
args[0],
Tree.Ast.Constant(this)
));
Tree.Expression[] expr = new Tree.Expression[args.Count-1];
for ( int i = 0; i < expr.Length; i++ )
expr[i] = args[i+1];
result.Target = result.MakeReturn(binder, Tree.Ast.ComplexCallHelper(Target, expr));
}
As you can see, we create a new StandardRule but a very simple one:
- The test part of the rule simply checks that the target object on which to invoke the call is the same as our instance. This is what tells the DLR if the call should or should not be made. Here's I'm simply doing an identity test, which seems to work fine in my unit tests. IronRuby, for example, instead uses a unique ID assigned to each Proc instance for the call test.
- The second part will create a new array with the actual arguments to the call (i.e. without the first 'this' argument) and use that to build the call using Ast.ComplexCallHelper().
One we return the rule, the DLR can cache that information to make following calls more efficient, for example.
Something to watch out for here and that really confused me at first: If you read the above code carefully, you'll notice that the DLR Actually gives us an object[] to GetRule<T>(). This will contain the actual values of arguments to the call (not expressions that evaluate to them), so you can examine them to decide how to best to create your rule. Notice however that I don't actually use them. Instead I use the already existing expressions in StandardRule<T>.Parameters, which will be already populated with the expressions for your call arguments.
The reason for this is that, as it turns out, is that if you used the original values to build your own Expressions to create the test (and possibly parameters) for the call, you will run into an issue that can cause the DLR to go into an infinite recursion of nested DynamicSite.Invoke() calls. Not fun to debug, and looking at the rules or AST dumps won't tell you why it's failing. I think the relevant issue is mentioned in a comment in the DLR source code that reads "The test should be such that it does not become invalid immediately. Otherwise, ActionBinder.UpdateSiteAndExecute can potentially loop infinitely."
Anyway, once you realize how Ast.Action.Call() works and what it expects from you, it all starts making a lot more sense.
ActionBinders
It is also important to note that already in these simple scenarios your language's ActionBinder implementation starts to kick in. In particular, you need to ensure that it implements the required type conversions so that the DLR can make any conversions necessary between the actual argument values to a call and the types of the parameters as declared in the CodeBlock (through ActionBinder.ConvertExpression()).
Next time I'll talk a bit about InvokeMember actions.