Closures have been around a long time yet have experienced a resurgence in the last decade. They were first implemented in the functional language Scheme then later introduced to object-oriented programming in Smalltalk. The ideas behind closures can be found in Java anonymous classes, C++ lambdas, Python nested functions, Ruby procs, Eiffel agents, and more. Closures are now supported in one form or another by many programming languages (C is a notable exception) but remain somewhat mysterious and are often not fully understood. Closures have major implications for how we program and also require significant effort behind the scenes to properly support them. Like any fundamental concept, it’s important for you, as a programmer, to know how they work.
In this article I will explain some basics. First I will work through a simple example, in case you are new to closures. Then I will show how the same semantics can be implemented without using closures.
A basic example
A closure is basically just a nested function which accesses variables from its parent scope. To demonstrate this we’ll start with a very simple example of a closure. I’ve used a pseudo-syntax based on my Cloverleaf project since I think it conveys the intent clearly. Look at the code first and then I will explain it a bit more. Don’t worry if you are new to closures and don’t quite understand. You may choose to skip ahead to how they are implemented and then come back to this section; understanding the mechanics actually makes matters clearer.
[sourcecode]
typedef int_function = () -> ( : integer );
function create_closure = ( x : integer ) -> ( : int_function )
{
function square = () -> ( : integer )
{
return x*x;
};
return square;
};
var close_foo = create_closure(4);
var close_bar = create_closure(8);
print( close_foo() ); //prints 16
print( close_bar() ); //prints 64
[/sourcecode]
Now let’s go over some of this code:
[sourcecode]
typedef int_function = () -> ( : integer );
[/sourcecode]
The typedef ‘int_function’ defines a function type taking no parameters and returning an integer. I am just using the typedef here to make the next line in the code clearer.
[sourcecode]
function create_closure = ( x : integer ) -> ( : int_function)
[/sourcecode]
You may not be familiar with the ‘=’ assignment being used to assign a block of code to a function name. We are not calling the ‘create_closure’ function at this point. First we create an anonymous function block and assign it to the variable ‘create_closure’. We can then use this variable to call the function. This is a concept known as first-class functions: functions themselves are treated like any other variable and can be assigned, returned, and passed as parameters.
[sourcecode]
var close_foo = create_closure(4);
[/sourcecode]
Here we actually call ‘create_closure’ which returns a so-called “closure”. Let’s take another look at the body of the ‘create_closure’ function:
[sourcecode]
function square = () -> ( : integer )
{
return x*x;
};
return square;
[/sourcecode]
Inside the ‘create_closure’ function, we define another function, ‘square’. Furthermore this inner function uses the parameter ‘x’ passed into its “parent”. What happens when ‘create_closure’ returns? Even though the variable ‘x’ goes out of scope, its value at the time ‘create_closure’ is called is captured behind the scenes along with the ‘square’ function itself. That’s what makes ‘square’ a closure and not just a straightforward reference to a function.
[sourcecode]
print( close_foo() );
[/sourcecode]
Because the closure that is returned from ‘create_closure’ keeps track of its “lexical scope” (the value of ‘x’ in this case), the call to ‘close_foo’ will print the value 16 (and calling ‘close_bar’ will print 64).
What really happens
To understand how this works I’ll show the same code written in an object-oriented pseudo code. Basically we will build a construct called a function object. It comprises an interface, an implementation, and a factory function (‘create_closure’).
[sourcecode]
interface int_function
{
int call();
}
class square implements int_function
{
int value;
int call()
{
return value*value;
}
}
int_function create_closure( int value )
{
square c = new square();
c.value = value;
return c;
}
int_function close_foo = create_closure(4);
int_function close_bar = create_closure(8);
print( close_foo.call() );
print( close_bar.call() );
[/sourcecode]
When written this way we can see that a closure is quite simple. Instead of defining an inner function, we instantiate an object, ‘square’. To mimic the closure’s ability to capture ‘value’ from the outer function’s scope, we copy it explicitly into the object. Finally, we return this object rather than the inner function in the previous example.
Using an interface is an important part of the concept here. The caller to ‘create_closure’ doesn’t care what instance it gets, so long as it obeys the ‘int_func’ interface. ‘create_closure’ could return any class, or a different class to each caller. In the previous version of the code, we actually only see this abstract type: the concrete function is anonymous and private to the ‘create_closure’ function.
By value or reference
In the object-oriented code above it is clear we have “captured” the variable ‘value’ by value. That is, we’ve made a separate copy of it. Another option is to capture it by reference. That is, what if our object, standing in for a closure, took a reference instead of copying? The example we’ve used so far is a bit too simple to show the difference so let’s create a new one (using a bit of C++ notation for the references).
[sourcecode]
class incrementer implements int_func //int_func remains the same as in our previous example
{
int & counter; //a C++ style reference
incrementer( int & start ) //constructor initializes counter
: counter(start)
{ }
int call()
{
return ++counter;
}
}
int_func create_incrementer( int & start_count)
{
incrementer c( start_count );
return c;
}
int x = 10;
int_func foo_incrementer = create_incrementer(x);
int_func bar_incrementer = create_incrementer(x);
print( foo_incrementer.call() ); //prints 11
print( bar_incrementer.call() ); //prints 12
print( foo_incrementer.call() ); //prints 13
[/sourcecode]
This example is similar to the one before it. The key difference is that instead of making a copy, we use a reference. While we have created two instances of the ‘incrementer’ class, they both refer to the same variable ‘x’. That’s why the sequence of calls results in the sequence 11,12,13 rather than 11,11,12 (which would happen if a copy of ‘x’ was taken).
When the designer of a language chooses to implement closures, they must decide how to handle this question of handling them by reference or by value. It varies greatly among current languages.
In C++, when creating a lambda (that is, a closure), you the programmer can specify whether variables from an outer scope will be copied directly (by value) or pointed to indirectly (by reference). However, as C++ requires manual memory control, you have to be careful when using references: after calling a function that returns a lambda, those variables captured by reference could be out of scope!
In JavaScript, at a logical level, you actually have a scope chain. The entire containing scope is kept, which is essentially the same as capturing all variables by reference.
In a functional language like Haskell, where most data is immutable, there is no distinction between “by value” or “by reference”. Since an immutable never changes, whatever value it has when the closure is created will be retained at the time it is executed.
In Java, anonymous inner classes must have all parameters to functions passed in with the ‘final’ keyword. This basically insures that you won’t manipulate those variables directly. It’s a bit confusing as to whether this is by value or reference; since objects are treated implicitly like pointers, copying a value is actually taking a reference.
Closing off
The examples I’ve provided here are quite simple. They don’t get into any complex data relations, but rest assured the same basic implementation can be used for all closures.
What isn’t shown is the memory management aspect. Finding a system which keeps closures safe, yet efficient, is an ongoing field of investigation. There are also many ways to optimize closures, some of which can completely erase the closure in the generated executable. What a compiler produces, or VM executes, could vary significantly from my example.
The concept however remains the same: a closure is an implicit creation of a data object attached to a function.
