Functions As Data, a.k.a. Higher-Order Functions

While the main way you use functions is to call them by name, a number of situations exist where it’s useful to be able treat functions as data. For instance, if you can pass one function as an argument to another, you can write a general-purpose sorting function while allowing the caller to provide a function that’s responsible for comparing any two elements. Then the same underlying algorithm can be used with many different comparison functions. Similarly, callbacks and hooks depend on being able to store references to code in order to run it later. Since functions are already the standard way to abstract bits of code, it makes sense to allow functions to be treated as data.9

In Lisp, functions are just another kind of object. When you define a function with **DEFUN**, you’re really doing two things: creating a new function object and giving it a name. It’s also possible, as you saw in Chapter 3, to use **LAMBDA** expressions to create a function without giving it a name. The actual representation of a function object, whether named or anonymous, is opaque—in a native-compiling Lisp, it probably consists mostly of machine code. The only things you need to know are how to get hold of it and how to invoke it once you’ve got it.

The special operator **FUNCTION** provides the mechanism for getting at a function object. It takes a single argument and returns the function with that name. The name isn’t quoted. Thus, if you’ve defined a function foo, like so:

CL-USER> (defun foo (x) (* 2 x))
FOO

you can get the function object like this:10

CL-USER> (function foo)
#<Interpreted Function FOO>

In fact, you’ve already used **FUNCTION**, but it was in disguise. The syntax #', which you used in Chapter 3, is syntactic sugar for **FUNCTION**, just the way ' is syntactic sugar for **QUOTE**.11 Thus, you can also get the function object for foo like this:

CL-USER> #'foo
#<Interpreted Function FOO>

Once you’ve got the function object, there’s really only one thing you can do with it—invoke it. Common Lisp provides two functions for invoking a function through a function object: **FUNCALL** and **APPLY**.12 They differ only in how they obtain the arguments to pass to the function.

**FUNCALL** is the one to use when you know the number of arguments you’re going to pass to the function at the time you write the code. The first argument to **FUNCALL** is the function object to be invoked, and the rest of the arguments are passed onto that function. Thus, the following two expressions are equivalent:

(foo 1 2 3) === (funcall #'foo 1 2 3)

However, there’s little point in using **FUNCALL** to call a function whose name you know when you write the code. In fact, the previous two expressions will quite likely compile to exactly the same machine instructions.

The following function demonstrates a more apt use of **FUNCALL**. It accepts a function object as an argument and plots a simple ASCII-art histogram of the values returned by the argument function when it’s invoked on the values from min to max, stepping by step.

(defun plot (fn min max step)
  (loop for i from min to max by step do
        (loop repeat (funcall fn i) do (format t "*"))
        (format t "~%")))

The **FUNCALL** expression computes the value of the function for each value of i. The inner **LOOP** uses that computed value to determine how many times to print an asterisk to standard output.

Note that you don’t use **FUNCTION** or #' to get the function value of fn; you want it to be interpreted as a variable because it’s the variable’s value that will be the function object. You can call plot with any function that takes a single numeric argument, such as the built-in function **EXP** that returns the value of e raised to the power of its argument.

CL-USER> (plot #'exp 0 4 1/2)
*
*
**
****
*******
************
********************
*********************************
******************************************************
NIL

**FUNCALL**, however, doesn’t do you any good when the argument list is known only at runtime. For instance, to stick with the plot function for another moment, suppose you’ve obtained a list containing a function object, a minimum and maximum value, and a step value. In other words, the list contains the values you want to pass as arguments to plot. Suppose this list is in the variable plot-data. You could invoke plot on the values in that list like this:

(plot (first plot-data) (second plot-data) (third plot-data) (fourth plot-data))

This works fine, but it’s pretty annoying to have to explicitly unpack the arguments just so you can pass them to plot.

That’s where **APPLY** comes in. Like **FUNCALL**, the first argument to **APPLY** is a function object. But after the function object, instead of individual arguments, it expects a list. It then applies the function to the values in the list. This allows you to write the following instead:

(apply #'plot plot-data)

As a further convenience, **APPLY** can also accept “loose” arguments as long as the last argument is a list. Thus, if plot-data contained just the min, max, and step values, you could still use **APPLY** like this to plot the **EXP** function over that range:

(apply #'plot #'exp plot-data)

**APPLY** doesn’t care about whether the function being applied takes **&optional**, **&rest**, or **&key** arguments—the argument list produced by combining any loose arguments with the final list must be a legal argument list for the function with enough arguments for all the required parameters and only appropriate keyword parameters.