Manipulating the Lexical Environment

The largest class of special operators contains the operators that manipulate and access the lexical environment. **LET** and **LET***, which I’ve already discussed, are examples of special operators that manipulate the lexical environment since they can introduce new lexical bindings for variables. Any construct, such as a **DO** or **DOTIMES**, that binds lexical variables will have to expand into a **LET** or **LET***.2 The **SETQ** special operator is one that accesses the lexical environment since it can be used to set variables whose bindings were created by **LET** and **LET***.

Variables, however, aren’t the only thing that can be named within a lexical scope. While most functions are defined globally with **DEFUN**, it’s also possible to create local functions with the special operators **FLET** and **LABELS**, local macros with **MACROLET**, and a special kind of macro, called a symbol macro, with **SYMBOL-MACROLET**.

Much like **LET** allows you to introduce a lexical variable whose scope is the body of the **LET**, **FLET** and **LABELS** let you define a function that can be referred to only within the scope of the **FLET** or **LABELS** form. These special operators are handy when you need a local function that’s a bit too complex to define inline as a **LAMBDA** expression or that you need to use more than once. Both have the same basic form, which looks like this:

(flet (function-definition*)
  body-form*)

and like this:

(labels (function-definition*)
  body-form*)

where each function-definition has the following form:

(name (parameter*) form*)

The difference between **FLET** and **LABELS** is that the names of the functions defined with **FLET** can be used only in the body of the **FLET**, while the names introduced by **LABELS** can be used immediately, including in the bodies of the functions defined by the **LABELS**. Thus, **LABELS** can define recursive functions, while **FLET** can’t. It might seem limiting that **FLET** can’t be used to define recursive functions, but Common Lisp provides both **FLET** and **LABELS** because sometimes it’s useful to be able to write local functions that can call another function of the same name, either a globally defined function or a local function from an enclosing scope.

Within the body of a **FLET** or **LABELS**, you can use the names of the functions defined just like any other function, including with the **FUNCTION** special operator. Since you can use **FUNCTION** to get the function object representing a function defined with **FLET** or **LABELS**, and since a **FLET** or **LABELS** can be in the scope of other binding forms such as **LET**s, these functions can be closures.

Because the local functions can refer to variables from the enclosing scope, they can often be written to take fewer parameters than the equivalent helper functions. This is particularly handy when you need to pass a function that takes a single argument as a functional parameter. For example, in the following function, which you’ll see again in Chapter 25, the **FLET**ed function, count-version, takes a single argument, as required by walk-directory, but can also use the variable versions, introduced by the enclosing **LET**:

(defun count-versions (dir)
  (let ((versions (mapcar #'(lambda (x) (cons x 0)) '(2 3 4))))
    (flet ((count-version (file)
             (incf (cdr (assoc (major-version (read-id3 file)) versions)))))
      (walk-directory dir #'count-version :test #'mp3-p))
    versions))

This function could also be written using an anonymous function in the place of the **FLET**ed count-version, but giving the function a meaningful name makes it a bit easier to read.

And when a helper function needs to recurse, an anonymous function just won’t do.3 When you don’t want to define a recursive helper function as a global function, you can use **LABELS**. For example, the following function, collect-leaves, uses the recursive helper function walk to walk a tree and gather all the atoms in the tree into a list, which collect-leaves then returns (after reversing it):

(defun collect-leaves (tree)
  (let ((leaves ()))
    (labels ((walk (tree)
               (cond
                 ((null tree))
                 ((atom tree) (push tree leaves))
                 (t (walk (car tree))
                    (walk (cdr tree))))))
      (walk tree))
    (nreverse leaves)))

Notice again how, within the walk function, you can refer to the variable, leaves, introduced by the enclosing **LET**.

**FLET** and **LABELS** are also useful operations to use in macro expansions—a macro can expand into code that contains a **FLET** or **LABELS** to create functions that can be used within the body of the macro. This technique can be used either to introduce functions that the user of the macro will call or simply as a way of organizing the code generated by the macro. This, for instance, is how a function such as **CALL-NEXT-METHOD**, which can be used only within a method definition, might be defined.

A near relative to **FLET** and **LABELS** is the special operator **MACROLET**, which you can use to define local macros. Local macros work just like global macros defined with **DEFMACRO** except without cluttering the global namespace. When a **MACROLET** form is evaluated, the body forms are evaluated with the local macro definitions in effect and possibly shadowing global function and macro definitions or local definitions from enclosing forms. Like **FLET** and **LABELS**, **MACROLET** can be used directly, but it’s also a handy target for macro-generated code—by wrapping some user-supplied code in a **MACROLET**, a macro can provide constructs that can be used only within that code or can shadow a globally defined macro. You’ll see an example of this latter use of **MACROLET** in Chapter 31.

Finally, one last macro-defining special operator is **SYMBOL-MACROLET**, which defines a special kind of macro called, appropriately enough, a symbol macro. Symbol macros are like regular macros except they can’t take arguments and are referred to with a plain symbol rather than a list form. In other words, after you’ve defined a symbol macro with a particular name, any use of that symbol in a value position will be expanded and the resulting form evaluated in its place. This is how macros such as **WITH-SLOTS** and **WITH-ACCESSORS** are able to define “variables” that access the state of a particular object under the covers. For instance, the following **WITH-SLOTS** form:

(with-slots (x y z) foo (list x y z)))

might expand into this code that uses **SYMBOL-MACROLET**:

(let ((#:g149 foo))
  (symbol-macrolet
      ((x (slot-value #:g149 'x))
       (y (slot-value #:g149 'y))
       (z (slot-value #:g149 'z)))
    (list x y z)))

When the expression (list x y z) is evaluated, the symbols x, y, and z will be replaced with their expansions, such as (slot-value #:g149 'x).4

Symbol macros are most often local, defined with **SYMBOL-MACROLET**, but Common Lisp also provides a macro **DEFINE-SYMBOL-MACRO** that defines a global symbol macro. A symbol macro defined with **SYMBOL-MACROLET** shadows other symbol macros of the same name defined with **DEFINE-SYMBOL-MACRO** or enclosing **SYMBOL-MACROLET** forms.