Package Gotchas

Once you’re familiar with packages, you won’t spend a bunch of time thinking about them. There’s just not that much to them. However, a couple of gotchas that bite most new Lisp programmers make the package system seem more complicated and unfriendly than it really is.

The number-one gotcha arises most commonly when playing around at the REPL. You’ll be looking at some library that defines certain interesting functions. You’ll try to call one of the functions like this:

CL-USER> (foo)

and get dropped into the debugger with this error:

attempt to call `FOO' which is an undefined function.
   [Condition of type UNDEFINED-FUNCTION]
Restarts:
  0: [TRY-AGAIN] Try calling FOO again.
  1: [RETURN-VALUE] Return a value instead of calling FOO.
  2: [USE-VALUE] Try calling a function other than FOO.
  3: [STORE-VALUE] Setf the symbol-function of FOO and call it again.
  4: [ABORT] Abort handling SLIME request.
  5: [ABORT] Abort entirely from this (lisp) process.

Ah, of course—you forgot to use the library’s package. So you quit the debugger and try to **USE-PACKAGE** the library’s package in order to get access to the name FOO so you can call the function.

CL-USER> (use-package :foolib)

But that drops you back into the debugger with this error message:

Using package `FOOLIB' results in name conflicts for these symbols: FOO
   [Condition of type PACKAGE-ERROR]
Restarts:
  0: [CONTINUE] Unintern the conflicting symbols from the `COMMON-LISP-USER' package.
  1: [ABORT] Abort handling SLIME request.
  2: [ABORT] Abort entirely from this (lisp) process.

Huh? The problem is the first time you called foo, the reader read the name foo and interned it in CL-USER before the evaluator got hold of it and discovered that this newly interned symbol isn’t the name of a function. This new symbol then conflicts with the symbol of the same name exported from the FOOLIB package. If you had remembered to **USE-PACKAGE** FOOLIB before you tried to call foo, the reader would have read foo as the inherited symbol and not interned a foo symbol in CL-USER.

However, all isn’t lost, because the first restart offered by the debugger will patch things up in just the right way: it will unintern the foo symbol from COMMON-LISP-USER, putting the CL-USER package back to the state it was in before you called foo, allowing the **USE-PACKAGE** to proceed and allowing for the inherited foo to be available in CL-USER.

This kind of problem can also occur when loading and compiling files. For instance, if you defined a package, MY-APP, for code that was going to use functions with names from the FOOLIB package, but forgot to :use FOOLIB, when you compile the files with an (in-package :my-app) in them, the reader will intern new symbols in MY-APP for the names that were supposed to be read as symbols from FOOLIB. When you try to run the compiled code, you’ll get undefined function errors. If you then try to redefine the MY-APP package to :use FOOLIB, you’ll get the conflicting symbols error. The solution is the same: select the restart to unintern the conflicting symbols from MY-APP. You’ll then need to recompile the code in the MY-APP package so it will refer to the inherited names.

The next gotcha is essentially the reverse of the first gotcha. In this case, you’d have defined a package—again, let’s say it’s MY-APP--that uses another package, say, FOOLIB. Now you start writing code in the MY-APP package. Although you used FOOLIB in order to be able to refer to the foo function, FOOLIB may export other symbols as well. If you use one of those exported symbols—say, bar--as the name of a function in your own code, Lisp won’t complain. Instead, the name of your function will be the symbol exported by FOOLIB, which will clobber the definition of bar from FOOLIB.

This gotcha is more insidious because it doesn’t cause an error—from the evaluator’s point of view it’s just being asked to associate a new function with an old name, something that’s perfectly legal. It’s suspect only because the code doing the redefining was read with a different value for ***PACKAGE*** than the name’s package. But the evaluator doesn’t necessarily know that. However, in most Lisps you’ll get an warning about “redefining BAR, originally defined in?”. You should heed those warnings. If you clobber a definition from a library, you can restore it by reloading the library code with **LOAD**.13

The last package-related gotcha is, by comparison, quite trivial, but it bites most Lisp programmers at least a few times: you define a package that uses COMMON-LISP and maybe a few libraries. Then at the REPL you change to that package to play around. Then you decide to quit Lisp altogether and try to call (quit). However, quit isn’t a name from the COMMON-LISP package—it’s defined by the implementation in some implementation-specific package that happens to be used by COMMON-LISP-USER. The solution is simple—change packages back to CL-USER to quit. Or use the SLIME REPL shortcut quit, which will also save you from having to remember that in certain Common Lisp implementations the function to quit is exit, not quit.

You’re almost done with your tour of Common Lisp. In the next chapter I’ll discuss the details of the extended **LOOP** macro. After that, the rest of the book is devoted to “practicals”: a spam filter, a library for parsing binary files, and various parts of a streaming MP3 server with a Web interface.

1The kind of programming that relies on a symbol data type is called, appropriately enough, symbolic computation. It’s typically contrasted to numeric programming. An example of a primarily symbolic program that all programmers should be familiar with is a compiler—it treats the text of a program as symbolic data and translates it into a new form.

2Every package has one official name and zero or more nicknames that can be used anywhere you need to use the package name, such as in package-qualified names or to refer to the package in a **DEFPACKAGE** or **IN-PACKAGE** form.

3COMMON-LISP-USER is also allowed to provide access to symbols exported by other implementation-defined packages. While this is intended as a convenience for the user—it makes implementation-specific functionality readily accessible—it can also cause confusion for new Lispers: Lisp will complain about an attempt to redefine some name that isn’t listed in the language standard. To see what packages COMMON-LISP-USER inherits symbols from in a particular implementation, evaluate this expression at the REPL:

(mapcar #'package-name (package-use-list :cl-user))

And to find out what package a symbol came from originally, evaluate this:

(package-name (symbol-package 'some-symbol))

with some-symbol replaced by the symbol in question. For instance:

(package-name (symbol-package 'car)) ==> "COMMON-LISP"
(package-name (symbol-package 'foo)) ==> "COMMON-LISP-USER"

Symbols inherited from implementation-defined packages will return some other value.

4This is different from the Java package system, which provides a namespace for classes but is also involved in Java’s access control mechanism. The non-Lisp language with a package system most like Common Lisp’s packages is Perl.

5All the manipulations performed by **DEFPACKAGE** can also be performed with functions that man- ipulate package objects. However, since a package generally needs to be fully defined before it can be used, those functions are rarely used. Also, **DEFPACKAGE** takes care of performing all the package manipulations in the right order—for instance, **DEFPACKAGE** adds symbols to the shadowing list before it tries to use the used packages.

6In many Lisp implementations the :use clause is optional if you want only to :use COMMON-LISP--if it’s omitted, the package will automatically inherit names from an implementation-defined list of packages that will usually include COMMON-LISP. However, your code will be more portable if you always explicitly specify the packages you want to :use. Those who are averse to typing can use the package’s nickname and write (:use :cl).

7Using keywords instead of strings has another advantage—Allegro provides a “modern mode” Lisp in which the reader does no case conversion of names and in which, instead of a **COMMON-LISP** package with uppercase names, provides a common-lisp package with lowercase names. Strictly speaking, this Lisp isn’t a conforming Common Lisp since all the names in the standard are defined to be uppercase. But if you write your **DEFPACKAGE** forms using keyword symbols, they will work both in Common Lisp and in this near relative.

8Some folks, instead of keywords, use uninterned symbols, using the #: syntax.

(defpackage #:com.gigamonkeys.email-db
  (:use #:common-lisp))

This saves a tiny bit of memory by not interning any symbols in the keyword package—the symbol can become garbage after **DEFPACKAGE** (or the code it expands into) is done with it. However, the difference is so slight that it really boils down to a matter of aesthetics.

9The reason to use **IN-PACKAGE** instead of just **SETF**ing ***PACKAGE*** is that **IN-PACKAGE** expands into code that will run when the file is compiled by **COMPILE-FILE** as well as when the file is loaded, changing the way the reader reads the rest of the file during compilation.

10In the REPL buffer in SLIME you can also change packages with a REPL shortcut. Type a comma, and then enter change-package at the Command: prompt.

11During development, if you try to :use a package that exports a symbol with the same name as a symbol already interned in the using package, Lisp will signal an error and typically offer you a restart that will unintern the offending symbol from the using package. For more on this, see the section “Package Gotchas.”

12The code for the “Practical” chapters, available from this book’s Web site, uses the ASDF system definition library. ASDF stands for Another System Definition Facility.

13Some Common Lisp implementations, such as Allegro and SBCL, provide a facility for “locking” the symbols in a particular package so they can be used in defining forms such as **DEFUN**, **DEFVAR**, and **DEFCLASS** only when their home package is the current package.