Delivering Applications

Another topic of practical importance, which I didn’t talk about elsewhere in the book, is how to deliver software written in Lisp. The main reason I neglected this topic is because there are many different ways to do it, and which one is best for you depends on what kind of software you need to deliver to what kind of user with what Common Lisp implementation. In this section I’ll give an overview of some of the different options.

If you’ve written code you want to share with fellow Lisp programmers, the most straightforward way to distribute it is as source code.8 You can distribute a simple library as a single source file, which programmers can **LOAD** into their Lisp image, possibly after compiling it with **COMPILE-FILE**.

More complex libraries or applications, broken up across multiple source files, pose an additional challenge—in order to load and compile the code, the files need to be loaded and compiled in the correct order. For instance, a file containing macro definitions must be loaded before you can compile files that use those macros. And a file containing **DEFPACKAGE** forms must be loaded before any files that use those packages can even be **READ**. Lispers call this the system definition problem and typically handle it with tools called system definition facilities or system definition utilities, which are somewhat analogous to build tools such as make or ant. As with make and ant, system definition tools allow you to specify the dependencies between different files and then take care of loading and compiling the files in the correct order while trying to do only work that’s necessary—recompiling only files that have changed, for example.

These days the most widely used system definition tool is ASDF, which stands for Another System Definition Facility.9 The basic idea behind ASDF is that you define systems in ASD files, and ASDF provides a number of operations on systems such as loading them or compiling them. A system can also be defined to depend on other systems, which will be loaded as necessary. For instance, the following shows the contents of html.asd, the ASD file for the FOO library from Chapters 31 and 32:

(defpackage :com.gigamonkeys.html-system (:use :asdf :cl))
(in-package :com.gigamonkeys.html-system)
(defsystem html
  :name "html"
  :author "Peter Seibel <peter@gigamonkeys.com>"
  :version "0.1"
  :maintainer "Peter Seibel <peter@gigamonkeys.com>"
  :license "BSD"
  :description "HTML and CSS generation from sexps."
  :long-description ""
  :components
  ((:file "packages")
   (:file "html" :depends-on ("packages"))
   (:file "css" :depends-on ("packages" "html")))
  :depends-on (:macro-utilities))

If you add a symbolic link to this file from a directory listed in asdf:*central-registry*,10 then you can type this:

(asdf:operate 'asdf:load-op :html)

to compile and load the files packages.lisp, html.lisp, and html-macros.lisp in the correct order after first making sure the :macro-utilities system has been compiled and loaded. For other examples of ASD files, you can look at this book’s source code—the code from each practical chapter is defined as a system with appropriate intersystem dependencies expressed in the ASD files.

Most free and open-source Common Lisp libraries you’ll find will come with an ASD file. Some will use other system definition tools such as the slightly older MK:DEFSYSTEM or even utilities devised by the library’s author, but the tide seems to be turning in the direction of ASDF.11

Of course, while ASDF makes it easy for Lispers to install Lisp libraries, it’s not much help if you want to package an application for an end user who doesn’t know or care about Lisp. If you’re delivering a pure end-user application, presumably you want to provide something the user can download, install, and run without having to know anything about Lisp. You can’t expect them to separately download and install a Lisp implementation. And you want them to be able to run your application just like any other application—by double-clicking an icon on Windows or OS X or by typing the name of the program at the command line on Unix.

However, unlike C programs, which can typically rely on certain shared libraries (DLLs on Windows) that make up the C “runtime” being present as part of the operating system, Lisp programs must include a Lisp runtime, that is, the same program you run when you start Lisp though perhaps with certain functionality not needed to run the application excised.

To further complicate matters, program isn’t really well defined in Lisp. As you’ve seen throughout this book, the process of developing software in Lisp is an incremental process that involves making changes to the set of definitions and data living in your Lisp image. The “program” is just a particular state of the image arrived at by loading the .lisp or .fasl files that contain code that creates the appropriate definitions and data. You could, then, distribute a Lisp application as a Lisp runtime plus a bunch of FASL files and an executable that starts the runtime, loads the FASLs, and somehow invokes the appropriate starting function. However, since actually loading the FASLs can take some time, especially if they have to do any computation to set up the state of the world, most Common Lisp implementations provide a way to dump an image--to save the state of a running Lisp to a file called an image file or sometimes a core. When a Lisp runtime starts, the first thing it does is load an image file, which it can do in much less time than it’d take to re-create the state by loading FASL files.

Normally the image file is a default image containing only the standard packages defined by the language and any extras provided by the implementation. But with most implementations, you have a way to specify a different image file. Thus, instead of packaging an app as a Lisp runtime plus a bunch of FASLs, you can package it as a Lisp runtime plus a single image file containing all the definitions and data that make up your application. Then all you need is a program that launches the Lisp runtime with the appropriate image file and invokes whatever function serves as the entry point to the application.

This is where things get implementation and operating-system dependent. Some Common Lisp implementations, in particular the commercial ones such as Allegro and LispWorks, provide tools for building such an executable. For instance, Allegro’s Enterprise Edition provides a function excl:generate-application that creates a directory containing the Lisp runtime as a shared library, an image file, and an executable that starts the runtime with the given image. Similarly, the LispWorks Professional Edition “delivery” mechanism allows you to build single-file executables of your programs. On Unix, with the various free and open-source implementations, you can do essentially the same thing except it’s probably easier to use a shell script to start everything.

And on OS X things are even better—since all applications on OS X are packaged as .app bundles, which are essentially directories with a certain structure, it’s not all that difficult to package all the parts of a Lisp application as a double-clickable .app bundle. Mikel Evins’s Bosco tool makes it easy to create .app bundles for applications running on OpenMCL.

Of course, another popular way to deliver applications these days is as server-side applications. This is a niche where Common Lisp can really excel—you can pick a combination of operating system and Common Lisp implementation that works well for you, and you don’t have to worry about packaging the application to be installed by an end user. And Common Lisp’s interactive debugging and development features make it possible to debug and upgrade a live server in ways that either just aren’t possible in a less dynamic language or would require you to build a lot of specific infrastructure.