Listing a Directory

You can implement the function for listing a single directory, list-directory, as a thin wrapper around the standard function **DIRECTORY**. **DIRECTORY** takes a special kind of pathname, called a wild pathname, that has one or more components containing the special value :wild and returns a list of pathnames representing files in the file system that match the wild pathname.2 The matching algorithm—like most things having to do with the interaction between Lisp and a particular file system—isn’t defined by the language standard, but most implementations on Unix and Windows follow the same basic scheme.

The **DIRECTORY** function has two problems that you need to address with list-directory. The main one is that certain aspects of its behavior differ fairly significantly between different Common Lisp implementations, even on the same operating system. The other is that while **DIRECTORY** provides a powerful interface for listing files, to use it properly requires understanding some rather subtle points about the pathname abstraction. Between these subtleties and the idiosyncrasies of different implementations, actually writing portable code that uses **DIRECTORY** to do something as simple as listing all the files and subdirectories in a single directory can be a frustrating experience. You can deal with those subtleties and idiosyncrasies once and for all, by writing list-directory, and forget them thereafter.

One subtlety I discussed in Chapter 14 is the two ways to represent the name of a directory as a pathname: directory form and file form.

To get **DIRECTORY** to return a list of files in /home/peter/, you need to pass it a wild pathname whose directory component is the directory you want to list and whose name and type components are :wild. Thus, to get a listing of the files in /home/peter/, it might seem you could write this:

(directory (make-pathname :name :wild :type :wild :defaults home-dir))

where home-dir is a pathname representing /home/peter/. This would work if home-dir were in directory form. But if it were in file form—for example, if it had been created by parsing the namestring "/home/peter"--then that same expression would list all the files in /home since the name component "peter" would be replaced with :wild.

To avoid having to worry about explicitly converting between representations, you can define list-directory to accept a nonwild pathname in either form, which it will then convert to the appropriate wild pathname.

To help with this, you should define a few helper functions. One, component-present-p, will test whether a given component of a pathname is “present,” meaning neither **NIL** nor the special value :unspecific.3 Another, directory-pathname-p, tests whether a pathname is already in directory form, and the third, pathname-as-directory, converts any pathname to a directory form pathname.

(defun component-present-p (value)
  (and value (not (eql value :unspecific))))
(defun directory-pathname-p  (p)
  (and
   (not (component-present-p (pathname-name p)))
   (not (component-present-p (pathname-type p)))
   p))
(defun pathname-as-directory (name)
  (let ((pathname (pathname name)))
    (when (wild-pathname-p pathname)
      (error "Can't reliably convert wild pathnames."))
    (if (not (directory-pathname-p name))
      (make-pathname
       :directory (append (or (pathname-directory pathname) (list :relative))
                          (list (file-namestring pathname)))
       :name      nil
       :type      nil
       :defaults pathname)
      pathname)))

Now it seems you could generate a wild pathname to pass to **DIRECTORY** by calling **MAKE-PATHNAME** with a directory form name returned by pathname-as-directory. Unfortunately, it’s not quite that simple, thanks to a quirk in CLISP’s implementation of **DIRECTORY**. In CLISP, **DIRECTORY** won’t return files with no extension unless the type component of the wildcard is **NIL** rather than :wild. So you can define a function, directory-wildcard, that takes a pathname in either directory or file form and returns a proper wildcard for the given implementation using read-time conditionalization to make a pathname with a :wild type component in all implementations except for CLISP and **NIL** in CLISP.

(defun directory-wildcard (dirname)
  (make-pathname
   :name :wild
   :type #-clisp :wild #+clisp nil
   :defaults (pathname-as-directory dirname)))

Note how each read-time conditional operates at the level of a single expression After #-clisp, the expression :wild is either read or skipped; likewise, after #+clisp, the **NIL** is read or skipped.

Now you can take a first crack at the list-directory function.

(defun list-directory (dirname)
  (when (wild-pathname-p dirname)
    (error "Can only list concrete directory names."))
  (directory (directory-wildcard dirname)))

As it stands, this function would work in SBCL, CMUCL, and LispWorks. Unfortunately, a couple more implementation differences remain to be smoothed over. One is that not all implementations will return subdirectories of the given directory. Allegro, SBCL, CMUCL, and LispWorks do. OpenMCL doesn’t by default but will if you pass **DIRECTORY** a true value via the implementation-specific keyword argument :directories. CLISP’s **DIRECTORY** returns subdirectories only when it’s passed a wildcard pathname with :wild as the last element of the directory component and **NIL** name and type components. In this case, it returns only subdirectories, so you’ll need to call **DIRECTORY** twice with different wildcards and combine the results.

Once you get all the implementations returning directories, you’ll discover they can also differ in whether they return the names of directories in directory or file form. You want list-directory to always return directory names in directory form so you can differentiate subdirectories from regular files based on just the name. Except for Allegro, all the implementations this library will support do that. Allegro, on the other hand, requires you to pass **DIRECTORY** the implementation-specific keyword argument :directories-are-files **NIL** to get it to return directories in file form.

Once you know how to make each implementation do what you want, actually writing list-directory is simply a matter of combining the different versions using read-time conditionals.

(defun list-directory (dirname)
  (when (wild-pathname-p dirname)
    (error "Can only list concrete directory names."))
  (let ((wildcard (directory-wildcard dirname)))
    #+(or sbcl cmu lispworks)
    (directory wildcard)
    #+openmcl
    (directory wildcard :directories t)
    #+allegro
    (directory wildcard :directories-are-files nil)
    #+clisp
    (nconc
     (directory wildcard)
     (directory (clisp-subdirectories-wildcard wildcard)))
    #-(or sbcl cmu lispworks openmcl allegro clisp)
    (error "list-directory not implemented")))

The function clisp-subdirectories-wildcard isn’t actually specific to CLISP, but since it isn’t needed by any other implementation, you can guard its definition with a read-time conditional. In this case, since the expression following the #+ is the whole **DEFUN**, the whole function definition will be included or not, depending on whether clisp is present in ***FEATURES***.

#+clisp
(defun clisp-subdirectories-wildcard (wildcard)
  (make-pathname
   :directory (append (pathname-directory wildcard) (list :wild))
   :name nil
   :type nil
   :defaults wildcard))