ASDF 2.26 in Quicklisp

[fare]

I am pleased to announce that Zach Beane (Xach) recently updated Quicklisp to use ASDF 2.26 (from October 2012).

At Xach's suggestion, I will describe the many changes made since the previous ASDF 2.014.6 (of April 2011) used by Quicklisp. Note however that since Quicklisp first tries to load an implementation-provided ASDF and only loads its own if none was found that is more recent, the improvement is only massive on those old implementations that haven't upgraded ASDF in a while. If you are using a recent version of a maintained implementation, a lot of these features were already made available to you while using Quicklisp.
Major Features

• The new inherited :around-compile attribute enables you to specify a function to be evaluated around the compilation of Lisp source (2.019). This way, you may notably bind special variables including the *package*, proclaim optimization settings, rename packages with package-renaming, enable λ instead of LAMBDA with lambda-reader, more generally tweak the reader with named-readtables or replace it completely with reader-interception, muffle warnings with asdf-condition-control, etc.
• The new :compile-check argument to the function called by the :around-compile hook, allows you to invalidate compilation of a file when it fails some invariant of yours (2.23). This is notably used by asdf-finalizers to ensure all finalizers were included in the fasl.
• The new inherited :encoding attribute lets you specify the character encoding of your Lisp files (2.21). Only :utf-8 is supported, and is recommended; as well as the default :default for backwards compatibility (i.e. let the implementation pick some implementation-dependent, environment-variable-dependent, configuration-dependent encoding). If your system :defsystem-depends-on (:asdf-encodings) you can specify whatever encodings your implementation supports, or let the encoding be autodetected, Emacs-style.

Minor Features

• A :force-not (sys1 sys2 ...) feature was added to complement the recently fixed :force (sys3 sys4 ...) feature. Based on it, a require-system function was implemented that avoids trying to reload existing systems (2.21). On implementations that support it, the require hook has been changed to use that instead of load-system (2.22).
• Classes asdf:cl-source-file.cl and asdf:cl-source-file.lsp are now provided for people using these common alternate file types (2.015).
• load-system now uses asdf:*load-system-operation* instead of directly using 'asdf:load-op so you may change the default, which is nice for users of asdf-bundle and/or poiu (2.24).

Semantic Improvements

• The source registry now eagerly populates a database of .asd files rather than querying the file system at each search (2.015). You may flush the cache with (asdf:initialize-source-registry) or more generally flush all configuration with (asdf:clear-configuration) (from 2.007) - we particularly recommend you do the latter before you dump an image.
• :defsystem-depends-on can now be used to define classes for the system and its modules and components, with keywords being accepted to denote same-named classes from package ASDF (2.016).
• Better integration with Quicklisp. More generally, the find-system protocol was both extended and made more robust (2.015, 2.016), allowing for systems that are not backed by files and eliminating some infinite loops.
• When a system is not found, we offer a restart to reload after re-initializing your source-registry (2.019).
• Self-Upgrade: ASDF is more robust when it upgrades itself (2.015 to 2.019). You must still, more than ever, upgrade ASDF as the first thing before you load any other system, if it is upgraded at all. See manual for details on self-upgrade. 2.27 will automate it some more, but that won't be effective until all implementations have upgraded to 2.27.
• Internal APIs: notable refactoring and simplification of many internals, making them easier to extend and fixing many bugs along the way (2.015 to 2.24). The guts of ASDF have been slowly but completely rewritten since ASDF 1.
• More robustness. For instance, survive invalid version strings (2.015), better support for pathnames in general and logical pathnames in particular (2.015 to 2.23), proper configuration directories on Windows (2.016 to 2.26), fix many bugs including very old ones from ASDF 1 days, and add more tests. Example old bugs include surviving incompatible redefinitions of an .asd file (2.019), surviving clock skews (2.018), having :default-component-class do something useful (2.22), fixing :weakly-depends-on (2.20), etc.
• Some updates to the documentation.

Portability

There have been plenty of implementation-specific fixes and improvements, for each and every single platform. They are too numerous to list; some are minor, some are major, all are meaningful. Note the new support for two implementations, MKCL and XCL. Also note that Cormanlisp, GCL, Genera and RMCL are dead: though support for every one of them has been significantly improved since 2.014.6, I am now unable to test any of them. The actively supported implementations are as follows: ABCL, Allegro, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, SBCL, Scieneer CL.
Not Improved

If you are looking for radical improvement on to how to build Common Lisp code, ASDF is the wrong place to look. For more declarative dependency declarations, reproducible builds, parallel or distributed builds, try XCVB. For integration of dependency-management with the package system, try faslpath or the upcoming quick-build (ask drewc or nyef). For a Lisp that has a sensible module system and proper evaluation staging, try Racket (no, it won't run CL code, but you could make it if you really wanted).
Libraries

Related to ASDF, but not part of ASDF itself, here are some improved libraries.

• asdf-utils: ASDF includes a lot of utilities, notably to portably manage pathnames; these utilities are not exported from package ASDF anymore (2.25). If you're writing an ASDF extension, you might want to use them using the asdf:: prefix. But in general, we now recommend you should use the symbols exported by the new package asdf-utils.
• inferior-shell: We strongly disrecommend the use of asdf:run-shell-command: it's a non-portable crock copied over from mk-defsystem, that does the wrong thing on many levels. Instead you should use inferior-shell:run or the underlying xcvb-driver:run-program/.
• asdf-condition-control: Also using xcvb-driver underneath, you may want to use asdf-condition-control to control which warnings should be ignored during the build, and which should trigger conspicuous breakage.
• asdf-encodings: If you want to use latin1, koi8-r, euc-jp, or some other national encoding in your Lisp source files, you can, with asdf-encodings. We still recommend you use utf-8 everywhere.
• asdf-bundle: On supported implementations you can now want to deliver your code as a single fasl file, or one fasl file per system, using asdf-bundle, which was evolved from the now obsolete asdf-ecl.lisp. All active implementations are now supported except ABCL which has its own abcl-jar contrib. [NB: asdf-bundle may be merged into asdf in the near future.]
• asdf-finalizers: ever needed to evaluate toplevel code from a deftype or from deep within a lexical macro expander? Now you can do it correctly with asdf-finalizers and its eval-at-toplevel. Example use with a monomorphic list-of.
• poiu: compile in parallel, load serially in the current image; poiu can boost the compilation speed of your large systems. It has been updated to work well with the latest ASDF, and several bugs have been fixed.
• cl-launch: cl-launch enables you to invoke your Lisp code from the shell in a uniform way over all Lisp implementations. On supported implementations, you can dump precompiled images or even executable images. It was updated to work with ASDF 2.

Better integration with SLIME

Also related to ASDF, I recently rewrote SLIME's swank-asdf extension. If you grab a recent SLIME (late November 2012), it will do the Right Thing™ when you compile a .asd file. I recommend you add the following incantation to your ~/.swank.lisp, so it will also compile asdf-controlled Lisp files the same way as ASDF:
(in-package :swank) (pushnew 'try-compile-file-with-asdf *compile-file-for-emacs-hook*) Using this hook, SLIME will heed your output-translations, which avoids pollution of current directory with fasls; but more importantly, it will use the same :encoding as ASDF and its :around-compile hooks, which are essential to the proper compilation of components that depend on them. While I'm at it, I recommend you also use these in your ~/.swank.lisp, so you always use the latest installed ASDF:
(require "asdf") (asdf:load-system :asdf) Indeed, the only proper time to upgrade ASDF is early during start up, so if you have a version that's later than what your implementation provides, the right time to upgrade ASDF is just after loading it; if there is no installed ASDF source code from which to upgrade, the load-system won't do anything bad (indeed, it won't do anything at all). If you want to use an old or unmaintained implementation that doesn't provide ASDF 2, you'll have to replace the above require by more complex code that you may copy and adapt from slime/contrib/swank-asdf.lisp.

If you, like me, believe slime-asdf should do all these things by default, please gently tell your SLIME hackers.