an atypical entry

[zanfur]

I've found that I have a hard time thinking in Microsoft. I can think in C, C++, Smalltalk, Java, Bourne, BASIC, Perl, LISP, and any kind of scripting you could care to name. But, hand me C#, VB, J, or any of the other language Microsoft has designed in the last ten years, and I just can't hack it. Well, I can, but it just seems ... the long way

Comments

[zanfur]

Well, although I agree that macros in general are not first class citizens of the programming environment, LISP is an exception: LISP's macro system is actually turing-complete, making it a full-blown programming language unto itself. Actually, the macro system isn't a separate language -- it's just more LISP -- but you can use all of LISP to generate the code. Except for the macro you're currently defining, of course. This is an obvious extension of LISP, because its code syntax and list syntax are identical, and the code therefore easy to programmatically generate, but there are other systems (such as Slate, a smalltalk derivative) that manage to have a turing-complete macro system as well. You still run into problems of badly written code, but it's a lot easier to avoid (and easier to catch) than in most languages.

Badly written programs will have badly written bugs, of course. Because LISP macros are literally programs unto themselves, badly written macros will have badly written bugs. But, there are ways (actually, it's standard practice) of making it impossible to have namespace clashes, using unique identifier generators. This gets rid of most of those "whoops I just clobbered a variable!" mistakes, and those "whoops I just evaluated that twice!" mistakes. Also, in LISP, because the code generators are just LISP programs ... if you're coding in LISP in the first place, you can figure out what the macros do. You can also inspect the various levels of macro expansion using the (macroexpand-1) function, which returns the result of the expansion. LISP is nice that way. (There's also a function that just expands everything recursively for you, giving you the final output.)

The entire class system of LISP is actually a collection of macros. The macro system of LISP is really quite slick, and amazing in its power, and even more amazing in its ease of use to get at that power. There are even "java" and "python" macros that do just what you think they'd do: let you write java or python code, inside of a LISP form, where the macro compiles that code to LISP. The only requirement is that the parentheses be matched, and you can make macros generate code from pretty much any input at all. That is definitely a first class citezen of the programming environment.

Code generation is rather impossible to delay until runtime, yeah. But, as LISP is run in a stateful environment (LISP "programs" are usually just images of a LISP environment, a small LISP-interpreting kernel that's been pre-loaded with lods of stuff), you can re-generate the code at any time. For instance, if I change a macro definition, that change doesn't "cascade" to every function that calls that macro -- that code is already generated and compiled. (I usually don't want it to, either, so this is a good thing.) But, if I do want to reload a function, or two, or all of them, I can just reload them, and they'll all use the current definition of the macro.

You do run into problems where two functions use two different definitions of the same macro. Usually, though, this is fixed by a simple reload of the affected source files, which redefines everything in a consistent manner (consistent with the contents and layout of the source file, that is). The really nice thing about this, though, is that after I make temporary/debugging changes to the image, I just close down the image and reload it, and it's clean. Usually, I just need to reload the source files, or just a few of them. That's much, much easier than tracking down where I've modified the source files, modify them back, and recompiling everything that depends on those changes.

Also, because LISP programs are (typically) run from an image that can evaluate LISP expressions, you can delay code generation until runtime: You can change any function at runtime, and you can even create new ones. Using lambdas (you can think of those as anonymous functions), you can programmatically generate new functions based upon user input, then call those functions.

[lumiere]

I've programmed in Chez Scheme and am familiar with hygenic macros and other nice properties of LISP-style macros.

Making macros Turing-complete, even making them expressed in the host programming language, does not make them first-class citizens of the programming language. It does make them far more powerful and far easier to write.

Here's another example of the second bullet point: static analysis. Your generic static analysis tool will operate on the fully-expanded syntax; how do results get mapped back onto the source syntax? There are workable approaches, but it's not a simple problem.