I've been looking forward to this. I hope you keep improving it, especially the documentation.
ASDF was one of those things that, as a Lisp programmer I use a lot, but only because I have to. It's under-documented, often confusing, occasionally unreliable, and has lousy error messages. In desperation, I even tried out mk-defsystem, only to find out that it's even worse.
If XCVB can (despite its even sillier name) manage to make packages in Common Lisp not suck, you will be my hero.
The name XCVB came in a 15 minute session with Spencer Brody when I had to give a code-name for the project in a hurry. I later backronymed it, and the name stuck.
XCVB isn't ready for primetime yet, but I hope that as I build up a "plausible promise" I will attract a community to enhance it.
If you don't want to collaborate on writing software with other people, XCVB will hardly force you to do it. Stay in your hole, nobody cares. Someday, you may understand the concept of "division of labor". In the meantime, why don't you build computers out of parts you made yourself out of raw materials you extracted or grew yourself?
As for the rest of us, we understand that individuals benefit from collaborating with each other, and that good collaboration tools (as opposed to bureaucracy) are enablers that allow each one to do what he could never have dreamt to do all by himself.
Note that in the Lisp family, the PLT Scheme community has no problem providing a separate compilation module system that offers all that XCVB promises and much more (except that they don't handle distribution and caching at this point). People casually develop new languages and dialects that integrate in the PLT virtual machine, and can mix and match between modules created in many such languages.
"that old oxymoron, "team productivity" (teams, by their very nature, can never run at maximum individual efficiency - they need the bureaucracy precisely BECAUSE they're made up of individuals"
It's no oxymoron. Teams have productivity. Just because each piece of a system isn't running at perfect efficiency doesn't mean it's meaningless to measure the whole.
"If collectivism is truly the way of the future, if there is no longer any place for the lone wolf programmer in this world"
Using Lisp doesn't preclude using libraries. In fact, they're far more powerful than libraries in other languages, since Lisp libraries can add syntax (like ITERATE).
If you don't want ad-hoc collaboration with other Lisp programmers, why stop with libraries? Drop all of Common Lisp, too -- no need for standard functions like REMOVE-IF-NOT, when each "lone wolf" can just implement it herself. We'll give everybody McCarthy's Lisp 1 just to spite Java PHBs!
any extension you use makes such side-effects exponentially more likely to confuse other people and programs, to the point that if you want your code to be reused by other people, you must refrain from using such extensions altogether, and stick to the base languageI find this part of XCVB's motivation rather weak. it doesn't seem to me that the main reason for people's reluctance to use language extensions in their libraries is that those extensions side-effect the Lisp image. it rather seems to me that language extensions are not widely used mostly because straying from the base language means fewer people would consider working on your code due to the increased cognitive load - and if the code is not for public consumption in the first place, then "autism" is not a problem at all
( ... )
I mustn't have been clear because that was exactly my point!
Those who want to write libraries reused by other people have to refrain from modifying the language too much, because their modifications diminish the amount of people who'll contribute to the library (if you depend on non-standard macros) or worse, those people who'll use it at all (if you depend on non-standard reader-macros).
Those who don't care for other people reusing their software don't have other people reusing their software, anyway.
(As for why hygiene matters -- once again, if you refrain yourself from building ambitious higher-order macro extensions that other people build macro extensions on top of, packages are good enough indeed. Once again, limitations in the language lead to limitations in the ambition of programmers.)
Those who want to write libraries reused by other people have to refrain from modifying the language too much, because their modifications diminish the amount of people who'll contribute to the library (if you depend on non-standard macros) or worse, those people who'll use it at all (if you depend on non-standard reader-macros).
indeed. but the reasons here are more social (having to do with the cognitive load introduced by the modified language) than technical, and I don't see how XCVB or any other piece of language infrastructure is going to solve that.
My point is precisely that technical and social aspects are deeply linked in ways that are obvious when you think of it, yet are ignored by most people who usually see only one aspect and don't understand the other.
XCVB is solving the problem by shielding modules from compile-time side-effects required by other modules by compiling each module in its own Lisp image, therefore allowing one to not have to know anything about how other modules are built, only about the interfaces he cares for.
By contrast, ASDF and other variants of defsystem, with their model of non-deterministically side-effecting a shared Lisp image, force the programmer to master the exponentially complex interactions between all the compile-time side-effects of each and every module that is part of the system.
Hence XCVB reduces the cognitive load required for the combination of multiple modules, and will enable the delimited use of advanced metaprogramming that wouldn't be socially manageable with ASDF.
(bah, comment editing seems to be a paid feature. oh well)
EDIT: I do understand why one needs compile-time/run-time side effect separation in order to support massive parallelized builds, say. but practically speaking you just need "cfasl" support for that, right?
CFASLs are a speed optimization. With enough discipline and at a speed cost, you could do everything with FASL.
What is *necessary* for parallelization of compilation is lack of compile-time side-effects across modules.
Note that in practice, incremental compilation with ASDF *requires*, but does not *enforce* such meta-level purity. Which leads to incremental compilation not working on big projects. At ITA, most everyone got used to compiling our huge projects from clean, because incremental build *may* unpredictably break, or worse, yield a false positive that you think passes the regression test suite but really won't if you commit it and other people build from clean.
Comments 14
Reply
ASDF was one of those things that, as a Lisp programmer I use a lot, but only because I have to. It's under-documented, often confusing, occasionally unreliable, and has lousy error messages. In desperation, I even tried out mk-defsystem, only to find out that it's even worse.
If XCVB can (despite its even sillier name) manage to make packages in Common Lisp not suck, you will be my hero.
Reply
XCVB isn't ready for primetime yet, but I hope that as I build up a "plausible promise" I will attract a community to enhance it.
Reply
Reply
As for the rest of us, we understand that individuals benefit from collaborating with each other, and that good collaboration tools (as opposed to bureaucracy) are enablers that allow each one to do what he could never have dreamt to do all by himself.
Note that in the Lisp family, the PLT Scheme community has no problem providing a separate compilation module system that offers all that XCVB promises and much more (except that they don't handle distribution and caching at this point). People casually develop new languages and dialects that integrate in the PLT virtual machine, and can mix and match between modules created in many such languages.
Reply
It's no oxymoron. Teams have productivity. Just because each piece of a system isn't running at perfect efficiency doesn't mean it's meaningless to measure the whole.
"If collectivism is truly the way of the future, if there is no longer any place for the lone wolf programmer in this world"
Using Lisp doesn't preclude using libraries. In fact, they're far more powerful than libraries in other languages, since Lisp libraries can add syntax (like ITERATE).
If you don't want ad-hoc collaboration with other Lisp programmers, why stop with libraries? Drop all of Common Lisp, too -- no need for standard functions like REMOVE-IF-NOT, when each "lone wolf" can just implement it herself. We'll give everybody McCarthy's Lisp 1 just to spite Java PHBs!
Reply
Reply
Those who want to write libraries reused by other people have to refrain from modifying the language too much, because their modifications diminish the amount of people who'll contribute to the library (if you depend on non-standard macros) or worse, those people who'll use it at all (if you depend on non-standard reader-macros).
Those who don't care for other people reusing their software don't have other people reusing their software, anyway.
(As for why hygiene matters -- once again, if you refrain yourself from building ambitious higher-order macro extensions that other people build macro extensions on top of, packages are good enough indeed. Once again, limitations in the language lead to limitations in the ambition of programmers.)
Reply
indeed. but the reasons here are more social (having to do with the cognitive load introduced by the modified language) than technical, and I don't see how XCVB or any other piece of language infrastructure is going to solve that.
Reply
XCVB is solving the problem by shielding modules from compile-time side-effects required by other modules by compiling each module in its own Lisp image, therefore allowing one to not have to know anything about how other modules are built, only about the interfaces he cares for.
By contrast, ASDF and other variants of defsystem, with their model of non-deterministically side-effecting a shared Lisp image, force the programmer to master the exponentially complex interactions between all the compile-time side-effects of each and every module that is part of the system.
Hence XCVB reduces the cognitive load required for the combination of multiple modules, and will enable the delimited use of advanced metaprogramming that wouldn't be socially manageable with ASDF.
Reply
EDIT: I do understand why one needs compile-time/run-time side effect separation in order to support massive parallelized builds, say. but practically speaking you just need "cfasl" support for that, right?
Reply
What is *necessary* for parallelization of compilation is lack of compile-time side-effects across modules.
Note that in practice, incremental compilation with ASDF *requires*, but does not *enforce* such meta-level purity. Which leads to incremental compilation not working on big projects. At ITA, most everyone got used to compiling our huge projects from clean, because incremental build *may* unpredictably break, or worse, yield a false positive that you think passes the regression test suite but really won't if you commit it and other people build from clean.
Reply
Leave a comment