Some ramblings on the importance of culture in tech, especially around choice of programming tools
[A friend asked why, if Lisp was so great, it never got a look-in when Ada was designed.]
My impression is that it’s above all else cultural.
There have long been multiple warring factions depending on deeply-felt beliefs about how computing should be done. EBDCIC versus ASCII, RISC vs CISC, C vs Pascal, etc. Now it’s mostly sorted inasmuch as we all use Unix-like OSes - the only important exception, Windows, is becoming more Unix-like - and other languages etc. are layered on top.
But it goes deeper than, e.g., C vs Pascal, or BASIC or Fortran or whatever. There is the imperative vs functional camp. Another is algebraic expressions versus non-algebraic: i.e. prefix or postfix (stack-oriented RPN), or something Other such as APL/I/J/A+; manual memory management versus automatic with GC; strongly versus weakly typed (and arguably sub-battles such as manifest versus inferred/duck typing, static vs dynamic, etc.)
Mostly, the wars settled on: imperative; algebraic (infix) notation; manual memory management for system-level code and for externally-distributed code (commercial or FOSS), and GC Pascal-style languages for a lot of internal corporate s/w development (Delphi, VB, etc.).
FP, non-algebraic notation and things like were thus sidelined for decades, but are now coming back layered on top of complex OSes written in C-like languages. This is an era of proliferation in dynamic, interpreted or JITTed languages used for specific niche tasks, running on top of umpteen layers of GP OS. Examples range across Javascript, Perl 6, Python, Julia, Clojure, Ruby and tons more.
Meanwhile, new safer members of the broader C family of compiled languages, such as Rust and Go, and stretching a point Swift, are getting attention for more performance-critical app programming.
All the camps have strong arguments. There are no single right or wrong answers. However, cultural pressure and uniformity mean that outside of certain niches, we have several large camps or groups. (Of course, individual people can belong to more than one, depending on job, hobby, whatever.)
C and its kin are one, associated with Unix and later Windows.
Pascal and its kin, notably Object Pascal, Delphi/FPC, another. Basic now means VB and that means .NET family languages, another family. Both have historically mainly been part of the MS camp but now reaching out, against some resistance, into Unix land.
Java forms a camp of its own, but there are sub-camps of non-Java-like languages running on the JVM - Clojure, Scala, etc.
Apple’s flavour of Unix forms another camp, comprising ObjC and Swift, having abandoned outreach efforts.
People working on the development of Unix itself tend to strongly favour C above all else, and like relatively simple, old-fashioned tools - ancient text editors, standalone compilers. This has influenced the FOSS Unix GUIs and their apps.
The commercial desktop app developers are more into IDEs and automation; these days this covers .NET and JVM camps, and spans all OSes, but the Pascal/VM camp are still somewhat linked to Windows.
The people doing niche stuff, for their own needs or their organisations, which might be distributed as source - which covers sysadmins, devops and so on - are more into scripting languages, where there’s terrific diversity.
Increasingly the in-house app devs are just using Java, be they desktop or server apps. Indeed “desktop” apps of this type might now often mean Java server apps generating a remote UI via web protocols and technologies.
Multiple camps and affiliations. Many of them disdain the others.
A summary of how I’m actually addressing your question:
But these ones are the dominant ones, AFAICS. So when a new “safe” “secure” language was being built, “weird” niche things like Lisp, Forth, or APL never had a chance of a look-in. So it came out looking a bit Pascal- and BASIC-like, as those are the ones on the safe, heavily-type-checked side of the fence.
A more general summary:
I am coming to think that there are cultural forces stronger than technical forces involved in language choice.
Some examples I suspect that have been powerful:
Lisp (and FP) are inherently complex to learn and to use and require exceptionally high intelligence in certain focussed forms. Some people perfectly able to be serviceable, productive coders in simple imperative languages find themselves unable to fathom these styles or methods of programming. Their response is resentment, and to blame the languages, not themselves. (Dunning Kruger is not a problem confined to those of low intelligence.)
This has resulted in the marginalisation of these technologies as the computing world became vastly more commoditised and widespread. Some people can’t handle them, and some of them end up in positions of influence, so teaching switched away from them and now students are taught in simpler, imperative languages. Result, there is a general perception that some of these niche tools are exotic, not generally applicable or important, just toys for academics. This isn’t actually true but it’s such a widespread belief that it is self-perpetuating.
This also applies to things like Haskell, ML/OCaml, APL, etc.
On the flip side: programming and IT are male-dominated industries, for no very good reason. This results in masculine patterns of behaviour having profound effects and influences.
So, for instance, languages in the Pascal family have safety as a priority and try to protect programmers from errors, possibly by not allowing them to write unsafe code. A typically masculine response to this is to resent the exertion of oppressive control.
Contrastingly, languages in the BCPL/C/C++ family give the programmer extensive control and require considerable discipline and care to write safe code. They allow programmers to make mistakes which safer languages would catch and prevent.
This has a flip side, though: the greater control potentially permits or offers theoretically higher performance.
This aligns with “manly” virtues of using powerful tools - the appeal of chainsaws, fast cars and motorcycles, big powerful engines, even arguably explicitly dangerous things like knives and guns. Cf. Perl, “the Swiss Army chainsaw”.
Thus, the masculine culture around IT has resulted in people favouring these languages. They’re dangerous in unskilled hands. So, get skilled, then you can access the power.
Of course, again, as Dunning Kruger teach us, people cannot assess their own skill, and languages which permit bugs that others would trap have been used very widely for 3 decades or more, often on the argument of performance but actually because of toxic culture. All OSes are written in them; now as a result it is a truism that only these languages are suitable for writing OSes.
(Ignoring the rich history of OSes in safer languages - Algol, Lisp, Oberon, perhaps even Mesa, or Pascal in the early Macs.)
If you want fast code, you need a fast language! And Real Men use C, and you want to be a Real Man, don’t you?
Cf. the story of Mel The Real Programmer.
Do it in something low-level, manage your own memory. Programming is a game for the smart, and you must be smart because you’re a programmer, so you can handle it and you won’t drop a pointer or overflow an array.
Result, decades of complex apps tackling arbitrary complex data - e.g. Web browsers, modern office suites - written in C, and decades of software patching and updating trying to catch the legions of bugs. This is now simply perceived as how software works, as normal.
Additionally, in many cases, any possible performance benefits have long been lost due to large amounts of protective code, of error-checking, in libraries and tools, made necessary by the problems and inherent fragility of the languages.
The rebellion against it is only in the form of niche line-of-business app developers doing narrow, specific stuff, who are moving to modern interpreted languages running on top of tens of million of lines of C written by coders who are only just able to operate at this level of competence and make lots of mistakes.
For people not facing the pressures of commercial releases, there was an era of using safer, more protective compiled languages for in-company apps - Turbo Pascal, Delphi, VB. But that’s fading away now in favour of Java and .NET, “managed” languages running under a VM, with concomitant loss of performance but slight improvement in safety and reliability.
And because this has been widespread for some 2-3 decades, it’s now just _how things are done_. So if someone presents evidence and accounts of vastly better programmer productivity in other tools, decades ago, in things like Lisp or Smalltalk, then these are discounted as irrelevant. Those are not manly languages for manly programmers and so should not be considered. They’re toys.
People in small enough niches continue to use them but have given up evangelising about them. Like Mac users, their comments are dismissed as fanboyism.
So relatively small cultural effects have created immensely strong cultures, dogmas, about what is or isn’t a good choice for certain categories of problem. People outside those categories continue to use some of these languages and tools, while others languish.
This is immensely sad.
For instance, there have been successful hybrid approaches.
OSes written in Pascal derivatives, or in Lisp, or in Smalltalk, now lost to history. As a result, processor design itself has shifted and companies make processors that run C and C-like languages efficiently, and processors that understood richer primitives - lists, or objects - are now historical footnotess.
And languages which attempted to straddle different worlds - such as infix-notation Lisp derivatives, readable and easily learnable by programmers who only know infix-based, imperative languages - e.g. Dylan, PLOT, or CGOL - are again forgotten.
Or languages which developed down different avenues, such as the families of languages based on or derived from Oberon, or APL, or ML. All very niche.
And huge amounts of precious programmer time and effort expended fighting against limited and limiting tools, not well suited to large complex projects, because they simply do not know that there are or were alternatives. These have been crudely airbrushed out, like disappearing Soviet commissars.
“And so successful was this venture that very soon Magrathea itself became the richest planet of all time, and the rest of the galaxy was reduced to abject poverty. And so the system broke down, the empire collapsed, and a long, sullen silence settled over the galaxy, disturbed only by the pen-scratchings of scholars as they laboured into the night over smug little treatises on the value of a planned political economy. In these enlightened days, of course, no one believes a word of it.”
(Douglas Adams)
My impression is that it’s above all else cultural.
There have long been multiple warring factions depending on deeply-felt beliefs about how computing should be done. EBDCIC versus ASCII, RISC vs CISC, C vs Pascal, etc. Now it’s mostly sorted inasmuch as we all use Unix-like OSes - the only important exception, Windows, is becoming more Unix-like - and other languages etc. are layered on top.
But it goes deeper than, e.g., C vs Pascal, or BASIC or Fortran or whatever. There is the imperative vs functional camp. Another is algebraic expressions versus non-algebraic: i.e. prefix or postfix (stack-oriented RPN), or something Other such as APL/I/J/A+; manual memory management versus automatic with GC; strongly versus weakly typed (and arguably sub-battles such as manifest versus inferred/duck typing, static vs dynamic, etc.)
Mostly, the wars settled on: imperative; algebraic (infix) notation; manual memory management for system-level code and for externally-distributed code (commercial or FOSS), and GC Pascal-style languages for a lot of internal corporate s/w development (Delphi, VB, etc.).
FP, non-algebraic notation and things like were thus sidelined for decades, but are now coming back layered on top of complex OSes written in C-like languages. This is an era of proliferation in dynamic, interpreted or JITTed languages used for specific niche tasks, running on top of umpteen layers of GP OS. Examples range across Javascript, Perl 6, Python, Julia, Clojure, Ruby and tons more.
Meanwhile, new safer members of the broader C family of compiled languages, such as Rust and Go, and stretching a point Swift, are getting attention for more performance-critical app programming.
All the camps have strong arguments. There are no single right or wrong answers. However, cultural pressure and uniformity mean that outside of certain niches, we have several large camps or groups. (Of course, individual people can belong to more than one, depending on job, hobby, whatever.)
C and its kin are one, associated with Unix and later Windows.
Pascal and its kin, notably Object Pascal, Delphi/FPC, another. Basic now means VB and that means .NET family languages, another family. Both have historically mainly been part of the MS camp but now reaching out, against some resistance, into Unix land.
Java forms a camp of its own, but there are sub-camps of non-Java-like languages running on the JVM - Clojure, Scala, etc.
Apple’s flavour of Unix forms another camp, comprising ObjC and Swift, having abandoned outreach efforts.
People working on the development of Unix itself tend to strongly favour C above all else, and like relatively simple, old-fashioned tools - ancient text editors, standalone compilers. This has influenced the FOSS Unix GUIs and their apps.
The commercial desktop app developers are more into IDEs and automation; these days this covers .NET and JVM camps, and spans all OSes, but the Pascal/VM camp are still somewhat linked to Windows.
The people doing niche stuff, for their own needs or their organisations, which might be distributed as source - which covers sysadmins, devops and so on - are more into scripting languages, where there’s terrific diversity.
Increasingly the in-house app devs are just using Java, be they desktop or server apps. Indeed “desktop” apps of this type might now often mean Java server apps generating a remote UI via web protocols and technologies.
Multiple camps and affiliations. Many of them disdain the others.
A summary of how I’m actually addressing your question:
But these ones are the dominant ones, AFAICS. So when a new “safe” “secure” language was being built, “weird” niche things like Lisp, Forth, or APL never had a chance of a look-in. So it came out looking a bit Pascal- and BASIC-like, as those are the ones on the safe, heavily-type-checked side of the fence.
A more general summary:
I am coming to think that there are cultural forces stronger than technical forces involved in language choice.
Some examples I suspect that have been powerful:
Lisp (and FP) are inherently complex to learn and to use and require exceptionally high intelligence in certain focussed forms. Some people perfectly able to be serviceable, productive coders in simple imperative languages find themselves unable to fathom these styles or methods of programming. Their response is resentment, and to blame the languages, not themselves. (Dunning Kruger is not a problem confined to those of low intelligence.)
This has resulted in the marginalisation of these technologies as the computing world became vastly more commoditised and widespread. Some people can’t handle them, and some of them end up in positions of influence, so teaching switched away from them and now students are taught in simpler, imperative languages. Result, there is a general perception that some of these niche tools are exotic, not generally applicable or important, just toys for academics. This isn’t actually true but it’s such a widespread belief that it is self-perpetuating.
This also applies to things like Haskell, ML/OCaml, APL, etc.
On the flip side: programming and IT are male-dominated industries, for no very good reason. This results in masculine patterns of behaviour having profound effects and influences.
So, for instance, languages in the Pascal family have safety as a priority and try to protect programmers from errors, possibly by not allowing them to write unsafe code. A typically masculine response to this is to resent the exertion of oppressive control.
Contrastingly, languages in the BCPL/C/C++ family give the programmer extensive control and require considerable discipline and care to write safe code. They allow programmers to make mistakes which safer languages would catch and prevent.
This has a flip side, though: the greater control potentially permits or offers theoretically higher performance.
This aligns with “manly” virtues of using powerful tools - the appeal of chainsaws, fast cars and motorcycles, big powerful engines, even arguably explicitly dangerous things like knives and guns. Cf. Perl, “the Swiss Army chainsaw”.
Thus, the masculine culture around IT has resulted in people favouring these languages. They’re dangerous in unskilled hands. So, get skilled, then you can access the power.
Of course, again, as Dunning Kruger teach us, people cannot assess their own skill, and languages which permit bugs that others would trap have been used very widely for 3 decades or more, often on the argument of performance but actually because of toxic culture. All OSes are written in them; now as a result it is a truism that only these languages are suitable for writing OSes.
(Ignoring the rich history of OSes in safer languages - Algol, Lisp, Oberon, perhaps even Mesa, or Pascal in the early Macs.)
If you want fast code, you need a fast language! And Real Men use C, and you want to be a Real Man, don’t you?
Cf. the story of Mel The Real Programmer.
Do it in something low-level, manage your own memory. Programming is a game for the smart, and you must be smart because you’re a programmer, so you can handle it and you won’t drop a pointer or overflow an array.
Result, decades of complex apps tackling arbitrary complex data - e.g. Web browsers, modern office suites - written in C, and decades of software patching and updating trying to catch the legions of bugs. This is now simply perceived as how software works, as normal.
Additionally, in many cases, any possible performance benefits have long been lost due to large amounts of protective code, of error-checking, in libraries and tools, made necessary by the problems and inherent fragility of the languages.
The rebellion against it is only in the form of niche line-of-business app developers doing narrow, specific stuff, who are moving to modern interpreted languages running on top of tens of million of lines of C written by coders who are only just able to operate at this level of competence and make lots of mistakes.
For people not facing the pressures of commercial releases, there was an era of using safer, more protective compiled languages for in-company apps - Turbo Pascal, Delphi, VB. But that’s fading away now in favour of Java and .NET, “managed” languages running under a VM, with concomitant loss of performance but slight improvement in safety and reliability.
And because this has been widespread for some 2-3 decades, it’s now just _how things are done_. So if someone presents evidence and accounts of vastly better programmer productivity in other tools, decades ago, in things like Lisp or Smalltalk, then these are discounted as irrelevant. Those are not manly languages for manly programmers and so should not be considered. They’re toys.
People in small enough niches continue to use them but have given up evangelising about them. Like Mac users, their comments are dismissed as fanboyism.
So relatively small cultural effects have created immensely strong cultures, dogmas, about what is or isn’t a good choice for certain categories of problem. People outside those categories continue to use some of these languages and tools, while others languish.
This is immensely sad.
For instance, there have been successful hybrid approaches.
OSes written in Pascal derivatives, or in Lisp, or in Smalltalk, now lost to history. As a result, processor design itself has shifted and companies make processors that run C and C-like languages efficiently, and processors that understood richer primitives - lists, or objects - are now historical footnotess.
And languages which attempted to straddle different worlds - such as infix-notation Lisp derivatives, readable and easily learnable by programmers who only know infix-based, imperative languages - e.g. Dylan, PLOT, or CGOL - are again forgotten.
Or languages which developed down different avenues, such as the families of languages based on or derived from Oberon, or APL, or ML. All very niche.
And huge amounts of precious programmer time and effort expended fighting against limited and limiting tools, not well suited to large complex projects, because they simply do not know that there are or were alternatives. These have been crudely airbrushed out, like disappearing Soviet commissars.
“And so successful was this venture that very soon Magrathea itself became the richest planet of all time, and the rest of the galaxy was reduced to abject poverty. And so the system broke down, the empire collapsed, and a long, sullen silence settled over the galaxy, disturbed only by the pen-scratchings of scholars as they laboured into the night over smug little treatises on the value of a planned political economy. In these enlightened days, of course, no one believes a word of it.”
(Douglas Adams)