Software Lessons from Lego

[chriscoxadobe]

Bear with me a moment, I'm going to try and explain a computer science problem by analogy.
Lessons from Lego
When I was young I would build anything I imagined from Lego(R). I had the small, generic blocks - and lots of them. Because they were so simple, because they connected so easily - I could build anything. Ok, so not all the edges were perfect, sometimes I had mixed colors, and sometimes I lost pieces to the vacuum cleaner. But I could put up with a few odd edges and mixed colors - as long as I got the shape/structure that I wanted.
What I didn't have were kits that make a castle, or kits that make a helicopter. Those kits reduced the complexity and increased the "realism" by creating specialized parts. But just how many ways can you use a catapult shaped part? Are those helicopter shaped pieces going to be useful when you want to build a race car? Probably not.
Heck, there are even theme parks decorated/built with Lego blocks. If you look at the pictures of the parks, you may notice that they're mostly built with the simple blocks, with only a few specialized pieces added for flourish. You know, I did that same thing when I was building with Lego - lots and lots of generic pieces to build the structure, and only a few decorative pieces on the edges. Hmm, that seems to be a pattern.
The lessons: You get much more flexibility building with generic pieces than with specialized pieces.
When building something complex, you need the generic pieces far more often than you need the decorative pieces on the edges.

Conversely: Specialized pieces look good, for their specialized purpose, but aren't generally useful for much else.

How do Legos relate to software?
Software development certainly isn't as simple as snapping together Lego blocks, so it is not a perfect analogy. But the lessons above apply directly to software development if you conceptually substitute software widgets or APIs for the Lego pieces.

In software, widgets are more or less the pieces that the end user sees - like buttons, windows, or a list display. But widgets can be made up of smaller pieces - just as a car is made of motor and brake shaped pieces, a web browser is made up of text display, button, image, and menu shaped pieces.
APIs (Application Programming Interfaces) are how the software developer interacts with the OS or other libraries of code - not so much parts as connections and requests to perform actions. APIs can range from simple to very, very complex.
Simple API:

1. draw a line here

Complex API:

1. create a global drawing context
2. set graphics clipping context
3. set line antialiasing style
4. set line width
5. set line dash style
6. set line end cap style
7. set line edge style
8. set line fill style
9. set line starting point
10. set line ending point
11. render the line
12. commit changes to drawing context
13. force drawing context update to video buffer
14. destruct any styles or contexts that will not be reused
(non-developers are thinking this is a contrived example, and experienced developers are thinking "you forgot..." )

But from what I can tell as a developer, Operating System and development package vendors don't understand these lessons. These vendors spend a lot of time creating large, showy, over-specialized pieces (uber-widgets) that do what they think developers want. (Look, it's an HTML viewer shaped piece!) And it certainly seems as if like the vendors believe they know what developers want, without necessarily asking or listening. I really think they are spending too much time on uber-widgets, and not nearly enough on the simpler pieces, and the links between them, that developers need. The end result is that the vendors build a lot of things that professional developers can't use, and the developers struggle to make the available pieces do what they need.
Looking at recent operating system widgets and APIs, toolbox libraries, and even many web kits -- the over-specialized pieces are everywhere. From graphics APIs (like the line drawing example above), to programming languages, UI widgets, and on down to what should be simple operations ("sorry, no square or rectangle pieces, just chrome plated fenders with flashing lasers on top"). And this isn't coming from just one vendor or one group: it looks like a full-on trend. Somewhere along the way, I think the concept of abstraction got way out of hand. (But it will take a whole essay to explain software abstraction to non-developers, so I'll save that for later.)
Judging by their marketing materials and developer training, many vendors think that these uber-widgets are great, and even tout them as their competitive advantage. A few want to go so far as to take away access to the simpler building blocks. (You know: the ones you need most often) But have they really tried to build a professional application with all those over-specialized pieces? (Remember that race car built from helicopter parts?)
Sure, those uber-widgets make it easy to build good-looking demos really quickly, if your vendor wrote the widget needed for that demo. And if someone is making a proof-of-concept application (thesis-ware), then the uber-widgets might be what they need. They don't have to worry about localization, or compatibility, or stability -- they need something that works just well enough to demo on their platform in their language. But "works just well enough" isn't going to cut it for professional software development.
When developers write professional software applications, they have to think about things like making them:

• work the same on multiple platforms
• work well in many languages
• work the same across multiple OS versions (including versions that haven't shipped yet)
• never, ever lose the user's data (crashing or corrupting data)
• work well within the constraints of the customer's system
• maintainable for a very long time (you don't want to rewrite something that already works well)

In comparison, uber-widgets often:

• have their own behaviors, which can't easily be matched across platforms
• change behavior across OS versions or library releases
• may or may not work correctly in different languages
• have serious bugs, which may change across OS versions or library releases
• include more code or use more memory than necessary for the task
• get removed or replaced with new OS versions or library releases

The more a developer relies on uber-widgets:

• the more time they spend trying to make behaviors match
• the more often they have to update their code to deal with changes in widget behavior
• the more time they spend working around language (localization) problems
• the more time they spend chasing bugs in things outside their control
• the more they have to fight to keep things working well on a reasonably equipped system (or give in and bump up the minimum requirements)
• the more often they have to rewrite functionality, or at least rewrite interface/glue layers, because the widgets they use have been removed or replaced

Hmm, there's something that doesn't have a simple analogy in Lego: if a developer relies on a certain widget, and the vendor no longer supplies that widget - the developer has to find another widget to replace it (and change the interface), or write a replacement. The more complex that widget was, the more effort it will take to replace (and I suspect the effort is non-linear with the widget complexity, but I'm not yet sure if it is geometric or exponential). And if the vendor removes or replaces a lot of widgets at one time - the effort to incorporate those changes can be staggering.
Imagine if you built a helicopter from a Lego kit, then someone swapped all your helicopter pieces for submarine pieces. Or if they just took away all the helicopter pieces and left you with nothing. To build the structure that you wanted, you would have to rethink your design, find new parts, and possibly start over - or just give up and find something else to do.
Sure, that sounds extreme, and maybe unlikely for Lego, but I've been watching it happen more and more often in the software industry. Every time developers use an uber-widget, it becomes another chain that vendors can yank - making the developers spend time and resources to update their own software to meet someone else's goals. Using smaller, simpler widgets means that developers spend less time chasing vendor changes - because the connections aren't as complex, and most of the behavior is in the developer's code instead of the vendor's. And personally, I like setting my own direction for my application (helping customers) - not having that direction set for me by a third party that doesn't care about my application or my customers.
Even when the widgets remain available and unchanged, complex OS supplied widgets are still likely to have bugs that can only be fixed by the OS vendor. Bugs in more complex OS widgets always seem to be more difficult to work around, and frequently more complex to diagnose. In my experience, bugs in simpler OS widgets can usually be worked around, or patched around, far more easily. And customers really don't like to hear that they have to wait for the next major OS release that may, or may not, fix their bug.
Building with smaller widgets and simpler APIs, the developer controls more of the code and can control the size of the generated code, the performance of the code, and the memory footprint. If the developer doesn't want or need the chrome (or the flashing lasers on top), then they don't have to pay for it or use it. With uber-widgets, the developer has to accept the existing code size, performance, and memory footprint - even when they don't meet the developer's goals. Using an uber-widget means you pay for all the features, all the time. And because the code for the uber-widget is in someone else's control: what might have met the goals today might not meet those goals in the next widget release.
In Summary:
When writing professional applications, it is usually better for developers to avoid uber-widgets and write their own functionality based on simpler vendor widgets. This is so that the developer has control over the behaviors, can fix the bugs, can add just the features they need, can tune the performance to meet their needs, and isn't nearly so dependent on vendor code. For these reasons, professional software development needs more of the lower level, simpler widgets and APIs, not so many uber-widgets.

P.S. I also find it amusing that many of the people making decisions for developers are people who have never written a professional application. If you are going to tell other people how to build a better car, you really need more experience than a Lego car kit (Ages 7-12).