On (not) sucking at my job
Previously on posts tagged with 'angst' (mostly friendslocked): our hero became depressed, sought help, and is now (mostly) feeling better. Now read on...
John Walker, in his excellent book The Hacker's Diet, distinguishes two approaches to tackling problems, which he calls the Manager's approach and the Engineer's approach. Management is about monitoring and ameliorating chronic problems to keep their symptoms to within an acceptable level; engineering is about solving problems outright. Most difficult problems, he claims, must be tackled using a combination of both approaches.
My problem, which might be summarised as "I became depressed because I suck at my job", has so far responded well to a management approach: leaning on friends, attending CBT workshops, prioritizing exercise, talking to my supervisor about my problems. I'm very grateful to all of you for your support, and to the Glasgow NHS mental health team. Now that I'm feeling a bit more spoonful, it's time to apply the engineer's approach, and suck less at my job.
[A perhaps more honourable alternative would be to find another job at which I wouldn't suck, but that would be fraught with risk, and my current job has much to recommend it. Besides, I'm not sure that such a job exists.]
I have three basic problems:
On the first point, I'm reasonably happy with the readability of my code, but I'm unhappy with its testability and correctness, and I'm very unhappy with the time it takes me to produce it. I'm frequently struck by analysis paralysis. I've spent most of my programming career working with high-level languages, so I'm not very good at lower-level programming. I think the only solution to this problem is to write (hundreds of) thousands of lines of code, at as low a level as possible.
On the second point: before starting this job, I'd previously worked at a compiler vendor and at a web startup which did some machine-learning; back in high school, I'd done some assembly programming for an embedded system. I'd also done a bit of background reading on compiler theory. It turned out that this was insufficient preparation for a job using machine-learning to improve the state of the art in compilers targetting embedded systems. Astonishing, I know.
There used to be a cute slide in the Edinburgh University first compilers course:
Now, that's true for compiler theory, and it explains why compiler research is still a going concern after sixty years, but it doesn't explain why day-to-day hacking on compilers is hard. For me at least, that's because hacking on compilers is systems programming. You need to know, at least to the level of "understand the basic principles of and can look up the details if needed", about things like addressing modes, instruction set architecture design, executable layout, and calling conventions. Forget the fat guys who know C++ who apparently keep Silicon Valley running; I work with guys who know Verilog and the GNU ld control language.
[Betcha didn't know that the GNU linker embeds a Turing-equivalent programming language :-)]
Now, none of this stuff is especially difficult as far as I can see. But it's a lot of background knowledge to have, and if you lack it then you'll be constantly hitting problems that you lack the mental toolkit to address. So here's what I'm doing about it:
I have no real idea how to start addressing my third problem, namely my ignorance of experimental design. Go to the library and get another book out, I guess. Anyone got any recommendations?
John Walker, in his excellent book The Hacker's Diet, distinguishes two approaches to tackling problems, which he calls the Manager's approach and the Engineer's approach. Management is about monitoring and ameliorating chronic problems to keep their symptoms to within an acceptable level; engineering is about solving problems outright. Most difficult problems, he claims, must be tackled using a combination of both approaches.
My problem, which might be summarised as "I became depressed because I suck at my job", has so far responded well to a management approach: leaning on friends, attending CBT workshops, prioritizing exercise, talking to my supervisor about my problems. I'm very grateful to all of you for your support, and to the Glasgow NHS mental health team. Now that I'm feeling a bit more spoonful, it's time to apply the engineer's approach, and suck less at my job.
[A perhaps more honourable alternative would be to find another job at which I wouldn't suck, but that would be fraught with risk, and my current job has much to recommend it. Besides, I'm not sure that such a job exists.]
I have three basic problems:
- I'm not a good enough programmer;
- I don't know enough about the problem domain;
- I am now effectively an experimental scientist, but I don't know anything about experiment design.
On the first point, I'm reasonably happy with the readability of my code, but I'm unhappy with its testability and correctness, and I'm very unhappy with the time it takes me to produce it. I'm frequently struck by analysis paralysis. I've spent most of my programming career working with high-level languages, so I'm not very good at lower-level programming. I think the only solution to this problem is to write (hundreds of) thousands of lines of code, at as low a level as possible.
On the second point: before starting this job, I'd previously worked at a compiler vendor and at a web startup which did some machine-learning; back in high school, I'd done some assembly programming for an embedded system. I'd also done a bit of background reading on compiler theory. It turned out that this was insufficient preparation for a job using machine-learning to improve the state of the art in compilers targetting embedded systems. Astonishing, I know.
There used to be a cute slide in the Edinburgh University first compilers course:
Now, that's true for compiler theory, and it explains why compiler research is still a going concern after sixty years, but it doesn't explain why day-to-day hacking on compilers is hard. For me at least, that's because hacking on compilers is systems programming. You need to know, at least to the level of "understand the basic principles of and can look up the details if needed", about things like addressing modes, instruction set architecture design, executable layout, and calling conventions. Forget the fat guys who know C++ who apparently keep Silicon Valley running; I work with guys who know Verilog and the GNU ld control language.
[Betcha didn't know that the GNU linker embeds a Turing-equivalent programming language :-)]
Now, none of this stuff is especially difficult as far as I can see. But it's a lot of background knowledge to have, and if you lack it then you'll be constantly hitting problems that you lack the mental toolkit to address. So here's what I'm doing about it:
- To address the patchiness of my knowledge about machine-learning, I went to the library and got out a couple of ML textbooks, one of which I've now read. I've also signed up to the free online machine-learning class from Stanford, and, while I was at it, the Introduction to AI class too.
- To address my ignorance of architectural issues, I'm auditing the computer design undergrad course at Edinburgh; when I've finished that, I'll audit the follow-on course in computer architecture. So far it's mostly been the sort of boolean algebra and digital electronics that I learned at my mother's knee, but there's a lot of stuff I don't know later on in the course.
- Linkers are a particular problem for me; I think linker errors are the Universe's way of telling me what it's like for a non-techie who thinks they need to press the "any" key. zeecat kindly lent me a copy of Levine's book Linkers and Loaders, which I am now reading; as an unexpected bonus, one of the early chapters is a 30,000ft tour of computer architecture. To my delight, the book walks you through the construction of a simple linker in Perl.
- To address my lack of C and assembly language experience, to solidify my understanding of basic compiler theory, and to give me a testbed for implementing some optimisation algorithms later on, I started writing a simple compiler in C. Currently it accepts a simple term/factor/expression grammar and outputs x86 assembly; the plan is to extend it so it accepts a subset of C. "Compiler" is currently a bit of a joke; it is technically a compiler, but one that does really aggressive constant folding :-) I haven't hacked on this for a while, because work got in the way, but intend to pick it up again soon.
- To address my ignorance of C minutiae, I started reading through the comp.lang.c FAQ, at Aaron Crane's suggestion. This is also stalled, as is a project to convert said FAQ to a collection of Mnemosyne flashcards.
- The world seems to be standardising on Hadoop as a clustering solution; I should try that out at some point.
I have no real idea how to start addressing my third problem, namely my ignorance of experimental design. Go to the library and get another book out, I guess. Anyone got any recommendations?