Some random thoughts on programming

Since this post actually got read by people I made a follow up to clarify some of what I was saying. Thanks internet for helping me realize how bad I am at making my point.

As I near the end of my third decade programming, I keep finding myself coming back to pondering the same kind of questions.

  • Why are some people good at this and some are not?
  • Can the latter be trained to be the former?
  • What is the role of language, development environment, methodology, etc. in improving productivity?

I’ve always been partial to the “hacker”. The person who can just flat out code. For some people this seems to come easy. I have been lucky enough to work with a few people like this and it has been awesome. As you talk with them about a problem they are already breaking it down and have an idea of how the final system will work. I don’t mean that other aspects of professionalism, experience, etc. aren’t important its just that I believe in the end there is a raw kind of talent that you can’t teach. My wife can sing, I can’t, and not for lack of practice.  She practices, but she can also belt out a perfect tune without hardly trying.

What are the core talents that makes someone a good hacker? I think there are two:

  1. The ability to break down an idea/system into an abstract set of “concepts and actions”
  2. The ability to understand how these “concepts and actions” execute in a particular environment.

Note that I didn’t say anything about coding. Coding is actually the least interesting part. If you can do the others well, coding is just another kind of typing. Coding is the thing your fingers do while your mind is flipping between 1) and 2). Learning a new language is less about learning the syntax than it is about learning the execution model.

Writing good, correct code involves not just conceiving of how something should be 1) but also understanding all that can happen 2).

I honestly don’t know if these things can be taught.

In 30 years I have only seen a few examples where a mediocre programmer became good or a good one became great and zero examples of a mediocre programmer becoming great.

What does this mean for people who want to build large software systems? The easy answer is to only hire great people. Which is fine when you can do it but not everyone can be or hire the best so what is left for the majority?

What should be done to make things easier for people aren’t in the top 5%?

There are lots of nice things that have been invented like source code control, higher level languages, garbage collection, etc. which reduce or abstract away some of the problems. And while sometimes these abstractions are leaky, they are really one of the few weapons we have against the beast called Complexity. Complexity is the key problem we face, especially if we are not wizards because:

A persons talent with regard to 1)  and 2) above determine how much complexity they can deal with.

When I was in gradual school at UNC-CH Fred Brooks (the mythical man month guy) used to talk about Complexity as being a major thing which makes programming hard. Its not something easily cast aside:
Quoting Dr Brooks:

The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence.

I wish I had appreciated everything Fred said when I was a cocky grad student as much as I appreciate it today… sigh.

Some things are hard because they are hard and not much can be done about them. Still our goal needs to be to not try to make it worse than it is.

To this end, some people say:

Do the simplest thing that will work.

I agree but prefer this formulation:

Strive for the simplest thing that can’t possibly fail.

If you think about what can go wrong you are more likely to find the problems than if you think about what will go right.

I wish I could say we were making progress in reducing complexity but we seem to be adding it into our programming environments rather than removing it. One of the things I have thought about recently is the complexity of these abstractions themselves. Programming languages of today are so much more abstracted and complex than what I learned on (assembler, basic, pascal, C).

For example the world (or at least the web) is currently engaged in writing or rewriting everything in Javascript. Not because JS is so awesome, but because 15 years ago it got stuck into browsers, then we all got stuck with browsers and thus we are all stuck with JS. Sure there’s lots of server stuff done in various other languages but really how long can that last? How long can one justify having to write 2 parts of an application in 2 different languages?

The future looks like node.js.

I am not a JS hater. Its just that I used to think Javascript was a simple language. As someone who wrote 3D Game engine code in C++, my thought was “if these webkids can do JS then it can’t be that complicated”. After building a webapp using Javascript/Dojo/svg to create a realtime browser for genetic information, I realize how wrong I was.

Sure its all nice and simple when you are doing this:

$(“#imhungry”).click(function() { alert(“food”);});

However a lot of complexity lurks just under hood. And its the way it all interacts that concerns me. Crockford called it  Lisp in C’s clothing. This should excite a few people and scare the crap out of the rest.

JS has lots of opportunity for expressing yourself in interesting ways. Expressiveness is good, but too much of it can create more problems than it solves. Back when C++ was a bunch of source code you downloaded from at&t and you had to compile the complier yourself before you could use it, we grad students stayed up late eating bad fried chicken and debating the merits of O-O programming. It was at this point I began to wonder if the productivity gains of O-O were more the offset by the time spent debating about what good O-O practice is. This tape has played in my head more than a few times in my career.

Its not all bad of course: On balance, garbage collection is a win and I see lots of benefits to using closures in JS, but the underlying complexities of things escape most people and their interactions are understood by even fewer. If you have to slog through this (and you should) to understand what happens when you call a function, you are cutting out alot of people from fully understanding what is going on when thier code executes.

I hope that some of this will be address by standards of practice. Things like Google’s guidelines seem to be a good start.  As with C++, I think that the best hope for improving productivity will actually be to agree on limits on how we use these tools rather than seeing how far we can push the envelope.

Well except for wizards and we know who we are 🙂






10 responses to “Some random thoughts on programming”

  1. Jason P Sage Avatar

    Right On. Nice Article… I’m on year 28 of programming – and I really liked this article.

    P.S. I am a JS hater and .Net hater too (though good at both) because P-Code and Compile On demand are Slower and Make it necessary to obfuscate your code if you have intellectual property concerns.

  2. Jonathan Palethorpe Avatar
    Jonathan Palethorpe

    22 years ago when I started in programming I could mess about at home doing a bit of BASIC and at work I wrote green screen sales and invoicing apps in COBOL ’74 using ISAM files. Nowadays programming is many times more difficult- ISAM files have been replaces with all singing all dancing SQL databases with a host of frameworks designed to access them (no in-language stuff as in COBOL with ISAM) – the green screens have been replaced with name your weird and wonderful GUI framework of choice (WPF in my case) and the languages themselves are now many, varied and unreadable to those not in the know.

    I much prefer doing development work now but there are times when I still hanker after a bit of green screen/ISAM/MOVE W01-FOO TO W02-BAR

  3. Michael Fever Avatar

    Yeah, I’ll have to admit that was pretty random. I am an ok programmer, not great, but better than mediocre. I approach every job the same however by looking at what needs to get done and then by what is going to be required later to maintain it. Sometimes being OK is all you need to be.

  4. Bob Gibson Avatar

    Very good points.

    I can relate… I started programming, in Basic & Fortran, … 40 years ago. 😉

  5. Jeffrey Lanham Avatar
    Jeffrey Lanham

    Right on. After working in computers and programming for 27+ years, I have to agree. Programming is, unfortunately, something you inheritly understand or you don’t. You can teach semantics, but not the thought processes that go along with it. That being said, great programmers start as mediocre programmers, but quickly rise. Mediocre programmers tend to rise to somewhat competent and then stay there. That’s, unfortunately, the way it is.

    Great article.

  6. Alan Balkany Avatar
    Alan Balkany

    Sometimes the “hacker” is just someone who’s very familiar with the problem domain. An average programmer may appear to be a “hacker” to their coworkers when they’re re-implementing a system very similar to what they implemented at two other jobs.

    They already know what works best, and the mistakes they’ve made in previous implementations.

    I think most people don’t program as well as their potential because the best practices haven’t been communicated to them. For example, I see many programmers who don’t seem to know that reducing function length is a good way to keep complexity manageable.

    The best practices are still being worked out; the field of software development is still in its infancy, compared to more mature disciplines such as mathematics and mechanical engineering.

  7. Oisín Avatar

    Alan is spot on here. Perhaps the idea that many programmers overlook useful, basic heuristics for programming (such as “shorter functions imply more manageable code, so break down your functions until they are small”) ties in with one theme of the OP, that programming has become more complex in terms of frameworks and maybe language expressiveness.

    When I started programming, it was on the horrible bare-bones no-frills Commodore 64 Basic interpreter, followed a couple of years later by a lovely language called GFA Basic on the Atari ST, and then by 68k assembly and C. Maybe those formative years of just playing about, writing programs with minimal focus on libraries, industry standards and “frameworks”, were helpful in learning how to solve problems and write code.
    But these days I spend more time reading articles about flashy programming/library techniques than actually thinking about and writing programs.

    It could be the case that novice programmers nowadays jump too early onto this wheel, without spending the time doing simple, pure programming, so that they don’t sufficiently develop these key abstractions and heuristic tricks for managing complexity.

  8. Joel Wilson Avatar
    Joel Wilson

    Couldn’t agree more about programming skill or JS. The best programmers I’ve ever known generally don’t have a comp-sci education (myself included). In fact, the single best I’ve ever known personally was an English major a couple of papers away from his Doctorate. He found programming and never looked back. As far as JS, I agree that its “appeal” (for lack of a better word) is simply because it’s the only trick in town. As a language it sucks for anything more than basic functionality. Although I use jQuery professionally, I find it sad that we have to resort to bloated frameworks and layers of abstraction to try and hide the basic weakness of the underlying language.

  9. Leon Avatar

    Oh, I guess for that post you’ll earn a lot of flames from mediocre guys who think that they will be great one day. Especially nowadays when everyone and his mother see themselves as coders because they can hack up some javascript/ruby code 🙂

  10. Maintenance Man Avatar

    Say it ain’t so. Us mediocre programmers need some hope for the future. Heheh.

Leave a Reply

Your email address will not be published. Required fields are marked *