Back in touch with tech; or, How about a real programming language?

There's a singular joy in learning a new programming language, especially when you've been in the one- or one-and-a-half language rut for three years.

I tore through The Pragmatic Programmer, which is just as good as advertised, the past few days. While much of the book was already obvious to me and had already been part of my programming practice for a while, just as much was cleverly and brightly presented. "Wow, I can write my own code generators and source analyzers?" Not that it's beyond my ability to do so, but it had somehow never occurred to me to actually try it. Or I just never had a need.

Anyway, they also suggest keeping your "knowledge portfolio" up to date. It's become clear to me in the past few months that a lot of my knowledge is dating rapidly. A new language sounds like just the fix. While I'm at it, why not pick one that can help me write whizzy-bang code generators and analyzers?

I've done Perl in the past, but in my semi-regular checkins I've become impatient with the interminable Perl 6 gestation period. I'm not really interested in spending a lot of time to learn something experimental, and Perl 5 is 10 years old at this point with hardly any changes in that time. Perl is a really neat hack and a cool thing to work with, and I'm certainly not opposed to it, but as I find my own identity as an engineer and develop my own style, I feel that it's not the direction I want to be going in technically.

Next you're saying, "how about Python?" Well, I looked. Cool community, neat ideas, huge install base, pretty mature. Yet... there's something about it that just sounds foreign to my ears. While my company does a lot of internal work in Python, little of it is in spaces that scratch my itches. I think I'll pass for now. Maybe one day I'll check it out again, possibly even learn it, but right now I'll focus on the other alternative: Ruby, the other language we use internally.

This Ruby thing is neat. It works for me because:

1. it's different enough from Java and C# and D to be a good mind-stretcher;
2. it kicks ass at text processing;
3. it's dynamically typed and puristically object-oriented;
4. Rails is the new hotness in web architecting.

My Ruby education is just beginning, but already there are some ideas in it that have made me sit back and clap excitedly. Here's one neat nugget from an exception handling mechanism:

rescue ProtocolError
  if @esmtp then
    @esmtp = false
    retry
  else
    raise
  end
end

See what that does there? rescue is an exception handler, like a catch in the C++ family. Yet the keyword retry tells the control flow to reenter the exception scope after attempting to repair the error. Are you freaking kidding?! That kicks ass! That one little bit just blew my mind enough to make me completely interested in what Ruby has to say.

Where the hell does this class go?

Software development, however you want to define it, is really hard.

I've been doing this professionally for 10 years now, and recently I've been struck forcefully by how, well, bad a lot of software is. It breaks, it crashes, it doesn't do what it needs to, it decides to destroy unrelated data innocently minding its own business, it lies to your face about what will happen when you push the button. It doesn't matter who the developers are, or how big the team is, or how much time and effort are put into it; the goal, asymptotically approached, is making something that doesn't suck *that* bad.

Some developers are idiots. I have my moments, and I think there's a lot of truth in the simple "try not to make mistakes" approach as opposed to "find the best possible solution". But some are just straight-up, envy-inducing geniuses, and they work on really powerful and complex systems that wind up being terrible. How much effort has Microsoft spent on developing the Windows family? Is the quality of the end product proportional to the amount of effort invested in it? How many brilliant, motivated people worked on it through the years?

I got The Art Of Computer Programming for Christmas, and I eagerly tore into it. All right! The undisputed classic that lays out the guts of classical computer science! Surely in here I'll find my answers, clearly and cleverly presented in inky-black awesomeness, explaining why I can't architect an entire J2EE application correctly on the first attempt!

...yeah, it does sound pretty dumb, but I've always been the kind of person who figured that if he read just one more book, or if he found the right teacher, or if he learned the proper technique, then the formerly difficult and frustrating task would become easy, clear and fun. There was a time when this was true. When I first taught myself how to program, I was able to make the computer do damn near anything I wanted (so I thought). This part of Real Programmers Don't Use Pascal strikes too close to home for comfort:

When I got out of school, I thought I was the best programmer in the world. I could write an unbeatable tic-tac-toe program, use five different computer languages, and create 1000 line programs that WORKED (Really!). Then I got out into the Real World. My first task in the Real World was to read and understand a 200,000 line Fortran program, then speed it up by a factor of two. Any Real Programmer will tell you that all the Structured Coding in the world won't help you solve a problem like that-- it takes actual talent.

Well, I don't know whether I've got talent or not. I know I'm not a genius, because if I were I'd be off creating awesome things already. And I want to succeed in the Real World, not in the Happytime Sugar World Where No Challenge Trips You While You Saunter About Listening To The Sound Of How Awesome You Are. If a genius is just someone with supreme talent, like a Mozart or a Nietzsche, they don't need to develop skills in the same way we mortals do.

I'll save what I've found in TAOCP for the future, but suffice it to say that it wasn't exactly what I was expecting.

I read other programs and I can discern their structure; if it's Java (and it usually is, since that's my job), I can grok a method on one reading. I can understand an entire class in 15 minutes or so. After an hour I can fix bugs in the javadoc, write unit tests, and leave it a much better place than when I found it. If I have a package, it's harder to do. Some packages are just a velcro strip to stick loosely related functional units (java.text), while others provide a (hopefully simplified) abstraction over a much larger problem (javax.swing). The former are much easier to understand than the latter, which is fine since the latter is much more complex. But what do I do when I have to write something that's somewhere in the middle? How do you go top-down in architecture as opposed to bottom-up?

I want to be a better developer. To do that I must become at least competent in top-down design. I can't do it well, so I look for help in books. No help there. Can't learn it from geniuses since they don't have to think about what they're doing in the same way that I do. Can't read existing code because even if it is designed well it doesn't tell me anything about where it came from. So... what the hell do I do?