30 November, 2006
Sorry for the interruption in posting lately. I fell sick with something vile very early Tuesday morning and have been recovering. I’m on the mend, though, and probably not going to die soon. Probably….
Recently I went to a job interview for a position at a game company with a heavy emphasis on programming, and one part of the interview was a written programming test. I’ve ranted about game design tests before, so I figured it was only fair that I write a bit about programming tests.
If you read my rant on game design tests, you might already know part of my feeling on this type of test. I think that programming tests are generally more acceptable than a designer test, since you can test things for correctness easier. There’s a bit of subjectiveness in “is this clean code”? If the test is for C++, for example, the issue of where to place the opening brace could introduce some potentially unfair bias.
There are actually two areas that I think need improvement. First, this test was a written test. Now, most programmers I know have terrible handwriting. Actually, I can write fairly nice if I take my time. But, worrying about handwriting on a timed programming test isn’t exactly at the top of my list of things to consider. I see no reason not to give the applicant a computer with a text editing program to type in responses. Let the applicant print off copies if you need something physical to pass around. In addition to handwriting issues, the backspace key is much cleaner than scratching out incorrect bits of writing. (Or smudging the sheet because it was written in pencil instead.)
“But, Brian,” you interject, “if they have access to a computer, they could cheat and look up the answers online!”
Well, this gets my second criticism: doing these tests in isolation. Am I really a better programmer for remembering the rules for virtual functions in C++ without a reference? I don’t think so. Especially for a test taken on site with a time limit, I see little harm in allowing a person a reference. After all, if someone can teach themselves the language well enough in an hour to pass such a test, then they can probably do a pretty good job despite it all.
And, really, it’s silly to think that a test-taker is always going to be in isolation. I was left alone in a cubicle with the test and my cell phone; my cell phone is internet-enabled and I could have used it to look up the answers, anyway, if that were possible. (I didn’t, though.)
Further, I think not having a reference is also unfair to otherwise good programmers that know a lot of languages. I know half a dozen programming languages well at this point. I’ve also been working mostly in Python and a proprietary scripting language over the past few years, so my C++ skills aren’t razor-honed. So, it took me a bit to remember the specific syntax for a while loop in C++ as opposed to another language. Something you’d catch pretty easily during compiling anyway.
Finally, you’re not necessarily going to be able to test the applicant very well with a simple test. Even if I remember C++ syntax, that doesn’t mean that I can understand a particular piece of dense code. Most game code isn’t for the faint of heart, so it’s one thing to write your own implementation of atoi() without calling C runtime functions, but it’s another to trace through dozens of function calls through half a dozen classes to figure out how a spell is actually cast. (This is a about how complex it is to cast a spell in M59.)
In the end, programming tests are necessary, though. Unfortunately, a few people overstate their competence in coding, so you need to weed out the good from the bad. But, I think there are some things that can be done to improve the system and help really separate the pretenders from the people that have a clue.
What are your thoughts?