Putting the magic in the machine since 1980.

Thursday, December 4, 2008

Simple Programming Questions Reveal a Lot

This semester I am again teaching our Software Engineering Lab class. This is a senior-level class where we form groups of 4-5 students and develop a large software project. This semester I asked the students to write short programs on the whiteboard. I added this requirement because, after talking to our recent graduates and scouring the blogs, it is clear that nearly all employers are now using these questions to weed out possible candidates. It is not just Microsoft and Google anymore, it is also the small local insurance processing companies, etc. Everyone is asking programming questions, and I can see why.


I wanted to use really simple questions that real employers use, so I went to stackoverflow.com and asked them to give me some questions. They pointed out some simple popular questions like, among others, implementing the following:

  • int strstr(String a, String b): return the first location of string a within string b, or -1 if it does not appear.
  • String trim(String a): return a string that is like a but eliminates any spaces before and after a.
  • String strtok(Stream a): each time it is called it returns the next token in a, where a tokens are separated by spaces.
  • boolean isprime(int a): return true if a is a prime number, false otherwise.

I also came across the fizzbuzz question, repeatedly. The question is:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.
The author claims that more than half of the people he interviews for programming position are unable to write this program, other commenters report seeing similar numbers. I also used this question for some students.

Luckily, my students performed better than that. However, not everyone was able to tackle these programs with complete ease, as I would have wanted. I found that surprising, especially since we require large amounts of programming in our undergraduate curriculum. Note that I do not require a perfect bug-free implementation. For these questions a solid answer simply has to implement most of the logic needed to solve the problem, even if the final program breaks for some boundary cases.

On the other hand, I also noted that all the students know the basic programming constructs. They know about strings, arrays, for loops, etc. The difficulty was always in the transformation from the English description to code. That is, the problem lies in algorithm generation.

At this point some might look to studies, such as The camel has two humps and assume that there is some gene that gives some people the ability to program. I don't believe that is correct. Current research in psychology and neuroscience shows that the vast majority of abilities are learned, not inherited (what a surprise!). That is, I believe people who are good at programming become so because they practice. My hypothesis is that some students manage to take all the classes but end up doing little programming. This is easy to do since most programming experience comes from homework problems which are extremely easy to copy.

I remember that when I was an undergraduate a study revealed that about 37% of MIT's undergraduate students taking a certain introductory-level programming class had copied some of their homework programs. Copying homework programs, or doing no programming in a group project, is an unfortunately common occurrence even in the best schools.

Beside being an honor code violation, the problem with copying someone else's program is that it leads to a vicious cycle. A student that copies one program misses out on some practice so the next program is much harder for him to write, so he has even more incentive to copy the next one, and so on. This accumulates not just over one class but over the whole undergraduate experience: programming is a skill that takes years to build. Thus, it is possible that by graduation some students have had 10 times as much programming experience as others, which we know will make them 1000 times faster programmers.

To summarize, from this experiment I have derived a couple of conclusions. For students I recommend to

  1. Remember to practice the craft of programming. It is easy in a CSE degree, with all the high-level talk about hashing functions, AVL trees, NP-Completeness, multiple inheritance, etc., to forget to practice the basics. Every practising programmer will tell you that most of the programming they do is of the simple kind, the kind reflected by the simple questions above. Complex algorithms, theory, and language structures will help you become a great programmer, but first you have to master the basic craft: writing small programs.
  2. Don't fall into the trap of believing that because you can read and understand a program that you can write code. That is like me saying that I can write a good novel because I can read and understand novels. Yes, you have to be able to read programs, but you also need to practice writing them: turning an English description of the problem into code. I think some students justify copying someone else's code by thinking that it is enough to read and understand what others wrote. It is not.
  3. Practice writing small programs on the whiteboard, or on paper, especially if you are going to be looking for a job in the near future. It is a different experience from typing with Intellisense on.
  4. In an inteview, first make sure you understand the question by repeating it back to the interviewer. Write a few sample input-output pairs on the board. Then implement the simplest solution you can think of. If you are worried that the interviewer might be looking for a better answer simply check with him. Say, “Well, a brute-force way of doing that is yada yada yada.” He will either ask you implement that or ask you for a faster solution.
  5. Don't copy that program. It might take you all night to write it but next time you will be able to do it 10 times faster, and the next time after that you will be another 10 times faster. If at first it seems like it takes you for-fscking-ever to write a simple program it is because it does, and that is normal! Everyone is slow to start. We only get faster with practice. Oh, and that kid in your class that can write code at ninja speed, he started programming in middle school. Don't worry though, one can only get so fast with practice and after a handful of years you will reach that limit.

I also have a few suggestions for employers:

  1. Students are easily intimidated by the whiteboard because most have never used one. Some will be too nervous to perform. Pencil and paper might be a better way.
  2. It is important to give feedback and hints during this process. I've found that several students erroneously (probably, my fault) assumed the problem was a different and much harder one than the one I was actually asking them solve. Other students immediately tried to find a, sometimes non-existent, super clever solution instead of starting with a brute-force approach.
  3. Ask programming questions. It is an amazingly simple and quick way to determine if someone can program. It will save your company a lot of money. But, ask several different questions of each applicant.

In any case, I will continue to use whiteboard questions as they provide valuable experience for the students. I am also planning on using more of this "face to face" type of quizing in my other classes. It is a good and quick way of getting a lot of information on a person's technical capabilities.

PS. If you want more programming practice check out my Programming Questions blog for some more examples.

1 comment:

Jason Rikard said...

I went for an interview this semester and programmed the exact same "fizzbuzz" example you spoke of. Luckily it was really easy with %.