Critical Section

Codility ... testing, testing, 1 ... 2 ... 3

Sunday,  02/28/16  11:01 AM

Codility in actionRecently a potential client asked me to take a coding test on Codility.  (Well, they asked "us", not knowing that "us" was me :)  I took the test and failed miserably.  Herewith, lessons learned...

At right: the Codility interface ... (click to enbiggen)

My first mistake was taking the whole thing lightly.  I'm a great C++ programmer (if I say so myself) so I was confident in my ability to code.  However, there is more to passing a Codility test than writing good/great code.  1) you have to understand the interface, 2) you have to code an efficient solution, as opposed to a simple one that works, and 3) you have to do it in a limited amount of time.  Oh, and 4) you have to pay attention to boundary conditions.

1) To understand the interface, Codility recommend you take their sample test.  This is good advice which I did not take.  The interface is just fine, but it does take a few minutes to acclimate to it, especially the means of entering and running test cases.  In particular, they allow you to run any test cases you want, but they do not tell you whether the tests get the right answer, only whether they compile and run.  This isn't immediately obvious, and it's important.

2) Codility specify the O-notation of the expected best solutions.  Most of their algorithmic problems are pretty simple, but coming up with a most efficient solution may require tricks.  It's a mistake to go after a tricky / efficient solution first, however, as I found to my chagrin.  As I note below, code a simple solution that works first, then develop optimizations.

3) You are typically given 30 minutes to complete a test (I was given 90 minutes to complete three).  This is plenty of time to code and test a simple algorithm.  However, it is not plenty of time if you spend a lot of it learning the interface and investigating interesting optimal solutions.  Plan your time wisely.  Out of 30 minutes I suggest 10 minutes for a bolt dumb solution, 5 minutes creating a wide variety of test cases, and 15 minutes for optimization.

4) In many of their tests (I have now taken six) Codility specify boundaries like "integer in the range -2,147,483,647 ... +2,147,483,647.  You will recognize these are the max values for a 32-bit int.  They will give you these values in a test case.  If you want to add 1 to such an int, you'll need a long long.  This sort of stuff doesn't often come up in everyday coding, but you need to be ready for it.

My suggestions for anyone taking a Codility -type test:

  • Take a sample test or two first, to get used to the interface.  This post discusses their sample problem (take that one first) and has links to two others.
  • Develop your solutions in your favorite IDE, and then copy-and-paste them into Codility.  Really really.  At first you will think "this interface is fine, no worries" (as I did), but when you're pressed for time and want to run many test cases, you'll be happy to have your own IDE.  Mine is Visual Studio, yours might be Xcode or Eclipse, but use it.
  • For each problem, develop the absolute most bolt-dumb simple solution first.  Make sure it works (with "printf" debugging, or whatever), and then use it as a check against your more complicated more efficient solutions.  This does two things for you: 1) if you run out of time, you have a solution that works, 2) having an instant test for any would-be more-efficient solution is most helpful.  You can run though many, many possible optimizations rapidly once you have an automated test.

TestDome ... another test site similar to CodilityIn addition to Codility, other sites which offer these types of tests include TestDome and HackerRank.  They're actually a lot of fun once you get into the mode of taking them.

A final comment: In Googling I found quite a few people online have commented that this type of test is a bad measure of programming ability.  Okay, I agree it isn't perfect, and of course you will want other data points before hiring someone.  But it's a pretty good screen; ideally a good programmer would pass such a test, easily (*ahem*), and the fact that they've taken the time to do so is an indication of their sincere interest in your opportunity.

Cheers, and happy testing!


About Me

Greatest Hits
Correlation vs. Causality
The Tyranny of Email
Unnatural Selection
Aperio's Mission = Automating Pathology
On Blame
Try, or Try Not
Books and Wine
Emergent Properties
God and Beauty
Moving Mount Fuji The Nest Rock 'n Roll
IQ and Populations
Are You a Bright?
Adding Value
The Joy of Craftsmanship
The Emperor's New Code
Toy Story
The Return of the King
Religion vs IQ
In the Wet
the big day
solving bongard problems
visiting Titan
unintelligent design
the nuclear option
estimating in meatspace
second gear
On the Persistence of Bad Design...
Texas chili cookoff
almost famous design and stochastic debugging
may I take your order?
universal healthcare
triple double
New Yorker covers
Death Rider! (da da dum)
how did I get here (Mt.Whitney)?
the Law of Significance
Holiday Inn
Daniel Jacoby's photographs
the first bird
Gödel Escher Bach: Birthday Cantatatata
Father's Day (in pictures)
your cat for my car
Jobsnotes of note
world population map
no joy in Baker
vote smart
exact nonsense
introducing eyesFinder
to space
where are the desktop apps?