I used to read Joel On Software a lot during my initial days as a software engineer, after some time when the influence of Feynman’s writings regained its ground (don’t read something that’s not going to contribute to your earnings), I somewhat lost interest in the utopian notion of ‘company for, by and of engineers’ that Joel Spolsky so romantically (and, often) eulogizes in all his writings. I’ve also realized the importance of the phrase that goes like… “Birds of a feather flock together.”
Appears so true while interviewing a prospective team member – but of course, there are other times too when you feel it materializing, especially during the coffee/lunch breaks etc.. A vast majority of interviewers unknowingly recommend a candidate that is closest to them in aptitude and attitude.
During the last couple of years I’ve put the theory of FEPS questions to test and found remarkable results while screening candidates. I’ve also experimented with a few interviews as an observer. Candidates goofing up 40% or more questions failed invariably whenever the interviewer was resorting to the FEPS benchmark… And got instant advancement for the next rounds whenever they played by the rules 🙂
This also proves that the software world is too small and our repertoire of tele-screen questions is even smaller – sans standardization. In this situation, any one with a smart preparation strategy can get a ticket to the in-person interviews – and if the interviewing panel is not awake to the realities still, then such a candidate can ruin the future of the team too – especially when the team is small. So how to get across to the like-minded candidates and at the same time find out where does she stand as a prospective team-mate?
The strategy of using collaborative editors like collabedit, Google Docs etc. worked for sometime, but after a period, it became less effective. Many candidates started declining the calls even before the interview started for the fear of exposing their coding skills (or lack of it).
Cut to the time of netviewer sessions while asking the candidate to solve a TopCoder practice room problem (have to say this was very effective as it gave a standardized score for the same problem, the number of times solution is compiled unsuccessfully before spotting generic mistakes etc.), but this eventually found very few takers as no one wanted to undergo the grind of almost F2F interview while taking the tele one. Besides, computers are still a luxury for many. Even this strategy was fraught with loopholes, because the word of mouth notoriety of interviewer(s) spreads pretty fast among the collaborating candidates, the questions-set travels even faster.
So it was back to the classic ‘hello, xyz… tell me about the most challenging bug… list-sort, recursion… about synthesis’ over phone. I was amazed to see that after trying all the other modes, coming back to telephone wasn’t all that bad, only difference was that while asking the questions to the candidate, I was now making sure that the candidate gets reasonable time to write his code (if he chooses to plagiarize from the www, so be it) in seclusion after the phone screen is done and later, sends it across for evaluation – after all that’s how most of the software developers work — reuse/recycle.
Results after that? Almost as effective as making them write the code in front… May be better.
Almost 70% of the candidates couldn’t code a bug-free, non recursive singly linked-list reversal (in 0.5 hour), 40% couldn’t do it even by 1 hour, 90%+ did it in 2 hours.
Upping the ante, 95% candidates couldn’t write a proper Comeau compilable heap-sort routine even after 3 hours.
Average time for candidates to write the square matrix spiral print (clockwise, anticlockwise) that takes run-time input was a staggering 5 hours – this because the solutions available in the open are either too trivial or incorrect. The hit rate here was a paltry 2%. The ones who submitted genuine (albeit buggy) solutions though, made it all the way to the offer — that tells something.
The above illustration might lead you to believe that unconventional questions lead to better filtering. Incorrect, even conventional and popular questions like reverse a string in-place by words can do the job, only condition is that these questions should come as follow up to something that has already given away candidate’s thought process. Asking bit-manipulation question to a candidate who isn’t comfortable with sizes of data types is a waste. Recursions based questions are best suited to the candidates that have shown enough procedural reasoning. Space-time complexity should always be left for the face to face rounds.
Now, nothing is fool proof, so best strategy for any interviewer is to prepare a standardized set of unconventional questions, practice with modified versions, do time-boxed paper coding and dry runs of the code to be actively involved with the candidate as you’d do during the pair programming exercises, never ask for the perfect solution. Make the candidate ask a lot of questions – sometimes by giving hints – to gauge the thought process, remember, it’s not only the destination, but also the journey that’s important. Don’t fall in the trap of asking behavioral questions, leave it for the HR. And finally, as someone said, pray, as good developers hardly surface and hunt in open.
Disclaimer: The author doesn’t claim to have any degree of authority in the art(?) of interviewing, neither does he claim to be giving out an effective strategy or warranty of efficacy or a semblance thereof. The above piece is but a page of journal for posterity and is maintained for interested readers who might find the study to be useful, dreadful or amusing based on their respective tastes. Lastly, the views expressed above are of the author’s alone and his employers present or past have nothing whatsoever to do with the aforementioned.