Portfolio Based Interviews

An argument against tests, algorithms, and take home assignments.

The problem

In her excellent book, Cracking the Coding Interview by Gayle L. McDowell:

False negatives are acceptable. This is sad (and frustrating for candidates), but true. From the company’s perspective, it’s actually acceptable that some good candidates are rejected… They’re far more concerned with false positives: people who do well in an interview but are not in fact very good.

McDowell argues for algorithm based technical interviews as the best way to determine someone’s skill. After reading it, I felt in my gut that something was off. It’s probably because I’ve never done well with algorithm based tests. Given any topic, I’m confident that I can find, learn, and master the topic. However, put me in a contrived situation, with unnatural presures and all my confidence and creativity plays a game of hide-n-seek.

Consider this: If I were a designer, I’d show my portfolio and answer a few questions to prove my skill. Why then do we treat software developers so differently?

The software development industry has struggls to determine a developer’s skill but isn’t it already shown in the work they’ve done? From my perspective, here are the approaches I’ve seen companies take:

1. Informal interviews - General questions used to “tease out” the skill and underlying knowledge of a candidate. I’ve always done fairly well with these interview styles because the people who give them are usually very experienced.
2. Take-home Assignments - A problem statement that the interviewee must solve in code. Sometimes this is combined with algorithm interviews.
3. Algorithm interviews - Computed science related technical tests.

I’d like to suggest a fourth approach:

4. Portfolio interviews - Put forward the work you’ve done. Show your technical, communication, and teamwork skills.

Why have another apporach? Isn’t the current model working?

• I spend a large amount of time on each job opportunity. This means I’m limited by time. I’m usually only able to keep 1-3 interviews going at a time, especially when I’m employed. This traps me to a limited subset of viable matches.
• I don’t usually do well on algorithm tests unless I happen to have recently worked on a related problem. Companies often do not have time to prepare tailored assignments related directly to the actual job.
• Most importantly, we lose out on great candidates. Great people are ruled out for blunders, mistakes, life circumstance, and just having an off day. This has happened to me, perhaps it’s happened to you as well.
• I think companies would do better to honor our time. If I can show you my work, teach it to you, and answer any of your questions we’ll both save so much time.

Portfolio Interviews

An applicant should show their technical prowess by demonstrating an actual project completed by them. These are projects chosen and completed by the candidates long before they interview.

How it works:

1. The candidate has solved a real-world problem in code. They own the code in their repository. It is important that candidates have chosen their own projects so that they may be reused and improved.
2. A candidate shows up for the technical interview ready to teach the problem/solution of their project.
3. All questions by the interviewers are “on the table.” The applicant should be ready to discuss all aspects of the project: problem/solution, other viable solutions, architecture, specific programming choices, etc. This proves that they wrote it (or at least deeply understand it) while demonstrating communication and technical skill.

Madam, you may make proof of him, and if your ladyship find him too saucy or wasteful, you may return him whence you had him. — James Howell, letter, 2 June 1630

To put it simply:

The proof [evidence] is in the pudding [project].

Key things to look for:

1. What well did they communication their work? - Was their approach to explaining the problem clear and helpful? Do you understand the solution they gave? What happened when you challenged their approach? What happened when you gave a suggestion? Did they inspire you?
2. What programming skill did they show off? How well was the code organized? What technical challenges did they face? How did they solve for them? What was the scope of the solution? How difficult would you rate the challenge of the problem they took on? How satisfying was the solution they gave?
3. What are their teamwork qualities? Gives an accurate depiction of who a candidate was or is. It does nothing to show who they could be. I leave it up to the interviewer to lean into questions along the lines of aptitude. Was their tone something you could work alongside each day? Did you find anything they are passionate about? Are they committed to excellence? Can they be disagreed with? Are they teachable? Are they willing to teach?

Advantages

It is my view that in the public/private GitHub (or similar) of every developer is a project that demonstrates a problem they took upon themselves to solve. I think that is a fair ask. I freely admit that I am wrong to some degree and in some situations. Here are the reasons I think this approach works:

1. A project clearly demonstrates initiative and self-direction. It forces the candidate to spend their time investing in a problem. They must manage their work/life balance. They must create the requirements. They only have themselves to answer to.
2. A completed projects shows the ability to understand a problem, come up with a solution, and apply it concretely (not in abstractly.) You can avoid candidates who talk a good game or interview well. You can also avoid discarding candidates who struggle to interview well.
3. A project respects the company. Internal employees are not required to come up with all manor technical questions, tests, and assignments. They may avoid boring interview sessions or know quickly they have an undesired candidate before them.
4. A project provides space for mentorship. Candidates may work with friends to help them improve their weaknesses before a project interview. For myself, I’d love to help someone improve their project. This can only be a good thing.
5. A projects shows the unguided thought process behind a candidate. It shows what interests them. Possibly what they are passionate about. It’s a form of professional vulnerability.
6. A completed project is superior to “tests” for multiple reasons (a) the candidate may work at their pace (b) uses skills they feel comfortable with (c) not all candidates test well for valid reasons (d) tests do not match the real world while projects do (e) applied knowledge is always better than conceptual knowledge, it is easier to evaluate concrete solution over abstract solutions
7. A project respects the candidates time. They may work on the project in off hours and reuse it repeatedly. They may go above and beyond to create something truly compelling or inspired.
8. Since projects are concrete they are believable. If I talk you may doubt but if I show then you are convinced.

Conclusion

This is how I’d like interviewing to work for software developers. For myself, I only want to work at companies who are thoughtful. If they do not have the time to think through their approach then it’s likely they do not have time for other important questions.