There is no scientific study behind this blog and one based in 100% opinion but it is made to get you thinking of a potential new paradigm in hiring and training.
Let me quote a data scientist from Twilio to start
"Many interviews out there test for algorithms including Google, Amazon and I can do very well on those but your interview was the first interview showing productivity of a developer and I do horrible in that"
I have been studying (for fun) individual productivity and team productivity for 15 years. We even tested out some patterns that made team speed much much faster so everyone's code was similar with great success. These patterns spread to other teams within Twitter organically. Another team speed win was something called Feature Testing. To work on team speed, we need to make sure we are hiring top-talent that is extremely productive.
I wanted to have two parts of the interview and this article concentrates on the first part of the interview with "How productive are you at raw coding?". Clearly your interview process will need something to weed disagreeable people out of it. After all, that is going back to team speed. Here in this article though, I want to discuss interviewing for individual speed.
I have given the same interview over 15 years probably to more than 300 people and in 4 different languages. I started out with the intention of hiring the most productive developers and ones that may not know algorithms (like Google/Amazon interview for). Not only did it result in some really fast coders but it also brought insights and patterns. I started to learn what the fastest developers do differently during the interview. I also found out, they continued using these same patters in the job.
Productivity Differences
Let's first talk about what I have been seeing in differences in productivity in a very basic interview with no fancy thrills or anything. With 4 tests to get passing, we see some Senior Developers not even complete one test in 60 minutes(I cut the interview off at 60 minutes to not waste any more time). I have seen a junior engineer complete all 4 tests in 22 minutes. This means a junior developer was averaging 22 / 6 = 3.666 minutes. The Junior developer performed 16 times better than the senior engineer.
I see many business people hire for skill instead of talent. Not only do I find hiring talented developers is a must-have, they learn the skill/domain way way faster and beat the 'skilled' senior engineer who is 16x slower any day of the year. There is also a good study and science backing this resulting in the book First, break all the rules. This is why I always invite business people to watch these interviews live and gain a deep understanding of who they really want to hire. Do you want to hire someone with blockchain, web3 experience or do you want to hire someone 16 times faster? Make sure you create the correct interview and hire the correct person!
Patterns
Here is a list of patterns that I have noticed over the years
Ability to read a stack trace is solid
Run the test, fix the bug and learn while we code
The first one eliminates guessing by top talent which I see the slower developers guessing much more and so they end up being less productive. They may be great algorithm developers but in real business coding, we only deal with algorithms 10% of the time and then once we have decided on an algorithm it is more about "How productive are you at delivering working code". Learning to read a stack trace is so critical that these days, I make sure everyone is familiar with reading a stack trace and what it really means. Chained exceptions are especially important. Sidenote: The typescript/javascript community in the interviews has been really bad about reading stack traces and the average interview score seems to be far behind more enterprise languages so far because they start guessing.
The next item was really interesting to me because for years I have heard "It is better to understand the system first, then to go code after you understand it". For years, I thought this made sense to me. As I studied this more though, I came up with a more advanced interview and I watched those who had the 'learn first, implement after' get 100% stuck. I had quite a few that I told them to "Look, run the test and fix the first bug" and then they ended up crushing the interview after that and understood it after they code. This leads me to a new guess/hypothesis "Coding and implementing is a faster way to learn even if you throw away the code". Perhaps we need to start training junior engineers to learn in this way. I started my career learning that way but became quite a bit fluent where I feel I can read the code and understand it much more quickly now.
Algorithms vs. Business Logic
Up to this point, I still did not even talk about the actual interview I give, but basically we wrote up a few unit tests on top of an API with just enough implementation to compile but all the tests will fail. We ensure they are not dealing with any technologies they are not familiar with. We throw in a listener or lambda callback as well somewhere in the mix to test the understanding of different common patterns in software. We then interview the developer in the language they are most comfortable with as we find we can retrain that person in a new language quite quickly. Basically, we "Hire talent over skill" like the book First, break all the rules talks about from it's study on what the best managers do.
Since 10% (or even less really) of our job is algorithms, it becomes interesting hiring not for algorithms but for productivity as it turns out we randomly get some percent of engineers that are really good at algorithms. When we need to think of 'which algorithm should we use', we defer the team to these engineers to discuss and review. After that, it is back to productivity coding and getting the job done.
Warning
Please take all this with a grain of salt. There is no scientific study done here but is 15 years of subjective opinion. It would be great to run a more formal study on the subject. Until then, I keep giving the same interview and watching the results after that.