Home

Category: Interviewing & Hiring

How I interview engineering candidates

“What the hell do I ask the candidate?”

Probably the most common and immediate problem that engineers have when they start interviewing candidates is, “What the hell do I ask them?”

I’ve read innumerable books, talked to dozens of hiring managers/recruiters/fellow engineers, and tested a whole bunch of my own questions in candidate interviews. I’ve come to believe there are only a handful of questions you need to assess an engineering candidate thoroughly.

Question: What happens when you type a URL into your browser and hit enter?

This is an age-old question, and it can be quite effective for some roles (backend engineering and devops, especially). The usefulness of the question is that the answer can go in any number of directions and depth. I once interviewed a candidate who spent twenty minutes answering before I stopped him–and he hadn’t yet got outside of the local machine!

Your typical candidate will usually start their answer with DNS and move on to web servers, which is an excellent starting point for anyone working on webapps. The real answer starts when you begin asking questions, though. For example, let’s say your candidate is talking about web servers: ask them how the answer changes with SSL enabled/disabled, how HTTP keepalives play into it, and what the impact of a CDN or caching layer would have on the request/response. You can use this question as a jumping-off point to discuss when things go wrong and their ability to troubleshoot, as well: what might be going wrong if we’re seeing intermittent HTTP 500 errors? Where should we look?

Be aware that some candidates have studied the question, given how common it is. There is even a Github repo that attempts to answer the entire question in its entirety, so be sure to dig into the candidate’s answer.

Question: Thinking back on everything you’ve done in your career, what stands out as the most significant? Tell me about it.

Inspired by Lou Adler, Hire With Your Head

This is a variation on “tell me about a project,” but sets you up for better followup questions. If a candidate struggles to think of an answer here, chances are good that you have a dud candidate.

The candidate will usually think for a bit, then explain the project in whatever way comes to mind. When you sense they’ve answered it and are struggling for more words, it’s time to start asking your followup questions.

Pick and choose any/all of these as you see fit:

  • What were some of the big challenges in the project that you had to work through?
  • What impact did the project have on your team and company?
  • How did you get this project? Were you chosen, or did you volunteer? Why you? What was your role in the project?
  • How long did the project take to complete?
  • What skills were necessary to get the project done? Did you learn any new skills?
  • Give me some examples of you coaching or helping others during the project.
  • Did you have to influence or persuade anyone to change their opinion during the project? Tell me about that.
  • What was the biggest interpersonal conflict you had during the project? How did you resolve it?
  • What did you like the most about the project? The least?
  • Personally-speaking, how did you change or grow as a result of the project?
  • In retrospect, what would you have done differently?

The follow-up questions allow you focus on the facts and judge the candidate’s accomplishments on merit, rather than their ability to tell a story.

Asking these series of questions a few times for multiple accomplishments is a great way to determine talent and growth of a candidate. If the candidate can only think of one accomplishment to talk about, or if the other accomplishments are a little weak, be wary.

Question: How would you solve X?

Use an actual, real problem. Try to go for open-ended problems that don’t necessarily have a right/wrong answer, as you want to see how they approach problems and reason about them. Avoid any problems that require leaps-of-insight or any knowledge about the specifics of your environment. When in doubt, pick a different question.

Some example questions:

  • It’s important to us to ship a new feature by $somedate. From an engineering perspective, how would you go about ensuring we hit that deadline?
  • Here is an architecture diagram of a new app we’re working on. Do you see any potential issues, risk factors, or ways to improve it?
  • We’re considering ways to improve our code review process. How would you design a code review process for a quickly-growing, multi-disciplinary team?
  • We’re considering a migration from AWS to GCP. What would be your questions about such a move? What should we be concerned with? How would you handle the migration without any downtime?
  • We’re considering breaking our monolith into microservices. How would you approach that problem?

What’s the difference in interviewing a contractor vs FTE?

There is, but that not big of a difference. Let’s make a distinction first.

There are really two situations in which you hire contractors: staff augmentation and expert help.

With staff augmentation, you’re hiring a contractor because you just need more hands. Your contractors probably a look a lot like your FTEs and you might even convert one to FTE from time to time. With expert help, you’re hiring a contractor because of their specific expertise in a certain topic–an expertise your team lacks or isn’t up-to-speed on.

If you’re hiring for staff augmentation, treat the interview like an FTE interview. If you’re hiring for expert help, treat them like a vendor–ask for client references/testimonials, prior example work, etc.

When interviewing engineers, how deep should you assess their technical chops?

Probably not as deep as you’d think.

Many companies will spend the majority of an interview drilling in deep on a candidate’s technical ability, and spend comparatively little time assessing their interpersonal skills, which are arguably more important than the technical skills (especially at the senior IC level).

You can pretty easily get a sense of a candidate’s ability in about half an hour. The rest of the time is typically spent finding the depth of their knowledge in various areas. So how deep should you go? As deep as you need until you’re comfortable making a yes/no decision. Some people can come to that in an hour, some need a few hours. It all depends on your team culture and the specific needs of the role.

How to hire for highly specialized roles

This depends on what your situation is. Do you need someone who can immediately begin teaching your team how to improve at the speciality, or do you need someone who will take the share in the work with the rest of the team?

In the first, what you’re looking for is an existing domain expert. In the second, you have the option of training someone to become that domain expert. The second is much easier to do.

If you’re hiring for the expert-level role, look at other companies with a similar role and see what you can do about enticing one of them to join your team. You also have the option of hiring a consultant (who tend to focus more on specializations than FTEs do) here.

If you’re in the second scenario, reframe your thinking from, “Does this person have these specific skills?” to “Does this person have the capability to learn these skills?” With an existing, capable team around them, you can create your own domain experts and that’s so much easier (and less expensive) than hiring existing experts.

Interview processes in a small team

Hiring for a small team can be tough when there aren’t enough of you on the team to do what’s usually thought of as a standard interview.

The main reason why you involve so many people in interviewing a single candidate is because you want as many perspectives as possible. When you have a very small team (such as two people), the failure mode is that you only get two perspectives. That said, if you both like the candidate, then maybe there’s not really an issue.

My advice is to just do the interview like you would with a larger team: schedule multiple slots, and each person handles more than one interview topic. I’d also focus on interpersonal skills over raw technical ability. When you have such a small team, getting along well is more important than technical skills, in my experience.

Interviewing someone when you don’t know anything about their role

It’s not uncommon to find yourself in a position of having to hire someone for a role you know little-to-nothing about. Perhaps the most common occurrence of this is when engineering managers have to hire their first DevOps Engineer. Hiring for a role you don’t know personally know about can be a daunting proposition, but thankfully, it’s not too difficult of a challenge with a little bit of work.

My tips:

  • Go heavy on behavioral questions.
  • Focus on past experience when it comes to tech. Ask them to explain past projects. It’s okay to admit you don’t know much about what they do (and that’s why you’re hiring someone). Ask them to explain why they made certain choices over others, how they came to their decisions, etc.
  • Pay an outside person to do the technical bits of the interview. You can probably find someone in your network that knows this role (such as someone on a colleague’s team). Pay them (and pay them well!) to do some interviews for you and provide feedback on candidates.

That Time I Interviewed With 22 People AT ONCE

Some years ago, I interviewed for a position and was invited to the on-site. My recruiter walked me to the conference room where two people were waiting. After exchanging greetings, one of my interviewers says, “We’re still waiting on a few more people, they’ll be here soon.”

Another person walked in. And another. Yet another.

They just kept coming. Seemingly no end!

Finally, everyone had arrived, and so I counted: 22 people! I later learned that this was the entirety of three different teams, all to come interview me, a lowly engineer, for a couple of hours. Insanity.

This is the panel interview.

What is the panel interview?

An interview with more than three or more interviewers. This format is an easy way to get several perspectives and stakeholders into the interview process without requiring several days of non-stop interviews. It isn’t without its challenges, though.

The panel interview can take a few different formats:

Presentation: The candidate presents something to the panel and answers questions about it. This is a bit like the thesis defense common in Masters and Doctoral programs. For example, for a senior engineering role, you might ask the person to prepare an architecture for a sophisticated, hypothetical application and then present it during the panel interview.

The Gauntlet: The candidate answers question after question from the panel on a range of topics. This can be an incredibly intimidating format for the candidate due to the rapid-fire, all-over-the-place nature of it.

Topical: The candidate answers questions related to a narrow range of topics (or one topic). This is the most common panel format. For example, you might discuss the merits of different deployment methodologies in-depth.

The Good

It’s true that a panel interview allows you to get multiple perspectives in the interview at the same time. Rather than two different interviewers asking a similar question from a different perspective, you can get both of them in a single interview and explore both facets of a topic in one interview. This is considerable upside for everyone, as you can get a more holistic answer from a candidate.

I once gave a presentation to a panel of six people. Ahead of the panel, my take-home work was to design a monitoring architecture and strategy for a large-scale network spanning continents and dozens of stakeholders with different use cases. During the panel, I presented my architecture and answered questions about it for hours. Overall, it was one of the best interviews I’ve had. The presentation format was a very useful way to discuss my experience and ability on something that would have a sweeping impact on the organization.

When done well, a panel interview can also save quite a bit of time. Five people in a single panel interview can easily cut the interview time to two hours, rather than having to deal with five, one-hour interviews.

The Bad

There are some challenges with a panel interview, though.

They can be rather expensive for the company to conduct. Putting multiple engineers into an interview at the same time means business-as-normal could completely halt during that time. Some companies might find that they’re putting their entire team into an interview at once, so everything stops while the interview is in session.

A more significant concern is when your interviewers aren’t experienced in how to run a panel interview effectively. When you have five or six people on a panel, it can be difficult to determine who is asking questions and when leading to uncomfortable situations for the candidate when two interviewers start trying to ask questions at the same time. Because of this, a panel interview can be somewhat unwieldy without proper preparation.

Lastly, the nature of a panel interview is inherently intimidating, and without careful attention paid to the experience, it can feel awkward (at best) or even adversarial (at worst) pretty quickly. Having a moderator with a friendly demeanor is an excellent way to make sure it remains congenial, and the candidate doesn’t get completely overwhelmed by it. When I interviewed with that panel of 22 people, the room was so packed that there was standing room only. Some people didn’t speak at all, but there seemed to have been no preparation by the panelists for the topics of discussion. As a result, I was answering questions that had no rhyme or reason to them and continually turning my head back and forth to answers questions from my left and my right. Overall, it was not a great experience for myself, and I can’t imagine it was a valuable interview for the people present either. With only a few changes, it could have made for an incredibly useful interview for both sides.

Takeaways

A panel interview is a great interview format, but it isn’t without its challenges. With some careful preparation, however, it can be a valuable tool for interviewing some roles.

The key takeaways for a great panel interview are:

  1. No more than six people on a panel. Any more than that and it becomes difficult to manage.
  2. Go longer than an hour so you can really take advantage of having multiple perspectives. I recommend aiming for 2-3 hours (with bathroom breaks!).
  3. Every panelist should be prepared. Ensure everyone on the panel knows what the topics of discussion are and what they are assessing. Meet as a group before the interview to ensure everyone is on the same page.
  4. Be cautious about the candidate’s experience, as a panel can be intimidating and overbearing even for the most extroverted candidate.

How To Recruit Senior Engineers

One of the biggest challenges in our industry is how to hire more senior engineers. It’s a big challenge in a fallow market and perhaps counter-intuitively, even tougher in a strong and competitive market (eg, San Francisco, New York City, London). But the good news is that it’s not impossible–it just requires a different strategy.

The first thing you have to know about hiring senior engineers is that they don’t apply to your job postings cold.

A senior engineer (by definition) has been in the industry for a while already, and will generally have a strong personal network of current/former colleagues. They don’t need to hit the job boards to find their next job–they just grab lunch or drinks with an old coworker. They’re never really on the market, because they’re never really actively looking for a job. Many senior engineers will change jobs without ever applying anywhere or even conducting what we’d normally think of as a job search.

This stands at odd with how you hire more junior engineers (who apply to your job postings en masse). To hire senior engineers, you have to change your strategy.

Senior engineers must be recruited

You have to find them and convince them to join you. Unfortunately, there isn’t a magical website full of senior talent looking for their next job, so hiring senior engineers takes some time and patience. Be prepared to be flexible with your interview process: a senior engineer who is happily employed isn’t going to apply to your role just because you invited them to do so.

Where your hiring process for a junior/mid level engineer looks this like:

Post the req on job boards –> Candidate submits application –> You interview them

The process for hiring a senior engineer looks something like this:

Find them –> Gauge their interest in your role –> One or more informal interviews over coffee/drinks/lunch –> They formally apply for the role –> Standard interview process takes over

Where to find them

Okay, so senior engineers aren’t magically flocking to you and you know you have to go find them yourself. But where do start?

There are two main methods you can use: referral and cold, direct outreach.

Referral

This is by far the most effective method of recruiting anyone, but works especially well for senior engineers. It’s also really simple: ask your colleagues and extended network for who you should talk to.

Start by talking to your own team and ask, “Who do you know that we should hire for this role?”

A variation, if you have some experienced engineers on the team already, is, “Who should we hire but probably can’t get?” This one is especially good because chances are, your experienced engineers know a few amazing people that, in their mind, could never be convinced to leave their current job. We know better, of course: people can be enticed to change jobs for many reasons, and it’s all a matter of understanding what someone wants and then giving it to them.

Next, talk to other engineering managers, whether in your company or at other companies. They are often aware of great candidates they can’t (or couldn’t) hire, but that doesn’t mean they wouldn’t be a great fit for your roles. So ask them who they know might be interested in a new job.

Cold, direct outreach

Sometimes you hit a wall with referrals, and building your own network is a long game. For that, we have cold, direct outreach. This isn’t nearly as effective as referrals, but it still works great, and is best done in parallel with the referral strategy. Caveat though: your success rate won’t be great. If you’re doing well, you’ll get a 50% initial response rate. Don’t get too discouraged if you’re only getting 20-30% initial response rate.

Some ideas for where to find senior engineers:

* Conferences & Meetups: They’ll likely be speaking, not merely attending–though not always (there are many amazing engineers who don’t speak at events)
* Podcasts: Look at the top engineering podcasts and see who hosts them and who has guested on them
* Blogs: Many great engineers maintain an active blog
* Github, StackOverflow, Hacker News: Top contributors on these sites can often be wonderful hires.
* Niche sites: If your role involves a niche technology (eg Clojure), then involve yourself in that community. You’ll find great engineers rather quickly.

How to reach out

Once you’ve found a few people you’d like to reach out to, whether they came via referrals or your own direct research, it’s time to make contact. There are a few rules you should keep in mind to increase your chances of success and to not be a bother to the person.

Rule 1

Be a company they’d want to work for. This can mean different things to different people. How do you find out what they want in their next role? Simple: you ask them.

Rule 2

You must do the outreach, not a recruiter/HR. Recruiters, rightly or wrongly, have a reputation for taking a spray-and-pray approach to finding candidates. Even if your recruiters don’t take that approach, there’s no way someone outside of your company would know that. However, when an engineering manager makes direct contact themselves, it shows real effort has been put in, and you will be much more likely to get a response to your email.

Rule 3

Your outreach must be extremely personalized. Do your research. If you’re looking at someone who is well-known in the Rails world and has spent a huge chunk of their career working with Rails, there’s a good chance they want their next role to be something involving Rails–don’t reach out about a completely unrelated role. Likewise, if they have any recent, public work, it can be a good hint at what they might be interested in working on next.

 

Recruiting senior engineers this way will take time, so be patient and keep at it. I know engineers receiving a dozen or more emails from recruiters a day and they ignore most of them. The key to standing out is doing your homework and making your outreach about them.

Building An Engineering Interview Process

Creating an interview process can be tough. An effective interview process is a series of hard tradeoffs. On the one hand, you want to figure out how well a candidate will into your team and their ability to the do the job. On the other, the candidate wants to know how much they’re going to enjoy working with you and the role. Balancing these needs can be expensive, time-wise.

Spending eight hours interviewing a candidate comes with a non-trivial cost to you: that’s several hours taken away from development time, multiplied by the number of candidates you’re interviewing. That also means the candidate is spending eight hours with you, on top of any other companies they’re interviewing with and their day job. But, alas, the less time you spend, the worse the signal is for both you and the candidate. Your goal is to spend as little time as possible to get the highest signal possible. This is harder than it sounds.

Candidate experience is essential

One caution before we get into this: pay attention to the candidate experience during the interview. Even if the candidate doesn’t get an offer, you want them walking away talking about what a great team you have and how great their experience was. They may not be a good fit today, but they might be in the future. And never forget that candidates talk: if you have a crappy interview experience, other potential candidates will find out. That’s bad news for you.

So be kind. Be thoughtful. Make your interview process a delight to go through.

A Starter Process

If you’re not sure where to start with an interview process, I’ve built one out for you. This works well as-is, but please feel free to modify it to suit your own needs. Here’s the high-level:

Step 1: Job description & team prep
Step 2: Screen resumes/applications
Step 3: Phone/video screen
Step 4: Homework
Step 5: On-site
Step 6: Review feedback from interviewers
Step 7: Make an offer/send the rejection
Step 8: Collect feedback from the candidate

Step 1: Job description & team prep

Hiring someone starts well before you ever post the job somewhere and start interviewing candidates. I’ll cover writing great job descriptions and how to figure out what you want in another post, but suffice it to say, your team plays a big part in determining what sort of work this new role will be doing and the kind of person that would fit well. When it comes time to start interviewing, your team needs to be crystal clear on what matters for this role and what doesn’t.

It sounds silly, but one of the most common challenges you face in hiring is not knowing what you want. “How could I post a job and not know what I want?! I wrote the job posting!” you might be thinking. Unfortunately, it’s not uncommon to have a laundry list of “desirable skills and experiences” and for those to cloud your judgment. Before long, you end up with an ideal candidate profile that is more fiction than reality. Without getting clear about what you need and what you want, you may end up rejecting many otherwise-great candidates simply because they didn’t meet your nebulous, vague idea of what you’re looking for. Every interviewer must be on the same page here.

Before you begin interviewing candidates, hold a team prep meeting. This is a great way to go over the job posting and confirm what your team wants in the new-hire. I also recommend going over some effective interview questions for assessing various skills. Each person involved with interviewing should know what they’re going to be discussing, why, and what they’re looking for. Be sure every interviewer will be taking notes during their interviews — this is vitally important to reviewing candidates later.

Step 2: Screen resumes/applications

Once your job is posted, resumes will (hopefully) start coming in. Depending on the role, your location, and a few other factors, you may have a deluge of resumes…or a trickle.

Either way, you have resumes to screen. This is a pretty quick process, though it can be tiresome and daunting, especially if you’ve been in the situation of having a few hundred resumes to sift through.

Who does the screening?

Some managers like to have an engineer on the team screen resumes, while others prefer to handle this task themselves. Either way works well (though I, personally, prefer to have managers screen them).

How do I screen resumes?

I’ve found there are two ways to screen resumes: filtering out and filtering in. Both are useful, depending on what you’re trying to hire for.

Filtering out

If you have hard technical needs, this is an effective way to screen. Review each resume, looking for those specific experiences you can’t do without and you can’t train for. Toss any that don’t meet your criteria. The rest move on to the phone screen. Easy as that.

Of course, the more hard requirements you have, the fewer resumes that make it through this step.

Filtering in

This is an effective way to screen for most roles, as it biases to the question, “Does this person have the potential ability to do this job?”

It’s much harder to screen this way since you’re not comparing against some set criteria. The upside is that many more candidates will make it through this filter.

Examples of each approach

Filter-out: Your team works on search technologies, making heavy use of Elasticsearch. You’ve recently lost a member of your team and are replacing them. You need someone already familiar with Elasticsearch and can’t spend the time to train anyone. A filter-out screening approach would toss any resume that doesn’t have Elasticsearch experience.

Filter-in: Your team works on search technologies, making heavy use of Elasticsearch. You’re adding new team members to an already stable and productive team. A filter-in screening approach wouldn’t pay much attention to Elasticsearch experience but would pay more engineering to general engineering background.

If you’re not sure which approach to use, go with the filter-in approach.

I encourage a response to all resumes at this stage, including rejections. Rejecting a candidate at this stage can be done with a form response and doesn’t need a personal response (unless the candidate came in via referral — then it should be a personal response).

Step 3: Phone/video screen

Time: 20–30m

Once you have candidates that have passed your resume screening, it’s time to talk to them. I prefer video calls instead of the phone here. With video, you both get the benefit of non-verbal communication. This call doesn’t need to be very long.

What to talk about

Ask what they’re looking for in their next role. Get to know their interests in a company and the kind of work they’re looking for. Talk about how your role maps to their interests and desires. The candidate will probably have some questions about the role as well.

You may be inclined to do a short coding test here, such as fizz buzz or another engineering puzzle/brain teaser. I recommend against it. These sorts of tests are low-signal, high-pressure, and rarely map to actual work. When was the last time you had to implement a sorting algorithm or traverse a linked list? Not since college? So don’t do it here.

Instead, ask questions about their experience. Open-ended questions are very effective: “Tell me about a recent project you worked on,” and keep asking more questions about it.

Not sure how to dig into something? Here are some questions to follow up with:

  • How did you come to the decision to do X?
  • What was challenging about X?
  • What was the most interesting thing about X?

Remember that your goal of this interview isn’t to decide whether or not to hire them, but whether you want to continue and see how well they would fit into your team and company. Keep in mind that the candidate is probably interviewing with more than one company; you want to leave a great impression on them, so they want to continue with your company.

If you want to take them on to the next step, a quick introduction to your interview process is a good idea (phone screen, homework, on-site), as well as the typical timeline.

Lastly, ask about compensation expectations. Some candidates like to leave that discussion to the very end of the interview process, while some like to get it out of the way up front. If you know your company doesn’t pay market rate, it’s worth letting them know your budget for the role now rather than taking them through to the end of the process and finding out you’re nowhere near their price. Do not ask about their current/past compensation: not only is it now illegal in some US jurisdictions, it’s not actually helpful to you. If you really must know, just ask, “What are your compensation expectations?” They may turn around and ask about your budget — answer truthfully. There’s no need to be cagey about any of this.

As soon as the candidate has spoken with a person, you’re out of the territory of form responses. If you reject a candidate at this stage or any subsequent one, either the hiring manager or the recruiter should do so personally.

Step 4: Homework

Time: 2–3 hours (for the candidate), 5–10 hours (designing the homework), 1 hour (reviewing the homework)

Asking a candidate to do homework is a contentious point with many people, and there are great arguments for and against it. On the one hand, it is one of the strongest signals you will get from the entire interview process. On the other, many people don’t have time to do homework due to work schedules and family commitments. In essence, by choosing to provide homework, you’re making a tradeoff: in exchange for higher signal, you’re potentially filtering out entire groups of candidates.

That said, I think the benefits outweigh the downsides, and the downsides can be managed and mitigated to a large extent. One of the biggest issues with homework is the time commitment it requires, so your homework assignment must be very well time-boxed, preferably to no more than 2–3 hours.

Designing an effective but not time-consuming homework assignment is tricky. The best homework tests real-world work, so make it as close to the work they’d do for you as possible.

Most candidates will want to go above-and-beyond, spending way more time than you expect. On top of that, I’ve seen several homework assignments that went untested entirely, or were only tested by the person who wrote it. The best test: pay a couple people to go through it and provide feedback on it. For a few hundred dollars, you can be sure your test is both effective and not time-intensive.

A caution

Some teams like to have the candidate write an actual feature or fix an actual bug. While that is as close to real work as you’re going to get, it has two enormous problems:

  1. It requires more context and knowledge of your app than they have, so you are unlikely to see them at their best.
  2. It is valuable to you on its own. That is, you benefit from the task even if you don’t hire them. This amounts to free work. Not cool.

If a candidate submits homework, someone should review it. Rejecting a candidate without reviewing their homework assignment is disrespectful of the candidate’s effort. If your homework takes too much time to review, consider redesigning it, so the review is easier.

Step 5: On-site

Time: 4.5 hours

The on-site is where you finally get the candidate and your team together and really understand how well they can do the job and mesh with your existing staff. Don’t forget that the candidate is also interviewing you here. This is not one-sided — leave room for the candidate to get to know you and your company in every interview.

I’ve put together a sample schedule below. Some notes about this schedule:

  • Modify it to suit your needs! This is only a sample.
  • Pay attention to candidate experience. The small things can have a big impact. For example, have a couple bottles of water in every interview room. Restock as necessary; your candidates will be thankful for it.
  • Send the schedule to the candidate a couple days before the interview, so they know what to expect. Even better if you can tell them ahead of time who they’ll be speaking with, so they have time to look them up on LinkedIn/Twitter/Google. It’s much easier to build rapport when you can establish common ground beforehand.
  • If you schedule a lunch, be sure to ask the candidate about any dietary restrictions when you send the schedule. You really don’t want to take a candidate to a lunch they can’t eat.

Some special notes about lunch:

Rule #1: no alcohol. Period. End of discussion. That goes for your team as well. Alcohol during an interview will signal the worst of tech culture to the candidate, and if it happens that the candidate doesn’t drink alcohol, they may feel awkward — especially if your team is all drinking and they’re not.

As for who to go to lunch with, pick two of the previous interviewers. If possible, not the manager or recruiter. Going to lunch with the manager is too high-pressure since they are perceived as having the final say on whom to hire (where that’s actually true or not for your company is irrelevant). Going to lunch with the recruiter isn’t helpful to the candidate since they’re not going to be working with the recruiter.

So pick two of your team to go to lunch with them. Why two? If it’s just one team member and the candidate, it’s possible that there’s little rapport with that specific person and awkward pauses are too common. More than two and it’s easy to fall into the trap of your team talking to each other and leaving the candidate out, not to mention it can be a bit overbearing having so many people. The sweet spot is two people, as the social dynamics will allow for conversation to flow more freely.

Sample On-site Schedule

Building An Engineering Interview Process
Sample on-site schedule

Step 6: Review feedback from interviewers

Within a couple of days following the on-site, all feedback needs to be reviewed. I recommend doing this as a group so you can discuss the feedback. This is where the value of the note-taking comes into play: if you don’t have actual notes to review, then you’re only able to discuss instincts and gut feelings, which we know aren’t accurate. Without notes, you can’t suitably review a candidate.

Step 7: Make an offer/send the rejection

Offer

Oh, happy day — you found a candidate you want to hire! But you’re not done yet — the offer hasn’t been accepted, and you haven’t really made a hire until the candidate is sitting at one of your desks, working for you.

Making an offer is straightforward and should come in two conversations:

The first conversation is to call the candidate and let them the company would like to extend an offer. Now it’s time to talk real details: salary, non-cash compensation, leave time, benefits, etc. Learn what is important to the candidate — not everyone cares about maximizing salary.

The second conversation is the verbal offer itself. Call the candidate and let them know the details of their offer. After the call, send them the offer letter via email, along with all details of the benefits package.

Some companies have what’s known as an “exploding offer,” which is an offer with a deadline attached to it. Please don’t do this. It’s unfair to the candidate who might need time to wrap up other interviews or discuss it with their family. Instead, ask about their expected timeline for an offer. No one is going to sit on an offer for months and expect it to be around still, so don’t worry too much about it. Just keep communication open with the candidate, and it’ll be fine.

After the candidate has spent time reviewing the offer, some negotiation may be needed. If everything is great for both sides, then you end up with an accepted offer. Congratulations!

Rejection

Rejection at this stage is hard on the candidate. At this point, if you’ve used my schedule, the candidate has potentially spent eight hours with you, potentially more if they put additional time into the homework. They’re invested at this point.

While you may casually reject a candidate, know that it is anything but casual for them. Any rejection deserves a personal response from the hiring manager. A vague rejection (“Sorry, the team has decided it’s not a good fit at this time”) is rough, especially if the candidate felt the interviews had gone well. Have empathy for their position.

If the basis for the rejection was technical in nature, you should tell the candidate so and make recommendations about how they can improve. If the rejection was non-technical, that’s a bit touchier. Your HR and Legal teams will likely have a complete fit over this — I recommend asking them to walk you through what the risks are. Just remember, a vague rejection is rough and unfair to the candidate. Keep that in mind.

Step 8: Collect feedback from the candidate

After the entire process is complete, regardless of whether you made an offer or rejected them, you should collect feedback from the candidate. This can take the form of a few questions over email or even a Google Form. The feedback from the candidate is valuable information for improving your interview processes, spotting interviewers that need additional training, and much more. If you’re sending this to someone, you made an offer to, wait after the offer is accepted and make sure the candidate knows it won’t have any impact on their offer.

So there you have it: a starter interview process. What do you think? How does your own process work? How well does it work for you?

No More (Third-Party) Recruiters

We are all familiar with third-party recruiters. You may have used them to find a job, and many of you have probably used them to source candidates. Third-party recruiters (both individual and agencies) serve a very needed function in the tech world: finding candidates for you. They differ from in-house recruiters in that they don’t work for your company and instead work with many companies at the same time.

One of the big selling points of third-party recruiters is that they work with many different companies and many different candidates, allowing them to build relationships with people you might never have come across. This equates to often working with hundreds of companies and potentially thousands of candidates, allowing you to easily and quickly fill a job req by hiring a third-party recruiter. They put their network to work for you.

This all sounds great, and it often works fairly well.

But there’s a nasty downside to the entire model, and it’s a big one: the incentive structure is in their favor, not yours.

How does a third-party recruiter work?

There are two ways third-party recruiters charge for their services: flat fee and contingency. By far the most common is the contingency fee, which is what I’ll be writing about here.

What is a contingency fee? Simply put, they get paid only if you make a hire. The fee is anywhere between 15% and 25% of the new hire’s first-year salary, depending on your agreement with the recruiter. Most agreements come with a guarantee period of between 30 and 90 days, so if the new-hire quits or is terminated during that period, the recruiter will find you a new candidate on their dime.

Here’s the rub

Third-party recruiters are incentivized to find someone acceptable to you, at a price you’re not totally upset about, is likely to stay with you through the guarantee period, and to do all of this as quickly as possible. Third-party agencies are not incentivized to find you the best candidate possible.

I’ll go ahead and take this opportunity for an aside before every third-party recruiter on the planet starts screaming, “We’re not like them!” True, not every third-party recruiting company will lead you astray. There are many third-party recruiters with a strong sense of ethics and a commitment to long-term customer relationships. However, most of them still operate on contingency-for-placement, which sets up the faulty incentive structure at play here. As long as this incentive structure exists, you should be asking who really benefits from the arrangement.

The longer a recruiter takes to fill a req, the more their profits take a hit. If they find a so-so candidate, you’re satisfied with them, and it only takes them a week — that’s a slam dunk on their books.

But what if an absolutely perfect, ideal, pie-in-the-sky candidate was only another two weeks of work away? Many companies will gladly let a req sit open for another few weeks if it means getting someone substantially better. After all, you want to keep this person for as long as possible, and the upside to your business is immense.

But you’re out of luck because this game isn’t in your favor.

A tale of two candidates

Let’s look at two hypothetical candidates.

Candidate A was presented to you the second day you hired the recruiter, and they’re pretty good. You’re not thrilled with them, but they’d probably be fine. They will cost you $100,000 in salary.

Candidate B was presented to you three weeks after you hired the recruiter, and they’re phenomenal. You believe they’ll dramatically improve the whole team. They’ll cost you $140,000 in salary.

If you hire Candidate A, the third-party recruiter gets paid $25,000. If you hire Candidate B, though, the recruiter makes $35,000 — not a whole lot more. Especially considering that the recruiter had to do substantially more work finding them.

Your upside from the extra work is huge. But the recruiters? Not so much.

On top of that, third-party recruiters are often working for multiple companies at the same time. It’s entirely conceivable that a recruiter might have run across your Candidate B in the first week, but had them interviewing at another company and didn’t present them to you because the candidate was pretty happy with how things were progressing at the other company. You might have lost that great hire entirely.

So what now?

Using a third-party recruiter is great for convenience sake, but it isn’t the only way to hire great engineers. In fact, I’d argue that it’s one of the least effective methods of finding the best engineers.

My first rule of hiring is this: control your own pipeline.

Having control of your own candidate pipeline aligns incentives with everyone involved. By working with your own engineers and in-house recruiters, everyone is incentivized to only hire the best people for your company and roles, rather than make a quick-but-not-great hire.

Imagine if every time you opened a new req, you had a dozen people in mind right away, and all of them were interested in talking. Here are a few ideas to get you started:

  1. Be active in the tech community. Send your team (and go yourself, too) to speak at meetups and conferences. Host engineering-centric events at your office.
  2. Ask your team for referrals to other engineers they know on a regular basis. Include the “ungettable” people, too: you never know when someone is interested in changing jobs.
    Cultivate relationships with off-market people. So important it’s worth mentioning twice! People change jobs for all sorts of reasons, and the best engineers rarely hit the open job market.
  3. A recruiting machine is like a flywheel: it takes a bit to get it moving, but once it does, it works reliably. If you want to have the best people for your organization, you’ve got to control your own destiny and find them yourself.

What do you think? Do you work with third-party recruiters? How has it gone for you?