My Grand Tour of Pentest Interviews

Late last year, I began looking for a new job.  Earlier this year, I finally got one!  I was interested in branching out into the broader world of penetration testing and red teaming, with more external clients and more broadly-scoped sorts of engagements.  This was something of a sell to prospective employers though.  I do have close to a decade of infosec experience, but only a few years of that is pentesting and I’ve always been an in-house pentester doing mostly web app and mobile stuff.  That means that I am something of a noob when it comes to breaking in from the outside; I’m familiar with a lot of the tech and methodology, just haven’t done a lot of it hands-on (outside of CTFs and stuff like that).  I’ve been in the broader industry for a while, meaning my salary requirements are a little higher, and I absolutely wasn’t going to relocate again so soon after my last move for my old job.

All of this and extremely high demand for pentesters at the moment meant I went through A LOT of interviews.  Some of them broke down over salary expectations.  Some of them I quit early because I could tell it wasn’t what I was looking for.  Some of them weren’t budging on relocation.  One I completely hosed myself on because I bluffed too hard during the salary negotiation phase.  At least one of them probably thought I was a complete dumbass.  But in the end, one employer won out and I’m now happily hacking clients, mostly from the comfort of my own home.

Besides having lots of experience being interviewed for pentest jobs, I also have some experience in interviewing people for pentest jobs.  At one of my previous employer, I was involved in telephone screenings and in-person interviews of a dozen or so different candidates to join our team there.

Because I went through so many different interviews recently and have experience trying to assess pentest candidates, I figured that put me in a unique position to grade these different companies and throw in my own opinion on the best way to do it.

My intent is half to just be amusing for those who are curious about how different companies are interviewing people, maybe those trying to find out what to expect the next time they start looking for a new gig; but I’m also writing this and hoping that some recruiters and hiring managers will see this.  I hope this will give you some insight into how your competition might be assessing candidates, what you’re doing right in your own process, what you’re doing wrong, and how you could be doing it better.

**ALL NAMES HAVE BEEN ANONYMIZED TO PROTECT MY FUTURE EMPLOYMENT PROSPECTS**

(Names were chosen using this random Company Name Generator)

Bigity

To start with, let’s talk about one of my previous employers, codenamed “Bigity,” and how we used to do technical assessments and interviews.  The first phase, like with most companies, is one of the recruiters from HR would reach out to the candidate.  This was usually in the form of a brief phone interview, 30-60 minutes, in which the recruiter would mostly ask questions about background, experience, salary expectations, and other non-technical stuff.  If the candidate passed the sniff test here, and there weren’t any show-stoppers, like no being willing to relocate or asking for too much money, the HR folks would forward their résumés on to us.

The next phase would be a telephone interview with one or more of us on the pentest team.  This interview would mostly be stump-the-chump questions, by which I mean we’d ask a bunch of infosec quiz questions; things like asking what’s the difference between encryption, hashing, and encoding or explaining what cross-site request forgery is.  Most candidates get weeded out at this point.  Not that we put much faith in your ability to remember infosec trivia; we realize you’re under the gun and there’s plenty of stuff you don’t memorize and offload to just googling for it when you need to know it (I know I do)…but if you ask questions right, especially ones that lead you through someone’s thought process, you can usually tell if they’re full of crap.

My friend Garrett’s favorite Bigity interview story is from the time he was doing a phone screening, would ask a question, the candidate would stall, he would hear the guy furiously typing, then start reciting the answer.  It wasn’t hard to figure out what was going on, so Garrett started googling along with him and could see he was just reading off the first search result for each question asked.

For those who passed the telephone interview, who would bring them onsite to our office for a couple of in-person interviews with team members and the management, take them out for lunch, and the culminating event was a technical assessment.  The technical assessment was in the style of a capture-the-flag (CTF) challenge, where the candidate would have to attack and gain access to different web apps and servers.  My friend Gabe was the mastermind behind these challenges and would host them as VMs or Docker containers, either on a laptop or on Digital Ocean VPS’s.  We’d give the candidate a laptop with Kali Linux to use and then observe them as they attacked these targets and occasionally feed them hints.  In most cases, candidates were flying to our location to interview, so they had all day to be with us.  With that in mind, we tried to be as generous as possible and give the candidate several hours at the end of the day to do the technical assessment.

The technical assessment was really the make-or-break event, in our experience.  We had several cases where people sailed through the phone interviews, had extremely impressive résumés, really sounded like they knew what they talking about, convinced us they’d probably be eligible for a principal-level position…then completely fall to pieces in the technical assessment!  I’m talking people who didn’t even know how to do an nmap scan!  They might still get an offer from us…but it was going to be at a much more junior level.

I was participating in these interviews at a time when the pentest team was expanding significantly, so we interviewed a lot of people.  In my experience, the best candidates were those who already had pentesting experience (naturally) or those who might only have some infosec job experience, but who’d gotten the OSCP cert.  We posted a job listing in the OffSec OSCP alumni forum and it was one of our best sources of qualified candidates.  I went through this same interview process before I’d gotten my OSCP and did OK, but it did give me some good reasons to sign up for it and take it.

Verdict: Bigity’s process is fairly speedy, which is good, as I’ll discuss more later.  A long in-person interview is also something I like, as it gives a better opportunity to feel out what a candidate is like, personality-wise.  It also gives the candidate a chance to see what the culture and people are like at your workplace.  Having the technical assessment take place in-person also has some pros and cons.  As a facilitator, watching over their shoulder, you get to see step-by-step how they approach a pentest and give more junior candidates the little nudges they need to succeed (that you’d give them anyway if you were their team leader or mentor).  The downside is that, being in-person during office hours, there’s a pretty hard time limit on it.  One thing I have to ding Bigity on is reaction speed.  In our case, even when the team and hiring manager knew a candidate was good and wanted to make an offer immediately, we sometimes lost them because HR would take longer than a week before they’d even reach out to candidates to tell them we were making an offer.

4/5

Linefix

This one followed the somewhat standard formula: non-technical screening with a recruiter, technical interview with a team member or manager, a technical assessment, and a final interview with team members/management.  Most companies would ask for a written report on the technical assessment, but Linefix wanted me to put together a client presentation and deliver it to their team over a web meeting.  This part, I liked; presenting it not only makes sure the candidate is literate, but that they have the necessary social skills to put them in front of clients.

The only problem I had with this was that the technical assessment was a VulnHub VM.  Not like they gave me a VM that’s also available on VulnHub…they told me to go to VulnHub, download it, get root on it, and do the presentation on that.  If you too are a fan of Vulnhub, then you’ll know that there’s a big problem here: the fact that the VulnHub page for any CTF is full of links to detailed walkthroughs of how to own it!

Having anybody download a VM image has the inherent problem that a candidate could just mount the virtual hard drive and comb the file system for the flag file…but at least that way you’d catch cheaters when you find out they’re completely clueless as to how to actually exploit the machine the intended way.  Most VulnHub walkthoughs walk you through the entire thought process of the attacker, though.  At least they politely asked me to not rely on walkthroughs and try to do my best on my own.

It wasn’t too hard, but I did run into a part that seemed to require finding a buffer overflow in a binary.  I’m a newb at buffer overflows and only really know the limited techniques you’re taught in the OSCP labs, so I peeked at one of the walkthroughs to make sure I wasn’t crazy.  It turns out it required a much simpler exploit technique that I didn’t know before.  After that, I was able to finish it all on my own.  I freely admitted in the presentation I gave them that I did use the walkthrough for that part, since I’m a n00b when it comes to exploit dev and reverse engineering.  After the presentation was about a half-hour or so of technical questions from members of the pentest team and that was the end of the interview cycle.

Verdict: So while the challenge was fun and interesting, and making a presentation and delivering it is definitely a great way to gauge a candidate’s soft skills, I feel it’s a little lazy to rely on someone else’s VulnHub submission (I looked up the author of the VM and he’s never worked for this company or anyone affiliated with them).  Also, what stops someone from not even trying, just reading one of the walkthroughs (that give very detailed explanations of what to do and why it works when hacking this VM)?  It would be very easy for a low-skill pentester to BS their way through this one.  If they would create their own, in-house challenge VM to test candidates with, they’d be hitting that sweet spot of quality and speed.

3/5

Funlux

This company was probably the closest to hitting the right balance of speed of interview and quality.  After speaking with a recruiter, I had the usual technical screening call with a senior member of the team.  It was very conversational and not ball-bustingly hard or anything.

The next phase was a scheduled technical assessment.  I had to choose a 48-hour period and provide them with my home IP address, during which they would spin up an AWS box with their challenge VM image and whitelist my IP so only I could touch it.  This challenge actually had a storyline and theme to it, with a clear mission to accomplish and a final report to write and hand in.  It was actually a really fun little CTF challenge to root.  Accomplishing the mission, I wrote up an attack narrative report and sent it for them to review.

In short order, the recruiter got back to me and scheduled the final interview, which was a discussion with two hiring managers about my background, skills, etc.  The entire interview process was all over internet or phone.  I can’t really dock them for not having any in-person component, as this particular company is almost exclusively work-from-home, so there wouldn’t really be any office for a candidate to be flown in to.

Verdict: As I said, this one came the closest to being perfect.  The technical challenge was custom-made and hosted (no chance of anyone looking up a walkthrough or cheating by mounting the virtual disk image), only four phases from initial recruiter reach-out to final interview, and each step in the process was very prompt.  This would’ve been a perfect process if they’d asked me to present the findings over a phone call or videoconference, sort of like how Linefix did in it’s final phase.

4½/5

Tincore

A recruiter from Tincore found me on LinkedIn and we had a phone chat about the position and my experience.  After that, I had to use some online system to record myself giving answers to canned interview questions, which took about 45 minutes or so, if I remember right.  I’ve been told by others that this is getting more common for initial candidate screening, but it feels really impersonal and robotic to me.

The next step was downloading a VM to conduct their technical assessment.  It was a custom-developed CTF VM and definitely the hardest of all the challenges I undertook.  Without giving away too much, I got stuck on a portion where I was expected to escape from a container.  I actually asked for a mild hint (just to ask if something I was seeing in it was real or a troll) and finally had a “duh” moment when I realized the escape was right in front of me the whole time.  The final portion was also time-consuming, as it forced me to write a decryption program in Go.  This was actually the first time I’d messed with Go; it’s like C from the future.  Maybe not surprising considering that Ken Thompson and Rob Pike of Bell Labs/Unix/C fame developed it for Google.  Anyway, it reinforced why I hate crypto but was a really interesting challenge.

There were two downsides to this, however.  One is that it’s pretty trivial to just mount the VM hard disk image and pick it apart.  They even politely ask the candidate not to do this in the assessment instructions.  Of course, being able to view all the protected portions of the filesystem may or may not help you out and further interviewing can bear out whether a person knows their stuff or not.

The second downside is that this particular VM was a hog!  It was several gigabytes large to download off the bat and it was set to use several gigabytes of RAM when running in VMware.  I didn’t test out if it really required that much RAM (I suspect it probably didn’t), but still, challenge VMs should be as slimmed down as possible for ease of downloading and running on modest hardware.  You should consider that people doing this are probably using Windows or macOS as their primary OS, then running a distro like Kali to have all the pentesting tools, and then running your challenge VM to hack against.  Not everyone’s got 16 gigs of RAM in their machine to accommodate all that.

After that I finally got to have a technical interview with two pentesters…and they proceeded to frickin’ destroy me.  It was the most brutal Q&A interview I had during this whole exhausting job search.  The recruiter was nice enough to call me the next week to tell me how they didn’t feel I was competent enough and would be pursuing other candidates.  Thanks, I guess.

(My bruised ego did get a little feeling of schadenfreude when I heard from a friend at a large company that they had previously hired Tincore to do pentests for them and that Tincore had farmed it out to low-skill offshore workers who tried to pass off vuln scans as a pentest.  Maybe that’s why they were so hard in their interview process, trying to raise the quality of their consultant pool and restore some reputation.)

From what I understood, had I not looked like a moron during that final interview, there may or may not have been more phone interviews and eventually an onsite visit to their office for a final round.

Verdict: I give them massive points for a very interesting and challenging assessment and an extremely tough interview, but I have to subtract some for the resource-intensive VM, the cheating potential, the very cold and impersonal webcam video résumé thing, and for the length of the whole interview process.  If you total up all different interviews, that would make for at least a six-phase process.  On one hand, that allows for ample time to feel out a candidate, decide if they really know their stuff, and if they’re a good fit for the company.  On the other, in the highly competitive world of infosec hiring, you’re almost certainly not the only company they’re talking to and 6+ phases is a lot of time for a company with a faster interview process to make an offer before you and snatch up the best candidates first.

3/5

Voljob

This was a very prestigious pentesting company and I was super-excited to hear from one of their recruiters.

…and then I found out what the assessment was.

The first part was essential a multiple-choice English literacy test.  While it seemed odd, I can understand.  Nothing is going to turn a client off faster than a consultant who can’t spell or complete a grammatically-correct sentence.  I’m pretty sure I did ok on that one.

The second part was a code review.  I knew I was screwed then!  While I did what we called “code-assisted pentesting” in my last job, and I can read and understand most code, I am in *no way* qualified to perform a detailed code review and catch every potential problem or error.  I’d like to be a little better in it, but I have no wish to do it for a living, honestly.

The questions were a mix of pseudo-PHP, Java, C, and .NET languages and I was told there were twenty or more vulnerabilities in them.

Yeah, I only found like ten.  And I racked my brain on how there were twice that many lurking in these little code snippets.

I did my best, but I wasn’t surprised when I heard back that I was being passed up in favor of other candidates.

Verdict: I honestly don’t know how to grade this one.  Part of me wants to say “I’m grading on pentester interviews in this article, not code reviewer interviews” and give them a bad mark.  Part of me wants to feel guilty that I’m not a good enough code reviewer to pass their assessment.  And ideally a pentester could do code review…and security architecture…and forensic analysis…and reverse engineering…and hardware hacking…and lots of other things, but very few (genius and/or Adderall-abusing) people are going to have the time and inclination to become extremely competent in all these areas.  Also, I have no idea what the rest of their process is beyond the literacy test and code review.  So I’ll just give them a middle-of-the-road 3 out of 5: if they want their pentesters to be code reviewers, that’s great…I’m obviously not their guy.

3/5

Viahouse

This one was unique, as you will see, as it had two very different technical assessments.

The process began like many of the others.  In this case, it was an external headhunter that found me on LinkedIn and called me to discuss the listing.  The second call was your usual technical interview with some questions about my background and the usual stump-the-chump infosec trivia; relaxed and not too difficult.

The next phase was the first technical assessment, which was an AWS-hosted web app.  Both assessments I took for this company were not time-limited in any way and I was free to take as much time as I needed to get through them.  Poking it and doing some reconnaissance, I saw that it wasn’t some custom-developed app, but was existing open-source software.  Enumerating the version turned up public exploits for it on ExploitDB and other places, but none of them were particularly juicy (ie, they weren’t going to get me a shell).  At this point, I taught myself a valuable skill: reversing exploits from CVE descriptions.  Lots of apps have CVEs on them where the researcher never released a public exploit into the wild…and in the case of web apps, it can be stupidly easy to figure out an exploit (or at least substantially narrow down where to look).  For example, a CVE that says something like “SQL injection in sucky-code.php via the ‘query’ parameter” practically writes itself.  One CVE I saw was for a newer version of this web app that could allow for code execution.  The name of the page and parameter had changed, but it wasn’t hard to figure out where it was in the version I was attacking, especially since it was open source and I could just read the code for that version.  I was able to find the injection point and get a shell on the box.  I’m not sure if I was supposed to get this far, as per the original intent of the designer of this challenge, because there was no readily apparent path to escalate to root privileges.  DirtyCOW was all the rage around this time, and since I’d exhausted all other avenues, I used that to pop the box.  With that done, I wrote a web app pentest report for them, detailing all the findings I’d uncovered, and sent it in.

Having passed that, the next challenge was by far the most unique I’d seen.  This challenge involved reverse-engineering a network protocol to try to discover vulnerabilities in it.  Another AWS box was put up, with a web interface for interacting with the protocol.  Using that web interface, I was able to point the protocol traffic to a box I controlled and proxy it.  I then figured out just how the packets were structured, how they worked, and started building some tools to interact with it (the pwntools Python library makes whipping up POCs and hacking tools really easy).  I was able to discover some odd behavior and undocumented functions, wrote up my findings, and handed that report in.  To be an over-achiever, I even wrote up a Python tool for interacting with and exploiting the protocol and handed the code in.

The final step was flying out to their office for an in-person interview.  The in-person portion consisted of several different sessions with different sets of consultants and managers, gauging my general consulting skills, like how I would go about scoping projects, how I would approach each phase of a pentest, dealing with difficult clients, etc.

Verdict: this interview process was very thorough and, from the perspective of the company, does a great job of teasing out where your strengths are, how you approach problems, and your soft skills (ie, talking to clients).  I also give them credit for the protocol reversing challenge, which was unlike any of the other CTF challenges I undertook.  Another nice thing is that they realize candidates might be busy with their existing jobs or other obligations, so you’re encouraged to take as much time as you need to finished the assessments.

There’s one big problem, however…the process is long.  Even if you’re blowing through the CTFs as quick as you can, there’s still five phases to get through.  The recruiter happened to catch me at the right time that I ended up going through all the phases, but I’m certain this company probably loses a lot of high-end candidates because someone else makes them an offer first.  Having two technical assessments is awesome, but it might result in diminishing returns if it means other companies are snatching them from you before they’ve even been scheduled for the final interview.

4/5

Lamcon

This was actually two different companies, but nearly identical interview processes.  Both were infosec consulting firms and neither did any sort of technical assessment.  Both followed a pattern of a screening call with a recruiter to ask some HR sort of questions and answer mine about the firm, a second interview with team members asking some technical questions to gauge my knowledge, then a final one with a hiring manager.

Verdict: A very quick and traditional interview cycle, following the same template that I’ve encountered in a lot of job interviews in my working years.  However, with something as technically-involved as pentesting, I feel like a halfway-savvy person could really BS their way through this process.  In fact, I’ve seen it, as I mentioned when I helped with hiring at one of my previous employers.  So while this very traditional way of interviewing candidates is just fine for a lot of other jobs, my own opinion is that you need to have some sort of technical challenge to gauge candidates’ practical skills.

2/5

Dishonorable Mention: Dongquote

Yes, I saved the worst randomly-generated name for these guys.

I found these guys via the Reddit r/netsec quarterly hiring thread.  Their process was another traditional three-phase one like Lamcon, but absolutely the worst technical Q&A I’ve ever had.

The guy I was interviewing with seriously asked me the “What happens when you type google.com into your browser’s address box and press enter?” question!  This was a phone screening, so thankfully there was no whiteboard around for me to reverse a binary tree on.  It was all downhill from there.  I swear he asked me more questions about DNS than he even did about security or pentesting.

After this atrocious interview, they seemed upbeat and said they’d follow up.  They never emailed or called me again, nor did I.  I’d started reading up on the company on GlassDoor and some other websites and the reviews from former employees were awful.  Granted, most review sites are gonna be biased in favor of complainers, since people are usually way more motivated to get online and vent after a bad experience, rather than go praise after an average or good one.  But in the case of this company, there was only one positive review and it was clearly written by a recruiter or HR person trying to propagandize…followed by dozens of bad reviews warning people to stay the hell away!  I wasn’t exactly heartbroken that they never followed up.

Verdict: Don’t ask the “What happens when you type google.com in your browser?” question.  Just don’t.

1/5

Conclusions for Pentest Teams and Hiring Managers

Having been on both sides, interviewer and interviewee, and seen how it’s done with several large firms, here are my own take-aways for how to effectively evaluate and select candidates.

1. You have to have a technical assessment.  This is by far the most important piece of knowledge I can impart, and thankfully, most pentest teams already do it.  This is absolutely imperative!  You can play stump-the-chump all you want, but nothing demonstrates skill like actually doing it.
2. Make your own challenge VM.  Don’t cheap out and just download something off VulnHub.  It’s fine to look at VulnHub or conference CTFs for inspiration, but make your own VM and make it reflect the sort of work you do.  If your team is more into web apps, make it some sort of web app hacking challenge.  If you’re into infrastructure, have them get a foothold and escalate privileges on Windows or Linux boxes.  If you do a lot of reversing, offer a crack-me or a protocol reversing challenge.  Or do some combination of all of them!  Whatever it is you do on your team, challenge your candidates to show you they can do it.  The other advantage of making the challenge VM yourself is you know they can’t just go find a write-up or walkthrough online.  Chances are you’ve got guys on your pentest team who’d love to make their own CTF challenge.  Do peer-review it among other pentesters and make sure the difficult level is appropriate.  I know from experience that the creator of a CTF challenge often thinks it’s going to be a lot easier for others to solve than it really turns out being.
3. Host it yourself (whether on the Internet or in-person is up to).  Another way to both avoid cheating, as you can easily mount and read .vmdk files, and also not require the candidate to have a ton of spare hard drive space or RAM, if the challenge VM is particularly resource-hungry.  Most places opt to host this on the Internet and give candidates ample time to finish the assessment.  There is something to be said for the Bigity way of doing it, hosting in-office and getting to watch the candidate as they work.  It’s really up to you, but it’s probably best to do this part remotely, just for time considerations.
4. Technical interview questions are great, but don’t turn it into “Hacker Jeopardy”.  When you’re doing the actual interview and trying to feel out how smart a candidate is, technical questions are of course important…but quiz them more on their thought process, methodology, and how they approach problems.  Some stump-the-chump questions are OK, but don’t rely on them to have memorized all the same crap as you.  I know there’s a lot of technical minutiae that I’ve offloaded to Google…that my screwed-up brain has then replaced with weird Unix history trivia that will never do me any good on any future job interview.  Ask instead how they do pentests, maybe even some scenario questions, see how they would tackle a problem or research answers.  Even if they slam-dunk it in this sort of interview, you still need to do a technical assessment so they can prove they know how to do it.
5. Have an in-person component (if it makes sense for your organization).  I’ll say this is optional.  If you’re an org with an almost entirely remote workforce, then you can probably skip this part.  If you do have an office culture, and even if you’re hiring for a remote position, bring them in to meet the in-office folks.  It’ll help you to feel out the candidate and for them to feel you out too, to make sure you’re both good fits for each other, as far as corporate culture and all.  Most places save this for the final phase, when they’re already 90% certain you’re going to make the guy or gal an offer.
6. Consider having candidates present, rather than just deliver, a report.  Again, this kinda depends on how your organization does business, but I suggest you have your assessment process mirror that.  If your usually mode of operation is to pentest, write a report, and present those findings to a client vocally…do it that way in your interviews.  Have your candidate write a report on their technical assessment, then present all of their findings to you over the phone, videoconference, or in-person (if you chose to do that from the last step).  In my own opinion, this is a good step even if you’re not a consulting firm.  When I was an in-house pentester, I always had to present findings to developers and application/system owners, and they’re usually an even tougher crowd than a paying client!  If you want, you can even play good cop/bad cop with the candidate and make them defend their findings and explain to you why they pose a threat.  Make it realistic and reflect how you do business.
7. For God’s sake, be quick.  Probably the other key take-away I learned, both as interviewer and interviewee.  If your process isn’t quick, you’ll lose good talent to someone else.  As much as I appreciate the thoroughness of some of the more exotic forms of interview I underwent, the classic four-phase interview process is probably the best: recruiter screening, phone technical interview, technical assessment, and final interview with presentation.  Or you could roll the last two into one giant step, like Bigity does.  Any longer than that and you risk getting undercut by faster organizations.  The one aspect that might be out of your control is how swift HR is in issuing offers, as Bigity had this issue.  Communicate up your chain of command as best you can the importance of swift action.  These are highly-skilled professionals we’re talking about and their talents are in high demand; tell them they need to act accordingly.

I hope you’ve found this article entertaining and informative!  If you conduct pentest team hiring for a company, I hope you’ve enjoyed this little peak behind the curtain at how other orgs are doing it and can learn a thing or two to improve the process at your own organization.  Good luck out there!