HealthTap Builds a Skills-Based Hiring Growth Machine with HackerRank

What if sharing your symptoms with a doctor was as easy as sharing a photo with a friend?

HealthTap is one startup that’s making this possible. Having received more than 23,000 notes from people thanking HealthTap for saving lives, the company is making healthcare information more accurate and accessible. 

Its latest innovation is the launch of Doctor A.I., which combines the world’s largest network of 105,000 doctors
and 5.4 billion healthcare questions served with a personal artificial intelligence (AI)-powered “physician” that routes users to doctor-recommended insights and care immediately.

But you can’t make real progress in disrupting an industry as dated as healthcare without an army of entrepreneurially minded leaders and engineering talent that isn’t afraid of going the extra mile.
How do you find this kind of talent? If you’re a growth manager, you hack it, of course.

As one of the first product growth managers in the health tech space, Steijn Pelle (pictured above), along with lead designer Jordan Pease tell us how HealthTap is saving lives while building the first growth team for the company:

 

First, give us a little background about yourself and what you’re doing at HealthTap.

Steijn: We basically code….and save lives. I’m not exaggerating. In our product, we give people the opportunity to rate the impact of the experience with HealthTap and Dr. A.I. More than 23,000 people thanked us for saving their lives or the life of a loved one, and told us the story of how we’ve done so–it’s beyond inspiring!

Okay so you’re in charge of both growth and building the first product growth team at HealthTap — finding the right skills and mission-driven people. 

Steijn: Yes. One of the most valuable resources you have at a product company is obviously the designers, data scientists, and engineers that are building a product people love to use. When you need to take time away from your team to evaluate other engineers or candidates you’re bringing in, it becomes very expensive.

Especially when you’re doing it at a scale that we were doing it in. We were reviewing hundreds of candidates per week. Any way you can optimize that adds tons of value to the process.

What kind of process was it?

Steijn: To give you an impression of what our hiring process is like, we looked at more than 3,000 applications to find five people. That makes the ratio of hires to candidates 1 to 600. Especially because we have a mission-driven culture; we have certain core values and we’re in an entrepreneurial stage it’s important to find the right match.

How long did it take you to screen people when you were doing things the traditional way?

As a growth guy, I like to measure things and look at things from an optimization perspective. We basically spent 30 minutes reviewing or doing technical assignments manually–and we did that back in the day through Skype conversations or phone calls. A bottleneck in our funnel.

So we decided to use HackerRank to create custom coding assessments, which are automatically graded for us. This helps us evaluate thousands of people by their skills.

And how did this impact your hiring speed?

Jordan: What we saw with HackerRank was an immediate, massive uptick in productivity and our ability to filter candidates out that didn’t quite cut it without wasting engineering resources.   There was a very significant gap that we jumped across from the time before we had a HackerRank system to after HackerRank.

With HackerRank, we brought that 30 minutes down to five minutes. An engineer only has to look at the assignment, which is already pre-set. So, it’s mostly just directional. Jordan and I were able to get a sense of candidates even if we don’t have HTML or engineering skills. I can get a sense of someone’s skills with coding assessments.

Note: The “time spent screening” is based on HealthTap’s estimation on the average time it takes to perform technical phone screenings. 

But did you see better quality candidates as well? People who were truly skilled?

Jordan: It allowed us to bring through–with less resources, time, and effort–much higher-level candidates through the testing component of HackerRank. It gave us the chance to vet people for technical skills and cultural values.

It’s interesting. I saw a lot of qualified candidates who you would have think would do well on the test because they went to a great school actually not pass the test.

How did HackerRank and this skills-based hiring approach impact your bottom line?

Jordan: With this time back, it’s much easier to ship things faster. Hiring is a time-intensive process. Especially if you’re involved in multiple stages of the process.

At HealthTap we have five stages to a good interview process. If you can eliminate any time away from me as a designer and not a recruiter–Steijn as a growth PM not a recruiter–that’s immense value you’re giving back to the company.

You increase speed in bringing candidates, but also you get to commit your own time to the goals you were initially hired for. It basically turns you into an hiring machine!

Doctor A.I. sounds truly transformative. How long did it take for you to build and ship Doctor A.I.?

Steijn: The knowledge and data-foundation building has taken six years to build up. The engine behind it was built and iterated multiple times over these years. The ontology was built and evolved with thousands of doctors contributing and reviewing since the company was founded. Only within the last year was machine learning and AI applied to the data to train the engine and to create the Doctor A.I. functionality.

Building the UI/UX part was a matter of 4 months from start to finish.

Do you think you shipped faster because of your hiring growth engine?

Steijn: That’s the other thing about hiring, it’s important to consider and measure how much time can you delegate to tools like HackerRank? How many hours do you need to bring in candidates?

Back in the day, before HackerRank, we needed 30 hours to bring in a candidate. We’re able to do it in five hours right now. That’s a big benefit.

What’s next for you?  What are your hiring goals?

Steijn: The rubber meets the road this year. The business is growing very rapidly and we want to at least double the team, while keeping quality as high as it’s always been. What’s holding us back from growing faster is adding more talented and entrepreneurial engineers, designers, product managers, and data scientists. It’s a great problem to have.

We’re excited to work together with HackerRank and others to see how can we save a lot of time to make sure we’re making the right matches. Jordan and I are going to be contributing to that challenge.

I think there’s a great potential on the sourcing side. As we start picking up hiring for the start of the year, we can accelerate the pace of high-quality sourcing of great candidates. The idea of having pre-tested, pre-vetted candidates is phenomenal.

Jordan: We have experience using tools and services like Hired, A-list, Angel list, and they have their own set of vetting. You only get into that batch or pool if there’s a demand for you and if you meet a certain set of requirements. We love those services, and I think having the HackerRank assignment vetted by top engineers adds a layer that I like and I think would add a lot of value.

 

Interested in learning more about HealthTap’s skills-based hiring approach? Say hello.

 

 

Top Performing Coding Bootcamps

 It used to be that companies mainly look for developers from top universities, like Stanford, MIT, UC Berkeley. Now, things are changing. Computer science education is more accessible online. We’re standardizing technical skill assessment.

Many companies are hiring people who learned their coding skills through ‘untraditional’ means. VMware, JP Morgan Chase and Puppet…to name a few. By ‘untraditional,’ we mean:

 

• Bootcamps
• Junior colleges (or acceleration programs at universities)
• Online resources (Self-taught)

By 2024, there will be 1.4 million new developer jobs with only 400 thousand CS grads. This mismatch means employers must look at elsewhere to fill the skills gap.

One big hesitation from employers has been: How do you know if these folks actually have the right skills?

 

At HackerRank, we help companies find the right developer based on their skills. Not pedigree. Using practical coding challenges, employers can standardize the way the measure skills. This is the evolution toward skills-based hiring.

 

The White House launched the ‘TechHire Initiative’ in 2015 to promote skills-based hiring as a solution to the skills gap in America. They partnered with us last year to host a nationwide online hackathon. ‘Untraditional’ developers from underrepresented cities solved coding challenges. Employers included Jaguar, Agile, WebMD among many others.

 

The beauty of standardized skill assessments is that you get an apples-to-apples comparison. We were curious: Which bootcamps performed the strongest on HackerRank coding challenges?

TechHire, which is powered by the Opportunity@Work nonprofit, targeted developer communities in 9 states. Fifteen bootcamps participated, and The Software Guild stood out with the most participants. And they had the strongest performers too.

 

We looked at bootcamps with the most developers who made it to the 70th percentile. Dozens of employers sponsored the event in kind, and we sent them contact info of the 70th percentile.

 

 

 

Which Languages Were Most Popular in the TechHire Hackathon?

It turns out, bootcamps focused on Javascript more so than traditional colleges. Javascript’s popularity has been skyrocketing. Because it’s — well — everywhere. It’s used by 94% of all websites.

“Bootcamps have a limited amount of time to teach, and Javascript is a more practical skill in the industry,” says Dr. Heraldo Memelli, the lead content curator at HackerRank. “Javascript has increasingly become of the one of the most sought after skills. It’s a tool that new developers can use to build things fast.”

The Bottom Line

Strong programming skills can come from anywhere, from unlikely towns and unlikely backgrounds. It’s hard to measure someone’s skills based on a certificate of completion. Standardized skill assessment is expanding the talent pool for companies. It uncovers untraditional talent.

Untraditional developers may not look as good on paper. But they could have the drive, ability and skills to do the job. All you have to do is give them the opportunity to prove their skills to you first.

---

If you enjoyed this piece, subscribe here to get a quick note the next time we have an interesting, new piece on tech hiring.

---

If you liked this piece, you might like:

• Which Universities Have the Best Coders in the World?
• Which Country Would Win the Programming Olympics?
• Which Countries Have the Best Female Developers?

 

Skills-Based Hiring Can Narrow the Tech Talent Gap

This article was originally published on SHRM.org blog.

---

Recruiters are adopting data-driven practices and tools to find skilled candidates, relying less on methods like poring over resumes and following their gut instincts.

The tech industry especially bemoans a lack of qualified job seekers to fill its expanding ranks, but some say part of the problem lies in the traditional hiring process, rigid job qualifications and restricted searches. Tech firms tend to hire from top-name schools or competitors, a practice that overlooks talented programmers who lack that pedigree, according to HackerRank CEO Vivek Ravisankar.

HackerRank, which develops computer programming testing and a ranking system for tech professionals, and companies like it are interjecting coding challenges into the typical hiring process, which relies on resume screens instead of skills challenges as a gate to an initial interview.

HackerRank claims 1 million users globally, is free for job seekers and collects fees from employers. Programmers are scored on the accuracy of their solutions to programming challenges and ranked globally on a leaderboard. Employers can set up specific challenges related to jobs they want to fill.

Ravisankar sat down with SHRM Online at the Talent Acquisition Tech Conference in Austin, Texas, in November.

SHRM Online: HackerRank was founded in 2012. How did it come about?

Ravisankar: I was a developer at Amazon and my university colleague Hari Karunanidhi was an engineer at IBM. There was a general sentiment in the developer community that the rounds of interviews we experienced were not the most effective use of time. A resume is a poor indicator of how good a programmer is.

We figured that’s the reason it took so long for programmers to move through the hiring process. But if you could figure out a prospective candidate’s abilities before the interview, that would have a better impact on the interview process.

That got us thinking. We wanted to solve the problem of better matching developers and companies. After a couple of failed attempts to solve the problem, we launched HackerRank. As developers started using it, we expanded our idea. This doesn’t just have to be a screening product but can be used to build a talent community from which employers can start to engage with and source from as well.

SHRM Online: What’s next for the company?

Ravisankar: We have evolved to automatically matching candidates to companies using machine learning techniques. That’s a huge shift. If you go to a company and ask, “What kind of developers are you looking for?” you will probably get a generic answer, something like “Somebody really smart and who is a hard worker.”

We are quantifying what that really means. I think we’re in the middle of a shift to skills-based hiring. Our goal is to continually write a better matching algorithm.

SHRM Online: You’ve said that the skills gap in tech is a myth. How so?

Ravisankar: Employers are applying very restrictive parameters, and the amount of hidden talent is way higher than what people imagine. If most tech companies apply the same filters—that candidates should have graduated from Stanford or Berkeley and worked at Facebook or Google—clearly the sample size will be limited. One factor is location. Another is background.

VMware [a cloud and virtualization software and service company] recently hired a person who was working as a dishwasher.There’s no way his resume would have passed the tech recruiter’s review.  The only reason he got so far was that he took a HackerRank challenge, scored above the cut-off and got the attention of the company. Resumes are used as a proxy for showcasing skills.

A resume doesn’t tell you anything. If you want to have a job as a reporter, the first thing they will ask you for are your clips. But for developers, it’s hard to show the code you’ve been working on from a previous company because of proprietary reasons.

Was this article useful? SHRM offers thousands of tools, templates and other exclusive member benefits, including compliance updates, sample policies, HR expert advice, education discounts, a growing online member community and much more. Join/Renew Now and let SHRM help you work smarter.

 

One Way to Close the Gender Gap

Believe it or not, there’ve been more *articles written on diversity in tech in 2016 than in the last three years combined. You know the deal: Why Diversity Matters, The Grim Diversity Numbers and The Problem’s Getting Worse.

What’s not written enough is: How do we individually move the needle to close the gender gap in tech? What are some changes we can make to the hiring process that will diversify our talent pipeline?

If you have the technical skills, motivation and persistence, you should have a fair shot to the most in-demand jobs of the 21st century—whether or not you possess a Y chromosome. Here at HackerRank,  we’re on a mission to create opportunities for people of all backgrounds.

But it’s not all one-sided. Companies carry the burden to level the playing field, as well. Our hypothesis (which is laid out by our co-founder Vivek here) for solving Tech’s diversity problem is two fold:

• Use technology to reach more people in a very targeted way
  

• Create standard, relevant and blind assessments to limit biases

While we know this problem can’t be solved overnight, we wanted to test our theory. Our question was clear: Do targeted recruiting and blind assessments really move the needle in closing the gender gap?

We teamed up with college recruiting startup Door of Clubs to host a 24-hour online hackathon on October 14th, 2016. The goal of this event was to reach underrepresented students, and then test their coding skills in an objective way. What we found was that a small, targeted push can, in fact, boost the ratio of female coders, building a diverse talent pipeline.

How the Experiment Went Down

Door of Clubs tapped into its national network to engage college students  in engineering clubs to participate in a friendly, one-day online coding competition comprising five sets of challenges. We also held an onsite event at San Jose State University (SJSU) in partnership with the SJSU Colony of Alpha Omega Epsilon, a sorority for female engineering majors. While the event was open to all students, we targeted outreach efforts around women.

Top 3 winners could win $500, $250 and $100, respectively. Recruiters from tech companies, including Asana and Symantec, participated as well, to scout for interns.

So, What Was the Outcome?

Of the 50 students who participated in the hackathon, 47 percent were women. It’s interesting to see this ratio in contrast to the industry average. Diversity champion Tracy Chou’s public spreadsheet of over 80 companies, for instance, averages 19.37 percent women in engineering positions.

What’s more, two of the top three highest performers were women — Huiyu Yang ranked second and Minglu Liu came in third. This was the first coding competition for both women.

Liu and Yang are both working towards masters degrees in computer science at SJSU. Liu has a masters in geochemistry from the University of Wisconsin, while Yang holds a masters in geology from Stony Brook University.

“I actually just search online how to do things. I merge pieces together. I was an outsider, and didn’t get official training,” Liu says. “Anyone can code.”

On paper, both women didn’t go to elite schools and have backgrounds in other sciences. Their resumes—alone—wouldn’t make it past traditional, blunt proxies that most companies use to find talent. And unconscious biases against women engineers doesn’t help their case either. 

But given that they placed in the top three out of 50 people, our hypothesis in this experiment deems to be proven true. Many people point to the “pipeline problem,” for the lack of women in tech. But we built a pipeline of talent with an almost even ratio of men and women on one college campus. 

And we’re excited to surface two talented women who might have otherwise been overlooked in any given talent pool.

This means, making even the slightest proactive effort in reaching out to targeted groups, and then using blind assessments, can help move the needle to close the gender gap.

“I hope every woman can realize her dreams,” Yang says.

So do we.

 

What kind of tangible efforts have worked for your organization?

 

*This statement’s based on a quick Google search of the term “diversity in tech,”  filtered by year

Yet Another Diversity Program is Not Enough

This article was originally published on Forbes

---

If our unconscious biases trigger stereotypes that influence our judgement about people, what exactly should we do when we’re on the ground?

For the past 50 years or so, companies have been trying to close the diversity gaps in the workplace with diversity training programs.

But real change can only come with true innovation in creating not only opportunities to reach more diverse people but also standard, objective ways to evaluate candidates. Today, companies say diversity is important while still using bias-ridden recruiting proxies, like resumes from top universities.

Yet another diversity program is not enough. The only way we’ll start to achieve change is if we change hiring and focus on what matters: Skills to do the job well.

The Roots of Diversity Hiring:

‘To Invent the Future, you Must Understand the Past’ – Leslie Berlin

Roots of employment diversity trace back as early as World War I, when the government tasked a committee of psychologists to assess the skills of over 1.7 million men in less than 2 years. Their job was to sift through massive volumes of people quickly and efficiently—a sentiment that feels uncannily familiar to corporations today. So they created an intelligence test to screen people for the skills they needed.

This started becoming ubiquitous and businesses used it to gauge the abilities of their own candidates. The only problem was that no one stopped to consider its fairness.

It wasn’t until the 1970’s landmark Griggs vs. Duke case when the pressure grew heavy on companies to be able to prove that employment tests didn’t discriminate against subsets of people. For instance, one group of 13 African American employees filed a class action lawsuit against Duke Power Corporation because the company required that everyone score the national median IQ, except the lowest-paying department.

This requirement was unfair because the questions called for knowledge of education out of reach for underserved communities, which were primarily minorities. It created an barrier to entry for a number of African American employees (see pie chart above).

Duke lost the case, and set the precedent for companies to carry the burden of proof of the effectiveness of their selection process.

Shifting Mindset: The True Value of Diversity

While the post-Civil Rights investment in diversity stemmed from lawsuits, more recent studies in the value of diversity prove that diversity is impactful to the business. Of the several benefits of diversity in the workplace, here are two critical:

Diversity of thought

Even the slightest differences in background at the table could improve your team’s decisions when solving complex problems. This theory comes from Scott Page in the early 2000s, a professor at the University of Michigan, who researches how teams work together.To prove his theory, he created a mathematical model to test whether diversity of thought triumphs pure ability. He divided the algorithms into two teams:

• Team 1: Made up of experts who thought very similarly
• Team 2: Made up of random algorithms with different approachesTurns out, Team 2 solved problems faster than the experts. Diverse teams have the advantage of diverse perspectives that homogenous teams would have never think of.

Attracting more talent

Another reason for investing in diversity is to attract more people, period. We need more people in STEM jobs more than any other professions.

Programming jobs are growing 12% faster than the market average, according to a new Burning Glass report, which analyzed 25 million job listings.

But what’s even more interesting is that programming is no longer just limited to the tech industry. Technical skills (like coding) are increasingly important for other professionals, like business analysts who work with data, designers and marketers who create websites, alongside engineers who build products and technologies.

“More than half of set designers, for example, are expected to use 3D modeling software such as AutoCAD, the same tools used to design an iPhone or a new car,” Burning Glass report

If we keep sourcing people from the same homogeneous places, the gap between companies’ demand for talent will widen. If we don’t diversify the places we look for talent, who will fill these jobs?

Every business is powered by its people, and—without diversity—homogeneous teams are limited in perspectives. And, therefore, limited in its impact. It’s why tech companies are investing millions in improving diversity:

2016: Different Mindset, Same Programs

Companies understand the value of diversity and are clearly investing heavily in diversity programs, but they’re still having a difficult time closing the diversity gaps.

Despite the fact that sociologists have found no positive impact of short-term diversity training programs, some versions of these programs persist 50 years later.

Dissecting some of these programs uncovers elements that are reminiscent of diversity programs of the 1960s and learnings that are crucial for any team today:

• Diversity policies are backfiring.
  New research from Harvard Business Review points out that mandatory diversity programs fail. This coercion goes against what social scientists know about what makes people change their behavior. People won’t change their mindset just because they’re told to do so.
• Adding incentives for recruiters is not enough.

  Recruiters may push for diverse candidates, but hiring managers are the ones who ultimately make the decision. Incentive-based programs for recruiters are band-aids to the problem.

• C-level diversity leaders signal long-term impact, but only if they’re not working in silos.

  A string of tech companies, like Salesforce, Airbnb and Pinterest, have hired heads of diversity to hold the company accountable for progress. This is a positive move as long as they’re integrated in the hiring process, versus add-on.

While helpful in some ways, these types of initiatives don’t effectively help companies reach their goal of building more diverse teams.

The Path Forward is Skill-Based Hiring

 

If traditional proxies neglect to account for diverse aspects of people’s backgrounds, then why aren’t we using technology to reach more people and assess skills outside of these buckets?

Even with the best of intentions, most companies have too many variables in interviews, putting them at risk for biases. The most impactful solutions are two-fold:

Use Tech to Reach More People in a Targeted Way

You can’t create lofty hiring goals without a systematic process to reach more diverse people. Let’s take university recruiting, for instance. It’s logical to prioritize elite or local universities to find candidates fast.

But today there are innovative ways to cast a wider net. Using online hackathons, social networks, diversity clubs or building exciting challenges for students are scalable, new ways to reach targeted communities virtually.

For instance, Asana—a productivity platform company— is one company goes beyond in-person university recruiting. Instead, it partnered with Door of Clubs (an organization that provides jobs and funding to student clubs) and HackerRank (a community of skilled developers of which I am cofounder), to host a Diversity Club Hacks event. Over 100 minority computer science majors signed up to participate.

To corroborate this anecdote: Harvard Business Review’s study found:

Five years after a company implements a college recruitment program targeting female employees, the share of white women, black women, Hispanic women, and Asian-American women in its management rises by about 10%, on average. A program focused on minority recruitment increases the proportion of black male managers by 8% and black female managers by 9%.

Then Assess them Objectively using Blind Coding Tests

The most objective filter for talent is raw skills.

Slack, a communications platform, is a prime example of successful skill-based hiring. Its engineering team has more African Americans than the rest of the company.

The company’s been focusing its efforts on minimizing unconscious bias in the selection process. For instance, they got rid of the whiteboard tests in the coding interview, which often incites unnecessary anxiety for candidates who otherwise would perform well. Leslie Miley, the director of engineering at Slack, says they give relevant coding tests that don’t reveal an applicant’s name or background.

“So we don’t know where you went to school, we don’t know what company you worked at, we don’t know your name, we don’t know anything about you when it’s graded.” – Miley on NPR

In fact, the same HBR study showed that—when executed well in a standardized way—hiring skill tests are one of the most effective ways to boost diversity in the workplace. By eliminating biases at the first step of the hiring process, a candidate’s pure skill (relevant to the job) can help mitigate discrimination.

People have been trying to solve the diversity problem in corporate America for over 50 years, but one of the biggest lessons we can learn is that true progress can’t come from add-on diversity programs.

Tactics like coercion, policies or incentives are not enough to reach underrepresented people who don’t have clear paths to your door. It’s hard to change biases that are ingrained in our subconscious, but you can challenge biases with real data from skill-based hiring.

Start a Free Trial of Skill-Based Hiring Here

 

 

Correction:An earlier version of this article incorrectly stated that Facebook’s unconscious bias training is mandatory for all employees.

Introducing HackerRank Jobs: Connect with Our Community of 1M+ Skilled Developers

With traditional recruiting methods, it can take well over 30 days to find the right candidate. You spend most of this time through a lot of manual resume sifting, online profile perusing, phone screen scheduling, etc. If you’re still using old techniques, chances are you’re not only hiring slower but also using the quickest proxies that overlook highly-skilled candidates. Talented developers who may not have attended a great school, may have dropped out or simply took an alternate path to programming. To unlock these engineers, you have to be able to is test their skills at scale.

Just as Uber unlocked more cars and Airbnb unlocked more places to rent, HackerRank Jobs is unlocking more developers and job opportunities for developers, faster.  The app unleashes the power of skilled programmers all over the world, no matter what credentials are next to your name. By going beyond resumes and eliminating the “resume black hole,” HackerRank Jobs is the new bridge between companies and developers.

HackerRank Jobs: For Anyone Having Trouble Finding Great Engineers

Over 40 companies, including Booking.com VMware and Box, are already using HackerRank Jobs and have trimmed down their time-to-hire from months to just days.

Here’s how it works:

1. You set your criteria
2. Post a coding challenge on HackerRank Jobs
3. We fill your dashboard with hand-selected candidates who meet your criteria

Posting a coding challenge on HackerRank Jobs is a great way to get your brand in front of thousands of developers who might not have applied to your company otherwise.

The Bottom Line

Every week, you get a list of candidates’ code challenge scores on either web or mobile, and you have the opportunity to call engineers immediately. What’s more, you won’t have to deal with the endless back-and-forth emails when scheduling interviews with candidates. Scheduling conflicts alone unnecessarily extend recruiting time by weeks.

Of all the companies who have used HackerRank Jobs, each one of them have wondered why this hiring system didn’t exist before. Since the quality of candidates is so high and selective, many customers are shaving weeks off their time-to-hire. On the flip side, candidates have expressed rave reviews on using HackerRank Jobs to apply for jobs versus any other job application process. Hear it from them directly:

“HackerRank is the most convenient way to apply for a software engineering position. All other ways are still very weak compared to HackerRank. HackerRank can tell the exact skills I have without any other proof,” Anas Salman, engineering manager at CASHU

 

“I like the common app on HackerRank Jobs because it makes the job search easy. It’s not like a multiple choice test. It’s real evaluation. It’s real insight from people,” Phil Hord at Pure Storage

Interested in trying out HackerRank Jobs to solve sourcing for your team? Email support@hackerrank.com to learn how to get started.

Should Engineers Really be Judged by Their Resume Writing Skills?

Martin Harriman wanted to make his way back to the software industry after a decades-long break building hardware, so he did what most people do when they’re trying to change industries.

He carefully tailored his resume and cover letter to downplay his hardware experience and—in turn—emphasize his skills, knowledge and expertise in software. A quick CTRL-F command could tell you that he mentions the keyword “software” 17 times in his resume and “hardware” only six.

Technically, his resume should successfully pass through most companies’ applicant tracking systems and into the hands of software engineering managers. One problem though: The last time he was in the software game, the languages, tools and concepts that were considered “modern” included: LISP, Smalltalk, PHP, LALR, and AJAX. In fact, if you navigate to his LinkedIn profile, his most upvoted skill is Perl.

“It’s funny because I don’t even like Perl.”  — Harriman.

Of all the programming languages and tools that Harriman knows, LinkedIn’s handy skill endorsement section is far from an accurate representation of his actual skills.  Harriman can pick up today’s neatly packaged tools of Ruby on Rails or even newer languages like Objective-C in his sleep.

“I mean, after you learn a dozen or so languages, they’re all kind of very similar,” Harriman says.

After all, if he had the mental model, tenacity and problem-solving skills to be able to port the gcc backend to odd architectures, like the DEC PDP-10, today’s modern software development is not exactly rocket science.

But even if he makes it through the ATS system, regardless of how good he is at software engineering, the reality is there are blockades of biases all over this resume. When hiring managers have to go through stacks of resumes daily, it’s hard to move him onto the next step with a resume filled with:

• Older technologies. The reality is that the evolving state of software development makes it really hard for people to trust folks with older technologies on their resume — even if they’ve had decades of past experience in software. Will he be able to catch onto newer stacks? Can’t leave this to chance.
  
  
• ‘Hardware’ is a red flag.’ It’s fairly clear cut. Engineering managers think hardware is generally removed from software development. Harriman describes the general sentiment as:  
  

  “He’s a hardware guy, why are we wasting our time?”

• Potential age discrimination. It’s hard to talk about, but age discrimination in Silicon Valley and tech in general is rampant. After all, the Harvard Business Review reports that the average age of Silicon Valley founders is just over 31. Given that Harriman has experience dating back to the 70s, this could potentially trigger an unconscious bias.

Despite his best efforts of resume tweaking, the traditional, resume-driven screening and hiring process by and large works against him. He got a few call backs, but not nearly as many as highly sought after engineers today. Such traditional processes lock massive pools of talent, like Harriman, out of the running.

So, what happens when you stop depending on resumes to screen candidates? Resumes are ingrained so deeply into our hiring culture that it’s hard to imagine the tangible benefits that might come from replacing resumes with something more practical, like a coding challenge.

Pure Storage is one such company that has always aimed at being as objective in everything they do, including their hiring techniques. So, instead of relying heavily on the subjectivity of resumes, the team has always asked candidates to solve a coding challenge—or what they call a “quiz.” It’s a completely blind way to gauge the skills of anyone, no matter what background or universities they have on their resume.

Harriman applied to Pure Storage, aced its coding quiz, and he’s been an integral component of team for almost four years. It turns out, contrary to what most engineering managers think when they see a hardware resume, hardware experience helps you become a better software engineer. We sat down with Harriman to learn more about his experience and how he ended up earning a coveted job as as a software engineer at Pure Storage with odds stacked against him:

Martin, could you tell us a little about when you first started coding?

I started out in software when I was in grade school because my father was in programming. My first computer-related jobs were in software. I worked in computer software roughly from the mid-70’s to mid-90’s.

Wow, you’ve really seen the boom in software from the beginning. Why did you make the switch from software engineering to hardware in the ‘90s?

I switched to hardware because I was working with a lot of hardware engineers and it was fun stuff.  I literally got a 5-min course on doing chip design and switched right into it. When it comes to doing chip design, it’s basically like writing software in a really bad programming languages.

Wait, you said literally a 5-min crash course on chip design? What do you mean by that?

There was more of a demand for hardware designers at the company I was working at (Silicon Engineering in Scotts Valley). All they had to teach me were the particular constraints in the programming language that you need to follow to compile into a chip. It’s pretty rudimentary stuff. The company that makes the compiler actually publishes a fairly thin volume on writing Verilog for synthesis. I got a 5-minute condensed version. I knew how hardware worked. I knew what design was and given that background, and knowing what a For Loop is, the rest was really trivial.

That’s awesome. So do you think your hardware experience helped you become a better software engineer, looking back?

From an engineer’s point of view, the actual work is kind of similar. What you end up with at the end of the day is a physical object that embodies what you do. That’s satisfying…it’s fun. You just made something that you can’t change. It has to survive for a couple of years for it to be worth it.

No pressure? 🙂

Well, there’s pressure in all of these jobs, even here at Pure Storage. It would be very significant if we created software that corrupted a company’s data. It’s a different mindset on the nature of the testing you need to avoid the disaster…the exact consequences of the disaster. It helps you prevent shipping sloppy work. Because in hardware design, the world is much more test-driven than the software world. The consequences of missed bugs is much higher. It helps in taking the testing side of things seriously.

So, how was your job search overall when you decided to go back to software?

I applied to Pure Storage because I think it’s a great company overall, and had heard really great things. They sent me a coding challenge, first and foremost, and they didn’t really look too closely at my resume.

I heard that you got a perfect score on that test. Did you prepare?

I did not prepare for it at all. I thought, great, I’ve been doing this for many years and I should be able to do this, let’s see what happens. I got a perfect score and I got the fastest time they ever had it.

Wow, why do you think you scored well? What was the quiz?

It’s a lot to do with having a lot of experience in both software and hardware. They did a good job of testing things that are more than just rote knowledge. Things that are fundamental of how computers work and how you think about programming languages.

There’s one question that we got the most heat on, but it was the most predictive of how people would do in the rest of our interview process. It had to do with the binary representation of a fractional number. It’s one of those things where it’s not something you think about everyday in your job. But if you know how binary computers work, you can answer the question instantly. That correlated very well for people’s ability to come into the interview process and produce code for us and talk about their process.

Also read: Why Should Senior Engineers Balance Trees in an Interview?

Do you think these are good questions for screening candidates?

We got a lot of heat from unsuccessful candidates, of course, in the grounds that this is silly because they don’t need to know this to do their job. And they have a point. I guarantee that nobody at Pure needs to know what an IEEE floating-point number looks like, for instance. It’s not something I think of day to day in my work. But it’s one of those things that it’s part of knowing how a computer works. Having that kind of mental model is important on producing code.

Would you have gotten this job if Pure Storage used the same resume process as everyone?

The embarrassing part of that for me is that, I carefully crafted my resume for Pure to emphasize the software experience that we had. They would read that resume and said ‘Oh he’s a hardware engineer, I don’t know why we’re bothering.’ That shows my resume writing skills, glad you’re not hiring me for that.

In fact, many of the most talented engineers at Pure are from interesting backgrounds that go against the grain of what most hiring managers look for in a resume. People come from a diverse set of backgrounds, whether it be their technical background or the university from which they graduated.

Pure Storage now scales the early quiz model by using automated coding challenges. It’s the same idea of asking candidates to solve problems in lieu of sending their resume, except now they can do so for thousands of candidates at once.

So, what kind of cool stuff are you doing now for Pure?

The most recent achievement was a team effort. I’m part of a team that works on space accounting, calculating the amount of space in use. It’s a surprising difficult problem at scale because the internal metadata is optimized for things like read and write performance and as a result almost pessimistic for the task of figuring out what space is in use and by what entity. So, we’ve had a couple of iterations of this over the last three years, and we’re very happy with the most recent one because it’s basically eliminated a significant portion of customer problem calls. It used to be a regular feature for our support organization.

Interesting. And do you think Pure had any problems or skepticism hiring you since you came from hardware?

I’m sure they did, they didn’t explicitly say that to me. Not everyone makes the transition. The good news is that once you have the job, and once you make the transition, you can show them what you can do. That’s the fun part.

That’s awesome. If there’s someone out there who doesn’t have software experience, like yourself, but wants to make the switch, what advice would you give?

Go find a job, which will be hard because people won’t want to listen to you. Try to find a job that’s actually a software engineering job, not just close to the software engineers. If you don’t get the software work you want, then keep looking. As long as you’re sharpening your skills overall, there’s no reason why you can’t prove your coding skills, especially when more and more employers are seeing the value of innovating their hiring process to find software engineers from different backgrounds, even hardware.

 

Enjoy this post? Please subscribe to the HackerRank Blog.

 

Why Should Senior Engineers Balance Trees in an Interview?

For nearly as long as companies have hired programmers, managers have asked engineering candidates to solve fundamental algorithm and data structure problems. And for nearly just as long, engineers have debated the validity of these challenges in job interviews (2005, 2015).

The argument is: If I’m never going to balance a tree on the job, why would you ask these fundamental coding questions to gauge my skillset? At first pass, this can be infuriating for senior engineers. Who’s going to remember basic tree-traversal from computer science (CS) courses when you’ve been using easier, faster standard libraries for years?

But what’s not as emphasized as often is the value of basic CS fundamentals for most roles. Everyone knows the best strategy for screening candidates is to test for whatever’s important for the job, but simple algorithm questions actually play an important role in uncovering what engineers can and can’t do. If you dig deeper, engineers who can’t complete basic algorithmic code challenges in an interview are actually less productive hires in the long run.

You Get Unqualified Hustlers with Quick Wins

If you don’t test for CS fundamentals, you’ll risk hiring programmers who are only good at gettings things done in the short-term. They can put together decent code using APIs and build a glowing portfolio. But if you ask them why their program works the way it does, they’d see opaque black boxes. It’s like they’re assembling parts together without a toolkit.

Over the past several years, there’s been a sharp boost in the number of APIs and standard libraries. For instance, Salesforce, alone, has over 3 million applications in its third-party app system. Look at the sharp rise in APIs in the last 10 years, according to the ProgrammableWeb

The uptick of these neat packages make it easy for programmers to get by without revisiting the fundamentals. And that’s fine if you just want to hustle, get a quick win and build a stunted product.

But most–if not all–accomplished programmers, from Donald Knuth to Ken Thompson, value the importance of knowing why code works in building revolutionary products. For instance, Knuth’s 1968 masterpiece The Art of Programming, was the first time coders could understand why algorithms work the way they do. “So my book sort of opened people’s eyes: ‘Oh my gosh, I can understand this and adapt it so I can have elements that are in two lists at once. I can change the data structure.’ It became something that could be mainstream instead of just enclosed in these packages.”

Testing for algorithms and data structures also tests for lifelong curiosity. Engineers should be “continually interested in keeping themselves up to speed, in revising the fundamentals and taking on intriguing programming problems. Those are the people I want to work with,” says Soham Mehta, CEO and cofounder of Interview Kickstart.

You Build Fragile Products

If you don’t test for CS fundamentals, it’s going to be really difficult for you to provide for your growing base of customers. When scaling out architecture, you have to understand how components work on a simpler, more fundamental level before applying them across multiple machines. If your engineers open enough logic-related bugs, you could lose valuable customer information or create bottlenecks, resulting in a slow customer experience.

This happened to Ben Sigelman, an ex-Googler who founded a company called LightStep, which builds monitoring and performance tools for developers of large distributed systems. He recently worked with a well-intentioned engineer who decided to use Redis for scalable, consistent and durable storage. But Redis is best as an in-memory data structure server and does not – and can not – scale well when placed into its “AOF” consistency mode. In that configuration, Sigelman says it’s much slower and less resilient than true distributed databases that append to cluster-level file systems. He makes a solid point:

“Formal CS training would have triggered a ‘too good to be true’ alarm, well before [the engineer] deployed it, and irrevocably lost user data in the process,” he says.

You End Up Reinventing the Wheel

If you don’t test for CS fundamentals, optimizing your codebase is going to take a lot longer than it should. Opponents argue that smart programmers use standard libraries to save time. Why reinvent the wheel when someone else has already solved this problem for you?

But, remember, we’re not asking advanced algorithmic interview questions because you’ll be writing algorithms from scratch on the job. We’re testing basic knowledge of fundamentals to ensure you’re not just relying on other people’s code, Stackoverflow or Google. Otherwise, when you need to scale and optimize, you’ll waste a lot of time trying to figure out optimal solutions. It’s not just about memorizing how to implement algorithms. Learning the trade-offs between algorithms is valuable in boosting efficiency. Simply testing candidates on knowledge of where trees fit in relative to sets or maps or linked lists is valuable in and of itself.

Gayle Laakmann McDowell, founder and CEO of CareerCup and author of Cracking the Coding Interview, offers a great example of what happens when a senior engineer doesn’t revise fundamentals:

“A more senior engineer building a parsing engine might not understand how she can leverage graph theory or trees. She could spend hours reinventing the wheel, only to come up with something less optimal in the end.”

It’s the same for debugging. The most efficient way to debug requires fundamental knowledge of how components behave with one another. Someone who doesn’t really know how things work might put in logging everywhere in hopes of catching errors by trial and error. A better way would be to systematically isolate issues by spotting patterns in the errors. You can only do this if you know the system and its algorithm.

It’s especially important if you’re not quite sure which specific tools you’ll need. If you’re building a long-lasting product, it’s crucial to test for timeless fundamentals that will be the foundation of future programs. “The breadth-first search algorithm, for instance, was invented in 1959 as the solution to the shortest path in a maze, but it’s still indirectly important to programmers today through some layer of abstraction. ” says Dr. Heraldo Memelli, who oversees all of the code challenges at HackerRank.

Programming tools come and go, but fundamentals are forever. The assumption that you don’t need to know CS fundamentals on the job couldn’t be further from the truth.

But the Interview Can’t be ‘One-Size-Fits-All’

Of course, you can’t rely on general CS questions—alone—to hire for every role. The coding challenges you select have to be appropriate for the role you need filled, and basic fundamentals are one bar that should be cleared.

Leo Polovets has had a lot of experience in designing great screening processes as the second non-founding engineer at LinkedIn and engineer at Google. He offers a solid example:

“For a backend candidate, you might give them a problem where they need lists, sets, and hashmaps, and you want to make sure they use the right structure at the right time. For a front-end candidate, a good question might be to asks them to do some basic DOM manipulation. These could be 10-20 line programs, but they’ll still reveal a lot about what the person can and cannot do,” Polovets says.

More Companies Should Prepare Candidates

The reality is, algorithm and data structure interview questions should be really easy—as long as you have some warning, good prep material and context for what interviewers are really want to see.

Algorithmic coding challenges aren’t designed to evaluate how well you think on your feet. In fact, if you’re pop quizzing your candidates on algorithms, you’re most likely turning away really great people who happen to test poorly. The best tech companies are preparing their candidates as much as possible to create a stronger, more successful talent pool.

Facebook invests in teaching an interview prep class for all of their candidates. They realize that senior engineers or folks who are self-taught will need to prepare. It covers exactly what kind of algorithmic coding challenges they plan to ask and explain why. Of the three phases of Facebook’s technical interview, one is called “Ninja,” which screens the ability to solve tough coding challenges, like sorting algorithms. Any engineer who applies to Facebook has to do really well on these interviews. It’s one of the key reasons why Facebook has a world-class engineering team.

The assumption that you don’t need to know CS fundamentals on the job couldn’t be further from the truth for most jobs. Well-designed basic algorithm and data structures challenges are a good way to gauge depth of technical skills for sustainable products.

---

This article originally appeared on Forbes.

---

If you like what you see, please subscribe to our blog to get a quick note when we occasionally write thoughtful, long form posts.

---

To help hiring managers create better code challenges, analyzed hundreds of code submissions and spoke with several interviewers to create this in-depth guide:

Step 0. Before You Do Anything

Step 1. Designing Impactful Challenges

1a. The Challenge Checklist

Step 2. Setting Expectations, Warming Your Candidates

Step 3. Calibrating After the Screening

 

Screening | Step 4: Calibrate

Welcome to the final part of a 4-part series on screening senior engineers.

Step 1. Before You Do Anything
Step 2. Designing Impactful Challenges
Step 2a. The Challenge Checklist
Step 3. Setting Expectations, Warming Your Candidates 
Step 4. Calibrating After the Screening (This piece)

---

In steps 1, step 2 and step 3, we strategized on the best types of challenges to tease the engineers you need. But what good is a code challenge screening if you’re testing different candidates with different questions? Consistency can make the data you’re collecting meaningful. HackerRank’s head of challenge curation Dr. Heraldo Memelli finds that all too often, companies change the question after it’s already gone out to some candidates, ruining your data set. Periodically review which questions have been directly correlated with successful candidates.

After the initial screening, it’s crucial to mirror your code challenges with the rest of your interviewing rounds. It’s logical — folks who clear the automated code challenges online are more likely to perform well in-person if the questions are consistent. Automated code challenges are the best tool to predict who will perform well in the in-person interview. So, target your questions to mirror the style of questions asked in the actual interview as much as possible.

Choose questions that you’ve actually asked in on-site interviews so that you can see how candidates might react. This can help you frame and word the question appropriately as well. Of course, this means you’ll have to retire those questions from your in-person interviews. Aim to keep your code challenge banks fresh. Trying out new code challenges on your own time is a great way to not only benchmark your own team against newcomers but also keep your engineering teams’ skills sharp. Investing the time in revamping and optimizing your set of interview code challenges is minuscule compared to the cost of hiring wrong engineers. 

Tieing it All Together

Asking senior engineers to revisit fundamentals in an interview sounds outrageous. And it is if you simply send a cold email without any proper preparation. But automated code challenges are the most objective and successful predictors of hiring we have to filter through candidates at scale.

Gauging not only their intelligence but also how much they value fundamentals through algorithm and data structure questions are strong instruments to find the best engineering talent. If you set expectations and are mindful of senior engineers’ complicated hubris, you’ll have a stronger pool of senior engineers. And you’ll be set up with a replicable system to build and scale your engineering team. A set of well-designed mix of questions that test fundamentals, knowledge and depth of thinking, is a good way to weed out candidates who don’t have the basic skills you need to build revolutionary software.

---

Please subscribe to our blog to get a quick note when we occasionally write thoughtful, long form posts.

---

Welcome to the final part of a 4-part series on screening senior engineers.

Step 1. Before You Do Anything
Step 2. Designing Impactful Challenges
Step 2a. The Challenge Checklist
Step 3. Setting Expectations, Warming Your Candidates
Step 4. Calibrating After the Screening  (This piece)

 

Screening | Step 3: Set Expectations

Welcome to the 3rd part of a 4-part series on screening senior engineers.

Step 1. Before You Do Anything
Step 2. Designing Impactful Challenges 
2a. The Challenge Checklist
Step 3. Setting Expectations, Warming Your Candidates (This piece)
Step 4. Calibrating After the Screening

---

In step 1 and step 2, we figured out what you want and designed the challenges to help surface the right engineers tailored to your needs. Now you can focus on the most important part of executing your new screening process: practicing tactful communication to engage senior engineers.

Senior engineers love a good challenge, but not if it’s time-consuming and futile. The truth is, accomplished engineers will scoff at a cold email prompting them to spend 1-2 hours solving a fundamental code challenge for a company they may never hear from again. It’s understandable — why should they? Chances are they’ve already earned their stripes putting in late hours building software that’s reshaping our world one way or another. When designed and delivered haphazardly, automated code challenges will be perceived as insulting.

It’s critical to infuse an element of empathy in the delivery and design of your challenges. Cold emails with no explanation as to what they’ll be asked and why, are a surefire way to lose senior engineers. Those who make an attempt to make their candidates feel valued while scaling their screening systems will be the winners of today’s competitive war for talented engineers.

There are a couple of things you can do to mitigate this:

1. Be very clear about what you intend to ask and what you expect. Give them easy-to-digest prep material.

This is an important step in ensuring that you’re not rejecting great people just because they lack CS fundamental knowledge. Most senior engineers are not going to have CS fundamental knowledge right off the bat. First of all, nearly 60% of working software engineers don’t even have a proper CS background in one study. Second of all, even if they did, it was likely too long ago to remember anything that’s applicable in the real world. Expecting too much with no warning results in this: 

Before sending testing engineers on fundamentals, explain that we’re not trying to quiz them on random things they don’t even need to use on the job. Instead, fundamental binary tree questions help gauge problem solving and critical thinking skills, as we explained in Step 0. Although many great candidates get rejected because they failed to adequately prepare, it’s actually not that hard for smart developers to learn or re-learn the fundamentals. It’s part of why companies are fine with requiring them. And, as Kickstart CEO Soham Mehta said, the best senior engineers want to brush up on their fundamentals anyway. 

2. Calculate the amount of time you need to take the challenges (usually between 45 and 90 minutes) and give them more than enough time to actually complete it. Communicate this to your candidates and explain that it shouldn’t take you more than an hour, but you have X hours to take it. It helps to simply articulate that you recognize that they’re extremely busy. Their time is valuable, so there’s no intense time limit to submit.

3. Explain why you’re even asking them to do this. For instance, if the code challenge replaces the resume or initial phone screening, let them know. Explain: “We ask you to do this code challenge so you don’t have to spend more time on the phone with multiple engineers.”  If you’re sending code challenges with no explanation, reasoning or preparation, your screening tool is an arbitrary knowledge test. You’re bound to anger senior engineers.

Bonus Tip: We did a quick analysis of dozens of code challenges and found a consistently lower completion rate when using the word “test.” Avoid this term like the plague. Instead, always use “challenges.”

So after you’ve implemented great code challenges, what can you do to make sure they’re successfully producing the best senior engineering candidates?

Read onto Step 4. Calibrating After the Screening

---

If you like what you see, please subscribe to our blog to get a quick note when we occasionally write thoughtful, long form posts.

---

Welcome to the 3rd part of a 4-part series on screening senior engineers.

Step 1. Before You Do Anything
Step 2. Designing Impactful Challenges
Step 2a. The Challenge Checklist
Step 3. Setting Expectations, Warming Your Candidates (This piece)
Step 4. Calibrating After the Screening