Hiring TestersBest of TestRail Quality Hub

Contents

Do We Still Need Dedicated Software Testers? 3

The Ideal Tester to Developer Ratio 9

Find, Interview and Hire Great Software Testers 14

25 Challenging Tester Interview Questions 25

21 Questions to Ask When You Are Being Interviewed 31

Retaining Your Software Testers 38

Making Do When You’re Understaffed 45

How to Hire the Agile Team You Need 52

One of the most important responsibilities of a QA leader is to build a testing team. But

it isn’t easy. In this era of Shift-Left testing, you may have to first justify why your team

needs testers at all. In addition, an effective tester must understand testing principles

and have technical abilities. But there is a relative lack of higher education courses in

testing. So it can be hard to find someone with the knowledge and skills you need.

At TestRail, we believe in the essential contributions that testers make to the develop-

ment of quality software. So, we’ve gathered some of the best articles from the TestRail

Quality Hub to help you hire great testers.

3

Do We Still Need Dedicated Software Testers?

Hiring Testers

Simon Knight, TestRail Product Manager

4

Even without dedicated testers and even if the developers get it wrong, you can have

other safeguards in place, right? Just like Facebook does. You can do things like:

• Have the developers write lots of automated tests so that you don’t need to spend any time manually testing our software.

• Have your own people dogfood for problems with a pre-release version of the software.

• Build some more logging and traceability into the software so that you can easily pinpoint where problems are occurring if you see any.

• Monitor current activities with previous versions of the software to see whether there’s some indication of different user behaviours and whether those behaviours are desired or not.

Well, maybe. And maybe not. Let’s take a look at some of those ideas in some more

detail and see whether they hold up to scrutiny.

Most of you will have heard at some point or another about startups or much bigger or-

ganisations that have decided they have no need for dedicated testers. Facebook is an

often cited example of a large company who has decided that by using various mech-

anisms including dogfooding and production monitoring, they can eliminate testers.

They are choosing instead to release lower quality products directly into the public

domain.

Such stories beg the question that, if it’s okay for Facebook and other companies not

to have dedicated testers, can my organisation save time and money by eliminating

the testing function also? Instead of having a tester do it, can’t I just get the developer

responsible for building the software in the first place to check his or her own code?

Life Without Testers

5

Automated Testing

Apparently, automated testing gets all the focus now. It seems like more or

less every team in the world is varying degrees of Agile and is now writing

code using some kind of model driven development – typically test, behaviour

or acceptance test driven (TDD, BDD and ATDD respectively).

Normally in a model-driven-development kind of a scenario, the developer is

attempting to write their code using their model of choice, mocking out any

interfaces or integration points so that the code they’re working on can be

tested in isolation and when any resulting automated testing is run as part of

the build process, the test is run isolated from any other code dependencies.

There are a few problems here though.

For a kick-off, the very notion of automated testing is problematic. You can’t

really automate testing. Testing is a cognitive, analytical, human activity – not

something that can be performed by a machine. A machine can only do what

you tell it to. If you tell it to assert against a boundary or edge-case, that’s

what it will do. If you don’t, it won’t. So that means that for an automated test

to run, you have to know in advance exactly what it is you are going to test

for. It doesn’t take into account the things that you might find out about your

code along the way.

When humans test, they generate information about your program. When

a machine tests, they generate bits of data. 1’s and 0’s might give you some

information about your software, but it’s a black and white kind of a situation.

If you want the full spectrum of greys, you need to pull in some human

intelligence.

Dogfooding

Having your own people test pre-release code is actually a pretty decent

strategy. But it largely depends on what the nature of your software is and

how much people can use it without disrupting their normal, business as

usual activities.

6

Microsoft has had some success with this approach over the years, but its

bread and butter is productivity software. Microsoft can have its people

dogfood an email or word-processing application very easily since a good

proportion of their people are going to be using that stuff all the time anyway!

For more niche applications, this approach might prove more difficult.

Dogfooding works well for web applications. It can help to reduce the cost of

up front testing where the complexities of the production environment are

prohibitively difficult to simulate. The nature of web applications also makes

it possible to carry out controlled experiments like A/B-split testing or con-

trolled test flights where potentially dangerous code is floated out to a small

percentage of the user base. Combined with monitoring, dogfooding can be a

very powerful tool indeed.

Logging and Production Monitoring

Adding logging to the software seems like a no-brainer. Why wouldn’t you

want logging? Anytime you get an unhandled exception in the code you can

trace the activity through the stack and see what the user was doing when

things went wrong, and on what kind of an environment or browser they were

working in.

Likewise with monitoring. If you have historical data that we can compare

the current situation with, then changes in user behaviour can be monitored

over time and you can see whether or not new features are having the desired

impact. If you find that users are changing their behaviour in ways either you

or they don’t like, you can take action accordingly.

Monitoring also helps you to see whether or not there are any non-function-

al concerns. You can see for example how long it takes a user to complete a

specific task like searching for and landing on a specific item, or making a

purchase. You can identify how real users are carrying out their various activ-

ities and look for improvements in both flow and response times.

7

None of the activities listed above require a dedicated tester to carry them out. In

many organisations, they can and are being carried out by developers, operations and

support staff. With that in mind, it may seem like testers shouldn’t be much in demand.

But that’s actually not the case. Though it’s fair to say that the market for testers has

changed considerably over the last 5 years or so.

There’s very little call for Manual Functional Testers as they used to be called, for

example. The market’s all about Agile Testers and Developers in Test these days. There

are obviously some specific competencies associated with those job titles. Whether

you have those vogue skills will have some bearing on your career trajectory, without a

doubt. But there’s some skills you can work on that never go out of fashion.

If you can prove your worth on an agile development team by employing critical thinking

skills throughout the backlog design and grooming phases of the project, exploring

requirements with domain knowledge and specific examples, employing the art of

humble enquiry along the way, you’ll make many developer friends and help them deliver

a top quality product as a result.

Then, using your “mad experiment design skillz” while they’re cutting code, you can help

them to design effective automated tests/checks that help constrain development to

the functionality that needs to be delivered and prove it works the way the business

The Opportunity

You can also look for potential security issues, some of which may relate to

business logic, as well as the more specific vulnerabilities that inevitably rise

to the surface over time.

8

intended. Having that layer of automation will ultimately make life easier as you carry

out some manual testing, since you’ll be able to use it to ensure you get a reliable build

and potentially as a taxi-service to get the build exactly where it needs to be for you to

test it.

The Bright Future

Do you think your services as a tester are no longer in demand? Do you think the job

market has no place for testing professionals any longer? You couldn’t be more wrong.

But you need to make sure you have the right skills. By working on your test design and

critical thinking skills:

You can be responsible for the dogfooding – working directly with the business users

to help them test the pre-release product, capturing any issues as they arrive and

reporting them back to the development team.

You can help define the areas that, based on your exploratory testing, you think need

additional logging in case they do go wrong and help the team with analysing faults if

they do occur.

You can work with the production monitoring team or system to carry out various kinds

of testing in production, providing valuable results and feedback to your team to help

them in the next iteration.

If you can add value to your team by helping provide information about what users really

experience in the production or pre-production environments, by designing dogfood

and production experiments – you’ll always be in demand.

9

The Ideal Tester to Developer Ratio

Hiring Testers

Justin Rohrman, Excelon Development

10

I was working on a pretty tough project early in my career. This was back in the bad

old days, when waterfall was the go-to project management tool. We were at the end

of a release, in the testing phase, and all the testers were working evenings and some

weekend days to get through the queue of work. We were all left feeling burned out at

the end of that release.

The Monday after the release, I asked if we could hire a new tester for our team. The

answer I got from my manager was that I needed to show how that would make sense

financially. I’m not an accountant, so I never did the analysis.

What is the right test-to-dev ratio, and how can you figure out what’s right for your team

without working in the accounting department?

Software development is often a process of handoffs — even in agile, and even when we

do two-week sprints.

Developers might start a sprint with a handful of tickets to work on apiece. They work

for a day or two, then I check in and ask how things are going. The developer tells me

that everything seems to be going well, but they need a little more time. They don’t

really have anything testable just yet.

After a day more I get a small change from one developer and start working on that.

I find a problem where entering a bad date in a date field caused an exception to be

thrown when I submit the page, and another where a button is overlapping a text field

in older versions of Internet Explorer. I report those issues to the developer, and he lets

me know he’ll get them in his queue of work.

Analyze The Problem

11

New changes ready to test start rolling in a couple of days before the end of the sprint

and our inevitable deploy to production. We have to test a new report, a new manage-

ment workflow, and a new page for managing those workflows — oh, and the two bugs I

found a few days before. All of that is a lot for a team of two testers with two days to go

before a release.

Presto: We now have too much work and not enough testers to do a good job. If we did

have enough testers, we would have the new problem of fixing and retesting all the

problems they find in time to release.

We had several problems. First, we started the sprint with large swaths of work, and

a single card in our bug-tracking system might take around five days to complete, on

average. Next we have varying amounts of developer testing. Some developers created

unit tests, and some did nothing aside from quickly open up a browser after they were

done writing code. Last, we had physical and mental separation between roles on the

development team.

Our tester-to-developer ratio was completely inappropriate for how we were develop-

ing software, especially right before we wanted to put out new software. We had three

options: recruit and hire an additional tester, work to make our development process

more effective and efficient, or both of those things. Here is where I would start.

Break Work Into Smaller Chunks

The first thing I would do in this scenario is learn to break down work into smaller

pieces. In my experience, a day or two is a good upper limit for completing a change.

Smaller changes are easier to understand, isolate and, more importantly, test.

12

When I look at longer running changes I have worked on, ranging from a week to several

months, there were inevitably surprises. We discovered new scope, forgot to test

things or had to rework parts of the code that were implemented poorly. This philos-

ophy can easily stink of big, upfront design if we aren’t careful. The goal here isn’t 100

percent reduction of uncertainty; it is understanding a change just enough that you can

get it done in a day or two.

Developers tend to start performing some testing after you get in the habit of making

very small changes. Because very small changes are easier to understand, I see devel-

opers begin to consider how and what to test.

I like pairing with developers. We start by thinking about what we would test, and then

we write that test. After that, we write the product code that will make the test pass. On

a normal change, we do this back-and-forth process of writing tests and then produc-

tion code until we get a gut feeling. That feeling lets us know that we are probably far

enough along to create a new test environment and do some exploring.

This isn’t about pairing, though; this is about managing the tester-to-developer ratio.

So why does this matter?

Software that is written in very small chunks and is built by people who care about

quality tends to be pretty good. I don’t generally see dead-on-arrival builds, back-and-

forth bug reports and bug fixes, or time spent in bug triage meetings trying to figure out

what matters, what is a bug and what is a “feature request.” There are fewer bugs, and

the bugs that are there tend to be more interesting and harder to find.

This workflow requires fewer testers — they just also need to be more highly skilled.

13

It is easy to look at a bottleneck — getting a large amount of testing done right before a

release, for example — and automatically think you need to hire another person. But the

tester-to-developer ratio tends to solve itself if you examine the actual problems you

are having.

How long does it take to get one code change done? How much developer testing and

quality emphasis do you see in the development process? How closely do the different

roles on your development team work?

If you are working on small changes that last somewhere around a day, test-driving

where you can and pairing sometimes, and you still have a bottleneck, then maybe you

do need another tester. By that point, though, it should be apparent that the next step

is hiring a new person; in this situation, there’s no accountant needed.

Figure Out Your Ratio

14

Find, Interview and Hire Great Software TestersSimon Knight, TestRail Product Manager

Hiring Testers

15

Are you a test manager, QA lead or recruiter tasked with building and growing your QA

team? Or were you asked to find and recruit your company’s best next tester, quickly? If

so, you can probably empathize with my fictitious test manager’s diary entry.

Another day, another inbox full of recruiter emails. All of them have got CV’s or resumés attached that I somehow need to decipher and correlate to the skills I’m looking for. Most of the emails I’m sent are a complete waste of time.

If the candidates actually made it to an interview, I’d quickly discover that they don’t have the experience their CV says they do, they can’t challenge assump-tions the way I think they need to, they’re not committed to learning and they don’t have the confidence and the communication skills I need from them.

Recruitment is a time-consuming, energy-draining task that takes me away from the things I should be focusing my attention on. I could live with it, if ulti-mately we managed to find the right people and hire them. But we can’t even seem to do that.

Every so often I’ll find someone who actually is a good fit, and has the skills I’m looking for. But then we don’t manage to get them through the door. Somehow, they fall out of the process.

Maybe someone makes them a better offer, or they decide to stay put, or something else happens to make them change their mind. I feel like I’m wasting my time here.

Why can’t we find the right candidates? Why aren’t the wrong candidates being filtered out of the process? Why doesn’t the offer and joining process work? Am

I the only one who feels like this?

Why is hiring testers such a challenge? Where are all the awesome testers hiding? How

do you know whether a tester is awesome or not? And once you’ve found one, how do

you attract him or her to your company?

In this article I’m going to tackle the various stages of tester recruitment, namely:

Finding and attracting them. Interviewing them. Hiring them. Feel free to skip ahead if

a specific section provides the information you’re looking for. But before we get into any

of that, let’s get the most important part out of the way first.

16

Figuring Out What You’re Looking For

Do you really have a clear understanding of what attributes are important for the role

you’re trying to fill? If you don’t, then inevitably at some point further down the line you

are going to encounter some difficulties.

In the best case scenario you will discover that the CV’s you’re receiving don’t match the

kind of person you think you’re looking for. Or, during the interview stage you’ll realise

that the candidate can’t solve the kind of problems you need them for.

Wouldn’t it be better to figure out what your ideal candidate looks like in advance, so

that you can target them more effectively from the outset? This way, when you get their

CV or start to interview them, you already know that they have the skills you’re looking

for, and how to test for them.

Starting the process with a checklist or set of heuristics could work. For example, you

might have a list of questions like the ones below:

• Does the candidate understand what testing is? Can they explain it, clearly and simply?

• Do they have experience of working on software development projects? Do they understand the nuances and pitfalls?

• Can they distinguish between different approaches and how they may be more or less appropriate depending on the project context?

• Can they work well within a team?

• Are they smart?

• Do they get things done?

• Can they design a test? Can they perform testing work? Can they operate, design and build tools to help them perform testing more skilfully?

• Can they communicate effectively?

• Can they think critically and creatively?

• Do they have some domain knowledge for the industry in which you operate?

17

Or you could use someone else’s list: Huib Schoots has done a great job of identifying

a set of criteria in his article, Heuristics for recognising professional testers. And Ilari

Henrik has some superb ideas about what makes a World Class Tester that may also

be useful in your search. Johanna Rothman has some fantastic resources in her Tips to Streamline Your Hiring. And Cem Kaner has a detailed guide to Recruiting Software Testers that contains a number of lists that can help you tighten-up your requirements.

Ultimately, if you want to hire the best testers for your team, it would be wise to spend

some time figuring out what skills and attributes are most important to your organisa-

tion. Once you’ve gone through the process of trying to answer all of those questions,

you can document and communicate the answers, so that all of the people involved in

the hiring process can see exactly what success looks like, making it easy for them to

objectively decide whether or not a candidate meets your requirements.

How to Find Great Candidates

The thing you need to do next is to take those qualities and try to find some testers

that have them. But here’s the bad news. The days of just publishing a job posting in

a newspaper, or on the web, and then waiting for the applications to roll in, are over.

All the great candidates, i.e. the testers you want to hire, already have a great job,

and plenty of options, without needing to scan the job listings or the internet for new

opportunities.

Those testers aren’t looking for you. And they’re not paying attention to the job market.

So how can you find them? You could rely on a third party. Maybe you don’t feel like you

18

even have a choice, because the HR department normally takes care of all that. Them

or the preferred recruitment company that insists on spamming your inbox. Maybe

though. Just maybe, you could get a bit more hands on, by:

• Going to a community event: You may not know it, but testers are everywhere! There’s almost certainly a meetup near you. And if there isn’t, why not think about starting one? You don’t have to go far. You could even hold it in-house. Plenty of other people have!

• Going to a testing conference: Just go along and put the word out. Alternatively, ask the organiser if you can be a sponsor, or if you can make an announcement of some kind.

• Connecting with testers on the net: You can start following the testing community online via Twitter or LinkedIn, and the latest happenings in blogs and testing forums. The testing community is very vocal. You won’t have to look too hard!

• Publishing material testers want to read: If you’re starting to engage with testers on social media, you should definitely think about how you can contribute to the conversation. Creating blog posts and articles that intrigue and excite testers is a great way of drawing them into your sphere of influence.

• Setting testers a challenge they can’t ignore: If you feel like you still need to advertise, make it interesting. Attention spans aren’t what they used to be so your copy really needs to catch the eye. And what stands out more to the kind of tester you want to hire than a testing conundrum?

If you’ve followed some of the advice above, then it’s quite likely that you will find

someone you want to recruit. Perhaps you bumped into them at a meetup or a confer-

ence, or read a blogpost or Twitter discussion that piqued your interest.

But now you’ve found them, how do you attract them into your organisation? Well – you

could just ask the person in question if they want to come and work for you. But, maybe

they’re not ready for that yet. Perhaps they’re already in an [employment] relationship.

19

Play the long game You can frequent some of the places where your tester hangs out. Get to

know the forums or groups they’re a part of on LinkedIn for example. Or find a

great blog post they wrote and comment on it. Let them know you’re interest-

ed in their work.

If you manage to get their attention, then you can take it a step further and

tell them about the opening you’ve got and how you think they might be a

perfect fit. Let them know how impressed you are with their work, and their

background; reference an article they wrote or a time you saw them speak at

a conference – you’ll be on safe territory.

Sell the role. Tell them all about how awesome your organisation is and why

they wouldn’t want to let the opportunity pass them by. Focus on the benefits

to them. Can they work from home? Do they get the best technology?

Fantastic training? Once you’ve got them hooked, it’s time to take the next

step.

How to Interview Testers

By this stage you should have a tester that you think maybe you want to hire. Perhaps

you’re in contact with them because you’ve engaged with them directly, or maybe their

CV has arrived on your desk via some other channel. Irrespective – you need to figure

out whether or not they’re right for your team, and whether they have the skills you’re

looking for.

20

Traditionally, the way that folk have gone about this is by means of an interview. But, if

you pay any attention to advances in psychology then you’ll probably already know that

most interviews are basically an exercise in confirmation bias. In the classic interview

scenario, whoever does the interviewing will make a decision based on their gut

reaction to the candidate within the first 30 seconds. Then, they’ll spend the rest of the

time looking for reasons to justify the decision they’ve already made. If you want a more

scientific outcome, you need to move away from this model. I have a couple of sugges-

tions for ways you might proceed instead.

Give them a call If you’re going to follow my recommendations, then you should a) warn the

tester in advance what they can expect and b) be certain you are investing

your time and energy into the right people.

Use the telephone call script below to gauge whether or not it’s worth con-

tinuing the process, tailoring the easy and hard questions so that they’re

specific to your own requirements:

1. Introduction

2. Question about recent project candidate worked on

3. Easy Testing Question

4. Hard Testing Question

5. Are you satisfied?

6. Do you have any questions?

If they make it through the screening call, then you can take them to the

next stage.

21

Give them a work test Setting your candidate a task that is representative of the kind of work they’ll

be doing for you is a great way to get a better idea of how they think and

behave in a work situation. And there’s lots of different approaches you could

take to keep things fun, or more serious, both in and out of the office.

You should keep in mind the objectives of this approach though. What is it

exactly that you want to know about your candidate and how will you measure

those things by giving them a work test?

If you’ve followed my earlier advice and have gotten really clear on what

specific skills, competencies, personality or attributes it is that you’re

looking for from your tester, you should have a good idea of what information

you’re trying to get out of the work test. You just need to tailor it so that it

answers your specific questions, and any additional considerations like:

• Can the candidate communicate their thinking well?

• Can the candidate complete the exercise within the time allocated?

• Does the candidate have specific domain knowledge?

• Does the candidate have specific skills that can be applied – tools, automation, critical thinking etc.

If you’re stuck for more ideas, Richard Robinson has a couple of great

examples in his Needle in a Haystack article, available in the July 2014 edition

of Testing Circus magazine. What you really need to know next though is what

“good” looks like, once you get the results.

Stick to a script Understanding what good looks like gets much easier when you follow

exactly the same process every time you speak to a candidate.

If every candidate does the same work test, then over the course of time you

will know what good, bad and mediocre looks like for that test. If you already

have an example of what good, or even excellent looks like – even better.

22

The same can be said of interviews. You need to stick to a script. Try to make

sure whoever is doing the interviewing asks exactly the same questions

every time. And record the results so that you have data from which you can

start to draw meaningful conclusions, even if you don’t actually hire anyone.

The results might surprise you and provide some actionable insights about

your hiring process. The kind of people you’re targeting for interview in the

first place, for example.

Give them a work day If you want to go the extra mile, or still have some reservations – get the

candidate in to work with your team. You can start to build a picture of how

good of a cultural fit they are and how they relate to and communicate with

their potential future colleagues.

Inform the candidate that they’ll be in for 3-4 hours, then start off with some

introductions and the formal interview. After the formal interview, introduce

the candidate to your team and leave them to get on with some work for a

couple of hours.

Ideally, try to have a specific piece of work lined up. It should be something

that will require them to collaborate with at least one other member of the

team in order to deliver successful results.

After the work, give the candidate a break to gather their thoughts while you

catchup with the team and get their feedback.

Finish with a debrief based on how things have gone so far. Do they have any

questions or insights based on the experience. Do they still want the job? Do

you still want to hire them?

If the work day is a success, then you’re ready for the final stage.

23

Okay, so you’ve attracted an ideal tester and you now know: a) they can do the work and

b) they’re a fit for your organisation. It’s time to seal the deal. How are you going to make

sure that your tester actually makes it through your door, and not one of the other op-

portunities that they may have available to them? What are you going to offer them that

makes your opportunity stand out among all of the other roles that are available to the

brightest, best, most high-flying testers out there?

It’s probably not about the money, right? Maybe you have some flexibility with how much

you’ll remunerate the right person. But there’s a cap. Once you’ve reached it, there’s

no more wiggle room. And let’s get this straight. Beyond a certain amount of money,

folk aren’t motivated to work harder or perform better anyway. So we need to think a

bit more creatively. What else can your organisation offer that may be attractive to a

certain kind of hire?

• Flexible working hours?

• Remote working?

• Unlimited holiday?

• Training and personal development?

• Coaching?

• The latest most powerful hardware?

• A free lunch?

• Time off work to attend conferences?

How to Hire the Ideal Candidate

24

At the beginning of this article, my fictional test manager had quite the recruitment

headache. She could have just outsourced the entire process, but I’ve recommended

the following course of alternative therapies:

Identifying the specific characteristics, attributes and skills a tester needs to have in

her organisation.

• Engaging with the testing community in order to find awesome testers that have the abilities and personality she’s looking for.

• Striking up a relationship with the tester directly.

• Screening potential candidates with an initial telephone call.

• Testing candidates with a work simulation.

• Sticking to a script during the entire process – making it more rigorous and scientific.

• Inviting the candidate to a work day – soliciting feedback from both them and the team.

• Negotiating on working practices rather than money during the offer stage.

Recruiting testers may be challenging, but it’s not impossible.

Curing the recruitment headache

At this point, it’s not even necessarily about what you can or can’t offer. You just need to

be willing to have a conversation about it. Find out what it is that drives and motivates

your new hire. Once you know what it is that will get them through the door, if you can,

give it to them.

Work with them. Figure out how to bridge the gap between what they want and what

you can offer. Attracting the right people is hard enough. Don’t let them get away once

you’ve found someone that you definitely want.

25

25 Challenging Tester Interview QuestionsSimon Knight, TestRail Product Manager

Hiring Testers

26

In the previous article, I wrote about an approach to finding, interviewing and hiring the

best testers. In it, I suggested that when carrying out an interview you should stick to a

script so that, if you need to compare results across a number of interviews, you can at

least be assured that the same questions were asked each time.

To help you out even further, we thought we’d go the extra mile and provide you with

a list of questions too! And to make sure we had the very best questions, we even

approached some of the best thinkers in the testing community and included their

responses below.

When you start your interview, you want to give your candidate every possible opportu-

nity to shine and show you their best qualities. Use the questions below to help break

the ice and get them warmed up ready for some tougher follow-up questions.

1. What is testing to you? Do you have a testing paradigm? Can you explain it?

(Suggested by Kim Knup @punkmik)

2. Why did you choose software testing as a career, and what motivates you to stick

with it? (Suggested by Dan Billing @TheTestDoctor)

3. Tell me about the work you’ve been doing recently. What’s the most interesting bug

that you’ve found, and why? (Suggested by Katrina Clokie @katrina_tester)

4. What kind of challenges does testing present? Can you tell me about some specific

software testing challenges you’ve faced, and how you overcame them?

Questions For Warming Up the Candidate

27

Hiring an awesome tester isn’t just about making sure they have the right skills for the

work you need them to do. It’s of paramount importance that they are a fit for your

organisation and the team they’ll be working with. Use the questions below to help you

figure out where their strengths and weaknesses lie, and whether their values align with

yours.

5. Have you encountered any challenges working with your colleagues? Tell me about

a specific instance when you were in a difficult situation, and how you dealt with it.

(Suggested by Katrina Clokie @katrina_tester)

6. How about situations where you have to make decisions… Have you ever made a

bad decision? What contributed to you making that decision? How did you deal with

the consequences? (Suggested by Kim Knup @punkmik)

7. How have you added value to the organisations you’ve worked with? Can you give me

a specific example from your last or current position? (Suggested by Dan Billing

@TheTestDoctor)

8. What process are you using for testing currently? Can you describe how you might

improve it? (Suggested by Katrina Clokie @katrina_tester)

Questions to Test for Behavioral and Character Traits

28

Now we’re getting to the good stuff. What kind of a tester are you dealing with here?

Do they know their edges from their corner cases? Can they distinguish between a test

and a check? Do they understand their biases and models, and how a worldview can

influence their approach and decision making process? Use the questions below to dig

deep into their testing toolkit and to make sure they don’t view every testing problem as

a nail to be pounded with a blunt instrument.

9. What kind of tests have you been doing? What do you enjoy about them? How do you

develop those tests?

10. When you perform a test, what steps do you take? What’s your process?

11. Have you ever written a test plan? What would you put in one?

12. How do you prioritise your testing? What factors might influence your decisions?

13. How do you know when it’s time to stop testing?

14. What’s the role of risk in your testing? How do you analyse and measure it?

15. Do you measure how effective (or not) your testing is? What metrics do you use?

16. If I left you testing for two hours, what would you have to show me when I returned?

(Suggested by Richard Bradshaw @FriendlyTester)

Questions to Determine A Candidate’s Testing Skills

29

No doubt many readers will be hiring for automation skills; but being able to automate

testing activities isn’t just about understanding the right tool, language or framework.

It’s about identifying where the risks to the product are and then being able to pinpoint

those risks by developing intelligent, reliable and repeatable automated tests. You want

your tester to help you speed up the development and delivery of your product, not slow

it down. Use the questions below to help you figure out whether they have the depth of

knowledge to do so.

17. Have you automated any of your tests? How so?

18. What’s your favourite testing tool? Why? If some technical constraint meant you

were unable to use it, what would you do instead? (Suggested by Richard Bradshaw

@FriendlyTester)

19. How do you know when you (or your automation) has found a bug? What makes

it a bug? Are some bugs more important than others? How do you report them?

(Suggested by Richard Bradshaw @FriendlyTester)

20. What do you do if the developers decide the bug is not a bug?

21. How do you decide which tests to automate? Which tests don’t you automate, and

why? (suggested by Katrina Clokie @katrina_tester)

Questions to Determine A Candidate’s Testing Skills

30

Finally, if you’ve made it to the self-development questions and are still interested in the

answers, you may have found a keeper! But are they in it for the long term? Is software

testing a passion, or just filler until they can find a proper job? How do they learn, and

what keeps them motivated to do so? Use the questions below to probe your testers

long-term commitment to the role.

22. Testing can be challenging. What keeps you motivated? (suggested by Kim Knup

@punkmik)

23. How do you stay at the top of your game? What self-learning do you do? (suggested by

Dan Billing @TheTestDoctor)

24. What have you been criticized for in the past? How did you respond to that

criticism? What did you do about it?

25. Describe the characteristics of your ideal boss, and why.

This completes the list of 25 questions that you might pose to a candidate in an

interview. In the next article, we’ll take a look at 21 questions that a candidate should

ask in return.

Questions to Determine A Candidate’s Testing Skills

31

21 Questions to Ask When You Are Being InterviewedJustin Rohrman, Excelon Development

Hiring Testers

32

In the previous article, Simon Knight described questions that a hiring manager can

ask candidates during an interview. In response to that article, I wrote one on the bias and problems built into different styles of interviews, and some ways we can counter

that to get a better understanding of the candidate. Ideally, we ask a person the right

questions, and in the right conditions so that we can learn about their skill level and

how they might contribute to our teams.

I also assess candidates based on the questions they ask during an interview. These

questions show that they are curious and paying attention. They show that the person

is able to discover when they don’t know something, and figure out how to seek that

information. And, they of course can learn more about whether or not they want the job

based on the answers.

So, what questions should you ask when you are the one being interviewed?

33

Job advertisements tend to be pretty generic. Most will say something about discov-

ering and reporting bugs, creating and executing test scripts, and generally working to

help the development group produce the best software they can. But, in the same way

that a test case isn’t testing, the job specification isn’t the job. There are whole catego-

ries of things that people forget to describe when they make a job advertisement, and

others that are hard to turn into bullet point lists. These questions are designed to help

you learn what a tester really does at this company.

1. What does a normal day look like for a tester there? What meetings do they go to?

Who do they talk with? How much time do they spend testing software?

2. What skill set do you expect the person you are hiring to have? Can you give some

examples of what tasks you would expect them to be able to perform?

3. What technology and tool stack do your testers need to know? What is the learning

path for someone that is not familiar with a piece of technology or process?

4. How do testers work with developers, product people, and decision makers?

5. How is the relationship between testers, developers, and product people?

Clarifying the Role

34

Companies, and even teams within a company, approach testing from vastly different

perspectives. Some groups deliver software hundreds of times every day and do

most of their testing by layering automation and using radical collaboration tools like

Extreme Programming. Others companies completely separate the testing, develop-

ment, and product groups. Testers are expected to do the testing equivalent of big

design up front, and then perform those plans once there is software to look at. Most

companies today are somewhere between those two extremes. You will probably want

to understand how the company thinks about and approaches software testing.

6. How much time do your testers spend doing things that aren’t testing, such as doc-

umentation, meetings, emails, and status reports?

7. How free are your testers to make decisions on how their testing work is done?

8. How close to development do your testers work? Do you ever have developers and

testers pairing on a code change?

9. What tools do your testers use? What happens if they want to use something new or

different?

10. How does a team know when they are ready to stop testing and push a code change

to production?

Test Approach

35

Culture

Culture is shorthand for the ways people in a company interact, the goals they strive

for, their value system and ethics, and how employees treat each other. Some cultures

foster an environment where lone individuals can control broad swaths of the product,

and are responsible for its problems as well. Others are collaborative, congenial, and

encourage open discussion about where improvements can be made and who the

appropriate people to do that work are. There are also the very important questions of

diversity and inclusion practices. Getting a feel for culture can help the interviewer, or

you, to figure out if you are a fit for this organization and how long you might want to

work there.

11. Do you give your technical team members the opportunity to work from home?

12. How often do you find yourself in a ‘crunch time’ situation where you require people

to come in early, stay late, or work on weekends?

13. What sort of diversity and inclusion practices do you use to make sure you can build

and maintain diverse, high-functioning teams?

14. Does your company financially help testers that want to attend conferences or

attend training. If not, do they offer time off without having to take PTO?

15. How do you prevent burnout of your technical team members?

36

Team compositions vary quite a bit. The last company I worked with had a completely flat

structure. There were 6 or 7 developers, 2 testers including me, and we all sat in the same

room. We had some people that were acting as leads in practice, but they didn’t have the

title or the power that comes along with that title. We all reported up to one development

manager when we needed something, or when he needed something from us.

Other teams I have worked with were steep pyramids. One company had a handful of

testers per group, each group had a lead tester, and a couple of groups combined would

work with someone called a test architect. There were also managers and directors in the

mix. Everyone reported up to someone else and the ladder was tall. My personal preference

is to work in a more flat structure. I feel like it is easier to get things done, and decisions

come through much faster. These questions can help understand how flat or deep the

organizational hierarchy is at a company, and how that will affect the testing work.

16. What team would I be working on assuming you decide to hire me?

17. How exactly do the different roles work together? Do they sit in the same room?

Do they sit at the same table? Do they have to schedule meetings to talk?

18. Who would I report directly to? What is that person’s relationship to the team?

19. Why are you wanting to add a tester to the team?

20. What is the biggest complaint your testers have now?

21. What do your testers like the most about the team and company?

Clarifying the Team

37

Emergent Questions

This is a tricky category because it is dependent entirely on the flow of conversation. I

usually have a laundry list of questions I want to ask people I’ll be working with and what

you see above is an example of that. But, I also come up with a lot of questions based on

what we are talking about at any point in time. Technology work is full of lingo, some of

it is overloaded to mean different things at different points in time. In testing, we don’t

have a lexicon, or a standard dictionary. A lot of my questions focus on that. An inter-

viewer might start talking about their regression testing process and I’ll say “When I say

that term I mean the testing we do to discover new problems introduced by a change. Is

that what you mean?” I also like to flip questions back on the interviewer to learn about

them. If they ask me how I would test something, after I give my answer I like to ask

them the same question. This can be a learning experience, and can also be a litmus

test for how the conversation is going.

This is just a few of the questions I like to ask. They help me learn much more about

what sort of job I would be getting into. Sometimes, they help the people on the inter-

viewing side to better understand what they are looking for, and what they really need in

a new tester.

38

Retaining Your Software TestersSimon Knight, TestRail Product Manager

Hiring Testers

39

I’ve had a bit of experience with wanting to leave software testing gigs. During my

career as a tester I’ve worked for a number of organizations in various different capac-

ities. From my days as a rookie through various layers of senior, technical and automa-

tion roles right up to the dizzying heights of test management.

Over that time, as you might well imagine, I’ve had a range of experiences in the

groups, teams, departments and corporations within which it’s been my pleasure (and

sometimes my displeasure) to work.

In total I have worked for 14 different companies during my testing career so far.

Granted, 11 of them were on a contract basis – so I was never in any real danger of

having to stick around for too long, but nevertheless; a few patterns have started to

emerge. I have been able to make a number of observations about which companies

I would have chosen to stick around with and work for in the longer term, and which

companies I definitely wouldn’t.

In my experience, those factors break down into several commonalities which I believe

can probably be applied to most testers. They are as follows:

• Opportunities for learning and development, or lack thereof.

• Recognition for contributions, or lack thereof.

• Pay, relative to the organization and the wider marketplace.

And that’s basically it. In any situation where you’ve got someone wanting to move on

from a role, there’s probably any number of other contributing factors at work. But in

the technology field and particularly where software testers are concerned, I think

these are the main things that will determine whether or not a tester decides to stick

around, or decides to twist and look for a role elsewhere. So let’s take a look at the

problems in more detail.

40

Learning and Development

Testers along with many of our more or less technical colleagues working on software

development projects are a curious bunch. They like to learn as much new stuff as they

possibly can. Which provides employers with a couple of interesting challenges.

If you have a tester who is not curious, you most likely don’t have a very good tester.

All of the best testers I have met are innately curious and have a burning intrinsic mo-

tivation to poke and pry and break or improve things. They may have various degrees

of aptitude for some of the ancillary skills you’ll be looking for in your particular brand

of tester, like coding and automation ability for example, but at their core they will be a

curious little beaver.

If your tester isn’t curious – you hired the wrong tester. Read the earlier article in this

ebook on hiring the best software testers and try to do a better job next time.

If your tester is curious however: You had better be giving them plenty of rope with

which to exercise their curiosity. Otherwise you’re going to end up with a sad little

tester because they have nothing new to learn. And sad testers tend to develop a desire

to move on and explore new pastures instead. Pastures where their curiosity will be

recognised and rewarded, not hampered or curtailed.

I’ll provide you with some strategies for exercising your testers curiosity at the end of

the article. But next, let’s take a look at recognition.

41

Recognition and Rewards

Testers don’t like to be treated as 2nd class citizens. There. I said it. Hopefully this will

not be news to most readers. But just in case you need some further clarification on

this point.

I have seen and been in the position of being treated like the work I’m doing doesn’t

matter. Like it doesn’t add value to the project on which I’m working. It’s not fun. I don’t

like it. I’ve never met a tester who does like it. You wouldn’t like it. So don’t do it.

Here’s an idea instead. If your development team don’t feel like they need a dedicated

testing professional, then don’t hire one. Simple, right?

Or, if you’ve hired a tester who genuinely isn’t adding value, and who is for some reason

unable or unwilling to put in the work to improve themselves to a point where they can

and are adding value – sack them. Try to hire better the next time.

If on the other hand, you have a tester who is adding value -- who is helping your team

to deliver a superior product or service -- celebrate them and their work (along with the

rest of the team, of course). Because delivering great quality products is a team effort

and good testers play a big part in that. How big of a part will likely determine the next

point.

42

Compensation and Pay

How much folk are paid for their efforts is clearly a tricky subject. People don’t generally

like to talk about pay (certainly in the UK where I come from anyway) but they surely do

like to feel as though they’re being remunerated at a level that correlates with the value

they add.

When I was a junior tester I got paid a junior salary. As my experience and skills

increased my remuneration increased accordingly. Now I consult and charge a fee for

my services. Clients are happy to pay that fee because they can see the value that is

being added.

Clearly that path is not for everyone and is most likely not a great model for your team or

organisation if you’re reading this as a test manager. You may also have any number of

constraints around how much you’re able to pay people. But I’m just making the point.

There are plenty of opportunities out there for ambitious and skilled testers. So if you

have a good one and you want to keep a hold of them, you should be prepared at some

point to have a serious conversation about their benefits package.

If they’re maxed out on the payscale, try to think about whether there are some other

things you can do for them by way of perks and bonuses. Some new hardware for

example. Remote working. 20% time. Travel. Etc. Try to think out of the box. It’ll be

worth it.

43

Additional Strategic Advice

If you’ve taken some of those points on board, then you’re most likely looking for some

ways to actually address the issues. Good News! You’re in the right place. Try imple-

menting some of the ideas on the checklist below.

Your Tester Is Curious

• Maintain a testing book library (the books in this list will make a great start)

• 20% of their time can be set aside for skills and interest development (so long as it can be tied back to work)

• Send them off to learn other areas of the business so they can expand their skills and increase their domain knowledge

• Pair them with developers so they can learn some coding and development skills to supplement their testing knowledge

• Start a regular brown bag lunch session so that testers (and anyone else who is interested) can eat lunch together and share knowledge, skills, useful experiences, lessons etc.

• Start a meetup and invite people from outside of your company to it so your tester gets exposed to other ways of thinking and approaching software testing

• Send your tester for some training

• Send your tester off to a conference

44

Your Tester Needs Recognition

• Try to observe when they’re doing an awesome job, in the moment, and praise them for it

• Implement some kind of public recognition program and nominate them for it

• Credit them publicly for their work, in a standup or show and tell for example

• Give them some more responsibility and make sure they and others know why

• Spend time with them one-on-one and tell them how much you appreciate their work, over lunch for bonus points!

Your Tester Needs Rewarding

• Give them some training budget – even a few books can make a world of difference

• Send them to a conference or some training

• Upgrade their hardware

• Give them some extra time off or a more flexible working arrangement

• Cut them some slack – even if you can’t promote them, you can provide them with greater leeway and authority with which to do their work

Bear in mind that these are just ideas to try and get you started. Hopefully as you read

through them, you’ll find some of your own ideas being stimulated. As you do, add them

to the list and try them out. By paying attention to their needs, you’ll be more likely to

keep your best testers on board.

45

Making Do When You’re UnderstaffedErik Dietrich, DaedTech LLC

Hiring Testers

46

Are you feeling spread thin when it comes time to test your application? Is your team

missing deadlines and pushing back releases to validate your product? This is a common

feeling. It manifests itself in many ways.

For example, picture the following scenario. Your team is understaffed. You can’t

get through testing fast enough to get features out quickly. It’s all hands on deck—

the testers, developers, analysts, and managers are all stepping in to help test new

features. Deadlines slip and morale drops as testing takes center stage.

Or how about this. Your team is writing software with a hard deadline. Regulations are

changing and you need to be ready. But as usual, development took longer than

expected. Your QA team wanted two weeks to test, but they’re only going to get three

days. And even after pulling extra hours, you’re still behind.

So now you know you’re not alone. But where do you go from here? The short answer to

your problem is “automate.” To which your response will probably be, “But we don’t have

time to automate! We don’t even have time to test!”

OK, let’s look at the long answer then.

47

You have your unit tests that run on the developers’ machines, contract tests, and tests

that run in some other system. Acceptance and regression tests are handled by your

offshore team using SoapUI and reported back weekly. The QA team performs manual

acceptance and UI testing based off their Excel sheet.

All the results are collated, reviewed, and shown to management. Plans to enhance

testing and add new feature testing are discussed, put in a separate document, and

reviewed by the appropriate people.

Then when issues are found, you capture that data in yet another system like Jira,

Pivotal Tracker, or Zendesk. Sometimes you track it in more than one system.

Therefore, your team spends hours each week going from one system to another trying

to keep tabs on it all.

I hope that illustrates the point that all these systems and all this back and forth cause

waste. Take a hard look at what’s really going on with each feature and test. Find spots

where things are repeated and recommunicated. When you have a clear picture, look

for solutions that will let you manage all your tests in one place. Make sure it’s visible

and integrates well with your existing systems.

Manage Your Testing

48

Automate Your Tests

If the testing process itself takes a lot of time and effort, one of your first steps might

be to automate it. Therefore, review all manual tests and look for streamlining opportu-

nities. Take a look at what tools are available to help your team get the repetitive grunt

work done quickly and easily.

However, don’t fall into the trap of testing everything from the UI. Keep the testing

pyramid in mind. And note that the acceptance test doesn’t need to be at the UI level. It

can be at the API or boundary level. Having too many tests at the top of the pyramid will

slow you down. The tests will be flakier, run more slowly, and be harder to maintain. If you

notice your tests are slowing down, it’s time to see if the focus is on the right functionality.

Once you automate your tests, the next step is easier. This is a great example of how

you can make the change easy, then make the easy change.

Test Early, Test Often

49

Prioritize

Prioritization comes in two parts. On one side you’ll have to prioritize which tests need

attention first. On the other, you’ll have to prioritize which issues to fix first.

For the first half, again we have two things to consider:

• What features are a priority

• What types of tests are a priority

After that, prioritize bugs and changes that come out of testing. Make sure everyone

knows the difference between a minor formatting bug and a major “all hands on deck”

bug that needs to be the top priority. Be explicit about prioritization rules. Don’t leave it

to assumptions.

With software testing, the sooner the better. This is why test-driven development (TDD)

pushes testing first. TDD ensures you’re testing before the code is even written. Though

even if you don’t want to take on something like TDD, don’t wait to test everything at the

end. Make sure to test features as soon as they’re ready.

And the more often you test, the more processes you’ll be forced to automate. Making

frequent testing an easy process will ensure you’ve got a streamlined and efficient way

to run these tests.

50

Adopt the Right Tools

The QA team needs tools to be more efficient. And sometimes they need training. Don’t

just throw out new tools and expect the team to use them efficiently. Take time to learn

the tools, share that knowledge, and find the most effective ways to test with them.

Fix Small Inefficiencies

Sometimes test teams get into a rut. They perform the same tests day after day.

Eventually, they stop seeing inefficiencies and parts of the operation that could be

improved.

Have someone sit with a tester for a day or two. Better yet, have someone outside the

regular work group watch what testing involves for a few days. The visitor should note

anything that seems repetitive or redundant. Those steps are the first places you can

look for automation opportunities.

51

Also, encourage testers who know all the shortcuts and tricks to the tools they use to

share that knowledge with others. Testing is a skill that requires knowledge of the tools

and practice using them. It can’t be left to those who hunt and peck their way to a result.

Another opportunity for streamlining is test data. From what I’ve seen, setting up test

data often takes longer than the actual test. If this is the case for your team, work on

automating this data entry. Or figure out how to get around it.

Many opportunities exist for improvement and automation. It’s up to you and your team

to find them.

Come Back To The Short Answer

Now let’s come back around to the short answer of “automate.” Is that the silver bullet?

Yes, if you know where to aim and focus your efforts.

You can’t go from no automation to having everything fully automated and efficient

overnight. Take time to review inefficiencies in your process and start working on

improving them.

In addition, when looking at automation, look beyond just the tests themselves. Are

there other processes that can be automated? Consider tests that report and monitor

themselves. Or tickets that close themselves after a successful test. When we talk

about test automation, it shouldn’t just be the tests that we’re automating, but the

whole process of ensuring a quality product.

52

How to Hire the Agile Team You NeedJeff Langr, Langr Software Solutions, Inc.

Hiring Testers

53

You’re interested in hiring for an existing team that purports to be “agile,” whatever that

means. Tell me, if you can:

• What’s core to the culture of that team — what do they hold near and dear about how they work?

• Which one technical element is a must-have — if there are any?

If you can’t answer these questions succinctly, you’re jumping the gun with your hiring.

Here are six major points to consider — three job requirements essentials, and three

important aspects about hiring — before you draft a job requisition for an agile team.

Agile Essentials

I believe the success of agile is predicated on two things: creating a highly collaborative,

cohesive team, and fostering the willingness to keep learning and adapting.

When I say highly collaborative, I mean team members must work with one another

throughout the day, and they must work on as few things as possible. If your team

members spend most of their day in their own cube, or if you have as many current work

items as team members, your team is not working in a highly collaborative fashion.

If collaboration and adaptation are paramount, what should you put into your job requi-

sition (“req”)? Keep it simple: Describe briefly how you currently work and your interest

in making people comfortable with continual learning.

“We mob occasionally, and when we don’t, we prefer to pair. We seek folks comfortable

working in this fashion and with regularly evolving team practices and dynamics.”

54

Technical Essentials

Delivering software, agile or otherwise, still demands expertise in the core technol-

ogies we employ, but we don’t need everyone to be an expert on every technology. In

fact, we can do just fine with a few experienced folks in the core technologies as long as

we are truly collaborating. Techniques like mobbing and pairing can do wonders to help

savvy people rapidly ramp up.

Lest you think I’m discounting expertise, I firmly believe it’s essential for at least a

couple of people to know each technology very well. You might survive without that

expertise, but you will also waste a lot of time as a result. (Hint: You can rent expertise.

Just make sure the person you rent it from excels at helping others ramp up on the

technology.)

The worst thing you can put into a job req is a laundry list of technologies. All it accom-

plishes is chasing off candidates who might adapt well to your world.

At one place I worked, we sought an experienced C++ developer to help with test-driven

development mentoring and coaching. Before I got my hands on the job req, it was an

HR-produced morass of a dozen or more technical demands. I simplified the demands

to the following:

“Our developers predominantly work in C++, so we seek people with solid C++ experience—

and people who still want to work in this challenging language. You must also have experi-

ence helping others learn TDD.”

55

People Essentials

One more characteristic I consider important to software development in general (agile

or not) is good problem-solving skills. While we can all learn these skills, some folks

don’t appear to have much aptitude for it. I’ve seen employment end in tears when we

realized that recent hires weren’t able to contribute sufficiently to development efforts,

no matter how much we tried to help them along.

While you don’t need to ask your candidates to solve complex puzzles, they can be

expected to think and work through mildly interesting problems and mundane, typical

challenges. Doing so is pretty much our minute-to-minute job. You can explain this

expectation in the job req if you like, but you could also wait to reveal it during an initial

conversation.

Ultimately, however, what’s most important from an individual stance is that they want

to work the way your team works.

“In addition to regular pairing, we expect programmers to produce developer-focused

tests and to contribute to BDD efforts.”

56

Selling

While you may view hiring as having candidates sell themselves and their skills, the

first mentality should be that you are selling them on your culture — you are trying to

find people experienced with, and happy about, working the way you do. As such, the

first focus of your req should be on making the job sound appealing. Don’t overdo it, but

remember that every hire is a give-and-take negotiation process, and thus so should be

the job req itself.

I mentioned earlier rewriting the job req for a TDD coach. Before doing so, an external

agency sent me a couple of dozen resumes, most with neither C++ nor TDD empha-

sized or even mentioned. Out of the candidates, only two were anywhere near what we

needed, and neither of those survived an interview.

After simplifying the job req to the essentials, we received resumes for five candidates.

Four of those five looked good enough to interview, and all did well in the interviews. We

ended up hiring a top-notch candidate.

57

Interviewing

To give candidates the best opportunities to shine, you can play out the interview

across a number of forums. I’ve done this a few different ways, but here’s a three-step

approach that worked well:

Initial phone conversation: A 15-minute chat to describe the company and

what we were doing, and to get a feel for what the candidate is interest-

ed in (and capable of) doing. Where I worked, this conversation was run by

business folk.

Initial technical conversation: A 15- to 30-minute chat with development

team members to clarify the technical landscape and get answers to a

question or two about the candidate’s background. More importantly, this call

sets the stage for the core of the technical interview.

We asked developers to build code for a handful of increasingly interesting, related

features. Developers were told to get back to us when done to schedule a follow-up. We

expected them to spend perhaps an hour or two on their homework. We would explain that

they would be expected to pair with us when ready, to add one more interesting feature.

If it took a few days for them to get back to us, that was fine; if it took a couple weeks,

that was fine too. Rushing to hire a candidate is never a good idea.

Importantly, we explained that they were free to use whatever technologies they felt

most comfortable with: operating system, programming language, editor and so on.

58

The goal was on the problem-solving, not learning a new technology. We wanted them

to be able to approach the forthcoming demonstration session with as much comfort

as possible. (It turned out that choosing one of our desired technologies, something

they weren’t so comfortable in, was always a mistake for the candidate. We reminded

them of this reality.)

Demonstration session: A 60- to 90-minute session with at least a couple

of us. The candidate was asked to step us through their solution and explain

choices they made. Sometimes we’d ask what they thought about alternative

approaches; I remember dismissing a candidate almost immediately when

they took offense at the thought that there might be a better way.

We then presented the candidate with a new feature, something that would add an

interesting wrinkle to their existing solution. They were expected both to talk and work

through coding a solution at the same time. Our goal wasn’t to put them on the spot; in

fact, we sought to help them as they worked.

The candidates we dismissed were people who weren’t capable of interacting with us

effectively. Some wanted to turn their back and scribble things for a while; while that’s

perfectly acceptable, the candidates we appreciated more were the ones who could

talk us through their thought processes. We most rapidly dismissed the candidates who

weren’t open to listening to our suggestions.

Apart from information about their ability to solve problems, the most important part of

this interaction was discerning whether we thought we wanted to work with them every

day as part of our team.

We made it clear that either the candidate or we could choose to terminate the process

at any step.

Sometimes we ended up on the fence. For a while, we had a second pair of developers run

another interactive session, but this second session always resulted in the same uncertainty.

We decided to end the interview process, one way or another, after a single session, and I

believe that was the best choice. Either we knew they were a good hire, or we passed on them.

All of the above steps can be accomplished online, though of course, face to face is

usually far more revealing. If at all possible, get the candidate to visit in person.

59

Diversity and Culture

A successful organization embraces diversity of thought and approach. While I firmly

believe this diversity is important, I also believe that building a culture is an important

aspect to building a successful team — which also implies that I believe teams within

the same organization can and should have differing cultures.

The culture represents the shared beliefs, attitudes and preferred ways of working for a

team. A healthy team understands its culture and accepts that its culture will change as

it matures. It also accepts that “immature” new ideas might benefit the team, and that

experimentation might be worthwhile.

While keeping open to new ways of thinking, a team should not feel like it needs to hire

people who will directly subvert their current ways of working: “Oh, you do pairing? I

hate pairing, and here’s why it’s stupid …” No, thanks. We don’t need the drama.

Diversity is great as long as it doesn’t threaten to destroy the team coherence. A team’s

set of core values should be small and strongly held, and it should be open to practices

that improve the team without jeopardizing its values.

The core Agile Manifesto values promote individuals, interactions, collaboration and

responding to change. Your approach to hiring should be no different.

60

About TestRail

We build popular software testing tools for QA and development teams. Many of the

world’s best teams and thousands of testers and developers use our products to build

rock-solid software every day. We are proud that TestRail – our web-based test man-

agement tool – has become one of the leading tools to help software teams improve

their testing efforts.

Gurock Software was founded in 2004 and we now have offices in Frankfurt (our HQ),

Dublin, Austin & Houston. Our world-wide distributed team focuses on building and

supporting powerful tools with beautiful interfaces to help software teams around the

world ship reliable software.

Gurock part of the Idera, Inc. family of testing tools, which includes Ranorex, Kiuwan,

Travis CI. Idera, Inc. is the parent company of global B2B software productivity brands

whose solutions enable technical users to do more with less, faster. Idera, Inc. brands

span three divisions – Database Tools, Developer Tools, and Test Management Tools –

with products that are evangelized by millions of community members and more than

50,000 customers worldwide, including some of the world’s largest healthcare, financial

services, retail, and technology companies.