Thursday, December 8, 2016

Would You Survive These Test Manager Interview Questions?

You probably want a change of scenery. It’s probably a new city. You’re looking to meet new people, make new friends. The possibilities are endless.
The role itself is familiar to you. You’ve already put in enough time building experience and expertise in Testing. You’re going through the motions of preparing for the interview for that exciting new role of a Test Manager.
Then again, you could be looking to move internally within your company – in a multinational environment, it is common for people to apply for roles within the organisation and move to a new, better, or similar role with a different team – within the same city, to a different city, to a different country, to a different continent. If your company provides this flexibility, then good for you.
No matter your circumstances, when you’re preparing for an interview for a Test Manager role, you want to be as well prepared as is humanly possible. You surf online forums, brush up on fundamentals, you talk to friends about the role, about the department, about the company.
Yes – let’s agree that despite your best efforts, things could go wrong on the day. Your car may breakdown. There may be a delay on the subway. Your clothes may get soiled by a passing car carelessly splashing stagnant rain water. You may feel unwell. You may clam up in front of the interviewer. The interviewers may have had a bad day – and (unprofessional though it is) take it out on you.
Anything could go wrong. Unforeseen circumstances could derail things completely. Yet, you can’t let what can be paralyse you. You want to do the best you can, prepare for all eventualities – and hope that the worst doesn’t happen.
In the spirit of preparing for the unforeseen, I’ve pulled together some simple yet (what can be in the moment) challenging Test Manager Interview questions. Today, we’ll look at these patently challenging Test Manager interview questions, and how you can tackle them should your interviewers decide to throw them at you.
This is not an exhaustive list of all the interview questions you can expect to be asked when interviewing for a Test Manager role. The internet provides ample resources for the basic questions.
In this blog, we’ll look at some of the trickier ones, and how you can handle them with aplomb, and come out shining.
Why are they tricky? Not because they are difficult to answer, but because it can at times seem like whatever you say doesn’t satisfy the question or the people that fire the question at you. So, we’ll look at how you can avoid the uncomfortable silence either side of the table that can sometimes follow the question, or the response.

If you join our team/organisation as a Test Manager, what are the first three things that you would do?

This is a trap. Why?
You probably don’t have enough understanding or appreciation of the team or organisation you’re looking to join, yet. In which case, you don’t clearly know what challenges the team are facing currently. So you cannot provide an informed response that suits the particular situation of the team or company.
While they don’t really expect you to provide world changing ideas in response, they are open to the possibility.
So what can you do? Not provide a response? Well, in that case, the interviewer might question the value you bring to the table. After all, how can they be sure that you’re the right candidate if you don’t provide a considered response?
Does ‘between a rock and a hard place’ sound familiar?
This is where conducting prior research on the company or the team can help – immensely. Invariably, you’re being asked this question because the interviewers are trying to measure how much effort you have put in to understand what they do, and how you can help improve things if you join.
While they don’t really expect you to provide world changing ideas in response, they are open to the possibility. Some of the best ideas I’ve picked up are from brilliant candidates that I interviewed for a role in my team.
So this question isn’t a blank bullet. It’s quite possible for you to create a great impression with your response, if you’re sufficiently prepared.
Go with a few commonly applicable ideas. Like ‘I’d encourage everyone in my team to take up Agile Testing certifications’.
Before you answer this question, however – remember: you’re probably already half way through the interview. This Test Manager interview question usually surfaces during the second half of an interview.
So you should try and understand as much as you can from the interviewer about their team, department, organisation.
In my experience, the best interviewers spend most of the interview silent – and make the candidates talk. And the best candidates get their interviewers to talk and divulge more about the job role, team, company. If you made your interviewers feel comfortable enough to talk about the role, team, organisation, you may already know some of the challenges they are facing day-to-day. And you will be able to respond knowledgeably to this question.
Note:
On some occasions, you might understand the role, the team, their challenges beforehand. How? This is applicable for situations where you have closely worked with the team you’re applying to join. Therefore, you’re able to demonstrate sufficient understanding of their common challenges. And provide a suitable response to each.
Don’t stop at three – rattle off as many as you believe are apt for the question. Then again, remember, it’s not about how many – it is about how helpful your ideas are going to be.

What Testing tools do you use, and Why?

This is perhaps the most testing of all Test Manager interview questions. And I’ve covered this previously. Tooling as an enabler is important in making a tester’s life easy and simple. The idea behind tools is that they take up the repetitive, redundant and repeatable activities; that they improve the flow of information across the organisation.
Tools essentially help free up some of your time, so you can spend that time thinking. That is, so you can focus on improving your Testing processes and practices, so you can come up with better strategies for managing the Testing function in your organisation.
In that sense, yes, Tooling is important. Because with Tooling, you’re free to focus on the top of the tree and leave the low hanging fruit to the tools.
But (of course there’s a but!), the challenge is to not get too hung up with Tools.
Remember, people have preferences.
I love Honda cars – why? My first car was a Honda, and I remember friends complementing me on my choice and the silent yet powerful performance of my car, which by the way had a youthful look and feel, yet was sensible (and light on my purse) as a family car.
Every single car I subsequently purchased, I tried to get that particular combination of feelings from. It didn’t work out that way. Even when I bought a Range Rover, even when I tested out a Tesla, the experience, the satisfaction I had with my first Honda was missing. Every time we see a new version of my Honda on the road, my wife and I reminisce about that car that we used to own. And how no other vehicle – not even a Rolls – can ever give us the same satisfaction and value.
The takeaway here is, that where people are concerned, everything can be tainted with emotions. Emotions are involved even with matters like Tools that enable Testing in your organisation.
“Emotions are involved even with matters like Tools that enable Testing in your organisation.”
Maybe you use a particular Software Testing tool (despite it not being the top three in the industry), because your Global Head of Testing loves the product simply since they have had it since it’s first release, and their career success coincided with similar success for the tool. Even if the tool doesn’t make business sense today, at least in comparison with better alternatives readily available in the market, they may have personal ties with a product that they may not want to severe that easily.
Or, maybe that character is you. Let me explain:
You like A better than B because, well, you know all the intricacies there are to know about A, and will be able to use A more effectively than others can. You may not like B because it’s A’s competitor, and is rated better than A (yeah I know – it sounds counterintuitive).
Leave personal preferences aside when discussing key Testing enablers like Tooling. Tools are important, yes, but more important is whether the Tester that uses the tool can really make it work for them.

How do you decide the channels, devices, OS we should test for?

When it comes to customer facing systems, device strategy is paramount. Especially if you are in the Digital line of business (yes, Digital is now a Line of Business – it is being recognised as it’s own Business Function), a sound device strategy will help you manage product support effectively.
Do your apps still support Android Ginger Bread? Really, you do? What about iOS 6? You should continue to provide support for Samsung Galaxy SII right?

There’s more to device strategy than a techie or tester arbitrarily deciding to cull older devices, operating systems from support. This is a business decision.
What percentage of your customers still use Android 2.3.3? Or Android 3.1? Are you going to try and support such customers? Or are you going to politely tell them to upgrade to a better phone, tablet and OS?
Do your regulators have a say in your device strategy?
There are a lot of firms that still provide employees laptops with Windows 7 and Office 2010 or 2013. This is simply because their staff continue to use specific software that won’t yet work with Windows 10. Or, their IT security haven’t yet figured out how to fix all the vulnerabilities that a new OS version introduces to their internal systems and servers.
Device Strategy is a complicated beast, and is a decision not made lightly or by one individual. It’s a collective decision that has far reaching implications for your customers, colleagues and regulators. And your response to this question should reflect that thinking.

What do you know/think about Automation?

Automation has been a hot topic for well over a decade now. Ever since the benefits of Test Automation were demonstrated visibly, there has been a clamour for more and more Automation in Software Development.
While the wider Automation efforts include other disciplines like Coding and Integration, this Test Manager Interview question tries to test your understanding of, and experience in, Test Automation.
Don’t get me wrong – not being experienced in Test Automation isn’t unacceptable. It’s quite understandable that so far in your career, your employers may not have provided you a platform to automate Testing. If that is true, you should say as much.
What the question tries to also bring out, is whether you understand about Test automation, and whether you’re open and willing to give Automation a chance to make people’s lives better. And, more importantly, whether you have a balanced approach to Automation.
Why is this question important?
You’d be surprised at how many candidates have either gone the full monty with their support for Automation, so much so at times it sounded like they prefer working with machines more than humans. While others have shown unfathomable opposition to the idea, without providing clear justification for their hatred of Test Automation.
At the end of the day, Automation is here to stay, and when done right, can make people’s lives better. Test Automation can improve your team’s productivity manifold by taking away repetitive and redundant activities, allowing your team to focus on the higher value activities.

What do you know about our organisation, and our software testing practices?

You’re probably thinking, “Hmm, is that a question extraordinaire? Isn’t that a staple for all interviews?”
Well, sure it is. This Software Test Manager Interview question is quite fundamental, and common. Yet, time and time again, I’ve sat in an interview talking to a candidate who has no clue about what my company does, or how we conduct testing.
See, matters like internal Testing practices are naturally going to be hard for a candidate to uncover or understand. Unless of course they have friends within the organisation that can provide this type of information.
Demonstrating that you understand how Software Testing works with your prospective employer is a good way of emphasising your networking and research skills.
Your ability to know and understand a team or process that you are not necessarily involved with shows your skills in gathering information that is hard to come by. That is a skill in itself.
Yet, not many candidates put in the effort necessary to understand enough about a target employer. Knowing my organisation’s history, or key leaders’ names, or our client database or our press releases doesn’t equal knowing how my Software Testing operations work in daily life.

What is the difference between Agile and Scrum?

Yup – fundamental question. Yup – you’d think it’s simple enough to answer.
Really, if you’ve got any experience of Agile, you should know the difference. If you possess an Agile certification, well, this one’s a no brainer.
Yet, I’ve noticed how many candidates totally clam up when it comes to this one question. Hence it has a special place in my list of go to Test Manager Interview questions.
Even if you do understand the difference, how much do you know? How experienced are you in Agile and Scrum? Have you worked on ‘real’ Agile projects, or the typical ‘Waterfall in Agile clothing’ ones?
This question is usually a precursor to more prodding discussions about Agile and Scrum, and the other Agile development methodologies out there. A good interviewer doesn’t stop with asking you about the differences. They will try to test how much depth and breadth you’ve accumulated in knowledge and experience of Agile methodologies.
The interviewer is really interested in what value you can bring with your Agile experience. The demand for Agile coaches is at an all time high currently, and no wonder. As more and more companies adopt what is inarguably the most successful software development methodology out there, there is a need to help organisations go through an Agile transformation. Software Testing is no exception to this rule.
Try to provide an answer that leads to further discussion about Agile methodologies in general, and Scrum specifically – if you want to demonstrate experience in these topics.

Bringing it all together

No amount of preparation can beat knowledge gained through hard experience. While I have attempted to provide you a quick reference guide of the trickier questions out there and how you can navigate them, I cannot help you come up with a fluent, congruent response that works in the moment. That comes out of experience gained through diligent hard work, and preparing thoroughly for your interview.
If you do not possess the experience, try to gain some first. Or, demonstrate why and how you can succeed if given an opportunity to gain experience as a Test Manager. A half-decent interviewer will see through scripted responses and lack of experience in no time.
Do you have other tricky Test Manager Interview Questions to share? Let’s discuss in the comments section below.

Wednesday, December 7, 2016

Pointers for interviewing testers

Or you could use someone else’s list: Huib Schoots has done a great job of identifying a set of criteria in his article, Heuristics for recognising professional testers. And Ilari Henrik has some superb ideas about what makes a World Class Tester that may also be useful in your search. Johanna Rothman has some fantastic resources in her Tips to Streamline Your Hiring. And Cem Kaner has a detailed guide to Recruiting Software Testers that contains a number of lists that can help you tighten-up your requirements.

https://blog.gurock.com/interview-recruit-testers/

Wednesday, October 12, 2016

What to ask an agile tester / test manager

http://reqtest.com/testing-blog/agile-methodology-interview-questions/?utm_medium=email&utm_source=drip&utm_content=bp_263_agile-methodology-interview-questions_2016.06.30&utm_campaign=tm&utm_source=ReQtest+newsletter&utm_campaign=989dd52006-TM_Workflow_v1&utm_medium=email&utm_term=0_17bd351788-989dd52006-320212129



When recruiting for Agile Testing talent, I’m sure you agree that testing their Agile and Scrum knowledge is essential. Asking solid and probing interview questions about Agile and Scrum methodology will help you zero in on the Agile Testing talent that you need.
But you’re probably wondering: which interview questions on Agile and Scrum will help you evaluate your candidate better?
I’ve pulled together the best Agile Methodology interview questions and answers for experienced Testers, so you can conduct efficient interviews and grab the best talent out there in the market.
In this blog post, we’ll look at simple questioning techniques to test your prospective candidate’s knowledge and proficiency in Agile and Agile Testing hacks.
Want to make your quest to find your next testing superstar easier? Read on…

1. What is Agile, or What is Agile Testing?

Yes, this is a fundamental question. And yes, the answers are also fundamental. Yet, there is more to asking this question than there is to any other.
Why do you need to ask this question?
Especially when you’re recruiting for the more senior positions in your team, asking this Agile interview question can unseat the candidates a bit.
With experience, the average Agile Test Lead/Test Manager stops expecting such questions during interviews. They prepare for the more complex Agile and Agile Testing questions – like exploratory tests, test-driven development, etc.
Probing your prospective candidates with this fundamental Agile interview question will push the best among them to pause and think about what they want to say. More than getting an expected answer, this question is a good way to push your prospective Agile Testing superstar out of their comfort zone.
Being anopen-ended interview question, you allow the candidate an opportunity to express their Agile experience and thinking freely. In this way, this Agile interview question also helps you measure the prospective talent on their ability to articulate Agile thinking clearly.

2. Do testers sit within or outside the Scrum team?

Again a fundamental interview question. This time, however, there is more to it than meets the eye.
Because, the answer is: It depends.
It depends on how agile a team really is.
The easiest answer is that the testing team sits within the Scrum team, and deliver to sprints. This is true for well-run Agile test initiatives. Having the testing team embedded within the Scrum team helps drive code development to primarily pass test cases.
By nature of how Agile and Scrum work, a Scrum team is expected to possess all the skills necessary to deliver an Agile project – including Testing. And, we all know that anyone in the team can deliver any skill – when the team is truly Agile.
Obviously, there are pros and cons to this approach of a multi-disciplined, multi-skilled team – as opposed to one in which individual are identified and earmarked for particular skills like Business Analysis, Development, Testing etc. We won’t get into that today.
When embedded within a Scrum team (and even otherwise), the hallmark of a good Agile tester is that they help reduce the defects count and percentage of defects found per line of code written.
How do they do this? By working with the rest of the team to deliver to requirements. And what better way to do this than by working day-to-day as part of the Scrum team.
On the other hand, the more experienced agile tester knows that at times, traditional test cycles need to be run outside the Scrum.

When Traditional test cycles are at times necessary

For instance, if you work for a large multinational company that is overhauling its mobile app for all of its markets, they probably have a product team that is delivering some ‘core’ features for the app, to then be customised for individual markets before deployment. The ‘core’ team’s focus will be on fixing major bugs that block progress of product development; not make the code 100% bug-free.
Depending on which release of the product is being deployed to each market, it is quite possible that a release needs to be thoroughly tested before deployment to ensure the code isalmost bug-free. This is when you will need traditional System Integration Testing (SIT) and User Acceptance Testing (UAT) cycles to ‘clean’ the code to make it deployment-ready.
Unless the candidate has worked primarily in self-contained projects where there is no dependency on teams external to the Scrum team, you should expect to hear about dedicated traditional testing cycles outside of Scrum. Knowledge of this seemingly non-Agile way of working demonstrates valuable experience in working on distributed Agile projects and working Agile alongside Waterfall teams.
“Ability to make progress despite challenging and non-ideal work environments is a hallmark of a good candidate — and a pre-requisite for your next testing superstar.”
This agile methodology interview question and the answer you receive in response, is important because it helps you measure the length and breadth of experience a candidate brings, and the level of exposure they have to working in non-ideal Agile working environments. The ability to make progress despite challenging and non-ideal work environments is a hallmark of a good candidate – and a pre-requisite for your next testing superstar.

3. Burndown chart is a popular Agile technique. What is a better alternative for reporting Agile Testing, and why? Explain with real-life project example.

This a tricky question to handle.
You’re acknowledging the importance of the Burndown chart in the Agile pecking order to the candidate. You’re treating the candidate as an equal in this knowledge – that they know and understand the Burndown chart and its uses. You’re also not giving them an opportunity to demonstrate that knowledge because, well, it is assumed that they possess it.
The kicker? You’re asking your potential Agile Testing superstar to go further and elaborate on what can better the Burndown chart and why. You’re making it clear that basic responses about how the Burndown chart do not interest you. This is in stark contrast to the fundamental questions you have been asking so far. It’s a bit like switching from the first to the fourth gear without shifting through second and third.
You expect a knowledgeable and considered response to this question.
The answer could, of course, be anything. Some of them could have had exposure to the Burnup Chart, and its many uses and advantages over the Burndown. They could tell you about how they used Defect Trends to augment Testing Burndown reports, to present a more holistic view.
By challenging them to go beyond the routine answer (and by taking the routine answer off the table), you will push boundaries with your candidate. And you will discover how hands on they have been with leading a Test function. For prospective Test Leads and Managers, this Agile interview question and its answer is a great way to measure maturity and depth in understanding. This interview question in particular will help you gauge the level of responsibility your candidate can take on from day one.
“You should expect candidates to have prepared well to discuss project examples in general. But rarely have I seen someone that had a well-prepared answer to this one.”
The clincher in the answer to this Agile and Scrum methodology interview question is the real-life project example.
You see, you should expect candidates to have prepared well to discuss project examples in general. But rarely have I seen someone that had a well-prepared answer to this one.
By trying to provide a real-life project example, your candidate is forced to think on their feet and be spontaneous.
This act of stepping outside their comfort zone will cause them to display their original personality traits. Any concerning individual traits will show up. It gets better: their response to this one will be original and not sound premeditated, so you can get to see and hear how they talk, sound and behave in reality rather than what they have shown you of themselves so far.
This way, you get the best of both worlds: an assessment of their experience level, and a peek into the real them.

4. Explain how you manage Requirements Traceability on an Agile project?

Requirements traceability applies universally. For a tester, even more so.
Why, you ask? Let me tell you why.
“A good Test Lead or Test Manager optimises their team’s efforts towards designing and executing just the right amount of test cases necessary to cover the requirements scope for their project.”
What this Agile and Scrum methodology interview question seeks is to understand, quite simply, how committed your prospective testing superstar is to Requirements Traceability. I normally expect the candidate to explain in straightforward terms, the tools and processes they use to minimise the number of test cases their team need to execute, and yet maximise requirements coverage.
The fundamental necessity for testing is simple: to check that the requirements of the Product Owner have been met. If your candidate can demonstrate ample knowledge and experience with gearing their testing towards achieving this basic purpose, don’t let them get away!
Test Case Management is an art in itself. I have covered Case Management in detail previously. See if they bring up Case Management in the same breath as Requirements Traceability. If they do, they have worked on or managed large and complex testing assignments, and understand how good Case Management goes hand in hand with ensuring Requirements Traceability.

5. What is the best way to reduce defects count in an Agile project?

Look: we need good Agile Testing talent. No doubt about it. One of the easiest ways to evaluate talent and potential is to assess how lazy the candidate is. The lazier the better.
Yeah you heard me right – you have to go for the laziest of the lot. This is crazy – I know. But hear me out.
“One of the easiest ways to evaluate talent and potential is to assess how lazy the candidate is. The lazier the better.”
Talent isn’t about taking the wheel and making a better, more efficient wheel. Yes, that is improvement. But just making improvements alone doesn’t deserve the tag ‘talent’. There is more to talent than that.

Talent is about taking the wheel, and making the cart. Then, it is about moving on to a carriage. Going further, it is about imagining and building the automobile. That is talent.

And all that came about because – I postulate – talented lazy people thought, “Hmm… now that I have the wheel, I wonder how I could use it to make my life easier?” Someone else that was equally talented and lazy put an animal in front of the cart so they could luxuriate in the laziness of it all. And so on until we hit the automobile.
Talent is about taking what you do and completely transforming it, so in the best case, you don’t actually have to do it anymore but still derive the expected outcome.
When you ask this particular Agile and Scrum interview question, you’re testing your potential candidates to see if they’re mere doers or innovators.
The innovators will tell you they’d like to adopt Test-Driven Development (TDD) to help reduce defects count. The innovators will think about minimising the actual number of test cases they need to execute. They’ll further look at ways to reduce the amount of testing necessary to unearth all the bugs in your system.
Innovators will first and foremost look to make their jobs easier and simpler to accomplish.
This attitude of doing the least necessary to achieve the desired results is what will produce transformative ideas. Such ideas will free up your testers to focus on enabling the rest of the team to build a product to the Business Analyst’s specifications. Testers will have more time to build a Case Management repository and maintain it diligently. Testers will execute lesser test cases. Defects will be found sooner during development. Your testers will be free to do exploratory tests to make the system better.

Now It’s Your Turn

As much as it is crucial for a candidate to prepare diligently for an interview, it is equally important for the recruiter to be prepared as well.
You need to be prepared to ask the right Agile methodology interview questions that will help you judge a candidate better. Tune your Agile and Scrum interview questions to elicit responses that give you an insight into your potential testing superstar’s experience, capability and psyche.
The questions I shared in this article will help improve your chances of making the right decision when recruiting for Agile Testing roles.
If you found these Agile methodology interview questions useful, why not share it with friends using one of the options below?
What other questions have worked for you when recruiting Agile or Agile Testing talent? Share your views in the comments section.