Let’s Talk Testing with Paul Holland and learn the difference between Context Driven Testing and Exploratory Testing, a tester’s work routine if there is one and if software testing is changing for better at all.
Paul also gives a glimpse of how he feels about the work that he does as a change agent and the misbeliefs as testers that we need to shatter.
Q: What is the difference between Context Driven Testing and Exploratory Testing?
A: Context-driven testing(CDT) is a school/approach/mindset in which the type of testing you are going to do depends on the situation that you are in.
The biggest contributor to any context that I have been involved in has always been the PEOPLE. More so than the domain/environment/project that you are working in.
The biggest context changer is the people that you are dealing with. Because they are the ones who are looking for particular things in your report, how you interact with them before you start your testing, the types of questions that need to be asked typically are dependent upon the people. CDT is the approach to testing, how you deal with the team, how you plan your testing, how you change your testing based on the ongoing changes.
How you test it when you are actually interacting with the product that’s the Exploratory Testing(ET) part or it might be scripted testing, there are some contexts where scripted testing makes sense. In the Medidata context, we do scripted validation when we need to show to the Food and Drug Administration(FDA) in the US, or to the Medicines and Healthcare products Regulatory Agency(MHRA) in the UK that our software can do what the story says it does. In that context, scripted testing makes sense. But when we are actually testing the feature before the validation part we wouldn’t have a script and we would prefer to do exploratory testing because that is the right type of testing in the context of looking for bugs. While the scripted test is the right type of testing in the context of validation. That’s probably the biggest difference between CDT and ET. So ET is a piece of CDT in most situations.
Q: What do you have to say to the new aspiring testers?
A: The one thing that makes good testers stand apart to me is inquisitiveness or creativity. So allow yourself to be curious, and allow yourself to indulge that curiosity. And the more creative you can be with your curiosity, the more effective you will be at testing.
There are three parts to this answer. You need to be curious, you need to be creative with your curiosity, and allow yourself to indulge that curiosity even if somebody says why are you doing that you can say because probably there are good bugs there. Having to justify your tests before you do them or while you are doing them is probably the best way to not find as many bugs. If you are curious about a proof of that ask any tester if they have ever found a bug by mistake. And the answer to that will be yes.
Q: What needs to be a tester’s daily routine if there is one?
A: There is none.
Show up to work, ask questions, repeat.
Q: How do you feel about the work that you do during the working hours?
A: I really like what I do. I spend most of my time trying to figure out ways to make testing better at Medidata Solutions and in the testing community in general and then trying to implement those ideas. So far the most effective way I have found is just sitting down beside people and having them show me stuff and give them feedback. I tend to walk around the New York office a lot and I just say hey how’s it going and then people talk to me about what they want to talk to me about and then I say do you have an example of that, can I see your mind map and then I just give them feedback. So I was coaching this one time spontaneously and the response that they gave me was this was helpful and will really make me do my job better. That’s the part that I like about my job, to be able to go and just help people see what they are doing that could be done better or more in line with what the Medidata approach to testing philosophies are.
Paul, you are doing it because you like to do it and there is nobody opposing you doing it.
Paul: There used to be people opposing it, but not here at Medidata Solutions. From the time I started introducing RST in another firm that I worked with everybody around us was like why are you doing this and our director of testing, when he became our boss told I know nothing about testing, you do what you need to do and ask me to help if you do need anything.
Rhetorically: And that’s the kind of leaders we want around creative minds who allow people to work creatively and not oppose ideas just for the sake of opposing.
Q: What misbeliefs about testing do the new age testers need to shatter? And why?
A: Scripted testing does anything that’s helpful as far as finding bugs.
The way to look at that is if I write a script as I execute that script the first time I’m learning something about the product, the product is going to behave in a certain way based on those inputs and I’m going to learn that the product is either behaving properly or poorly and that’s it. Every other time from then on that I execute the same script if the product is behaving, in the same way, I’m learning nothing and the vast majority of the time the product is going to be acting in the same way. The only time it isn’t is when somebody has checked in some code that caused a change in behavior but that type of change should be caught by unit test not by me wasting my time doing the exact same thing. So scripted testing is beneficial especially if it’s automated to sort of be a safety net but it won’t find bugs. We need people to be creative and to go and explore the software to find bugs.
Paul, is that why the popular belief in the testing world ‘anybody or everybody can test’ exists?
Paul: Anybody can test those happy paths if you are given the script I can push the buttons.
Q: How is testing changing for the better?
A: I don’t think it is yet, but I think it will. The CDT community is roughly about two hundred people or maybe four to five hundred core people who are really trying to do good testing. The community is growing but we are just still a tiny voice. I know a lot of people are talking more about Exploratory Testing and are allowing their testers to do it. But most of them are doing monkey/bash testing, completely unstructured and try to break the system with some forcible tests but there is no real structured Exploratory Testing happening. I think it’s probably going to take multiple big companies some catastrophic, stock changing events for them to say how did we miss this, our automation suite is perfect.
A large American Insurance company implemented RST very effectively and saw an improvement in their quality. Companies like Microsoft have gone back to have their software testers. So there is a movement but still, a vast majority are looking for automation testers.
0