Paul Holland in this last Q&A of the series ‘Let’s Talk Testing with Paul Holland’ shares his views on automation, the journey as a change agent in transforming traditional testing to Context-Driven Testing, the challenges encountered and the wins. And read along to know about the conferring experience for testers.
Q: Your views on test automation and specifically who is qualified to automate?
A: A role which doesn’t define you either as a developer or as a tester shouldn’t exist in my opinion. If there is a good tester without a computer science degree or background and who has taught themselves coding in a programming language and you put them in automation as an automation developer/tester or whatever title they have then you are likely to have lost a good tester and gained a bad coder. To be a really good coder, you probably need to have gone to a university and studied coding/development. But most people will not be good coders without good experience. I personally like to see developers be writing their own automation and the unit tests(at integration and API level) as part of their development process. The use of Test Driven Development (TDD) is beneficial even though it should not be called TDD, it should be called unit check first development. It is a development methodology. Having a developer create solid unit test framework around all the code that they develop and not rely on somebody else to write them will make their code better. The benefit that you get out of it is the way it failed again cost, maintenance cost, the debugging cost, the effort of writing it and how long it takes to execute it. I am yet to see a very successful end to end GUI automation suite.
But GUI level automation is almost definitely going to take more effort for the benefit it gives you plus automation is only going to see only what it is coded to see.
So who should do automation? Developers should do the unit test and integration test automation and usually, nobody should be doing end to end automation. Or if they are it could be a tester. But I much prefer to see testers using tools that will help them. Automated assisted testing where I use a tool that will generate 100’s of users or will configure the system in a way faster than I do. Using tools like that is beneficial to testers.
Behavior Driven Development(BDD) and Testing Driven Development(TDD) are both development methodology. BDD – the context of given, when, then is to have a conversation about the scenario. And is not supposed to develop automation. The misunderstanding here is the developer who thinks BDD is for the testers to do. This is the education part and the way it is introduced to the teams.
The transformation from traditional to Context-Driven Testing
Q: How to overcome the challenges when introducing CDT in a traditional environment?
A: That’s a good question and that’s what we are working on. If someone at the top is against it, the best success I have had was a neutral comment from above – ok we can try that. That’s worked twice. The senior most management has said – yes we support it and that’s been successful at Medidata. If someone at the senior level is speaking against it, then it’s better to find some other place. While you are looking to switch, do some exploratory testing – stay an hour late and give them some proof about the type of testing you are doing. Here’s the mindmap I used, the charter I created and the bugs I found – if that doesn’t convince them then get out (the stop criteria). And if someone says they do not want to implement these new methods then say – electricity was new once, or the Internet was new once. But it’s not even new 😉 How long did it take for the Internet/electricity to get distributed to everybody. ET started to be talked about commonly from 2000’s when James started doing RST. But then new things do take time to ramp up.
Q: What to risk/not risk when introducing CDT in a highly regulated pharmaceutical industry?
A: Griffin Jones doesn’t do any scripted testing in the highly regulated pharma industry, he does ET and the auditors are looking for indisputable proof if the testers can provide that then that should be sufficient. Video of what you did, and step by step evidence. Every industry will tolerate this proof expect the telecom industry which requires a 95% pass rate.
Q: What if ‘None Of The Above’ methods tried as a change agent work?
A: I would say if you go a few months and don’t see any progress, as long as you are taking steps and started introducing charters it took almost two and a half years.
If you are at a company and are trying and trying for two or three months and not made any progress then it’s about time to leave.
Q: What are you sick of hearing when it comes to the ‘firm’ unwilling to bring any change without even trying?
A: You mean – I like that idea but they keep doing what they are doing. It’s people saying that they are gonna change but not knowing what to do.
I don’t think it’s not that they are unwilling to change but them not knowing what steps to take to change. They are unsure what to do and don’t what to ask because it makes them uncomfortable to ask and learn.
Q: How having a strong support system (in the organization) can help when bringing about a/any change? Who is your support system?
A: Internally at Medidata, we have a good team there. Right from the CTO who supports the change, Su Finley who implemented the change. 3 or 4 directors who are well known in the community, so we have a good senior level support. We make sure that the managers are more onboard and involve them, we introduced them to more training and some more details. I think we need to keep adding. The biggest supporters are the director level, my level and above right up to the CTO.
Overall, the whole of the context-driven community is supportive.
Q: What if there was no support from the above/management?
A: But if I don’t have the support I focus on what I have influence over. At this level, I would implement it just in my team. If I don’t have support from above, it’s ok. As long as it is not the opposite of support. If I am a manager, then it’s me and my team. If I am a director, me, my managers and the team. If I am an individual contributor, how I do my work. So look downwards, if you have support from above it’s good.
Conferring experience for the Testers
Q: What do you get out of networking and conferring?
A: Friendship, support. That’s it.
Q: Can you elaborate on this?
A: Besides this, nothing really.
Encouragement, it’s really a big extended family. Once you get into that community of probably a few hundred CD Testers who go to the conferences, and by the conferences I mean ATD, Let’s Test, Test Bash, CAST, COCO – the core group of people when you see them over and over again that’s really like a family reunion, there is support, that level of camaraderie, of sharing experiences of seeing how the other people are doing things is really encouraging and it’s a lot of fun.
Q: Can you say that this keeps you going?
A: I don’t know if this keeps me going, but I need to take off a year or two without going to a conference to see what it’s like. It encourages to keep me going and involved, but I think I’m at a level where there’s no need to be in person. There is the online community that is there.
I think if I was cut off from both the online and offline community, then that might be discouraging to keep going.
Q: How does gaming help testers? What kind of games do you recommend?
A: Gaming doesn’t help, Martin Hynie did a study and a talk much to my displeasure. Not that he did the talk, but the findings that he had an MRI done while he was at rest, and when he played the games. Comparing the mind working of a tester who hasn’t play games and a tester who did play games – it didn’t show anything of interest when they did the MRI.
The testing games help you approach the world differently and more like a testing mindset.
The dice games is a really good opportunity for the testers to alternate between focusing and defocusing, randomizing their outputs, simplifying their inputs, looking for patterns, analyses, the same skills you use in the dice games can be applied to software debugging.
Recommended games – The pen game, the mind reader game – they all involve putting limits on how you figure out how to learn and analyze better.
Paul Holland an exemplary rapid software testing coach.
Thank you Paul Holland for sharing some of the gems of testing wisdom with us all in this elaborate interview with Moolya Software Testing Services.
