Writing about Heuristics & Oracles, I will start this post by sharing these lines which I had read. After giving it a thought, I feel it is connected to the testing I do.
“The test doesn’t find the bug. The tester finds the bug, and the test plays a role in finding the bug.”
The above line tells the skills of the testers. One such capability is to be conscious in applying Heuristics and Oracles while testing. This is one of the principles I follow in my projects. I have heard about it many times from my seniors while working. This concept may sound new to any beginner, but these are the skills that are consistently honed by knowledge and practice.
All about the Heuristics
“Heuristics are a fallible way of learning. These are the instinctive approaches we follow to tackle any problem”.
Your test ideas can also be Heuristics.
- Let’s imagine that I have lost a key, and I have got a bunch of keys from the owner to find mine. Few of the heuristics that could be applied here are –
- I will try with some random keys
- I will look for keys of the same size
- I will look for the key number
- To see if the lock and key have got the same manufacturer name
- The key which looks similar to one I have used before
I may fail in all the above five chances or may pass in one or two of the runs.
- Another example is to find the techniques with the shortest path:
About the Oracles
“Oracle is the principle or mechanism used to identify the problem. Oracle helps in making decision about the fault”
- The Sun rises in the east
- Opposite poles attract each other
- A sign that says “Humps for a mile”, as it warns travellers about the road condition.
Are we applying it in our day-to-day life?
Yes. All of us apply heuristics and oracles in our daily life and also in testing. But, that necessarily doesn’t mean we are aware of it. Be it a mom at home, colleagues in the office, students in school, everybody uses these ideas in daily life. But the effectiveness depends on the people and the context.
Heuristics and Oracles in Software Testing:
When it comes to software testing, Oracles could also be the specifications and requirements of the software we test. So if there is a difference between oracles and the actual result, it is a bug because there is a deviation in the requirement or the expectations. Heuristics are the different approaches we apply to learn and test the software and its associated systems.
Suppose while testing software, I find an inconsistent crash. Now the Oracle here is — the software should crash in this flow or this usage pattern. As a result of the crash, I have to restart the flow which is not part of the product’s claim.
I am aware now there is a crash in the build and with a pattern and internal/external state of the system. I explore and find this pattern to experience and observe this crash again. Here the heuristics are the different paths I chose to learn and experience the crash again by knowing the root cause. However, the heuristic is fallible!
Do Oracles fail?
Yes, if the heuristic can fail, then oracle can fail too. But what do I mean by ‘fail’ here? I mean that it will not assist me to learn or/and identify the problem precisely in a given context. It does not mean, it will always fail, but it is fallible because we humans are fallible too.
So, the heuristic could be an oracle and an oracle could be a heuristic.
Here are the few instances where my oracles might fail:
- I expect rain when the sky is full of dark clouds. But, after some time it may disappear. If I believe it will rain, my oracle fails here. This is a case where an Oracle can also be a Heuristic.
- There is a board that shows “Roadwork Ahead”. Imagine if this work is completed, but the board is not removed for a long time.
I would follow the speed recommended along with instruction on board saying work in progress. Wait, “ROAD WORK AHEAD” and 80 kmph!? Shows me I was wrong in this context.
Which Oracle is right here? 80 km or Road Work Ahead? Or both?
The board with the message “ROAD WORK AHEAD” is an Oracle asking people to slow down or hinting to be in control while driving. With this, 80 kmph signage is also an Oracle which says, I’m allowed to travel in 80 kmph speed on the road where work is in progress. Now, these two Oracles are contradicting each other and one of these can fail or both can fail if the road is fine and the sign was not removed.
The heuristic and oracle are also used in the simple code we write for automation to assist our testing.
I would like to highlight that I too had the idea that only veteran testers were good in heuristics and oracles. Then my senior showed me that everybody has her/his unique way of testing. There are no universal best practices, but there are practices that work for the context. So I feel that with each passing day, I’ll learn much more about these concepts and will come up with a better understanding of the same, by unlearning.
You may also read this conversation between Michael Bolton with his tester friend Tony about Heuristics and Oracles.
4