I don’t think I was the only one who invented this but I have become personally good at building this as a skill and an approach to testing software and I am also starting to teach people how to do it. I mean the idea of using twitter and other social networking platforms to understand what users want versus what is being delivered to them, what requirements should be changed to, who our real customers are and more.
One of Moolya’s customer, hundreds of million USD in revenues, hired us to consult them in doing brainual testing. So, we as a services company didn’t say, “OMG! Why do we need to tell you our secrets?” but instead we said, “That’s cool. We can help you do that”. There is no secret ingredient in our secret ingredient soup.
I am not supposed to tell who the customer is and who their customers are nor the projects they work on. So, those things remain a secret. We consulted for about 12 different customers they work for.
One project was a portal targeted to be used by North Americans and if I tell you the website, you’d agree that the website receives at least millions of hits per month. Americans use it to buy something. No, it is not Amazon, eBay or something like that. Let’s assume they buy something with it.
The users of that portal have plenty of alternatives. So, if the website doesn’t help the users do what they want it to, users are likely to go to another website and still buy that because they need something When a lot of users move away from a specific website, the company is losing a lot of revenues. This could result in heavy losses.
I went through tons of test case documentation that were being executed for every build and was trying to determine if the test coverage is as good as the claim made to me was. I logged on to twitter and searched for the customer twitter handle and searched for their company-related tweets on twitter.
I was shocked that there were a lot of people complaining about not being able to buy what they want to. Was the company paying attention to it? Sure, they had a large support team to listen to those who were kind enough to call them. Hey, most of them have started tweeting than calling up customer service.
As an example: Let me take some website used by Indians irctc.co.in which is known for its poor quality yet people have no alternative so we are forced to use it. Imagine, if they had an alternative, a good one, then irctc.co.in would lose all the money they are making.
So, let me get on to twitter and find out what users are telling about it:
At this moment, if you think you can gain the skill I am talking about just because you too know to search on twitter, you might be mistaken. Read on.
Look at the following tweets
“@IRCTC Shame on you guys. You should change your site name to ‘serviceunavailable.co.in’ or ‘wefoolscantscale.co.in A friend’s status in FB” Source: read more
“IRCTC … u r a disaster …. atleast u ve become one !”Source: Read more
“There is this permanent scroll on irctc site-”Due to technical issues the website may be sometimes slow.The inconvenience is regretted.” Gr8″ Source: Read more
“In 2 months IRCTC has wasted my 24 hrs.. F’in hate IRCTC…” Source: Read more.
The above tweets determine the emotions of people using the software. Emotion is an important oracle in software testing to determine if there is a problem with the software. I would use the above tweets to help the stakeholders what emotions people are undergoing while using the software. It is important information I am going to provide to the management and ask them,” Is this is how you want to treat your end-users?”
Look at the tweet below:
“So IRCTC breaks my heart yet again. One extra evening in kgp vs. additional 4.5k on flight tickets. Decisions… decisions…” Source: Read more.
I’d use the above tweet as metadata to what I want to go find out more about. So, my metadata becomes IRCTC & Flight to determine how many people want to use the train but are forced to taking a flight. I would investigate and present my findings on how many such cases are reported in a week and how many cases might not even be reported and the potential loss estimate.
The other idea on twitter is about knowing the hashtag FAIL or #Fail, so I associate what I want to search with #fail and see if there are people reporting tweets with those hashtags as well.
So, I found this one interesting:
“Dear irctc,don’t you understand meaning of seat preference?I ask for a window seat and you don’t give when you have around 100 vacant? #fail” Source: Read more.
The above tweet helps me direct my testing to determine if there are problems with the algorithm that allocates seats based on users preferences.
As an interesting extension to that, I would direct my tests to determine if the user was informed that there are no seats of their choice available but they were informed about it in a real estate of the website that most users are not paying attention to.
The other set of ideas are about tweets like the one below:
“Tired of IRCTC. Can we have private railways? Atleast we’ll have better customer care :-/” Source: Read more.
This is someone who appears to have written to customer service or the so-called “care” division. I may want to search for more patterns of what people have thought about its customer service. That helps me find tweets like the ones below indicating how the customer service might be appearing to work in this organization.
“IRCTC website takes customer for a ride to claim refunds: Booking tickets on the Indian Railways ” Source: Read more.
and “Dilli IRCTC cust care guy takes my call n spells Madaan as M for Monkey; A for Awaara; D for Dog and N for New Delhi… #AnnoyinglyHillarious”Source: Read more.
So far, I have exposed many different aspects of the website and what users want and don’t want and have shown how what these users say impacts my testing. So, there is more to it than you see.
Just check how much of power this offers to you as a tester on bug advocacy and how much power it offers to you as a tester to sell the bugs you find to developers. I did try it out. I said, “Hey look, this is not what I say, this is for whom you are writing the code is saying. You don’t care about what they say?”
One other thing I do is to check for various quality criteria problems being reported. Hey, users are not testers so they don’t use our language. No one would tell you, “I am facing a reliability problem here” but instead they’d tell something like this:
“Yes irctc, your homepage is very pretty and informative, now let me login!!!! :@” Source: Read more.
No one would tell you, “I think the session time out value should be increased to 60 more seconds but they’d tell something like this:
“My session has expired :S WHAT IS WRONG WITH IRCTC! Aaaaarhg!” Source: Read more.
“IRCTC site is pretty irritating, session expires after a couple of clicks” Source: Read more.
So, deciphering the quality criteria you want to search for and mapping it to what the user language could be is an interesting exercise.
I spend time figuring out any detailed explanations or conversations between people about the service or product. One such is shown below:
“Looks like Cleartrip.com fetches the data from irctc.co.in ! interesting. So if irctc.co.in is down.. so is cleartrip .. Yup. The only difference is the interface. I use it for searching for trains and then book it using irctc. ” Source:
You may wonder why this is important. These people might be willing to spend time if I want to talk to them. In Feb 2010, I was hired to test a portal here in India that had half of the internet using population of India using it. There’s one thing we did is to figure out which of its users were willing to spend time providing feedback. We tracked those people based on their conversations and time spent on those conversations about the service on twitter. You must know they were pretty excited that the organization called them up and wanted to listen to more of what they had to tell.
Now, some testers and techie folks are also on twitter using such software and they may end up writing bug reports that are a good read. For instance, take a look at this one:
“Gotta firebug #irctc and write a script, seriously. Stupid captcha din’t load in last booking”Source: Read more,
I have just spilt a few beans from the can of worms you can uncover by using twitter driven exploratory approach to software testing. You could help in spreading this message to more testers. Most importantly, you could share your practice of this approach in the comments section or contribute new ideas to us.
I tried spreading this to some Indian testers in some Indian IT services companies. They only had one problem, “Our company blocks all social networking websites and hence we can’t do this testing” to which I replied, “Fellas, I see smartphones with GPRS / 3G enabled in your pockets, I am sure your organizations cant block internet access to that. Sashaboyee!”.
At Moolya, this idea is a part of our testing culture. Tomorrow, if you are getting a proposal from any services company that tells “we do twitter driven exploratory testing”, maybe you’d tell who taught them how to do it. Skadoosh!
0