A Case of User Field Testing

“An Italian design company best known for luxury clothing aimed at young adults is internationally acclaimed for its jeanswear. Headquartered in Braganze, Italy, the brand has the presence in over 80 countries, with over 400 company-owned stores worldwide.” – website

Moolya partnered with this international brand to test its first Customer Loyalty App which they are very soon going to launch for the Indian markets. The app allows buyers to accumulate loyalty points on every purchase. The customer also gets upgraded to different levels depending on the amount of purchase made. The app features a catalog of products from the same brand which can be exclusively purchased using the loyalty points.

Our testers initially began testing the app and its functionality from the backend. Each feature was well scrutinized and all possibilities were checked from the technical aspect. So, what’s new about testing this app you may ask?

WhatsApp Image 2017-02-04 at 16.33.08
Testers – Bhavana and Sandhya

Moolya not only just claims to provide an end-user perspective in their testing approaches but proves the same with their out of the box testing techniques. Since the app in question is going to be used by people purchasing items from the brand, a team of testers was placed out in the field to do some real-time testing. Yes! our testers from Moolya posed as customers and executed user field testing to check each functionality from an end-user perspective. Test cases were created keeping in mind a real purchase environment and the results were tracked based upon the outcome. For example, if a purchase worth Rs. 50000 was made the customer had to be upgraded to the next level where instead of earning 15 points (purchase below Rs. 50000), he would be credited 30 points.

Testing Phases

Test Case 1: Purchase

When the buyer made a purchase for an amount, the right number of points had to be credited to their account. The calculations for crediting points against purchase had to be checked under various circumstances.

Test Case 2: Redemption

After accumulating a certain amount of points the customer can utilize those points to purchase items from the catalogue. Each item on the catalogue is worth a certain number of points. the redemption process had to be tested so that when a customer redeems his points he can make the right purchase.

Challenges faced during real-time testing

  • Generating real-time bills without any actual monetary transactions.
  • The testers had to face some backend locks as the billing department was hesitant to generate bills without real purchase
  • Very high amount of purchases
  • The calculation of points against purchase was complex. Multiple test cases had to be tested to check whether the points were credited appropriately to the customer’s account
  • We were also successful in finding an important tweak with the requirement that we later raised as an improvement/recommendation.
  • The store did not have any return policy. The testers were initially not aware of this scenario so test cases for any return policy that were created were not applicable
  • Membership up-gradation calculations posed as a complex challenge. After spending an amount the buyer had to be upgraded to the next level. In case of return/exchange, the points were deducted in a different manner which in some scenarios brought the score below zero. Test cases had to be created keeping in mind various scenarios for items worth different amounts.

All the thinking, All the effort, All the enthusiasm finally paid off when the team found issues that could only be found in the real environment.

You May Also Like