Why web testing stops when a new release goes live

Posted by Richard Blundell on Wednesday 2 August 2017
Find me on:

Testing’s good in pre-production

A quick glance at the BugFinders blog pages will show that, as an organisation, we never stop shouting about the importance of web testing to help increase conversion rates, ensure brand loyalty and improve customer satisfaction. You’ll be pleased to hear this blog will not cover the same ground! Suffice to say, the IT teams that build and maintain the truly great websites, make sure they test their site endlessly in a pre-production environment to prevent embarrassing issues emerging in the days and weeks that follow the launch. Many of them take advantage of our intensive crowdsourced pre-release tests to uncover any critical, conversion and CX issues.

But testing stops on release day! Yes, monitoring happens, but not testing

What we’ve discovered to be unusual is that once a new release has actually gone live, most web testing stops. At this stage, monitoring web quality seems to become either reactive or machine-driven. The reactive piece is when a customer (or God forbid the CEO) spots a bug - and they can then let the development team know and a fix can be found. And the machine-driven piece is the automation scripts that are written at launch to test the 10-20 key customer journeys or the digital monitoring tools that can measure how well the site is performing.

But they will only alert the development or product teams once the ecommerce revenue damage has already been done. It normally takes a pretty significant drop in ecommerce income for the alarm to be raised and by then the horse has bolted. W e are often contacted by organisations who have seen an alarming drop in revenues coming from a section of their site, looking for a crowdsourced team to dive deep to find out why.

The solution: always on, human testing on a live website

What would be ideal is if there was such a thing as human testing on a live environment – this would be a highly proactive testing solution where the organisation could be contacted by a real consumer tester the minute a problem arose. We could let companies know when the stable door lock started to rattle, not when the horse is 2 miles down the road.

Maybe the site stops working on tablets, or a new data upload means that several top selling products can longer be bought. Maybe the product team have uploaded images of winter woollies when they meant to upload the new summer collection. Who knows – accidents happen to the best of us and mistakes are made. It is the speed at which these mistakes are spotted and rectified which is key – if caught early little damage will be done. If caught late, then this error could potentially cost the company tens of thousands of pounds.

On demand, always on human testing would also pick up any issues caused by the release of new devices. Samsung released 34 new mobile devices in 2016, iPhone released 11 and Sony a further 8. How does the modern-day IT leader hope to possibly keep up when so many new technology platforms are introduced into the market each year? Well if you had a live testing crowd who owned all of these devices anyway, they would pick up the issues in seconds; oh, and they would spot any issues caused by new operating systems as well.

The good news

Obviously, we wrote this blog for a reason! The good news is that we are about to launch a truly ground breaking new service called Always On – and like all good products it does what it says on the tin. Unlike most of the 3-5 day tests we run, Always On has been created to leave our crowd of over 55,000 testers checking the quality of our customers’ websites 24/7/365. We’ve pushed the bounties up for our testers too so they focus on only the most critical errors on a site or app. These are the ones that could cause the most reputational or commercial damage. Using their own devices (so they are completely familiar with their phone or tablet’s navigation) they check everything is working as it should and alert the customer via the BugFinders Always On Dashboard App if there is a problem.

Web Testing

As ever the bug is written up, a video produced to show the problem and a text and email sent to crucial members of the development team. In this way hundreds of days of testing can be delivered on thousands of different devices and operating system combinations. This reassures the business that their website is clean and not hampered with critical technical issues. The technical team are then able to confirm with certainty that if there are problems with the company’s ecommerce conversion rates, they are not down to the quality of the site.

And the price of this service for a year? Less than the cost of employing a single full-time internal tester. You see (to nick another good advertising strapline) testing doesn’t have to be taxing!

Are you looking to take your testing to the next level? Download our guide to revolutionising your web testing strategy now...

Web Testing

Topics: Web Testing, Featured