QA: How I Learned to Stop Breaking and Start Loving Tests

67 %
33 %
Information about QA: How I Learned to Stop Breaking and Start Loving Tests
Business & Mgmt

Published on March 18, 2014

Author: bmassey



Joel Harvey details new ways to look at quality assurance and outlines the approach his company, Conversion Sciences ensures it's tests don't break client websites

OPTIMIZATION QA: How to Avoid Launching Tests that Break Your Site When you break shit you pollute data, lose money and eventually get fired. Here’s how to avoid that pain and look like a hero in the process. @joeljharvey #convcon Joel Harvey, COO Conversion Sciences

@joeljharvey #convcon Goal

@joeljharvey #convcon What’s QA?

@joeljharvey #convcon Let’s say that …

@joeljharvey #convcon 30% of your visitors get a broken experience on your site

@joeljharvey #convcon Y O U R SITE IS BROKEN ON ONE OF T H E S E SCREENS

@joeljharvey #convcon No One On Your Team Knows

@joeljharvey #convcon 20% RPV IE BUG

@joeljharvey #convcon INTERNET EXPLORER

@joeljharvey #convcon You’re about to hire this guy as your CPA

@joeljharvey #convcon QA MATH

@joeljharvey #convcon Internet Explorer Traffic: Feb 2013 Internet Explorer Traffic: Feb 2014

@joeljharvey #convcon  120% Increase Internet Explorer (IE) Revenue  20% Increase in All Revenue  20% Increase in IE Average Order Value  167% Increase in IE Conversion Rate $1,500,000 Per Year

@joeljharvey #convcon Who Are You?

@joeljharvey #convcon Who Am I? - COO & Managing Partner, Conversion Sciences - Conversion Scientist - Data Geek - Entrepreneurial MBA

@joeljharvey #convcon


@joeljharvey #convcon 30% of your visitors get a broken experience on your site

QA DRIVES REVENUE @joeljharvey #convcon

OLD BORING DEFINITION @joeljharvey #convcon

@joeljharvey #convcon

QA is just a Cost Center We can’t support everything! QA prevents us from being agile People on really old devices probably aren’t buyers QA is an IT / Development Task @joeljharvey #convcon

QA IS A MARKETING FUNCTION @joeljharvey #convcon


@joeljharvey #convcon 1. CULTURE 3. TECH PARADIGMS 2. REAL USER USE CASES

QA IS OPTIMIZATION @joeljharvey #convcon

1. Understand your visitors @joeljharvey #convcon

2. Make sure control is not broken @joeljharvey #convcon

3. Generate great hypotheses & test them @joeljharvey #convcon

@joeljharvey #convcon 4. Don’t pollute your test data

5. Do what the data tells you to do @joeljharvey #convcon

BUILDING A QA CULTURE @joeljharvey #convcon

@joeljharvey #convcon #QAHAMMER

@joeljharvey #convcon Build Destroy

HAMMER UP @joeljharvey #convcon

BUY IN @joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon 20% RPV IE BUG

@joeljharvey #convcon

REAL USE CASES @joeljharvey #convcon

90% @joeljharvey #convcon


@joeljharvey #convcon SCREEN RECORDING

@joeljharvey #convcon USER TESTING

DOCUMENT AND QA ALL USE CASES @joeljharvey #convcon

YOUR JOB IS TO BREAK YOUR SITE @joeljharvey #convcon

@joeljharvey #convcon

WHAT SHOULD YOU SUPPORT? @joeljharvey #convcon

5% @joeljharvey #convcon

@joeljharvey #convcon Old New BROWSER SUPPORT  Internet Explorer 8  Chrome 7  Firefox 4  Safari 4.1  iOS 5.1 *  Internet Explorer 11  Chrome 33  Firefox 27  Safari 7.0  iOS 7.1 *

@joeljharvey #convcon OS SUPPORT Old New  Windows XP  OS X 10.4 (Tiger)  Android 2.3  iOS 5.1  Windows Phone 7  Windows 8  OS X 10.9 (Maverick)  Android 4.4  iOS 7.1  Windows Phone 8.1

5% MINIMUM @joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

QA TOOLS #4 @joeljharvey #convcon BUILD YOUR QA’TION STATION


iOS 7.1? @joeljharvey #convcon

@joeljharvey #convcon SOFTWARE vs. HARDWARE

@joeljharvey #convcon iPad, iOS 5 iPad, iOS 7.1 Windows Vista Android 4.3 iPhones: iOS4, 5, 7 Android: 2.3.6, 4.0.4, Windows 8, no updates OS X 10.6 Windows 8, regular updates + VM’s OS X 10.8 OS X 10.9 OS X 10.7 + VM’s Windows XP Windows 7

@joeljharvey #convcon

@joeljharvey #convcon ?

@joeljharvey #convcon 1. Browser 2. Operating System 3. “Other” - Visitor Internet Speed - Browser Plugins - Flash - Resolution QA’TION STATION CHECKLIST

QA PROCESS #4 @joeljharvey #convcon BE THE HAMMER

QA EXPERIENCES, NOT PAGES @joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

@joeljharvey #convcon

TEST QA PROCESS @joeljharvey #convcon

1. Build Your Test @joeljharvey #convcon

2. Use Cookie, Query String and/or Location Targeting so Only You See Experience @joeljharvey #convcon

3. Pause / Disable All But One Experience @joeljharvey #convcon

4. Review Live Experience In All Supported Browser/OS Combos (like REAL USERS) @joeljharvey #convcon

5. Clear Cookies, Pause Good-To-Go Experience and Unpause the Next One @joeljharvey #convcon

6. Repeat @joeljharvey #convcon







@joeljharvey #convcon Need to Borrow a QA Hammer? Call 1-888-961-6604 and mention this session for a chance to win a Free QA Audit ($15,000 Value) Conversion Conference San Francisco, 2014 Joel Harvey, Conversion Sciences

#convcon presentations

Add a comment

Related presentations

Canvas Prints at Affordable Prices make you smile.Visit http://www.shopcanvasprint...

30 Días en Bici en Gijón organiza un recorrido por los comercios históricos de la ...

Con el fin de conocer mejor el rol que juega internet en el proceso de compra en E...

With three established projects across the country and seven more in the pipeline,...

Retailing is not a rocket science, neither it's walk-in-the-park. In this presenta...

What is research??

What is research??

April 2, 2014

Explanatory definitions of research in depth...

Related pages

Dr. Strangelove - Wikipedia, the free encyclopedia

Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb, more commonly known as Dr. Strangelove, is a 1964 political satire black comedy film ...
Read more

Dr. Strangelove or: How I Learned to Stop Worrying and ...

Search for "Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb" on ... Test your knowledge of Dr. Strangelove or: ...
Read more

How to Stop Loving Someone: 9 Steps (with Pictures) - wikiHow

Read this guide to learn how to stop loving an ... Start with the obvious one ... online dating sites are a great way to set up dates and test the waters ...
Read more

Software testing - Wikipedia, the free encyclopedia

Rather than replace traditional QA ... Testing will determine the breaking ... Another practice is to start software testing at the same moment the ...
Read more

Alcohol Abuse Treatment and Self-Help: How to Stop ...

How to Stop Drinking and Start Recovery ... Any alcohol addiction treatment program should be customized to your unique problems and situation.
Read more

Braking and Turning Your Bicycle - Sheldon Brown-Bicycle ...

Test the brakes in this way ... you should use the rear brake alone to bring yourself to a safe stop. Braking a wheel that has ... If you start to ...
Read more

How to Ride a Motorcycle - How to NOT Stall at Stop Lights ...

Starting out in first gear is the hardest part about learning to ride a motorcycle ... Start On A Motorcycle ... Motorcycle Skills Tests ...
Read more

Anxiety Culture: How to Stop Worrying

... the habit of worrying seems hard to stop. We're raised to worry ... learned /conditioned ... Starts with ...
Read more

Q&A with Hans Rosling (Part 2): How to change Fidel Castro ...

And it’s difficult to stop Castro when he begins ... despite the fact that all your efforts are focused on breaking our realities ... TED Blog; Our ...
Read more