advertisement

Analysing quantitative data

53 %
47 %
advertisement
Information about Analysing quantitative data

Published on May 20, 2008

Author: stevebaty

Source: slideshare.net

Description

An introduction to the analysis of quantitative data arising from user research
advertisement

Analysing quantitative data with Steve Baty UX Strategist Web Directions User Experience ‘08

Data is important Web Directions User Experience ’08 - Analysing Quantitative Data

We expend a lot of effort to gather it Web Directions User Experience ’08 - Analysing Quantitative Data

We don’t always use it well Web Directions User Experience ’08 - Analysing Quantitative Data

We’ll be looking at: Web Directions User Experience ’08 - Analysing Quantitative Data

We’ll be looking at: * time-to-completion Web Directions User Experience ’08 - Analysing Quantitative Data

We’ll be looking at: * time-to-completion * task completion rates Web Directions User Experience ’08 - Analysing Quantitative Data

We’ll be looking at: * time-to-completion * task completion rates * a/b testing Web Directions User Experience ’08 - Analysing Quantitative Data

We’ll be looking at: * time-to-completion * task completion rates * a/b testing * page-view data Web Directions User Experience ’08 - Analysing Quantitative Data

time-to- completion Web Directions User Experience ’08 - Analysing Quantitative Data

Web Directions User Experience ’08 - Analysing Quantitative Data

1 min 24 secs Web Directions User Experience ’08 - Analysing Quantitative Data

1 min 23.8 secs Web Directions User Experience ’08 - Analysing Quantitative Data

1 min 23.77 secs Web Directions User Experience ’08 - Analysing Quantitative Data

1 min 23.768 secs Web Directions User Experience ’08 - Analysing Quantitative Data

83.768 secs Web Directions User Experience ’08 - Analysing Quantitative Data

Our data Task 1 Task 2 User 1 83.5 131.1 User 2 User... 97.3 165.5 might Task 3 Task 4 Task 5 54.5 97.8 118.0 45.5 88.2 143.3 look like Task 6 Task 7 243.9 22.9 309.0 23.9 this... Web Directions User Experience ’08 - Analysing Quantitative Data

We can calculate... Web Directions User Experience ’08 - Analysing Quantitative Data

We can calculate... mean - AVERAGE() variance - VAR() standard dev’n - STDEV() Web Directions User Experience ’08 - Analysing Quantitative Data

Low-variability Medium-variability High-variability 90.43 s Web Directions User Experience ’08 - Analysing Quantitative Data

Compare 2 sets of data - between iterations - between audience segments Web Directions User Experience ’08 - Analysing Quantitative Data

Low sample sizes restrict options Web Directions User Experience ’08 - Analysing Quantitative Data

non-parametric version == no assumed dist’n Web Directions User Experience ’08 - Analysing Quantitative Data

Rank-sum test Web Directions User Experience ’08 - Analysing Quantitative Data

Time for a practical demonstration Web Directions User Experience ’08 - Analysing Quantitative Data

Web Directions User Experience ’08 - Analysing Quantitative Data

Web Directions User Experience ’08 - Analysing Quantitative Data

1 3 3 3 5.5 5.5 7.5 7.5 9 10 Web Directions User Experience ’08 - Analysing Quantitative Data

1 3 3 3 5.5 5.5 7.5 7.5 9 10 } } 2+3+4 } 5+6 7+8 =9/3 =11/2 =15/2 Web Directions User Experience ’08 - Analysing Quantitative Data

3 7.5 1 10 3 5.5 7.5 S0 = 27 3 n=5 5.5 9 S1 = 28 m=5 Web Directions User Experience ’08 - Analysing Quantitative Data

⎡ n ( n + 1) ⎤ U 0 = nm + ⎢ ⎥ − S0 ⎣ 2 ⎦ ⎡ 5 ( 5 + 1) ⎤ = 5x5 + ⎢ ⎥ − 27 ⎣ 2 ⎦ = 13 Web Directions User Experience ’08 - Analysing Quantitative Data

⎡ n ( n + 1) ⎤ U 0 = nm + ⎢ ⎥ − S0 ⎣ 2 ⎦ ⎡ 5 ( 5 + 1) ⎤ = 5x5 + ⎢ ⎥ − 27 ⎣ 2 ⎦ = 13 90% --> 38 95% --> 41 99% --> 45 Web Directions User Experience ’08 - Analysing Quantitative Data

task completion rates Web Directions User Experience ’08 - Analysing Quantitative Data

Only 2 possible values: success or fail Web Directions User Experience ’08 - Analysing Quantitative Data

Small samples lead to very broad estimates Web Directions User Experience ’08 - Analysing Quantitative Data

4/6 successes = 66.67% 21% - 99.3% with 62.5% most likely Web Directions User Experience ’08 - Analysing Quantitative Data

With 30 users 47.7% - 81.9% with 64.8% most likely Web Directions User Experience ’08 - Analysing Quantitative Data

s +1 Most likely = p = n+2 p (1 − p ) Range = p±z n Web Directions User Experience ’08 - Analysing Quantitative Data

p (1 − p ) p±z n Web Directions User Experience ’08 - Analysing Quantitative Data

p (1 − p ) p±z n most likely Web Directions User Experience ’08 - Analysing Quantitative Data

p (1 − p ) p±z n confidence level Web Directions User Experience ’08 - Analysing Quantitative Data

p (1 − p ) p±z n variability Web Directions User Experience ’08 - Analysing Quantitative Data

A/B Testing Photo courtesy of www.dorothyphoto.com Web Directions User Experience ’08 - Analysing Quantitative Data

Compare two different approaches to the same problem Web Directions User Experience ’08 - Analysing Quantitative Data

Run both simultaneously; randomly divert users to option B Web Directions User Experience ’08 - Analysing Quantitative Data

Compare using a Chi- squared test Web Directions User Experience ’08 - Analysing Quantitative Data

Example: clicks on an ad banner Ignore Click Total A 10,119 275 10,394 B 962 38 1,000 Total 11,081 313 11,394 Web Directions User Experience ’08 - Analysing Quantitative Data

(e ) 2 − oij χ =∑ 2 ij eij The test statistic is a measure of distance between what we expect to see (e), and what we actually observed (o). For each cell, subtract what we expect from what we saw, square it to remove any negative values, and divide it by the expected value. Add it all together... Web Directions User Experience ’08 - Analysing Quantitative Data

Calculated expected values For each cell: row total x column total/grand total Web Directions User Experience ’08 - Analysing Quantitative Data

Ignore Click Total 10,108 = 286 = A 10,394x(11,081/11,394) 10,394x(313/11,394) 10,394 973 = 27 = B 1,000x(11,081/11,394) 1,000x(313/11,394) 1,000 Total 11,081 313 11,394 Web Directions User Experience ’08 - Analysing Quantitative Data

Ignore Click Total A 10,108 - 10,119 = -11 286 - 275 = 11 10,394 B 973 - 962 = 11 27 - 38 = -11 1,000 Total 11,081 313 11,394 Web Directions User Experience ’08 - Analysing Quantitative Data

(e ) 2 − oij χ =∑ 2 ij eij 2 2 2 2 11 11 11 11 = + + + 10,108 286 973 27 = 0.012 + 0.423 + 0.124 + 4.48 = 5.04 Web Directions User Experience ’08 - Analysing Quantitative Data

χ 2 α = 0.025 = 5.02 < χ 2 χ 2 α = 0.01 = 6.63 > χ 2 Web Directions User Experience ’08 - Analysing Quantitative Data

page views pre- & post comparison Web Directions User Experience ’08 - Analysing Quantitative Data

Can be cyclical Web Directions User Experience ’08 - Analysing Quantitative Data

Can be cyclical Web Directions User Experience ’08 - Analysing Quantitative Data

Can be trending Web Directions User Experience ’08 - Analysing Quantitative Data

Typically compare the average Web Directions User Experience ’08 - Analysing Quantitative Data

But ignores fluctuation Web Directions User Experience ’08 - Analysing Quantitative Data

But ignores fluctuation ? Web Directions User Experience ’08 - Analysing Quantitative Data

z= ( x1 − x2 ) 2 2 s s + 1 2 n1 n2 2 Test  1 : x1 , s , n1 1 2 Test  2 : x2 , s , n2 2 Web Directions User Experience ’08 - Analysing Quantitative Data

z= ( x1 − x2 ) 2 2 s s + 1 2 In order: mean, n1 n2 variance & Test  1 : x1 , s , n1 2 number of data 1 points in each 2 Test  2 : x2 , s , n2 2 test. Web Directions User Experience ’08 - Analysing Quantitative Data

Mean difference z= ( x1 − x2 ) 2 2 s s + 1 2 In order: mean, n1 n2 variance & Test  1 : x1 , s , n1 2 number of data 1 points in each 2 Test  2 : x2 , s , n2 2 test. Web Directions User Experience ’08 - Analysing Quantitative Data

Mean difference z= ( x1 − x2 ) 2 2 s s Combined + 1 2 In order: mean, standard error n1 n2 variance & Test  1 : x1 , s , n1 2 number of data 1 points in each 2 Test  2 : x2 , s , n2 2 test. Web Directions User Experience ’08 - Analysing Quantitative Data

Mean difference z= ( x1 − x2 ) 2 2 s s Combined + 1 2 In order: mean, standard error n1 n2 variance & Test  1 : x1 , s , n1 2 number of data 1 points in each 2 Test  2 : x2 , s , n2 2 test. If z < -1.96 or > 1.96 a significance difference exists Web Directions User Experience ’08 - Analysing Quantitative Data

Pre Post x 1,288 1,331 2 1,369 756.25 s ni 60 30 Web Directions User Experience ’08 - Analysing Quantitative Data

z= ( x1 − x2 ) 2 2 s s + 1 2 n1 n2 = (1288 − 1331) 1369 756.25 + 60 30 43 = = 6.205 6.93 Web Directions User Experience ’08 - Analysing Quantitative Data

1,288 1,331 Web Directions User Experience ’08 - Analysing Quantitative Data

Read more... Statistics without tears by Derek Rowntree Flaws & Fallacies in statistical thinking by Stephen K Campbell http://uxstats.blogspot.com Web Directions User Experience ’08 - Analysing Quantitative Data

Thank you Web Directions User Experience ’08 - Analysing Quantitative Data

Add a comment

Related pages

Analyze Quantitative Data « Pell Institute

Analyze Quantitative Data. Quantitative data analysis is helpful in evaluation because it provides quantifiable and easy to understand results.
Read more

Analyzing Quantitative Data - LearningStore

Analyzing Quantitative Data Ellen Taylor-Powell G3658-6. Table 1. Frequency distribution of Extension partici-pants by place of residence (n = 860) Place of
Read more

Analyse This!!! - quantitative data introduction - Learnhigher

Analyse This!!! Learning to analyse quantitative data . 1/19 Quantitative data analysis . First of all let's define what we mean by quantitative data analysis.
Read more

Part II: Chapter 4 : Analyzing Quantitative Data - NSF

Chapter 4 Analyzing Qualitative Data . ... data most often occur in more embedded and less easily reducible or distillable forms than quantitative data.
Read more

Data analysis - Wikipedia

All are varieties of data analysis. Data integration is ... practices for understanding quantitative data. ... is stated before analysing the data, ...
Read more

6.7 Analysing quantitative data - World Health Organization

In the manual analysis of quantitative data, it is best to first summarize the data in a so-called data master sheet to facilitate analysis. On a master ...
Read more

5. Analysing Quantitative data - University of Surrey

5. Analysing Quantitative data. The analysis of research in any project involve summarising the mass of data that has been collected and the presenting the ...
Read more

QUANTITATIVE data analysis - University of Hertfordshire

QUANTITATIVE data analysis . Let us commence our look at data analysis by looking at a hypothetical research study. Remember that there are different ways ...
Read more

Qualitative Data Analysis - SAGE Publications

Features of Qualitative Data Analysis Qualitative Data Analysis as an Art. Qualitative Compared With Quantitative . Data Analysis. Techniques of ...
Read more

4. Analysing Qualitative Research Data

4. Analysing Qualitative Research Data. The analysis of qualitative research involves aiming to uncover and / or understand the big picture - by using the ...
Read more