30 Sep Marketers use A/B testing to determine which marketing campaign, website layout, or product performs better. You show version A of a piece of marketing content to one half of your audience and ve
Marketers use A/B testing to determine which marketing campaign, website layout, or product performs better. You show version A of a piece of marketing content to one half of your audience and version B to another. You compare the average ratings of version A and version B to determine which version performs better. Perform A/B testing on A/B Testing Survey Data. Download A/B Testing Survey Data. Group 1 saw website layout version A and Group 2 saw version B. Read Statistical Analysis in Excel
Actions pages 1-2 and add in 'Data Analysis' toolkit. Read pages 5-7 and perform ANOVA (Analysis of Variance).
- What is the average rating for version A? What is the average rating for version B?
- If p-value is less than 0.1, it means that the difference between two groups is statistically significant. Is p-value lower than 0.1?
- Is version B liked more than version A?
So you have to create 2 separate surveys one with an open ended questionnaire. And one with semantic differential scale (1-5). This is just extra credit so just to be safe do two different ones!
1
Statistical Analysis in
Excel
Run linear regression
1. Click ‘File’ tab → ‘More’ (bottom-left) → ‘Options’
2. Click ‘Add-ins’ (bottom-left) → Select ‘Excel Add-ins’ → Click
‘Go…’
2
3. Check the box ‘Analysis ToolPak’ → Click ‘OK’
4. Click ‘Data’ on the top menu → Click ‘Data Analysis’ on your
top right.
3
5. Scroll down → Select ‘Regression’ → Click ‘OK’
6. Input “$A1:$C70” for X range and “$D1:$D70” for Y range –
Check ‘Labels’ and ‘Line Fit Plots’ → Click ‘OK’
4
7. The linear regression results will load on a separate tab.
Check the coefficient (effect) of X variables (TV, radio, and social
media ad budgets). Are they positive or negative? Does each TV,
radio, and social media ad influence sales positively?
Check the R-squared value. Is it higher than 0.2? Can you
conclude that the model is statistically significant?
5
Perform analysis of variance (ANOVA)
1. Compare the satisfaction ratings for two different website
layouts.
2. Group 1 rated the layout A, and Group 2 rated the layout B
6
3. Click ‘Data’ tab on the top menu → Click ‘Data Analysis’ on
your top right → Select ‘ANOVA Single Factor’ → Click ‘OK’
4. Scroll and select all cells ($A$1:$B$12) → Check “Labels in
First Rows” → Click ‘OK’
7
5. Excel gives the ANOVA results on a new tab.
What is the F score?
Is P-value lower than 0.1?
Are the ANOVA results statistically significant?
Can you say that satisfaction ratings significantly vary by
website layout?
,
Sheet1
| Group 1 | Group 2 |
| 5 | 10 |
| 6 | 9 |
| 8 | 7 |
| 4 | 6 |
| 5 | 9 |
| 8 | 8 |
| 6 | 10 |
| 7 | 8 |
| 5 | 9 |
| 6 | 10 |
| 7 | 7 |
