[vc_row][vc_column][vc_column_text]Marketo AB TestingWe’ve all heard advertising claims backed by sketchy statistics, like “9 out of 10 doctors prefer” this product or that. After hearing so many of these claims we suspect that the reality of the test was, “we gave 20 chosen doctors a choice between our product and another, we picked the 9 that chose our product, and threw in one that picked the other, and now we’re hoping you believe that 90% of doctors prefer our product.”

If you want a good A/B test, then make sure it is designed to be statistically relevant. Setup your test based on the right statistical foundation, and you can rely on the results. Ignore the statistic foundation, and you may as well guess. A solid foundation of statistical relevance involves several factors, but its cornerstone is the p-value.

In plain English, a p-value of .047 is saying is that there is a 4.7% chance that you could have obtained these results by random chance. There is a 4.7% chance that if you were to run this experiment again you would not see the same result.

A key here is to not consider the test a failure if the results are inconclusive (p-value is greater than .05). Knowing that changes to certain email content or timing won’t likely have an effect on your audience is just as useful for future communication strategies. If you still feel strongly that the first experiment wasn’t enough to capture the difference in your group’s responses, then replicate the experiment to add to the strength of your results.[/vc_column_text][vc_empty_space height=”40px”][/vc_column][/vc_row][vc_row][vc_column width=”1/6″][/vc_column][vc_column width=”2/3″][vc_column_text]

Once you know your p-value, you need to design your test correctly. Here are a few rules of thumb that can help:

[/vc_column_text][vc_empty_space height=”40px”][/vc_column][vc_column width=”1/6″][/vc_column][/vc_row][vc_row][vc_column width=”1/3″][edgtf_icon_with_text icon_pack=”simple_line_icons” simple_line_icons=”icon-people” icon_position=”top” icon_type=”normal” icon_size=”edgtf-icon-large” icon_animation=”” text=”Choose the right sample size”][/vc_column][vc_column width=”1/3″][edgtf_icon_with_text icon_pack=”simple_line_icons” simple_line_icons=”icon-puzzle” icon_position=”top” icon_type=”normal” icon_size=”edgtf-icon-large” icon_animation=”” text=”Don’t change too many variables at once”][/vc_column][vc_column width=”1/3″][edgtf_icon_with_text icon_pack=”simple_line_icons” simple_line_icons=”icon-bulb” icon_position=”top” icon_type=”normal” icon_size=”edgtf-icon-large” icon_animation=”” text=”Know exactly what you want to test before starting”][/vc_column][/vc_row][vc_row][vc_column][vc_empty_space height=”40px”][/vc_column][/vc_row][vc_row css=”.vc_custom_1476205271252{padding-top: 40px !important;padding-right: 20px !important;padding-bottom: 40px !important;padding-left: 20px !important;background-color: #babf33 !important;}”][vc_column][vc_column_text]

Want to dig deeper into A/B test design?

[/vc_column_text][vc_empty_space height=”30px”][vc_row_inner][vc_column_inner width=”2/3″][vc_single_image image=”6258″ img_size=”full” alignment=”center” onclick=”custom_link” img_link_target=”_blank” link=”https://digitalpi.com/wp-content/uploads/2016/10/DPI-eBook-ABtesting.pdf”][/vc_column_inner][vc_column_inner width=”1/3″][vc_empty_space height=”24px”][edgtf_button type=”outline” size=”huge-full-width” text=”DOWNLOAD eBOOK” target=”_blank” icon_pack=”simple_line_icons” simple_line_icons=”icon-book-open” font_weight=”700″ color=”#f2f0e4″ hover_color=”#f2f0e4″ hover_background_color=”#47592a” border_color=”#f2f0e4″ hover_border_color=”#47592a” font_size=”24″ link=”https://digitalpi.com/wp-content/uploads/2016/10/DPI-eBook-ABtesting.pdf”][vc_column_text]

“How to make the most out of your Marketo A/B test.”