{"id":12682,"date":"2015-07-15T00:00:00","date_gmt":"2015-07-15T00:00:00","guid":{"rendered":"https:\/\/e-dialog.group\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/"},"modified":"2026-02-26T13:37:57","modified_gmt":"2026-02-26T13:37:57","slug":"when-is-your-test-result-significant-the-statistics-behind-the-ab-test","status":"publish","type":"post","link":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/","title":{"rendered":"When Is Your Test Result Significant The Statistics Behind The Ab Test"},"content":{"rendered":"<p>&ldquo;Don&rsquo;t trust any statistics you haven&rsquo;t falsified yourself,&rdquo; Winston Churchill most likely never said. However, you often hear this sentence in everyday life when the validity of statistical results is questioned. And in fact: it never hurts to know how such a result came about, especially if you do it yourself<a href=\"https:\/\/e-dialog.group\/blog\/conversion-optimierung\/ab-test-fundierte-entscheidungsgrundlage-statt-ratespiel\/\">A\/B testing<\/a>for conversion optimization.<\/p>\n<p>In the endless expanses of the Internet there are numerous online calculators that determine whether the difference between the conversion rates of two test variants is significant. Also with<strong><a href=\"\/wissen\/signifikanzrechner\/\">our significance calculator<\/a><\/strong>see at a glance which result is significant and which uplift or downlift could have occurred by chance.<\/p>\n<p>However, we want to take a look at how these computers work. In this sense: &ldquo;Don&rsquo;t trust any statistics that you haven&rsquo;t calculated yourself!&rdquo;<\/p>\n<h2>Step 1: Observed and expected values<\/h2>\n<p>Most significance calculators use chi for their calculation<sup>2<\/sup>-Test (pronounced: ki-square). How this works and how it is calculated will be shown here using a fictitious example:<\/p>\n<table style=\"width: 100%;\" border=\"1\" cellspacing=\"1\" cellpadding=\"1\">\n<thead>\n<tr>\n<th scope=\"row\"><\/th>\n<th style=\"text-align: center;\" scope=\"col\">Visitors<\/th>\n<th style=\"text-align: center;\" scope=\"col\">Conversions<\/th>\n<th style=\"text-align: center;\" scope=\"col\">Conversion rate<\/th>\n<\/tr>\n<tr>\n<th scope=\"row\">Variant A<\/th>\n<td style=\"text-align: center;\" scope=\"col\">9,998<\/td>\n<td style=\"text-align: center;\" scope=\"col\">1,001<\/td>\n<td style=\"text-align: center;\" scope=\"col\">10.01%<\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<th scope=\"row\">Variant B<\/th>\n<td style=\"text-align: center;\">10,001<\/td>\n<td style=\"text-align: center;\">1,087<\/td>\n<td style=\"text-align: center;\">10.87%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The conversion rates are obtained by dividing the observed conversions of a variant by the total number of visitors to a variant and multiplying them by 100%. For variant A this results in a conversion rate of 10.01% and for variant B 10.87%. The measure by which variant B performs better than variant A is called the uplift. This is calculated as follows:<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/daum_equation_1472110857969.png\" sizes=\"auto, (max-width: 652px) 100vw, 652px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/daum_equation_1472110857969.png 652w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/daum_equation_1472110857969-300x97.png 300w\" alt=\"\" width=\"652\" height=\"210\">So variant B has an 8.55% higher conversion rate than variant A.<strong>However, this does not automatically mean that variant B is better than variant A.<\/strong>The different conversion rates can also arise by chance. This random deviation is called &ldquo;statistical noise&rdquo;.<\/p>\n<p>For example, if you throw a die 600 times, you can expect the six to fall 100 times. In fact, there is a very high probability that we will observe a value that deviates from this. Maybe six is &#8203;&#8203;rolled 110 times, maybe just 92 times. Only if we roll an infinite number of dice can we be sure that exactly one sixth of all rolls will show a 6.<\/p>\n<p>However, since an infinite number of observations is practically impossible, we must accept that the observed value may differ from the actual value. In order to be able to judge whether variant B actually performs better than variant A, we have to find out how likely the difference is due to statistical noise.<\/p>\n<p>To do this, a crosstab is created that contains the observed values &#8203;&#8203;of converted and non-converted visitors:<\/p>\n<table style=\"width: 100%;\" border=\"1\" cellspacing=\"1\" cellpadding=\"1\">\n<thead>\n<tr>\n<th scope=\"row\"><\/th>\n<th style=\"text-align: center;\" scope=\"col\">Visitors<\/th>\n<th style=\"text-align: center;\" scope=\"col\">Conversions<\/th>\n<th style=\"text-align: center;\" scope=\"col\">No conversions<\/th>\n<th style=\"text-align: center;\" scope=\"col\">Conversion rate<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<th scope=\"row\">Variant A<\/th>\n<td style=\"text-align: center;\">9,998<\/td>\n<td style=\"text-align: center;\">1,001<\/td>\n<td style=\"text-align: center;\">8,997<\/td>\n<td>10.01%<\/td>\n<\/tr>\n<tr>\n<th scope=\"row\">Variant B<\/th>\n<td style=\"text-align: center;\">10,002<\/td>\n<td style=\"text-align: center;\">1,087<\/td>\n<td style=\"text-align: center;\">8,915<\/td>\n<td>10.87%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The next step is to compare these observed values &#8203;&#8203;with the values &#8203;&#8203;that one would expect if there were no difference between the two variants. The expected values &#8203;&#8203;result from the common conversion rate of both variants:<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_gemeinsamecr_0.png\" sizes=\"auto, (max-width: 714px) 100vw, 714px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_gemeinsamecr_0.png 714w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_gemeinsamecr_0-300x66.png 300w\" alt=\"\" width=\"714\" height=\"156\">The number of visitors to each variant is now multiplied with this common conversion rate. This gives us the expected conversions of a variant:<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_erwarteteconversions_0.png\" sizes=\"auto, (max-width: 502px) 100vw, 502px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_erwarteteconversions_0.png 502w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_erwarteteconversions_0-300x137.png 300w\" alt=\"\" width=\"502\" height=\"229\">The expected non-conversions are easily determined by subtracting the expected conversions from the number of visitors of the two variants:<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_erwartetenichtconversions_0.png\" sizes=\"auto, (max-width: 735px) 100vw, 735px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_erwartetenichtconversions_0.png 735w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_erwartetenichtconversions_0-300x93.png 300w\" alt=\"erwartete nicht-conversions\" width=\"735\" height=\"229\">We now enter these values &#8203;&#8203;into our crosstab:<\/p>\n<table style=\"width: 100%;\" border=\"1\" cellspacing=\"1\" cellpadding=\"1\">\n<thead>\n<tr>\n<th scope=\"row\"><\/th>\n<th style=\"text-align: center;\" scope=\"col\">Visitors<\/th>\n<th style=\"text-align: center;\" colspan=\"2\" scope=\"col\">Conversions<\/th>\n<th style=\"text-align: center;\" colspan=\"2\" scope=\"col\">No conversions<\/th>\n<th style=\"text-align: center;\" scope=\"col\">CR<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<th scope=\"row\"><\/th>\n<td><\/td>\n<td>observed<\/td>\n<td>expected<\/td>\n<td>observed<\/td>\n<td>expected<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<th scope=\"row\">Variant A<\/th>\n<td style=\"text-align: center;\">9,998<\/td>\n<td style=\"text-align: center;\">1,001<\/td>\n<td style=\"text-align: center;\">1,044<\/td>\n<td style=\"text-align: center;\">8,997<\/td>\n<td>8,954<\/td>\n<td style=\"text-align: center;\">10.01%<\/td>\n<\/tr>\n<tr>\n<th scope=\"row\">Variant B<\/th>\n<td style=\"text-align: center;\">10,002<\/td>\n<td style=\"text-align: center;\">1,087<\/td>\n<td style=\"text-align: center;\">1,044<\/td>\n<td style=\"text-align: center;\">8,915<\/td>\n<td>8,958<\/td>\n<td style=\"text-align: center;\">10.87%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2>Step 2: Calculate deviation values<\/h2>\n<p>Now we calculate the deviation from the expected value for each observed value. This is done with the formula:<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung.png\" sizes=\"auto, (max-width: 411px) 100vw, 411px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung.png 411w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung-300x47.png 300w\" alt=\"Abweichung\" width=\"411\" height=\"65\">By squaring the deviation, we ensure that it does not matter whether the observed value is below or above the expected value and avoid positive and negative deviations canceling out or weakening each other. In addition, larger deviations are taken into account more than small deviations.<\/p>\n<p>By dividing by the expected value, we also accept a slightly larger deviation for higher expected values &#8203;&#8203;than for lower expected values.<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung11.png\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung11.png 932w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung11-300x51.png 300w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_abweichung11-768x131.png 768w\" alt=\"Abweichung\" width=\"800\" height=\"136\">We proceed similarly for the conversions of variant B and the values &#8203;&#8203;of the non-conversions. We now add all four values &#8203;&#8203;together and get the test value for the Chi<sup>2<\/sup>-Test:<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_pruefwert.png\" sizes=\"auto, (max-width: 479px) 100vw, 479px\" srcset=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_pruefwert.png 479w, https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/eq_pruefwert-300x18.png 300w\" alt=\"Pr&uuml;fwert\" width=\"479\" height=\"28\"><\/p>\n<h2>Step 3: Check for significance<\/h2>\n<p>In general, the higher this value is, the higher the probability that the two variants actually differ. It is not possible to make a 100% statement about this, so a confidence value must be chosen. This indicates the probability that the two variants are different. For this confidence value you now hit in one<a href=\"https:\/\/www.medcalc.org\/manual\/chi-square-table.php\">Chi<sup>2<\/sup>-Distribution table<\/a>how high the test value must be at least so that a difference between the two variants is at least as high as the confidence value. Popular confidence values &#8203;&#8203;and the associated minimum test values &#8203;&#8203;are:<\/p>\n<table style=\"width: 100%;\" border=\"1\" cellspacing=\"1\" cellpadding=\"1\">\n<thead>\n<tr>\n<th style=\"text-align: center;\" scope=\"col\">Confidence<\/th>\n<th style=\"text-align: center;\" scope=\"col\">Minimum test value<\/th>\n<th style=\"text-align: center;\" scope=\"col\">note<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: center;\">90%<\/td>\n<td style=\"text-align: center;\">2.71<\/td>\n<td style=\"text-align: center;\">tends to<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: center;\">95%<\/td>\n<td style=\"text-align: center;\">3.84<\/td>\n<td style=\"text-align: center;\">significant<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: center;\">99%<\/td>\n<td style=\"text-align: center;\">6.63<\/td>\n<td style=\"text-align: center;\">very significant<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: center;\">99.9%<\/td>\n<td style=\"text-align: center;\">10.83<\/td>\n<td style=\"text-align: center;\">highly significant<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>We decide on a confidence of 95%, for this we need a test value of at least 3.84. With 3.955 we skip this minimum value, so we can claim with 95% probability:<em>Variants A and B differ significantly from each other. The measured uplift actually has its origin in a higher conversion rate and is not just caused by statistical noise.<\/em>However, what we cannot say for sure is that the uplift is actually +8.55%. This is the most likely value that the uplift can take, but it is also subject to statistical noise. But it is very likely close to this value.<\/p>\n<p>Is all this too complicated for you? Then just use that<a style=\"display: inline-block; padding: 2px 6px; background-color: #0a6696; color: white; font-weight: bold; text-decoration: none;\" href=\"\/wissen\/signifikanzrechner\/\" target=\"_blank\" rel=\"noopener noreferrer\">e-dialog significance calculator &gt;<\/a><\/p>\n<h3>Are you not sure what to do with the calculated result? Or do you not know what to test?<br>\nWe are happy to help you!&nbsp;Write to us via<a href=\"mailto:kontakt@e-dialog.group\">kontakt@e-dialog.group<\/a><\/h3>\n","protected":false},"excerpt":{"rendered":"<p>In this blog post we want to show you how to calculate significance in three steps. It&#8217;s always good to know how such a result comes about and what&#8217;s behind it. That&#8217;s why we&#8217;ll take a look at how these computers work.<\/p>\n","protected":false},"author":1,"featured_media":4804,"comment_status":"closed","ping_status":"open","sticky":false,"template":"post-old.php","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[5,446],"channel":[459],"goal":[466],"technology":[],"c-year":[71],"class_list":["post-12682","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analytics","category-campaigns","channel-web","goal-performance-marketing","c-year-71"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.5 (Yoast SEO v27.5) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>When Is Your Test Result Significant The Statistics Behind The Ab Test - e-dialog<\/title>\n<meta name=\"description\" content=\"In this blog post we want to show you how to calculate significance in three steps. It&#039;s always good to know how such a result comes about and what&#039;s behind it. That&#039;s why we&#039;ll take a look at how these computers work.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"When Is Your Test Result Significant The Statistics Behind The Ab Test\" \/>\n<meta property=\"og:description\" content=\"In this blog post we want to show you how to calculate significance in three steps. It&#039;s always good to know how such a result comes about and what&#039;s behind it. That&#039;s why we&#039;ll take a look at how these computers work.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/\" \/>\n<meta property=\"og:site_name\" content=\"e-dialog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/edialog.group\" \/>\n<meta property=\"article:published_time\" content=\"2015-07-15T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-02-26T13:37:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/cookie-banner-ch.png\" \/>\n\t<meta property=\"og:image:width\" content=\"2200\" \/>\n\t<meta property=\"og:image:height\" content=\"1180\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"digitalists\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"digitalists\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/\"},\"author\":{\"name\":\"digitalists\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#\\\/schema\\\/person\\\/7b481e2ac90c21e4dc393821d35bb518\"},\"headline\":\"When Is Your Test Result Significant The Statistics Behind The Ab Test\",\"datePublished\":\"2015-07-15T00:00:00+00:00\",\"dateModified\":\"2026-02-26T13:37:57+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/\"},\"wordCount\":901,\"publisher\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/e-dialog.group\\\/wp-content\\\/uploads\\\/2025\\\/08\\\/cookie-banner-ch.png\",\"articleSection\":[\"Analytics\",\"Campaigns\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/\",\"url\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/\",\"name\":\"When Is Your Test Result Significant The Statistics Behind The Ab Test - e-dialog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/e-dialog.group\\\/wp-content\\\/uploads\\\/2025\\\/08\\\/cookie-banner-ch.png\",\"datePublished\":\"2015-07-15T00:00:00+00:00\",\"dateModified\":\"2026-02-26T13:37:57+00:00\",\"description\":\"In this blog post we want to show you how to calculate significance in three steps. It's always good to know how such a result comes about and what's behind it. That's why we'll take a look at how these computers work.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#primaryimage\",\"url\":\"https:\\\/\\\/e-dialog.group\\\/wp-content\\\/uploads\\\/2025\\\/08\\\/cookie-banner-ch.png\",\"contentUrl\":\"https:\\\/\\\/e-dialog.group\\\/wp-content\\\/uploads\\\/2025\\\/08\\\/cookie-banner-ch.png\",\"width\":2200,\"height\":1180,\"caption\":\"|\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"When Is Your Test Result Significant The Statistics Behind The Ab Test\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/\",\"name\":\"e-dialog\",\"description\":\"Data-driven Marketing &amp; Strategie\",\"publisher\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#organization\",\"name\":\"e-dialog\",\"url\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/e-dialog.group\\\/wp-content\\\/uploads\\\/2025\\\/06\\\/e-dialog-logo.svg\",\"contentUrl\":\"https:\\\/\\\/e-dialog.group\\\/wp-content\\\/uploads\\\/2025\\\/06\\\/e-dialog-logo.svg\",\"width\":1,\"height\":1,\"caption\":\"e-dialog\"},\"image\":{\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/edialog.group\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/e-dialog-gmbh\\\/\",\"https:\\\/\\\/www.instagram.com\\\/e_dialog\\\/\",\"https:\\\/\\\/www.tiktok.com\\\/@e_dialog\"],\"description\":\"Bei e-dialog gestalteten wir gemeinsam die Zukunft des data-driven Marketings - mit einem gro\u00dfartigen Team aus passionierten Expert*innen.\",\"email\":\"kontakt@e-dialog.group\",\"telephone\":\"+43 1 309 09 09\",\"legalName\":\"e-dialog GmbH\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/#\\\/schema\\\/person\\\/7b481e2ac90c21e4dc393821d35bb518\",\"name\":\"digitalists\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/8acd1abfb5ef9f78793e3164334d4533968653b26cbd89b6cf0eb4e5483434e4?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/8acd1abfb5ef9f78793e3164334d4533968653b26cbd89b6cf0eb4e5483434e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/8acd1abfb5ef9f78793e3164334d4533968653b26cbd89b6cf0eb4e5483434e4?s=96&d=mm&r=g\",\"caption\":\"digitalists\"},\"sameAs\":[\"https:\\\/\\\/e-dialog.group\"],\"url\":\"https:\\\/\\\/e-dialog.group\\\/en\\\/blog\\\/author\\\/digitalists\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"When Is Your Test Result Significant The Statistics Behind The Ab Test - e-dialog","description":"In this blog post we want to show you how to calculate significance in three steps. It's always good to know how such a result comes about and what's behind it. That's why we'll take a look at how these computers work.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/","og_locale":"en_US","og_type":"article","og_title":"When Is Your Test Result Significant The Statistics Behind The Ab Test","og_description":"In this blog post we want to show you how to calculate significance in three steps. It's always good to know how such a result comes about and what's behind it. That's why we'll take a look at how these computers work.","og_url":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/","og_site_name":"e-dialog","article_publisher":"https:\/\/www.facebook.com\/edialog.group","article_published_time":"2015-07-15T00:00:00+00:00","article_modified_time":"2026-02-26T13:37:57+00:00","og_image":[{"width":2200,"height":1180,"url":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/cookie-banner-ch.png","type":"image\/png"}],"author":"digitalists","twitter_card":"summary_large_image","twitter_misc":{"Written by":"digitalists","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#article","isPartOf":{"@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/"},"author":{"name":"digitalists","@id":"https:\/\/e-dialog.group\/en\/#\/schema\/person\/7b481e2ac90c21e4dc393821d35bb518"},"headline":"When Is Your Test Result Significant The Statistics Behind The Ab Test","datePublished":"2015-07-15T00:00:00+00:00","dateModified":"2026-02-26T13:37:57+00:00","mainEntityOfPage":{"@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/"},"wordCount":901,"publisher":{"@id":"https:\/\/e-dialog.group\/en\/#organization"},"image":{"@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#primaryimage"},"thumbnailUrl":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/cookie-banner-ch.png","articleSection":["Analytics","Campaigns"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/","url":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/","name":"When Is Your Test Result Significant The Statistics Behind The Ab Test - e-dialog","isPartOf":{"@id":"https:\/\/e-dialog.group\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#primaryimage"},"image":{"@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#primaryimage"},"thumbnailUrl":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/cookie-banner-ch.png","datePublished":"2015-07-15T00:00:00+00:00","dateModified":"2026-02-26T13:37:57+00:00","description":"In this blog post we want to show you how to calculate significance in three steps. It's always good to know how such a result comes about and what's behind it. That's why we'll take a look at how these computers work.","breadcrumb":{"@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#primaryimage","url":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/cookie-banner-ch.png","contentUrl":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/08\/cookie-banner-ch.png","width":2200,"height":1180,"caption":"|"},{"@type":"BreadcrumbList","@id":"https:\/\/e-dialog.group\/en\/blog\/when-is-your-test-result-significant-the-statistics-behind-the-ab-test\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/e-dialog.group\/en\/"},{"@type":"ListItem","position":2,"name":"When Is Your Test Result Significant The Statistics Behind The Ab Test"}]},{"@type":"WebSite","@id":"https:\/\/e-dialog.group\/en\/#website","url":"https:\/\/e-dialog.group\/en\/","name":"e-dialog","description":"Data-driven Marketing &amp; Strategie","publisher":{"@id":"https:\/\/e-dialog.group\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/e-dialog.group\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/e-dialog.group\/en\/#organization","name":"e-dialog","url":"https:\/\/e-dialog.group\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/e-dialog.group\/en\/#\/schema\/logo\/image\/","url":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/06\/e-dialog-logo.svg","contentUrl":"https:\/\/e-dialog.group\/wp-content\/uploads\/2025\/06\/e-dialog-logo.svg","width":1,"height":1,"caption":"e-dialog"},"image":{"@id":"https:\/\/e-dialog.group\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/edialog.group","https:\/\/www.linkedin.com\/company\/e-dialog-gmbh\/","https:\/\/www.instagram.com\/e_dialog\/","https:\/\/www.tiktok.com\/@e_dialog"],"description":"Bei e-dialog gestalteten wir gemeinsam die Zukunft des data-driven Marketings - mit einem gro\u00dfartigen Team aus passionierten Expert*innen.","email":"kontakt@e-dialog.group","telephone":"+43 1 309 09 09","legalName":"e-dialog GmbH"},{"@type":"Person","@id":"https:\/\/e-dialog.group\/en\/#\/schema\/person\/7b481e2ac90c21e4dc393821d35bb518","name":"digitalists","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/8acd1abfb5ef9f78793e3164334d4533968653b26cbd89b6cf0eb4e5483434e4?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/8acd1abfb5ef9f78793e3164334d4533968653b26cbd89b6cf0eb4e5483434e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8acd1abfb5ef9f78793e3164334d4533968653b26cbd89b6cf0eb4e5483434e4?s=96&d=mm&r=g","caption":"digitalists"},"sameAs":["https:\/\/e-dialog.group"],"url":"https:\/\/e-dialog.group\/en\/blog\/author\/digitalists\/"}]}},"_links":{"self":[{"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/posts\/12682","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/comments?post=12682"}],"version-history":[{"count":1,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/posts\/12682\/revisions"}],"predecessor-version":[{"id":13128,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/posts\/12682\/revisions\/13128"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/media\/4804"}],"wp:attachment":[{"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/media?parent=12682"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/categories?post=12682"},{"taxonomy":"channel","embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/channel?post=12682"},{"taxonomy":"goal","embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/goal?post=12682"},{"taxonomy":"technology","embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/technology?post=12682"},{"taxonomy":"c-year","embeddable":true,"href":"https:\/\/e-dialog.group\/en\/wp-json\/wp\/v2\/c-year?post=12682"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}