What does SplitSignal measure? Sessions, CTR, or Rankings?

  • SplitSignal measures organic sessions from websites connected to Google Analytics / Search Console. For hired pilot users, we can also manually retrieve clickthrough rate data from the Google Search Console.
  • We will explore other metrics in the future. Do you have any suggestions for other information? Feel free to contact splitsignal@semrush.com.

Why are there minimum requirements to perform statistical search engine optimization? What are these minima?

Three things are needed to complete the test:


  • The website must have a strong concept of product and category pages.
  • The website must have more than 500,000 monthly organic sessions.

For testing:

  • Organic traffic – 100,000 organic sessions per month for the test group.
  • Templatized Pages – Over 300+ templatized product / category pages per test group. Examples of branched pages are product pages, category pages, blog / article pages, etc.
Examples of websites with templatized pages


  • These thresholds significantly increase the probability of a statistically significant result.
  • The more traffic and pages a website has, the better the chances of a statistically significant test. We may review these thresholds as more information becomes available.

How are Variant and Control groups identified?

SplitSignal uses cohort analysis to split test group pages without a data processing team. We leverage more than 100 days of historical traffic data to form two statistically similar page groups. Such sharing increases test accuracy and controls external factors such as seasonality, Google algorithm updates, and more.

How does SplitSignal know when Googlebot (Chrome Headless Browser) visits Variant group URLs?

While we can’t be sure when Google will crawl your changes, SplitSignal displays an indicator of when Googlebot visited each page based on the URL information in the Google Search Console. In our experience, Googlebot visits approximately 60% of Variant pages in week 1 and the remaining 40% of Variant pages in week 2. It may take a few days for the changes to appear in Google SERPs after the page is visited. This all depends entirely on how much organic traffic your site receives each month.

The example below shows that all 460 pages in the conversion group have been visited (100%):

Screenshot of SplitSignal: “Pages Visited by Googlebot”

Option group change management: Client-Side vs. Server-Side

There are two types of SEO statistical tests.

Customer side testing occurs when the Javascript code changes to the Variant group after the page has been loaded into the browser (“client”). The code is stored in the CDN and is triggered each time a visitor loads the page. SplitSignal uses client-side change management.

Server-side SEO testing works by making changes to Variant pages directly on the web server. Instead of downloading the original version and updating after a few milliseconds, the server-side changes are fixed. They are launched before the page is delivered to the client (i.e. the browser).

Is there flickering?

What is flicker? In this case, users will see the Control version of the page in a web browser before it quickly changes to the Variant version.

We have not detected any significant flicker in the more than 100+ tests performed so far. SplitSignal allows you to preview changes made before starting the test so that the user can see the effect of the interface / interface before starting. Although our main stakeholder in these tests is Googlebot, the user experience is still very important to us.

What types of elements can SplitSignal test?

  • Header recognition tests
  • Meta tag tests
  • Header test (example: Change header from h1 to h2)
  • Move elements (e.g.

    transfer before or content transfer)

  • Remove elements (e.g. or delete content)
  • Add elements (e.g. Add new


We are constantly looking for alternatives to new test types.

Can we run multiple tests at once?

Users can run as many tests as your website allows, as long as each test meets the minimum traffic and page limits. It is important to note that once a page is selected for a test, it cannot be selected for a new test while it is still running in the first test. You should wait until the first test is complete or stopped before you can use that page again; We recommend waiting about 7-10 days for you to continue on these same pages. We call this the “cooling period.”

How does SplitSignal affect Core Web Vitals?

We take Core Web Vitials very seriously. The advantage of customer-side shared testing is that it has little impact on Core Web Vitals products. We have conducted extensive performance tests with our script.

It’s important to consider the impact of, among other things, third-party code and JavaScript execution time, as it can affect site performance. We have not yet seen any noticeable effect that the SplitSignal JS snippet has had on performance metrics. We’ve tested websites from 300,000 monthly organic session sites to 50 million monthly organic session. Before completing the test, we encourage developers / design teams to conduct a performance test in the form of a 2-day concept certificate through either a step-by-step development / test or a live / prod site.

For more information, contact the SplitSignal team at splitsignal@semrush.com

Can we preview what the test would look like before it is released?

Yes, the SplitSignal preview feature allows the user to see what the change looks like to ensure that not only Googlebot sees the change but also the user. Our main stakeholder is Googlebot, but we also value the user experience highly and don’t want it to be negatively impacted.

SplitSignal interface: “Test preview”

What kind of information does SplitSignal have access to?

SplitSignal analyzes request access logs with a SplitSignal JS code snippet to improve the quality of SplitSignal and for use in certain tool features (e.g. tracking Googlebot visits). The lifetime of the log data is 45 days from the original request. After 45 days, the logs will be deleted and SplitSignal will no longer have access to them.

  • SplitSignal or Semrush does not share information related to SEO split testing with third parties or other Semrush tools.
  • SplitSignal does not set cookies in the client’s browsers.
  • SplitSignal uses GA / GSC data only to share pages before the SEO split test.

NOTE: Users can revoke access / connection to GA / GSC at any time.

For more information, see the official privacy policy: Privacy Policy Semrush

Can I try SplitSignal?

From 1 July 2021 to 1 September 2021, we are offering qualified users the free use of SplitSignal to complete up to ten tests as a pilot / concept certificate.

We have two pilot options:

  • Done For You – a white glove service where the SplitSignal team prepares, launches and analyzes your first ten tests (price: $ 5,000 / time)
  • Done by Yourself – users are expected to create their own tests, receive these tests from the SplitSignal team, and then launch the tests within 30 days. (price: $ 0)

Learn more about SplitSignal and apply for a pilot / concept certificate here.


Please enter your comment!
Please enter your name here