The Fall of MTA and MMM: Enter AI-Powered Visit Scoring with Incrementality Calibration

Every time I see people discussing MTA and MMM, or even Incrementality Tests, I feel like they’re stuck in the past, trying to solve modern problems with outdated tools. Yes, you can design all kinds of wheels and engines for a car, but it won’t make it fly. A new invention comes to replace old things and solve new challenges.

SegmentStream is one of the pioneers driving innovation in incrementality measurement within today’s complex marketing ecosystem. After much thought, I’ve finally decided to write a comprehensive article on how we approach the incrementality measurement problem at SegmentStream.

No "whoo-whoo," no "black boxes," no "magic." No more empty buzzwords like "Clicks and Impressions MTA" or "Modern MMM" that people throw around without real explanation.

Consider this like when Elon Musk shared Tesla’s patents with the public, even competitors, to push the entire market forward. Today, I’m doing the same—laying it all out so the industry can evolve.

Let's begin

SegmentStream has introduced a breakthrough solution by using AI-powered visit scoring and incrementality-based calibration to provide brands with a more accurate and visible way of measuring the true effectiveness of their campaigns.

In this article, we’ll dive into how SegmentStream’s approach works, why it surpasses traditional methods, and how it helps marketers measure cross-device, cross-browser, and upper-funnel behavior more effectively. Along the way, we’ll explore specific examples and explain why calibration based on incrementality tests is key to success.

The Problem with ANY Cookie-Based Attribution

Traditional attribution models have long relied on cookies to track user behavior and assign credit to marketing touchpoints. However, these models suffer from several limitations:

1. Binary Results: Cookie-based models work on a black-and-white basis. A visit either results in a conversion or it doesn’t, with non-converting visits receiving zero credit. This simplistic view fails to capture the true complexity of the customer journey, where multiple touchpoints across different channels contribute to eventual conversions. Yes, you might argue that Multi-Touch Attribution (MTA) solves this problem by distributing credit across various touchpoints, but MTA still only assigns credit within the same browser and cookie. This means that 99% of upper-funnel interactions—such as clicks from different devices, browsers, or sessions where cookies have expired—still receive zero credit. This leaves a huge gap in understanding how these interactions influence the overall conversion path.
2. Cross-Device, Cross-Browser Challenges: Today’s users interact with brands across multiple devices—phones, tablets, desktops—and through various browsers like Safari, Chrome, and others. Cookie-based attribution models struggle to connect these touchpoints, leading to fragmented data and an incomplete view of marketing effectiveness. Yes, some MTA providers tout their "identity graph" technology as a solution, but let’s be honest: identity graphs rely on cross-device stitching through Personally Identifiable Information (PII) and user authentication. Now, how many of your upper-funnel campaign visitors actually leave their PII or authenticate on your website? Identity graph technology sounds promising, but that’s essentially all it is—just cool-sounding. In reality, it might provide a marginal 1% improvement at best, but it certainly won’t solve your cross-device attribution challenge in a meaningful way.
3. In-App Browsers: Many mobile users interact with brand content via in-app browsers like those found on Instagram, Facebook, and TikTok. These interactions are often missed by traditional cookie tracking, making it difficult to attribute credit for conversions coming from these platforms.
4. Attribution Blind Spots: A small fraction of visits, typically less than 1%, result in conversions. This leaves a vast majority of traffic unaccounted for in cookie-based models. As a result, marketers are left with little insight into the actual value of most of their traffic.

Why SegmentStream’s Visit Scoring is More Correlated with Actual Incrementality

Unlike traditional models, SegmentStream’s AI-powered visit scoring evaluates every single interaction, assigning each visit a score based on the likelihood that it will lead to a conversion. Each visit receives a credit from 0 to 1 depending on the engagement and intent shown by the user.

For example:

• A visit where the user displays shallow behavior, such as bouncing off the site quickly, might receive a score of 0.
• A user who spends time on several product pages and shows more interest in the product might get a score of 0.25.
• A highly engaged user who spends a lot of time browsing website, researching products, or even adds products to the cart but, for some reason, doesn’t convert might get a score as high as 0.9.

This model looks beyond just whether a conversion happened and instead evaluates the actual behavior on the website—how engaged the user is during the visit, whether the visitor shows genuine interest in the products you provide, and whether they put effort into research. These behaviors signal that the user is well-targeted and relevant, potentially sitting in the upper- or mid-funnel stages of the customer journey.

Detailed Examples of How Visit Scoring Works

For example, let’s say a user interacts with a brand across three devices: they browse on their smartphone, research further on their tablet, and finally convert on their desktop. Even the most sophisticated Multi-Touch Attribution (MTA) will credit only the final desktop interaction because it is still based on cookies. With visit scoring, each visit across the devices receives a score based on its role in the journey, giving marketers a more complete picture of how these visits work together to drive conversions.

Another scenario might involve two non-converting visits. In cookie-based models, both visits receive a zero score, ignoring the fact that they may have shown significant potential. One user might have spent time browsing multiple pages and adding products to their cart, while another left after only a few seconds. SegmentStream’s visit scoring can distinguish between these two cases, giving the high-intent visit a higher score based on its future conversion potential, even if no purchase happened during that session.

Cross-Device and Cross-Browser Attribution

SegmentStream’s visit scoring ensures that interactions across different devices and browsers are properly evaluated. For example, a user might browse a brand’s product catalog on their phone while commuting, and finally complete the purchase later at home on a desktop. SegmentStream doesn’t lose sight of the earlier interactions on different devices and browsers, providing credit to each touchpoint based on its contribution to the overall customer journey.

Increased Visibility: Measuring the Impact of Every Visit

One of the most significant advantages of visit scoring is the visibility it brings to marketing attribution. In traditional models, only a tiny fraction of visits—usually less than 1%—that lead directly to conversions are counted. The vast majority of visits that don’t convert are completely ignored, leaving marketers with a blind spot about the effectiveness of their campaigns.

With SegmentStream’s visit scoring, almost every single visit is assigned a score based on its incremental impact. This ensures that even non-converting visits are properly evaluated and credited. For example, if a user engages with several pages on a website but doesn’t immediately make a purchase, their visit still receives a score that reflects the likelihood of future conversion. This increased visibility gives marketers the ability to understand the value of each interaction, regardless of whether it leads directly to a sale.

Below is a short video that visually explains how AI-powered visit scoring works

The Importance of Calibration Based on Incrementality Tests

One might say, "How can we trust that visit scoring based on website behavior is correlated with incrementality?" And this is a fair statement. This is where incrementality test calibration comes into play. Visit scoring wouldn’t be as powerful without proper calibration, and incrementality testing helps fine-tune the scoring model to ensure it accurately reflects the true impact of marketing efforts.

One common method of testing incrementality is through geo-holdout tests, which are designed to compare the performance of users exposed to a marketing campaign against those in regions where the campaign has been paused or withheld. This comparison allows marketers to see how much incremental value the campaign truly delivers, providing the calibration necessary to ensure visit scores are accurate.

What Are Geo-Holdout Tests?

Geo-holdout tests are a type of controlled experiment where marketing activities (e.g., ads, email campaigns, or promotions) are turned off in certain geographical regions (holdout regions), while continuing in other regions (test regions). By analyzing the difference in conversions between these two regions, SegmentStream can isolate the true incremental impact of the campaign.

A Real-World Example of a Geo-Holdout Test

Let’s say you’re running a digital advertising campaign for a DTC brand. To measure the incremental value of your Facebook Ads, you decide to conduct a geo-holdout test:

• Control Region: In New York, you continue running your Facebook ads as normal.
• Holdout Region: In Boston, you pause all Facebook ads for the duration of the test (or just exclude Boston from targeting).

For several weeks, SegmentStream tracks and compares user behavior in both regions. This includes measuring key performance indicators such as website visits, product page views, and more importantly - conversions (sales). By comparing the data, SegmentStream can determine how much of the conversion activity in Boston was a direct result of the Facebook Ads, while New York serves as a baseline where no ads were running.

For example, if conversions in Boston decreased by 15% during the test period, while New York remained flat, SegmentStream can confidently attribute this 15% drop in Boston to the absence of Facebook Ads. This reflects the true incremental lift—the value added by Facebook Ads.

Applying the Calibration Coefficient

The calibration coefficient is then applied to the visit scores to adjust them based on the true incremental effect of the marketing campaign. This step ensures that the visit scores are not just theoretical predictions but are grounded in real-world performance.

• For example, based on visit scoring, the total contribution of Facebook Ads might result in 300 conversions. However, an incrementality test might reveal that the true incremental impact of Facebook Ads during the same period is actually 360 conversions. In this case, a calibration coefficient of 1.2 would be applied to all visit scores, adjusting them upward to reflect the true incremental value.
• On the other hand, for Brand Search, visit scoring might show 500 conversions, but the incrementality test could reveal that the true incremental impact is only 400 conversions. In this case, a calibration coefficient of 0.8 would be applied to the visit scores, scaling them down to reflect the lower actual contribution of Brand Search. This ensures that the visit scores are not overestimated, giving a more accurate picture of each channel's real impact.

Why Is This Important? Without this calibration, visit scores might either overestimate or underestimate the true value of marketing efforts. By applying the coefficient from the geo-holdout test, SegmentStream ensures that the visit scoring model remains accurate and aligned with real-world data.

Why Calibration Ensures Accurate Attribution

By applying a coefficient based on incrementality tests, SegmentStream ensures that every visit score reflects the actual contribution of the marketing campaign. This calibration process is crucial for maintaining the accuracy of the visit scoring model, allowing marketers to:

• Allocate budget more effectively: If the test shows that one channel (like Facebook) delivers a higher incremental lift than another (like Paid Search), marketers can confidently reallocate their budget to maximize returns.
• Optimize campaigns with precision: By continuously calibrating visit scores with real-world data, marketers can make more informed decisions about which channels, campaigns, or user segments to focus on.

In essence, geo-holdout tests are more than just a measurement tool—they are the key to ensuring that SegmentStream’s AI-powered visit scoring model remains accurate, actionable, and reflective of the true incremental value that marketing campaigns deliver.

Why Cookie-Based MTA Calibration Fails

Some might wonder, “Why not just apply an incrementality coefficient to cookie-based attribution models, like last-click?”

The issue is that multiplying any coefficient by zero still results in zero. Traditional models treat non-converting visits as zeros, so no matter how much you calibrate, the result will still be zero for those visits. This approach also breaks down for upper-funnel campaigns that don’t lead to direct conversions, resulting in inaccurate data at a granular level.

By contrast, visit scoring evaluates every visit, creating a wealth of signals that can be used to generate statistically significant data. This provides a far more robust foundation for applying incrementality coefficients, leading to more accurate insights and a clearer understanding of how each channel and campaign contributes to overall success.

Conclusion

In today’s increasingly complex digital landscape, where customer journeys span multiple devices, browsers, and touchpoints, SegmentStream’s AI-powered visit scoring combined with incrementality-based calibration provides a more transparent, precise, and actionable approach to marketing attribution. By analyzing every visit—whether on mobile, desktop, or within in-app browsers—and accurately scoring each interaction, SegmentStream gives marketers a comprehensive view of how each marketing effort truly contributes to conversions.

This innovative approach not only uncovers the true incremental impact of campaigns but empowers marketers to make data-driven decisions, optimize their marketing budgets, and achieve better overall performance. Traditional cookie-based models can no longer keep pace with today’s multi-channel, multi-device environment. SegmentStream is leading the evolution toward a more reliable and effective attribution model that addresses the complexities of modern marketing.

P.S. As usual, if you find this newsletter useful, please like and share it. Your support helps keep this newsletter alive and continue delivering value to our community!