How is the definition of "close" in googles "close variant" changing over time?

I already discussed in the past that there is quite a lot of movement when it comes to "close variant" matchtings in google ads: https://www.dhirubhai.net/posts/stefan-neefischer_ppc-googleads-closevariants-activity-6680580401326522368-mpBD/

I saw for different accounts and different markets that the share of close variants is increasing over time - and it also seemed that the additional traffic coming from close variants is not converting that well. Time to crunch some data...

Using string comparisons of Keyword vs. Query to give "close" a number

My thesis was that "close" can be very different within. You will find good matches but also some very poor ones. For that reason I used a string comparison function that tells me how good the match was between the keyword and the query. Maybe you have heard about levenshtein distance before for making string comparisons - I used something similar (I do not want to go into too much detail). In the end we have a matching value between 0 (bad) and 100 (perfect). I did this with around 100.000 keyword/query pairs to ensure we have enough data for the follow analysis.

Exact (close variant) is getting more fuzzy over time

The first thing I was interested in was the average matching score. Until October 2019 it was quite stable at 85 - then we have changes to a new level of 80 and when we look at May/June 2020 we reached a value below 75. Quite interesting!

The increase of the close variant share is mainly driven by matches becoming more fuzzy (poor matches)

How can google grow in ad revenues when the user demand of that market is quite flat? If the demand stays the same you can generate click/impressions growth by changing the way you match queries to keywords. And I think this is exactly happening here: the biggest growth of close variant matches takes place in the area of poor matches.

Poor Matching = Bad Performance

Let's group the performance KPIs by my new Matching Score. I grouped them into 4 different buckets to have > 10.000 Clicks per bucket as observations. When I was looking at the chart it is obvious why the additional traffic seems not convert that good:

The Conversion Rate of the "good matches" is more that 3 times higher than the "poor matches". Again: we are looking at the same exact match keywords that perform completely different depending on the queries they trigger! Sneaky.

How to get back control?

Still not sure how to solve it. One approach is described here that is using Bounce Rates on n-Gram level generated from search queries to block noise in the earliest stage:

I'm now thinking of an additional approach that is using the matching score to cut off the poor matchings in an automated way.

What is your solution on that problem? Have you realized a development that the performance of your exact match keywords decreased over the last months?