20X Faster Than . . . What?

Many times, in advertising and new product announcements, there can be some amazing claims about product speed, quality, efficiency, or any number of other metrics. "Our paper towel is 10 times stronger than the competition!" Or "our new cleaning product works five times faster than Brand C." While these claims may have some merit, it’s always difficult to find out the real story.?

In engineering software, the story is often the same. Some company releases a new solver, new mathematical method, or even some new compute acceleration algorithm. To make a big splash and garner attention, they make bold claims of “20X faster” or “5X more efficient workflow” For these comparisons, there needs to be some reference upon which these claims are made. Usually, engineers reading these boisterous claims of speed and efficiency are looking through the material for the ‘*’ denoting the caveats and testing conditions. Often a company will make a bold claim about how their new algorithm is 10X or 100X faster and won't tell the reader how that efficiency calculation was made. If it sounds too good to be true, it most likely is.?

There are, however, a couple of ways to make accurate and truthful claims of speed and efficiency. The company can take a well-published canonical problem, with documented solve times and simulation conditions to compare against. These unbiased references are extremely rare and are usually very simple engineering problems. Alternatively, they can reference a previous version of their own tool, such as, “the 2021 release is 2.5x faster than the 2020 release for calculating a specified set of problems.”?

At the end of the day, engineers just want to solve their design challenges as accurately as possible in as little time as possible. Many times, these fantastical claims of speed have little evidence of support. If you see wild claims of speed improvements for new products, it’s best to question the basis upon which those claims are made.?

Stay tuned for more of these conversations. I welcome any feedback you care to offer.