On the cost of deploying AI: Tesla's latest software upgrade and some reality checks

Today, my Tesla Model 3 updated its firmware (seamlessly, as always, and to Tesla's credit). On the dashboard, it said, "updated to V12.3.6 of the Full Supervised Driving (FSD) mode using end-to-end neural network trained on millions of video clips enabling replacement of 300k line of explicit C++ code".

Note: I got to test the 8K$ FSD feature to play with for free - probably because my lease is about to expire, and that seems like either a nice "goody" to continue attract customers or I'm being used as a guinea pig to help improve the feature without even me knowing

Now, as a technologist, I felt that the "neural net vs c++" statement was bold yet in line with Silicon Valley's general thinking that AI will surpass everything, and I continue to believe that such a mind state needs to be toned down. I always like to say that AI is a useful "tool" but, in its current form, certainly not "the holy grail".

Let's first analyse the claim from an engineering cost perspective:

Assuming a very "good" programmer produces ~50 lines of "bug-free" C++ per day with an average pay of $150000/year (a fully-loaded salary in California is probably 30% more), and assuming 1/2 of "effective" code generation time only (removing vacation, project overheard, etc...) - it would mean that the cost per line is around 150K/(50*365/2)=~17$ meaning that for 300K lines of C++, Tesla would have spend 5M$ which, IMO, isn't too bad (excluding design, testing, admin overhead, etc...). Because people are always optimistic about the true cost of software, let's say those 300K lines are closer to representing a 50M$ (x10) investment.

Now let's contrast this with the proclaimed neural-net alternative, where I see 4 main questions (1) is FSD safer than a human driver, and is the use of neural-net good news in terms of safety compared to C++ (2) does it scale/is it an engineering technique that can scale across regions and vehicle types (San Francisco is a "well structured / well behaved" environment but does it even work in messy round-about traffic in the middle of Bangalore? (3) how much did Tesla spend creating, storing and involving humans to review and annotate "millions of videos" (4) Are end-users even willing to trust and pay for FSD (or indirectly, is the feature a significant- enough differentiator that Tesla is able to sell more Teslas)? My personal opinions on the 4 questions are as follows:

1. "FSD is better than human": maybe. I must say that the "simple" use case of self-driving highway automation seems well understood and technologically mature, but dense urban traffic remains a challenge for the software "autonomy" stack (perception, control, mapping, etc...). The question is always: would an average driver have performed better in complex situations? It's often hard to quantify/answer.
2. Scalability/safety: in my opinion, the current state of the technology is not scalable despite all the investments already made (in simulation, mapping, cross-city deployment, etc...). In part, scaling means an ability to generalize, which has a lot of liability implications in a heavy safety-regulated industry. Many experts are saying that, the nature of neural net technologies makes it difficult to understand why they sometimes fail compared to traditional C++ coding techniques (it can be a lack of training data, lack of precision, lack of model performance, etc...). This is often referred to as "lack of explainability" even though, with massive amounts of curated training data, neural nets can show impressive results
3. Cost of switching from standard programming techniques (e.g., C++) to neural nets: the "true" cost of switching to a neural approach is particularly interesting, and I'll return to that point.
4. Who cares and who is willing to pay?: I am personally willing to pay for high-way automation (given my commute profile or the level of trust I have in the car in this context), but even as a technologist, I am not yet ready to pay $8K to see the FSD software drive me around in a city

Neural-net versus C++ costs: a few guesses

Tesla spent at least 10X more to replace the 300K lines of code of the original software, and here is why.

Let's say 10 million 60min videos were needed (average commute duration is 60min per day in the US) and that it costs ~5$/hour to manually review/confirm the annotation (people use semi-automated annotation tools, and offshore wages can be as low as $3-$5 an hour for this type of tasks). Then you need to add "$c1" to cover for IT storage and video-database access cost, "c2" for the GPU IT training infrastructure and "c3" energy cost to run the GPU farm (human brain functions at 20Wh whereas a "server-class" GPU is more in the ~3500Wh range).

• for c1, let's assume: ~2GB for 60min of compressed video (for 1 camera) @ 250$ per 1TB is ~5M$/year
• for c2, let's assume 10000 NVIDIA H100 GPUs (the infrastructure is probably much bigger than that today) = ~500M$ (one-time capital gain investment, to be fair)
• for c3, the energy bill is 3500 kWh*10000*0.30$Wh in California=~10M$ (assumes 100% utilization)

So all-in-all, Tesla probably spent a minimum of 5M*10M+5M+500M+10M or 550M$ to replace 300k lines of C++ code at 50M$ - or ~10x the original software cost.

I'm unsure if anyone can claim that this investment made cars safer, behaved better than the original code, or delivered a better driving experience than a human driver.

Conclusion

One thing that's for sure is that if people are willing to pay for the FSD feature, then Tesla only needs to secure 550M$/$8000=62K customers, so perhaps worth the bet :) Now if the new code perform as well as the original C++ code, than the value of the experiment comes from the belief that the switch to neural-net will help scale the platform capability over time (more use cases, regions of the world covered) which isn't clear given that the cost of training remains (it's true that it generally diminishes over time) but let's also remember that the $10M$ in energy cost represent the wage of ~50 software engineers that could have been assigned to continue improving the C++ code...

#Tesla #AI #FSD #C++ #trueengineeringcost

As always, feel free to contact me @ [email protected] if you have comments or questions about this article (I am open to providing consulting services).

More at www.lohier.com and also my book . You can subscribe to this free bi-weekly newsletter here and access former editions here .