Tracking All the Things (without the?fuss)

I built a robust tracking system using Apple Shortcuts, AI, and Google Sheets. The process was surprisingly simple.

Why track?

The gap between my impressions of my behaviors and my actual behaviors is too big. For example, Apple Health informed me that my average amount of sleep in 2024 was way lower than I thought. I consider myself an “early to bed, early to rise” kind of guy. The data tells me I’m only the latter.

After ~5 years of not tracking a few key things (e.g. spending, nutrition), and largely ignoring the reports from the things that are already being tracked for me by my watch (e.g. sleep, exercise), I committed to track several daily behaviors for at least a few months.

Requirements

I once meticulously tracked every thing I ate for two months as part of a contest at work. I hated it. If I’m going to have any chance of tracking several things for several months, I need a system that works for me:

• Super low friction. Unlocking a car with an app is cool, but not as cool as the car unlocking automatically when I touch the handle and it senses the key nearby. Indeed, the best interface would be no interface. If something can be tracked automatically, great. If not- I’m okay with more interaction than the purely-passive tracking of ye olde mint.com. I just don’t want something as involved as YNAB.
• No new apps. No downloads, updates, logins, ads, accounts. I don’t want something that might be acquired, shut-down, renamed, or need to be unlocked, upgraded, or agreed to.
• Flexible inputs. Most of the time I want to say “Hey Siri…” and just tell it stuff to track. But sometimes I will want to type or copy/paste. Ideally I could scan the barcode of a food item. I don’t want to adhere to a strict format. I should be able to enter one word for a single item of food, or a whole paragraph with several different activities listed together.
• All in one place. Some of the things I want to track are not only already being tracked, but I can view beautiful reports. However, I want the data all in one place, and I want a single point of entry for it to get there.

Solution v0.1-beta

Here’s the architectural overview:

• Google Sheet for the storage.?—?free, stable, flexible for future data analysis, and it integrates with Google Apps Script. I’ve been using this power couple for over 10 years.
• LLMs for the flexibility. After a little bit of prompt tuning, Claude and Gemini both do a good job at parsing out the various things I want to track from a block of text, typos and all.
• iOS Shortcuts for the inputs. SMS-based tracking (e.g. Twilio) was great a decade ago, but modern regulations make it impractical. Shortcuts are native to iOS and Mac, require no third-party service, and provide powerful building blocks for handling voice input, prompting for additional data, and calling external services. Using Toolbox Pro as a helper app*, it can also pull data directly from Apple Health. *yes this had to be downloaded and purchased, but it’s been around for years and had a one-time charge of $5.
• Nutritionix with the assist. I didn’t trust some LLM hallucinations I was seeing regarding nutrition macros on food entries. So for those, I instruct the LLM to make a call to the Nutritionix API (free for personal use). They have a natural-language endpoint, as well as a barcode endpoint. Once I discovered the built-in “scan a barcode” module in iOS Shortcuts it was about 10 minutes before I had a legit MyFitnessPal replacement.

So I’m now tracking these things automatically:

• Steps
• Exercise
• Sleep
• Body weight (my scale sends this to Apple Health)
• Most spending: amount, vendor, category, description (Capital One and Chase send me text alerts, which trigger the Shortcut to run with the content of the text)

and these things “manually”, by speaking/typing/scanning into my Shortcut

• Nutrition macros: calories, protein, carbs, fats, fiber
• Meals: who in our family cooked, when, and an optional note about it e.g. “would make again!”)
• Social: who, what activity, and optional notes
• Weightlifting: exercise, weight, sets, reps
• Miles driven: just the trips I take for work that can be expensed
• Work: hours, project, client
• Other Spending: (if not via credit card)

Here’s a 90-second screencast of a few examples in action:

LLMs for using LLMs

To me, this really is the most exciting part of this project. It’s incredible how quickly the first working version of the code came together. Not only does the LLM play a part in every entry I make, but it helped me write the code. Yes I could have written it all myself, but the first pass it wrote for me saved me hours. Even at the start of the project, I asked it for different options to create this system, talked through the pros and cons of each, and kept going back for guidance. Amazing.

For the UPC code scanning function, I noticed that clearance price stickers (huge fan!) have a different format, and it wasn’t working with Nutritionix. So I asked Claude to explain. After educating me on how UPC codes are generally constructed, it broke down for me how to translate a sale code into the correct format:

It then proceeded to write a function in Google Apps Script to handle this for me. ??

Phase #2: Reporting

I’ve had this running for about 2 weeks now. So far the only “dashboard” I’ve created is this simple pivot table in the Sheet that I view several times a day to see where I’m at with calories and protein.?

In a couple months, I’m going to tinker with using the LLM to help me query and create reports, and try using iOS/Mac widgets to show me the most timely info.?

Until then,