Employee Spotlight #2: Skylar Payne, Machine Learning Lead at HealthRhythms

Want news on the hottest startups delivered to your inbox? Subscribe?to my mailing list, Founder to Founder (F2F):?f2f.substack.com.?Check out my latest F2F stories:?

• Startup Spotlight #180: Humming Homes
• Case Study: Telgorithm's Journey Towards Product-Market Fit Involves "Celebrating Your Successes And Losses"

Skylar Payne leads the machine learning team at HealthRhythms, where he applies his expertise in large-scale, personalized machine learning systems to important challenges in mental health care. Notably, he has taken machine learning products from initial concept to international launch at large companies like LinkedIn as well as developed and maintained several other data products which have hundreds of millions of users. He has led machine learning teams to double their productivity through key investments in infrastructure and process improvement. He holds 3 patents related to large-scale, personalized machine learning systems.

Payne graduated Summa Cum Laude with a B.S. in Computer Science from the University of California, Irvine.

HealthRhythms Machine Learning Lead Skylar Payne.

Frederick Daso: What was your journey to becoming a Machine Learning Lead at HealthRhythms? What were the prior positions you had that set you up for success in your current role?

Skylar Payne: I was hooked on the concept of machine learning after taking my first course. At first, I was interested in the topic from a theoretical lens — I was amazed at how we could use math to make a computer learn, and we had theorems to?prove?their effectiveness. As I waddled into the industry with a job at Google, I quickly learned there was much more to machine learning than “just the math.” I was at Google at an interesting time; Tensorflow had just been developed. There was a lot of work going into using it well in production (these efforts eventually culminated in a platform known as TFX).

In my role at Google, I didn’t get to do much of what I would consider “machine learning,” — so I joined some friends who had their company acquired by LinkedIn (Connectifier). At LinkedIn, I learned how to do machine learning “well.”

We were building a new product, basically a “Tinder for hiring.” I built the core of a real-time personalization algorithm that would personalize your results based on who you interacted with — making hiring much simpler for hiring managers (the product is now known as Recommended Matches).

Over time, I realized that the algorithm changes I made rarely made a splash. I would spend hours pouring over literature, writing out theorems, and trying to understand how to eek more performance out. But it turns out: that fixing bugs and incorporating new data sources always performed way better.

So I spent a lot of time understanding how to do that better. And I found that no one really knew at the time (~2016). There were some whispers of best practices, but we were still figuring all this stuff out as an industry.

I watched LinkedIn go through a few cycles of building machine learning infrastructure — lots of hard-learned lessons about what works and what doesn’t. Eventually, I found a knack for figuring out how to make machine learning teams more productive. I was able to > 2x the productivity of my own team (in terms of the number of successful model iterations we shipped).

I’ve always been interested in working in the mental health field for the past ~2 years. I’ve been applying my skillset in that space.

Daso: There’s a lot of conventional career advice about being a successful Machine Learning engineer, but are there any unorthodox lessons that you’ve learned through experience or mentorship that more of your fellow ML engineers should know?

Payne: Be like glue, not like water. Most people are like water. They go where there is the least friction. But often, there’s friction. This is especially true in the data/machine learning space: the explosion of tools lacking interoperability has broken the workflow for many folks. We have a bunch of lego pieces that don’t fit together. The friction is where there is a leverage opportunity: glue the broken pieces together (whether it’s systems, people, etc.), and you can make many people more productive. But be forewarned: it’s not glamorous work.

Always be instrumentin’: the hardest part of machine learning, in my experience, is that you often don’t notice problems until the very end — and it is super challenging to trace back to find root causes. One thing that can help a lot is just making sure you are always instrumenting. Double-check every assumption; there will always be a time when your data breaks your assumptions.

Daso: What’s the toughest project (professionally or personally) that you worked on as an ML engineer? What were the most important lessons you learned from that project?

Payne: Improving the productivity of my team. Doing machine learning well AND fast is hard. Part of what made this hard was there wasn’t clearly established best practices in the industry yet — so knowing what we “should” do wasn’t obvious. The problem seemed extremely multi-faceted, so knowing how to focus and prioritize even was difficult. The biggest lesson in this project was how to simplify a complex, multi-cause problem. We couldn’t fix everything at once — and the data we had did not clearly point to one cause being the biggest problem, so we had to combine our intuition and data to chop the problem down into chunks that could be addressed. I learned another interesting lesson: making an individual more productive can often make the team less productive.

Daso: Who are some of the most inspirational people you’ve gotten to work with during your career??

Payne: Here's a nice list:

Alex Patry: Alex was my tech lead for quite some time. One of the sharpest ML practitioners I know. I learned SO much from him.

Hossein Attar: Hossein led machine learning infra at LinkedIn, and I learned a lot about the qualities of good (and bad) infrastructure from him.

Erik Buchanan: Erik was my manager for most of my time at LinkedIn. I really learned how to think in a “product first” mindset from him.

Ben McCan: I really learned how to ship quickly rather than “correctly” from Ben. I always wanted to ship the “perfect” thing, but I learned how to really embrace an iterative mindset from Ben.

Suju Rajan: Suju was my manager after Erik. I learned a lot about managing cross-functional relationships from her. I had a...harsh edge in my communication and she helped me identify and fix it.

David Stein: I always found David to be very calming. I remember some colleagues and me giving David some unfairly harsh feedback on a system he and his team built. Still, David was able to sift through our mostly rambling complaining to find the valuable bits of feedback. David is extremely good at understanding and synthesizing different points of view.

Ben Le: Ben is another one of the sharpest ML practitioners I know. When I think of a lot of really compelling work pushing the state of LinkedIn’s machine learning systems — Ben seemed always to be involved.

Kevin Kao: Kevin Kao worked on a partner team and I don’t know how he has all the energy. He spun up so many projects to help my team and delivered EXCELLENT quality every time. I’m truly in awe of how good Kevin is.

Daniel Hewlett: it’s rare for me to meet someone who is simultaneously deeply scientific but also could stand in as a great engineer. Daniel led one of the most difficult transformations of modeling approaches. I always appreciated his attention to detail and process in making this transformation a reality.

I would work with any of these people again in a heartbeat.

Daso: How does your company’s culture create an environment where you can do your best work?

Payne: My company builds its culture very consciously. We have a “culture committee” that anyone can join to discuss initiatives to help drive the culture we’re seeking to build. A huge part of that culture is being feedback-driven. At Google, there was a sign in some of the bathrooms with a quote from Astro Teller (head of Google X): “No one is so high performing that they couldn’t use some really good feedback.” If you’re building something hard, you will make mistakes. Having a team of people watching my back to help me improve (and vice versa) makes us adaptable to the challenges ahead. Beyond that, I’ve enjoyed the cultural norm of ownership we have built; I have a lot of agency to do the things I believe are most important (and if I’m wrong — I can take solace knowing someone will give me some feedback about it).

Daso: What’s one interesting thing (non-work related) that more people should know about you?

Payne: I’m really committed to my fitness — getting stronger, more mobile, and more flexible every week at Allegiate in Santa Monica. If you’re in the Los Angeles area — check out Allegiate (locations in Santa Monica, Century City, and Redondo). Without any hyperbole, Allegiate changed my life for the better. It could change yours too!

Want news on the hottest startups delivered to your inbox? Subscribe?to my mailing list, Founder to Founder (F2F):?f2f.substack.com.?Check out my latest F2F stories:?

• Startup Spotlight #180: Humming Homes
• Case Study: Telgorithm's Journey Towards Product-Market Fit Involves "Celebrating Your Successes And Losses"

If you enjoyed this article, feel free to check out my other work on?LinkedIn. Follow me on Twitter?@fredsoda, on Medium?@fredsoda,?and on Instagram?@fred_soda.