LightStep, And The Future of Application Management

There’s a saying, “the future is already here, it’s just not evenly distributed.” It means: “early adopters are doing today what everyone will do tomorrow.” The press often points to examples like Tesla’s autopilot, Dutch farming, automated factories, or how teenagers use social media. But the same is true for enterprise computing. In that case, the early adopters are developer-driven, cloud-native companies that have achieved scale within the past five years, such as GitHub, Lyft, and Twilio. Ask them how they build and manage applications and you get a glimpse of the future: they are all building microservices, and managing them with LightStep. 

To understand why, let’s start with one of the biggest trends in technology. Software has become the primary way that companies interact with their customers. For example, the COO of a major retail bank told me this week that, on average, their customers use the bank’s mobile app 272 times a year, but only visit a branch 6 times. Think of your own work, and how often you contact customers using email or apps, instead of phone calls or faxes as happened years ago. 

That makes software much more important than it used to be: a company’s software now directly impacts customer experience and revenue, not just costs. So the speed at which companies can improve their software, and respond to changing customer needs, is a major source of competitive advantage. More and more large companies understand this, which is why they are changing how they build and manage applications. 

The Move To Microservices

The great enabler of more flexible applications is Docker, the open source software container that makes it easy to fit application components together and move them around. Previously, large numbers of developers would all work on a single code base. They would create large, complex applications (now often referred to as “monoliths”) that were hard to change, because everything was interdependent. 

Today, companies have smaller, nimbler teams building separate components which run in containers. These components (for example, Walmart's shopping cart or Wells Fargo's balance checker) are loosely coupled, meaning each one can easily be changed without impacting any others. Each of these is a “microservice”, which together make up an application. In Gartner's recent survey of 711 companies, 53% of them are using (22%) or evaluating (31%) container deployments for microservices.

The Need For LightStep

This new, more flexible way of building applications makes them harder to manage. Before, in a single application, it was easy to see how the pieces fit together. Now, when someone clicks “Buy Now” in their shopping cart, that single transaction could pass through dozens – if not hundreds – of loosely-coupled system components. So if the site starts hanging when someone tries to buy, how do you figure out what’s wrong? 

That’s where LightStep comes in. It’s what enables developers to understand the customer experience, by tracing how customer transactions move from component to component. Using LightStep, developers draw connections between what a customer sees and what the application is doing. As a result, engineering teams can now respond in real time to customer needs.

Some of this is reactive: if the Lyft app is updating wait times too slowly, then engineers can use LightStep to isolate the problem and fix it, saving lost revenue because customers don't get frustrated and switch to Uber. Some uses are proactive: LightStep can track Twilio’s performance for key customers and immediately send alerts if the system falls below its SLA (Service Level Agreement). Either way, LightStep abstracts away the complexity, so that developers can manage applications to increase customer satisfaction and revenue. 

Sequoia Partnership

Today, the company is announcing its product and funding. All of us at Sequoia are thrilled to partner with the LightStep team, and our friends at Redpoint, Cowboy Ventures and Harrison Metal.

We met the founders (Ben S., Spoons, and Ben C.) years ago as they were leaving Google, where they built massively scalable internal systems for tracing (Dapper) and monitoring (Monarch) which are still in use today. Aside from their intellect, love of biking, and self-deprecating humor, we were immediately struck by two aspects of their vision. One is that LightStep captures all the data flowing between system components, rather than just sampling. As Stripe later put it to us, "they are solving this problem the right way." The other is that the founders don't talk about speeds and feeds. Their focus is on telling stories from the data, so engineers and ops teams can act more quickly. 

It's now clear that every company is becoming a software company, and moving towards microservices and platforms like LightStep. Today, both are "unevenly distributed", and used mainly by early adopters; tomorrow, they will be mainstream best practices. As that happens, it's the companies who move first that are the most likely to win out.