Elastic -- The Evolution of Open Source

In February of 2010, a developer called Shay Banon open sourced a piece of software called Elasticsearch. Shay was no novice to search engine software. After Doug Cutting released the Lucene search libraries (sort of like the “primitives” to a search server) to the Apache Foundation, there had been a few attempts to create an easy-to-use, distributed, and scalable search engine for any data. In fact, Shay had authored one of them called Compass. But he remained frustrated with the fact that there was still no scalable solution available anywhere in the world.

In order to focus on his project, Shay resigned from his lucrative job at Gigaspaces and spent nearly two years writing Elasticsearch. When he released it in 2010, the project took on a life of its own as developers started to flock to it to enhance, improve and build on it. They made it easier to ingest data in Elasticsearch, created a management console, and developed a graphic package that enabled analytics. The creativity and innovation of the open source community created significant momentum for this project and nurtured a rising movement. As of 2018, Elastic’s products, including Elasticsearch, had been downloaded 350 million times, a staggering number by any server software standards.

We encountered Shay and Elastic because several of the luminary CTOs in our portfolio companies wouldn’t stop talking about this ”miracle software” called Elasticsearch. Back in 2012, when we discussed industry trends with a number of our CTOs, they kept bringing up how amazing Elasticsearch was. The software was being used in a variety of applications including enterprise search, analytics and security. Even then its popularity was starkly apparent with 200,000 downloads per month - we had never seen anything like that in enterprise software. We reached out to the company through friends in the open source community and met with Steven Schuurman, an industry veteran who co-founded Elastic along with Shay. The pair had successfully launched the company, together with Uri Boness and Simon Willnauer, two search veterans and exceptional engineers. They were quickly recruiting the who’s who in the Lucene community to join Elastic and becoming the place to be for open source search.

Now, I’ll pause on the Elastic story to take quick detour and retrace the journey of open source companies. While today we’ve seen a lot of recognition for companies like Mulesoft, MongoDB, Hortonworks and Cloudera - who have all achieved multi-billion dollar valuations, just six years ago, in 2012, the conventional wisdom was that you couldn’t monetize open source companies. The common venture capital meme was, “How can you make money on free software?” Even Redhat, which was worth north of $10 billion at the time, was mostly cited as the exception that proves the rule. At Index, however, we believed that open source was a very interesting if contrarian opportunity for investment. As I wrotein a piece back in 2014, we felt that there was a sea-change underway in the world of open source software. The companies were becoming more “open core” by retaining value-added portions of the software in the commercial domain and by monetizing a subset of the customers who were really using the software in production. The development of the open source software itself was shifting from purely “community-based’ to largely company-led, which allowed for the control of roadmaps and rapid innovation cycles. Perhaps most importantly, users of key infrastructure software were shunning product from incumbent closed-source vendors like Oracle or SAP in favor of these new open source solutions.

Elastic was in the perfect place to capitalize on this industry shift. Elasticsearch, together with a couple of smaller but strategically critical software components - Logstash and Kibana - were quickly becoming the industry standard known as the ELK Stack, and today called the Elastic Stack. What proved to be so useful about the Elastic Stack was its enormous flexibility and ease of use. At its heart, Elasticsearch is a search engine, which means that it can read structured and unstructured documents (often referred to as “schema-free” in the industry lingo), index them (an index is like a map of where the content rests), find the right content, and return a search result. What makes Elasticsearch special is the fact that it’s distributed - it allows clusters to act like one big system; it’s multi-tenant - so many developers of parts of an organization can use it without interfering with each other; and it has a programmable web (HTTP) interface - so that developers can write programs that query Elasticsearch rather than just respond to typed searches. 

All of these things make the Elastic Stack very useful in a wide range of applications. Obviously, there is the basic search function — many of the websites and mobile apps you visit everyday use Elastic as their site and app search software — but, in addition to that, Elastic’s products are used in a range of other solutions: it’s commonly set up as a logging platform for IoT or weblogs; it’s used in security applications to identify anomalous behavior; and it’s used for analytics through its visualization engine - Kibana. The reality is that developers find new applications for Elastic all the time. In a sense, the team at Elastic has taken a core technology not dissimilar to that which powers Google and unleashed it on a massive range of data that Google never touches. You can begin to see why this is such an exciting company.

Shay would, of course, not only innovate on the product but also strive to build a special organization. Aside from the terrific culture and work environment that permeates Elastic, the company is truly global. Nearly 90% of its development team lives outside of Silicon Valley and it’s completely distributed throughout the world, and not just clustered in one or two other locations. This gives the company massive leverage to recruit the best and brightest talent from around the globe. The organization runs on “first principles” and tries not to rely on conventional wisdom, which is not surprising given that they have rejected ‘the tried and true’ path of startup convention every step of the way. To lead this amazing group of people, Elastic has brought together a very talented executive team. Kevin Kluge orchestrates their massively distributed engineering and product team that spans the globe. Aaron Katz provides his steady hand and prior experience at Salesforce to lead the go-to-market efforts of the company. Justin Hoffman - their first salesperson - leads global sales. Leah Sutton, who was also promoted through the ranks, leads the people function. Jeff Yoshimura - marketer extraordinaire - gave us his second tour of duty after his successes at Zuora. And, Janesh Moorjani, whom I’ve known since my Cisco days, completes this fantastic team as the CFO. And, of course, a big shoutout is owed to Steven Schuurman who fearlessly led this company until he handed off the reins to Shay two years ago.

As unbelievable as the Elastic journey has been - from its humble beginnings in Amsterdam, to one of the most respected young companies in enterprise software - we are still in the early days of this company’s potential. The ever-expanding volumes of data that are being created by companies around the world, compounded with the critical need to make sense of this information, will continue to drive more and more users to Elastic. No organization can be effective in the modern business world without wielding data as a competitive advantage, which is just what Elastic unlocks. For us at Index, we feel like the last six years with Elastic have happened in the blink of an eye. We are so incredibly proud of this unique day in Elastic’s story. An IPO is a very special moment; at the same time, we know that in many ways this is the first inning and that the best is yet to come. All of us at Index welcome Elastic to our family of publicly traded companies and wish all the Elasticians a hearty congratulations on this occasion -- we can’t wait for the next chapter.