Scala on the rise

There appears to be an ever-growing trend of companies either using or looking to switch to Scala as their chosen programming language. Just take a look at the list below:

Twitter, LinkedIn, Netflix, Nike, Sony, Starbucks, Apple, Amazon, PayPal, Bloomberg, eBay, TomTom, IBM, NASA, Sky, Cisco, HMRC and HSBC.

Why is this though?

Scala is an acronym for “Scalable Language”. Meaning that Scala grows with you. It is used for large mission critical systems in companies such as those mentioned earlier.

Scala is the preferred workhorse language for many mission critical server systems. The generated code is on a par with Java’s and its precise typing means that many problems are caught at compile-time rather than after deployment.

Scala combines objective-oriented programming (OOP) and functional programming (FP). Its goal is to essentially say OOP and FP don’t have to be these separate worlds. It’s a language with a notion, rather than trying to appeal to every programmer out there. Scala is trying to solve a specific intellectual problem.

Why do Twitter use it?

As Twitter grew, it needed a new language to manage its long-running heavy processes, message-queuing and caching layers for doing 20,000 operations a second.

They wanted something that was fast and fun, good for long-running process and could be productive quickly.

Here is an example of some of the Scala based services used at Twitter:

Hawkwind for People Search

Twitter’s people-search is powered by a Scala-built service called Hawkwind. It’s a bunch of user objects dumped out by Hadoop, where the request is fanned out to multiple machines and then pulled back together.

Flock to Store the Social Graph

Twitter uses Flock to store their social graph, as a denormalized list of user ids. Twitter performs over 20,000 operations a second right now, backed by a MySQL schema designed to keep as much as possible in memory. This have proved very efficient for they and not many servers are needed.

Thrift for Transferring Data

Twitter also use Thrift; it was built at Facebook then open-sourced at Apache. With Thrift, it allows them to define data structures and methods, and it deals with everything they don’t want to deal with to efficiently represent data and get it from point A to point B. As the system evolves, the method signatures change, and Thrift has a nice system for creating positional arguments and being backwards compatible.

Hosebird for Streaming

Twitter streams out tweets to public search engines, using a low-latency, HTTP-based, persistent connection system called Hosebird. This was built on top of Jetty using Scala to handle to volume of the load. The idea was developed from This was developed from looking at queuing systems that financial-services companies used.

I'd love to hear your thoughts and feelings about this, so feel free to comment below.

With Scala being the possible future of programming, if your company is interested in hiring top Scala talent, or if you are ‘that’ talent, feel free to get in touch with me.