A new paradigm for machine learning: probabilistic programming

Probabilistic programming is a new programming paradigm for managing large and complex sets of uncertain information. Machine learning is defined as the ability of computers to understand data and manage results and insights from uncertain information.

Machine learning has been the driving force behind several recent revolutions in computing. Many features like email spam filters, chat bots in smart phones and self-driving vehicles are all based on research advances in machine learning. But, as the demand for these capabilities is accelerating, machine learning experts are required to put a lot of effort and time in building these applications as tools currently available to them are often slow and can perform unpredictably against large, complex data sets. Probabilistic Programming for Machine learning addresses these challenges. Using probabilistic programming languages greatly helps in increasing the number of people who can successfully build machine learning applications.

Probabilistic programming has five tactical objectives:

• Shortening machine learning code to make models faster to write and easier to understand
• Reducing development time and cost to encourage experimentation
• Facilitating construction of sophisticated tools that can incorporate rich domain knowledge and separate queries from underlying code
• Reducing the level of expertise necessary to build machine learning applications
• Supporting the construction of integrated models across a wide range of domains.

At this point, there are several probabilistic programming languages, at varying levels of maturity with different groups experimenting with widely divergent approaches. Probabilistic programming languages can be classified based on following two features:

Expressivity of Probabilistic Programming

Determine how expressive the language is. Expressivity of probabilistic programming language is based on two factors: One is the theoretical expressivity of the language, i.e. the number of probabilistic programs that it can represent. The second determines the practical expressivity of the language, i.e. how easily and quickly developers can write complex programs. One potential drawback to expressivity is the ease with which a programmer can write ill-defined probabilistic programs in which efficient sampling is not possible due to the complexity of their resulting models.

Horizontal vs. Vertical Integration

The second way to classify probabilistic programming languages is based on the type of integration i.e. horizontal vs. vertical integration. There are two aspects to this. Horizontal integration includes a probabilistic programming language that offers everything, including libraries for tooling, visualization, etc, or whether probabilistic constructs are merely added to existing languages, while vertical integration includes probabilistic programming language that provides the full stack of functionality needed for probabilistic programming such as inference engines.

Probabilistic programming can help in revolutionizing machine learning capabilities in fields ranging from Intelligence, Natural Language Processing (NLP) to cyber security and predictive analytics. It will also aid experts in building useful machine learning applications. With reusable tools and new probabilistic programming languages, we can aim at reducing current barriers to machine learning and foster effectiveness, innovation, and productivity.