Computational Graphs in Deep Learning With Python

Deep Learning Computational Graphs

In fields like Cheminformatics and Natural Language Understanding, it is often useful to compute over data-flow graphs. Computational Graph form an integral part of Deep Learning. Not only do they help us simplify working with large datasets, they’re simple to understand. So in this tutorial, we will introduce them to you and then show you how to implement them using Python. For this, we will use the Dask library from Python PyPI.

Do you know about Python Library

What are Computational Graphs in Deep Learning?

A computational graph is a way to represent a mathematical function in the language of graph theory. Nodes are input values or functions for combining them; as data flows through this graph, the edges receive their weights.

So we said we have two kinds of nodes- input nodes and function nodes. Outbound edges from input nodes bear the input value, and those from function nodes bear the composite of the weights of the inbound edges. The graph above represents the following expression:

f(x,y,z)=(x+y)*z

Have a look at deep Learning vs Machine Learning

Of the five nodes, the leftmost three are input nodes; the two on the right are function nodes. Now if we’d have to compute f(1,2,3), we’d get the following computation graph-

Need of Computational Graph

Well, this was a simple computational graph with 5 nodes and 5 edges. But even simpler deep neural networks observe hundreds of thousands of nodes and edges- say, more than one million? In such a case, it would be practically impossible to calculate a function expression for it. Then, computational graphs come in handy.

Such graphs also help us describe backpropagation more precisely.

Computational Graphs Example – Composite Function

We take an example of the function f(x)=esin(x**2). Let’s decompose this-

f(x)=ex

g(x)=sin x

h(x)=x2

f(g(h(x)))=eg(h(x))

Let’s revise the Python Machine Learning Tutorial

We have the following computational graph for this-

Visualizing a Computation Graph in Python

Now let’s use the Dask library to produce such a graph.

1. >>> from dask import delayed,compute
2. >>> import dask
3. >>> @delayed
4. def square(num):
5. print("Square function:",num)
6. print()
7. return num*num
8. >>> @delayed
9. def sum_list(args):
10. print("Sum_list function:",args)
11. return sum(args)
12. >>> items=[1,2,3]
13. >>> computation_graph = sum_list([square(i) for i in items])
14. >>> computation_graph.visualize()

<IPython.core.display.Image object>

1. >>> computation_graph.visualize(filename='sumlist.svg')

<IPython.core.display.SVG object>

Note that for this code to work, you will need to perform three tasks-

1. Install the dask library with pip:
2. pip install dask
3. Download and unzip the Windows packages for Dask-

https://graphviz.gitlab.io/_pages/Download/Download_windows.html

1. Add the first line of code to your User Path and the next to your system path in environment variables:

C:\Users\Ayushi\Desktop\graphviz-2.38\release\bin (Use your own path for bin)

C:\Users\Ayushi\Desktop\graphviz-2.38\release\bin\dot.exe

Read Complete Article>>