Are you "a" positive or negative?

In this article I will tell you what an Hessian Matrix is and why you should care about it. Actually, we already faced Hessian or a simple form of Hessian couple of times during our high school times. Do you still remember about the quadratic function which we learnt in 10th or 11th grade? 

Suppose you have a function,

where a, b, and c are just numbers and x is your variable. If you can rewind back your memory a bit to the past, what was your teacher told you at that time about this function?

I believe he/she would say,

“If your value of "a" is positive, you will get a smiley graph and you will be happy for the rest of your day. But, if your "a" is negative, well your day will be bit gloomy because your graph is sad.”

Does it ring a bell to you? Have you heard something similar? These statement can be illustrated in the following figures.ey

So, do you get the point? Since, we are no longer a high schooler and we want to look cool, so our x now is no longer a single variable but multivariable instead. We can call it a vector x. But, do you want to be cooler?

In order to be cool, let's transform our happy and sad smiley graphs up there to a 3D surface. They will look like:

Now, the graphs have transformed, so does our x. It looks like this,

and our lovely function f(x) will also grow up and look like this 

in terms of vectors and matrix. Here x, b, and c are vectors and A a is matrix. If we take the first-order derivative of our function f(x), we will get what we called the gradient vector and the second-order derivatives will give us the Hessian! The gradient and Hessian can be presented as follows:

Let’s take a moment to relate back what we had learned in high school with this matrix-vector form of a function.

We had learned that in high school if the second-derivative of our function is positive, we will get a smiley graph or scientifically a concave upward curve. If the second-derivative of our function is negative, we will get a concave downward curve.

It is similar for the matrix-vector form of a function but now it is depending on the eigenvalues of the Hessian (Eigenvalues? Maybe I will cover it later).

In conclusion, Hessian is the second-order derivatives represents the curvature of the function or the objective function in optimization literature. The Hessian or the letter H is not something you should be afraid of since it just a letter. It would not bring you any harms though.

Yet, for those who are working in the data science, machine learning, and any related fields, you have no choice. Either to face your fear of this letter H or simply quit. But you will not quit, don’t you?

For someone like you, whom I believe will not simply quit, check out this links for more details:

https://en.wikipedia.org/wiki/Hessian_matrix