How Machine Learning is used in Predicting Stock Prices - LSTM

Algorithms can predict future stock prices. These models analyze huge amounts of data to detect complex patterns in the market and figure out how prices might change. Machine learning is increasingly being used by major financial institutions to gain an edge in the stock market. Top companies like Goldman Sachs and JP Morgan are leveraging sophisticated algorithms to predict future stock prices and make profitable trades.

Some of the machine learning models used for stock price forecasting include:

• Long short-term memory (LSTM) networks
• Support vector machines (SVMs)
• Random forests
• Gradient boosting machines (GBMs)
• Ensemble learning

In this article, we'll explore one of the Machine Learning algorithms used for stock price prediction which is LSTM. We'll delve into how it works with an example

Long short-term memory (LSTM) networks

LSTMs are a type of neural network that can learn long-term dependencies and are useful for predicting stock prices. They examine a sequence of stock prices over time to detect patterns and predict future prices.

Understanding LSTM

Imagine how humans process language. When we listen to someone speak, we comprehend each word within the context of preceding words and the overall conversation. Likewise, when predicting stock prices, LSTMs rely on their past experiences, just as we rely on our knowledge to understand the meaning of words in a sentence.

To illustrate LSTM's concept, let's look at predicting the next number in a sequence:

• In the sequence [6 - 7 - 8 - ], we expect the next number to be 9.
• However, in the sequence [2 - 4 - 8 - ], we anticipate the next number to be 16.

This is where LSTM networks shine. They excel at capturing dependencies over time, just like our brains adapt to changing contexts during a conversation. As time progresses, older inputs in a sequence have less impact on predicting the next value. LSTM networks are designed to learn when to remember important information and when to forget less relevant data, a critical feature when dealing with the ever-changing landscape of stock market data.

How LSTM Networks Work

At the heart of LSTM networks are memory cells that store and update information over time. These memory cells are connected through a network of gates that control the flow of information. The three fundamental gates in an LSTM network are:

1. Forget Gate: This gate determines what information from the previous time step should be discarded or forgotten.
2. Input Gate: The input gate decides which new information from the current time step should be stored in the memory cell.
3. Output Gate: The output gate selects what information from the memory cell should be used to make the prediction for the current time step.

These gates work in tandem to capture patterns and dependencies in the data. By adjusting the weights and biases associated with each gate during training, LSTM networks can effectively learn to remember and forget information as needed to make accurate predictions.

Example

I have done a team project using Machine Learning Algorithms at my University for Stock Price Prediction. we employed LSTM (Long Short-Term Memory) neural networks as one of the algorithms to predict Tesla stock prices.

The Process involved,

• Data preparation
• Visualization of historical price trends
• Feature engineering using a sliding window approach
• Building a Sequential neural network model with multiple LSTM layers and dropout regularization
• Training the model to predict the next day's stock price
• Visualizing the model's performance by comparing real and predicted prices over time

This approach allowed for effective forecasting of Tesla stock prices and showcased the capabilities of LSTM in financial time series prediction.

In our first attempt at using LSTM (Long Short-Term Memory) for stock price prediction, we want to be upfront about our results. Please don't expect perfect accuracy just yet. While we'll show you what we've done, keep in mind that improving predictions takes time, data, and experience.

Code

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from datetime import datetime
from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error, mean_absolute_error
from sklearn.model_selection import train_test_split
from keras.callbacks import EarlyStopping
from tensorflow.keras.optimizers import Adam, SGD

# Function to convert date format
def to_datetime(df):
    date = datetime.strptime(df, '%d.%m.%Y')
    return date.strftime("%Y-%m-%d")

# Read and preprocess the dataset
df = pd.read_csv('Price Tesla.csv', sep=';')
df['Date'] = df['Date'].apply(lambda x: to_datetime(x))
df = df.sort_values('Date').reset_index(drop=True)
df['Price'] = df['Price'].astype(float)

# Visualize Tesla stock price data
plt.figure(figsize=(20, 7))
plt.plot(df['Date'].values, df['Price'].values, label='Tesla Stock Price', color='red')
plt.xticks(np.arange(100, df.shape[0], 200))
plt.xlabel('Date')
plt.ylabel('Price ($)')
plt.legend()
plt.show()

# Prepare training data
X_train = []
y_train = []
window = 60

for i in range(window, num_shape):  # num_shape needs to be defined
    X_train_ = np.reshape(train_scaled[i-window:i, 0], (window, 1))  # train_scaled should be defined
    X_train.append(X_train_)
    y_train.append(train_scaled[i, 0])
X_train = np.stack(X_train)
y_train = np.stack(y_train)

# Initialize the Recurrent Neural Network (RNN) model
model = Sequential()
model.add(LSTM(units=50, return_sequences=True, input_shape=(X_train.shape[1], 1)))
model.add(Dropout(0.2))
model.add(LSTM(units=50, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=50, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=50))
model.add(Dropout(0.2))
model.add(Dense(units=1))
model.summary()

# Visualize real and predicted Tesla stock prices
plt.figure(figsize=(20, 7))
plt.plot(df['Date'].values[1800:], df_volume[1800:], color='red', label='Real Tesla Stock Price')  # df_volume needs to be defined
plt.plot(df['Date'][-predict.shape[0]:].values, predict, color='blue', label='Predicted Tesla Stock Price')  # predict needs to be defined
plt.xticks(np.arange(100, df[1800:].shape[0], 200))
plt.title('Tesla Stock Price Prediction')
plt.xlabel('Date')
plt.ylabel('Price ($)')
plt.legend()
plt.show()

# 20 days prediction
pred_ = predict[-1].copy()  # predict needs to be defined
prediction_full = []
window = 60
df_copy = df.iloc[:, 1:2][1:].values

for j in range(20):
    df_ = np.vstack((df_copy, pred_))
    train_ = df_[:num_shape]  # num_shape needs to be defined
    test_ = df_[num_shape:]  # num_shape needs to be defined

    df_volume_ = np.vstack((train_, test_))

    inputs_ = df_volume_[df_volume_.shape[0] - test_.shape[0] - window:]
    inputs_ = inputs_.reshape(-1, 1)
    inputs_ = sc.transform(inputs_)  # sc needs to be defined

    X_test_2 = []

    for k in range(window, num_2):  # num_2 needs to be defined
        X_test_3 = np.reshape(inputs_[k-window:k, 0], (window, 1))
        X_test_2.append(X_test_3)

    X_test_ = np.stack(X_test_2)
    predict_ = model.predict(X_test_)
    pred_ = sc.inverse_transform(predict_)  # sc needs to be defined
    prediction_full.append(pred_[-1][0])
    df_copy = df_[j:]

# Visualize real and predicted Tesla stock prices
plt.figure(figsize=(20, 7))
plt.plot(df['Date'].values[1700:], df_volume[1700:], color='red', label='Real Tesla Stock Price')  # df_volume needs to be defined
plt.plot(df_date['Date'][-prediction_full_new.shape[0]:].values, prediction_full_new, color='blue',
         label='Predicted Tesla Stock Price')  # prediction_full_new needs to be defined
plt.xticks(np.arange(100, df[1700:].shape[0], 200))
plt.title('Tesla Stock Price Prediction')
plt.xlabel('Date')
plt.ylabel('Price ($)')
plt.legend()
plt.show()
        

Output:

LSTM

MSE: 4840.804172281035

MAE: 63.15895608084543

RMSE: 69.5758878655604

The performance can certainly be improved by tuning the model further.

Limitations of LSTM stock price prediction

There are a number of limitations to using LSTM neural networks for stock price prediction. One limitation is that LSTM models require a large amount of historical data to train. Another limitation is that LSTM models are complex and can be difficult to tune. Additionally, LSTM models are susceptible to overfitting, which can lead to inaccurate predictions on new data.

Other Popular Algorithms for Price Prediction:

Support vector machines (SVMs)

SVMs are supervised learning models that analyze data and recognize patterns. They can classify stock price data into groups to predict if a stock price will increase or decrease.

Random forests

Random forests are an ensemble learning method that combines the predictions of multiple decision trees. They analyze many variables to determine which factors influence stock price changes the most.

Gradient boosting machines (GBMs)

GBMs are also ensemble methods that combine the predictions of multiple models to improve accuracy. They incrementally build new models to predict stock prices based on the errors of prior models.

Ensemble learning

Ensemble learning combines multiple ML models, like the ones above, into a meta-model that can make even more accurate predictions. For stock trading, ensemble models that incorporate both technical analysis and fundamental analysis tend to perform the best.

Conclusion

In the future, machine learning will become even more widespread in finance. Algorithms will continue to become more sophisticated, using techniques like deep learning, reinforcement learning and transfer learning. Firms will have AI systems that can analyze huge amounts of data, detect complex patterns and relationships, and make extremely accurate predictions about the market. Overall, machine learning has the potential to vastly improve how investments are made and how the finance industry operates.

Disclaimer:?The material is purely for educational and should not be taken as professional investment advice. Invest at your own discretion.