Home | Portfolio | Terms and Conditions | E-mail me | LinkedIn

Viewing LSTM predictions in a web application with Django

This is a program I created using Django that runs an LSTM model (long-short term memory network) on a dataset, and then outputs the predictions for that dataset in a Django web page.

The dataset in question is the brexit_wikipedia dataset included in this repository, and the purpose of LSTM is to forecast page views for this term on the Wikipedia website over time.

Here are the steps to creating the Django web application. Note that the below terminal commands were run using Linux.

  1. Open the terminal and input the following:

django-admin startproject mysite

This creates the mysite folder.

python3 manage.py runserver

This runs the server. Upon inputting http://127.0.0.1:8000/ into your browser, the following should display:

install_worked_successfully

  1. Now, a subfolder is created which we will call pyfiles.
python3 manage.py startapp pyfiles
  1. Visit mysite -> urls.py, and ensure the following is included in the .py code:
from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path('pyfiles/', include('pyfiles.urls')),
    path('admin/', admin.site.urls),
]
  1. Also ensure that the following is included in mysite -> pyfiles -> urls.py:
from django.urls import path

from . import views

urlpatterns = [
    path('', views.index, name='index'),
]

Under views.py, the Python code to run LSTM specifically is included:

# Create your views here.
from django.http import HttpResponse
from PIL import Image

# Import libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from pandas import read_csv
import math
import pylab
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from pandas import Series
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error
import os;
path="directory"
os.chdir(path)
os.getcwd()

# Form dataset matrix
def create_dataset(dataset, previous=1):
	dataX, dataY = [], []
	for i in range(len(dataset)-previous-1):
		a = dataset[i:(i+previous), 0]
		dataX.append(a)
		dataY.append(dataset[i + previous, 0])
	return np.array(dataX), np.array(dataY)

# fix random seed for reproducibility
np.random.seed(7)

# load dataset
dataframe = read_csv('brexit_wikipedia.csv', usecols=[1], engine='python', skipfooter=3)
dataset = dataframe.values
dataset = dataset.astype('float32')

std1=np.std(dataset)
mean1=np.mean(dataset)
cv1=std1/mean1 #Coefficient of Variation

from numpy import log
dataset = log(dataset)

std2=np.std(dataset)
mean2=np.mean(dataset)
cv2=std2/mean2 #Coefficient of Variation

# normalize dataset with MinMaxScaler
scaler = MinMaxScaler(feature_range=(0, 1))
dataset = scaler.fit_transform(dataset)

# Training and Test data partition
train_size = int(len(dataset) * 0.8)
test_size = len(dataset) - train_size
train, test = dataset[0:train_size,:], dataset[train_size:len(dataset),:]

# reshape into X=t-50 and Y=t (MODIFY TIME HERE)
previous = 50
X_train, Y_train = create_dataset(train, previous)
X_test, Y_test = create_dataset(test, previous)

# reshape input to be [samples, time steps, features]
X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))

# Generate LSTM network
model = Sequential()
model.add(LSTM(4, input_shape=(1, previous)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, Y_train, epochs=100, batch_size=1, verbose=2)

# Generate predictions
trainpred = model.predict(X_train)
testpred = model.predict(X_test)

# Convert predictions back to normal values
trainpred = scaler.inverse_transform(trainpred)
Y_train = scaler.inverse_transform([Y_train])
testpred = scaler.inverse_transform(testpred)
Y_test = scaler.inverse_transform([Y_test])
predictions = testpred

# calculate RMSE
trainScore = math.sqrt(mean_squared_error(Y_train[0], trainpred[:,0]))
print('Train Score: %.2f RMSE' % (trainScore))
testScore = math.sqrt(mean_squared_error(Y_test[0], testpred[:,0]))
print('Test Score: %.2f RMSE' % (testScore))

# Train predictions
trainpredPlot = np.empty_like(dataset)
trainpredPlot[:, :] = np.nan
trainpredPlot[previous:len(trainpred)+previous, :] = trainpred

# Test predictions
testpredPlot = np.empty_like(dataset)
testpredPlot[:, :] = np.nan
testpredPlot[len(trainpred)+(previous*2)+1:len(dataset)-1, :] = testpred

# Plot all predictions
inversetransform, =plt.plot(scaler.inverse_transform(dataset))
trainpred, =plt.plot(trainpredPlot)
testpred, =plt.plot(testpredPlot)
plt.title("Predicted vs. Actual Page Views (Logarithmic Format)")

path='mysite/graphs'
os.chdir(path)
plt.savefig('testpred.png')

img = Image.open('mysite/graphs/testpred.png')

def index(request):
    response = HttpResponse(content_type = 'image/png')
    img.save(response, "PNG")
    return response
  1. Now, the server is run:

python3 manage.py runserver

The graph can now be visualized by visiting http://localhost:8000/pyfiles/:

lstm_results