{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Utilizing daal4py in Data Science Workflows\n",
    "\n",
    "The notebook below has been made to demonstrate daal4py in a data science context.  It utilizes a Cycling Dataset for pyworkout-toolkit, and attempts to create a linear regression model from the 5 features collected for telemetry to predict the user's Power output in the absence of a power meter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.8.10 (default, May 19 2021, 18:05:58) \\n[GCC 7.3.0]'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import glob\n",
    "import sys\n",
    "%matplotlib inline\n",
    "sys.version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example will be exploring workout data pulled from Strava, processed into a CSV for Pandas and daal4py usage.  Below, we utilize pandas to read in the CSV file, and look at the head of dataframe with .head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>altitude</th>\n",
       "      <th>cadence</th>\n",
       "      <th>distance</th>\n",
       "      <th>hr</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>power</th>\n",
       "      <th>speed</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>185.800003</td>\n",
       "      <td>51</td>\n",
       "      <td>3.46</td>\n",
       "      <td>81</td>\n",
       "      <td>30.313309</td>\n",
       "      <td>-97.732711</td>\n",
       "      <td>45</td>\n",
       "      <td>3.459</td>\n",
       "      <td>2016-10-20T22:01:26.000Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>185.800003</td>\n",
       "      <td>68</td>\n",
       "      <td>7.17</td>\n",
       "      <td>82</td>\n",
       "      <td>30.313277</td>\n",
       "      <td>-97.732715</td>\n",
       "      <td>0</td>\n",
       "      <td>3.710</td>\n",
       "      <td>2016-10-20T22:01:27.000Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>186.399994</td>\n",
       "      <td>38</td>\n",
       "      <td>11.04</td>\n",
       "      <td>82</td>\n",
       "      <td>30.313243</td>\n",
       "      <td>-97.732717</td>\n",
       "      <td>42</td>\n",
       "      <td>3.874</td>\n",
       "      <td>2016-10-20T22:01:28.000Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>186.800003</td>\n",
       "      <td>38</td>\n",
       "      <td>15.18</td>\n",
       "      <td>83</td>\n",
       "      <td>30.313212</td>\n",
       "      <td>-97.732720</td>\n",
       "      <td>5</td>\n",
       "      <td>4.135</td>\n",
       "      <td>2016-10-20T22:01:29.000Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>186.600006</td>\n",
       "      <td>38</td>\n",
       "      <td>19.43</td>\n",
       "      <td>83</td>\n",
       "      <td>30.313172</td>\n",
       "      <td>-97.732723</td>\n",
       "      <td>1</td>\n",
       "      <td>4.250</td>\n",
       "      <td>2016-10-20T22:01:30.000Z</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     altitude  cadence  distance  hr   latitude  longitude  power  speed  \\\n",
       "0  185.800003       51      3.46  81  30.313309 -97.732711     45  3.459   \n",
       "1  185.800003       68      7.17  82  30.313277 -97.732715      0  3.710   \n",
       "2  186.399994       38     11.04  82  30.313243 -97.732717     42  3.874   \n",
       "3  186.800003       38     15.18  83  30.313212 -97.732720      5  4.135   \n",
       "4  186.600006       38     19.43  83  30.313172 -97.732723      1  4.250   \n",
       "\n",
       "                       time  \n",
       "0  2016-10-20T22:01:26.000Z  \n",
       "1  2016-10-20T22:01:27.000Z  \n",
       "2  2016-10-20T22:01:28.000Z  \n",
       "3  2016-10-20T22:01:29.000Z  \n",
       "4  2016-10-20T22:01:30.000Z  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "workout_data_dd= pd.read_csv('data/batch/cycling_dataset.csv', index_col=0)\n",
    "workout_data_dd.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data above has several key features that would be of great use here.  \n",
    "- Altitude can affect performance, so it might be a useful feature.  \n",
    "- Cadence is the revolutions per minute of the crank, and may have possible influence.  \n",
    "- Heart Rate is a measure of the body's workout strain, and would have a high possibly of influence.\n",
    "- Distance may have a loose correlation as it is highly route dependent, but might be possible.\n",
    "- Speed has possible correlations as it ties directly into power."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore and visualize some of the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, we are trying to predict on the 'power' in Watts to see if we can generate a model that can predict one's power output without the usage of a cycling power meter.  Below are some basic scatterplots as we explore the data.  Scatterplots are great for looking for patterns and correlation in the data itself.  Below, we can see that cadence and speed are positively correlated.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27928 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 26132 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 27428 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 26876 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 36492 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 35984 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "workout_data_dd.plot.scatter('cadence','power')\n",
    "plt.show()\n",
    "workout_data_dd.plot.scatter('hr','power')\n",
    "plt.show()\n",
    "workout_data_dd.plot.scatter('cadence','speed')\n",
    "plt.show()\n",
    "workout_data_dd.plot.scatter('speed','power')\n",
    "plt.show()\n",
    "workout_data_dd.plot.scatter('altitude','power')\n",
    "plt.show()\n",
    "workout_data_dd.plot.scatter('distance','power')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using daal4py for Machine Learning tasks\n",
    "\n",
    "In the sections below, we will be using daal4py directly.  After importing the model, we will arrange it in a separate independent and dependent dataframes, then use the daal4py's training and prediction classes to generate a workable model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import daal4py as d4p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is now the time to split the dataset into train and test sets.  This is demonstrated below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3902, 9)\n",
      "(3000, 9) (902, 9)\n"
     ]
    }
   ],
   "source": [
    "print(workout_data_dd.shape)\n",
    "train_set = workout_data_dd[0:3000]\n",
    "test_set = workout_data_dd[3000:]\n",
    "print(train_set.shape, test_set.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reduce the dataset, create X.  We drop the target, and other non-essential features.\n",
    "reduced_dataset = train_set.drop(['time','power','latitude','longitude'], axis=1)\n",
    "# Get the target, create Y\n",
    "target = train_set.power.values.reshape((-1,1))\n",
    "# This is essentially doing np.array(dataset.power.values, ndmin=2).T\n",
    "# as it needs to force a 2 dimensional array as we only have 1 target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "X is 5 features by 3k rows, Y is 3k rows by 1 column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3000, 5) (3000, 1)\n"
     ]
    }
   ],
   "source": [
    "print(reduced_dataset.values.shape, target.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the Linear Regression Model, and train the model with the data.  We utilize daal4py's linear_regression_training class to create the model, then call .compute() with the independent and dependent data as the parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "d4p_lm = d4p.linear_regression_training(interceptFlag=True)\n",
    "lm_trained = d4p_lm.compute(reduced_dataset.values, target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model has this number of features:  5\n"
     ]
    }
   ],
   "source": [
    "print(\"Model has this number of features: \", lm_trained.model.NumberOfFeatures)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prediction (inference) with the trained model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the model is trained, we can test it with the test part of the dataset.  We drop the same features to match that of the trained model, and put it into daal4py's linear_regression_prediction class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "subset = test_set.drop(['time','power','latitude','longitude'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create the Prediction object and use the reduced dataset for prediction.  The class's arguments use the independent data and the trained model from above as the parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "lm_predictor_component = d4p.linear_regression_prediction()\n",
    "result = lm_predictor_component.compute(subset.values, lm_trained.model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 32368 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(result.prediction[0:300])\n",
    "plt.plot(test_set.power.values[0:300])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The graph above shows the Orange (predicted) result over the Blue (original data).  This data is notoriously sparse in features leading to a difficult to predict target!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model properties\n",
    "Another aspect of the model is the trained model's properties, which are explored below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Betas: [[ 1.51003501e+01 -1.25075548e-01  1.32249115e+00  1.64363922e-03\n",
      "   8.53155955e-01 -1.09595022e+01]]\n",
      "Number of betas: 6\n",
      "Number of Features: 5\n"
     ]
    }
   ],
   "source": [
    "print(\"Betas:\",lm_trained.model.Beta) \n",
    "print(\"Number of betas:\", lm_trained.model.NumberOfBetas)\n",
    "print(\"Number of Features:\", lm_trained.model.NumberOfFeatures)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional metrics\n",
    "We can generate metrics on the independent data with daal4py's low_order_moments() class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.90063975e+01, 3.75882355e+01, 4.98258371e+03, 2.41394741e+01,\n",
       "        1.81623064e+00]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics_processor = d4p.low_order_moments()\n",
    "data = metrics_processor.compute(reduced_dataset.values)\n",
    "data.standardDeviation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Migrating the trained model for inference on external systems\n",
    "\n",
    "Occasionally one may need to migrate the trained model to another system for inference only--this use case allows the training on a much more powerful machine with a larger dataset, and placing the trained model for inference-only on a smaller machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('trained_model2.pickle', 'wb') as model_pi:\n",
    "    pickle.dump(lm_trained.model, model_pi)\n",
    "    model_pi.close"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The trained model file above can be moved to an inference-only or embedded system.  This is useful if the training is extreamly heavy or computed-limited.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('trained_model2.pickle', 'rb') as model_import:\n",
    "    lm_import = pickle.load(model_import)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The imported model from file is now usable again.  We can check the betas from the model to ensure that the trained model is present."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.51003501e+01, -1.25075548e-01,  1.32249115e+00,\n",
       "         1.64363922e-03,  8.53155955e-01, -1.09595022e+01]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm_import.Beta"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
