sagemaker-deployment/Tutorials/Boston Housing - XGBoost (Batch Transform) - Low Level.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting Boston Housing Prices\n",
    "\n",
    "## Using XGBoost in SageMaker (Batch Transform)\n",
    "\n",
    "_Deep Learning Nanodegree Program | Deployment_\n",
    "\n",
    "---\n",
    "\n",
    "As an introduction to using SageMaker's Low Level Python API we will look at a relatively simple problem. Namely, we will use the [Boston Housing Dataset](https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html) to predict the median value of a home in the area of Boston Mass.\n",
    "\n",
    "The documentation reference for the API used in this notebook is the [SageMaker Developer's Guide](https://docs.aws.amazon.com/sagemaker/latest/dg/)\n",
    "\n",
    "## General Outline\n",
    "\n",
    "Typically, when using a notebook instance with SageMaker, you will proceed through the following steps. Of course, not every step will need to be done with each project. Also, there is quite a lot of room for variation in many of the steps, as you will see throughout these lessons.\n",
    "\n",
    "1. Download or otherwise retrieve the data.\n",
    "2. Process / Prepare the data.\n",
    "3. Upload the processed data to S3.\n",
    "4. Train a chosen model.\n",
    "5. Test the trained model (typically using a batch transform job).\n",
    "6. Deploy the trained model.\n",
    "7. Use the deployed model.\n",
    "\n",
    "In this notebook we will only be covering steps 1 through 5 as we just want to get a feel for using SageMaker. In later notebooks we will talk about deploying a trained model in much more detail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 0: Setting up the notebook\n",
    "\n",
    "We begin by setting up all of the necessary bits required to run our notebook. To start that means loading all of the Python modules we will need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import os\n",
    "\n",
    "import time\n",
    "from time import gmtime, strftime\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.datasets import load_boston\n",
    "import sklearn.model_selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the modules above, we need to import the various bits of SageMaker that we will be using. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "\n",
    "# This is an object that represents the SageMaker session that we are currently operating in. This\n",
    "# object contains some useful information that we will need to access later such as our region.\n",
    "session = sagemaker.Session()\n",
    "\n",
    "# This is an object that represents the IAM role that we are currently assigned. When we construct\n",
    "# and launch the training job later we will need to tell it what IAM role it should have. Since our\n",
    "# use case is relatively simple we will simply assign the training job the role we currently have.\n",
    "role = get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Downloading the data\n",
    "\n",
    "Fortunately, this dataset can be retrieved using sklearn and so this step is relatively straightforward."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "boston = load_boston()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Preparing and splitting the data\n",
    "\n",
    "Given that this is clean tabular data, we don't need to do any processing. However, we do need to split the rows in the dataset up into train, test and validation sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First we package up the input data and the target variable (the median value) as pandas dataframes. This\n",
    "# will make saving the data to a file a little easier later on.\n",
    "\n",
    "X_bos_pd = pd.DataFrame(boston.data, columns=boston.feature_names)\n",
    "Y_bos_pd = pd.DataFrame(boston.target)\n",
    "\n",
    "# We split the dataset into 2/3 training and 1/3 testing sets.\n",
    "X_train, X_test, Y_train, Y_test = sklearn.model_selection.train_test_split(X_bos_pd, Y_bos_pd, test_size=0.33)\n",
    "\n",
    "# Then we split the training set further into 2/3 training and 1/3 validation sets.\n",
    "X_train, X_val, Y_train, Y_val = sklearn.model_selection.train_test_split(X_train, Y_train, test_size=0.33)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Uploading the data files to S3\n",
    "\n",
    "When a training job is constructed using SageMaker, a container is executed which performs the training operation. This container is given access to data that is stored in S3. This means that we need to upload the data we want to use for training to S3. In addition, when we perform a batch transform job, SageMaker expects the input data to be stored on S3. We can use the SageMaker API to do this and hide some of the details.\n",
    "\n",
    "### Save the data locally\n",
    "\n",
    "First we need to create the test, train and validation csv files which we will then upload to S3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is our local data directory. We need to make sure that it exists.\n",
    "data_dir = '../data/boston'\n",
    "if not os.path.exists(data_dir):\n",
    "    os.makedirs(data_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We use pandas to save our test, train and validation data to csv files. Note that we make sure not to include header\n",
    "# information or an index as this is required by the built in algorithms provided by Amazon. Also, for the train and\n",
    "# validation data, it is assumed that the first entry in each row is the target variable.\n",
    "\n",
    "X_test.to_csv(os.path.join(data_dir, 'test.csv'), header=False, index=False)\n",
    "\n",
    "pd.concat([Y_val, X_val], axis=1).to_csv(os.path.join(data_dir, 'validation.csv'), header=False, index=False)\n",
    "pd.concat([Y_train, X_train], axis=1).to_csv(os.path.join(data_dir, 'train.csv'), header=False, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload to S3\n",
    "\n",
    "Since we are currently running inside of a SageMaker session, we can use the object which represents this session to upload our data to the 'default' S3 bucket. Note that it is good practice to provide a custom prefix (essentially an S3 folder) to make sure that you don't accidentally interfere with data uploaded from some other notebook or project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "prefix = 'boston-xgboost-LL'\n",
    "\n",
    "test_location = session.upload_data(os.path.join(data_dir, 'test.csv'), key_prefix=prefix)\n",
    "val_location = session.upload_data(os.path.join(data_dir, 'validation.csv'), key_prefix=prefix)\n",
    "train_location = session.upload_data(os.path.join(data_dir, 'train.csv'), key_prefix=prefix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Train and construct the XGBoost model\n",
    "\n",
    "Now that we have the training and validation data uploaded to S3, we can construct a training job for our XGBoost model and build the model itself.\n",
    "\n",
    "### Set up the training job\n",
    "\n",
    "First, we will set up and execute a training job for our model. To do this we need to specify some information that SageMaker will use to set up and properly execute the computation. For additional documentation on constructing a training job, see the [CreateTrainingJob API](https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html) reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will need to know the name of the container that we want to use for training. SageMaker provides\n",
    "# a nice utility method to construct this for us.\n",
    "container = get_image_uri(session.boto_region_name, 'xgboost')\n",
    "\n",
    "# We now specify the parameters we wish to use for our training job\n",
    "training_params = {}\n",
    "\n",
    "# We need to specify the permissions that this training job will have. For our purposes we can use\n",
    "# the same permissions that our current SageMaker session has.\n",
    "training_params['RoleArn'] = role\n",
    "\n",
    "# Here we describe the algorithm we wish to use. The most important part is the container which\n",
    "# contains the training code.\n",
    "training_params['AlgorithmSpecification'] = {\n",
    "    \"TrainingImage\": container,\n",
    "    \"TrainingInputMode\": \"File\"\n",
    "}\n",
    "\n",
    "# We also need to say where we would like the resulting model artifacts stored.\n",
    "training_params['OutputDataConfig'] = {\n",
    "    \"S3OutputPath\": \"s3://\" + session.default_bucket() + \"/\" + prefix + \"/output\"\n",
    "}\n",
    "\n",
    "# We also need to set some parameters for the training job itself. Namely we need to describe what sort of\n",
    "# compute instance we wish to use along with a stopping condition to handle the case that there is\n",
    "# some sort of error and the training script doesn't terminate.\n",
    "training_params['ResourceConfig'] = {\n",
    "    \"InstanceCount\": 1,\n",
    "    \"InstanceType\": \"ml.m4.xlarge\",\n",
    "    \"VolumeSizeInGB\": 5\n",
    "}\n",
    "    \n",
    "training_params['StoppingCondition'] = {\n",
    "    \"MaxRuntimeInSeconds\": 86400\n",
    "}\n",
    "\n",
    "# Next we set the algorithm specific hyperparameters. You may wish to change these to see what effect\n",
    "# there is on the resulting model.\n",
    "training_params['HyperParameters'] = {\n",
    "    \"max_depth\": \"5\",\n",
    "    \"eta\": \"0.2\",\n",
    "    \"gamma\": \"4\",\n",
    "    \"min_child_weight\": \"6\",\n",
    "    \"subsample\": \"0.8\",\n",
    "    \"objective\": \"reg:linear\",\n",
    "    \"early_stopping_rounds\": \"10\",\n",
    "    \"num_round\": \"200\"\n",
    "}\n",
    "\n",
    "# Now we need to tell SageMaker where the data should be retrieved from.\n",
    "training_params['InputDataConfig'] = [\n",
    "    {\n",
    "        \"ChannelName\": \"train\",\n",
    "        \"DataSource\": {\n",
    "            \"S3DataSource\": {\n",
    "                \"S3DataType\": \"S3Prefix\",\n",
    "                \"S3Uri\": train_location,\n",
    "                \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "            }\n",
    "        },\n",
    "        \"ContentType\": \"csv\",\n",
    "        \"CompressionType\": \"None\"\n",
    "    },\n",
    "    {\n",
    "        \"ChannelName\": \"validation\",\n",
    "        \"DataSource\": {\n",
    "            \"S3DataSource\": {\n",
    "                \"S3DataType\": \"S3Prefix\",\n",
    "                \"S3Uri\": val_location,\n",
    "                \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "            }\n",
    "        },\n",
    "        \"ContentType\": \"csv\",\n",
    "        \"CompressionType\": \"None\"\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Execute the training job\n",
    "\n",
    "Now that we've built the dictionary object containing the training job parameters, we can ask SageMaker to execute the job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First we need to choose a training job name. This is useful for if we want to recall information about our\n",
    "# training job at a later date. Note that SageMaker requires a training job name and that the name needs to\n",
    "# be unique, which we accomplish by appending the current timestamp.\n",
    "training_job_name = \"boston-xgboost-\" + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "training_params['TrainingJobName'] = training_job_name\n",
    "\n",
    "# And now we ask SageMaker to create (and execute) the training job\n",
    "training_job = session.sagemaker_client.create_training_job(**training_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training job has now been created by SageMaker and is currently running. Since we need the output of the training job, we may wish to wait until it has finished. We can do so by asking SageMaker to output the logs generated by the training job and continue doing so until the training job terminates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-01-14 12:54:19 Starting - Starting the training job...\n",
      "2019-01-14 12:54:22 Starting - Launching requested ML instances......\n",
      "2019-01-14 12:55:47 Starting - Preparing the instances for training......\n",
      "2019-01-14 12:56:42 Downloading - Downloading input data...\n",
      "2019-01-14 12:57:20 Training - Training image download completed. Training in progress.\n",
      "2019-01-14 12:57:20 Uploading - Uploading generated training model.\n",
      "\u001b[31mArguments: train\u001b[0m\n",
      "\u001b[31m[2019-01-14:12:57:15:INFO] Running standalone xgboost training.\u001b[0m\n",
      "\u001b[31m[2019-01-14:12:57:15:INFO] File size need to be processed in the node: 0.02mb. Available memory size in the node: 8439.73mb\u001b[0m\n",
      "\u001b[31m[2019-01-14:12:57:15:INFO] Determined delimiter of CSV input is ','\u001b[0m\n",
      "\u001b[31m[12:57:15] S3DistributionType set as FullyReplicated\u001b[0m\n",
      "\u001b[31m[12:57:15] 227x13 matrix with 2951 entries loaded from /opt/ml/input/data/train?format=csv&label_column=0&delimiter=,\u001b[0m\n",
      "\u001b[31m[2019-01-14:12:57:15:INFO] Determined delimiter of CSV input is ','\u001b[0m\n",
      "\u001b[31m[12:57:15] S3DistributionType set as FullyReplicated\u001b[0m\n",
      "\u001b[31m[12:57:15] 112x13 matrix with 1456 entries loaded from /opt/ml/input/data/validation?format=csv&label_column=0&delimiter=,\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 8 extra nodes, 0 pruned nodes, max_depth=3\u001b[0m\n",
      "\u001b[31m[0]#011train-rmse:19.3632#011validation-rmse:18.8053\u001b[0m\n",
      "\u001b[31mMultiple eval metrics have been passed: 'validation-rmse' will be used for early stopping.\n",
      "\u001b[0m\n",
      "\u001b[31mWill train until validation-rmse hasn't improved in 10 rounds.\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 12 extra nodes, 0 pruned nodes, max_depth=3\u001b[0m\n",
      "\u001b[31m[1]#011train-rmse:15.8284#011validation-rmse:15.3715\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 10 extra nodes, 0 pruned nodes, max_depth=3\u001b[0m\n",
      "\u001b[31m[2]#011train-rmse:12.9626#011validation-rmse:12.6214\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 14 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[3]#011train-rmse:10.7185#011validation-rmse:10.5508\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[4]#011train-rmse:8.86813#011validation-rmse:8.84133\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 16 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[5]#011train-rmse:7.36134#011validation-rmse:7.5184\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[6]#011train-rmse:6.19707#011validation-rmse:6.44953\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[7]#011train-rmse:5.30523#011validation-rmse:5.68346\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[8]#011train-rmse:4.60973#011validation-rmse:5.10667\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[9]#011train-rmse:4.02146#011validation-rmse:4.5806\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[10]#011train-rmse:3.55149#011validation-rmse:4.17824\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 24 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[11]#011train-rmse:3.18392#011validation-rmse:3.88569\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[12]#011train-rmse:2.89589#011validation-rmse:3.65829\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 22 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[13]#011train-rmse:2.65742#011validation-rmse:3.51087\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 14 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[14]#011train-rmse:2.51823#011validation-rmse:3.39135\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[15]#011train-rmse:2.37619#011validation-rmse:3.31754\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 22 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[16]#011train-rmse:2.25016#011validation-rmse:3.24278\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 12 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[17]#011train-rmse:2.16371#011validation-rmse:3.1861\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 16 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[18]#011train-rmse:2.08104#011validation-rmse:3.11841\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[19]#011train-rmse:2.00517#011validation-rmse:3.07823\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[20]#011train-rmse:1.94873#011validation-rmse:3.03221\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[21]#011train-rmse:1.8943#011validation-rmse:2.99425\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 16 extra nodes, 6 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[22]#011train-rmse:1.83592#011validation-rmse:2.97329\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[23]#011train-rmse:1.78484#011validation-rmse:2.95064\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 6 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[24]#011train-rmse:1.75597#011validation-rmse:2.96291\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 14 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[25]#011train-rmse:1.7214#011validation-rmse:2.97564\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 16 extra nodes, 2 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[26]#011train-rmse:1.67278#011validation-rmse:2.97659\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 0 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[27]#011train-rmse:1.62868#011validation-rmse:2.98112\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 16 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[28]#011train-rmse:1.60692#011validation-rmse:2.99801\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 6 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[29]#011train-rmse:1.56415#011validation-rmse:2.98472\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 16 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[30]#011train-rmse:1.51245#011validation-rmse:2.98574\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 12 extra nodes, 6 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[31]#011train-rmse:1.49467#011validation-rmse:2.98412\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 16 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[32]#011train-rmse:1.45863#011validation-rmse:3.03229\u001b[0m\n",
      "\u001b[31m[12:57:15] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 4 pruned nodes, max_depth=5\u001b[0m\n",
      "\u001b[31m[33]#011train-rmse:1.44134#011validation-rmse:3.03158\u001b[0m\n",
      "\u001b[31mStopping. Best iteration:\u001b[0m\n",
      "\u001b[31m[23]#011train-rmse:1.78484#011validation-rmse:2.95064\n",
      "\u001b[0m\n",
      "\n",
      "2019-01-14 12:57:28 Completed - Training job completed\n",
      "Billable seconds: 46\n"
     ]
    }
   ],
   "source": [
    "session.logs_for_job(training_job_name, wait=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the model\n",
    "\n",
    "Now that the training job has completed, we have some model artifacts which we can use to build a model. Note that here we mean SageMaker's definition of a model, which is a collection of information about a specific algorithm along with the artifacts which result from a training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We begin by asking SageMaker to describe for us the results of the training job. The data structure\n",
    "# returned contains a lot more information than we currently need, try checking it out yourself in\n",
    "# more detail.\n",
    "training_job_info = session.sagemaker_client.describe_training_job(TrainingJobName=training_job_name)\n",
    "\n",
    "model_artifacts = training_job_info['ModelArtifacts']['S3ModelArtifacts']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Just like when we created a training job, the model name must be unique\n",
    "model_name = training_job_name + \"-model\"\n",
    "\n",
    "# We also need to tell SageMaker which container should be used for inference and where it should\n",
    "# retrieve the model artifacts from. In our case, the xgboost container that we used for training\n",
    "# can also be used for inference.\n",
    "primary_container = {\n",
    "    \"Image\": container,\n",
    "    \"ModelDataUrl\": model_artifacts\n",
    "}\n",
    "\n",
    "# And lastly we construct the SageMaker model\n",
    "model_info = session.sagemaker_client.create_model(\n",
    "                                ModelName = model_name,\n",
    "                                ExecutionRoleArn = role,\n",
    "                                PrimaryContainer = primary_container)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Testing the model\n",
    "\n",
    "Now that we have fit our model to the training data, using the validation data to avoid overfitting, we can test our model. To do this we will make use of SageMaker's Batch Transform functionality. In other words, we need to set up and execute a batch transform job, similar to the way that we constructed the training job earlier.\n",
    "\n",
    "### Set up the batch transform job\n",
    "\n",
    "Just like when we were training our model, we first need to provide some information in the form of a data structure that describes the batch transform job which we wish to execute.\n",
    "\n",
    "We will only be using some of the options available here but to see some of the additional options please see the SageMaker documentation for [creating a batch transform job](https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTransformJob.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Just like in each of the previous steps, we need to make sure to name our job and the name should be unique.\n",
    "transform_job_name = 'boston-xgboost-batch-transform-' + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "\n",
    "# Now we construct the data structure which will describe the batch transform job.\n",
    "transform_request = \\\n",
    "{\n",
    "    \"TransformJobName\": transform_job_name,\n",
    "    \n",
    "    # This is the name of the model that we created earlier.\n",
    "    \"ModelName\": model_name,\n",
    "    \n",
    "    # This describes how many compute instances should be used at once. If you happen to be doing a very large\n",
    "    # batch transform job it may be worth running multiple compute instances at once.\n",
    "    \"MaxConcurrentTransforms\": 1,\n",
    "    \n",
    "    # This says how big each individual request sent to the model should be, at most. One of the things that\n",
    "    # SageMaker does in the background is to split our data up into chunks so that each chunks stays under\n",
    "    # this size limit.\n",
    "    \"MaxPayloadInMB\": 6,\n",
    "    \n",
    "    # Sometimes we may want to send only a single sample to our endpoint at a time, however in this case each of\n",
    "    # the chunks that we send should contain multiple samples of our input data.\n",
    "    \"BatchStrategy\": \"MultiRecord\",\n",
    "    \n",
    "    # This next object describes where the output data should be stored. Some of the more advanced options which\n",
    "    # we don't cover here also describe how SageMaker should collect output from various batches.\n",
    "    \"TransformOutput\": {\n",
    "        \"S3OutputPath\": \"s3://{}/{}/batch-bransform/\".format(session.default_bucket(),prefix)\n",
    "    },\n",
    "    \n",
    "    # Here we describe our input data. Of course, we need to tell SageMaker where on S3 our input data is stored, in\n",
    "    # addition we need to detail the characteristics of our input data. In particular, since SageMaker may need to\n",
    "    # split our data up into chunks, it needs to know how the individual samples in our data file appear. In our\n",
    "    # case each line is its own sample and so we set the split type to 'line'. We also need to tell SageMaker what\n",
    "    # type of data is being sent, in this case csv, so that it can properly serialize the data.\n",
    "    \"TransformInput\": {\n",
    "        \"ContentType\": \"text/csv\",\n",
    "        \"SplitType\": \"Line\",\n",
    "        \"DataSource\": {\n",
    "            \"S3DataSource\": {\n",
    "                \"S3DataType\": \"S3Prefix\",\n",
    "                \"S3Uri\": test_location,\n",
    "            }\n",
    "        }\n",
    "    },\n",
    "    \n",
    "    # And lastly we tell SageMaker what sort of compute instance we would like it to use.\n",
    "    \"TransformResources\": {\n",
    "            \"InstanceType\": \"ml.m4.xlarge\",\n",
    "            \"InstanceCount\": 1\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Execute the batch transform job\n",
    "\n",
    "Now that we have created the request data structure, it is time to ask SageMaker to set up and run our batch transform job. Just like in the previous steps, SageMaker performs these tasks in the background so that if we want to wait for the transform job to terminate (and ensure the job is progressing) we can ask SageMaker to wait of the transform job to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform_response = session.sagemaker_client.create_transform_job(**transform_request)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "......................................!\n"
     ]
    }
   ],
   "source": [
    "transform_desc = session.wait_for_transform_job(transform_job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze the results\n",
    "\n",
    "Now that the transform job has completed, the results are stored on S3 as we requested. Since we'd like to do a bit of analysis in the notebook we can use some notebook magic to copy the resulting output from S3 and save it locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform_output = \"s3://{}/{}/batch-bransform/\".format(session.default_bucket(),prefix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completed 2.3 KiB/2.3 KiB (18.1 KiB/s) with 1 file(s) remaining\r",
      "download: s3://sagemaker-eu-west-1-177473201433/boston-xgboost-LL/batch-bransform/test.csv.out to ../data/boston/test.csv.out\r\n"
     ]
    }
   ],
   "source": [
    "!aws s3 cp --recursive $transform_output $data_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see how well our model works we can create a simple scatter plot between the predicted and actual values. If the model was completely accurate the resulting scatter plot would look like the line $x=y$. As we can see, our model seems to have done okay but there is room for improvement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y_pred = pd.read_csv(os.path.join(data_dir, 'test.csv.out'), header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Median Price vs Predicted Price')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm4HHWd7/H3J4eDHBY9iJELgRAUDYJAolF0og4giiOIcRkQl8HRKzrjxoDR4HAFFC5RXGfuDDMoKAqyKBhBVGQMDMoITGICyACPsnNAiJLIkggnyff+UdVJp9PVXb1Ur5/X85wn3dVVXb/unFPf+n1/myICMzMbXlO6XQAzM+suBwIzsyHnQGBmNuQcCMzMhpwDgZnZkHMgMDMbcg4EVpekkLRH+vjfJP2fHijTdElPSBrpdlk6TdI9kg5OH39a0jc6cM4DJD3Qxve7VdIB7Xo/a40DwQBJLxBPS3pOxfZl6cV8RqvniIgPRcTnWn2fSumFZn16cX9c0h2S/rZGOe6LiG0jYl27y9IqSd9K/x+ekPSopKsk7VnEuSLi/0bE/85ZplOLKEP6/iHpyfQzT0j6cq0gHRF7R8Q1RZXHGuNAMHjuBo4qPZG0D7B194rTkAcjYlvgmcCngK9L2qtyJ0lbdLxkjftC+ll2AR4BvlVtpz75LHntl37m1wLvBD5QucOAfd6B4UAweL4D/E3Z86OBb5fvIOkZkr4o6T5JD6fpnrGy1+dLekjSg5LeV3HshjtLSdtL+pGkFZJWpo93Kdv3Gkmfk3Rdepf/s8raSjWRWASsBPaSNCO943y/pPuAxWXbtkjP9WxJ30zLvFLSorJyHCZpuaRVkv5L0r7VzivpTElfrNj2Q0nHpY8/ld7tlmosr83xWVYD3wVenL7HyZK+L+k8SY8B75U0RdICSXdK+qOkiyU9u6wM75F0b/raP1aU72RJ55U9f1X6GVdJul/SeyUdA7wL+GR6x355uu/Oki5J///ulvSxsvcZS/+vV0r6H+Bl9T5r2We+HfhF2We+J/3ubgaelLRFRXprJE1x3Zl+t0sl7Zq+tmdao3o0/c6PyFsOy8+BYPBcDzxT0ovSqvk7gPMq9lkIvBCYBewBTAM+AyDpDcAngNcBLwAOrnGuKcA3gd2A6cAa4P9V7PNO4G+B5wJbpu9dU3phfAswDtxS9tJfAi8CDqly2HdIaj57p+f6Svpes4FzgA8COwD/Dlwm6RlV3uMC4EhJSo/dHng9cKGkmcBHgJdFxHZpGe7J8Vm2JbkILyvb/Gbg++nnOx/4KDAv/Xw7kwTAf0mP3ws4E3hP+toOJLWMaufaDfgJ8M/AVJL/3+URcVZ6ni+k6bQ3SZoCXA7cRPL//1rgWEml7/Yk4PnpzyEkNxS5pGV+dcVnPgo4FBiPiLUVhxyXvv5Gktrg+4DVkrYBriIJpM8l+V3+12q1RGtRRPhnQH5ILkwHAycCpwNvIPlD2gIIYAYg4Eng+WXHvRK4O318DrCw7LUXpsfukT7/FnBqxvlnASvLnl8DnFj2/O+Bn2YcewCwHlgFPAosB96RvjYjLcPzyvYvbdsC2Ck9dvsq73sm8LmKbXcAf1llXwH3Aa9Jn38AWJw+3oMkxXMwMFrn/+FbwJ/Tz/J74LLS9w2cDFxbsf9twGvLnu8ETKaf7TPAhWWvbQM8DRxc9n7npY9PAH5Qo0ynlj3fH7ivYp8TgG+mj+8C3lD22jHAAzU+cwCPkQSxO4FTgSllv5fvq/a7Wvb/8eYq73kk8IuKbf8OnNTtv7VB+3G+bjB9B7gW2J2KtBDJneLWwNL0xheSC2CpYW9nYGnZ/vdmnUTS1iR33m8Atk83bydpJDY24v6+7JDVwLY1yv1gRFS9203dn7F9V+DRiFhZ5bXdgKMlfbRs25Ykn3MTERGSLiS5O72WpDZzXvra7yQdS3Lh3VvSlcBxEfFgRpm+GBEn5vwcuwE/kLS+bNs6YMe0nBv2j4gnJf0x4313JbkI57EbsLOkVWXbRkhSOlSelxq/B2VeEhG/y3gt6/8Ossu9G7B/RRm3IPn9tjZyamgARcS9JI3GbwQurXj5DyQpnL0jYjz9eVYkjXwAD5H8YZZMr3Gq44GZwP4R8UzgNel2ZR/Skqypcu8Hni1pPOO108o+63hEbB0RF2S81wXA29M0y/7AJRtOHvHdiHgVyQUqgM+36XPcD/xVRRm3iogJKv4/0uC7Q8b73k+Sysl7zrsrzrldRLwxfb2R34M8ak1znFXu+4H/rCjjthHxdy2WxSo4EAyu9wMHRcST5RsjYj3wdeArkp4LIGlaWW74YpIGzL3Si85JNc6xHUlQWZU2btbatzAR8RBJbvxf0wbsUUmloPR14EOS9ldiG0mHStou472WkQTLbwBXRsQqAEkzJR2Uti38meRzr6/2Hk34N+C0NPggaaqkN6evfR84LG0E3hL4LNl/t+cDB0s6Im2Q3UHSrPS1h4Hnle17I/B42og7ljbYvlhSqVH4YuCE9PvchaQdoyjfAD4n6QXp/9G+knYAfgS8MG0sH01/XibpRQWWZSg5EAyoiLgzIpZkvPwp4HfA9Up6rvwHyZ09EfET4KvA4nSfxTVO81VgjOTCeT3w0/aUvinvIcmr306Syz8WIP0OPkDSiL2S5DO9t857fZekLeC7ZdueQdLI/geSdNdzSXLq7fA1knaEn0l6nOS73D8t/63Ah9OyPJR+hqoDuyLiPpJa4PFsbGfZL335bJIeWKskLUpTd4eRtOvczcbg96x0/1NI0kF3Az+j2HTMl0kCz89I2hnOBsYi4nGSxvp3AA+SfO+fJ/m/sDZShBemMTMbZq4RmJkNOQcCM7Mh50BgZjbkHAjMzIZcXwwoe85znhMzZszodjHMzPrK0qVL/xARU+vt1xeBYMaMGSxZktUT0szMqpGUZ0S4U0NmZsPOgcDMbMg5EJiZDTkHAjOzIedAYGY25Pqi15CZWactWjbBGVfewYOr1rDz+BjzD5nJvNnTul2sQjgQmJlVWLRsghMuvYU1k8n6ShOr1nDCpcmqqYMYDJwaMjOrcMaVd2wIAiVrJtdx/MU3sfuCK5i7cDGLlk10qXTt5xqBmfW9dqdxHly1pur2dem0/UXXEDqdlnKNwMz6WimNM7FqDcHGi3Qrd+w7j4/V3WfN5DrOuPKOps+RpYjPU48DgZn1taw0TisX6fmHzGRsdKTuflk1h1YU8XnqKTwQpGuhLpP0o/T5tyTdLWl5+jOr3nuYmWXJuhi3cpGeN3sap791H6aNjyFgRKq6X56aQ6OK+Dz1dKKN4OPAbcAzy7bNj4jvd+DcZtYh3epuufP4GBNVLpKtXqTnzZ62ofyVvYgAxkZHmH/IzJbOUU1Rn6eWQmsEknYBDiVZFNvMBlQ38tol1dI47b5IV9YQpo2Pcfpb9ykk0HXi81QqdPF6Sd8HTge2Az4REYdJ+hbwSuAp4OfAgoh4qsqxxwDHAEyfPv2l996bazZVM+uCuQsXV72LnTY+xnULDir8/IM2+Ktdn0fS0oiYU3e/ogKBpMOAN0bE30s6gI2BYCfg98CWwFnAnRHx2VrvNWfOnPB6BGa9a/cFV1DtSiLg7oWHdro4TRm0YAL5A0GRqaG5wOGS7gEuBA6SdF5EPBSJp4BvAi8vsAxm1gFZ+esi89rt1M3UVi8oLBBExAkRsUtEzADeASyOiHenNQIkCZgH/KaoMphZZ3Qjr91O3eiy2Uu6MbL4fElTSWqNy4EPdaEMZtZGpRRKVmql19Mu3eiy2Us6Eggi4hrgmvRx8S1HZtZx5d0ty/XDBG7d6LLZSzyy2MwK1Q9pl35PbbXKk86ZWaH6Ie1SL7U16BwIzKxQ/ZJ2yUptDQOnhsysUMOedukHrhGYWaGGPe3SDxwIzKxww5x26QdODZmZDTkHAjOzIedAYGY25BwIzMyGnAOBmdmQcyAwMxtyDgRmZkPOgcDMbMg5EJiZDTkHAjOzIedAYGY25Aqfa0jSCLAEmIiIwyTtTrKY/Q7AUuA9EfF00eUwM+sXnV7asxM1go8Dt5U9/zzwlYjYA1gJvL8DZTAz6wulpT0nVq0h2Li056JlE4Wds9BAIGkX4FDgG+lzAQcB3093OReYV2QZzMz6STeW9iw6NfRV4JPAdunzHYBVEbE2ff4AULW+I+kY4BiA6dOnF1xMM6un0+mKYdWNpT0LqxFIOgx4JCKWNnN8RJwVEXMiYs7UqVPbXDoza0Q30hXDKmsJzyKX9iwyNTQXOFzSPSSNwwcBXwPGJZVqIrsA/k0ya7NFyyaYu3Axuy+4grkLF7d8we5GumJYdWNpz8ICQUScEBG7RMQM4B3A4oh4F3A18PZ0t6OBHxZVBrNhVMTdezfSFcNq3uxpnP7WfZg2PoaAaeNjnP7WfQpNw3VjqcpPARdKOhVYBpzdhTKYDaxad+/NXkx2Hh9jospFv8h0xTDr9NKeHRlQFhHXRMRh6eO7IuLlEbFHRPx1RDzViTKYDYsi7t67ka6wzvHi9WYDpoi799LdaZ5eQ+5d1H8cCMwGzPxDZnLCpbdskh5qx917nnRFqX2idO5S+0TpeOtNDgRmA6aRu/d2K6J9Yhh1ulblQGA2gDrd2FjSbPuE00kbdaNW5dlHzaxtmhkM5cFqm+rGmA0HAjNrm2Z6F3mw2qYGaooJMxs+zQyG8mC1TXVjigm3EZhZWzXaPuHBapsqqtdXLa4RmFlXebDapoZligkzsw262d21V3W615cDgZl1Xbe6u1rCgcDMMrl/f3d4QJmZ9QRPF9EdHlBmZm3X7CI17t/fHYO4ZrGZdVErd5fu398dHlBmZm3Vyt1lNwY22YCtWSxpK0k3SrpJ0q2STkm3f0vS3ZKWpz+ziiqD2bBr5e7S/fu7oxvfe5GpoaeAgyLiCUmjwC8l/SR9bX5EfL/Ac5sZrY3adf/+7ujG915YIIiIAJ5In46mP1HU+cxsc61OV+D+/d0xUGsWSxqRtBx4BLgqIm5IXzpN0s2SviLpGUWWwWyYdWO6Aus/Sm7cCz6JNA78APgo8Efg98CWwFnAnRHx2SrHHAMcAzB9+vSX3nvvvYWX08xskEhaGhFz6u3XkV5DEbEKuBp4Q0Q8FImngG8CL8845qyImBMRc6ZOndqJYppZg5odo2C9pbA2AklTgcmIWCVpDHgd8HlJO0XEQ5IEzAN+U1QZzIZFN6aC8MjjwVFYakjSvsC5wAhJzePiiPispMXAVEDAcuBDEfFE9jvBnDlzYsmSJYWU06zfVV6QAUaniG232oJVqycLCwxzFy6u2iNp2vgY1y04qK3nsubkTQ0V2WvoZmB2le3+DbGh1u6792qDxibXBytXTwLF3al75HFxOl3Dq9tGoMS7JX0mfT5dUtW8vpnVVsRC7XkuvEXMVeORx8Uo4neknjyNxf8KvBI4Kn3+OPAvhZXIbIAVMaFY3gtvu+/UPfK4GN2YdC5PINg/Ij4M/BkgIlaSdP00swYVkU6pdkGupt136h6jUIxupNzytBFMShohHRWc9gZaX1iJzAZYEQu1V05J8KyxUZ58ei2T6zZ2BCnqTt0jj9uviN+RevIEgn8iGQz2XEmnAW8HTiysRGYDrNkpH+o1HlZekL2yWP9qdVqQZuTqPippT+C1JF0+fx4RtxVWoircfdQGSaMX6WrdQ8dGR5yGGWDtCuR5u4/WDQSSXgHcGhGPp8+fCbyobN6gwjkQ2DBzf31rVjunmDiTjbOIkj4+s9mCmVlj3F/fipYnECjKqg0RsR4vcWnWMe6vb0XLEwjukvQxSaPpz8eBu4oumFkrBmkyNPfXt6LlCQQfAv4CmAAeAPYnnR7arBd1Y2Rmkdxf34rWkfUIWuXGYstr0bIJjr/4JtZV+b1246oNm5YnnZP0yYj4gqR/psoSkxHxsRbLaNZWpZpAtSAAblwt53EGVq5Wo29prIBvxa0vVJujpZwbVxNeR8AqZQaCiLg8nVpin4j4RAfLZNaUWnf8blzdqNakZg4Ew6lmY3FErAPmdqgsZi3JuuMfkdy4WsbjEqxSnl5DyyVdJuk9kt5a+im8ZGYNyupm+aUj9nMQKONxCVYpTyDYCvgjcBDwpvTnsHoHSdpK0o2SbpJ0q6RT0u27S7pB0u8kXSTJU1pbW7ibZT4el2CV8sw19JyI+EPDb5wsTr9NRDwhaRT4JfBx4Djg0oi4UNK/ATdFRM0pK9x91Cy/PD2C3GtoOLSj++ibgHNI1iNYDxwREf+VtwDptBSlOYpG058gqVm8M91+LnAynrvIrC3y9gjyOgJWrlZq6DTg1RGxM/A24PRG31zSiKTlwCPAVcCdwKqIWJvu8gBQ9bdR0jGSlkhasmLFikZPbdYW/TZVRTeWObT+V2scwdqIuB0gIm6QtF2jb572OpolaZxkcZs9Gzj2LOAsSFJDjZ7brJpGUiL92N/ePYKsGbUCwXMlHZf1PCK+nPckEbFK0tXAK4FxSVuktYJdSOYwMitcoxf2bve3byaP341lDq3/1UoNfR3Yruyn8nlNkqamNQEkjQGvIxmtfDXJcpcARwM/bLbwZo1oNG3SzbvrZifOc48ga0atkcWntPjeOwHnpqOTpwAXR8SPJP0PcKGkU4FlwNktnscsl0Yv7N28u262NlK5kL17BFkehS0wExE3A7OrbL8LeHlR5zXL0uiFvV2LiDeT4mmlNuIeQdaoPAPKzAZCo2mTdgxQazbF0yujf/ut15Q1x0tO2tBoJm3S6t11symedtVGWtGPvaasObUGlB2X9Ro01mvIrFd0Om3SbIqnF3L93e41ZZ1Tq0ZQ6hk0E3gZcFn6/E3AjUUWymxQtNLg3O1cv8ckDI+6vYYkXQu8JCIeT5+fDFzRkdKZVdFP8+T0QoqnWR6TMDzyNBbvCDxd9vzpdJtZx/XbwvT9PCOqxyQMjzyNxd8GbpT0g/T5PJLJ4sw6rh/z1t1O8TSrF9ophlWna711A0FEnCbpJ8Cr001/GxHLCiuRWQ158tb9lDrqdf0axPpZN3pr5R1HsDXwWER8DXhA0u6FlMasjnr96/stdWRWqRszyNYNBJJOAj4FnJBuGgXOK6xEZjXUy1t7Gmbrd93orZWnjeAtJFNF/BogIh5sZkpqs3aol7fu5S6PnUhZOS3W/7rRWytPIHg6IkJSAEjaprDSmOVQK2/dK10eKy/IB+45lUuWThSa9/VI4MHQjS7HedoILpb07yTrCHwA+A/gG4WVyKwFebo8Fj1/TrV2ivOuv6/wlJXTYoOhG12O8/Qa+qKk1wGPkYwy/kxEXFVYicxaUC911Im75moX5CztTFn1clrMGtPp3lp1A4Gkz0fEp0jWHK7cZtZzav0RdWIcQiMX3lZTVuUpqCkS62LzVV09EtjqyZMael2VbX/V7oKYdUIn7przXnhbzftWpqCqBQGPBLY8MgOBpL+TdAuwp6Sby37uBm7pXBHN2ifrIj1FaltbQbV2impazftmpaBGpL6bzsK6q1Zq6LvAT4DTgQVl2x+PiEfrvbGkXUmmp9gRCOCsiPhaOmndB4AV6a6fjogfN1F2s9xKKZRqPYoguZtuV1tB6fiTL7uVVWsmq+4zbXys5fNk1WLWR3D3wkNbem8bLrVmH/0T8CdJXwMeLZt99JmS9o+IG+q891rg+Ij4dTruYKmkUjvDVyLii+34AGb1VDYQZ2lHW0Flt9G9d96O/7rzUcqTNu1ICZ1x5R1snghKuE3AGpVnHMGZwEvKnj9RZdtmIuIh4KH08eOSbgNcR7WWNTpoqlO9eKr1SHr0yad51yumc/XtK9oyyKteUHObgDUjTyBQxMZWqIhYL6mhJS4lzSAZnXwDMBf4iKS/AZaQ1BpWVjnmGOAYgOnTpzdyOhtgzXT/7FQvnqweSVffvoLrFhxU9Zh2BrVpHklsTcpzQb9L0sdIagEAfw/clfcEkrYFLgGOjYjHJJ0JfI6k3eBzwJeA91UeFxFnAWcBzJkzJ6sWbEOgXhfJeimdrNHGlVq9m260R1I7g5ogM9iY1ZOn++iHgL8AJoAHgP1J79TrkTRKEgTOj4hLASLi4YhYFxHrga8DL2+m4DYc8nSRhNp3/fMPmcnoFNU8Tzt62NSbGbVSMyOBGz2HWR51A0FEPBIR74iI50bEjhHxzoh4pN5xkgScDdxWvtC9pJ3KdnsL8JtmCm7DIW9+v9qF8MRFt/D8E37MsRctZ3J9dqVydERtSak0uqJXM2MavGqYFSEzNSTpkxHxBUn/DJt3UIiIj9V577nAe4BbJC1Pt30aOErSrPQ97wE+2EzBbTjkze+vfnoti5ZNbLiYn7joFs67/r5cx06ui7aMLM67olcrvX68apgVoVYbwW3pv0uaeeOI+CVJ6rKSxwxYbln5fbHp3cnK1ZMce9FyTr7sVk4+fG8uuOH+hs6Tpw0B6jfu1psjph29frxqmLVbrXEEl6f/en1i65pqU/KOTkkajKs1F6xaM8lxFy1nfYPnqdOEADTWuJsVMNzrx3pRrdTQ5VRJCZVExOGFlMisTGUq5Fljozz59FrW17jSNxoEALKaEJrpsVQrYLjXj/WiWqmh0sjftwL/i43LUx4FPFxkoczKladC5i5cnDltQ7tVXtDz9liq1RuoVxbOMSuX2WsoIv4zIv4TmBsRR0bE5enPO4FXd66IZolFyyZy5/IbNT42utm2Znss1eoN5F4/1ovyDCjbRtLzIuIuAEm7A16u0gqVtdRjEUaniJMP33uz7c0OQqt11+9eP9aL8gSCfwCukXQXSSpzN9zl01pUq/fNiYtu4fzr79vQQDWxas0mz9spq4F20bKJzXomlYxIrI/IvIjXW3PWvX6s1+RZqvKnkl4A7Jluuj0iniq2WDbIajWmAlX7/xcVBLIaaLP6+Qv40hH71byQ+67f+k2epSq3Bo4DdouID0h6gaSZEfGj4otngyirMfX4i29iq9E8s560rl5ePivPH+Rbr8B3/dZP8vzVfRN4Gnhl+nwCOLWwEtnAy7rIrovgyafzTRfdqPGxUaaNj+VeuSurF8809+6xAZSnjeD5EXGkpKMAImJ1Oo+QWVPyzgbaLmOjI5x8+N4N3aHXy/ObDZI8NYKnJY2RpmklPR9wG4E1bf4hM6vOPVJLo/u3um7vvNnTOP2t+zRUizDrV3lqBCcBPwV2lXQ+yWRy7y2yUDaYynsKNdr4u9XoFNauDybXbTxydEQQwWTFUOLRKeKMv97YoLto2QRzFy5uuOHWeX4bFjUDQZoCup1kdPErSG7MPh4Rf+hA2azPlS8YP0XZ0ziUE/CuV0znipsfYuXqjSOI10yuZ3SK2H7rUVatntwwtuCiG++nsk9ReVxoZvEXs2FTMzWULlH544j4Y0RcERE/chCwPMoXlIHGgsCp8/Zh6y03v0eZXB9sveUW3L3wUK5bcBBX376i6joD69bHhsVdmln8xWzY5EkN/VrSyyLivwsvjQ2MUy6/NfeC8ZD06pHg/Ovv4+rbV2Q2Jpf3OKq1VkHptWYWfzEbNnkCwf7AuyXdAzxJOhV8ROxbZMGsNzS6uHrpmPK0Th5PrV2/SfomS3m3zlq9j0r7eZI3s/ryBIJDmnljSbsC3wZ2JEninhURX5P0bOAiYAbJCmVHRMTKZs5hxWp2/v1GjUi5ag+V3TfnHzKT+d+7abP00MgU8eRTa9l9wRWMbz3K6BRtso+7gZptqtZ6BFuRLFy/B3ALcHZErG3gvdcCx0fEryVtByyVdBVJj6OfR8RCSQuABcCnmv0AVpxa+fVa8+83Ymx0JNdx5XMClQed8a1H+fPkOtakXYe22XKEp9eu3zBV9crVk4yOiLHRKRv26dToZbN+UatGcC4wCfwC+CtgL+Djed84Ih4CHkofPy7pNmAa8GbggLJzXIMDwSaaSccUIW9+Pe90zZVKNYE8PYoeXLWGM668gyX3PsolSyc2nG/l6knGRkf46pGzmDd7GnMXLt4sFTS5Llhb1u105epJ9xwyK1MrEOwVEfsASDobuLHZk0iaAcwGbgB2TIMEwO9JUkeW6kZ3x6zAkze/3mzDa2mhlzw9ioLsWUhL8xT9w0XLM8cnVDumHQvWmw2CWnXkDa19DaaENiFpW+AS4NiIeKz8tbR7atW/XUnHSFoiacmKFSuaPX3f6XR3x/JunqWL7QmX3sKiZRO5F1HpZMNr1oV+XUTDg9Ra6TlUGqS2+4IrmLtwMYuWFbNWglkn1AoE+0l6LP15HNi39FjSYzWO20DSKEkQOD8iLk03Pyxpp/T1nYBHqh0bEWdFxJyImDN16tT8n6jPtdrdsdELVFbgOeXyW3NPs1AtYJQGf/WCrOkpmg1gtYKnWT/KTA1FxEjWa3mko5LPBm6LiC+XvXQZcDSwMP33h62cZ9C00t2xmbRSVoBZuXqSRcsmck2zUHr9lMtv3dBtdJtnbMFJb9p7w8jiZoxUWSy+EYJNVjdr1wRyeRvRzfpFkd0n5gLvAQ6StDz9eSNJAHidpN8CB6fPLdXKmrbNpJVqBZjK4+rVNv5cNunPqjVJg+yBe05N5gRqQrUgkPedpo2PbRiBfOq8fdo6gZwHqdmgyTOOoCkR8Uuy/25fW9R5+10rq1s1c4Gaf8hMjr1oed3j6tU2soLQ1bev4Iy377dJbaFZ03JOX10tcLZzAjkPUrNB4w7VPWje7Glct+CgDXe0eS9gWReiWheoebOnMT5WPZdffly92katIDRv9jSWfeb1fPXIWZvVdgAkePcrpte82y9d3LMWhlH604npoluptZn1IgeCAdLsBerkw/euedyiZRN15/7JE4SqNT5/9chZ3H36oZw6b5+aAasUdOYfMpPRKZuHjC1GxFeOnNVQ4GyW1yqwQVNYasg6r9m0Uq3jSimhLKWLd7UVvQQcuOemPb5qpWiqvUe5Uu2iWpppcl10tLHWaxXYIHEgGDDNXqCyjqs3arh0oZ83expL7n10kwFfAVyydII5uz0790IwAMdffFPVhuJS0FmV0dbgxlqz5jg1ZDXVu7heffuKTR5njeDNa97saXzpiP1qpqqaaQsxs2wOBFZTvYvrg6vWbOhWmmcNgTzq5eB7obHWI4ttkChaGLDTKXN8DrUHAAANbUlEQVTmzIklS5Z0uxhDadGyiZpz+Gy/9Sh/nlxfM30kAUFbJ9Dr5sR81WZbHRsdcYOx9RxJSyNiTt39HAisnhkLrsh8bXxsdMOUz3kMwgUzq/YzbXyM6xYc1IUSmVWXNxA4NWR1ZfXd337rUf7UQBCAwVgv2COLbdA4EPSYZnPPReass3LyJ71p76YaaPv9gunGahs07j7aQ5pdiyDPca3k1OuNT2h0dbJ+XyGs2ngHjyy2fuZA0EOandWy3nEnLrplk/79zSx2kzXOoDJIPGtsFCnp65/V+vTU2vUZr/SHVuaDMutFDgQdkPduvNncc63jFi2byFzV6+TLbm3LxSsrSGQ1MudZkazXeWSxDZL+rqP3gUYWMWk291zruJMvuzXzznzVmslC+7+PqPo0clnbzaw7HAgK1sgaAc0OlMo67sA9p9bt2llkD56j9t+1oe1m1h1ODRWskXRPuyeNy3ORz9ODp9mG5lPn7QPABTfcz7oIRiSO2n/XDdvNrDd4QFnB2jX4qNrFGDZdHnJ8bJSTD997w0V69wVX1F3QvVSOrIt9tVG0oyNimy234E9rJt1QatbD8g4oK6xGIOkc4DDgkYh4cbrtZOADQGmmsk9HxI+LKkMvaEdXw2rdQ+d/7ybWA+vKWl5XrZlk/vduApJaQtZKWpXlqNX9tFpqa3JdbEg5lcpSOqeZ9Z/CagSSXgM8AXy7IhA8ERFfbOS9+rlGAK3Pi1NrQrdqRiS+dMR+wOZ9/NNpf5iijb13StsqTRsf48G0kbue8bFRlp/0+txlNLPidb1GEBHXSppR1Pv3k1a7GjY6EnddBCdcegunvzVZtL08CB2451Qu+u/7mVy38fKedaEvHZMnCDUy35CZ9ZZu9Br6iKSbJZ0jafusnSQdI2mJpCUrVqzI2m0oNDN1QfmAsvL1j6++fcUmQaDeeav1SDKzwdLpQHAm8HxgFvAQ8KWsHSPirIiYExFzpk6dmrXbwKk2Z1CzF+NqNYm8tYtS+0Hl2gBmNng6Gggi4uGIWBcR64GvAy/v5Pl7XdbgM6Cpi3G1mkSt2sWIVHUhmPJaxfZbj1Y9Nmu7mfW+jgYCSTuVPX0L8JtOnr/X5Rl8lrdpP6tn0vxDZjI6snk4GZ2SNDCXUkhZbRonvWlvRqZsevzIFHHSm/bOWTIz6zVFdh+9ADgAeI6kB4CTgAMkzSK5nt0DfLCo8/ejrLRNqWZQb4bPUu+faTV6JpW21Rp/UM8UYF3FczPrX0X2GjqqyuazizpfpzXSJTTvvlk9dEakXNM8B5vm9rO00ovpjCvvYLJi1rjJ9VF3hlQz612+mWtCIxPJNbJv1pxB6xoY61H0CmBencts8DgQNKGRieQa2beyh06p0TZrqcgsRV6UvTqX2eDxpHNNaOSuuNE76Ky0TdYI4WrGC+zB49W5zAaPawRNaOSuuB130NVqCl85chbjY9Uv+EXOI5hVa3H7gFn/co2gAaVG34lVaza7I6/VXbMdd9DVagr/cNHyqvv+qeDpHrw6l9lgcSDIqXKGzqCx7ppFrG+b1cvI+Xoza4QDQU7VGn1LQaDeugKVd9ClaSRaDQzzD5nJ/O/dtEl3ztEpcr7ezBriQJBTu7pN1pr7v6laQuUgYU8IZGYNcmNxTu3qNpm3O2m1yeeqvVflTKKT66LQcQRmNngcCHJqdmH5SnlqFnkHoXlwl5m1gwNBTu3qNpmnZpG31uDBXWbWDm4jaEA7uk3m6U6a906/2nuNThGrn17L7guu8MLyZpaLawQdlqdmkfdOv/K9xsdGQbBy9WTdeY3MzEoKW7y+nfp98fpGVfYsgqTWUC8VlbXIfZ4urmY2eLq+eL01r9lBaG48NrNmOBD0qGbaIzzS2Mya4TaCAdKuLq5mNlwKCwSSzpH0iKTflG17tqSrJP02/Xf7os4/jDwzqJk1o7DGYkmvAZ4Avh0RL063fQF4NCIWSloAbB8Rn6r3Xr3cWNzIkpVmZp2Ut7G4sBpBRFwLPFqx+c3Auenjc4F5RZ2/ExpZhtLMrFd1uo1gx4h4KH38e2DHrB0lHSNpiaQlK1as6EzpUnnm+YHGlqE0M+tVXWssjiQnlZmXioizImJORMyZOnVqx8rVyF2+u2ua2SDodCB4WNJOAOm/j3T4/HU1cpfvuX7MbBB0OhBcBhydPj4a+GGHz19XI3f5B+45dbPp/91d08z6TZHdRy8AfgXMlPSApPcDC4HXSfotcHD6vKfkvctftGyCS5ZObJLbEvC2l3o9XzPrL4WNLI6IozJeem1R52yHvIvNZy1defXtnW3YNjNrlaeYqJB3nh83FJvZoBjYQNDKQK888/x4Xh8zGxQDOddQJwZ6eV4fMxsUA1kjqNUFtF0Nuc1MFe3pKMysFw1kIOhU/r6RqaIrF5sp1VJK72Nm1i0DmRrqxYFeno7CzHrVQAaCXszfu5eRmfWqgQwEvTgvfy/WUszMYEDbCKC5pR6LlHegmplZpw1sIOg1zS5Ib2ZWNAeCDuq1WoqZGQxoG4GZmeXnQGBmNuQcCMzMhpwDgZnZkHMgMDMbckrWkO9tklYA93a7HC16DvCHbheih/j72Mjfxab8fWzU6nexW0RMrbdTXwSCQSBpSUTM6XY5eoW/j438XWzK38dGnfounBoyMxtyDgRmZkPOgaBzzup2AXqMv4+N/F1syt/HRh35LtxGYGY25FwjMDMbcg4EZmZDzoGgAJLOkfSIpN+UbXu2pKsk/Tb9d/tulrFTJO0q6WpJ/yPpVkkfT7cP6/exlaQbJd2Ufh+npNt3l3SDpN9JukjSlt0ua6dIGpG0TNKP0ufD/F3cI+kWScslLUm3Ff634kBQjG8Bb6jYtgD4eUS8APh5+nwYrAWOj4i9gFcAH5a0F8P7fTwFHBQR+wGzgDdIegXweeArEbEHsBJ4fxfL2GkfB24rez7M3wXAgRExq2z8QOF/Kw4EBYiIa4FHKza/GTg3fXwuMK+jheqSiHgoIn6dPn6c5A9+GsP7fUREPJE+HU1/AjgI+H66fWi+D0m7AIcC30ifiyH9Lmoo/G/FgaBzdoyIh9LHvwd27GZhukHSDGA2cAND/H2kqZDlwCPAVcCdwKqIWJvu8gBJsBwGXwU+CaxPn+/A8H4XkNwU/EzSUknHpNsK/1vxCmVdEBEhaaj67UraFrgEODYiHktu/BLD9n1ExDpglqRx4AfAnl0uUldIOgx4JCKWSjqg2+XpEa+KiAlJzwWuknR7+YtF/a24RtA5D0vaCSD995Eul6djJI2SBIHzI+LSdPPQfh8lEbEKuBp4JTAuqXRjtgsw0bWCdc5c4HBJ9wAXkqSEvsZwfhcARMRE+u8jJDcJL6cDfysOBJ1zGXB0+vho4IddLEvHpDnfs4HbIuLLZS8N6/cxNa0JIGkMeB1Ju8nVwNvT3Ybi+4iIEyJil4iYAbwDWBwR72IIvwsASdtI2q70GHg98Bs68LfikcUFkHQBcADJFLIPAycBi4CLgekkU2ofERGVDcoDR9KrgF8At7AxD/xpknaCYfw+9iVp8BshuRG7OCI+K+l5JHfFzwaWAe+OiKe6V9LOSlNDn4iIw4b1u0g/9w/Sp1sA342I0yTtQMF/Kw4EZmZDzqkhM7Mh50BgZjbkHAjMzIacA4GZ2ZBzIDAzG3IOBNb3JIWk88qebyFpRWk2ywbe5xpJc9LHPy7192+xbO9Ny7I8nYH1Axn7zZH0T62ez6wZnmLCBsGTwIsljUXEGpJBWi2NRo2IN7alZImLIuIj6bQBt0q6LCIeLr0oaYuIWAIsaeM5zXJzjcAGxY9JZrEEOAq4oPRCOmLznHQdgGWS3pxuH5N0oaTbJP0AGCs75h5Jz0kfL0onAbu1bCIwJD0h6bR0bYHrJdWcDCydNuBOYDdJJ0v6jqTrgO9IOqBsPv5tJX0znZf+ZklvS7e/XtKvJP1a0vfS+ZvMWuZAYIPiQuAdkrYC9iUZuVzyjyTTF7wcOBA4Ix3C/3fA6oh4Ecno75dmvPf7IuKlwBzgY+lIT4BtgOvTtQWuBaqmfUrSkaPPA36XbtoLODgijqrY9f8Af4qIfSJiX2BxGpROTPd/CUnt4bha5zPLy6khGwgRcXM6zfVRJLWDcq8nmdzsE+nzrUiG678G+Key42/OePuPSXpL+nhX4AXAH4GngVI7xFKSlFQ1R6ZTbTwFfDAiHk1nX70sTWVVOphk7p3SZ1uZztS5F3BdeuyWwK8yzmfWEAcCGySXAV8kmedph7LtAt4WEXeU71w+FXaWdA6cg4FXRsRqSdeQBBKAydg4R8s6sv+eLoqIj1TZ/mTdApQVBbiqSu3BrGVODdkgOQc4JSJuqdh+JfDRdCZUJM1Ot18LvDPd9mKSlFKlZwEr0yCwJ8lym0W7Cvhw6Um6Ru31wFxJe6TbtpH0wg6UxYaAA4ENjIh4ICKqdcH8HMmSkDdLujV9DnAmsK2k24DPkqR3Kv0U2CLdZyHJBblopwLbS/qNpJtI1rBdAbwXuCBNYf2KIV3QxtrPs4+amQ051wjMzIacA4GZ2ZBzIDAzG3IOBGZmQ86BwMxsyDkQmJkNOQcCM7Mh9/8BASSybH03UhEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(Y_test, Y_pred)\n",
    "plt.xlabel(\"Median Price\")\n",
    "plt.ylabel(\"Predicted Price\")\n",
    "plt.title(\"Median Price vs Predicted Price\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optional: Clean up\n",
    "\n",
    "The default notebook instance on SageMaker doesn't have a lot of excess disk space available. As you continue to complete and execute notebooks you will eventually fill up this disk space, leading to errors which can be difficult to diagnose. Once you are completely finished using a notebook it is a good idea to remove the files that you created along the way. Of course, you can do this from the terminal or from the notebook hub if you would like. The cell below contains some commands to clean up the created files from within the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First we will remove all of the files contained in the data_dir directory\n",
    "!rm $data_dir/*\n",
    "\n",
    "# And then we delete the directory itself\n",
    "!rmdir $data_dir"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p36",
   "language": "python",
   "name": "conda_pytorch_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}