yolov3_on_videos-tests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "InT7_Hnn-_4-"
   },
   "source": [
    "# Using YOLOv3-tiny and OpenCV 4 to detect objects\n",
    "\n",
    "I wanted to try to use OpenCV and YoloV3 in order to detect basics objects, this is what this notebook will be about.\n",
    "\n",
    "*Thank you Joseph Redmon and Ali Farhadi for their incredible work on YOLO object detector* : **YOLOv3: An Incremental Improvement**, *Redmon, Joseph and Farhadi, Ali* ; arXiv, 2018.\n",
    "\n",
    "This paper was my main inspiration for this notebook. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "VgFkipuK-_5E"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenCV version :  4.5.1\n"
     ]
    }
   ],
   "source": [
    "# import useful libraries\n",
    "import os\n",
    "import numpy as np\n",
    "import cv2\n",
    "print('OpenCV version : ', cv2. __version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J011UaOx-_5H"
   },
   "source": [
    "### Load YoloV3 models with pre-trained weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "b-_j5ie8-_5H"
   },
   "outputs": [],
   "source": [
    "# function to load our classes names\n",
    "def read_classes(file):\n",
    "    \"\"\" Read the classes files and extract the classes' names in it\"\"\" \n",
    "    classNames = []    \n",
    "    with open(file, 'rt') as f:\n",
    "        classNames = f.read().rstrip('\\n').split('\\n')\n",
    "   \n",
    "    return classNames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6lNsDmSE-_5I",
    "outputId": "9915621d-8c15-417f-ea2e-93befabc392c"
   },
   "outputs": [],
   "source": [
    "# test our function read_classes\n",
    "img_file = './data/coco.names'\n",
    "classNames = read_classes(img_file)\n",
    "# classNames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "17jo_5tzLNiZ"
   },
   "source": [
    "### YoloV3-tiny weights and cfg files\n",
    "\n",
    "In this part we'll upload YoloV3 Weights and cfg files in order to input them into ***cv2.dnn.readNetFromDarknet()*** and build our forward propagation with OpenCV."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 263
    },
    "id": "sYmLADJ2LNSA",
    "outputId": "aba64119-7ff5-4336-a35f-acc1016a9800"
   },
   "outputs": [],
   "source": [
    "# load the model config and weights\n",
    "modelConfig_path = './cfg/yolov3-tiny.cfg'\n",
    "modelWeights_path = './models/yolov3-tiny.weights'\n",
    "\n",
    "# read the model cfg and weights with the cv2 DNN module\n",
    "neural_net = cv2.dnn.readNetFromDarknet(modelConfig_path, modelWeights_path)\n",
    "# set the preferable Backend to GPU\n",
    "neural_net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)\n",
    "neural_net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)\n",
    "\n",
    "# confidence and non-max suppression threshold for this YoloV3 version\n",
    "confidenceThreshold = 0.3\n",
    "nmsThreshold = 0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HCoGUKRILNN1"
   },
   "source": [
    "## Function creation for object detection\n",
    "\n",
    "In this part, we'll define the object detection function which allows us to get **the output of the three YOLO layers (82,94,106)** and select **the best bounding box** to put on the detected object, the layers' outputs are in the format :\n",
    "- The confidence probability (if there is an object or not in the frame)\n",
    "- (x,y) coordinates of the center of the object\n",
    "- (height, width) of the frame\n",
    "- All the scores of the detected object (if there is an object detected, which one is it?)\n",
    "\n",
    "We know that the first 5 elements of the results ouputs are not the \"scores\", **i.e.** : *the 80 classes probabilities of what kind of object is detected*. That's why we'll start to store our outputs layers' values in different lists, the first one will stock all the **scores** from the fifth element to the last element (85th), then we'll collect the first 5 elements. \n",
    "\n",
    "For the 5th to 85th elements corresponding to *the objects scores*, we'll take the maximum value which will represent the **highest probability** about what's the object's class.\n",
    "\n",
    "#### Object detection function\n",
    "\n",
    "Based on what we've detailed above, we'll write an object detection function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def object_detection(outputs, input_frame):\n",
    "    \"\"\" This function will allow us to draw bounding boxes on detected objects in frames (images) \"\"\"\n",
    "    # first we'll collect the height, width and channel of the input frame (3 if the image is RGB, 1 if it's grayscale)\n",
    "    height, width, channel = input_frame.shape\n",
    "    \n",
    "    # we'll create category lists to store the layers' output values \n",
    "    bounding_boxes = []\n",
    "    class_objects = []\n",
    "    confidence_probs = []\n",
    "    \n",
    "    # Knowing that there are 3 YOLO layers, we'll browsing them and their outputs using this :\n",
    "    for result in outputs:        \n",
    "        for values in result:\n",
    "            \n",
    "            scores = values[5:] # we know that the class probabilities are from the 5th values\n",
    "            indices_object = np.argmax(scores) # get the indice of the max score\n",
    "            confidence_probability = scores[indices_object] # store the maximum value of the indice found\n",
    "            \n",
    "            # in order to have a proper detection, we'll eliminate the weakest probability detection by imposing a threshold\n",
    "            if confidence_probability > confidenceThreshold:   \n",
    "                \n",
    "                # get the pixel values corresponding to the scaling of the bounding box coordinates to the initial frame\n",
    "                box_detected = values[0:4] * np.array([width, height, width, height])                                 \n",
    "                # get the top left corner coordinates by extracting values from box_detected and perform calculations\n",
    "                x, y, w, h = box_detected\n",
    "                # we're converting the coordinates to int because OpenCV doesn't allow floats for bounding boxes\n",
    "                x = int(x - (w/2))\n",
    "                y = int(y - (h/2))\n",
    "                \n",
    "                # adding the good detected boxe in the bounding boxes list created\n",
    "                bounding_boxes.append([x,y,w,h])                \n",
    "                # adding the detected objects indices in the class objects list \n",
    "                class_objects.append(indices_object)                \n",
    "                # adding the max value of the object score (confidence) in the confidences_probs list\n",
    "                confidence_probs.append(float(confidence_probability))    \n",
    "    \n",
    "    return bounding_boxes, class_objects, confidence_probs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Non-max suppresion function\n",
    "\n",
    "When we're doing object detection with YoloV3, the algorithm outputs several bounding boxes. In order to keep the best bounding box we'll need to perform a maximizimation technique called **non-max suppression**, this method will allow us to keep the bounding box with the highest probability and eliminate the others."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def nms_bbox(bounding_boxes, confidence_probs, confidenceThreshold, nmsThreshold):\n",
    "    \"\"\"This function performs non-max suppression on all the bounding boxes detected and keeps the best one\"\"\"    \n",
    "    #Using OpenCV DNN non-max supression to get the best bounding box of the detected object (retrieve the indices)\n",
    "    indices_bbox = cv2.dnn.NMSBoxes(bounding_boxes, confidence_probs, confidenceThreshold, nmsThreshold)   \n",
    "    print('Number of objects detected : ', len(indices_bbox), '\\n')\n",
    "    \n",
    "    return indices_bbox"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Drawing box function\n",
    "\n",
    "Now, we have only one bounding box left (the best one) and we'll just need to draw the corresponding box."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def box_drawing(input_frame, indices, bounding_boxes, class_objects, confidence_probs, classNames, color=(0,255,255), thickness=2):\n",
    "    \"\"\" Drawing the detected objects boxes \"\"\"\n",
    "    # once we have the indices, we'll extract the values of x,y,w,h of the best bounding boxes and stores it.\n",
    "    for i in indices:\n",
    "        i = i[0]\n",
    "        final_box = bounding_boxes[i]\n",
    "    # we'll retrieve the bounding boxes values (coordinates) now and use them to draw our boxes.\n",
    "        x, y, w, h = final_box[0], final_box[1], final_box[2], final_box[3]\n",
    "        x, y, w, h = int(x), int(y), int(w), int(h)        \n",
    "        print('Bounding box coordinates in the frame : ', 'x : ', x,'|| y : ',y,'|| w : ',w,'|| h :',h , '\\n')\n",
    "    \n",
    "        cv2.rectangle(input_frame, (x,y), (x+w,y+h),  color, 2)\n",
    "        cv2.putText(input_frame, f'{classNames[class_objects[i]].upper()} {int(confidence_probs[i]*100)}%',\n",
    "                        (x, y-10), cv2.FONT_HERSHEY_DUPLEX, 0.6,  color, 2)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Blob function \n",
    "\n",
    "OpenCV uses blob techniques to extract features from an image and allows detection. In order to be able to do forward propagation on our images or videos, we'll need to convert our images to **OpenCV blob's format**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_to_blob(input_frame, network, height, width):\n",
    "    \"\"\" This function allow us to convert a frame/image into blob format for OpenCV DNN\"\"\"    \n",
    "    blob = cv2.dnn.blobFromImage(input_frame, 1/255, (height,width), [0,0,0], 1, crop=False)\n",
    "    network.setInput(blob)\n",
    "    # get the YOLO output layers numbers (names), these layers will be useful for the detection part\n",
    "    # the layer's name : yolo_82, yolo_94, yolo_106\n",
    "    yoloLayers = network.getLayerNames()\n",
    "    outputLayers = [(yoloLayers[i[0]-1]) for i in network.getUnconnectedOutLayers()]\n",
    "    # Doing forward propagation with OpenCV\n",
    "    outputs = network.forward(outputLayers)\n",
    "    \n",
    "    return outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test on video files\n",
    "\n",
    "These blocks will run object detection on videos in our **video** folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "NJAqm_Np-_5M"
   },
   "outputs": [],
   "source": [
    "# method to load our video from directory\n",
    "import os \n",
    "\n",
    "def load_video(video):    \n",
    "    \"\"\" Return the video by inputing the title of the video \"\"\" \n",
    "    # get the video path and load the video\n",
    "    video_path = os.path.join('videos', video)\n",
    "    \n",
    "    return video_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  415 || y :  471 || w :  177 || h : 319 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  387 || y :  473 || w :  177 || h : 336 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  383 || y :  478 || w :  173 || h : 323 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  374 || y :  465 || w :  195 || h : 352 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  372 || y :  458 || w :  195 || h : 355 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  367 || y :  452 || w :  193 || h : 377 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  350 || y :  459 || w :  186 || h : 371 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  364 || y :  478 || w :  197 || h : 361 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  366 || y :  471 || w :  199 || h : 374 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  248 || y :  494 || w :  83 || h : 158 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  377 || y :  476 || w :  197 || h : 379 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  322 || y :  653 || w :  314 || h : 257 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  367 || y :  480 || w :  232 || h : 400 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  388 || y :  471 || w :  193 || h : 417 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  128 || y :  475 || w :  81 || h : 176 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  411 || y :  463 || w :  195 || h : 440 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  410 || y :  466 || w :  203 || h : 437 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  49 || y :  469 || w :  86 || h : 173 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  410 || y :  450 || w :  203 || h : 470 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  411 || y :  451 || w :  203 || h : 475 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  387 || y :  472 || w :  236 || h : 439 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  392 || y :  451 || w :  227 || h : 478 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  380 || y :  479 || w :  200 || h : 419 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  356 || y :  484 || w :  244 || h : 410 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  349 || y :  480 || w :  195 || h : 414 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  338 || y :  478 || w :  206 || h : 337 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  300 || y :  464 || w :  254 || h : 361 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  296 || y :  488 || w :  221 || h : 401 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  288 || y :  486 || w :  214 || h : 401 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  265 || y :  476 || w :  203 || h : 343 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  251 || y :  464 || w :  211 || h : 366 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  233 || y :  475 || w :  193 || h : 335 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  126 || y :  738 || w :  229 || h : 191 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  221 || y :  469 || w :  206 || h : 365 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  197 || y :  467 || w :  192 || h : 365 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  189 || y :  463 || w :  203 || h : 363 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  177 || y :  467 || w :  200 || h : 333 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  156 || y :  478 || w :  194 || h : 332 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  141 || y :  466 || w :  199 || h : 359 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  135 || y :  472 || w :  202 || h : 335 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  111 || y :  483 || w :  197 || h : 318 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  97 || y :  482 || w :  201 || h : 327 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  77 || y :  484 || w :  196 || h : 313 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  64 || y :  481 || w :  195 || h : 321 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  44 || y :  478 || w :  188 || h : 321 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  23 || y :  479 || w :  198 || h : 326 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  0 || y :  643 || w :  237 || h : 223 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  3 || y :  485 || w :  179 || h : 323 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  -6 || y :  487 || w :  182 || h : 317 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  -2 || y :  474 || w :  148 || h : 345 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  2 || y :  480 || w :  131 || h : 322 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  0 || y :  477 || w :  122 || h : 326 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  37 || y :  664 || w :  56 || h : 124 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  640 || y :  563 || w :  25 || h : 12 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  3 || y :  608 || w :  93 || h : 217 \n",
      "\n",
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  0 || y :  598 || w :  80 || h : 224 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  370 || y :  550 || w :  9 || h : 32 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  681 || y :  576 || w :  13 || h : 18 \n",
      "\n",
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  0 || y :  602 || w :  67 || h : 212 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  645 || y :  570 || w :  15 || h : 15 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  333 || y :  546 || w :  10 || h : 34 \n",
      "\n",
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  6 || y :  687 || w :  33 || h : 71 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  683 || y :  569 || w :  16 || h : 22 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  302 || y :  544 || w :  11 || h : 31 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  267 || y :  543 || w :  10 || h : 33 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  607 || y :  592 || w :  101 || h : 110 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  237 || y :  545 || w :  11 || h : 34 \n",
      "\n",
      "Number of objects detected :  6 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  590 || y :  749 || w :  123 || h : 142 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  208 || y :  544 || w :  10 || h : 36 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  100 || y :  540 || w :  12 || h : 21 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  253 || y :  549 || w :  13 || h : 31 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  282 || y :  259 || w :  31 || h : 12 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  596 || y :  582 || w :  116 || h : 121 \n",
      "\n",
      "Number of objects detected :  4 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  178 || y :  539 || w :  10 || h : 36 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  560 || y :  750 || w :  135 || h : 132 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  560 || y :  578 || w :  136 || h : 116 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  228 || y :  547 || w :  13 || h : 30 \n",
      "\n",
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  193 || y :  549 || w :  12 || h : 31 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  150 || y :  539 || w :  9 || h : 36 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  538 || y :  718 || w :  162 || h : 134 \n",
      "\n",
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  120 || y :  541 || w :  10 || h : 35 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  496 || y :  577 || w :  116 || h : 99 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  171 || y :  549 || w :  12 || h : 27 \n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  138 || y :  550 || w :  12 || h : 28 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  93 || y :  542 || w :  8 || h : 35 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  479 || y :  577 || w :  137 || h : 107 \n",
      "\n",
      "Number of objects detected :  3 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  489 || y :  597 || w :  59 || h : 61 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  112 || y :  545 || w :  13 || h : 30 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  61 || y :  538 || w :  11 || h : 34 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  448 || y :  593 || w :  64 || h : 66 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  73 || y :  541 || w :  27 || h : 40 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  403 || y :  580 || w :  135 || h : 98 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  60 || y :  543 || w :  12 || h : 31 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  414 || y :  592 || w :  60 || h : 62 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  38 || y :  545 || w :  11 || h : 31 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  396 || y :  599 || w :  58 || h : 53 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  15 || y :  542 || w :  10 || h : 33 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  367 || y :  591 || w :  52 || h : 59 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  322 || y :  653 || w :  303 || h : 220 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  341 || y :  592 || w :  55 || h : 58 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  310 || y :  728 || w :  105 || h : 125 \n",
      "\n",
      "Number of objects detected :  1 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  336 || y :  593 || w :  61 || h : 58 \n",
      "\n",
      "Number of objects detected :  2 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  320 || y :  590 || w :  60 || h : 54 \n",
      "\n",
      "Bounding box coordinates in the frame :  x :  274 || y :  732 || w :  107 || h : 118 \n",
      "\n",
      "Can't receive frame (stream end?). Exiting ...\n"
     ]
    }
   ],
   "source": [
    "cap_video = cv2.VideoCapture(load_video('../videos/library1.mp4'))\n",
    "\n",
    "network = neural_net\n",
    "# we can decrease the height and width but the minimum is 320x320.\n",
    "height, width = 600,600\n",
    "\n",
    "# save the video with object detections\n",
    "frame_width = int(cap_video.get(3))\n",
    "frame_height = int(cap_video.get(4))\n",
    "video_frames_save = cv2.VideoWriter('../results/vids/testvid1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 20, (frame_width,frame_height))\n",
    "\n",
    "while cap_video.isOpened():\n",
    "    success, video_frames = cap_video.read()\n",
    "    # if 'video_frames' is read correctly 'success' is True\n",
    "    if not success:\n",
    "        print(\"Can't receive frame (stream end?). Exiting ...\")\n",
    "        break\n",
    "        \n",
    "    # using convert_to_blob function : \n",
    "    outputs = convert_to_blob(video_frames, network, height, width)    \n",
    "    # apply object detection on the video file\n",
    "    bounding_boxes, class_objects, confidence_probs = object_detection(outputs, video_frames)   \n",
    "    # perform non-max suppression\n",
    "    indices = nms_bbox(bounding_boxes, confidence_probs, confidenceThreshold, nmsThreshold)\n",
    "    # draw the boxes\n",
    "    box_drawing(video_frames, indices, bounding_boxes, class_objects, confidence_probs, classNames, color=(0,255,255), thickness=2)\n",
    "  \n",
    "    # save the video\n",
    "    video_frames_save.write(video_frames)\n",
    "    \n",
    "    cv2.imshow('Object Detection in videos', video_frames)         \n",
    "    \n",
    "    if cv2.waitKey(1) == ord('q'):\n",
    "        break\n",
    "        \n",
    "cap_video.release()\n",
    "cv2.destroyAllWindows()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "yolov3_ftn4.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}