Using PySpark in Google Colab

Restaurant Recommender.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Restaurant Recommender.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "0yR5iT_szD1o",
        "bIEdOY_2C5T4",
        "x_v6aF1HiQjw"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n7opPpCWbDPh"
      },
      "source": [
        "##Restaurant Recommender Based On PySpark\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0yR5iT_szD1o"
      },
      "source": [
        "#### Setting up the environment\r\n",
        "\r\n",
        "In Google Colab, we fetch the spark modules, configure and install them. \r\n",
        "We also import libraries like matplotlib that can help us visualize the data.\r\n",
        "We create an object of the SparkContext and name it 'sc'. The first cell may take longer to run.\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RpDA1bkZyGJ0"
      },
      "source": [
        "!apt-get install openjdk-8-jdk-headless -qq > /dev/null\n",
        "!wget -q https://www-us.apache.org/dist/spark/spark-2.4.7/spark-2.4.7-bin-hadoop2.7.tgz\n",
        "!tar xf spark-2.4.7-bin-hadoop2.7.tgz\n",
        "!pip install -q findspark"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5l76xcy5FKhy"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import os\n",
        "import findspark\n",
        "from pyspark import SparkContext"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kwpl_jKKFUBy"
      },
      "source": [
        "os.environ[\"JAVA_HOME\"] = \"/usr/lib/jvm/java-8-openjdk-amd64\"\n",
        "os.environ[\"SPARK_HOME\"] = \"/content/spark-2.4.7-bin-hadoop2.7\"\n",
        "findspark.init()\n",
        "sc = SparkContext(\"local\")\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l48Duzhtze7I"
      },
      "source": [
        "I have provided both 'entree_data.tar.gz' zipped folder as well as the extracted files and folder. You can use either one to run the project. Only run the below cell if you are using the zipped folder.\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UGW-z9JMFUgn",
        "outputId": "d33b6cb1-2a54-471f-bb83-ce8f074337d0"
      },
      "source": [
        "!tar xf entree_data.tar.gz"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "entree\t\t    sample_data\t\t       spark-2.4.7-bin-hadoop2.7.tgz\n",
            "entree_data.tar.gz  spark-2.4.7-bin-hadoop2.7  spark-2.4.7-bin-hadoop2.7.tgz.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TeT159JeFUmV"
      },
      "source": [
        "!ls"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Sai5xlxCC3h"
      },
      "source": [
        "\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bIEdOY_2C5T4"
      },
      "source": [
        "#### Processing the session files.\r\n",
        "\r\n",
        "Here, we will first use the session file to generate popularity score of each restaurant.\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NWuZWNXD_hX0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4b4f05a1-84a8-4a19-8a1a-59a13f5248e7"
      },
      "source": [
        "obj = {'Atlanta': [], 'Boston': [], 'Chicago':[], 'Los Angeles':[], 'New Orleans':[], 'New York':[], 'San Francisco':[], 'Washington D.C.':[]}\n",
        "\n",
        "def trail(x):\n",
        "  if len(x)==1:\n",
        "    return \"000000\"+x\n",
        "  elif len(x)==2:\n",
        "    return \"00000\"+x\n",
        "  elif len(x)==3:\n",
        "    return \"0000\"+x\n",
        "\n",
        "def locmap(x):\n",
        "    if(x[-1]=='A'):\n",
        "      return ('Atlanta',trail(x[:-1]))\n",
        "    elif(x[-1]=='B'):\n",
        "      return ('Boston',trail(x[:-1]))\n",
        "    elif(x[-1]=='C'):\n",
        "      return ('Chicago',trail(x[:-1]))\n",
        "    elif(x[-1]=='E'):\n",
        "      return ('New Orleans',trail(x[:-1]))\n",
        "    elif(x[-1]=='F'):\n",
        "      return ('New York',trail(x[:-1]))\n",
        "    elif(x[-1]=='G'):\n",
        "      return ('San Francisco',trail(x[:-1]))\n",
        "    elif(x[-1]=='D'):\n",
        "      return ('Los Angeles',trail(x[:-1]))\n",
        "    elif(x[-1]=='H'):\n",
        "      return ('Washington D.C.',trail(x[:-1]))\n",
        "\n",
        "def to_list(a):\n",
        "    return [a]\n",
        "\n",
        "def append(a, b):\n",
        "    a.append(b)\n",
        "    return a\n",
        "\n",
        "def extend(a, b):\n",
        "    a.extend(b)\n",
        "    return a\n",
        "\n",
        "sess = sc.textFile(\"entree/session/session.19*\").map(lambda s: s.split(\"\\t\")).map(lambda s: s[2])\n",
        "print(\"Number of rows before filtering: \", sess.count())\n",
        "sess = sess.filter(lambda x: x[0]!='0')\n",
        "\n",
        "\n",
        "print(\"Number of rows after filtering: \", sess.count())\n",
        "sess = sc.parallelize(sess.map(lambda x: locmap(x)).collect())\n",
        "sess=sess.combineByKey(to_list, append, extend).collect()\n",
        "\n",
        "print(\"Example of data being stored in the 'sess' object\")\n",
        "for i in sess:\n",
        "  print(i[0], i[1][:5], \"\\t \\t Total restaurants: \",len(i[1]))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of rows before filtering:  50672\n",
            "Number of rows after filtering:  5995\n",
            "\n",
            "Chicago ['0000260', '0000250', '0000010', '0000418', '0000418'] \t \t Total restaurants:  4990\n",
            "New York ['0000723', '0000816', '0000816', '0000017', '0000622'] \t \t Total restaurants:  278\n",
            "New Orleans ['0000024', '0000266', '0000221', '0000221', '0000174'] \t \t Total restaurants:  100\n",
            "Los Angeles ['0000180', '0000180', '0000180', '0000180', '0000180'] \t \t Total restaurants:  180\n",
            "Atlanta ['0000012', '0000012', '0000152', '0000245', '0000023'] \t \t Total restaurants:  71\n",
            "Washington D.C. ['0000325', '0000065', '0000365', '0000325', '0000372'] \t \t Total restaurants:  118\n",
            "San Francisco ['0000198', '0000078', '0000227', '0000406', '0000231'] \t \t Total restaurants:  129\n",
            "Boston ['0000312', '0000008', '0000403', '0000059', '0000140'] \t \t Total restaurants:  129\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pBgdYO-qCTaU"
      },
      "source": [
        "As can be seen above, after filtering almost 90% of the data is lost. We can conclude that this process is optional or is rather harming the correctness of the program. But it includes some important PySpark concepts and hence, I would rather not modify it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x_v6aF1HiQjw"
      },
      "source": [
        "#### Onto the Data folder\r\n",
        "\r\n",
        "In the following function, we shall only be considering 4 parameters as important: \r\n",
        "\r\n",
        "\r\n",
        "\r\n",
        "1.   The ID of the restaurant\r\n",
        "2.   The list of features\r\n",
        "3.   The city in which the restaurant is present\r\n",
        "4.   The name of the restaurant\r\n",
        "5.   The popularity score\r\n",
        "\r\n",
        "\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SVY4Q4OBuxCB"
      },
      "source": [
        "def stream(s, city):\n",
        "  for i in sess:\n",
        "    if(i[0]==city):\n",
        "      popularity = i[1].count(s[0])\n",
        "  return (s[1], s[2].split(), city,  s[0],popularity)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rjzol6-pB-dz"
      },
      "source": [
        "Here, we are creating RDDs and transforming them based on our needs. We are also maintaining counts of each city in the `resto` list.\r\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZW4xZx6w2E4S"
      },
      "source": [
        "resto=[]\r\n",
        "atlanta = sc.textFile('entree/data/atlanta.txt')\r\n",
        "atlanta = atlanta.map(lambda s: s.split('\\t')).map(lambda s:stream(s, 'Atlanta'))\r\n",
        "atcount = atlanta.count()\r\n",
        "resto.append()\r\n",
        "boston = sc.textFile('entree/data/boston.txt')\r\n",
        "boston = boston.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'Boston'))\r\n",
        "bocount = boston.count()\r\n",
        "resto.append(bocount)\r\n",
        "chicago = sc.textFile('entree/data/chicago.txt')\r\n",
        "chicago = chicago.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'Chicago'))\r\n",
        "chcount = chicago.count()\r\n",
        "resto.append(chcount)\r\n",
        "los_angeles = sc.textFile('entree/data/los_angeles.txt')\r\n",
        "los_angeles = los_angeles.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'Los Angeles'))\r\n",
        "lacount = los_angeles.count()\r\n",
        "resto.append(lacount)\r\n",
        "new_orleans = sc.textFile('entree/data/new_orleans.txt')\r\n",
        "new_orleans = new_orleans.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'New Orleans'))\r\n",
        "nocount = new_orleans.count()\r\n",
        "resto.append(nocount)\r\n",
        "new_york = sc.textFile('entree/data/new_york.txt')\r\n",
        "new_york = new_york.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'New York'))\r\n",
        "nycount = new_york.count()\r\n",
        "resto.append(nycount)\r\n",
        "san_francisco = sc.textFile('entree/data/san_francisco.txt')\r\n",
        "san_francisco = san_francisco.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'San Francisco'))\r\n",
        "sfcount = san_francisco.count()\r\n",
        "resto.append(sfcount)\r\n",
        "washington_dc = sc.textFile('entree/data/washington_dc.txt')\r\n",
        "washington_dc = washington_dc.map(lambda s: s.split('\\t')).map(lambda s: stream(s, 'Washington D.C.'))\r\n",
        "wacount = washington_dc.count()\r\n",
        "resto.append(wacount)\r\n",
        "\r\n",
        "\r\n",
        "\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bg4zfNl4fxmU"
      },
      "source": [
        "Here we have a visualization of the data distribution. As we can see, the data is not necessarily distributed uniformly but the numbers are fairly comparable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 352
        },
        "id": "AsmZP_l8fCJI",
        "outputId": "6a6a3a5f-f25c-4436-a85f-1699e573eb4d"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0,0,2,1])\n",
        "cities = ['Atlanta', 'Boston', 'Chicago', 'Los Angeles', 'New Orleans', 'New York', 'San Francisco', 'Washington D.C.']\n",
        "ax.bar(cities,resto)\n",
        "ax.set_title('Number of restaurants in the given locations')\n",
        "\n",
        "plt.show()\n",
        "\n",
        "print(\"Total Count: \" + atcount+bocount+chcount+lacount+nocount+nycount+sfcount+wacount)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA44AAAFPCAYAAAAC3JCsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwlVX3//9dbRtllnS/KgA4q0YBxnbgvGDWyGUjihgbQoHxNXINGMfoLRL8qagxKjAsKigRF3FFwQRQBFQQUWSVMEGVThlVRUcHP749zGi5Nd81Md093z/B6Ph79mLqntnOr7r1V7zqnalJVSJIkSZI0mbvNdQUkSZIkSfObwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6StBpJ8rEk/2+O1p0kH01yfZLvz0Ud1gRJzk+ywwwta4ckl8/EslZgXV9JsvdsrGtknS9McupsrrOv96Yk95vt9UrSfGZwlKRpSHJpkquTrD9S9uIkJ81htVaVJwBPB7aqqketyhX17fq0VbmO6ZhO/apq+6o6aYrrrSQPmMq801VVO1XVEXOx7lUpyUlJXjxaVlUbVNUlc1UnSZqPDI6SNH1rAa+a60qsrCRrreQs9wUurapfr+DyF6x8rVa9+VovSZLmM4OjJE3fu4DXJtl4/Igki3sr0YKRsttaOHpXvO8kOTjJDUkuSfK4Xn5Zb80c3z1w8yQnJPlVkm8nue/Ish/Ux12X5KIkzxkZ97EkH0hyfJJfA0+ZoL5bJjm2z780yUt6+T7AR4DH9m58/zbBvKPv5VrgwCRrJ/n3JD9L8oskH0yybp9+8yRf7u/7uiSnJLlbkiOB+wBf6ut6XZ/+00l+nuTGJCcn2X6ibTpSl1NHXleSlyW5GLi4l723b+NfJjkryRNHpj8wyTFJPt638/lJlvRxd6pfknWS/HeSa/v7OSPJFuO3UZ//ttbKofVMMN/JffBHfb3PHRn3mv5ZuSrJi0bKJ93+Eyx/rSTvTnJNkp8kefnoZ3dsG/dl3pDkwSPzLkzy2yT/p7/eNcnZfbrvJnnIuPf/2iTn9H35qSTrTFSnCer4uL5tb+z/Pm5k3KZpXamvTOtO/YVevkn/nC3r5V9OslUf91bgicD7+jZ9Xy+/rWU3yUZ9/yxL8tMkb0pytz7uhUlO7dv4+r7ddhqp0wvTvtO/6uNesCLvU5LmI4OjJE3fmcBJwGunOP+jgXOAzYBPAEcDfw48APg72kntBiPTvwB4C7A5cDZwFEBad9kT+jL+D/A84P1JthuZ9/nAW4ENgYnuHTsauBzYEngW8LYkf1FVhwEvBb7Xu/EdMPBeLgG26Os5CPgT4GH9/SwC/rVP+5q+roV9+n8Bqqr2BH4GPLOv6519+q8A2/b39oOx970Sdu/1G9seZ/R6bUrbZp8eF2D+qm+PjYFjgffRKjhR/fYGNgK2pu3HlwK/XcF6Tbie8arqSX3woX29n+qv79XXvQjYB/ivJJv0cUPbf7yXADv1aR9B214T1eN3wOeAPUaKnwN8u6quTvJw4HDg/9K2xYeAY5OsPW76HYFtgIcAL5ykTrdJsilwHHBIX+5/AMcl2axPciSwHrA97TNycC+/G/BRWov5fWj7ZWxfvhE4BXh536Yvn2DV/0nbvvcDngzsBbxoZPyjgYto38d3AoelWb/Xdaeq2hB4HO37KkmrJYOjJM2MfwVekWThFOb9SVV9tKpuBT5FCx9vrqrfVdXXgd/TTvrHHFdVJ/cT+DfSWgG3BnaldSX9aFXdUlU/BD4LPHtk3i9W1Xeq6o9VdfNoJfoyHg+8vqpurqqzaa2Me63Ee7myqv6zqm4Bbgb2Bf6pqq6rql8Bb6MFWoA/APcG7ltVf6iqU6qqJltwVR1eVb/q7/tA4KFJNlqJur291+O3fXn/XVXX9m31bmBt4IEj059aVcf3/XIk8NCBZf+BFmYeUFW3VtVZVfXLFazXyqxnsnW/uW/D44GbgAcmCcPbf7znAO+tqsur6npa6JzMJ8Yt5/m9jL7OD1XV6X1bHAH8DnjMyPSHVNWVVXUd8CVaWF2eXYCLq+rIvs8+CfwYeGaSe9NC70ur6vq+Lb4N0PfxZ6vqN30bvJUWAJcrrTv384A39M/epcC7gT1HJvtpVX24778jaJ/psdbmPwIPTrJuVV1VVeevyHolaT4yOErSDKiq84AvA/tPYfZfjAyPhZrxZaMtjpeNrPcm4DpaC+F9gUf37oE3JLmB1jp5r4nmncCWwFjAGPNTWivVihpd/kJaC9BZI/X5ai+H1sV3KfD13p1v0m3Xu1EelOR/k/wSuLSP2nyKdaN3l7ywd3u8gdaqNLq8n48M/wZYJ5PfH3kk8DXg6N5V8p1J7r6C9VqZ9Uzk2h7UR5exAcvf/uNtyR230dBn5VvAekkenWQxLfh9vo+7L/CacZ/Drfvyx4x/z6Of78lsSfs8jhr7fG5N++xeP36mJOsl+VDvZvpL4GRg46zYPb6bA3cft97x34nb3ktV/aYPbtDvBX4urfX5qiTHJXnQCqxTkuYlg6MkzZwDaN39Rk8qxx4ks95I2WiQm4qtxwZ6F9ZNgStpJ/rfrqqNR/42qKp/GJl30ha9voxNk2w4UnYf4IqVqNvo8q+hhd7tR+qzUVVtANBbcF5TVfejddfcL8lTJ6nn84HdgKfRAt7iXp7+769Z/ja+bZlp9zO+jtbKtklVbQzcOLK8lXmf9Bauf6uq7WhdEndl5VpqV4XB7T+Bq4CtRl5vPcl09Na1Y2jdVfcAvjxyweEy4K3jPofr9RbC6biSFkpHjX0+L6N9du90nzGtS/QDgUdX1T2BsS6/Y/t66DtxDa1Fd3S9K/ydqKqvVdXTaa2QPwY+vCLzSdJ8ZHCUpBlSVUtpXU1fOVK2jHaS+Xe91ezvgftPc1U7J3lCknvQ7nU8raouo7V4/kmSPZPcvf/9eZI/XcH6XwZ8F3h72sNeHkK7Z+6/p1LJqvoj7UT54JGHpixK8ow+vGuSB/QulTcCt9K69kFrhR39f/Q2pHV3vJYWEN82bnVnA3/TW5ce0Os9ZEPgFmAZsCDJvwL3XIm3d4f6JXlKkj/rrVi/pIWNP0428zSM3y6TWt72n8AxwKv6NBsDr1/OKj5Ba1F7Abd3U6Wv86W9NTJJ1k+yy7gLElNxPO3z/fwkC9IeDrQdLbReRbsH9v1pD8O5e5KxgLghLUDf0O+THH9/7qTbdCQgvzXJhmkPotqPFfhOJNkiyW79Xsff0boQr4rPhCTNCoOjJM2sNwPrjyt7CfDPtNCzPS2cTccnaCe/1wGPpD1Ah97i85e0e7KupHWhewft3r0VtQetNe9KWtfDA6rqG9Oo6+tp3VFP690Ev8Ht9xFu21/fBHwPeH9VfauPezvwpt7V8bXAx2ldBK8ALgBOG7eeg2n3gv6Cdp/Z8h6c8zVat83/6cu9meGumeONr9+9gM/QQuOFwLdp3Vdn2oHAEX29z1nexAxv//E+DHyd9qCmH9KC2i20QH8nVXU6raV3S1poGys/k/aZfx9wfV//C1egroOq6lpaS+5raN+l1wG7VtU1fZI9aYH9x8DVwKt7+XuAdWmth6fR9vuo9wLPSnsq6iETrPoVtPd5Ce2BUp+gPfxnee5GC5lX0r6rTwb+YXAOSZrHMvAcAkmSdBeV9t9KfLCqxncPlSTdBdniKEmSSLJukp17N9BFtFbtzy9vPknSXYMtjpIkiSTr0brYPoh2T+BxwKtW4r8VkSStwQyOkiRJkqRBdlWVJEmSJA0yOEqSJEmSBi2Y6woM2XzzzWvx4sVzXQ1JkiRJWuOdddZZ11TVwonGzevguHjxYs4888y5roYkSZIkrfGS/HSycXZVlSRJkiQNMjhKkiRJkgYZHCVJkiRJgwyOkiRJkqRBBkdJkiRJ0iCDoyRJkiRpkMFRkiRJkjTI4ChJkiRJGrTc4Jjk8CRXJzlvpOxdSX6c5Jwkn0+y8ci4NyRZmuSiJM8YKd+xly1Nsv/MvxVJkiRJ0qqwIi2OHwN2HFd2AvDgqnoI8D/AGwCSbAc8D9i+z/P+JGslWQv4L2AnYDtgjz6tJEmSJGmeW25wrKqTgevGlX29qm7pL08DturDuwFHV9XvquonwFLgUf1vaVVdUlW/B47u00qSJEmS5rmZuMfx74Gv9OFFwGUj4y7vZZOVS5IkSZLmuQXTmTnJG4FbgKNmpjqQZF9gX4D73Oc+M7VYSZKk5Vq8/3FzXYXV3qUH7TLXVZC0Cky5xTHJC4FdgRdUVfXiK4CtRybbqpdNVn4nVXVoVS2pqiULFy6cavUkSZIkSTNkSsExyY7A64C/qqrfjIw6FnhekrWTbANsC3wfOAPYNsk2Se5Be4DOsdOruiRJkiRpNiy3q2qSTwI7AJsnuRw4gPYU1bWBE5IAnFZVL62q85McA1xA68L6sqq6tS/n5cDXgLWAw6vq/FXwfiRJkiRJM2y5wbGq9pig+LCB6d8KvHWC8uOB41eqdpIkSZKkOTcTT1WVJEmSJK3BDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNGi5wTHJ4UmuTnLeSNmmSU5IcnH/d5NeniSHJFma5JwkjxiZZ+8+/cVJ9l41b0eSJEmSNNNWpMXxY8CO48r2B06sqm2BE/trgJ2AbfvfvsAHoAVN4ADg0cCjgAPGwqYkSZIkaX5bbnCsqpOB68YV7wYc0YePAHYfKf94NacBGye5N/AM4ISquq6qrgdO4M5hVJIkSZI0D031HsctquqqPvxzYIs+vAi4bGS6y3vZZOV3kmTfJGcmOXPZsmVTrJ4kSZIkaaZM++E4VVVAzUBdxpZ3aFUtqaolCxcunKnFSpIkSZKmaKrB8Re9Cyr936t7+RXA1iPTbdXLJiuXJEmSJM1zUw2OxwJjT0bdG/jiSPle/emqjwFu7F1avwb8ZZJN+kNx/rKXSZIkSZLmuQXLmyDJJ4EdgM2TXE57OupBwDFJ9gF+CjynT348sDOwFPgN8CKAqrouyVuAM/p0b66q8Q/ckSRJkiTNQ8sNjlW1xySjnjrBtAW8bJLlHA4cvlK1kyRJkiTNuWk/HEeSJEmStGYzOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEHTCo5J/inJ+UnOS/LJJOsk2SbJ6UmWJvlUknv0adfur5f28Ytn4g1IkiRJklatKQfHJIuAVwJLqurBwFrA84B3AAdX1QOA64F9+iz7ANf38oP7dJIkSZKkeW66XVUXAOsmWQCsB1wF/AXwmT7+CGD3Prxbf00f/9Qkmeb6JUmSJEmr2JSDY1VdAfw78DNaYLwROAu4oapu6ZNdDizqw4uAy/q8t/TpN5vq+iVJkiRJs2M6XVU3obUibgNsCawP7DjdCiXZN8mZSc5ctmzZdBcnSZIkSZqm6XRVfRrwk6paVlV/AD4HPB7YuHddBdgKuKIPXwFsDdDHbwRcO36hVXVoVS2pqiULFy6cRvUkSZIkSTNhOsHxZ8BjkqzX71V8KnAB8C3gWX2avYEv9uFj+2v6+G9WVU1j/ZIkSZKkWTCdexxPpz3k5gfAuX1ZhwKvB/ZLspR2D+NhfZbDgM16+X7A/tOotyRJkiRplixY/iSTq6oDgAPGFV8CPGqCaW8Gnj2d9UmSJEmSZt90/zsOSZIkSdIazuAoSZIkSRpkcJQkSZIkDTI4SpIkSZIGGRwlSZIkSYMMjpIkSZKkQQZHSZIkSdIgg6MkSZIkaZDBUZIkSZI0yOAoSZIkSRpkcJQkSZIkDTI4SpIkSZIGGRwlSZIkSYMMjpIkSZKkQQZHSZIkSdIgg6MkSZIkaZDBUZIkSZI0yOAoSZIkSRpkcJQkSZIkDTI4SpIkSZIGGRwlSZIkSYMMjpIkSZKkQQZHSZIkSdKgBXNdAUlrrsX7HzfXVVjtXXrQLnNdBUmSJFscJUmSJEnDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNGhawTHJxkk+k+THSS5M8tgkmyY5IcnF/d9N+rRJckiSpUnOSfKImXkLkiRJkqRVabotju8FvlpVDwIeClwI7A+cWFXbAif21wA7Adv2v32BD0xz3ZIkSZKkWTDl4JhkI+BJwGEAVfX7qroB2A04ok92BLB7H94N+Hg1pwEbJ7n3lGsuSZIkSZoV02lx3AZYBnw0yQ+TfCTJ+sAWVXVVn+bnwBZ9eBFw2cj8l/cySZIkSdI8Np3guAB4BPCBqno48Gtu75YKQFUVUCuz0CT7JjkzyZnLli2bRvUkSZIkSTNhOsHxcuDyqjq9v/4MLUj+YqwLav/36j7+CmDrkfm36mV3UFWHVtWSqlqycOHCaVRPkiRJkjQTphwcq+rnwGVJHtiLngpcABwL7N3L9ga+2IePBfbqT1d9DHDjSJdWSZIkSdI8tWCa878COCrJPYBLgBfRwugxSfYBfgo8p097PLAzsBT4TZ9WkiRJkjTPTSs4VtXZwJIJRj11gmkLeNl01idJkiRJmn3T/X8cJUmSJElrOIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBi2Y6wpIkmbP4v2Pm+sqrNYuPWiXua6CJElzwhZHSZIkSdIgWxy1xrAlZfpsTZEkSdJEbHGUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEHTfjhOkrWAM4ErqmrXJNsARwObAWcBe1bV75OsDXwceCRwLfDcqrp0uuuXJEnSmsuH302PD77TTJmJFsdXAReOvH4HcHBVPQC4Htinl+8DXN/LD+7TSZIkSZLmuWkFxyRbAbsAH+mvA/wF8Jk+yRHA7n14t/6aPv6pfXpJkiRJ0jw23RbH9wCvA/7YX28G3FBVt/TXlwOL+vAi4DKAPv7GPv0dJNk3yZlJzly2bNk0qydJkiRJmq4pB8ckuwJXV9VZM1gfqurQqlpSVUsWLlw4k4uWJEmSJE3BdB6O83jgr5LsDKwD3BN4L7BxkgW9VXEr4Io+/RXA1sDlSRYAG9EekiNJkiRpNeEDi6ZndX1g0ZRbHKvqDVW1VVUtBp4HfLOqXgB8C3hWn2xv4It9+Nj+mj7+m1VVU12/JEmSJGl2rIr/x/H1wH5JltLuYTyslx8GbNbL9wP2XwXrliRJkiTNsGn/P44AVXUScFIfvgR41ATT3Aw8eybWJ0mSJEmaPauixVGSJEmStAYxOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDFsx1BSRJuitbvP9xc12F1dqlB+0y11WQpLsEWxwlSZIkSYMMjpIkSZKkQQZHSZIkSdIgg6MkSZIkaZDBUZIkSZI0yOAoSZIkSRpkcJQkSZIkDTI4SpIkSZIGGRwlSZIkSYMMjpIkSZKkQQZHSZIkSdIgg6MkSZIkaZDBUZIkSZI0aMFcV2B1tnj/4+a6Cqu1Sw/aZa6rIEmSJGkF2OIoSZIkSRpkcJQkSZIkDTI4SpIkSZIGGRwlSZIkSYMMjpIkSZKkQQZHSZIkSdIgg6MkSZIkaZDBUZIkSZI0yOAoSZIkSRpkcJQkSZIkDZpycEyydZJvJbkgyflJXtXLN01yQpKL+7+b9PIkOSTJ0iTnJHnETL0JSZIkSdKqM50Wx1uA11TVdsBjgJcl2Q7YHzixqrYFTuyvAXYCtu1/+wIfmMa6JUmSJEmzZMrBsaquqqof9OFfARcCi4DdgCP6ZEcAu/fh3YCPV3MasHGSe0+55pIkSZKkWTEj9zgmWQw8HDgd2KKqruqjfg5s0YcXAZeNzHZ5L5MkSZIkzWPTDo5JNgA+C7y6qn45Oq6qCqiVXN6+Sc5McuayZcumWz1JkiRJ0jRNKzgmuTstNB5VVZ/rxb8Y64La/726l18BbD0y+1a97A6q6tCqWlJVSxYuXDid6kmSJEmSZsB0nqoa4DDgwqr6j5FRxwJ79+G9gS+OlO/Vn676GODGkS6tkiRJkqR5asE05n08sCdwbpKze9m/AAcBxyTZB/gp8Jw+7nhgZ2Ap8BvgRdNYtyRJkiRplkw5OFbVqUAmGf3UCaYv4GVTXZ8kSZIkaW7MyFNVJUmSJElrLoOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA0yOEqSJEmSBhkcJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEkaZHCUJEmSJA2a9eCYZMckFyVZmmT/2V6/JEmSJGnlzGpwTLIW8F/ATsB2wB5JtpvNOkiSJEmSVs5stzg+ClhaVZdU1e+Bo4HdZrkOkiRJkqSVMNvBcRFw2cjry3uZJEmSJGmeSlXN3sqSZwE7VtWL++s9gUdX1ctHptkX2Le/fCBw0axVcM2zOXDNXFdCd+A+mV/cH/OP+2T+cZ/ML+6P+cd9Mv+4T6buvlW1cKIRC2a5IlcAW4+83qqX3aaqDgUOnc1KramSnFlVS+a6Hrqd+2R+cX/MP+6T+cd9Mr+4P+Yf98n84z5ZNWa7q+oZwLZJtklyD+B5wLGzXAdJkiRJ0kqY1RbHqrolycuBrwFrAYdX1fmzWQdJkiRJ0sqZ7a6qVNXxwPGzvd67KLv8zj/uk/nF/TH/uE/mH/fJ/OL+mH/cJ/OP+2QVmNWH40iSJEmSVj+zfY+jJEmSJGk1Y3Ccx5LsnqSSPKi/fliSnUfGvzDJ+6a47I2T/ONM1fWuIsmtSc5O8qMkP0jyuCku519mum53JUnuleToJP+b5KwkxyfZN8mXJ5n+I0m2m+16ru6S3LQKlvmeJFckWSXHnyQHJnntqlj2XOrHgnePvH5tkgNneB3bJ/lmkouSXJzk/0uSSabdYbLvm263KvdbknWS/DjJn42U/XOSD63g/GvEdyXJG5Ocn+Scfnx+9Awsc3GS3/bljf3dYybqO8n6vruqlj3Tkhyc5NUjr7+W5CMjr9+dZL+VXOakn8XpbJv+OzWl87RJljd2Dnh+Pw98zWTHsknOU/5kpuoyVwyO89sewKn9X4CHATtPPvlK2RgwOK6831bVw6rqocAbgLdPcTkGxynqJ7KfB06qqvtX1SNp+2KLyeapqhdX1QWzVUdNrB9g/xq4DHjyHFdndfM74G+SbL4qFp5kXdpTzg+qqgcCDwUexwTHiSSz/nyE1dgq229VdTPwauD9aRYBLwX2X968a8o+TPJYYFfgEVX1EOBptN+XmfC//Xg/9vf7kfXO6ParqhkLN7PgO7TfhrHf9M2B7UfGPw6YsSA8zW2zA72uM2TsHHB74OnATsAB4yeaynnK6sLgOE8l2QB4ArAP8Lx+pevNwHP71Y7njpv+mUlOT/LDJN9IskUvPzDJ4UlOSnJJklf2WQ4C7t+X9a4kGyQ5sbeinZtkt1l8u6urewLXQ/uR6NvxvL79ntvL753k5L6dz0vyxCQHAev2sqP6dPv18eeNXcnrVzwvTPLhfnXr6/3k7q7uKcAfquqDYwVV9SPgFGCDJJ/pV+GPGmst6Z//JX14x/45/1GSE3vZo5J8r39/vpvkgb18vSTHJLkgyef7d2xsOXv0fX1eknfM8jaYM2k9H07rV/c/n2STXv7Kvp3OSXL0JLPvAJwPfIDbL4gN/U6R1up1UZJTk3xy7Kp0kvsn+Wq/kntKes+McXWdcJokz+777UdJTp6xjbNq3UJ72MM/jR+RZGGSzyY5o/89vpefm9a7JEmuTbJXL/94kqePW8zzge9U1dcBquo3wMvpIaTvoyOTfAc4ctz61+/77/v9O0/JOSsAAAuaSURBVLRbL1/ct/sPMtJDI60V4KRJvqsHjXyO/n3Gtt7cWaX7raq+ClwF7AUcDBwIbJTWcnxO2nH9Pn3+jyX5YJLTgXeOq8tLknxlNTzG3Bu4pqp+B1BV11TVlQBJ/rVv1/OSHDruePCO/nn9nyRPXJEV9c/tKUmOBS7oZV/ovy/nJ9l3ZNqbkry1/8acltvPybbov5s/6n9j34mb+r93Omfo5RMdtzbt6z+nr+MhM7NJl+u7wGP78PbAecCvkmySZG3gT4EfDGz/yY4V22XiY8DYthn63di5l52V5JAkX06ymHYh5Z/69nxi/02a7LtxSNrx/5Ikz1reRqiqq4F9gZeP1WPEhOcpVXXKCm/l+aqq/JuHf8ALgMP68HeBRwIvBN43Ms1tr4FNuP1hRy8G3t2HD+zzr027KnQtcHdgMXDeyLIWAPfsw5sDS8eW598d9sutwNnAj4EbgUf28r8FTqD9NzNbAD+jHdBeA7yxT7MWsGEfvmlkmY8EzgXWBzagnVg/vO+jW4CH9emOAf5urrfBXP8BrwQOnqB8h75PtqJdFPse8IQ+7iRgCbCQdjV6m16+af/3nsCCPvw04LN9+LXAh/rwg/v+WAJs2ffxwv7d+Saw+1xvm1WwrW+aoOwc4Ml9+M3Ae/rwlcDafXjjSZb3YWDPvr2vAO7eyyf7nfrz/n1bB9gQuBh4bZ/nRGDbPvxo4Jsjy1reNOcCi4bqOt/+gJv6drsU2Kh/Ng/s4z4x8lm/D3BhH/4gsEv/7J4BfLiXXwysP275/wG8aoL1Xt/XeyBwFrBuL98B+HIffhv9t4nWm+V/aL9n6wHr9PJtgTNH5r3TdxXYDLiI249lq8W+mcv91su3BC4HvtVffwnYuw//PfCFPvwx4MvAWqPfFdoFgi/Sv7+r0x/tmHl2/8y9n/7b1MdtOjJ8JPDMPnwSt58j7Qx8Y4LlLgZ+25d9NvBf/XP7a/rxY3QdwLq0ALVZf10j63sn8KY+/Cng1X14LWCjsc9J//dO5wxMftz6T+CAPvwXwNmzuN1/0j+z/5cWzt7St+XjgVOWs/3vdKxgkmPAuG2zAxP/bqwzbvt8ktt/mw6kHw9W4Lvx6b7c7YClk7zviY6JNwBbjCub8DxlTfhbI7oqrKH2AN7bh4/ur88bmH4r4FNJ7g3cg/alHnNctatxv0tyNRM3lQd4W5InAX8EFvXpfj6td7Hm+W1VPQxu6yLz8SQPpv14fbKqbgV+keTbtJPeM4DDk9yd9gN19gTLfALw+ar6dV/u54An0rqN/WRknrNoBzNN7vtVdTlAkrNp2+vUkfGPAU6uqp8AVNV1vXwj4Igk29IO+Hfv5U+gfw+r6rwk5/TyP6d1QVnW13UU8CTgC6vofc0LSTaiHei/3YuOoB1soQXKo5J8gQm2Q1qviZ2B/arqV73V4xm0E1mY+Hfq8cAXq3XJuznJl/qyNqB1P/r0yIXetcetb2ia7wAfS3IM8LkpbYw5UFW/TPJx2knJb0dGPY12tX7s9T37+z+F9rn8Ka2Vd9+07ozXj/3erKRjq+q3E5T/JfBXuf0epXVoJ5VXAu9L8jDaRbfR+3sm+q6eBtwMHJZ2/+QacQ/lqt5vVXVlkm9y+/Z6LPA3ffhI7ti6+Ol+nBqzF+2ke/eq+sM03uacqKqbkjySdsx8Cu08aP+q+hjwlCSvo13A2JR2UfZLfdax7/3QcfV/x4730Fq8aJ/b0fOrVyb56z68Ne0CybXA77l9f5xF69YILeDt1et+Ky0IjbrTOUNf70THrSfQLlpTVd9MslmSe1bVLyd5PzPpu7Tf18fRLjot6sM30n5fYfLtP9mxYqJjwOXj1jvR78ZNwCUj++WTtJbAiQx9N75QVX8ELhhrIdad2VV1HkqyKe3H5SNJLgX+GXgOLdxN5j9prY9/RrsCtM7IuN+NDN/KxP9/5wtoV7Ue2X8ofzFuGRqnqr5HuzK2cGCak2knAFfQTlT3WsnVrMi+u6s5n9ZKO5Gpbq+30K7WPxh4Jn72p2IX2lX5RwBn5M73AD2D1hp1bv9dewIj3VVZuX13N+CGuuP9R3+6otNU1UuBN9FO9M5KstlKvdO59R7aLQzrj5TdDXjMyPtcVFU3ASfTTqifSGtlWQY8ixZMxruAcd+rJPejXWEfOxGdLGwG+NuR9d+nqi6kdc/8Be1+ySW0i5pj7rS/q+oW4FHAZ2j3rX118s2w2llV+23MH/vf8ozfh+fSTr63WoF556WqurWqTqqqA2itp3+bZB1aC+Sz+nnRh5n4vGhlj6u3bb8e6J4GPLbacw9+OLKOP1RvelqZdczAOcNsGbvP8c9ojRqn0ULZ44DvLmf7T3asWJFjwKo8Jxpd9tD59u0Ttd/IW4Grx40aOk9ZrRkc56dnAUdW1X2ranFVbc3t3QI2nGSejWg/NAB7r8A6fjVuWRsBV1fVH5I8Bbjv1Kp+15F2v9RatKuLp9DuP10ryULaD//3k9wX+EVVfRj4CO2HEuAP/Yoifd7d0+6nW5/28JDVvx/8qvNNYO3c8X6Sh9BOspbnNOBJSbbp823ay0e/Py8cmf47tIs2pD2Vdezphd8Hnpxk8yRr0QLQt1nDVdWNwPW5/Z6gPYFvpz0gYeuq+hbwetr23GDc7HsAL+6/aYuBbYCnJ1lvYJXfAZ6Z9vTIDWhhgh5kfpLk2XDbPcYPHVfXSadJcv+qOr2q/pV2Ur71lDbIHOitDcfQQsiYrwOvGHvRW/ioqstoF7e2rapLaK3vr6UFk/GOAp6Q5Gl9GesChzDuXrhJfA14xcj9Rg/v5RsBV/Wr+HvSfi8n1ffxRlV1PC10PnRo+tXJKtxvE/ku8Lw+/AKGjyc/pF1sPjbJliu4/HkjyQN7T5ExD6O11I6FlGv652q596xNwUa0VuDf9POBx6zAPCcC/wDQzxc2Gh05yTnDZMetU2j7dyzEXjNLrY3QPmO7Atf14H4d7cLgY/u4Cbf/Ch4rVtZFwP3S7mkEGH0GyPhz3ZX5bgzq53ofpDXa1LjRE56nZAXvp53PDI7z0x60pzGN+ixwL1q3ljs9HIfWj/vTSc4CrlneCqrqWuA7aTctv4t20rAkybm0bhQ/nuZ7WFONPdTmbNq9Cnv37iafp3W/+BHtB+N1VfVzWp/8HyX5Ie3HbKz78aHAOUmOqqof0PrXfx84HfhIVf1wFt/TaqX/QP818LS0x1yfT3u67XK7VfeupfsCn0vyI9o+hHZy/Pa+n0avYL4fWJjkAuD/0a4i3lhVV9EeGvIt2j4/q6q+OCNvcH5ZL8nlI3/70S5MvSut2+7DaPc5rgX8d//9+CFwSFXdMLaQHg53BI4bK+td7k6ltfBOqKrOoHXZPgf4Cq11ZKxr1wuAffp+PB+Y6IFek03zrvQHG9FOJH60MhtlHng3LViMeSXt9/uc/ll96ci402n3f0E7SVrEHbtvA9C7oO4GvCnJRbRtfQawIv/l01to3bvP6d/Ht/Ty9wN79+3/ICZvsRyzIfDl/tk6FVipR/qvBmZ8v03iFcCL+nbcE3jV0MRVNRZMj8sqemrvKrQB7TaDC/r73Y52D+kNtFau82gXNs5YBev+KrAgyYW0Bw6etgLzvIrWhfNcWhfW8f9N1A6MO2cYOG4dCDyyv++DWLFGg5lyLu2zfNq4shurPaBosu0/eKyYiv7b9Y/AV/s58K+4/TjxJeCv+3nbE1nJ78YExs4Bzwe+Qbv4828ASbZMcnyv0+B5Sj+HXC3lziFZkgTtijDtBv2bk9yfdqB4YI08ll2rVpIN+n1M69FaXPbtF1skSRo9ToTWDfbiqjp4ruu1JvJ+KUma3HrAt3q34gD/aGicdYf2bsLrAEcYGiVJ47wkyd60+6h/CHxojuuzxrLFUZIkSZI0yHscJUmSJEmDDI6SJEmSpEEGR0mSJEnSIIOjJEmSJGmQwVGSJEmSNMjgKEmSJEka9P8Dn0DZWvwpTeAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yHd_7unO2Njx"
      },
      "source": [
        "We combine the data by taking a union of all RDDs. \r\n",
        "We filter the restaurants that have 'CLOSED' in their names. This enables us to only provide results of the available restaurants."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C9NpaJEyXWMj",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f9c7e213-24b4-422e-a4de-48cd8b36bddb"
      },
      "source": [
        "final = sc.union([atlanta,boston, chicago, los_angeles, new_orleans, new_york, san_francisco, washington_dc]).filter(lambda s: \"(\" not in s[0]).sortBy(lambda x: x[0].strip())\n",
        "\n",
        "final.take(5)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4160\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('101', ['125', '075', '205', '053', '166'], 'New York', '0001096', 0),\n",
              " ('101 Seafood',\n",
              "  ['125', '212', '075', '205', '053', '166'],\n",
              "  'New York',\n",
              "  '0000163',\n",
              "  1),\n",
              " ('103 NYC', ['184', '075', '204', '052', '166'], 'New York', '0000542', 0),\n",
              " ('103 WEST',\n",
              "  ['080',\n",
              "   '253',\n",
              "   '099',\n",
              "   '200',\n",
              "   '245',\n",
              "   '196',\n",
              "   '191',\n",
              "   '192',\n",
              "   '146',\n",
              "   '024',\n",
              "   '045',\n",
              "   '076',\n",
              "   '206',\n",
              "   '054',\n",
              "   '167'],\n",
              "  'Atlanta',\n",
              "  '0000230',\n",
              "  1),\n",
              " ('107 West', ['075', '204', '052', '165'], 'New York', '0000010', 0)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XQ2yXwJognZI"
      },
      "source": [
        "A list of all features being considered by the data. There is room for improvement here."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n15SdZmGXCrP",
        "outputId": "58fd6ebe-c59d-43a0-be67-4612d2623386"
      },
      "source": [
        "features = sc.textFile('entree/data/features.txt').map(lambda x: x.split(\"\\t\"))\n",
        "features.take(5)\n",
        "# features.collect()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[['000', 'A'],\n",
              " ['001', 'Authentic'],\n",
              " ['002', 'Afghanistan'],\n",
              " ['003', 'African'],\n",
              " ['004', 'After Hours Dining']]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s4mRX7LtbgBL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "414bfaf9-2786-4f0f-882b-09cf889de2f1"
      },
      "source": [
        "print(\"Input list of features wanted\")\n",
        "lis = input().split()\n",
        "print(\"\\n1. Atlanta\\n2. Boston\\n3. Chicago\\n4. Los Angeles\\n5. New Orleans\\n6. New York\\n7. San Francisco\\n8. Washington D.C.\\n\")\n",
        "location = int(input(\"Enter a number for location: \")) - 1\n",
        "\n",
        "def score(x,lis):\n",
        "  return   len( [value for value in lis if value in x[1]] )\n",
        "  \n",
        "\n",
        "\n",
        "temp = final.map(lambda x: (x[0],  score(x,lis), x[2], x[4])).filter(lambda x: x[1] !=0 and x[2]==cities[location]).sortBy(lambda x: (x[1], x[3]),ascending= False )#.sortBy(lambda x:x[-1], ascending = False)\n",
        "print(\"Hotels with at least one feature matching: \" + str( temp.count()))\n",
        "temp.take(10)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Input list of features wanted\n",
            "164 053 253\n",
            "\n",
            "1. Atlanta\n",
            "2. Boston\n",
            "3. Chicago\n",
            "4. Los Angeles\n",
            "5. New Orleans\n",
            "6. New York\n",
            "7. San Francisco\n",
            "8. Washington D.C.\n",
            "\n",
            "Enter a number for location: 1\n",
            "Hotels with at least one feature matching: 224\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(\"Houston's\", 3, 'Atlanta', 5),\n",
              " (\"Babette's Cafe\", 3, 'Atlanta', 2),\n",
              " ('Chile Tree', 3, 'Atlanta', 1),\n",
              " ('Haveli', 3, 'Atlanta', 1),\n",
              " ('Longhorn Steaks', 3, 'Atlanta', 1),\n",
              " ('Outback Steaks', 3, 'Atlanta', 1),\n",
              " ('SURIN OF THAILAND', 3, 'Atlanta', 1),\n",
              " (\"Altobeli's Fine Italian Cuisine\", 3, 'Atlanta', 0),\n",
              " ('August Moon', 3, 'Atlanta', 0),\n",
              " ('Azio', 3, 'Atlanta', 0)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6bYIzMk08Zid"
      },
      "source": [
        "The output displays the names of the restaurants, the number of matching features, their locations and their popularity score."
      ]
    }
  ]
}