{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Association Rule Mining\n",
    "\n",
    "This is a sample solution for the association rule mining exercise. This does not mean that this is the only way to solve this exercise. As with any programming task - and also with most data analysis tasks - there are multiple solutions for the same problem. \n",
    "\n",
    "## Libraries and Data\n",
    "\n",
    "The first part of the exercise is about association rule mining. In Python, you can use the ```mlxtend``` library for the mining of association rules. \n",
    "\n",
    "We use data about [store baskets](https://user.informatik.uni-goettingen.de/~sherbold/store_data.csv) in this exercise. You can use the following code to load the data. The code creates a list of records, where each record is a list of the items that are part of the transaction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "\n",
    "records = []\n",
    "# directly load from the url instead of using the file\n",
    "for line in urllib.request.urlopen(\"https://user.informatik.uni-goettingen.de/~sherbold/store_data.csv\"):\n",
    "    # this also means we need to decode the binary string into ascii\n",
    "    records.append(line.decode('ascii').strip().split(','))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding frequent itemsets\n",
    "\n",
    "Once you have the transactional records, use the apriori algorithm to find frequent itemsets with a suitable threshold for support for this data. Try to find a suitable threshold for the minimal support such that you can state a clear reason why you picked this threshold. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>support</th>\n",
       "      <th>itemsets</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.020397</td>\n",
       "      <td>(almonds)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.008932</td>\n",
       "      <td>(antioxydant juice)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.033329</td>\n",
       "      <td>(avocado)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.008666</td>\n",
       "      <td>(bacon)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.010799</td>\n",
       "      <td>(barbecue sauce)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.014265</td>\n",
       "      <td>(black tea)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.009199</td>\n",
       "      <td>(blueberries)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.011465</td>\n",
       "      <td>(body spray)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.033729</td>\n",
       "      <td>(brownies)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.008666</td>\n",
       "      <td>(bug spray)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.005866</td>\n",
       "      <td>(burger sauce)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.087188</td>\n",
       "      <td>(burgers)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.030129</td>\n",
       "      <td>(butter)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.081056</td>\n",
       "      <td>(cake)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.009732</td>\n",
       "      <td>(candy bars)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.015331</td>\n",
       "      <td>(carrots)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.025730</td>\n",
       "      <td>(cereals)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.046794</td>\n",
       "      <td>(champagne)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.059992</td>\n",
       "      <td>(chicken)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0.006133</td>\n",
       "      <td>(chili)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0.163845</td>\n",
       "      <td>(chocolate)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0.010532</td>\n",
       "      <td>(cider)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0.008399</td>\n",
       "      <td>(clothes accessories)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.080389</td>\n",
       "      <td>(cookies)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.051060</td>\n",
       "      <td>(cooking oil)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.031862</td>\n",
       "      <td>(cottage cheese)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0.013198</td>\n",
       "      <td>(eggplant)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.179709</td>\n",
       "      <td>(eggs)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.027063</td>\n",
       "      <td>(energy bar)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0.026663</td>\n",
       "      <td>(energy drink)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>695</th>\n",
       "      <td>0.005066</td>\n",
       "      <td>(ground beef, shrimp, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>696</th>\n",
       "      <td>0.005066</td>\n",
       "      <td>(ground beef, mineral water, soup)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>697</th>\n",
       "      <td>0.017064</td>\n",
       "      <td>(spaghetti, mineral water, ground beef)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>698</th>\n",
       "      <td>0.005466</td>\n",
       "      <td>(ground beef, tomatoes, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>699</th>\n",
       "      <td>0.006133</td>\n",
       "      <td>(ground beef, spaghetti, olive oil)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>700</th>\n",
       "      <td>0.006399</td>\n",
       "      <td>(ground beef, pancakes, spaghetti)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>701</th>\n",
       "      <td>0.005999</td>\n",
       "      <td>(ground beef, shrimp, spaghetti)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>702</th>\n",
       "      <td>0.005599</td>\n",
       "      <td>(ground beef, tomatoes, spaghetti)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>703</th>\n",
       "      <td>0.005999</td>\n",
       "      <td>(herb &amp; pepper, mineral water, spaghetti)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>704</th>\n",
       "      <td>0.005599</td>\n",
       "      <td>(spaghetti, mineral water, low fat yogurt)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>705</th>\n",
       "      <td>0.008532</td>\n",
       "      <td>(mineral water, milk, olive oil)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>706</th>\n",
       "      <td>0.007866</td>\n",
       "      <td>(pancakes, mineral water, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>707</th>\n",
       "      <td>0.007866</td>\n",
       "      <td>(shrimp, mineral water, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>708</th>\n",
       "      <td>0.008532</td>\n",
       "      <td>(mineral water, milk, soup)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>709</th>\n",
       "      <td>0.015731</td>\n",
       "      <td>(spaghetti, mineral water, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>710</th>\n",
       "      <td>0.006532</td>\n",
       "      <td>(tomatoes, mineral water, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>711</th>\n",
       "      <td>0.006133</td>\n",
       "      <td>(turkey, mineral water, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>712</th>\n",
       "      <td>0.005066</td>\n",
       "      <td>(whole wheat rice, mineral water, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>713</th>\n",
       "      <td>0.007199</td>\n",
       "      <td>(spaghetti, milk, olive oil)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>714</th>\n",
       "      <td>0.005866</td>\n",
       "      <td>(tomatoes, spaghetti, milk)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>715</th>\n",
       "      <td>0.005199</td>\n",
       "      <td>(mineral water, soup, olive oil)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>716</th>\n",
       "      <td>0.010265</td>\n",
       "      <td>(spaghetti, mineral water, olive oil)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>717</th>\n",
       "      <td>0.011465</td>\n",
       "      <td>(spaghetti, pancakes, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>718</th>\n",
       "      <td>0.006799</td>\n",
       "      <td>(spaghetti, mineral water, salmon)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>719</th>\n",
       "      <td>0.008532</td>\n",
       "      <td>(spaghetti, shrimp, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>720</th>\n",
       "      <td>0.007466</td>\n",
       "      <td>(spaghetti, mineral water, soup)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>721</th>\n",
       "      <td>0.009332</td>\n",
       "      <td>(spaghetti, tomatoes, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>722</th>\n",
       "      <td>0.006399</td>\n",
       "      <td>(spaghetti, turkey, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>723</th>\n",
       "      <td>0.006266</td>\n",
       "      <td>(spaghetti, whole wheat rice, mineral water)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>724</th>\n",
       "      <td>0.005066</td>\n",
       "      <td>(pancakes, spaghetti, olive oil)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>725 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      support                                      itemsets\n",
       "0    0.020397                                     (almonds)\n",
       "1    0.008932                           (antioxydant juice)\n",
       "2    0.033329                                     (avocado)\n",
       "3    0.008666                                       (bacon)\n",
       "4    0.010799                              (barbecue sauce)\n",
       "5    0.014265                                   (black tea)\n",
       "6    0.009199                                 (blueberries)\n",
       "7    0.011465                                  (body spray)\n",
       "8    0.033729                                    (brownies)\n",
       "9    0.008666                                   (bug spray)\n",
       "10   0.005866                                (burger sauce)\n",
       "11   0.087188                                     (burgers)\n",
       "12   0.030129                                      (butter)\n",
       "13   0.081056                                        (cake)\n",
       "14   0.009732                                  (candy bars)\n",
       "15   0.015331                                     (carrots)\n",
       "16   0.025730                                     (cereals)\n",
       "17   0.046794                                   (champagne)\n",
       "18   0.059992                                     (chicken)\n",
       "19   0.006133                                       (chili)\n",
       "20   0.163845                                   (chocolate)\n",
       "21   0.010532                                       (cider)\n",
       "22   0.008399                         (clothes accessories)\n",
       "23   0.080389                                     (cookies)\n",
       "24   0.051060                                 (cooking oil)\n",
       "25   0.031862                              (cottage cheese)\n",
       "26   0.013198                                    (eggplant)\n",
       "27   0.179709                                        (eggs)\n",
       "28   0.027063                                  (energy bar)\n",
       "29   0.026663                                (energy drink)\n",
       "..        ...                                           ...\n",
       "695  0.005066          (ground beef, shrimp, mineral water)\n",
       "696  0.005066            (ground beef, mineral water, soup)\n",
       "697  0.017064       (spaghetti, mineral water, ground beef)\n",
       "698  0.005466        (ground beef, tomatoes, mineral water)\n",
       "699  0.006133           (ground beef, spaghetti, olive oil)\n",
       "700  0.006399            (ground beef, pancakes, spaghetti)\n",
       "701  0.005999              (ground beef, shrimp, spaghetti)\n",
       "702  0.005599            (ground beef, tomatoes, spaghetti)\n",
       "703  0.005999     (herb & pepper, mineral water, spaghetti)\n",
       "704  0.005599    (spaghetti, mineral water, low fat yogurt)\n",
       "705  0.008532              (mineral water, milk, olive oil)\n",
       "706  0.007866               (pancakes, mineral water, milk)\n",
       "707  0.007866                 (shrimp, mineral water, milk)\n",
       "708  0.008532                   (mineral water, milk, soup)\n",
       "709  0.015731              (spaghetti, mineral water, milk)\n",
       "710  0.006532               (tomatoes, mineral water, milk)\n",
       "711  0.006133                 (turkey, mineral water, milk)\n",
       "712  0.005066       (whole wheat rice, mineral water, milk)\n",
       "713  0.007199                  (spaghetti, milk, olive oil)\n",
       "714  0.005866                   (tomatoes, spaghetti, milk)\n",
       "715  0.005199              (mineral water, soup, olive oil)\n",
       "716  0.010265         (spaghetti, mineral water, olive oil)\n",
       "717  0.011465          (spaghetti, pancakes, mineral water)\n",
       "718  0.006799            (spaghetti, mineral water, salmon)\n",
       "719  0.008532            (spaghetti, shrimp, mineral water)\n",
       "720  0.007466              (spaghetti, mineral water, soup)\n",
       "721  0.009332          (spaghetti, tomatoes, mineral water)\n",
       "722  0.006399            (spaghetti, turkey, mineral water)\n",
       "723  0.006266  (spaghetti, whole wheat rice, mineral water)\n",
       "724  0.005066              (pancakes, spaghetti, olive oil)\n",
       "\n",
       "[725 rows x 2 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from mlxtend.frequent_patterns import apriori\n",
    "from mlxtend.preprocessing import TransactionEncoder\n",
    "\n",
    "# we first need to create a one-hot encoding of our transactions\n",
    "te = TransactionEncoder()\n",
    "te_ary = te.fit_transform(records)\n",
    "data_df = pd.DataFrame(te_ary, columns=te.columns_)\n",
    "\n",
    "# use support of 0.005 - low threshold may include to many candidates\n",
    "# careful selection of rules based on other metrics required\n",
    "# this means that we use a higher confidence\n",
    "frequent_itemsets = apriori(pd.DataFrame(\n",
    "    data_df), min_support=0.005, use_colnames=True)\n",
    "frequent_itemsets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mining rules from the frequent itemsets\n",
    "\n",
    "Determine good rules from the results for this data. Use lift and confidence as metrics for your evaluations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>antecedents</th>\n",
       "      <th>consequents</th>\n",
       "      <th>antecedent support</th>\n",
       "      <th>consequent support</th>\n",
       "      <th>support</th>\n",
       "      <th>confidence</th>\n",
       "      <th>lift</th>\n",
       "      <th>leverage</th>\n",
       "      <th>conviction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>(ground beef, shrimp)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.011465</td>\n",
       "      <td>0.174110</td>\n",
       "      <td>0.005999</td>\n",
       "      <td>0.523256</td>\n",
       "      <td>3.005315</td>\n",
       "      <td>0.004003</td>\n",
       "      <td>1.732354</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>(frozen vegetables, ground beef)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.016931</td>\n",
       "      <td>0.174110</td>\n",
       "      <td>0.008666</td>\n",
       "      <td>0.511811</td>\n",
       "      <td>2.939582</td>\n",
       "      <td>0.005718</td>\n",
       "      <td>1.691742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>(frozen vegetables, olive oil)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.011332</td>\n",
       "      <td>0.174110</td>\n",
       "      <td>0.005733</td>\n",
       "      <td>0.505882</td>\n",
       "      <td>2.905531</td>\n",
       "      <td>0.003760</td>\n",
       "      <td>1.671444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>(frozen vegetables, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.007999</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.005066</td>\n",
       "      <td>0.633333</td>\n",
       "      <td>2.656954</td>\n",
       "      <td>0.003159</td>\n",
       "      <td>2.077178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>(olive oil, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.008932</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.005199</td>\n",
       "      <td>0.582090</td>\n",
       "      <td>2.441976</td>\n",
       "      <td>0.003070</td>\n",
       "      <td>1.822476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>(frozen vegetables, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011332</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.006532</td>\n",
       "      <td>0.576471</td>\n",
       "      <td>2.418404</td>\n",
       "      <td>0.003831</td>\n",
       "      <td>1.798297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>(milk, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.015198</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.008532</td>\n",
       "      <td>0.561404</td>\n",
       "      <td>2.355194</td>\n",
       "      <td>0.004909</td>\n",
       "      <td>1.736520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>(chocolate, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010132</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.005599</td>\n",
       "      <td>0.552632</td>\n",
       "      <td>2.318395</td>\n",
       "      <td>0.003184</td>\n",
       "      <td>1.702471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>(eggs, cooking oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011732</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.006399</td>\n",
       "      <td>0.545455</td>\n",
       "      <td>2.288286</td>\n",
       "      <td>0.003603</td>\n",
       "      <td>1.675590</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>(frozen vegetables, ground beef)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.016931</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.009199</td>\n",
       "      <td>0.543307</td>\n",
       "      <td>2.279277</td>\n",
       "      <td>0.005163</td>\n",
       "      <td>1.667711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>(turkey, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011332</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.006133</td>\n",
       "      <td>0.541176</td>\n",
       "      <td>2.270338</td>\n",
       "      <td>0.003431</td>\n",
       "      <td>1.659967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>(spaghetti, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.014265</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.007466</td>\n",
       "      <td>0.523364</td>\n",
       "      <td>2.195614</td>\n",
       "      <td>0.004065</td>\n",
       "      <td>1.597933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>(ground beef, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009732</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.005066</td>\n",
       "      <td>0.520548</td>\n",
       "      <td>2.183798</td>\n",
       "      <td>0.002746</td>\n",
       "      <td>1.588546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>(chocolate, chicken)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.014665</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.007599</td>\n",
       "      <td>0.518182</td>\n",
       "      <td>2.173871</td>\n",
       "      <td>0.004103</td>\n",
       "      <td>1.580745</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>(ground beef, pancakes)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.014531</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.007466</td>\n",
       "      <td>0.513761</td>\n",
       "      <td>2.155327</td>\n",
       "      <td>0.004002</td>\n",
       "      <td>1.566375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>(ground beef, eggs)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.019997</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.010132</td>\n",
       "      <td>0.506667</td>\n",
       "      <td>2.125563</td>\n",
       "      <td>0.005365</td>\n",
       "      <td>1.543848</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>(spaghetti, salmon)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.013465</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.006799</td>\n",
       "      <td>0.504950</td>\n",
       "      <td>2.118363</td>\n",
       "      <td>0.003589</td>\n",
       "      <td>1.538496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>(chocolate, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.016398</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.008266</td>\n",
       "      <td>0.504065</td>\n",
       "      <td>2.114649</td>\n",
       "      <td>0.004357</td>\n",
       "      <td>1.535749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>(ground beef, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.021997</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.011065</td>\n",
       "      <td>0.503030</td>\n",
       "      <td>2.110308</td>\n",
       "      <td>0.005822</td>\n",
       "      <td>1.532552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>(milk, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.017064</td>\n",
       "      <td>0.238368</td>\n",
       "      <td>0.008532</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.097595</td>\n",
       "      <td>0.004465</td>\n",
       "      <td>1.523264</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         antecedents      consequents  antecedent support  \\\n",
       "13             (ground beef, shrimp)      (spaghetti)            0.011465   \n",
       "6   (frozen vegetables, ground beef)      (spaghetti)            0.016931   \n",
       "9     (frozen vegetables, olive oil)      (spaghetti)            0.011332   \n",
       "8          (frozen vegetables, soup)  (mineral water)            0.007999   \n",
       "17                 (olive oil, soup)  (mineral water)            0.008932   \n",
       "7     (frozen vegetables, olive oil)  (mineral water)            0.011332   \n",
       "15                      (milk, soup)  (mineral water)            0.015198   \n",
       "2                  (chocolate, soup)  (mineral water)            0.010132   \n",
       "3                (eggs, cooking oil)  (mineral water)            0.011732   \n",
       "5   (frozen vegetables, ground beef)  (mineral water)            0.016931   \n",
       "16                    (turkey, milk)  (mineral water)            0.011332   \n",
       "19                 (spaghetti, soup)  (mineral water)            0.014265   \n",
       "12               (ground beef, soup)  (mineral water)            0.009732   \n",
       "0               (chocolate, chicken)  (mineral water)            0.014665   \n",
       "11           (ground beef, pancakes)  (mineral water)            0.014531   \n",
       "4                (ground beef, eggs)  (mineral water)            0.019997   \n",
       "18               (spaghetti, salmon)  (mineral water)            0.013465   \n",
       "1             (chocolate, olive oil)  (mineral water)            0.016398   \n",
       "10               (ground beef, milk)  (mineral water)            0.021997   \n",
       "14                 (milk, olive oil)  (mineral water)            0.017064   \n",
       "\n",
       "    consequent support   support  confidence      lift  leverage  conviction  \n",
       "13            0.174110  0.005999    0.523256  3.005315  0.004003    1.732354  \n",
       "6             0.174110  0.008666    0.511811  2.939582  0.005718    1.691742  \n",
       "9             0.174110  0.005733    0.505882  2.905531  0.003760    1.671444  \n",
       "8             0.238368  0.005066    0.633333  2.656954  0.003159    2.077178  \n",
       "17            0.238368  0.005199    0.582090  2.441976  0.003070    1.822476  \n",
       "7             0.238368  0.006532    0.576471  2.418404  0.003831    1.798297  \n",
       "15            0.238368  0.008532    0.561404  2.355194  0.004909    1.736520  \n",
       "2             0.238368  0.005599    0.552632  2.318395  0.003184    1.702471  \n",
       "3             0.238368  0.006399    0.545455  2.288286  0.003603    1.675590  \n",
       "5             0.238368  0.009199    0.543307  2.279277  0.005163    1.667711  \n",
       "16            0.238368  0.006133    0.541176  2.270338  0.003431    1.659967  \n",
       "19            0.238368  0.007466    0.523364  2.195614  0.004065    1.597933  \n",
       "12            0.238368  0.005066    0.520548  2.183798  0.002746    1.588546  \n",
       "0             0.238368  0.007599    0.518182  2.173871  0.004103    1.580745  \n",
       "11            0.238368  0.007466    0.513761  2.155327  0.004002    1.566375  \n",
       "4             0.238368  0.010132    0.506667  2.125563  0.005365    1.543848  \n",
       "18            0.238368  0.006799    0.504950  2.118363  0.003589    1.538496  \n",
       "1             0.238368  0.008266    0.504065  2.114649  0.004357    1.535749  \n",
       "10            0.238368  0.011065    0.503030  2.110308  0.005822    1.532552  \n",
       "14            0.238368  0.008532    0.500000  2.097595  0.004465    1.523264  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from mlxtend.frequent_patterns import association_rules\n",
    "\n",
    "# use a high confidence to counterbalance the low support threshold\n",
    "# order by lift to have the \"best\" rules at the top\n",
    "association_rules(frequent_itemsets, metric=\"confidence\",\n",
    "                  min_threshold=0.5).sort_values('lift', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The rules make sense, but are only helpful in a limited way, because we only have two different items as consequent: spaghetti and mineral water. The first three rules indicate that if a person buys a product that is often used to make pasta sauce (e.g., ground beef, vegetables, olive oil), they also often buy pasta. While this makes sense, this rule may also be there, because people tend to buy pasta a lot. However, the lift of about 3 indicates that these rules are three times as likely as a random effect. The shrimp does not really match this pattern. \n",
    "\n",
    "The other rules are likely effective, but an random artifact: the problem is that mineral water is part of many transactions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.23836821757099053"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the mean of a one hot encoded column is the percentage that this value occurs\n",
    "data_df['mineral water'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, the rule to predict mineral water is almost universally good. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validation of the rules\n",
    "\n",
    "Randomly split your records into two sets with roughly 50% of data each. Now use the Apriori algorithm to determine rules on both of these sets. Do you find similar rules on both sets? What does the similarity/the differences indicate?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# split the data into two sets with 50% of the data\n",
    "X_train, X_test = train_test_split(data_df, test_size=0.5, random_state=42)\n",
    "\n",
    "# create frequent itemsets\n",
    "frequent_itemsets_train = apriori(pd.DataFrame(\n",
    "    X_train), min_support=0.005, use_colnames=True)\n",
    "frequent_itemsets_test = apriori(pd.DataFrame(\n",
    "    X_test), min_support=0.005, use_colnames=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>antecedents</th>\n",
       "      <th>consequents</th>\n",
       "      <th>antecedent support</th>\n",
       "      <th>consequent support</th>\n",
       "      <th>support</th>\n",
       "      <th>confidence</th>\n",
       "      <th>lift</th>\n",
       "      <th>leverage</th>\n",
       "      <th>conviction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>(spaghetti, whole wheat pasta)</td>\n",
       "      <td>(milk)</td>\n",
       "      <td>0.010133</td>\n",
       "      <td>0.135733</td>\n",
       "      <td>0.005067</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>3.683694</td>\n",
       "      <td>0.003691</td>\n",
       "      <td>1.728533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>(grated cheese, eggs)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.009600</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.005867</td>\n",
       "      <td>0.611111</td>\n",
       "      <td>3.370098</td>\n",
       "      <td>0.004126</td>\n",
       "      <td>2.105143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>(frozen vegetables, soup)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.008533</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.005067</td>\n",
       "      <td>0.593750</td>\n",
       "      <td>3.274357</td>\n",
       "      <td>0.003519</td>\n",
       "      <td>2.015179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>(frozen vegetables, ground beef)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.016800</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.009600</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>3.151261</td>\n",
       "      <td>0.006554</td>\n",
       "      <td>1.910222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>(ground beef, shrimp)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.012267</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.006933</td>\n",
       "      <td>0.565217</td>\n",
       "      <td>3.117008</td>\n",
       "      <td>0.004709</td>\n",
       "      <td>1.882933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>(frozen vegetables, mineral water, milk)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.011200</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.006133</td>\n",
       "      <td>0.547619</td>\n",
       "      <td>3.019958</td>\n",
       "      <td>0.004102</td>\n",
       "      <td>1.809684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>(frozen vegetables, olive oil)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.011200</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.005867</td>\n",
       "      <td>0.523810</td>\n",
       "      <td>2.888655</td>\n",
       "      <td>0.003836</td>\n",
       "      <td>1.719200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>(cake, chocolate)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.013333</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.006933</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>2.867647</td>\n",
       "      <td>0.004516</td>\n",
       "      <td>1.705556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>(herb &amp; pepper, eggs)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.012800</td>\n",
       "      <td>0.181333</td>\n",
       "      <td>0.006400</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.757353</td>\n",
       "      <td>0.004079</td>\n",
       "      <td>1.637333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>(spaghetti, milk, eggs)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009067</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006133</td>\n",
       "      <td>0.676471</td>\n",
       "      <td>2.710219</td>\n",
       "      <td>0.003870</td>\n",
       "      <td>2.319418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>(frozen vegetables, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.008533</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005600</td>\n",
       "      <td>0.656250</td>\n",
       "      <td>2.629207</td>\n",
       "      <td>0.003470</td>\n",
       "      <td>2.182982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>(pancakes, cooking oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.008267</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005333</td>\n",
       "      <td>0.645161</td>\n",
       "      <td>2.584781</td>\n",
       "      <td>0.003270</td>\n",
       "      <td>2.114764</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>(pancakes, low fat yogurt)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010133</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006133</td>\n",
       "      <td>0.605263</td>\n",
       "      <td>2.424933</td>\n",
       "      <td>0.003604</td>\n",
       "      <td>1.901013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>(frozen vegetables, spaghetti, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010133</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006133</td>\n",
       "      <td>0.605263</td>\n",
       "      <td>2.424933</td>\n",
       "      <td>0.003604</td>\n",
       "      <td>1.901013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>(grated cheese, ground beef)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011467</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006667</td>\n",
       "      <td>0.581395</td>\n",
       "      <td>2.329308</td>\n",
       "      <td>0.003805</td>\n",
       "      <td>1.792622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>(frozen vegetables, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011200</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006400</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>2.289377</td>\n",
       "      <td>0.003604</td>\n",
       "      <td>1.750933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>(ground beef, pancakes)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.014667</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.008267</td>\n",
       "      <td>0.563636</td>\n",
       "      <td>2.258159</td>\n",
       "      <td>0.004606</td>\n",
       "      <td>1.719667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>(spaghetti, cooking oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.016267</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.009067</td>\n",
       "      <td>0.557377</td>\n",
       "      <td>2.233081</td>\n",
       "      <td>0.005007</td>\n",
       "      <td>1.695348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>(frozen vegetables, ground beef)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.016800</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.009333</td>\n",
       "      <td>0.555556</td>\n",
       "      <td>2.225783</td>\n",
       "      <td>0.005140</td>\n",
       "      <td>1.688400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>(olive oil, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009333</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005067</td>\n",
       "      <td>0.542857</td>\n",
       "      <td>2.174908</td>\n",
       "      <td>0.002737</td>\n",
       "      <td>1.641500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>(milk, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.016267</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.008800</td>\n",
       "      <td>0.540984</td>\n",
       "      <td>2.167402</td>\n",
       "      <td>0.004740</td>\n",
       "      <td>1.634800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>(ground beef, low fat yogurt)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009867</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005333</td>\n",
       "      <td>0.540541</td>\n",
       "      <td>2.165627</td>\n",
       "      <td>0.002871</td>\n",
       "      <td>1.633224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>(ground beef, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.020267</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.010933</td>\n",
       "      <td>0.539474</td>\n",
       "      <td>2.161353</td>\n",
       "      <td>0.005875</td>\n",
       "      <td>1.629440</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>(eggs, cooking oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.012533</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006667</td>\n",
       "      <td>0.531915</td>\n",
       "      <td>2.131069</td>\n",
       "      <td>0.003538</td>\n",
       "      <td>1.603127</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>(pancakes, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.018667</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.009867</td>\n",
       "      <td>0.528571</td>\n",
       "      <td>2.117674</td>\n",
       "      <td>0.005207</td>\n",
       "      <td>1.591758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>(grated cheese, eggs)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009600</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005067</td>\n",
       "      <td>0.527778</td>\n",
       "      <td>2.114494</td>\n",
       "      <td>0.002671</td>\n",
       "      <td>1.589082</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>(ground beef, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010133</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005333</td>\n",
       "      <td>0.526316</td>\n",
       "      <td>2.108637</td>\n",
       "      <td>0.002804</td>\n",
       "      <td>1.584178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>(spaghetti, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.015467</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.008000</td>\n",
       "      <td>0.517241</td>\n",
       "      <td>2.072281</td>\n",
       "      <td>0.004140</td>\n",
       "      <td>1.554400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>(milk, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.018667</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.009600</td>\n",
       "      <td>0.514286</td>\n",
       "      <td>2.060440</td>\n",
       "      <td>0.004941</td>\n",
       "      <td>1.544941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>(ground beef, spaghetti, chocolate)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009867</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005067</td>\n",
       "      <td>0.513514</td>\n",
       "      <td>2.057346</td>\n",
       "      <td>0.002604</td>\n",
       "      <td>1.542489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>(eggs, red wine)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009867</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005067</td>\n",
       "      <td>0.513514</td>\n",
       "      <td>2.057346</td>\n",
       "      <td>0.002604</td>\n",
       "      <td>1.542489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>(herb &amp; pepper, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010400</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005333</td>\n",
       "      <td>0.512821</td>\n",
       "      <td>2.054569</td>\n",
       "      <td>0.002737</td>\n",
       "      <td>1.540295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>(frozen vegetables, whole wheat rice)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010933</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.005600</td>\n",
       "      <td>0.512195</td>\n",
       "      <td>2.052064</td>\n",
       "      <td>0.002871</td>\n",
       "      <td>1.538320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>(spaghetti, salmon)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.015733</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.008000</td>\n",
       "      <td>0.508475</td>\n",
       "      <td>2.037158</td>\n",
       "      <td>0.004073</td>\n",
       "      <td>1.526676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>(eggs, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.013333</td>\n",
       "      <td>0.249600</td>\n",
       "      <td>0.006667</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.003205</td>\n",
       "      <td>0.003339</td>\n",
       "      <td>1.500800</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 antecedents      consequents  \\\n",
       "27            (spaghetti, whole wheat pasta)           (milk)   \n",
       "5                      (grated cheese, eggs)      (spaghetti)   \n",
       "15                 (frozen vegetables, soup)      (spaghetti)   \n",
       "10          (frozen vegetables, ground beef)      (spaghetti)   \n",
       "21                     (ground beef, shrimp)      (spaghetti)   \n",
       "33  (frozen vegetables, mineral water, milk)      (spaghetti)   \n",
       "14            (frozen vegetables, olive oil)      (spaghetti)   \n",
       "0                          (cake, chocolate)      (spaghetti)   \n",
       "6                      (herb & pepper, eggs)      (spaghetti)   \n",
       "32                   (spaghetti, milk, eggs)  (mineral water)   \n",
       "12                 (frozen vegetables, soup)  (mineral water)   \n",
       "2                    (pancakes, cooking oil)  (mineral water)   \n",
       "23                (pancakes, low fat yogurt)  (mineral water)   \n",
       "34      (frozen vegetables, spaghetti, milk)  (mineral water)   \n",
       "16              (grated cheese, ground beef)  (mineral water)   \n",
       "11            (frozen vegetables, olive oil)  (mineral water)   \n",
       "19                   (ground beef, pancakes)  (mineral water)   \n",
       "3                   (spaghetti, cooking oil)  (mineral water)   \n",
       "9           (frozen vegetables, ground beef)  (mineral water)   \n",
       "28                         (olive oil, soup)  (mineral water)   \n",
       "26                              (milk, soup)  (mineral water)   \n",
       "17             (ground beef, low fat yogurt)  (mineral water)   \n",
       "18                       (ground beef, milk)  (mineral water)   \n",
       "1                        (eggs, cooking oil)  (mineral water)   \n",
       "25                          (pancakes, milk)  (mineral water)   \n",
       "4                      (grated cheese, eggs)  (mineral water)   \n",
       "20                       (ground beef, soup)  (mineral water)   \n",
       "30                         (spaghetti, soup)  (mineral water)   \n",
       "24                         (milk, olive oil)  (mineral water)   \n",
       "31       (ground beef, spaghetti, chocolate)  (mineral water)   \n",
       "8                           (eggs, red wine)  (mineral water)   \n",
       "22                     (herb & pepper, milk)  (mineral water)   \n",
       "13     (frozen vegetables, whole wheat rice)  (mineral water)   \n",
       "29                       (spaghetti, salmon)  (mineral water)   \n",
       "7                          (eggs, olive oil)  (mineral water)   \n",
       "\n",
       "    antecedent support  consequent support   support  confidence      lift  \\\n",
       "27            0.010133            0.135733  0.005067    0.500000  3.683694   \n",
       "5             0.009600            0.181333  0.005867    0.611111  3.370098   \n",
       "15            0.008533            0.181333  0.005067    0.593750  3.274357   \n",
       "10            0.016800            0.181333  0.009600    0.571429  3.151261   \n",
       "21            0.012267            0.181333  0.006933    0.565217  3.117008   \n",
       "33            0.011200            0.181333  0.006133    0.547619  3.019958   \n",
       "14            0.011200            0.181333  0.005867    0.523810  2.888655   \n",
       "0             0.013333            0.181333  0.006933    0.520000  2.867647   \n",
       "6             0.012800            0.181333  0.006400    0.500000  2.757353   \n",
       "32            0.009067            0.249600  0.006133    0.676471  2.710219   \n",
       "12            0.008533            0.249600  0.005600    0.656250  2.629207   \n",
       "2             0.008267            0.249600  0.005333    0.645161  2.584781   \n",
       "23            0.010133            0.249600  0.006133    0.605263  2.424933   \n",
       "34            0.010133            0.249600  0.006133    0.605263  2.424933   \n",
       "16            0.011467            0.249600  0.006667    0.581395  2.329308   \n",
       "11            0.011200            0.249600  0.006400    0.571429  2.289377   \n",
       "19            0.014667            0.249600  0.008267    0.563636  2.258159   \n",
       "3             0.016267            0.249600  0.009067    0.557377  2.233081   \n",
       "9             0.016800            0.249600  0.009333    0.555556  2.225783   \n",
       "28            0.009333            0.249600  0.005067    0.542857  2.174908   \n",
       "26            0.016267            0.249600  0.008800    0.540984  2.167402   \n",
       "17            0.009867            0.249600  0.005333    0.540541  2.165627   \n",
       "18            0.020267            0.249600  0.010933    0.539474  2.161353   \n",
       "1             0.012533            0.249600  0.006667    0.531915  2.131069   \n",
       "25            0.018667            0.249600  0.009867    0.528571  2.117674   \n",
       "4             0.009600            0.249600  0.005067    0.527778  2.114494   \n",
       "20            0.010133            0.249600  0.005333    0.526316  2.108637   \n",
       "30            0.015467            0.249600  0.008000    0.517241  2.072281   \n",
       "24            0.018667            0.249600  0.009600    0.514286  2.060440   \n",
       "31            0.009867            0.249600  0.005067    0.513514  2.057346   \n",
       "8             0.009867            0.249600  0.005067    0.513514  2.057346   \n",
       "22            0.010400            0.249600  0.005333    0.512821  2.054569   \n",
       "13            0.010933            0.249600  0.005600    0.512195  2.052064   \n",
       "29            0.015733            0.249600  0.008000    0.508475  2.037158   \n",
       "7             0.013333            0.249600  0.006667    0.500000  2.003205   \n",
       "\n",
       "    leverage  conviction  \n",
       "27  0.003691    1.728533  \n",
       "5   0.004126    2.105143  \n",
       "15  0.003519    2.015179  \n",
       "10  0.006554    1.910222  \n",
       "21  0.004709    1.882933  \n",
       "33  0.004102    1.809684  \n",
       "14  0.003836    1.719200  \n",
       "0   0.004516    1.705556  \n",
       "6   0.004079    1.637333  \n",
       "32  0.003870    2.319418  \n",
       "12  0.003470    2.182982  \n",
       "2   0.003270    2.114764  \n",
       "23  0.003604    1.901013  \n",
       "34  0.003604    1.901013  \n",
       "16  0.003805    1.792622  \n",
       "11  0.003604    1.750933  \n",
       "19  0.004606    1.719667  \n",
       "3   0.005007    1.695348  \n",
       "9   0.005140    1.688400  \n",
       "28  0.002737    1.641500  \n",
       "26  0.004740    1.634800  \n",
       "17  0.002871    1.633224  \n",
       "18  0.005875    1.629440  \n",
       "1   0.003538    1.603127  \n",
       "25  0.005207    1.591758  \n",
       "4   0.002671    1.589082  \n",
       "20  0.002804    1.584178  \n",
       "30  0.004140    1.554400  \n",
       "24  0.004941    1.544941  \n",
       "31  0.002604    1.542489  \n",
       "8   0.002604    1.542489  \n",
       "22  0.002737    1.540295  \n",
       "13  0.002871    1.538320  \n",
       "29  0.004073    1.526676  \n",
       "7   0.003339    1.500800  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# rules for first set\n",
    "association_rules(frequent_itemsets_train, metric=\"confidence\",\n",
    "                  min_threshold=0.5).sort_values('lift', ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>antecedents</th>\n",
       "      <th>consequents</th>\n",
       "      <th>antecedent support</th>\n",
       "      <th>consequent support</th>\n",
       "      <th>support</th>\n",
       "      <th>confidence</th>\n",
       "      <th>lift</th>\n",
       "      <th>leverage</th>\n",
       "      <th>conviction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>(pancakes, olive oil)</td>\n",
       "      <td>(spaghetti)</td>\n",
       "      <td>0.010930</td>\n",
       "      <td>0.166889</td>\n",
       "      <td>0.005599</td>\n",
       "      <td>0.512195</td>\n",
       "      <td>3.069080</td>\n",
       "      <td>0.003774</td>\n",
       "      <td>1.707878</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>(tomatoes, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.007465</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005065</td>\n",
       "      <td>0.678571</td>\n",
       "      <td>2.987466</td>\n",
       "      <td>0.003370</td>\n",
       "      <td>2.404455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>(chocolate, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.008798</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>2.935055</td>\n",
       "      <td>0.003867</td>\n",
       "      <td>2.318582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>(olive oil, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.008531</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005332</td>\n",
       "      <td>0.625000</td>\n",
       "      <td>2.751614</td>\n",
       "      <td>0.003394</td>\n",
       "      <td>2.060962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>(turkey, milk)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011464</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006931</td>\n",
       "      <td>0.604651</td>\n",
       "      <td>2.662026</td>\n",
       "      <td>0.004328</td>\n",
       "      <td>1.954883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>(shrimp, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009064</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005332</td>\n",
       "      <td>0.588235</td>\n",
       "      <td>2.589754</td>\n",
       "      <td>0.003273</td>\n",
       "      <td>1.876947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>(milk, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.014130</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.008264</td>\n",
       "      <td>0.584906</td>\n",
       "      <td>2.575095</td>\n",
       "      <td>0.005055</td>\n",
       "      <td>1.861891</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>(frozen vegetables, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011464</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006665</td>\n",
       "      <td>0.581395</td>\n",
       "      <td>2.559641</td>\n",
       "      <td>0.004061</td>\n",
       "      <td>1.846278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>(chocolate, chicken)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.012797</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.007198</td>\n",
       "      <td>0.562500</td>\n",
       "      <td>2.476452</td>\n",
       "      <td>0.004291</td>\n",
       "      <td>1.766538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>(eggs, cooking oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010930</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006132</td>\n",
       "      <td>0.560976</td>\n",
       "      <td>2.469741</td>\n",
       "      <td>0.003649</td>\n",
       "      <td>1.760405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>(eggs, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009064</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005065</td>\n",
       "      <td>0.558824</td>\n",
       "      <td>2.460267</td>\n",
       "      <td>0.003006</td>\n",
       "      <td>1.751817</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>(pancakes, chicken)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.009064</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005065</td>\n",
       "      <td>0.558824</td>\n",
       "      <td>2.460267</td>\n",
       "      <td>0.003006</td>\n",
       "      <td>1.751817</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>(chocolate, olive oil)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.014396</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.007998</td>\n",
       "      <td>0.555556</td>\n",
       "      <td>2.445879</td>\n",
       "      <td>0.004728</td>\n",
       "      <td>1.738936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>(frozen vegetables, ground beef)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.017062</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.009064</td>\n",
       "      <td>0.531250</td>\n",
       "      <td>2.338872</td>\n",
       "      <td>0.005189</td>\n",
       "      <td>1.648769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>(spaghetti, soup)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.013063</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006931</td>\n",
       "      <td>0.530612</td>\n",
       "      <td>2.336064</td>\n",
       "      <td>0.003964</td>\n",
       "      <td>1.646529</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>(ground beef, eggs)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.018662</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.009864</td>\n",
       "      <td>0.528571</td>\n",
       "      <td>2.327079</td>\n",
       "      <td>0.005625</td>\n",
       "      <td>1.639401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>(ground beef, turkey)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.010397</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005332</td>\n",
       "      <td>0.512821</td>\n",
       "      <td>2.257734</td>\n",
       "      <td>0.002970</td>\n",
       "      <td>1.586398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>(whole wheat rice, spaghetti)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.013863</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006931</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.201291</td>\n",
       "      <td>0.003783</td>\n",
       "      <td>1.545721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>(light cream)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.012263</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006132</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.201291</td>\n",
       "      <td>0.003346</td>\n",
       "      <td>1.545721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>(spaghetti, salmon)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.011197</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.005599</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.201291</td>\n",
       "      <td>0.003055</td>\n",
       "      <td>1.545721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>(ground beef, tomatoes)</td>\n",
       "      <td>(mineral water)</td>\n",
       "      <td>0.012263</td>\n",
       "      <td>0.227139</td>\n",
       "      <td>0.006132</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>2.201291</td>\n",
       "      <td>0.003346</td>\n",
       "      <td>1.545721</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         antecedents      consequents  antecedent support  \\\n",
       "20             (pancakes, olive oil)      (spaghetti)            0.010930   \n",
       "16             (tomatoes, olive oil)  (mineral water)            0.007465   \n",
       "4                  (chocolate, soup)  (mineral water)            0.008798   \n",
       "15                 (olive oil, soup)  (mineral water)            0.008531   \n",
       "13                    (turkey, milk)  (mineral water)            0.011464   \n",
       "14               (shrimp, olive oil)  (mineral water)            0.009064   \n",
       "12                      (milk, soup)  (mineral water)            0.014130   \n",
       "9     (frozen vegetables, olive oil)  (mineral water)            0.011464   \n",
       "1               (chocolate, chicken)  (mineral water)            0.012797   \n",
       "5                (eggs, cooking oil)  (mineral water)            0.010930   \n",
       "7                       (eggs, soup)  (mineral water)            0.009064   \n",
       "2                (pancakes, chicken)  (mineral water)            0.009064   \n",
       "3             (chocolate, olive oil)  (mineral water)            0.014396   \n",
       "8   (frozen vegetables, ground beef)  (mineral water)            0.017062   \n",
       "18                 (spaghetti, soup)  (mineral water)            0.013063   \n",
       "6                (ground beef, eggs)  (mineral water)            0.018662   \n",
       "11             (ground beef, turkey)  (mineral water)            0.010397   \n",
       "19     (whole wheat rice, spaghetti)  (mineral water)            0.013863   \n",
       "0                      (light cream)  (mineral water)            0.012263   \n",
       "17               (spaghetti, salmon)  (mineral water)            0.011197   \n",
       "10           (ground beef, tomatoes)  (mineral water)            0.012263   \n",
       "\n",
       "    consequent support   support  confidence      lift  leverage  conviction  \n",
       "20            0.166889  0.005599    0.512195  3.069080  0.003774    1.707878  \n",
       "16            0.227139  0.005065    0.678571  2.987466  0.003370    2.404455  \n",
       "4             0.227139  0.005865    0.666667  2.935055  0.003867    2.318582  \n",
       "15            0.227139  0.005332    0.625000  2.751614  0.003394    2.060962  \n",
       "13            0.227139  0.006931    0.604651  2.662026  0.004328    1.954883  \n",
       "14            0.227139  0.005332    0.588235  2.589754  0.003273    1.876947  \n",
       "12            0.227139  0.008264    0.584906  2.575095  0.005055    1.861891  \n",
       "9             0.227139  0.006665    0.581395  2.559641  0.004061    1.846278  \n",
       "1             0.227139  0.007198    0.562500  2.476452  0.004291    1.766538  \n",
       "5             0.227139  0.006132    0.560976  2.469741  0.003649    1.760405  \n",
       "7             0.227139  0.005065    0.558824  2.460267  0.003006    1.751817  \n",
       "2             0.227139  0.005065    0.558824  2.460267  0.003006    1.751817  \n",
       "3             0.227139  0.007998    0.555556  2.445879  0.004728    1.738936  \n",
       "8             0.227139  0.009064    0.531250  2.338872  0.005189    1.648769  \n",
       "18            0.227139  0.006931    0.530612  2.336064  0.003964    1.646529  \n",
       "6             0.227139  0.009864    0.528571  2.327079  0.005625    1.639401  \n",
       "11            0.227139  0.005332    0.512821  2.257734  0.002970    1.586398  \n",
       "19            0.227139  0.006931    0.500000  2.201291  0.003783    1.545721  \n",
       "0             0.227139  0.006132    0.500000  2.201291  0.003346    1.545721  \n",
       "17            0.227139  0.005599    0.500000  2.201291  0.003055    1.545721  \n",
       "10            0.227139  0.006132    0.500000  2.201291  0.003346    1.545721  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# rules for second set\n",
    "association_rules(frequent_itemsets_test, metric=\"confidence\",\n",
    "                  min_threshold=0.5).sort_values('lift', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The \"mineral water\" rules seem to be fairly stable and are present in both splits. The \"spaghetti\" rules seem to be more random. While there are such rules in both splits, they are not the same in the splits. In general, it seems like spaghetti may also be just something that is bought very often, same as mineral water, and that the rules may be good suggestions for cross-sell, but that there is probably not a strong association between the antecedents and the consequents for all rules, i.e., there does not seem to be causality. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}