Skip to content

Instantly share code, notes, and snippets.

@lluang

lluang/WilkinsonStatisticsQuiz.ipynb

Last active December 20, 2015 08:39
Show Gist options
  • Save lluang/6101786 to your computer and use it in GitHub Desktop.
Save lluang/6101786 to your computer and use it in GitHub Desktop.
Download ZIP
This is an implementation of the Wilkinson Statistics Quiz of the Python data analysis libraries applied to the **NASTY** dataset. Currently based on Pandas. Todo: (1) Finish implementation and confirm that these are correct (2) Apply to other libraries (3) Write up bug submissions as appropriate.
Display the source blob
Display the rendered blob
Raw
WilkinsonStatisticsQuiz.ipynb
{
"metadata": {
"name": "wilkinson_statistics_quiz.ipynb"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Wilkinson Statistics Quiz\n",
"=========================\n",
"\n",
"McCullogh and Yalta (2013) [1] recently evaluated the statistical functions in several well known cloud-based spreadsheet programs using a fairly basic set of tests [2]. They found, like the majority of their PC based counterparts, that the statistical functions in the spreadsheets fail. They note that there are well known algorithms that can in fact pass these tests using double precision therefore the problem is in the naive choices of algorithms used in computing these standard statistics. This is an implementation of the Wilkinson Statistics Quiz [3] of the Python data analysis libraries applied to the **NASTY** dataset. The data set dictionary *nastydict* below was created by using the pandas.read_csv('nasty.dat') as found at [1], then it was re-exported using pandas.to_dict(). Finally, some newlines were removed in the interest of aesthetics. No other processing was done to the data (i.e. the *nan* values found in dataset **MISS** were processed correctly by pandas on read.)\n",
"\n",
"[1] McCullough, Bruce D, and A Talha Yalta. 2013. \u201cSpreadsheets in the Cloud - Not Ready Yet.\u201d Journal of Statistical Software 52 (7): 1\u201314. http://www.jstatsoft.org/v52/i07.\n",
" \n",
"[2] Wilkinson, Leland. 1994. \u201cPractical Guidelines for Testing Statistical Software.\u201d In Computational Statistics: Papers Collected on the Occasion of the 25th Conference on Statistical Computing at Schlo\u00df Reisensburg (Contributions to Statistics), edited by Peter Dirschedl and R\u00fcdiger Ostermann, 553. Physica-Verlag HD. ISBN: 379080813X\n",
" \n",
"[3] Wilkinson, Leland. 1985. \u201cSTATISTICS QUIZ.\u201d http://www.stanford.edu/~clint/bench/wilk.txt."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import numpy as np\n",
"import pandas as pd\n",
"print(np.__version__)\n",
"print(pd.__version__)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"1.7.1\n",
"0.12.0\n"
]
}
],
"prompt_number": 1
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"nastydict = {'BIG': {0: 99999991, 1: 99999992, 2: 99999993, 3: 99999994, 4: 99999995,\n",
" 5: 99999996, 6: 99999997, 7: 99999998, 8: 99999999},\n",
" 'HUGE': {0: 1000000000000.0, 1: 2000000000000.0, 2: 3000000000000.0,\n",
" 3: 4000000000000.0, 4: 5000000000000.0, 5: 6000000000000.0,\n",
" 6: 7000000000000.0, 7: 8000000000000.0, 8: 9000000000000.0},\n",
" 'LITTLE': {0: 0.99999990999999999, 1: 0.99999992000000004, 2: 0.99999992999999998,\n",
" 3: 0.99999994000000003, 4: 0.99999994999999997, 5: 0.99999996000000002,\n",
" 6: 0.99999996999999996, 7: 0.99999998000000001, 8: 0.99999998999999995},\n",
" 'MISS': {0: nan, 1: nan, 2: nan, 3: nan, 4: nan, 5: nan,\n",
" 6: nan, 7: nan, 8: nan},\n",
" 'OBS': {0: 'ONE', 1: 'TWO', 2: 'THREE', 3: 'FOUR', 4: 'FIVE', 5: 'SIX',\n",
" 6: 'SEVEN', 7: 'EIGHT', 8: 'NINE'},\n",
" 'ROUND': {0: 0.5, 1: 1.5, 2: 2.5, 3: 3.5, 4: 4.5, 5: 5.5,\n",
" 6: 6.5, 7: 7.5, 8: 8.5},\n",
" 'TINY': {0: 9.9999999999999998e-13, 1: 2e-12, 2: 2.9999999999999997e-12,\n",
" 3: 3.9999999999999999e-12, 4: 5.0000000000000005e-12, 5: 5.9999999999999995e-12,\n",
" 6: 7.0000000000000001e-12, 7: 7.9999999999999998e-12, 8: 8.9999999999999996e-12},\n",
" 'X': {0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9},\n",
" 'ZERO': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0}}"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 2
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"nasty = pd.DataFrame.from_dict(nastydict)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 3
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"nasty"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>BIG</th>\n",
" <th>HUGE</th>\n",
" <th>LITTLE</th>\n",
" <th>MISS</th>\n",
" <th>OBS</th>\n",
" <th>ROUND</th>\n",
" <th>TINY</th>\n",
" <th>X</th>\n",
" <th>ZERO</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td> 99999991</td>\n",
" <td> 1.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> ONE</td>\n",
" <td> 0.5</td>\n",
" <td> 1.000000e-12</td>\n",
" <td> 1</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td> 99999992</td>\n",
" <td> 2.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> TWO</td>\n",
" <td> 1.5</td>\n",
" <td> 2.000000e-12</td>\n",
" <td> 2</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td> 99999993</td>\n",
" <td> 3.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> THREE</td>\n",
" <td> 2.5</td>\n",
" <td> 3.000000e-12</td>\n",
" <td> 3</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td> 99999994</td>\n",
" <td> 4.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> FOUR</td>\n",
" <td> 3.5</td>\n",
" <td> 4.000000e-12</td>\n",
" <td> 4</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td> 99999995</td>\n",
" <td> 5.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> FIVE</td>\n",
" <td> 4.5</td>\n",
" <td> 5.000000e-12</td>\n",
" <td> 5</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td> 99999996</td>\n",
" <td> 6.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> SIX</td>\n",
" <td> 5.5</td>\n",
" <td> 6.000000e-12</td>\n",
" <td> 6</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td> 99999997</td>\n",
" <td> 7.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> SEVEN</td>\n",
" <td> 6.5</td>\n",
" <td> 7.000000e-12</td>\n",
" <td> 7</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td> 99999998</td>\n",
" <td> 8.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> EIGHT</td>\n",
" <td> 7.5</td>\n",
" <td> 8.000000e-12</td>\n",
" <td> 8</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td> 99999999</td>\n",
" <td> 9.000000e+12</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> NINE</td>\n",
" <td> 8.5</td>\n",
" <td> 9.000000e-12</td>\n",
" <td> 9</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"output_type": "pyout",
"prompt_number": 4,
"text": [
" BIG HUGE LITTLE MISS OBS ROUND TINY X ZERO\n",
"0 99999991 1.000000e+12 1 NaN ONE 0.5 1.000000e-12 1 0\n",
"1 99999992 2.000000e+12 1 NaN TWO 1.5 2.000000e-12 2 0\n",
"2 99999993 3.000000e+12 1 NaN THREE 2.5 3.000000e-12 3 0\n",
"3 99999994 4.000000e+12 1 NaN FOUR 3.5 4.000000e-12 4 0\n",
"4 99999995 5.000000e+12 1 NaN FIVE 4.5 5.000000e-12 5 0\n",
"5 99999996 6.000000e+12 1 NaN SIX 5.5 6.000000e-12 6 0\n",
"6 99999997 7.000000e+12 1 NaN SEVEN 6.5 7.000000e-12 7 0\n",
"7 99999998 8.000000e+12 1 NaN EIGHT 7.5 8.000000e-12 8 0\n",
"8 99999999 9.000000e+12 1 NaN NINE 8.5 9.000000e-12 9 0"
]
}
],
"prompt_number": 4
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Real Numbers\n",
"============\n",
"\n",
"Means\n",
"-----\n",
"\n",
"C. Compute basic statistic on all the variables. The means should \n",
"be the fifth value of all the variables (case FIVE)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['X'].mean())\n",
"nasty['X'].mean()==nasty['X'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"5.0\n"
]
},
{
"output_type": "pyout",
"prompt_number": 5,
"text": [
"True"
]
}
],
"prompt_number": 5
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['ZERO'].mean())\n",
"nasty['ZERO'].mean()==nasty['ZERO'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"0.0\n"
]
},
{
"output_type": "pyout",
"prompt_number": 6,
"text": [
"True"
]
}
],
"prompt_number": 6
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['MISS'].mean())\n",
"nasty['MISS'].mean()==nasty['MISS'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"nan\n"
]
},
{
"output_type": "pyout",
"prompt_number": 7,
"text": [
"False"
]
}
],
"prompt_number": 7
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['BIG'].mean())\n",
"nasty['BIG'].mean()==nasty['BIG'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"99999995.0\n"
]
},
{
"output_type": "pyout",
"prompt_number": 8,
"text": [
"True"
]
}
],
"prompt_number": 8
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['LITTLE'].mean())\n",
"nasty['LITTLE'].mean()==nasty['LITTLE'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"0.99999995\n"
]
},
{
"output_type": "pyout",
"prompt_number": 9,
"text": [
"True"
]
}
],
"prompt_number": 9
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['HUGE'].mean())\n",
"nasty['HUGE'].mean()==nasty['HUGE'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"5e+12\n"
]
},
{
"output_type": "pyout",
"prompt_number": 10,
"text": [
"True"
]
}
],
"prompt_number": 10
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['TINY'].mean())\n",
"nasty['TINY'].mean()==nasty['TINY'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"5e-12\n"
]
},
{
"output_type": "pyout",
"prompt_number": 11,
"text": [
"False"
]
}
],
"prompt_number": 11
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['ROUND'].mean())\n",
"nasty['ROUND'].mean()==nasty['ROUND'][4]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"4.5\n"
]
},
{
"output_type": "pyout",
"prompt_number": 12,
"text": [
"True"
]
}
],
"prompt_number": 12
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I currently have **TINY** failing the test, although it does print out the correct results of 5e-12. This is probably due to floating point rounding so this should be considered as a Pass. Note that *nan* always fails tests of equality for **MISS**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Standard deviation\n",
"------------------\n",
"\n",
"C. Compute basic statistic on all the variables. The standard\n",
"deviations should be \"undefined\" or missing for MISS, 0 for ZERO,\n",
"and 2.738612788 (times 10 to a power) for all the other variables.\n",
"\n",
"The standard deviation for all numerical columns is 2.738612788 times a power, except for **ZERO** and **MISS**"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(nasty['X'].std())\n",
"print(nasty['ZERO'].std())\n",
"print(nasty['MISS'].std())\n",
"print(nasty['BIG'].std())\n",
"print(nasty['LITTLE'].std())\n",
"print(nasty['HUGE'].std())\n",
"print(nasty['TINY'].std())\n",
"print(nasty['ROUND'].std())"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"2.73861278753\n",
"0.0\n",
"nan\n",
"2.44948974278\n",
"2.98023223877e-08\n",
"2.73861278753e+12\n",
"2.73861278753e-12\n",
"2.73861278753\n"
]
}
],
"prompt_number": 13
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using pandas 0.12, BIG and LITTLE fail, providing only one accurate digit each."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"PLOTTING\n",
"--------\n",
"\n",
"B. Plot HUGE against TINY in a scatterplot. The values should fall \n",
"on a line. Plot BIG against LITTLE. Again, the values should fall\n",
"on a line. Plot X against ZERO. Some programs cannot produce this\n",
"last plot because they cannot scale an axis for a constant."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import matplotlib.pyplot as plt"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 14
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"plt.plot(nasty['HUGE'], nasty['TINY'])"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 15,
"text": [
"[<matplotlib.lines.Line2D at 0x3ddd390>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEPCAYAAABfmE8WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X1UlGX6B/AvroQmRtZY7LY0WrIyEyCQCJTi5BqwsuRL\ntoJ7VkP77djZFUrUTlub0JaWpmJkOrV7PKZmW2a+4NsCnnFSXhwFlSAqDSJdrYDjqCjbCM/vj2dh\noYGZAWbmmWfm+zmncxjnnnkut7z29ppnvrePIAgCiIhItgZIXQAREfUPGzkRkcyxkRMRyRwbORGR\nzLGRExHJHBs5EZHMuayRz5s3D3fffTfCwsIc8n5JSUkYNmwYUlJSuvz6W2+9hVGjRmHAgAFoampy\nyLWIiNyZyxp5eno6Dh486LD3W7p0KbZs2WLx6+PHj0dRURGUSqXDrkVE5M5c1sgnTJiAYcOGdfm1\nCxcuYMmSJYiLi8PcuXNRW1tr9/tNmjQJ/v7+Fr8eERHBJk5EXkXSGflLL72E1NRUlJSUYNasWVi5\ncqWU5RARydJAqS5sNpuxf/9+lJeXWzy3c+dOLFu2zOLXf/nLX+LAgQOuKI+ISDYka+RtbW0YMGAA\nSktL4efn1+W5GTNmYMaMGTbfw8fHx1nlERHJhmSjFT8/P0yZMgUbNmxAa2srBEHAmTNnevUetvK+\nmAdGRN7AZiO357bB559/Hvfddx8efPBB1NTUdLsmLS0NDz30EL744gsEBQVh06ZNyMnJwaVLlzB2\n7FiEhoZiz549dhc+YcIE/O53v0NRURGCgoJQUFAAAHjzzTcRFBSECxcuIDw8HH/84x/tfk8iIjny\nsRVj++mnn8Lf3x9z5sxBZWWlxfPHjx/HokWLsGfPHhw6dAjbtm1Dfn6+0womIqKubO7Iu7ttsLOy\nsjLMnDkTd9xxB9LS0vD55587tEAiIrKu3zPy48ePQ61WdzwePnw4zp0719+3JSIiO/X7rhVBECw+\nVOzubhLeYUJE1De2btzo9448JiYG1dXVHY9/+OEH3HfffT0W4+7/LFu2TPIaWCdrZJ2ss/0fezik\nkX/88cdobGzE+++/D5VK1d+3JCKiXrA5WklLS8ORI0fQ0NCAoKAg5OTkwGw2AwC0Wi3GjRuH8ePH\nY+zYsbjjjjuwdetWpxdNRET/Y7ORb9++3eabvPbaa3jttdccUpDUNBqN1CXYhXU6jhxqBFino8ml\nTnvYvI/cYRfy8bF73kNERCJ7eidPCCIikjk2ciIimWMjJyKSOTZyIiKZYyMnIpI5NnIiIpljIyci\nkjk2ciIimWMjJyKSOTZyIiKZYyMnIpI5NnIiIpljIycikjk2ciIimWMjJyKSOZuN3GAwQKVSITg4\nGHl5eRbPX716FVlZWYiIiEBcXBzOnTvnlEKJiKh7Nht5ZmYmdDodCgsLsX79ejQ0NHR5fvv27TCb\nzTh16hTWrFmDpUuXOq1YIiKyZLWRm0wmAEB8fDyUSiUSEhJQVlbWZc3hw4eRnJwMAIiLi8PZs2ed\nVCoRkXf5/nv71lk9s9NoNCIkJKTjsVqtRmlpaUfjBoDExERs374d8fHxKCgoQGVlJWprazFy5EiL\n98vOzu74WaPReNSZeUREjqDX63H4sB5lZcCnn9r3GpuHL9sya9YsnD9/HhMnTsTo0aMRHBwMPz+/\nbtd2buRERGSptVWDHTs0CAoCKiqAkJAcm6+xeviyyWSCRqNBRUUFAGDhwoVISkrqsiPv7Nq1axg/\nfjxOnTpleSEevkxE1KP6eiArCzhxAli7Fpg6FfDxccDhywEBAQDEO1fq6upQUFCAmJiYLmtMJhN+\n/PFHXL9+HStWrMCjjz7az98OEZH3aGkBXn0ViIoCQkOB6mpg2jSxidvL5mglNzcXWq0WZrMZGRkZ\nUCgU0Ol0AACtVovq6mo8+eSTaGtrQ1xcHDZu3Njn3xARkTfJzweeeQYIDxd34iNG9O19rI5WHImj\nFSIi0dmzYgP/6ivgzTeBxMSe1/Z7tEJERI7T3Ay88AIQGwvExwOVldabuL3YyImInEwQgI8+AlQq\noK4OOH0aWLoUuOUWx7x/v28/JCKinlVVAQsXAo2NwNat4k7c0bgjJyJyApMJWLQI0GiAGTOAkyed\n08QBNnIiIodqawM2bxbHKFeuiDvyP/8ZGOjE+QdHK0REDlJeLjbtmzeBXbuAceNcc13uyImI+qmx\nEViwAJgyBZg/HygtdV0TB9jIiYj6rLUV2LhRHKPccgvw+ediIx/g4s7K0QoRUR8UF4tjlKFDgcJC\n8duZUmEjJyLqhUuXgOeeA4qKgFWrgNTU3uWiOANHK0REdjCbgTVrxGCrwEBxjJKWJn0TB7gjJyKy\nqahI/FJPUBBw7BgwerTUFXXFRk5E1IOeMsLdDUcrREQ/4YiMcFfijpyIqBNHZYS7Ehs5ERG6ZoSv\nX++YeFlXsTlaMRgMUKlUCA4ORl5ensXzN27cwNy5cxEZGYmJEydi9+7dTimUiMgZnJUR7ko2d+SZ\nmZnQ6XRQKpVITExEWloaFApFx/ObN2/GkCFDUFFRgW+++QaTJk3CY489Bh93HSYREUHMCN+xQ/ww\nc8IEMSP8nnukrqpvrDZyk8kEAIj/b/ZiQkICysrKkJyc3LEmICAAV69ehdlsRlNTE2699VY2cSJy\na1VVQEYG0NDgvIxwV7LayI1GI0JCQjoeq9VqlJaWdmnkaWlp2Lt3LxQKBW7evImSkpIe3y87O7vj\nZ41GA41G0/fKiYh6yWQCcnKALVuAl14Cnn7aufGyfaHX66HX63v1mn7/Ft566y0MHDgQFy9eRGVl\nJZKTk/HNN99gQDepMZ0bORGRq7S1ic37+efFhMKqKuCuu6Suqns/3eTm5OTYfI3VDzujo6NRU1PT\n8biqqgqxsbFd1hgMBvz+97/HrbfeipiYGPziF7/Al19+2cvSiYico7wcGD9evBNl1y7g73933ybe\nV1YbeUBAAACxWdfV1aGgoAAxMTFd1vz617/G3r170dbWhq+//hpNTU1dxjFERFKQOiPclWyOVnJz\nc6HVamE2m5GRkQGFQgGdTgcA0Gq1SE1NRXV1NcaOHYvhw4dj3bp1Ti+aiKgnra3Au++KM/DUVDHc\natgwqatyLh9BEASXXMjHBy66FBF5qc4Z4Xl50maEO4o9vdPNPq8lIuo9d8wIdyWGZhGRbLlzRrgr\ncUdORLLk7hnhrsRGTkSyIpeMcFfiaIWIZEFuGeGuxB05Ebk9OWaEuxIbORG5LTlnhLsSRytE5HY8\nISPcldjIichtCALw0UeASgXU1YkZ4UuXArfcInVl7o2jFSJyC9XV4u2EnpIR7krckRORpEwmYNEi\nQKMBpk8HTp5kE+8tNnIikkRbG/Dee+IY5coV4LPPxJwUdzvoQQ74PxkRuVx5udi0b94UM8I9NV7W\nVbgjJyKXaWwUj1fzhoxwV2IjJyKna20FNm4E1GrA11cMt5o/H+jmREjqA45WiMipOmeEFxR4Rka4\nu7H5/4cGgwEqlQrBwcHIy8uzeP6NN95AZGQkIiMjERYWhoEDB+Ly5ctOKZaI5OPSJWDuXOB3vwOW\nLAH0ejZxZ7F5QlBkZCTWrVsHpVKJxMREHD16FAqFotu1+fn5yM3NRWFhoeWFeEIQkVcwm4G33gKW\nLwfmzQNefFHcjVPf9PuEIJPJBACI/+9NnQkJCSgrK0NycnK3699//32kpaX1pVYi8gCHD4tf6vnl\nL4GjR707I9yVrDZyo9GIkJCQjsdqtRqlpaXdNvLr16/j0KFDePvtt3t8v+zs7I6fNRoNNBpN7ysm\nIrdTXw8sXgwYjeKJPYyX7Tu9Xg+9Xt+r1zjsw869e/di/PjxuP3223tc07mRE5H8tbQAq1eLBzws\nXAhs3gwMHix1VfL2001uTk6OzddY/bAzOjoaNTU1HY+rqqoQGxvb7doPPviAYxUiL7Jvn3jAw8mT\nYkb4smVs4lKx+8POe++9F0lJSd1+2GkymXDffffh/PnzGNzDv0l+2EnkGTpnhL/5JuNlna3fH3YC\nQG5uLrRaLcxmMzIyMqBQKKDT6QAAWq0WALBr1y4kJib22MSJSP6am4EVK8Qv9ixdCuzcyXhZd2Fz\nR+6wC3FHTiRLggDs2CF+mDl+PLByJXDPPVJX5T0csiMnIu/VOSN8yxbGy7orJh0QkQVmhMsLGzkR\ndWBGuDzxXw8RAWBGuJxxR07k5ZgRLn9s5EReihnhnoOjFSIvxIxwz8JGTuRFLl0CnnsOKCoCVq0C\nUlMZbuUJ+JcoIi9gNovBVmFhQGCgOEZJS2MT9xTckRN5uKIiICODGeGejI2cyEN1zghfuxaYOpU7\ncE/F0QqRh2lpAV59FYiKAh54QPyaPQ968GzckRN5kH37gMxM8S6UEyeAESOkrohcgY2cyAN0zghf\nv54Z4d6GoxUiGWtuFk+pj40VQ60qK9nEvZHNRm4wGKBSqRAcHIy8vLxu1xiNRkRHR0OlUvFAZSIX\nEATgo4/Eb2XW1gKnT4uHPfCgB+9k91FvSqUSiYmJFke9CYKA8PBwrF27FpMnT0ZDQ4PFUXAAD5Yg\ncpTOGeF5eYyX9XT29E6rO3KTyQQAiI+Ph1KpREJCAsrKyrqsOXHiBMLDwzF58mQA6LaJE1H/MSOc\nemK1kRuNRoSEhHQ8VqvVKC0t7bLm0KFD8PHxwYQJE5CSkoJDhw45p1IiL8WMcLKl3/8ptLS04NSp\nUygsLMT169fx6KOP4rPPPuv2IObs7OyOnzUaDefpRDYwI9z76PV66PX6Xr3G6ozcZDJBo9GgoqIC\nALBw4UIkJSUhOTm5Y82+ffug1+uxatUqAMCsWbMwb948JP7ko3POyIns19go3o3yySfil3vS0xkv\n6636PSMPCAgAIN65UldXh4KCAsTExHRZExsbiyNHjuD69etoampCRUUFHn744X6WTuSdmBFOfWFz\ntJKbmwutVguz2YyMjAwoFArodDoAgFarxZ133on09HSMHTsWw4cPx8svvwx/f3+nF07kaZgRTn1l\n8/ZDh12IoxWibjEjnKzp92iFiJzHbAbWrAFCQ5kRTv3DG5iIJFBUJH6pJygIOHaMGeHUP2zkRC5U\nXw9kZYnJhMwIJ0fhaIXIBTpnhIeGMiOcHIs7ciIny88XI2aZEU7OwkZO5CTMCCdX4WiFyMGam4EX\nXmBGOLkOGzmRg7RnhKtUzAgn1+JohcgBqqr+lxG+ZQswcaLUFZE34Y6cqB9+mhFeXs4mTq7HRk7U\nB21twObNQEiImBHeviNnRjhJgf/ZEfVS54zw3buZEU7S446cyE6NjcCCBcCUKWK0bGkpmzi5BzZy\nIhtaW4ENG8S7UZgRTu6IoxUiK44d65oRPmaM1BURWWIjJ+rGxYtiRvjhw8DKlYyXJfdm8y+HBoMB\nKpUKwcHByMvLs3her9cjICAAkZGRiIyMxCuvvOKUQolcoT0jPCwM+PnPxTHK7Nls4uTebO7IMzMz\nodPpoFQqkZiYiLS0NCgUii5rJk6ciD179jitSCJXKCwEMjLEjPCjR8VbC4nkwOqO3GQyAQDi4+Oh\nVCqRkJCAsrIyi3U8wo3krL4eeOIJ4P/+D1i+HDh4kE2c5MVqIzcajQjp9F+0Wq1GaWlplzU+Pj4o\nLi5GREQEFi1ahHPnzjmnUiIHa2kBXnkFiIwEHniAGeEkX/3+sDMqKgrffvstfH19sXnzZmRmZiI/\nP7/btdnZ2R0/azQaaDSa/l6eqE/y84HMzP9lhI8cKXVFRCK9Xg+9Xt+r1/gIVuYiJpMJGo0GFRUV\nAICFCxciKSkJycnJ3a4XBAGBgYGor6+Hn59f1wvZcRI0kbN99ZWYEX72LPDmm4yXJfdnT++0OloJ\nCAgAIN65UldXh4KCAsTExHRZ891333VcZO/evQgPD7do4kRS65wRPnEiM8LJs9gcreTm5kKr1cJs\nNiMjIwMKhQI6nQ4AoNVqsWPHDmzYsAEDBw5EeHg4Vq9e7fSiiezVnhG+eDEwfjxw5gxwzz1SV0Xk\nWFZHKw69EEcr5GKdM8Lz8hgvS/LU79EKkRyZTMCzzzIjnLwHGzl5jM4Z4VevMiOcvAf/EyePcPKk\nGG7V2sqMcPI+3JGTrLVnhCcnA089xYxw8k5s5CRLzAgn+h+OVkh2iouBP/2JGeFE7djISTYuXRIz\nwouKmBFO1Bn/Ikpurz0jPDQUCAxkRjjRT3FHTm6tqEi8hZAZ4UQ9YyMnt1RfD2RlicmEa9cCU6dy\nB07UE45WyK20tACvvsqMcKLe4I6c3EZ+vhgxGxbGjHCi3mAjJ8mdPSs28K++AtavZ7wsUW9xtEKS\n6ZwRHh/PjHCivmIjJ5drzwhXqYDaWuD0aWDpUuCWW6SujEieOFohl+qcEb5lC+NliRzB5o7cYDBA\npVIhODgYeXl5Pa4zGo0YOHAgdu7c6dACyTOYTMCiRcwIJ3IGm408MzMTOp0OhYWFWL9+PRoaGizW\ntLa24rnnnkNSUhJPAaIuOmeEX7nCjHAiZ7D6x8lkMgEA4uPjAQAJCQkoKytDcnJyl3V5eXmYOXMm\njEajk8okOSovFzPCb95kRjiRM1lt5EajESGdvhOtVqtRWlrapZFfuHABu3fvxuHDh2E0GuFj5Zsb\n2dnZHT9rNBpoNJq+V05uq7FRvBtl1y7xyz3p6YyXJbKXXq+HXq/v1Wv6/RfcZ555Bq+99lrHAaHW\nRiudGzl5ntZW4N13gZdeAmbNEsOthg2TuioiefnpJjcnJ8fma6w28ujoaCxZsqTjcVVVFZKSkrqs\nOXnyJFJTUwEADQ0NOHDgAHx9ffHYY4/1pnaSueJicYzi78+McCJXs9rIAwICAIh3rtx7770oKCjA\nsmXLuqz5+uuvO35OT09HSkoKm7gXYUY4kfRsjlZyc3Oh1WphNpuRkZEBhUIBnU4HANBqtU4vkNyT\n2Qzk5QHLl4tHrH3+uXhiDxG5no/govsF22foJH+dM8LXrWNGOJEz2dM7eTcv2Y0Z4UTuiTeFkU3M\nCCdyb9yRk1XMCCdyf2zk1C1mhBPJB0cr1AUzwonkh42cADAjnEjOOFohZoQTyRx35F6MGeFEnoGN\n3AsxI5zIs/CPrpdhRjiR5+GO3Es0NgILFgBTpojZKKWlbOJEnoKN3MO1tgIbN4p3o9xyixhuNX8+\nD3og8iQcrXiw9ozwoUOBwkIgPFzqiojIGdjIPVDnjPBVq4DUVOaiEHky/gXbg5jNwJo1QGgoEBgo\njlF40AOR5+OO3EN0zgg/dgwYPVrqiojIVWzuyA0GA1QqFYKDg5GXl2fx/O7duzFmzBhEREQgOTkZ\nRqPRKYVS9+rrgSeeAJ56Sjyt5+BBNnEib2PzhKDIyEisW7cOSqUSiYmJOHr0KBQKRcfzzc3NGDJk\nCADgyJEj+Otf/wqDwWB5IZ4Q5FAtLcDq1eIBDwsXirkogwdLXRUROZo9vdPqjtxkMgEA4uPjoVQq\nkZCQgLKysi5r2pt4+/pBgwb1tV6yU36+OAc/eVLMCF+2jE2cyJtZnZEbjUaEdDqQUa1Wo7S0FMnJ\nyV3WffLJJ3j22Wdx7do1nDx5ssf3y87O7vhZo9FAo9H0rWovxYxwIs+n1+uh1+t79RqHfNg5ffp0\nTJ8+Hf/85z8xbdo0VFRUdLuucyMn+zU3i/NvnU4coezcyXhZIk/1001uTk6OzddYHa1ER0ejpqam\n43FVVRViY2N7XD9r1iz8+9//xo0bN+wol2zpnBFeV8eMcCLqntVGHhAQAEC8c6Wurg4FBQWIiYnp\nsubcuXMdg/j9+/fjwQcfxGAObPutqgr49a+BV14Btm4Ftm0D7rlH6qqIyB3ZHK3k5uZCq9XCbDYj\nIyMDCoUCOp0OAKDVavHxxx/jvffeg6+vLyIjI7Fy5UqnF+3JTCYgJ0c84GHZMjHoivGyRGSNzdsP\nHXYh3n5oVVub2Lyff15MKFy+HLjrLqmrIiKp2dM7uddzA50zwnftYrwsEfUOs1YkxIxwInIENnIJ\nMCOciByJoxUXY0Y4ETkaG7mLMCOciJyFf5l3MmaEE5GzcUfuRMwIJyJXYCN3gvp6ICtLTCZcuxaY\nOpU7cCJyHo5WHKilBXj1VSAqShylVFcD06axiRORc3FH7iD79gGZmeJdKCdOACNGSF0REXkLNvJ+\nYkY4EUmNo5U+am4GXnwRiI0F4uOByko2cSKSBht5L7VnhKvVQG0tM8KJSHocrfRCdbV4O2FDg5hU\nGB8vdUVERNyR28VkAhYtAjQaYPp08dBjNnEichds5Fa0tQHvvSeGW125Anz2mZiTwoMeiMid2NXI\nDQYDVCoVgoODkZeXZ/H8tm3bMGbMGIwZMwazZ8/Gl19+6fBCXa28HBg/HnjrLTEj/O9/50EPROSe\n7DohKDIyEuvWrYNSqURiYiKOHj0KhULR8XxJSQnUajUCAgKwefNmFBYWYsuWLV0vJJMTghobxbtR\nPvlE/HJPejrjZYlIOvb0TpstymQyAQDi4+OhVCqRkJCAsrKyLmvi4uI6DmpOTk7GkSNH+lqzZNoz\nwtVqwNeXGeFEJB82p71GoxEhISEdj9VqNUpLS5GcnNzt+nfeeQcpKSndPpednd3xs0ajgUaj6V21\nTtKeEe7vDxQUMCOciKSj1+uh1+t79RqHfmxXWFiIrVu3ori4uNvnOzdyd8CMcCJyNz/d5Obk5Nh8\njc3BQXR0NGpqajoeV1VVITY21mLdmTNnsGDBAuzZswe33367nSVLw2wWUwnDwpgRTkTyZ7ORt8++\nDQYD6urqUFBQgJiYmC5r6uvr8fjjj2Pbtm0YNWqUcyp1kKIiICICOHgQOHoUeP118dg1IiK5smu0\nkpubC61WC7PZjIyMDCgUCuh0OgCAVqvFyy+/jKamJixYsAAA4Ovri+PHjzuv6j6orwcWLwaMRmaE\nE5Fnsev2Q4dcSKLbD1tagNWrxea9cKGYizJ4sMvLICLqE3t6p0d/RzE/X4yYZUY4EXkyj2zkzAgn\nIm/iUV93aW4GXniBGeFE5F08opG3Z4SrVEBdHTPCici7yH60UlUlfojZ2Ahs3cp4WSLyPrLdkXfO\nCJ8xgxnhROS9ZNfI29qAzZv/lxFeVcWMcCLybrJqf+XlYtO+eVPMCB83TuqKiIikJ4sdeWMjsGAB\nMGWKGC1bWsomTkTUzq0beXtGuEol3oHCjHAiIktuO1ppzwgfOhQoLGRGOBFRT9yukTMjnIiod9xm\nSGE2A2vWAKGhzAgnIuoNt9iRFxWJX+oJCgKOHQNGj5a6IiIi+ZC0kdfXA1lZYjIhM8KJiPrGrtGK\nwWCASqVCcHAw8vLyLJ6vqalBXFwcBg0ahNWrV9t8v5YW4NVXgagocZRSXQ1Mm+YeTby3h55KhXU6\njhxqBFino8mlTnvY1cgzMzOh0+lQWFiI9evXo6Ghocvzd955J/Ly8rB48WKb75WfLzbvkyfFnfiy\nZe510INc/uWyTseRQ40A63Q0udRpD5uN3GQyAQDi4+OhVCqRkJCAsrKyLmuGDx+OsWPHwtfX1+p7\n/fa34ihl/Xpg504e9EBE5Ag2G7nRaERISEjHY7VajdLS0j5djBnhRESOZ/PMzsLCQvzjH//A9u3b\nAQAbN27EhQsX8Le//c1ibU5ODvz9/ZGVlWV5IXcYgBMRyVC/z+yMjo7GkiVLOh5XVVUhKSnJ4YUQ\nEVHf2BytBAQEABDvXKmrq0NBQQFiYmK6XctmTUTkejZHKwBw5MgRLFiwAGazGRkZGcjIyIBOpwMA\naLVaXLp0CdHR0bhy5QoGDBiAoUOHorq6Gv7+/k7/DRAReT3BydLT04W77rpLCA0Ndfal+qy+vl7Q\naDSCWq0WJk6cKGzbtk3qkrp148YNYdy4ccKYMWOEmJgYYc2aNVKXZNXNmzeFiIgI4be//a3UpfRI\nqVQKYWFhQkREhBAdHS11OT26du2aMGfOHCE4OFhQqVRCSUmJ1CVZqKmpESIiIjr+ue2224R169ZJ\nXZaFd955R4iLixOioqKEzMxMqcvp0bZt24T4+HhBrVYL7777rtW1Tm/kBoNBKC8vd+tGfvHiRaGi\nokIQBEH44YcfhJEjRwpXrlyRuKruNTc3C4IgCC0tLcIDDzwgfPXVVxJX1LPVq1cLs2fPFlJSUqQu\npUcjRowQGhsbpS7DpqysLOHFF18Ubty4IZjNZuHy5ctSl2RVa2urEBgYKNTX10tdSheNjY3CiBEj\nhGvXrgmtra3Cb37zG+HgwYNSl2Xh8uXLwq9+9SuhqalJuHr1qhAdHW3137nTQ7MmTJiAYcOGOfsy\n/RIYGIiIiAgAgEKhwAMPPIATJ05IXFX3br31VgDAtWvXcPPmTfj5+UlcUffOnz+P/fv346mnnnL7\nz07cvT5AvHvsL3/5CwYNGoSBAwd2fHblrgoLC3H//fcjKChI6lK6GDx4MARBgMlkwo0bN3D9+nW3\n7E/FxcWIiorCsGHD4O/vj0ceeQQlJSU9rneb9EN3cfbsWVRVVWGcmx5B1NbWhjFjxuDuu+/Gn//8\nZ7f7g9Lu2WefxapVqzDAzU8B8fHxwaRJkzBt2jTs2bNH6nK6df78ebS0tODpp59GTEwMXn/9dbS0\ntEhdllUffPABZs+eLXUZFgYPHowNGzZgxIgRCAwMxMMPP+yWf9bj4+Nx/Phx1NbW4uLFi9i/fz+K\ni4t7XO/ef8pc7OrVq5g1axbWrl2LIUOGSF1OtwYMGIDTp0/j7NmzePvtt1FRUSF1SRby8/Nx1113\nITIy0u13u8eOHcPp06exYsUKLFq0CJcuXZK6JAstLS348ssv8fjjj0Ov16Oqqgoffvih1GX16Mcf\nf8TevXvxxBNPSF2KhR9++AFPP/00qqurUVdXh5KSEuzbt0/qsiwMGTIEubm5+NOf/oSZM2ciLCwM\ngwYN6nE9G/l/mc1mPP744/jDH/6AqVOnSl2OTSNGjMCUKVMs4hLcQXFxMfbs2YORI0ciLS0Nhw8f\nxpw5c6Quq1s///nPAQAqlQqPPfYY9u7dK3FFlkaNGoXRo0cjJSUFgwcPRlpaGg4cOCB1WT06cOAA\nHnzwQQzYdASKAAADTElEQVQfPlzqUiwcP34csbGxGDVqFO6880488cQTMBgMUpfVrZSUFOzfvx/H\njh1DW1ub1e/vsJFDnJHOnz8foaGheOaZZ6Qup0cNDQ24fPkyAKCxsRH/+te/3PL/dJYvX45vv/0W\ntbW1+OCDDzBp0iS89957Updl4fr167h69SoAcad26NChPn3ZzRWCg4NRVlaGtrY27Nu3D5MnT5a6\npB5t374daWlpUpfRrQkTJuDEiRNoamrCf/7zHxw4cAAJCQlSl9Wt77//HoD4eUNlZSWioqJ6XOv0\nPPK0tDQcOXIEjY2NCAoKwssvv4z09HRnX7ZXjh07hq1btyI8PByRkZEAgBUrVrjdH+qLFy9i7ty5\naG1tRWBgIBYvXtyxo3Rn7hrP8N1332H69OkAxATPrKwst/3M4Y033sCcOXPQ0tKCyZMnIzU1VeqS\nutXc3IzCwkK8++67UpfSrdtuuw0vvvgipk+fjuvXryMpKQmPPPKI1GV1a+bMmfj+++8xdOhQbNq0\nyepau74QRERE7oujFSIimWMjJyKSOTZyIqJ+mjdvHu6++26EhYXZXGswGBAVFQVfX198/PHHHb9+\n6tQpPPTQQwgLC0NiYiIOHjxo9/XZyImI+ik9Pd3uxqtUKrF582aLL0wNGTIEW7ZsQWVlJdavX4+M\njAy7r89GTkTUT91FkVy4cAFLlixBXFwc5s6di9raWgBiIw8LC7P41nNwcDDuv/9+AOJ3B/z8/Dpe\nYwsbORGRE7z00ktITU1FSUkJZs2ahZUrV9r92qNHj+JnP/sZRo4cadd6p99HTkTkbcxmM/bv34/y\n8vJev/bChQuYP38+tm7davdr2MiJiBysra0NAwYMQGlpqdWE0p9+We7KlStISUnBihUrEB0dbff1\nOFohInIwPz8/TJkyBRs2bEBraysEQcCZM2e6rBHE8yA6Hv/444+YPn06nnzyScyYMaNX12MjJyLq\np7S0NDz00EP44osvEBQUhE2bNiEnJweXLl3C2LFjERoa2hGTbDQaERQUhB07dkCr1Xbcsvjhhx/i\n008/xaZNmxAZGYnIyEiL5t8TfkWfiEjmuCMnIpI5NnIiIpljIycikjk2ciIimWMjJyKSOTZyIiKZ\n+3+suggPFbS8IAAAAABJRU5ErkJggg==\n",
"text": [
"<matplotlib.figure.Figure at 0x3d931d0>"
]
}
],
"prompt_number": 15
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"plt.plot(nasty['BIG'], nasty['LITTLE'])"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 16,
"text": [
"[<matplotlib.lines.Line2D at 0x4117550>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAWgAAAEPCAYAAABvFF43AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtQ1XX+x/HXMU3RTF3cFV0ImLgIiFwUBAs5qAlh0G1a\nwSZvqY3TRc1tnXacpClxEiO1DddWM5tddEycQoN0DY/uioFQipJRFqzpYgooIXIReP/+IM/vsBzO\nBc7lezivxwwzeDjfL2/Z7dOzD99zvioRERARkeIMsPcARESkHxdoIiKF4gJNRKRQXKCJiBSKCzQR\nkUJxgSYiUqg+L9CLFi3CmDFjEBwcbIl5sG7dOkRGRiIyMhLp6ekmH9fQ0IBVq1YhNDQU0dHR+OGH\nH/Q+r6ioCI8++igCAgLwwgsvoKOjw+jx2dnZiI2NRVBQELZv3659vKKiAk8//TQCAwORkpKCpqYm\nAICIYO3atZg8eTJCQ0Nx6tQp7TGHDh1CfHw8AgIC8MYbb2gfP378OMLDwzFo0CDk5OSY/Pc25OOP\nP0ZQUBDuuusufPXVVxY5JxHZkPTR8ePH5auvvpIJEyb09VTyzTffSEhIiNy+fVtaW1tl4sSJ8u23\n33Z7nqenZ7fHtm3bJi+++KKIiBQWFsoTTzyh93tER0dLcXGxtLe3y7JlyyQ3N9fg8Tdu3BA/Pz+p\nq6uThoYGiYiIkPr6ehERSU1Nlb1794qIyPr162XLli0iInLo0CF57LHHpLW1VSorKyU6OlpERNrb\n28XX11cuXrwoLS0tkpycLGVlZSIiUlVVJWVlZTJv3jzZt29fr35+/+v8+fNSUVEharVaSktLLXJO\nIrKdPhd0TEwMRo0a1eWxy5cv45VXXkF0dDTmz5+PyspKk851zz33oLW1FY2NjWhoaEBrayuGDx/e\n7XkqlarbYwUFBZg9ezYAIDo6GhcuXOj2nMbGRvznP/9BREQEBgwYgKSkJJw4ccLg8YWFhQgPD8eo\nUaNwzz33IC4uDoWFhQAAjUaDpKQkAEBycnKXcyUkJGDQoEHw8vKCSqVCY2MjKioq4OrqCg8PD9x9\n991ISEjQHuPp6Yng4GAMGND9f5KPP/4YjzzyCGJiYvD++++b9LMEgPHjx8PPz8/k5xORslhlD/q1\n115DSkoKTp48iTlz5mDDhg0mHefh4YEVK1ZgzJgxGDduHF5++WWMGzfOpGPj4+Oxe/duNDU1ITc3\nF2fPnu32L4Zhw4bBx8cHeXl5aGxsRHZ2tnax7en4adOmobi4GJWVlaiurkZeXh5OnjwJAHjooYfw\n4YcfoqWlBbt27epyrv379+PGjRsoLS3FqVOnUFxcjICAANTW1uL06dO4fv069u3bpz2mJ1VVVdi3\nbx8++eQTfPHFF8jOzkZ1dbVJPxMicmwDLX3C27dvIy8vT++e5/79+7F27dpuj7u7uyM/Px9nzpxB\nVlYWfvzxR4gIHn74YUyZMgUTJ07E888/r13M/vvf/yIsLAwA8Ic//AGvvvoq5syZg0uXLiE2Nhb+\n/v7w9fXF4MGDu32vd999F5mZmVizZg0eeOAB/PzzzwDQ4/HDhg3Dpk2b8Pzzz6O+vh7BwcHa877+\n+uvYuHEjoqKiMGPGDLi4uAAA1Go1Tp8+jdmzZ8PV1RURERHaY3bs2IE333wTFy9exMSJE/UWs66c\nnBwUFxcjIiICQOd/BRQUFODpp5/GzJkztfPrSk9P15Y9ETkwS+yTVFZWavegm5ubZdy4cdLc3Gz2\ned577z1ZvXq19s9/+tOfJCsrq9vzvLy8DJ6noaFBQkJCjH6/rKwsefvtt806fs6cOXr3c/Py8rR7\n2P/L399f2tvbuz2+evVqycnJ6fLYggULujy2YcMGSUtLM/j3MIZ70ESOyeJbHIMHD0ZiYiK2bt2K\n9vZ2iAjKyspMOnbGjBkoKCjQ7kF/8cUXmDVrlknH1tfXo7W1Fbdu3cL69evx0EMP6X3etWvXAADf\nffcdPvroI8THxxs9/urVqwCAI0eO4OzZswgPD+9yrsuXLyMrK0t7rqamJjQ2NqKtrQ1ZWVld9pbv\nnKukpAS5ubmYMWNGl/lEBKLz/lUpKSnIycnBxYsXtd/rzvc1h/A9sYgcj7EV/P3335fo6GgJDw+X\n5cuXd/t6SkqKjB07VgYNGiTu7u7ywQcfyOXLl2X16tUSGhoqgYGB8sYbb5j8b4yMjAyJioqSqKgo\nyczM1Pscb2/vbo+dPHlS/Pz8xMfHR5555hlpbGzUfi0xMVGqq6tFROSVV16R8ePHi7+/v+zYscOk\n42NiYsTf318mT54sRUVF2sc3b94sfn5+4uvrK+vWrdM+XlVVJf7+/uLj4yNJSUly9epV7ddSU1Nl\n/PjxEhQUJAcPHtQ+XlxcLO7u7jJs2DBxdXXtclVMTk6OPPzwwxIcHCxTp06VH3/80aSf5f79+8Xd\n3V2GDBkiY8aMkYSEBJOOIyJlUIn0nFZ1dXWYNGkSzp07BxcXFzzyyCNYvny5thSJiMh6DP6S0MXF\nBSKC+vp6AMCtW7e6XVJHRETWYXAP2sXFBVu3boWXlxfc3NzwwAMPIDIy0lazERE5N0P7H1evXhVP\nT0/5/vvvpaamRuLi4rrsm/66PcIPfvCDH/zoxYcxBgu6uLgYUVFR8PHxgaurK5566ikcP3682/Pk\n1ysPlPyxdu1au8/AOTkn53TOGW/fFqSnC0aPFmzbJujoEENLr5bBBTomJgYlJSWoq6tDS0sL8vPz\nTb7sjYiIgPJyIDoaKCgASkqApUsBPe9WoZfBBfree+/FmjVr8Pjjj+PBBx9ESEgI4uLiLDEzEVG/\n1tYGrF8PqNXAkiXA4cOAp6d55zD6Uu8FCxZgwYIFvZtQQdRqtb1HMAnntCzOaVmOMKcSZiwvBxYs\nAEaO7KxmcxfmOwxeB23SCVQq9PEURET9QlsbkJEBZGYC69Z1lnNP2xmmrJ0Wf7MkIiJnZKlq1sVb\nXhER9YEl9pp7woImIuola1SzLhY0EZGZrFnNuljQRERmsHY162JBExGZwFbVrIsFTURkhC2rWRcL\nmoioB/aoZl0saCIiPexVzbpY0EREOuxdzbpY0EREv1JCNetiQROR01NSNetiQRORUzt3Dli4UDnV\nrIsFTUROqa0NSE9XXjXrYkETkdM5d65zr3nUKKC0VHkL8x0saCJyGrrVvHSpMqtZFwuaiJyCo1Sz\nLhY0EfVrjlbNuowu0BUVFQgLC9N+jBgxAlu2bLHFbEREfXLuHBAVBRw92lnN5txRWwnMuidhR0cH\nfv/736O4uBgeHh6dJ+A9CYlIYdragA0bOu8NmJ5u+N6A9mLxexIeOXIE999/v3ZxJiJSGkfca+6J\nWXvQe/bswdy5c601CxFRrznyXnNPTC7o1tZWHDhwAG+99Va3r6WlpWk/V6vVUKvVlpiNiMgkjlDN\nGo0GGo3GrGNM3oP+9NNPsXXrVnz++eddT8A9aCKyE0fYa+6JRfegd+/ejdTU1D4PRURkCY5QzX1l\nUkE3NjbC09MTlZWVGD58eNcTsKCJyIYcuZp1Wayghw0bhpqaGosMRUTUW85Qzbr4SkIiUrz+eIWG\nKfheHESkaM5WzbpY0ESkSM5azbpY0ESkOLr3BnS2atbFgiYixVDqvQHthQVNRIqgtDtqKwELmojs\nitXcMxY0EdkNq9kwFjQR2Ryr2TQsaCKyKVaz6VjQRGQTrGbzsaCJyOpYzb3DgiYiq2E19w0Lmois\ngtXcdyxoIrIoVrPlsKCJyGJYzZbFgiaiPmM1WwcLmoj6hNVsPSxoIuoVVrP1GV2gGxsbMX/+fPj5\n+SEwMBBffvmlLeYiIgUrLweio4GCgs5qXrrUMW/cqnRGF+i1a9fivvvuQ1lZGcrKyhAQEGCLuYhI\ngVjNtmV0D/rIkSM4efIkhgwZAgAYMWKE1YciIuXhXrPtGVygL126hObmZixbtgznz5/HE088geXL\nl2sX6zvS0tK0n6vVaqjVamvMSkR20NYGZGQAmZnAunWd5cztDPNpNBpoNBqzjlGJiPT0xQsXLsDP\nzw+ffvopZs6cieeeew4zZ87EvHnz/v8EKhUMnIKIHJhuNW/fzmq2JFPWToN70D4+PvD390dSUhJc\nXFyQmpqK/Px8iw5JRMrDvWZlMLoH7evri6KiIkREROCzzz7DzJkzbTEXEdkJ95qVw+hVHBs3bsTy\n5csRHh6OIUOGICUlxRZzEZGNsZqVx+AetEkn4B40kcPjXrPt9XkPmoj6N1azsvG9OIicFPealY8F\nTeRkWM2OgwVN5ERYzY6FBU3kBFjNjokFTdTPsZodFwuaqJ9iNTs+FjRRP8Rq7h9Y0ET9iG41L13K\nanZ0LGiifuJONY8aBZSWAvfdZ++JqK9Y0EQO7n+r+dAhLs79BQuayIGxmvs3FjSRA2I1OwcWNJGD\nYTU7DxY0kYNgNTsfFjSRA2A1OycWNJGCsZqdGwuaSKFYzcSCJlIYVjPdYVJBe3l54d5778Vdd92F\nQYMGobi42NpzETklVjPpMmmBVqlU0Gg0+M1vfmPteYicUlsbkJEBZGYC6enA4sWASmXvqcjeTN6D\n5p27iayD1Uw9Mbmgp0+fDm9vbyxatAjJycldvp6Wlqb9XK1WQ61WW3JGon5Jt5rXret8z2ZWc/+l\n0Wig0WjMOkYlJqRxdXU1xo4di/PnzyMpKQn//ve/4ebm1nkClYp1TWQm3Wrevp3V7IxMWTtNuopj\n7NixAICAgAAkJyfjwIEDfZ+OyAnxCg0yh9Etjlu3bqG9vR3Dhw/HtWvXcOjQIaxcudIWsxH1K9xr\nJnMZXaB//vlnPP744wAAV1dXrFq1Ch4eHlYfjKi/4BUa1Fsm7UEbPAH3oIl6xL1m6onF9qCJyDzc\nayZL4HtxEFkY95rJUljQRBbCaiZLY0ETWQCrmayBBU3UB7rVvGQJq5ksiwVN1Et3qnnkSKCkBPD0\ntPdE1N+woInM9L/VfPgwF2eyDhY0kRlYzWRLLGgiE7CayR5Y0ERGsJrJXljQRD1gNZO9saCJ9GA1\nkxKwoIl0sJpJSVjQRL8qLwcWLgRGjGA1kzKwoMnp6Vbz4sWsZlIOFjQ5NVYzKRkLmpwSq5kcAQua\nnA6rmRwFC5qcBquZHI1JBd3e3o7JkyfD3d0dBw4csPZMRBbHaiZHZFJBb968GYGBgVDxVsTkYFjN\n5MiMLtCXLl1CXl4eFi9ezLt3k0MpLwemTgUKCjqreelSgI1BjsToFsfKlSuRkZGBX375pcfnpKWl\naT9Xq9VQq9WWmI2oV9ragIwMIDMTePNNLsykDBqNBhqNxqxjVGIgiw8ePIj8/Hy899570Gg0ePvt\nt7vtQatUKpY1KYbuXvP27dzOIOUyZe00uMVRWFiI3NxceHt7IzU1FQUFBZg3b55FhySyBO41U39k\nsKB1HTt2DBs3bmRBk+KwmskR9bmg9Z2QSClYzdTfmVzQPZ6ABU12wGomR2fxgiayN1YzORO+Fwc5\nDL4akJwNC5oUj9VMzooFTYrGaiZnxoImRWI1E7GgSYFYzUSdWNCkGKxmoq5Y0KQI5eXAggXAyJGs\nZqI7WNBkV7rVvGQJq5lIFwua7IbVTGQYC5psjtVMZBoWNNkUq5nIdCxosglWM5H5WNBkdaxmot5h\nQZPVsJqJ+oYFTVbBaibqOxY0WRSrmchyWNBkMaxmIstiQVOfsZqJrMNoQTc3NyM2NhYtLS0YMmQI\n5syZg5UrV9piNnIArGYi6zHpprG3bt3C0KFD0dLSgkmTJuGTTz6Bj49P5wl401in1NYGZGQAmZnA\nunWd5cybvhOZzpS106Q96KFDhwIAbt68iba2NgwePLjv05HDYjUT2YZJC3RHRwfCwsJQXl6OTZs2\nwcPDo8vX09LStJ+r1Wqo1WpLzkgKwWom6j2NRgONRmPWMSZtcdxRVVWFxMRE/OMf/0BYWFjnCbjF\n4RR0q3n7dlYzUV+ZsnaadRWHl5cXEhMTUVRU1KfByHHwCg0i+zG6xVFTU4OBAwdi5MiRqK2txeHD\nh7Fq1SpbzEZ2xr1mIvsyukBXV1dj/vz5aG9vh5ubG/74xz9i7NixtpiN7IR7zUTKYNYetN4TcA+6\nX+FeM5FtWHwPmvov7jUTKQ/fi4O410ykUCxoJ8ZqJlI2FrSTYjUTKR8L2smwmokcBwvaibCaiRwL\nC9oJsJqJHBMLup87dw5YuJDVTOSIWND9VFsbkJ7OaiZyZCzofujcuc695lGjgNJSLsxEjooF3Y/o\nVvPSpaxmIkfHgu4nWM1E/Q8L2sGxmon6Lxa0A2M1E/VvLGgHxGomcg4saAfDaiZyHixoB8FqJnI+\nLGgHwGomck4saAVjNRM5N6ML9E8//YS4uDgEBQVBrVYjOzvbFnM5vXPngKgo4OjRzmpeupQ3biVy\nNkZvGnvlyhVcuXIFoaGhqKmpQWRkJM6cOYPhw4d3noA3jbWotjZgw4bOO2qnp/OO2kT9lSlrp9E9\naDc3N7i5uQEARo8ejaCgIJSUlCAuLs4yU5IW95qJSJdZvyS8cOECysvLERkZ2eXxtLQ07edqtRpq\ntdoSszkNVjNR/6fRaKDRaMw6xugWxx0NDQ1Qq9V47bXX8Oijj/7/CbjF0Se61bx9O6uZyFmYsnaa\ndBXH7du38eSTT+KZZ57psjhT7/EKDSIyxugWh4jg2WefxYQJE7BixQpbzNTvca+ZiExhtKBPnDiB\nv//97ygoKEBYWBjCwsLw+eef22K2fofVTETmMHkPuscTcA/aJNxrJiJdFtuDpt5jNRNRb/G9OKyI\ne81E1BcsaCtgNRORJbCgLay8vLOaR45kNRNR37CgLaStDVi/vrOalyxhNRNR37GgLUC3mktKuDAT\nkWWwoPuA1UxE1sSC7iVWMxFZGwvaTKxmIrIVFrQZWM1EZEssaBOwmonIHljQRrCaicheWNA9YDUT\nkb2xoPVgNRORErCgdbCaiUhJWNC/YjUTkdI4fUGzmolIqZy6oFnNRKRkTlnQrGYicgRGF+hFixZh\nzJgxCA4OtsU8VldeDkRHAwUFndW8dCmgUtl7KiKi7owu0AsXLuwXd/FmNRORozG6Bx0TE4Oqqiob\njGI93GsmIkdkkV8SpqWlaT9Xq9VQq9WWOG2ftbUBGRlAZiawbl1nOXM7g4jsQaPRQKPRmHWMSkTE\n2JOqqqqQlJSEs2fPdj+BSgUTTmFz5eXAwoXAiBHA9u2sZiJSFlPWzn53FYfuXvPixdxrJiLH1a+u\ng9atZu41E5GjM1rQqampmDp1Kr777jt4eHhg586dtpjLLKxmIuqPTNqDNngCO+9Bc6+ZiBxRv96D\nZjUTUX/nkHvQ3GsmImfgUAXNaiYiZ+IwBc1qJiJno/iCZjUTkbNSdEGzmonImSmyoFnNREQKLGhW\nMxFRJ8UUNKuZiKgrRRQ0q5mIqDu7FjSrmYioZ3YraFYzEZFhNi9oVjMRkWlsWtCsZiIi09mkoFnN\nRETms3pBs5qJiHrHagXNaiYi6hurFDSrmYio74wW9PHjxxEQEABfX1+8++67Bp+r5GrWaDT2HsEk\nnNOyOKdlOcKcjjCjqYwu0MuXL8e2bdtw5MgRvPfee6ipqdH7vPJyYOpUoKCgs5qXLgVUKovP22uO\n8j8a57QszmlZjjCnI8xoKoMLdH19PQBg2rRp8PT0xKxZs1BUVNTteUqtZiIiR2ZwgT516hTGjx+v\n/XNgYCC+/PLLbs9TajUTETkylRi47/eRI0ewY8cO7N69GwDw17/+FZcvX8Ybb7zx/yfgikxE1CsG\nll8ARq7iiIiIwCuvvKL9c3l5ORISEsz6BkRE1DsGtzhGjBgBoPNKjqqqKvzzn//ElClTbDIYEZGz\nM3od9KZNm/Dcc8/h9u3beOmllzB69GhbzEVE5PSMXmYXGxuL8+fP48KFC3jppZe6fM2ca6TtZdGi\nRRgzZgyCg4PtPYpBP/30E+Li4hAUFAS1Wo3s7Gx7j6RXc3MzpkyZgtDQUERFReGdd96x90g9am9v\nR1hYGJKSkuw9So+8vLwwceJEhIWFITIy0t7j9KixsRHz58+Hn59fjxcL2FtFRQXCwsK0HyNGjMCW\nLVvsPZZef/vb3zB16lRMmjQJK1as6PmJ0gehoaFy7NgxqaqqEn9/f7l27VpfTmcVx48fl6+++kom\nTJhg71EMqq6ulq+//lpERK5duybe3t7yyy+/2Hkq/RobG0VEpLm5WYKCguT777+380T6vf322zJ3\n7lxJSkqy9yg98vLyktraWnuPYdSqVatkzZo10tTUJLdv35YbN27YeySD2tvbxc3NTS5evGjvUbqp\nra0VLy8vuXnzprS3t8vDDz8sn3/+ud7n9vq9OEy9RtreYmJiMGrUKHuPYZSbmxtCQ0MBAKNHj0ZQ\nUBBKSkrsPJV+Q4cOBQDcvHkTbW1tGDx4sJ0n6u7SpUvIy8vD4sWLFf+LbKXPB3Re0fXnP/8ZQ4YM\nwcCBA7W/n1KqI0eO4P7774eHh4e9R+nGxcUFIoL6+no0NTXh1q1bPa5RvV6gTb1Gmsx34cIFlJeX\nK/Y/eTs6OhASEoIxY8bghRdeUOQ/BCtXrkRGRgYGDFDMfZH1UqlUmD59Oh577DHk5ubaexy9Ll26\nhObmZixbtgxTpkzBW2+9hebmZnuPZdCePXswd+5ce4+hl4uLC7Zu3QovLy+4ubnhgQce6PGfdWX/\nv9cJNTQ0YM6cOXjnnXcwbNgwe4+j14ABA3DmzBlcuHABWVlZ+Prrr+09UhcHDx7E7373O4SFhSm+\nTk+cOIEzZ85g/fr1ePnll3HlyhV7j9RNc3MzvvvuOzz55JPQaDQoLy/H3r177T1Wj1pbW3HgwAE8\n9dRT9h5Fr2vXrmHZsmX45ptvUFVVhZMnT+Kzzz7T+9xeL9ARERH49ttvtX8uLy9HVFRUb09HAG7f\nvo0nn3wSzzzzDB599FF7j2OUl5cXEhMTFbe1VVhYiNzcXHh7eyM1NRUFBQWYN2+evcfSa+zYsQCA\ngIAAJCcn48CBA3aeqDsfHx/4+/sjKSkJLi4uSE1NRX5+vr3H6lF+fj4mTZqE3/72t/YeRa/i4mJE\nRUXBx8cHrq6ueOqpp3D8+HG9z+31As1rpC1LRPDss89iwoQJhn+ra2c1NTW4ceMGAKC2thaHDx9W\n3L9M0tPT8dNPP6GyshJ79uzB9OnT8dFHH9l7rG5u3bqFhoYGAJ1VdejQoW4vBFMKX19fFBUVoaOj\nA5999hlmzpxp75F6tHv3bqSmptp7jB7FxMSgpKQEdXV1aGlpQX5+PmbNmqX/yX35baRGo5Hx48fL\n/fffL5s3b+7LqawmJSVFxo4dK3fffbe4u7vLBx98YO+R9PrXv/4lKpVKQkJCJDQ0VEJDQyU/P9/e\nY3VTVlYmYWFhMnHiRJk1a5bs2rXL3iMZpNFoFHsVx48//ighISESEhIi06dPlx07dth7pB5VVFTI\nlClTJCQkRFatWiU3b96090h63bx5U1xdXRV7BdQdO3fulGnTpsnkyZNlzZo10t7ervd5Bt+Lg4iI\n7Ie/JCQiUigu0ERECsUFmohsqqKiAk8//TQCAwORkpKCpqYmvc/Lzs5GbGwsgoKCsH37dqPHiwjW\nrl2LyZMnIzQ0FKdOndIec+jQIcTHxyMgIKDL2yVXV1fjueeew4QJEzBr1qwud4zasmULoqOjMWHC\nBBw8eFD7+F/+8hf4+PhgwIABqKurM/r3zc7O7vIS9LvuugtlZWWm/bBsuTFORM7l6NGjsmDBgi6P\npaamyt69e0VEZP369bJly5Zux924cUP8/Pykrq5OGhoaJCIiQurr6w0ef+jQIXnsscektbVVKisr\nJTo6WkQ6X/bt6+srFy9elJaWFklOTpaysjIREXn11VclIyNDRESys7Pl5ZdfFpHOX4pGRkZKY2Oj\n1NTUSHBwsHa2r7/+Wqqqqnr1Mv2zZ8+Kj4+Pyc9nQROR1ei7oYdGo9G+gVVycjJOnDjR7TmFhYUI\nDw/HqFGjcM899yAuLg6FhYUGjy8oKEBCQgIGDRoELy8vqFQqNDY2oqKiAq6urvDw8MDdd9+NhISE\nLsckJydrz3Xy5EkAwBdffIHp06dj6NChcHV1RUBAgPZ1H6GhofDUc1+/5uZmZGZmIjY2FrNnz9Z7\nb8Ts7GykpKSY/PPjAk1EViN6LhJ76KGH8OGHH6KlpQW7du3SLry6pk2bhuLiYlRWVqK6uhp5eXna\nxbOn4+Pj47F//37cuHEDpaWlOHXqFIqLixEQEIDa2lqcPn0a169fx759+7Tnio+Px65du9Da2oqd\nO3eitLQUra2tSEhIQH5+Pq5cuYIffvgBGo1Ge0xP9uzZg4EDB+LYsWP44IMPsHr16m7P2bt3r1nX\naBt9P2giInNFRUWhpaUFN2/eRF1dHcLCwgAAGzZswOuvv46NGzciKioKM2bMgIuLS7fjhw0bhk2b\nNuH5559HfX09goODtW/K1dPxarUap0+fxuzZs+Hq6oqIiAjtMTt27MCbb76JixcvYuLEidr3aHnx\nxReRkZGBqKgoPPjgg3B1dYVKpYK3tzdWrlypfT8P3XP1JCcnB1VVVdi5cycA4Pr166isrIS3tzcA\noKioCEOHDkVgYKDpP0izNlCIiMyg0Wi67UHrysvLkxdffNHoeebMmSOlpaVmHe/v76/3BSCrV6+W\nnJycbo9/88038sgjj+g9V3R0dLe3U/7fPejExEQ5duxYj3+HFStWyPr163v8uj7c4iAiqxE9WxzX\nrl0DAFy+fBlZWVmIj4/Xe+zVq1cBdL516NmzZxEeHm7w+KamJjQ2NqKtrQ1ZWVkIDg7WlvKdc5WU\nlCA3NxczZswA0Pl2BR0dHbh+/ToyMjK0L7Xv6OhAbW0tRAT79u3DgAED9N5NSvfvN3fuXGzbtk37\n8n3dNxHr6OjAxx9/bNb+851vQERkFRqNRhYuXNjlsc2bN4ufn5/4+vrKunXrtI9fvnxZEhMTtX+O\niYkRf39/mTx5shQVFRk9/s6NQ3x8fCQpKUmuXr2q/VpqaqqMHz9egoKC5ODBg9rH9+3bJ76+vuLj\n4yMrVqxOYfbUAAAAc0lEQVSQtrY2ERFpamqSwMBA8fX1ldjY2C43pdi8ebO4u7vLoEGDZNy4cbJk\nyRIR6byBxaZNmyQ6OlqCgoK6vMXA0aNHtVeVmIMv9SYiUihucRARKRQXaCIiheICTUSkUFygiYgU\nigs0EZFCcYEmIlKo/wOM9kBz9aFClAAAAABJRU5ErkJggg==\n",
"text": [
"<matplotlib.figure.Figure at 0x3def5d0>"
]
}
],
"prompt_number": 16
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"plt.plot(nasty['X'], nasty['ZERO'])"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 17,
"text": [
"[<matplotlib.lines.Line2D at 0x41482d0>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD9CAYAAAC1DKAUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFzFJREFUeJzt3H9MVff9x/HXNSZqrWWl6FymXIveyr1aCwpczCy7OkQ2\nxrARU2mmTcUF7VZQi1k0LrVNWtfZHxhilZrQtUNmmvlHqUodbEFalXthNXZFTNVC1KqtYKSgsKKe\n7x9u9zvGRYGrvdjP85HchHPP59z7birnyT33gs2yLEsAAOMMCfUAAIDQIAAAYCgCAACGIgAAYCgC\nAACGIgAAYKigA1BdXS2n0ymHw6HCwsKAa9auXauoqCjNmDFDx44d899/+fJlPfnkk3rooYfkcrlU\nU1MT7DgAgD4KOgB5eXkqKipSZWWltmzZoubm5m77fT6fPvzwQ9XV1Sk/P1/5+fn+fc8995wiIyP1\nySef6JNPPpHT6Qx2HABAHwUVgNbWVklSUlKS7Ha7UlJS5PV6u63xer3KzMxUeHi4srKy1NDQ4N9X\nWVmpdevWafjw4Ro6dKjCwsKCGQcA0A9BBaC2tlbR0dH+7UCXcXw+n1wul3979OjR+vzzz3XmzBl1\ndnZqxYoVcrvdevnll9XZ2RnMOACAfhh6p5/AsiwF+msTnZ2d+uyzz7Rp0yYlJycrJydH7777rpYs\nWdJtnc1mu9MjAsB30q3+0k9QrwDi4+O7valbX1+vxMTEbmvcbreOHj3q375w4YKioqI0adIkTZ48\nWenp6RoxYoSysrJUXl4e8Hn+E5HBfHvuuedCPgNzMufdOiNz3v5bXwQVgP9cs6+urlZTU5MqKirk\ndru7rXG73dq1a5daWlpUWlra7Y1eh8Mhr9er69eva8+ePUpOTg5mHABAPwR9CaigoEA5OTnq6upS\nbm6uIiIiVFRUJEnKyclRQkKCZs2apbi4OIWHh6ukpMR/7CuvvKIlS5aos7NTycnJWrRoUbDjAAD6\nyGb19bVCiNhstj6/nAmlqqoqeTyeUI9xS8x5e90Nc94NM0rMebv15dxJAADgO6gv507+FAQAGIoA\nAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAICh\nCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAAGIoAAIChCAAA\nGCroAFRXV8vpdMrhcKiwsDDgmrVr1yoqKkozZszQsWPHuu27du2aYmNjlZ6eHuwoAIB+CDoAeXl5\nKioqUmVlpbZs2aLm5uZu+30+nz788EPV1dUpPz9f+fn53fZv3rxZLpdLNpst2FEAAP0QVABaW1sl\nSUlJSbLb7UpJSZHX6+22xuv1KjMzU+Hh4crKylJDQ4N/35kzZ7R3714tW7ZMlmUFMwoAoJ+CCkBt\nba2io6P92y6XSzU1Nd3W+Hw+uVwu//bo0aP1+eefS5JWrVqlTZs2acgQ3ooAgG/b0Dv9BJZlBfzp\nfvfu3RozZoxiY2NVVVV108fYsGGD/2uPxyOPx3N7hwSAu1xVVdUtz6X/y2YFce2ltbVVHo9Hhw8f\nliQ988wzSk1NVVpamn9NYWGhrl69qlWrVkmSJk6cqJMnT2rdunX605/+pKFDh6qzs1Nff/21FixY\noHfeeaf7gDYbl4cAoJ/6cu4M6tpLWFiYpBufBGpqalJFRYXcbne3NW63W7t27VJLS4tKS0vldDol\nSS+99JJOnz6txsZG7dy5U3PmzOlx8gcA3DlBXwIqKChQTk6Ourq6lJubq4iICBUVFUmScnJylJCQ\noFmzZikuLk7h4eEqKSkJ+Dh8CggAvl1BXQL6NnAJCAD6745fAgIA3L0IAAAYigAAgKEIAAAYigAA\ngKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEI\nAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKGCDkB1\ndbWcTqccDocKCwsDrlm7dq2ioqI0Y8YMHTt2TJJ0+vRpzZ49W1OmTJHH41FpaWmwowAA+sFmWZYV\nzAPExsZq8+bNstvtmjdvnj766CNFRET49/t8Pq1evVplZWXat2+fduzYod27d+v8+fM6f/68YmJi\n1NzcrISEBB05ckSjRo3qPqDNpiBHBADj9OXcGdQrgNbWVklSUlKS7Ha7UlJS5PV6u63xer3KzMxU\neHi4srKy1NDQIEkaO3asYmJiJEkRERGaMmWK6urqghkHANAPQ4M5uLa2VtHR0f5tl8ulmpoapaWl\n+e/z+XxavHixf3v06NE6efKkJk6c6L/vxIkTqq+vV0JCQsDn2bBhg/9rj8cjj8cTzNgA8J1TVVWl\nqqqqfh0TVAD6wrKsHi9DbDab/+u2tjY9/vjjev311zVy5MiAj/HfAQAA9PS/Pxw///zztzwmqEtA\n8fHx/jd1Jam+vl6JiYnd1rjdbh09etS/feHCBUVFRUmSurq6tGDBAi1evFgZGRnBjAIA6KegAhAW\nFibpxieBmpqaVFFRIbfb3W2N2+3Wrl271NLSotLSUjmdTkk3XhlkZ2dr6tSpWrlyZTBjAAAGIOhL\nQAUFBcrJyVFXV5dyc3MVERGhoqIiSVJOTo4SEhI0a9YsxcXFKTw8XCUlJZKkAwcOqKSkRNOmTVNs\nbKwkaePGjUpNTQ12JABAHwT9MdA7jY+BAkD/3fGPgQIA7l4EAAAMRQAAwFAEAAAMRQAAwFAEAAAM\nRQAAwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMRQAA\nwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMRQAAwFAEAAAMFXQAqqur5XQ65XA4VFhY\nGHDN2rVrFRUVpRkzZujYsWP9OhYAcGfYLMuygnmA2NhYbd68WXa7XfPmzdNHH32kiIgI/36fz6fV\nq1errKxM+/bt044dO7R79+4+HStJNptNQY4IAMbpy7kzqFcAra2tkqSkpCTZ7XalpKTI6/V2W+P1\nepWZmanw8HBlZWWpoaGhz8cCAO6cocEcXFtbq+joaP+2y+VSTU2N0tLS/Pf5fD4tXrzYvz169Gid\nPHlSjY2Ntzz2P2y2Df+15fn3DQDw/6r+feu7oALQF5Zl9XgZYrPZ+vkYG27jRADwXeTRf/9wbLM9\nf8sjgroEFB8f3+1N3fr6eiUmJnZb43a7dfToUf/2hQsXFBUVpbi4uFseCwC4c4IKQFhYmKQbn+Zp\nampSRUWF3G53tzVut1u7du1SS0uLSktL5XQ6JUnf+973bnksAODOCfoSUEFBgXJyctTV1aXc3FxF\nRESoqKhIkpSTk6OEhATNmjVLcXFxCg8PV0lJyU2PBQB8O4L+GOidxsdAAaD/7vjHQAEAdy8CAACG\nIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAA\nYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgCAACGIgAAYCgC\nAACGCioAbW1tysjIUGRkpObPn6/29vaA66qrq+V0OuVwOFRYWOi/f82aNXI6nZo+fbpWrlypjo6O\nYMYBAPRDUAHYunWrIiMjdfz4cY0bN07btm0LuC4vL09FRUWqrKzUli1b1NLSIklKSUlRfX296urq\ndPnyZZWWlgYzDgCgH4IKgM/nU3Z2toYNG6alS5fK6/X2WNPa2ipJSkpKkt1uV0pKimpqaiRJc+fO\n1ZAhQzRkyBDNmzdP+/fvD2YcAEA/DA3m4NraWkVHR0uSoqOj5fP5brpGklwul2pqapSWltZt3fbt\n27Vs2bKAz7Nhwwb/1x6PRx6PJ5ixAeA7p6qqSlVVVf065pYBmDt3rs6fP9/j/hdffFGWZfXryXrz\nwgsvaNSoUVq4cGHA/f8dAABAT//7w/Hzzz9/y2NuGYCKiope97399ttqaGhQbGysGhoaFB8f32NN\nfHy81qxZ49+ur69Xamqqf/uPf/yj9u3bp7/97W+3HBYAcPsE9R6A2+1WcXGxOjo6VFxcrMTExB5r\nwsLCJN34JFBTU5MqKirkdrslSR988IE2bdqksrIyDR8+PJhRAAD9ZLOCuI7T1tamX/7ylzp8+LCm\nT5+ukpIS3XvvvTp79qx+9atfac+ePZKk/fv3a/ny5erq6lJubq5yc3MlSQ6HQ998843Cw8MlSTNn\nztQbb7zRfUCb7bZdagIAU/Tl3BlUAL4NBAAA+q8v505+ExgADEUAAMBQBAAADEUAAMBQBAAADEUA\nAMBQBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQ\nBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQBAAADEUAAMBQAw5AW1ubMjIyFBkZ\nqfnz56u9vT3guurqajmdTjkcDhUWFvbY/+qrr2rIkCG6ePHiQEcBAAzAgAOwdetWRUZG6vjx4xo3\nbpy2bdsWcF1eXp6KiopUWVmpLVu2qLm52b/v9OnTqqiokN1uH+gYAIABGnAAfD6fsrOzNWzYMC1d\nulRer7fHmtbWVklSUlKS7Ha7UlJSuq1bvXq1/vCHPwx0BABAEAYcgNraWkVHR0uSoqOj5fP5brpG\nklwul2pqaiRJ7733nsaNG6dp06YNdAQAQBCG3mzn3Llzdf78+R73v/jii7Isa0BPaLPZ1NHRoZde\nekkVFRX++2/2eBs2bPB/7fF45PF4BvTcAPBdVVVVpaqqqn4dY7MGeCZfsGCB1q9fr9jYWP3jH//Q\nxo0b9Ze//KXbmtbWVnk8Hh0+fFiS9Mwzzyg1NVV2u10/+clPdM8990iSzpw5ox/+8Ify+XwaM2ZM\n9wFttgHHBgBM1Zdz54AvAbndbhUXF6ujo0PFxcVKTEzssSYsLEzSjU8CNTU1qaKiQm63W1OnTtWX\nX36pxsZGNTY2aty4cfr44497nPwBAHfOgAOwYsUKnTp1SpMnT9YXX3yh5cuXS5LOnj2rtLQ0/7qC\nggLl5OQoOTlZTz/9tCIiIno8ls1mG+gYAIABGvAloG8Ll4AAoP/u6CUgAMDdjQAAgKEIAAAYigAA\ngKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEI\nAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEIAAAYigAAgKEGHIC2\ntjZlZGQoMjJS8+fPV3t7e8B11dXVcjqdcjgcKiws7LbvrbfektPp1JQpU/Tb3/52oKMMClVVVaEe\noU+Y8/a6G+a8G2aUmDMUBhyArVu3KjIyUsePH9e4ceO0bdu2gOvy8vJUVFSkyspKbdmyRc3NzZKk\nTz/9VG+++abKyspUX1+v/Pz8gY4yKNwt/yiY8/a6G+a8G2aUmDMUBhwAn8+n7OxsDRs2TEuXLpXX\n6+2xprW1VZKUlJQku92ulJQU/7ry8nJlZ2fL4XBIkkaPHj3QUQAAAzDgANTW1io6OlqSFB0dLZ/P\nd9M1kuRyuVRTUyNJ2rdvnz799FPFxcVp2bJlOnr06EBHAQAMhHUTycnJ1tSpU3vc3nvvPWv8+PFW\nR0eHZVmWdfnyZSsyMrLH8RUVFdaiRYv821u3brV+97vfWZZlWbNmzbIWL15sXblyxSorK7Nmz54d\ncAZJ3Lhx48ZtALdbGaqbqKio6HXf22+/rYaGBsXGxqqhoUHx8fE91sTHx2vNmjX+7fr6eqWmpkqS\nEhMT5fF4NGLECKWnpysnJ0ednZ0aPnx4t8e40QAAwO024EtAbrdbxcXF6ujoUHFxsRITE3usCQsL\nk3Tjk0BNTU2qqKiQ2+2WJM2cOVPl5eWyLEter1cTJ07scfIHANw5Aw7AihUrdOrUKU2ePFlffPGF\nli9fLkk6e/as0tLS/OsKCgqUk5Oj5ORkPf3004qIiJAkZWRk6OrVq3K5XPr973+v1157Lcj/FABA\nv9zyIlGIPPXUU9aYMWOsqVOnhnqUmzp16pTl8Xgsl8tl/fjHP7Z27NgR6pEC6ujosBISEqxHHnnE\ncrvd1muvvRbqkXp19epVKyYmxvr5z38e6lF6ZbfbrYcfftiKiYmx4uPjQz1Or9rb260lS5ZYDofD\ncjqd1qFDh0I9Ug/Hjh2zYmJi/Lf77rvP2rx5c6jHCujNN9+0Zs6caU2fPt3Ky8sL9Ti92rFjh5WU\nlGS5XC5r+/btva4btAGorq62Pv7440EfgHPnzlmHDx+2LMuyLly4YD344IPW119/HeKpArt8+bJl\nWZbV2dlpTZkyxTp+/HiIJwrs1VdftZ544gkrPT091KP0asKECVZLS0uox7ilZ5991lq/fr3V0dFh\ndXV1WZcuXQr1SDd17do1a+zYsdapU6dCPUoPLS0t1oQJE6z29nbr2rVr1k9/+lPrgw8+CPVYPVy6\ndMl66KGHrIsXL1ptbW1WfHx8r//fB+2fgnj00Ud1//33h3qMWxo7dqxiYmIkSREREZoyZYrq6upC\nPFVg99xzjySpvb1dV69e1bBhw0I8UU9nzpzR3r17tWzZskH/AYDBPp8kVVZWat26dRo+fLiGDh3q\nf19usKqsrNTEiRM1fvz4UI/Sw4gRI2RZllpbW9XR0aErV64MynPUwYMHNX36dN1///269957NXv2\nbB06dCjg2kEbgLvRiRMnVF9fr4SEhFCPEtD169f1yCOP6Pvf/75+85vfDMpvslWrVmnTpk0aMmRw\n/9O02WyaM2eO5s+fr7KyslCPE9CZM2fU2dmpFStWyO126+WXX1ZnZ2eox7qpnTt36oknngj1GAGN\nGDFCW7du1YQJEzR27Fj96Ec/GpTf60lJSfL5fGpsbNS5c+e0d+9eHTx4MODawf1ddhdpa2vT448/\nrtdff10jR44M9TgBDRkyREeOHNGJEyf0xhtv6PDhw6EeqZvdu3drzJgxio2NHfQ/XR84cEBHjhzR\nxo0btXr1ap0/fz7UI/XQ2dmpzz77TAsWLFBVVZXq6+v17rvvhnqsXn3zzTd6//33tXDhwlCPEtCF\nCxe0YsUKHT16VE1NTTp06JD27NkT6rF6GDlypAoKCvTrX/9amZmZevjhh3v9hCUBuA26urq0YMEC\nLV68WBkZGaEe55YmTJign/3sZwH/fEcoHTx4UGVlZXrwwQeVlZWlv//971qyZEmoxwroBz/4gSTJ\n6XTqF7/4hd5///0QT9TTpEmTNHnyZKWnp2vEiBHKyspSeXl5qMfqVXl5uWbMmDFo/yyMz+dTYmKi\nJk2apAceeEALFy5UdXV1qMcKKD09XXv37tWBAwd0/fp1/+9f/S8CECTLspSdna2pU6dq5cqVoR6n\nV83Nzbp06ZIkqaWlRX/9618HXaxeeuklnT59Wo2Njdq5c6fmzJmjd955J9Rj9XDlyhW1tbVJuvFT\n4b59+3r9Bgs1h8Mhr9er69eva8+ePUpOTg71SL3685//rKysrFCP0atHH31UdXV1unjxov71r3+p\nvLxcKSkpoR4roK+++krSjfdU/vnPf2r69OkB1930N4FDKSsrS/v371dLS4vGjx+vF154QU899VSo\nx+rhwIEDKikp0bRp0xQbGytJ2rhx46A7IZw7d05PPvmkrl27prFjxyo/P9//U+xgZbPZQj1CQF9+\n+aUee+wxSdIDDzygZ599dlC+nyJJr7zyipYsWaLOzk4lJydr0aJFoR4poMuXL6uyslLbt28P9Si9\nuu+++7R+/Xo99thjunLlilJTUzV79uxQjxVQZmamvvrqK40aNUpvvfVWr+ts1mC/2AoAuCO4BAQA\nhiIAAGAoAgAAhiIAAGAoAgAAhiIAAGCo/wNKquKxz912XgAAAABJRU5ErkJggg==\n",
"text": [
"<matplotlib.figure.Figure at 0x411c6d0>"
]
}
],
"prompt_number": 17
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Correlation matrix\n",
"------------------\n",
"\n",
"D. Compute a correlation matrix on all the variables. All the\n",
"correlations, except for ZERO and MISS, shoud be exactly 1. ZERO\n",
"and MISS should have undefined or missing correlations with the\n",
"other variables. The same should go for SPEARMAN corelations, if\n",
"your program has them."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"nasty.corr()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>BIG</th>\n",
" <th>HUGE</th>\n",
" <th>LITTLE</th>\n",
" <th>MISS</th>\n",
" <th>ROUND</th>\n",
" <th>TINY</th>\n",
" <th>X</th>\n",
" <th>ZERO</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>BIG</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>HUGE</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>LITTLE</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>MISS</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ROUND</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>TINY</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>X</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ZERO</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"output_type": "pyout",
"prompt_number": 18,
"text": [
" BIG HUGE LITTLE MISS ROUND TINY X ZERO\n",
"BIG 1 1 1 NaN 1 1 1 NaN\n",
"HUGE 1 1 1 NaN 1 1 1 NaN\n",
"LITTLE 1 1 1 NaN 1 1 1 NaN\n",
"MISS NaN NaN NaN NaN NaN NaN NaN NaN\n",
"ROUND 1 1 1 NaN 1 1 1 NaN\n",
"TINY 1 1 1 NaN 1 1 1 NaN\n",
"X 1 1 1 NaN 1 1 1 NaN\n",
"ZERO NaN NaN NaN NaN NaN NaN NaN NaN"
]
}
],
"prompt_number": 18
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"nasty.corr(method='spearman')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>BIG</th>\n",
" <th>HUGE</th>\n",
" <th>LITTLE</th>\n",
" <th>MISS</th>\n",
" <th>ROUND</th>\n",
" <th>TINY</th>\n",
" <th>X</th>\n",
" <th>ZERO</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>BIG</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>HUGE</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>LITTLE</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>MISS</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ROUND</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>TINY</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>X</th>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ZERO</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"output_type": "pyout",
"prompt_number": 19,
"text": [
" BIG HUGE LITTLE MISS ROUND TINY X ZERO\n",
"BIG 1 1 1 NaN 1 1 1 NaN\n",
"HUGE 1 1 1 NaN 1 1 1 NaN\n",
"LITTLE 1 1 1 NaN 1 1 1 NaN\n",
"MISS NaN NaN NaN NaN NaN NaN NaN NaN\n",
"ROUND 1 1 1 NaN 1 1 1 NaN\n",
"TINY 1 1 1 NaN 1 1 1 NaN\n",
"X 1 1 1 NaN 1 1 1 NaN\n",
"ZERO NaN NaN NaN NaN NaN NaN NaN NaN"
]
}
],
"prompt_number": 19
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Tabulate\n",
"\n",
"E. Tabulate X against X, using BIG as a case weight. The values \n",
"should appear on the diagonal and the total should be 899999955.\n",
"If the table cannot hold these values, forget about working with \n",
"census data. You can also tabulate HUGE against TINY. There is no\n",
"reason a tabulation program should not be able to digtinguish \n",
"different values regardless of their magnitude."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Regression\n",
"==========\n",
"\n",
"Regress BIG on X. The constant should be 99999990 and the\n",
"regression coefficient should be 1."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import statsmodels\n",
"import statsmodels.api as sm\n",
"statsmodels.__version__"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 20,
"text": [
"'0.4.3'"
]
}
],
"prompt_number": 20
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"X = nasty['X']\n",
"Y = nasty['BIG']\n",
"X = sm.add_constant(X)\n",
"big_x = sm.OLS(Y, X).fit()\n",
"big_x.summary()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stderr",
"text": [
"/home/lluang/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/statsmodels/tools/tools.py:306: FutureWarning: The default of `prepend` will be changed to True in 0.5.0, use explicit prepend\n",
" FutureWarning)\n",
"/home/lluang/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scipy/stats/stats.py:1276: UserWarning: kurtosistest only valid for n>=20 ... continuing anyway, n=9\n",
" int(n))\n"
]
},
{
"html": [
"<table class=\"simpletable\">\n",
"<caption>OLS Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>BIG</td> <th> R-squared: </th> <td> 1.000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>OLS</td> <th> Adj. R-squared: </th> <td> 1.000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>Least Squares</td> <th> F-statistic: </th> <td>1.478e+16</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Sun, 28 Jul 2013</td> <th> Prob (F-statistic):</th> <td>6.73e-55</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>22:20:08</td> <th> Log-Likelihood: </th> <td> 137.48</td> \n",
"</tr>\n",
"<tr>\n",
" <th>No. Observations:</th> <td> 9</td> <th> AIC: </th> <td> -271.0</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Df Residuals:</th> <td> 7</td> <th> BIC: </th> <td> -270.6</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Df Model:</th> <td> 1</td> <th> </th> <td> </td> \n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>t</th> <th>P>|t|</th> <th>[95.0% Conf. Int.]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>X</th> <td> 1.0000</td> <td> 8.23e-09</td> <td> 1.22e+08</td> <td> 0.000</td> <td> 1.000 1.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>const</th> <td> 1e+08</td> <td> 4.63e-08</td> <td> 2.16e+15</td> <td> 0.000</td> <td> 1e+08 1e+08</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <th>Omnibus:</th> <td> 2.026</td> <th> Durbin-Watson: </th> <td> 0.047</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Prob(Omnibus):</th> <td> 0.363</td> <th> Jarque-Bera (JB): </th> <td> 0.796</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Skew:</th> <td>-0.061</td> <th> Prob(JB): </th> <td> 0.672</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Kurtosis:</th> <td> 1.548</td> <th> Cond. No. </th> <td> 12.6</td>\n",
"</tr>\n",
"</table>"
],
"output_type": "pyout",
"prompt_number": 21,
"text": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: BIG R-squared: 1.000\n",
"Model: OLS Adj. R-squared: 1.000\n",
"Method: Least Squares F-statistic: 1.478e+16\n",
"Date: Sun, 28 Jul 2013 Prob (F-statistic): 6.73e-55\n",
"Time: 22:20:08 Log-Likelihood: 137.48\n",
"No. Observations: 9 AIC: -271.0\n",
"Df Residuals: 7 BIC: -270.6\n",
"Df Model: 1 \n",
"==============================================================================\n",
" coef std err t P>|t| [95.0% Conf. Int.]\n",
"------------------------------------------------------------------------------\n",
"X 1.0000 8.23e-09 1.22e+08 0.000 1.000 1.000\n",
"const 1e+08 4.63e-08 2.16e+15 0.000 1e+08 1e+08\n",
"==============================================================================\n",
"Omnibus: 2.026 Durbin-Watson: 0.047\n",
"Prob(Omnibus): 0.363 Jarque-Bera (JB): 0.796\n",
"Skew: -0.061 Prob(JB): 0.672\n",
"Kurtosis: 1.548 Cond. No. 12.6\n",
"==============================================================================\n",
"\"\"\""
]
}
],
"prompt_number": 21
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Missing data\n",
"============\n",
"\n",
"A. Use the NASTY dataset above on the following transformation:\n",
"\n",
" IF MISS = 3 THEN TEST = 1\n",
" ELSE TEST = 2\n",
"\n",
" If a package does not have an else statement (a serious omission if\n",
" you are doing a lot of IF-THEN transformations), you can code the\n",
" second statement as \"IF MISS <> 3 THEN TEST = 2\" where <> is \"not\n",
" equal\". TEST should have the value 2 for all cases because MISS\n",
" does not anywhere equal 3 (i.e. missing values do not equal 3).\n",
" Some packages have an \"indeterminate\" value if MISS=<missing> and\n",
" assign neither 1 or 2 to TEST. That is OK provided they assign\n",
" a missing value to TEST. IF the package assigns any other value to\n",
" TEST (say, 1), don't trust it for any logical comparisons.\n",
"\n",
"B. Use the NASTY dataset on the following calculation:\n",
"\n",
" IF MISS = <missing> THEN MISS = MISS + 1\n",
"\n",
" This transformation should leave the values of MISS as missing (you\n",
" cannot add 1 to something that is missing).\n",
" \n",
"C. Use the NASTY dataset and tabulate MISS against ZERO. You\n",
" should have one cell with 9 cases in it. This ability to tabulate\n",
" missing values against other variables is essential for analyzing\n",
" patterns of missing data in a file. Now tabulate MISS against ZERO\n",
" excluding missing values from the tablulation. You should be \n",
" notified that there are no non-missing values to tabulate."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"[1 if nasty['MISS'][i]==3 else 2 for i in range(len(nasty['MISS']))]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 22,
"text": [
"[2, 2, 2, 2, 2, 2, 2, 2, 2]"
]
}
],
"prompt_number": 22
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"[np.nan if np.isnan(nasty['MISS'][i]) else nasty['MISS'][i] +1 for i in range(len(nasty['MISS']))]"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 23,
"text": [
"[nan, nan, nan, nan, nan, nan, nan, nan, nan]"
]
}
],
"prompt_number": 23
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Regression\n",
"==========\n",
"\n",
"A. Take the NASTY dataset above.\n",
"--------------------------------\n",
"\n",
"Use the variable X as a\n",
" basis for computing polynomials. Namely, compute X1=X, X2=X*X,\n",
" X3=X*X*X, and so on up to 9 products. Use the algebraic\n",
" transformation language within the statistical package itself. You\n",
" will end up with 9 variables. Now regress X1 on X2-X9 (a perfect\n",
" fit). If the package balks (singular or roundoff error messages),\n",
" try X1 on X2-X8, and so on. Most packages cannot handle more than\n",
" a few polynomials.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"X = [[nasty['X'][i], nasty['X'][i]*nasty['X'][i], nasty['X'][i]*nasty['X'][i]*nasty['X'][i]] for i in range(len(nasty['X']))]\n",
"Y = nasty['X']\n",
"X = sm.add_constant(X)\n",
"x9_x = sm.OLS(Y, X).fit()\n",
"x9_x.summary()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<table class=\"simpletable\">\n",
"<caption>OLS Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>X</td> <th> R-squared: </th> <td> 1.000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>OLS</td> <th> Adj. R-squared: </th> <td> 1.000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>Least Squares</td> <th> F-statistic: </th> <td>4.734e+27</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Sun, 28 Jul 2013</td> <th> Prob (F-statistic):</th> <td>4.74e-69</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>22:20:08</td> <th> Log-Likelihood: </th> <td> 263.15</td> \n",
"</tr>\n",
"<tr>\n",
" <th>No. Observations:</th> <td> 9</td> <th> AIC: </th> <td> -518.3</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Df Residuals:</th> <td> 5</td> <th> BIC: </th> <td> -517.5</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Df Model:</th> <td> 3</td> <th> </th> <td> </td> \n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>t</th> <th>P>|t|</th> <th>[95.0% Conf. Int.]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>x1</th> <td> 1.0000</td> <td> 1.15e-13</td> <td> 8.68e+12</td> <td> 0.000</td> <td> 1.000 1.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>x2</th> <td> 6.661e-16</td> <td> 2.61e-14</td> <td> 0.026</td> <td> 0.981</td> <td>-6.64e-14 6.77e-14</td>\n",
"</tr>\n",
"<tr>\n",
" <th>x3</th> <td>-1.665e-16</td> <td> 1.72e-15</td> <td> -0.097</td> <td> 0.927</td> <td>-4.59e-15 4.26e-15</td>\n",
"</tr>\n",
"<tr>\n",
" <th>const</th> <td> 7.994e-15</td> <td> 1.41e-13</td> <td> 0.057</td> <td> 0.957</td> <td>-3.53e-13 3.69e-13</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <th>Omnibus:</th> <td> 2.114</td> <th> Durbin-Watson: </th> <td> 0.098</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Prob(Omnibus):</th> <td> 0.347</td> <th> Jarque-Bera (JB): </th> <td> 1.170</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Skew:</th> <td> 0.839</td> <th> Prob(JB): </th> <td> 0.557</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Kurtosis:</th> <td> 2.448</td> <th> Cond. No. </th> <td>2.77e+03</td>\n",
"</tr>\n",
"</table>"
],
"output_type": "pyout",
"prompt_number": 24,
"text": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: X R-squared: 1.000\n",
"Model: OLS Adj. R-squared: 1.000\n",
"Method: Least Squares F-statistic: 4.734e+27\n",
"Date: Sun, 28 Jul 2013 Prob (F-statistic): 4.74e-69\n",
"Time: 22:20:08 Log-Likelihood: 263.15\n",
"No. Observations: 9 AIC: -518.3\n",
"Df Residuals: 5 BIC: -517.5\n",
"Df Model: 3 \n",
"==============================================================================\n",
" coef std err t P>|t| [95.0% Conf. Int.]\n",
"------------------------------------------------------------------------------\n",
"x1 1.0000 1.15e-13 8.68e+12 0.000 1.000 1.000\n",
"x2 6.661e-16 2.61e-14 0.026 0.981 -6.64e-14 6.77e-14\n",
"x3 -1.665e-16 1.72e-15 -0.097 0.927 -4.59e-15 4.26e-15\n",
"const 7.994e-15 1.41e-13 0.057 0.957 -3.53e-13 3.69e-13\n",
"==============================================================================\n",
"Omnibus: 2.114 Durbin-Watson: 0.098\n",
"Prob(Omnibus): 0.347 Jarque-Bera (JB): 1.170\n",
"Skew: 0.839 Prob(JB): 0.557\n",
"Kurtosis: 2.448 Cond. No. 2.77e+03\n",
"==============================================================================\n",
"\n",
"The condition number is large, 2.77e+03. This might indicate that there are\n",
"strong multicollinearity or other numerical problems.\n",
"\"\"\""
]
}
],
"prompt_number": 24
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"B. Regress X on X \n",
"------------------\n",
" \n",
"The constant should be exactly 0 and the\n",
"regression coefficient should be 1. This is a perfectly valid\n",
"regression. The program should not complain.\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"X = nasty['X']\n",
"Y = nasty['X']\n",
"X = sm.add_constant(X)\n",
"x_x = sm.OLS(Y, X).fit()\n",
"x_x.summary()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<table class=\"simpletable\">\n",
"<caption>OLS Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>X</td> <th> R-squared: </th> <td> 1.000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>OLS</td> <th> Adj. R-squared: </th> <td> 1.000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>Least Squares</td> <th> F-statistic: </th> <td>1.674e+30</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Sun, 28 Jul 2013</td> <th> Prob (F-statistic):</th> <td>4.35e-104</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>22:20:08</td> <th> Log-Likelihood: </th> <td> 283.10</td> \n",
"</tr>\n",
"<tr>\n",
" <th>No. Observations:</th> <td> 9</td> <th> AIC: </th> <td> -562.2</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Df Residuals:</th> <td> 7</td> <th> BIC: </th> <td> -561.8</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Df Model:</th> <td> 1</td> <th> </th> <td> </td> \n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>t</th> <th>P>|t|</th> <th>[95.0% Conf. Int.]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>X</th> <td> 1.0000</td> <td> 7.73e-16</td> <td> 1.29e+15</td> <td> 0.000</td> <td> 1.000 1.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>const</th> <td> 2.887e-15</td> <td> 4.35e-15</td> <td> 0.664</td> <td> 0.528</td> <td> -7.4e-15 1.32e-14</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <th>Omnibus:</th> <td> 1.380</td> <th> Durbin-Watson: </th> <td> 0.013</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Prob(Omnibus):</th> <td> 0.501</td> <th> Jarque-Bera (JB): </th> <td> 0.781</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Skew:</th> <td>-0.327</td> <th> Prob(JB): </th> <td> 0.677</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Kurtosis:</th> <td> 1.714</td> <th> Cond. No. </th> <td> 12.6</td>\n",
"</tr>\n",
"</table>"
],
"output_type": "pyout",
"prompt_number": 25,
"text": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: X R-squared: 1.000\n",
"Model: OLS Adj. R-squared: 1.000\n",
"Method: Least Squares F-statistic: 1.674e+30\n",
"Date: Sun, 28 Jul 2013 Prob (F-statistic): 4.35e-104\n",
"Time: 22:20:08 Log-Likelihood: 283.10\n",
"No. Observations: 9 AIC: -562.2\n",
"Df Residuals: 7 BIC: -561.8\n",
"Df Model: 1 \n",
"==============================================================================\n",
" coef std err t P>|t| [95.0% Conf. Int.]\n",
"------------------------------------------------------------------------------\n",
"X 1.0000 7.73e-16 1.29e+15 0.000 1.000 1.000\n",
"const 2.887e-15 4.35e-15 0.664 0.528 -7.4e-15 1.32e-14\n",
"==============================================================================\n",
"Omnibus: 1.380 Durbin-Watson: 0.013\n",
"Prob(Omnibus): 0.501 Jarque-Bera (JB): 0.781\n",
"Skew: -0.327 Prob(JB): 0.677\n",
"Kurtosis: 1.714 Cond. No. 12.6\n",
"==============================================================================\n",
"\"\"\""
]
}
],
"prompt_number": 25
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"C. Regress X on BIG and LITTLE (two predictors)\n",
"-----------------------------------------------\n",
" \n",
"The program should tell you that this model is \"singular\" because BIG and\n",
"LITTLE are linear combinations of each other. Cryptic error\n",
"messages are unacceptable here. Singularity is the most\n",
"fundamental regression error.\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"Y = nasty['X']\n",
"X = [[nasty['BIG'][i], nasty['LITTLE'][i]] for i in range(len(nasty['BIG']))]\n",
"X=sm.add_constant(X)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 26
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"X"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 27,
"text": [
"array([[ 9.99999910e+07, 9.99999910e-01, 1.00000000e+00],\n",
" [ 9.99999920e+07, 9.99999920e-01, 1.00000000e+00],\n",
" [ 9.99999930e+07, 9.99999930e-01, 1.00000000e+00],\n",
" [ 9.99999940e+07, 9.99999940e-01, 1.00000000e+00],\n",
" [ 9.99999950e+07, 9.99999950e-01, 1.00000000e+00],\n",
" [ 9.99999960e+07, 9.99999960e-01, 1.00000000e+00],\n",
" [ 9.99999970e+07, 9.99999970e-01, 1.00000000e+00],\n",
" [ 9.99999980e+07, 9.99999980e-01, 1.00000000e+00],\n",
" [ 9.99999990e+07, 9.99999990e-01, 1.00000000e+00]])"
]
}
],
"prompt_number": 27
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"big_little_x = sm.OLS(Y, X).fit()\n",
"big_little_x.summary()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<table class=\"simpletable\">\n",
"<caption>OLS Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>X</td> <th> R-squared: </th> <td> 0.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>OLS</td> <th> Adj. R-squared: </th> <td> 0.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>Least Squares</td> <th> F-statistic: </th> <td> inf</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Sun, 28 Jul 2013</td> <th> Prob (F-statistic):</th> <td> nan</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>22:20:08</td> <th> Log-Likelihood: </th> <td> -21.307</td>\n",
"</tr>\n",
"<tr>\n",
" <th>No. Observations:</th> <td> 9</td> <th> AIC: </th> <td> 44.61</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Df Residuals:</th> <td> 8</td> <th> BIC: </th> <td> 44.81</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Df Model:</th> <td> 0</td> <th> </th> <td> </td> \n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>t</th> <th>P>|t|</th> <th>[95.0% Conf. Int.]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>x1</th> <td> 5e-08</td> <td> 9.13e-09</td> <td> 5.477</td> <td> 0.001</td> <td> 2.89e-08 7.11e-08</td>\n",
"</tr>\n",
"<tr>\n",
" <th>x2</th> <td> 5e-16</td> <td> 9.13e-17</td> <td> 5.477</td> <td> 0.001</td> <td> 2.89e-16 7.11e-16</td>\n",
"</tr>\n",
"<tr>\n",
" <th>const</th> <td> 5e-16</td> <td> 9.13e-17</td> <td> 5.477</td> <td> 0.001</td> <td> 2.89e-16 7.11e-16</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <th>Omnibus:</th> <td> 0.824</td> <th> Durbin-Watson: </th> <td> 0.133</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Prob(Omnibus):</th> <td> 0.662</td> <th> Jarque-Bera (JB): </th> <td> 0.567</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Skew:</th> <td> 0.000</td> <th> Prob(JB): </th> <td> 0.753</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Kurtosis:</th> <td> 1.770</td> <th> Cond. No. </th> <td> nan</td>\n",
"</tr>\n",
"</table>"
],
"output_type": "pyout",
"prompt_number": 28,
"text": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: X R-squared: 0.000\n",
"Model: OLS Adj. R-squared: 0.000\n",
"Method: Least Squares F-statistic: inf\n",
"Date: Sun, 28 Jul 2013 Prob (F-statistic): nan\n",
"Time: 22:20:08 Log-Likelihood: -21.307\n",
"No. Observations: 9 AIC: 44.61\n",
"Df Residuals: 8 BIC: 44.81\n",
"Df Model: 0 \n",
"==============================================================================\n",
" coef std err t P>|t| [95.0% Conf. Int.]\n",
"------------------------------------------------------------------------------\n",
"x1 5e-08 9.13e-09 5.477 0.001 2.89e-08 7.11e-08\n",
"x2 5e-16 9.13e-17 5.477 0.001 2.89e-16 7.11e-16\n",
"const 5e-16 9.13e-17 5.477 0.001 2.89e-16 7.11e-16\n",
"==============================================================================\n",
"Omnibus: 0.824 Durbin-Watson: 0.133\n",
"Prob(Omnibus): 0.662 Jarque-Bera (JB): 0.567\n",
"Skew: 0.000 Prob(JB): 0.753\n",
"Kurtosis: 1.770 Cond. No. nan\n",
"==============================================================================\n",
"\n",
"The smallest eigenvalue is -8.18e-15. This might indicate that there are\n",
"strong multicollinearity problems or that the design matrix is singular.\n",
"\"\"\""
]
}
],
"prompt_number": 28
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that this is a failure because it did not report a sigularity"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"D. Regress ZERO on X.\n",
"---------------------\n",
" \n",
"The program should inform you that ZERO has\n",
"no variance or it should go ahead and compute the regression\n",
"and report a correlation and total sum of squares of exactly 0."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"X = nasty['ZERO']\n",
"Y = nasty['X']\n",
"X = sm.add_constant(X)\n",
"zero_x = sm.OLS(Y, X).fit()\n",
"zero_x.summary()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<table class=\"simpletable\">\n",
"<caption>OLS Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>X</td> <th> R-squared: </th> <td> 0.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>OLS</td> <th> Adj. R-squared: </th> <td> 0.000</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>Least Squares</td> <th> F-statistic: </th> <td> nan</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Sun, 28 Jul 2013</td> <th> Prob (F-statistic):</th> <td> nan</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>22:20:08</td> <th> Log-Likelihood: </th> <td> -21.307</td>\n",
"</tr>\n",
"<tr>\n",
" <th>No. Observations:</th> <td> 9</td> <th> AIC: </th> <td> 44.61</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Df Residuals:</th> <td> 8</td> <th> BIC: </th> <td> 44.81</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Df Model:</th> <td> 0</td> <th> </th> <td> </td> \n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>t</th> <th>P>|t|</th> <th>[95.0% Conf. Int.]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>ZERO</th> <td> 0</td> <td> 0</td> <td> nan</td> <td> nan</td> <td> 0 0</td>\n",
"</tr>\n",
"<tr>\n",
" <th>const</th> <td> 5.0000</td> <td> 0.913</td> <td> 5.477</td> <td> 0.001</td> <td> 2.895 7.105</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <th>Omnibus:</th> <td> 1.861</td> <th> Durbin-Watson: </th> <td> 0.133</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Prob(Omnibus):</th> <td> 0.394</td> <th> Jarque-Bera (JB): </th> <td> 0.567</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Skew:</th> <td> 0.000</td> <th> Prob(JB): </th> <td> 0.753</td>\n",
"</tr>\n",
"<tr>\n",
" <th>Kurtosis:</th> <td> 1.770</td> <th> Cond. No. </th> <td> inf</td>\n",
"</tr>\n",
"</table>"
],
"output_type": "pyout",
"prompt_number": 29,
"text": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: X R-squared: 0.000\n",
"Model: OLS Adj. R-squared: 0.000\n",
"Method: Least Squares F-statistic: nan\n",
"Date: Sun, 28 Jul 2013 Prob (F-statistic): nan\n",
"Time: 22:20:08 Log-Likelihood: -21.307\n",
"No. Observations: 9 AIC: 44.61\n",
"Df Residuals: 8 BIC: 44.81\n",
"Df Model: 0 \n",
"==============================================================================\n",
" coef std err t P>|t| [95.0% Conf. Int.]\n",
"------------------------------------------------------------------------------\n",
"ZERO 0 0 nan nan 0 0\n",
"const 5.0000 0.913 5.477 0.001 2.895 7.105\n",
"==============================================================================\n",
"Omnibus: 1.861 Durbin-Watson: 0.133\n",
"Prob(Omnibus): 0.394 Jarque-Bera (JB): 0.567\n",
"Skew: 0.000 Prob(JB): 0.753\n",
"Kurtosis: 1.770 Cond. No. inf\n",
"==============================================================================\n",
"\n",
"The smallest eigenvalue is 0. This might indicate that there are\n",
"strong multicollinearity problems or that the design matrix is singular.\n",
"\"\"\""
]
}
],
"prompt_number": 29
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"ANALYSIS OF VARIANCE\n",
"====================\n",
"\n",
"A. Simple contrasts.\n",
"---------------------\n",
"\n",
"The following data contain an unbalanced\n",
" design with a significant interaction. A least squares analysis\n",
" shows the main efefct for A is not significant, but this test is\n",
" not particularlrly meaningful because of the interaction. Test,\n",
" therefore, the simple constrast between A1 and A2 within B1. Then\n",
" test A1 vs. A2 within B2. Both tests should use the same residual\n",
" error term (separate t-tests are unacceptable). Several widely\n",
" used mainframe programs fail this test. Unless the program can\n",
" constrast any terms in a model (not just main effects), it cannot\n",
" handle this frequently encountered type of problem."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 29
}
],
"metadata": {}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment