数据结构的内置方法.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# 数据结构的内置方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "这一节介绍常用的pandas数据结构内置方法。很重要的一节。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "创建本节要用到的数据结构。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "index  = pd.date_range('1/1/2000', periods=8)\n",
    "s = pd.Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e'])\n",
    "df = pd.DataFrame(np.random.randn(8, 3), index=index, columns=['A', 'B', 'C'])\n",
    "wp = pd.Panel(np.random.randn(2,5,4), items=['Item1', 'Item2'], major_axis=pd.date_range('1/1/2000',periods=5),\n",
    "             minor_axis=['A', 'B', 'C', 'D'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Head() Tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "想要预览Series或DataFrame对象，可以使用head()和tail()方法。默认显示5行数据，你也可以自己设置显示的行数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "long_series = pd.Series(np.random.randn(1000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.023058\n",
       "1   -0.074953\n",
       "2    0.561164\n",
       "3    1.955282\n",
       "4    0.737479\n",
       "dtype: float64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "long_series.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "997   -1.318555\n",
       "998    0.047255\n",
       "999    0.218609\n",
       "dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "long_series.tail(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 属性和 ndarray"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas对象有很多属性，你可以通过这些属性访问数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "* **shape**: 显示对象的维度，同ndarray\n",
    "* 坐标label\n",
    "* * **Series**: index\n",
    "* * **DataFrame**: index(行)和columns\n",
    "* * **Panel**:  items, major_axis and minor_axis\n",
    "\n",
    "可以通过属性进行安全赋值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>1.173233</td>\n",
       "      <td>-0.147631</td>\n",
       "      <td>-1.312819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>-1.266111</td>\n",
       "      <td>-1.125446</td>\n",
       "      <td>0.578458</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01  1.173233 -0.147631 -1.312819\n",
       "2000-01-02 -1.266111 -1.125446  0.578458"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df.columns = [x.lower() for x in df.columns] #将列名重置为小写"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>1.173233</td>\n",
       "      <td>-0.147631</td>\n",
       "      <td>-1.312819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>-1.266111</td>\n",
       "      <td>-1.125446</td>\n",
       "      <td>0.578458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-1.595159</td>\n",
       "      <td>-0.654289</td>\n",
       "      <td>-1.256511</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>-0.907272</td>\n",
       "      <td>-0.054247</td>\n",
       "      <td>-0.596867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>-1.259856</td>\n",
       "      <td>3.229433</td>\n",
       "      <td>1.971463</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>0.108100</td>\n",
       "      <td>0.019998</td>\n",
       "      <td>1.320694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>1.719391</td>\n",
       "      <td>-0.506610</td>\n",
       "      <td>1.039305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>1.107930</td>\n",
       "      <td>-0.191327</td>\n",
       "      <td>0.547800</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   a         b         c\n",
       "2000-01-01  1.173233 -0.147631 -1.312819\n",
       "2000-01-02 -1.266111 -1.125446  0.578458\n",
       "2000-01-03 -1.595159 -0.654289 -1.256511\n",
       "2000-01-04 -0.907272 -0.054247 -0.596867\n",
       "2000-01-05 -1.259856  3.229433  1.971463\n",
       "2000-01-06  0.108100  0.019998  1.320694\n",
       "2000-01-07  1.719391 -0.506610  1.039305\n",
       "2000-01-08  1.107930 -0.191327  0.547800"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "只想得到对象中的数据而忽略index和columns，使用values属性就可以"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.61338428, -0.31247701, -0.54252781,  0.25162093, -0.55944074])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.17323268, -0.14763078, -1.312819  ],\n",
       "       [-1.26611102, -1.12544635,  0.57845752],\n",
       "       [-1.59515867, -0.6542893 , -1.25651072],\n",
       "       [-0.90727235, -0.05424687, -0.59686748],\n",
       "       [-1.25985608,  3.229433  ,  1.97146342],\n",
       "       [ 0.10810032,  0.01999776,  1.32069354],\n",
       "       [ 1.71939091, -0.50660987,  1.03930483],\n",
       "       [ 1.1079299 , -0.19132729,  0.54780008]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(df.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[-1.54942434,  0.29147405,  1.04168054, -0.63775426],\n",
       "        [-0.68954438, -0.30728114,  0.93123234,  1.04587453],\n",
       "        [ 0.52590821, -0.87797709, -1.4053343 , -1.26458272],\n",
       "        [ 0.07689833, -0.62319929, -0.03040175,  0.28789213],\n",
       "        [ 0.26621579,  0.2508008 , -0.13140246,  1.46492144]],\n",
       "\n",
       "       [[ 1.06565079, -0.09858943, -0.39071567, -0.67769326],\n",
       "        [ 0.07779078, -0.16233238, -0.42585241, -0.28715586],\n",
       "        [ 2.41442006,  0.05856983,  0.33624126, -1.19499484],\n",
       "        [ 1.22066297, -1.40060425,  0.53552757,  0.79119371],\n",
       "        [-0.54880785,  0.43709614,  0.42075427, -0.58245336]]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wp.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果DataFrame或Panel对象的数据类型相同(比如都是 int64)，修改object.values相当于直接修改原对象的值。如果数据类型不相同，则根本不能对values属性返回值进行赋值。\n",
    "\n",
    "**注意**:\n",
    "\n",
    "如果对象内数据类型不同，values返回的ndarray的dtype将是能够兼容所有数据类型的类型。比如，有的列数据是int，有的列数据是float，.values返回的ndarray的dtype将是float。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 加速的操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas从0.11.0版本开始使用numexpr库对二值数值类型操作加速，用bottleneck库对布尔操作加速。\n",
    "\n",
    "加速效果对大数据尤其明显。\n",
    "\n",
    "这里有一个速度的简单对比，使用100,000行\\* 100列的DataFrame:\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/04/14/570f692c2e8a1.png)\n",
    "\n",
    "所以，在安装pandas后也要顺便安装numexpr, bottleneck。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 灵活的二元运算"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "在所有的pandas对象之间的二元运算中，大家最感兴趣的一般是下面两个：\n",
    "* 高维数据结构（比如DataFrame）和低维数据结构（比如Series）之间计算时的广播(broadcasting)行为\n",
    "* 计算时有缺失值\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 广播"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "DataFrame对象内置add(), sub(), mul(), div()以及radd(), rsub(),...等方法来进行二元操作。\n",
    "\n",
    "使用这些方法时，你可以通过axis参数来决定广播时匹配index还是columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'one' : pd.Series(np.random.randn(3), index=['a', 'b', 'c']),\n",
    "                       'two' : pd.Series(np.random.randn(4), index=['a', 'b', 'c', 'd']),\n",
    "                       'three' : pd.Series(np.random.randn(3), index=['b', 'c', 'd'])})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.930595</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.693425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-1.452673</td>\n",
       "      <td>0.021665</td>\n",
       "      <td>-0.406850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.284763</td>\n",
       "      <td>0.236374</td>\n",
       "      <td>-0.568421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.660751</td>\n",
       "      <td>0.675628</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  0.930595       NaN  0.693425\n",
       "b -1.452673  0.021665 -0.406850\n",
       "c  0.284763  0.236374 -0.568421\n",
       "d       NaN -0.660751  0.675628"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one     -1.452673\n",
       "three    0.021665\n",
       "two     -0.406850\n",
       "Name: b, dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "row = df.ix[1]\n",
    "row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "column = df['two']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    0.693425\n",
       "b   -0.406850\n",
       "c   -0.568421\n",
       "d    0.675628\n",
       "Name: two, dtype: float64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>2.383268</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.100275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>1.737436</td>\n",
       "      <td>0.214709</td>\n",
       "      <td>-0.161571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.682416</td>\n",
       "      <td>1.082478</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  2.383268       NaN  1.100275\n",
       "b  0.000000  0.000000  0.000000\n",
       "c  1.737436  0.214709 -0.161571\n",
       "d       NaN -0.682416  1.082478"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sub(row, axis='columns')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>2.383268</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.100275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>1.737436</td>\n",
       "      <td>0.214709</td>\n",
       "      <td>-0.161571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.682416</td>\n",
       "      <td>1.082478</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  2.383268       NaN  1.100275\n",
       "b  0.000000  0.000000  0.000000\n",
       "c  1.737436  0.214709 -0.161571\n",
       "d       NaN -0.682416  1.082478"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sub(row, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>one</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>three</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       one  three  two\n",
       "a      NaN    NaN  NaN\n",
       "b      NaN    NaN  NaN\n",
       "c      NaN    NaN  NaN\n",
       "d      NaN    NaN  NaN\n",
       "one    NaN    NaN  NaN\n",
       "three  NaN    NaN  NaN\n",
       "two    NaN    NaN  NaN"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sub(row, axis='index')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>one</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>three</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>two</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       one  three  two\n",
       "a      NaN    NaN  NaN\n",
       "b      NaN    NaN  NaN\n",
       "c      NaN    NaN  NaN\n",
       "d      NaN    NaN  NaN\n",
       "one    NaN    NaN  NaN\n",
       "three  NaN    NaN  NaN\n",
       "two    NaN    NaN  NaN"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sub(row, axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 填充缺失值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "在Series和DataFrame中，算术运算方法(比如add())有一个fill_value参数，含义很明显，计算前用一个值来代替缺失值，然后再参与运算。注意，如果参与运算的两个object同一位置(同行同列)都是NaN，fill_value不起作用，计算结果还是NaN。\n",
    "\n",
    "看例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.930595</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.693425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-1.452673</td>\n",
       "      <td>0.021665</td>\n",
       "      <td>-0.406850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.284763</td>\n",
       "      <td>0.236374</td>\n",
       "      <td>-0.568421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.660751</td>\n",
       "      <td>0.675628</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  0.930595       NaN  0.693425\n",
       "b -1.452673  0.021665 -0.406850\n",
       "c  0.284763  0.236374 -0.568421\n",
       "d       NaN -0.660751  0.675628"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df2 = pd.DataFrame({'one' : pd.Series(np.random.randn(3), index=['a', 'b', 'c']),\n",
    "                       'two' : pd.Series(np.random.randn(4), index=['a', 'b', 'c', 'd']),\n",
    "                       'three' : pd.Series(np.random.randn(4), index=['a', 'b', 'c', 'd'])})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>-0.213464</td>\n",
       "      <td>0.984343</td>\n",
       "      <td>0.832706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-0.592199</td>\n",
       "      <td>-0.131885</td>\n",
       "      <td>-0.599882</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>-0.092918</td>\n",
       "      <td>-1.011125</td>\n",
       "      <td>-1.246759</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.658080</td>\n",
       "      <td>1.424326</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a -0.213464  0.984343  0.832706\n",
       "b -0.592199 -0.131885 -0.599882\n",
       "c -0.092918 -1.011125 -1.246759\n",
       "d       NaN -0.658080  1.424326"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.717132</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.526131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-2.044872</td>\n",
       "      <td>-0.110220</td>\n",
       "      <td>-1.006732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.191845</td>\n",
       "      <td>-0.774751</td>\n",
       "      <td>-1.815180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.318831</td>\n",
       "      <td>2.099954</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  0.717132       NaN  1.526131\n",
       "b -2.044872 -0.110220 -1.006732\n",
       "c  0.191845 -0.774751 -1.815180\n",
       "d       NaN -1.318831  2.099954"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df + df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.717132</td>\n",
       "      <td>0.984343</td>\n",
       "      <td>1.526131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-2.044872</td>\n",
       "      <td>-0.110220</td>\n",
       "      <td>-1.006732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.191845</td>\n",
       "      <td>-0.774751</td>\n",
       "      <td>-1.815180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.318831</td>\n",
       "      <td>2.099954</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  0.717132  0.984343  1.526131\n",
       "b -2.044872 -0.110220 -1.006732\n",
       "c  0.191845 -0.774751 -1.815180\n",
       "d       NaN -1.318831  2.099954"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    " df.add(df2, fill_value=0) #注意['a', 'three']不是NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 灵活的比较操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas引入了二元比较运算方法：eq, ne, lt, gt, le。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     one  three    two\n",
       "a   True  False  False\n",
       "b  False   True   True\n",
       "c   True   True   True\n",
       "d  False  False  False"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.gt(df2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    one three   two\n",
       "a  True  True  True\n",
       "b  True  True  True\n",
       "c  True  True  True\n",
       "d  True  True  True"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.ne(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "意思操作返回一个和输入对象同类型的对象，值类型为bool，返回结果可以用于检索。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 布尔降维 Boolean Reductions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas提供了三个方法(any(), all(), bool())和一个empty属性来对布尔结果进行降维。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     one  three    two\n",
       "a   True  False   True\n",
       "b  False   True  False\n",
       "c   True   True  False\n",
       "d  False  False   True"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df>0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one      False\n",
       "three    False\n",
       "two      False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df>0).all() #与操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one      True\n",
       "three    True\n",
       "two      True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df > 0).any()#或操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "同样可以对降维后的结果再进行降维。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df > 0).any().any()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "使用empty属性检测一个pandas对象是否为空。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.empty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(columns=list('ABC')).empty"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "对于只含有一个元素的pandas对象，对其进行布尔检测，使用bool():"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([True]).bool()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([False]).bool()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([[True]]).bool()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([[False]]).bool()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 比较对象是否相等"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "一个问题通常有多种解法。一个最简单的例子：df+df和df\\*2。为了检测两个计算结果是否相等，你可能想到：(df+df == df\\*2).all()，然而，这样计算得到的结果是False："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     one  three   two\n",
       "a   True  False  True\n",
       "b   True   True  True\n",
       "c   True   True  True\n",
       "d  False   True  True"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df + df == df*2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one      False\n",
       "three    False\n",
       "two       True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df+df == df*2).all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "为什么df + df == df\\*2 返回的结果含有False？因为NaN和NaN比较厚结果为False！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.nan == np.nan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "还好pandas提供了equals()方法解决上面NaN之间不想等的问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df+df).equals(df*2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**注意**：\n",
    "\n",
    "在使用equals()方法进行比较时，两个对象如果数据不一致必为False。甚至是如果label的顺序不一致也返回False。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     c\n",
       "0    f\n",
       "1    0\n",
       "2  NaN"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = pd.DataFrame({'c':['f',0,np.nan]})\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     c\n",
       "2  NaN\n",
       "1    0\n",
       "0    f"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.DataFrame({'c':[np.nan, 0, 'f']}, index=[2,1,0])\n",
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.equals(df2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.equals(df2.sort_index()) #对df2的索引排序，然后再比较"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 不同类型的对象之间 逐元素比较"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "你可以直接对pandas对象和一个常量值进行逐元素比较："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1    False\n",
       "2    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(['foo', 'bar', 'baz']) == 'foo'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True, False, False], dtype=bool)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Index(['foo', 'bar', 'baz']) == 'foo'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "不同类型的对象(比如pandas数据结构、numpy数组)之间进行逐元素的比较也是没有问题的，**前提是两个对象的shape要相同**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(['foo', 'bar', 'baz']) == pd.Index(['foo', 'bar', 'qux'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(['foo', 'bar', 'baz']) == np.array(['foo', 'bar', 'qux'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Can only compare identically-labeled Series objects",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-55-a98f3718cffe>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSeries\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'foo'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'bar'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'baz'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSeries\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'foo'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'bar'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m#长度不相同\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/usr/local/lib/python2.7/dist-packages/pandas/core/ops.pyc\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(self, other, axis)\u001b[0m\n\u001b[1;32m    810\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_indexed_same\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mother\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    811\u001b[0m                 \u001b[0mmsg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'Can only compare identically-labeled Series objects'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 812\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    813\u001b[0m             return self._constructor(na_op(self.values, other.values),\n\u001b[1;32m    814\u001b[0m                                      index=self.index, name=name)\n",
      "\u001b[0;31mValueError\u001b[0m: Can only compare identically-labeled Series objects"
     ]
    }
   ],
   "source": [
    "pd.Series(['foo', 'bar', 'baz']) == pd.Series(['foo', 'bar']) #长度不相同"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "但要知道不同shape的numpy数组之间是可以直接比较的！因为广播！即使无法广播，也不会Error而是返回False。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False,  True, False], dtype=bool)"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([1,2,3]) == np.array([2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/ipykernel/__main__.py:1: DeprecationWarning: elementwise == comparison failed; this will raise an error in the future.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([1, 2, 3]) == np.array([1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "###  combine_first()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "看一下例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B\n",
       "0  1.0  NaN\n",
       "1  NaN  2.0\n",
       "2  3.0  3.0\n",
       "3  5.0  NaN\n",
       "4  NaN  6.0"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = pd.DataFrame({'A' : [1., np.nan, 3., 5., np.nan],\n",
    "                    'B' : [np.nan, 2., 3., np.nan, 6.]})\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B\n",
       "0  5.0  NaN\n",
       "1  2.0  NaN\n",
       "2  4.0  3.0\n",
       "3  NaN  4.0\n",
       "4  3.0  6.0\n",
       "5  7.0  8.0"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.DataFrame({'A' : [5., 2., 4., np.nan, 3., 7.],\n",
    "                        'B' : [np.nan, np.nan, 3., 4., 6., 8.]})\n",
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B\n",
       "0  1.0  NaN\n",
       "1  2.0  2.0\n",
       "2  3.0  3.0\n",
       "3  5.0  4.0\n",
       "4  3.0  6.0\n",
       "5  7.0  8.0"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.combine_first(df2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "解释：\n",
    "\n",
    "对于df1中NaN的元素，用df2中对应位置的元素替换！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### DataFrame.combine()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "DataFrame.combine()方法接收一个DF对象和一个combiner方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combiner = lambda x,y: np.where(pd.isnull(x), y,x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B\n",
       "0  1.0  NaN\n",
       "1  2.0  2.0\n",
       "2  3.0  3.0\n",
       "3  5.0  4.0\n",
       "4  3.0  6.0\n",
       "5  7.0  8.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.combine(df2, combiner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 统计相关 的方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series, DataFrame和Panel内置了许多计算统计相关指标的方法。这些方法大致分为两类：\n",
    "* 返回低维结果，比如sum(),mean(),quantile()\n",
    "* 返回同原对象同样大小的对象，比如cumsum(), cumprod()\n",
    "\n",
    "总体来说，这些方法接收一个坐标轴参数:\n",
    "* Series不需要坐标轴参数\n",
    "* DataFrame 默认axis=0(index), axis=1(columns)\n",
    "* Panel 默认axis=1(major), axis=0(items), axis=2(minor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.930595</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.693425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-1.452673</td>\n",
       "      <td>0.021665</td>\n",
       "      <td>-0.406850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.284763</td>\n",
       "      <td>0.236374</td>\n",
       "      <td>-0.568421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.660751</td>\n",
       "      <td>0.675628</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  0.930595       NaN  0.693425\n",
       "b -1.452673  0.021665 -0.406850\n",
       "c  0.284763  0.236374 -0.568421\n",
       "d       NaN -0.660751  0.675628"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one     -0.079105\n",
       "three   -0.134238\n",
       "two      0.098445\n",
       "dtype: float64"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.mean() #axis=0, 计算每一列的平均值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    0.812010\n",
       "b   -0.612619\n",
       "c   -0.015761\n",
       "d    0.007438\n",
       "dtype: float64"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.mean(1) #计算每一行的平均值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "所有的这些方法都有skipna参数，含义是计算过程中是否剔除缺失值，skipna默认值为True。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one           NaN\n",
       "three         NaN\n",
       "two      0.393782\n",
       "dtype: float64"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sum(0, skipna=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    1.624021\n",
       "b   -1.837858\n",
       "c   -0.047284\n",
       "d    0.014876\n",
       "dtype: float64"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sum(axis=1, skipna=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**这些函数可以参与算术和广播运算。**\n",
    "\n",
    "比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ts_stand = (df-df.mean())/df.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "one      1.0\n",
       "three    1.0\n",
       "two      1.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts_stand.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "xs_stand = df.sub(df.mean(1), axis=0).div(df.std(1), axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    1.0\n",
       "b    1.0\n",
       "c    1.0\n",
       "d    1.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xs_stand.std(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "注意cumsum() cumprod()方法 保留NA值的位置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.930595</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.693425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-0.522077</td>\n",
       "      <td>0.021665</td>\n",
       "      <td>0.286575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>-0.237314</td>\n",
       "      <td>0.258039</td>\n",
       "      <td>-0.281846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.402713</td>\n",
       "      <td>0.393782</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a  0.930595       NaN  0.693425\n",
       "b -0.522077  0.021665  0.286575\n",
       "c -0.237314  0.258039 -0.281846\n",
       "d       NaN -0.402713  0.393782"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.cumsum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "下面列出常用的方法及其描述。提醒每一个方法都有一个level参数用于具有层次索引的对象。\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "|    方法     | 描述  | \n",
    "| ------------- |:-------------:| \n",
    "| count      | 沿着坐标轴统计非空的行数| \n",
    "| sum      | 沿着坐标轴取加和|   \n",
    "| mean | 沿着坐标轴求均值|    \n",
    "|mad|沿着坐标轴计算平均绝对偏差|\n",
    "|median|沿着坐标轴计算中位数|\n",
    "|min|沿着坐标轴取最小值|\n",
    "|max|沿着坐标轴取最大值|\n",
    "|mode|沿着坐标轴取众数|\n",
    "|abs|计算每一个值的绝对值|\n",
    "|prod|沿着坐标轴求乘积|\n",
    "|std|沿着坐标轴计算标准差|\n",
    "|var|沿着坐标轴计算无偏方差|\n",
    "|sem|沿着坐标轴计算标准差|\n",
    "|skew|沿着坐标轴计算样本偏斜|\n",
    "|kurt|沿着坐标轴计算样本峰度|\n",
    "|quantile|沿着坐标轴计算样本分位数，单位%|\n",
    "|cumsum|沿着坐标轴计算累加和|\n",
    "|cumprod|沿着坐标轴计算累积乘|\n",
    "|cummax|沿着坐标轴计算累计最大|\n",
    "|cummin|沿着坐标轴计算累计最小|\n",
    "\n",
    "\n",
    "**Note**：所有需要沿着坐标轴计算的方法，默认axis=0，即将方法应用到每一列数据上。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series还有一个nunique()方法返回非空数值 组成的集合的大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "series = pd.Series(np.random.randn(500))\n",
    "series[20:500]=np.nan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "series[10:20]=5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      0.466212\n",
       "1     -1.572517\n",
       "2     -1.789153\n",
       "3     -0.649339\n",
       "4      0.874092\n",
       "5     -0.591584\n",
       "6      0.596416\n",
       "7     -0.257149\n",
       "8      0.822657\n",
       "9     -0.340937\n",
       "10     5.000000\n",
       "11     5.000000\n",
       "12     5.000000\n",
       "13     5.000000\n",
       "14     5.000000\n",
       "15     5.000000\n",
       "16     5.000000\n",
       "17     5.000000\n",
       "18     5.000000\n",
       "19     5.000000\n",
       "20          NaN\n",
       "21          NaN\n",
       "22          NaN\n",
       "23          NaN\n",
       "24          NaN\n",
       "25          NaN\n",
       "26          NaN\n",
       "27          NaN\n",
       "28          NaN\n",
       "29          NaN\n",
       "         ...   \n",
       "470         NaN\n",
       "471         NaN\n",
       "472         NaN\n",
       "473         NaN\n",
       "474         NaN\n",
       "475         NaN\n",
       "476         NaN\n",
       "477         NaN\n",
       "478         NaN\n",
       "479         NaN\n",
       "480         NaN\n",
       "481         NaN\n",
       "482         NaN\n",
       "483         NaN\n",
       "484         NaN\n",
       "485         NaN\n",
       "486         NaN\n",
       "487         NaN\n",
       "488         NaN\n",
       "489         NaN\n",
       "490         NaN\n",
       "491         NaN\n",
       "492         NaN\n",
       "493         NaN\n",
       "494         NaN\n",
       "495         NaN\n",
       "496         NaN\n",
       "497         NaN\n",
       "498         NaN\n",
       "499         NaN\n",
       "dtype: float64"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### descrieb()， 数据摘要"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "describe()方法非常有用，它计算数据的各种常用统计指标(比如平均值、标准差等)，计算时不包括NA。拿到数据首先要有大概的了解，使用describe()方法就对了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "series = pd.Series(np.random.randn(1000))\n",
    "series[::2]=np.nan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    500.000000\n",
       "mean       0.008860\n",
       "std        0.955782\n",
       "min       -2.968920\n",
       "25%       -0.686382\n",
       "50%        0.018842\n",
       "75%        0.635701\n",
       "max        2.574386\n",
       "dtype: float64"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "frame = pd.DataFrame(np.random.randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "frame.ix[::2]=np.nan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>500.000000</td>\n",
       "      <td>500.000000</td>\n",
       "      <td>500.000000</td>\n",
       "      <td>500.000000</td>\n",
       "      <td>500.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.005870</td>\n",
       "      <td>-0.056518</td>\n",
       "      <td>0.007845</td>\n",
       "      <td>0.045578</td>\n",
       "      <td>-0.001946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.984246</td>\n",
       "      <td>1.045085</td>\n",
       "      <td>1.012088</td>\n",
       "      <td>1.038450</td>\n",
       "      <td>1.048679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-2.752012</td>\n",
       "      <td>-3.414155</td>\n",
       "      <td>-2.518559</td>\n",
       "      <td>-2.884816</td>\n",
       "      <td>-2.871252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-0.645610</td>\n",
       "      <td>-0.769178</td>\n",
       "      <td>-0.730541</td>\n",
       "      <td>-0.630003</td>\n",
       "      <td>-0.712117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-0.008707</td>\n",
       "      <td>-0.077536</td>\n",
       "      <td>-0.009839</td>\n",
       "      <td>0.013494</td>\n",
       "      <td>0.031158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.631591</td>\n",
       "      <td>0.631172</td>\n",
       "      <td>0.675323</td>\n",
       "      <td>0.762697</td>\n",
       "      <td>0.688557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.689364</td>\n",
       "      <td>2.846510</td>\n",
       "      <td>3.668186</td>\n",
       "      <td>3.027345</td>\n",
       "      <td>2.877428</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                a           b           c           d           e\n",
       "count  500.000000  500.000000  500.000000  500.000000  500.000000\n",
       "mean     0.005870   -0.056518    0.007845    0.045578   -0.001946\n",
       "std      0.984246    1.045085    1.012088    1.038450    1.048679\n",
       "min     -2.752012   -3.414155   -2.518559   -2.884816   -2.871252\n",
       "25%     -0.645610   -0.769178   -0.730541   -0.630003   -0.712117\n",
       "50%     -0.008707   -0.077536   -0.009839    0.013494    0.031158\n",
       "75%      0.631591    0.631172    0.675323    0.762697    0.688557\n",
       "max      2.689364    2.846510    3.668186    3.027345    2.877428"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frame.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "默认describe()只包含25%, 50%, 75%, 也可以通过percentiles参数进行指定。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    500.000000\n",
       "mean       0.008860\n",
       "std        0.955782\n",
       "min       -2.968920\n",
       "5%        -1.508605\n",
       "25%       -0.686382\n",
       "50%        0.018842\n",
       "75%        0.635701\n",
       "95%        1.658959\n",
       "max        2.574386\n",
       "dtype: float64"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series.describe(percentiles=[.05, .25, .75, .95])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果Series内数据是非数值类型，describe()也能给出一定的统计结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(['a', 'a', 'b', 'b', 'a', 'a', np.nan, 'c', 'd', 'a'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     9\n",
       "unique    4\n",
       "top       a\n",
       "freq      5\n",
       "dtype: object"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果DataFrame对象有的列是数值类型，有的列不是数值类型，describe()仅对数值类型的列进行计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "frame = pd.DataFrame({'a':['Yes', 'Yes', 'NO', 'No'], 'b':range(4)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.290994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              b\n",
       "count  4.000000\n",
       "mean   1.500000\n",
       "std    1.290994\n",
       "min    0.000000\n",
       "25%    0.750000\n",
       "50%    1.500000\n",
       "75%    2.250000\n",
       "max    3.000000"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frame.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果非要知道非数值列的统计指标呢？describe提供了include参数，取值范围{'object', 'number', 'all'}。\n",
    "\n",
    "看一下例子, 注意'object'和'number'都是在列表中，而'all'不需要放在列表中："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          a\n",
       "count     4\n",
       "unique    3\n",
       "top     Yes\n",
       "freq      2"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frame.describe(include=['object']) #只对非数值列进行统计计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.290994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              b\n",
       "count  4.000000\n",
       "mean   1.500000\n",
       "std    1.290994\n",
       "min    0.000000\n",
       "25%    0.750000\n",
       "50%    1.500000\n",
       "75%    2.250000\n",
       "max    3.000000"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frame.describe(include=['number'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>Yes</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1.290994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>NaN</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          a         b\n",
       "count     4  4.000000\n",
       "unique    3       NaN\n",
       "top     Yes       NaN\n",
       "freq      2       NaN\n",
       "mean    NaN  1.500000\n",
       "std     NaN  1.290994\n",
       "min     NaN  0.000000\n",
       "25%     NaN  0.750000\n",
       "50%     NaN  1.500000\n",
       "75%     NaN  2.250000\n",
       "max     NaN  3.000000"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frame.describe(include='all')#'all'不是列表"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 最大/最小值对应的index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "idxmin(), idxmax()返回最小值和最大值对应的index。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.366003\n",
       "1   -0.169499\n",
       "2   -0.624169\n",
       "3   -0.307402\n",
       "4   -0.599977\n",
       "dtype: float64"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1 = pd.Series(np.random.randn(5))\n",
    "s1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 0)"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1.idxmin(), s1.idxmax() #最小值：-0.296405, 最大值：1.735420"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.996110</td>\n",
       "      <td>1.576533</td>\n",
       "      <td>-0.950779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.140296</td>\n",
       "      <td>0.499200</td>\n",
       "      <td>1.119654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1.187044</td>\n",
       "      <td>0.449324</td>\n",
       "      <td>0.830870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.661936</td>\n",
       "      <td>-0.241894</td>\n",
       "      <td>0.086323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.748093</td>\n",
       "      <td>-0.496080</td>\n",
       "      <td>0.972968</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A         B         C\n",
       "0 -0.996110  1.576533 -0.950779\n",
       "1  0.140296  0.499200  1.119654\n",
       "2 -1.187044  0.449324  0.830870\n",
       "3  1.661936 -0.241894  0.086323\n",
       "4 -1.748093 -0.496080  0.972968"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = pd.DataFrame(np.random.randn(5,3), columns=list('ABC'))\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    4\n",
       "B    4\n",
       "C    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.idxmin(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    B\n",
       "1    C\n",
       "2    C\n",
       "3    A\n",
       "4    C\n",
       "dtype: object"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.idxmax(axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果多个数值都是最大值或最小值，idxmax() idxmin()返回最大值、最小值第一次出现对应的索引值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>e</th>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A\n",
       "e  2.0\n",
       "d  1.0\n",
       "c  1.0\n",
       "b  3.0\n",
       "a  NaN"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3 = pd.DataFrame([2, 1, 1, 3, np.nan], columns=['A'], index=list('edcba'))\n",
    "df3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'d'"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3['A'].idxmin()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**实际上，idxmin和idxmax就是NumPy中的argmin和argmax**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "###   value_counts() 数值计数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "value_counts()计算一维度数据结构的直方图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 5, 4, 6, 0, 5, 6, 6, 2, 0, 6, 5, 6, 6, 3, 2, 1, 3, 3, 4, 3, 6, 5,\n",
       "       4, 3, 1, 3, 1, 0, 5, 2, 1, 2, 4, 4, 4, 1, 2, 4, 0, 6, 2, 5, 5, 2, 4,\n",
       "       4, 3, 5, 3])"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = np.random.randint(0, 7, size=50)\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4    9\n",
       "6    8\n",
       "5    8\n",
       "3    8\n",
       "2    7\n",
       "1    6\n",
       "0    4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4    9\n",
       "6    8\n",
       "5    8\n",
       "3    8\n",
       "2    7\n",
       "1    6\n",
       "0    4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.value_counts(data) #也是全局方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "虽然前面介绍过mode()方法了，看两个例子吧："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s5 = pd.Series([1,1,3,3,3,5,5,7,7,7])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3\n",
       "1    7\n",
       "dtype: int64"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s5.mode()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df5 = pd.DataFrame({\"A\": np.random.randint(0, 7, size=50),\n",
    "                    \"B\": np.random.randint(-10, 15, size=50)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "      <td>-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>-6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>-4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>-5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>5</td>\n",
       "      <td>-3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0</td>\n",
       "      <td>-7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>2</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>6</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>6</td>\n",
       "      <td>-5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>5</td>\n",
       "      <td>-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>4</td>\n",
       "      <td>-3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>5</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>1</td>\n",
       "      <td>-5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>2</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>6</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>1</td>\n",
       "      <td>-4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>6</td>\n",
       "      <td>-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>5</td>\n",
       "      <td>-5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    A   B\n",
       "0   5  -9\n",
       "1   4  -6\n",
       "2   2  11\n",
       "3   3  11\n",
       "4   4   6\n",
       "5   0   3\n",
       "6   5   2\n",
       "7   5   4\n",
       "8   3  -4\n",
       "9   2  -5\n",
       "10  3  -9\n",
       "11  4   8\n",
       "12  5  -3\n",
       "13  0  -7\n",
       "14  4   3\n",
       "15  0  11\n",
       "16  3   1\n",
       "17  4   8\n",
       "18  0   8\n",
       "19  6   8\n",
       "20  1  11\n",
       "21  2  13\n",
       "22  6  10\n",
       "23  6  -5\n",
       "24  1   1\n",
       "25  5 -10\n",
       "26  4  -3\n",
       "27  4   7\n",
       "28  4   0\n",
       "29  2   7\n",
       "30  5  13\n",
       "31  1  -5\n",
       "32  6   7\n",
       "33  2  -1\n",
       "34  1  14\n",
       "35  1   7\n",
       "36  3   4\n",
       "37  2   5\n",
       "38  6  14\n",
       "39  4   2\n",
       "40  4   4\n",
       "41  5  12\n",
       "42  2   5\n",
       "43  4   2\n",
       "44  6   5\n",
       "45  6   2\n",
       "46  1  -4\n",
       "47  0   7\n",
       "48  6  -8\n",
       "49  5  -5"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   A  B\n",
       "0  4  7"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df5.mode()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 区间离散化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "cut() qcut()方法可以对连续数据进行离散化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "arr = np.random.randn(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "factor = pd.cut(arr, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(-0.597, 0.476], (-1.674, -0.597], (-1.674, -0.597], (-1.674, -0.597], (1.548, 2.621], ..., (-0.597, 0.476], (0.476, 1.548], (-0.597, 0.476], (-1.674, -0.597], (-1.674, -0.597]]\n",
       "Length: 20\n",
       "Categories (4, object): [(-1.674, -0.597] < (-0.597, 0.476] < (0.476, 1.548] < (1.548, 2.621]]"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "factor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "factor = pd.cut(arr, [-5, -1, 0, 1, 5]) #输入 离散区间"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(-1, 0], (-5, -1], (-5, -1], (-1, 0], (1, 5], ..., (-1, 0], (0, 1], (-1, 0], (-1, 0], (-1, 0]]\n",
       "Length: 20\n",
       "Categories (4, object): [(-5, -1] < (-1, 0] < (0, 1] < (1, 5]]"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "factor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "qcut()方法计算样本的分位数，比如我们可以将正态分布的数据 进行四分位数离散化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "arr = np.random.randn(30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "factor = pd.qcut(arr, [0, .25, .5, .75, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0.216, 0.851], [-2.228, -0.68], (0.851, 2.636], [-2.228, -0.68], (-0.68, 0.216], ..., (-0.68, 0.216], (0.216, 0.851], (0.216, 0.851], (-0.68, 0.216], [-2.228, -0.68]]\n",
       "Length: 30\n",
       "Categories (4, object): [[-2.228, -0.68] < (-0.68, 0.216] < (0.216, 0.851] < (0.851, 2.636]]"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "factor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.851, 2.636]     8\n",
       "[-2.228, -0.68]    8\n",
       "(0.216, 0.851]     7\n",
       "(-0.68, 0.216]     7\n",
       "dtype: int64"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.value_counts(factor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "离散区间也可以用极限定义"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "arr = np.random.randn(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "factor = pd.cut(arr, [-np.inf, 0, np.inf])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, inf], (-inf, 0], (-inf, 0], (-inf, 0], (0, inf], ..., (-inf, 0], (0, inf], (0, inf], (0, inf], (0, inf]]\n",
       "Length: 20\n",
       "Categories (2, object): [(-inf, 0] < (0, inf]]"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "factor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 函数应用 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果你想用自己写的方法或其他库方法操作pandas对象，你应该知道下面的三种方式。 具体选择哪种方式取决于你是想操作整个DataFrame对象还是DataFrame对象的某几行或某几列，或者逐元素操作。\n",
    "\n",
    "* **管道 pipe()**\n",
    "* **基于列或行的函数应用 apply()**\n",
    "* **对DataFrame对象逐元素计算 applymap()**\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 管道"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "DataFrame和Series当然能够作为参数传入方法。然而，如果涉及到多个方法的序列调用，推荐使用pipe()。看一下例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "#f, g 和h是三个方法，接收DataFrame对象，返回DataFrame对象\n",
    "f(g(h(df), arg1=1), arg2=2, arg3=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "上面一行代码推荐用下面的等价写法:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "(df.pipe(h).pipe(g, arg1=1).pipe(f, arg2=2, arg3=3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "注意 f g h三个方法中DataFrame都是作为第一个参数。如果DataFrame作为第二个参数呢？方法是为pipe提供(callable, data_keyword)，pipe会自动调用DataFrame对象。\n",
    "\n",
    "比如，使用statsmodels处理回归问题，他们的API期望第一个参数是公式，第二个参数是DataFrame对象data。我们使用pipe传递(sm.poisson, 'data'):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "![](https://ooo.0o0.ooo/2016/04/16/5712057590c85.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pipe灵感来自于Unix中伟大的艺术：管道。pandas中pipe()的实现很简洁，推荐阅读源代码pd.DataFrame.pipe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 基于行或者列的函数应用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "任意函数都可以直接对DataFrame或Panel某一坐标轴进行直接操纵，只需要使用apply()方法即可，同描述性统计方法一样，apply()方法接收axis参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A   -0.082000\n",
       "B   -0.039589\n",
       "C   -0.226022\n",
       "dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(np.mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-01    0.031020\n",
       "2000-01-02    0.754696\n",
       "2000-01-03   -0.861633\n",
       "2000-01-04    0.707869\n",
       "2000-01-05   -0.191752\n",
       "2000-01-06    0.329149\n",
       "2000-01-07   -1.213012\n",
       "2000-01-08   -0.483304\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(np.mean, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    1.991008\n",
       "B    3.883607\n",
       "C    3.555682\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(lambda x: x.max() - x.min())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.142261</td>\n",
       "      <td>-0.060360</td>\n",
       "      <td>0.295683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.460755</td>\n",
       "      <td>2.086278</td>\n",
       "      <td>-0.189882</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-0.532777</td>\n",
       "      <td>0.994217</td>\n",
       "      <td>-0.689186</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.464698</td>\n",
       "      <td>2.055012</td>\n",
       "      <td>-0.623848</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>0.784671</td>\n",
       "      <td>1.499974</td>\n",
       "      <td>-0.964040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>0.516696</td>\n",
       "      <td>0.778774</td>\n",
       "      <td>1.012581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>0.193691</td>\n",
       "      <td>-0.958194</td>\n",
       "      <td>-0.566480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-0.656002</td>\n",
       "      <td>-0.316716</td>\n",
       "      <td>-1.808178</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.142261 -0.060360  0.295683\n",
       "2000-01-02  0.460755  2.086278 -0.189882\n",
       "2000-01-03 -0.532777  0.994217 -0.689186\n",
       "2000-01-04  0.464698  2.055012 -0.623848\n",
       "2000-01-05  0.784671  1.499974 -0.964040\n",
       "2000-01-06  0.516696  0.778774  1.012581\n",
       "2000-01-07  0.193691 -0.958194 -0.566480\n",
       "2000-01-08 -0.656002 -0.316716 -1.808178"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(np.cumsum)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>0.867394</td>\n",
       "      <td>0.941425</td>\n",
       "      <td>1.344044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>1.827624</td>\n",
       "      <td>8.556042</td>\n",
       "      <td>0.615349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>0.370266</td>\n",
       "      <td>0.335524</td>\n",
       "      <td>0.606953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>2.711427</td>\n",
       "      <td>2.888666</td>\n",
       "      <td>1.067520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>1.377091</td>\n",
       "      <td>0.574050</td>\n",
       "      <td>0.711634</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>0.764927</td>\n",
       "      <td>0.486169</td>\n",
       "      <td>7.218311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>0.723970</td>\n",
       "      <td>0.176053</td>\n",
       "      <td>0.206169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>0.427546</td>\n",
       "      <td>1.899287</td>\n",
       "      <td>0.288893</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01  0.867394  0.941425  1.344044\n",
       "2000-01-02  1.827624  8.556042  0.615349\n",
       "2000-01-03  0.370266  0.335524  0.606953\n",
       "2000-01-04  2.711427  2.888666  1.067520\n",
       "2000-01-05  1.377091  0.574050  0.711634\n",
       "2000-01-06  0.764927  0.486169  7.218311\n",
       "2000-01-07  0.723970  0.176053  0.206169\n",
       "2000-01-08  0.427546  1.899287  0.288893"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(np.exp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "灵活运用apply()方法可以统计出数据集的很多特性。比如，假设我们希望从数据中抽取每一列最大值的索引值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>1.508512</td>\n",
       "      <td>-0.337924</td>\n",
       "      <td>-0.322681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.546119</td>\n",
       "      <td>-0.569760</td>\n",
       "      <td>-0.475934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>0.479077</td>\n",
       "      <td>0.141438</td>\n",
       "      <td>-1.540489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>-0.829857</td>\n",
       "      <td>-0.823538</td>\n",
       "      <td>1.643723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>0.255760</td>\n",
       "      <td>1.210986</td>\n",
       "      <td>0.931286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>2.070021</td>\n",
       "      <td>1.276612</td>\n",
       "      <td>-0.115412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>-0.473810</td>\n",
       "      <td>0.291923</td>\n",
       "      <td>-1.695630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>0.602344</td>\n",
       "      <td>-1.268760</td>\n",
       "      <td>0.181028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-09</th>\n",
       "      <td>1.004540</td>\n",
       "      <td>-0.616671</td>\n",
       "      <td>-0.362409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-10</th>\n",
       "      <td>0.485239</td>\n",
       "      <td>0.971017</td>\n",
       "      <td>-0.087146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-11</th>\n",
       "      <td>0.133297</td>\n",
       "      <td>-0.348421</td>\n",
       "      <td>-1.032877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-12</th>\n",
       "      <td>-0.940945</td>\n",
       "      <td>-0.507634</td>\n",
       "      <td>0.575378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-13</th>\n",
       "      <td>0.606164</td>\n",
       "      <td>-0.641187</td>\n",
       "      <td>-1.049049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-14</th>\n",
       "      <td>0.646394</td>\n",
       "      <td>-0.504994</td>\n",
       "      <td>-0.112327</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-15</th>\n",
       "      <td>-0.347899</td>\n",
       "      <td>0.132691</td>\n",
       "      <td>0.400609</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-16</th>\n",
       "      <td>0.621119</td>\n",
       "      <td>-0.004572</td>\n",
       "      <td>-0.340887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-17</th>\n",
       "      <td>-0.736580</td>\n",
       "      <td>0.399046</td>\n",
       "      <td>0.088845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-18</th>\n",
       "      <td>-0.912036</td>\n",
       "      <td>0.840877</td>\n",
       "      <td>-0.074514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-19</th>\n",
       "      <td>1.287786</td>\n",
       "      <td>0.779607</td>\n",
       "      <td>-0.067636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-20</th>\n",
       "      <td>-0.599940</td>\n",
       "      <td>0.094742</td>\n",
       "      <td>-0.163426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-21</th>\n",
       "      <td>0.935834</td>\n",
       "      <td>-0.144410</td>\n",
       "      <td>-0.085487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-22</th>\n",
       "      <td>1.226581</td>\n",
       "      <td>-1.117103</td>\n",
       "      <td>0.246942</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-23</th>\n",
       "      <td>1.530435</td>\n",
       "      <td>-1.438474</td>\n",
       "      <td>0.542377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-24</th>\n",
       "      <td>-0.838566</td>\n",
       "      <td>-0.324698</td>\n",
       "      <td>1.184812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-25</th>\n",
       "      <td>-0.311246</td>\n",
       "      <td>-1.377314</td>\n",
       "      <td>-0.431623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-26</th>\n",
       "      <td>0.801814</td>\n",
       "      <td>0.438467</td>\n",
       "      <td>-1.915434</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-27</th>\n",
       "      <td>0.815048</td>\n",
       "      <td>-1.009805</td>\n",
       "      <td>0.501989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-28</th>\n",
       "      <td>0.897600</td>\n",
       "      <td>0.628330</td>\n",
       "      <td>0.475561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-29</th>\n",
       "      <td>0.696502</td>\n",
       "      <td>-0.822199</td>\n",
       "      <td>-0.448787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-30</th>\n",
       "      <td>0.471048</td>\n",
       "      <td>-1.841923</td>\n",
       "      <td>0.158116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-08-28</th>\n",
       "      <td>-0.184911</td>\n",
       "      <td>1.282684</td>\n",
       "      <td>0.951569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-08-29</th>\n",
       "      <td>-0.656969</td>\n",
       "      <td>-0.103318</td>\n",
       "      <td>-0.464619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-08-30</th>\n",
       "      <td>-1.051274</td>\n",
       "      <td>0.722578</td>\n",
       "      <td>-0.795858</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-08-31</th>\n",
       "      <td>1.869913</td>\n",
       "      <td>-2.879455</td>\n",
       "      <td>-0.092262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-01</th>\n",
       "      <td>-1.186259</td>\n",
       "      <td>0.897260</td>\n",
       "      <td>-0.770680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-02</th>\n",
       "      <td>0.898835</td>\n",
       "      <td>-0.175212</td>\n",
       "      <td>-0.485235</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-03</th>\n",
       "      <td>-1.657890</td>\n",
       "      <td>0.588649</td>\n",
       "      <td>0.802933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-04</th>\n",
       "      <td>-0.184906</td>\n",
       "      <td>-0.596059</td>\n",
       "      <td>0.759456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-05</th>\n",
       "      <td>1.069876</td>\n",
       "      <td>-1.375438</td>\n",
       "      <td>0.821087</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-06</th>\n",
       "      <td>-0.721034</td>\n",
       "      <td>-0.424185</td>\n",
       "      <td>0.150196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-07</th>\n",
       "      <td>0.048521</td>\n",
       "      <td>0.938075</td>\n",
       "      <td>0.420614</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-08</th>\n",
       "      <td>1.710716</td>\n",
       "      <td>-0.988647</td>\n",
       "      <td>-1.014892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-09</th>\n",
       "      <td>0.051754</td>\n",
       "      <td>0.758752</td>\n",
       "      <td>0.068914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-10</th>\n",
       "      <td>-0.404318</td>\n",
       "      <td>-1.102603</td>\n",
       "      <td>-0.641363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-11</th>\n",
       "      <td>-0.711666</td>\n",
       "      <td>-0.850081</td>\n",
       "      <td>1.223433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-12</th>\n",
       "      <td>0.105811</td>\n",
       "      <td>0.540562</td>\n",
       "      <td>-0.662475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-13</th>\n",
       "      <td>-0.182622</td>\n",
       "      <td>-0.815966</td>\n",
       "      <td>-1.001960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-14</th>\n",
       "      <td>-0.831867</td>\n",
       "      <td>-0.664711</td>\n",
       "      <td>-1.958620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-15</th>\n",
       "      <td>-0.178157</td>\n",
       "      <td>-0.865264</td>\n",
       "      <td>0.403366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-16</th>\n",
       "      <td>0.775322</td>\n",
       "      <td>-0.339516</td>\n",
       "      <td>0.419001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-17</th>\n",
       "      <td>1.363952</td>\n",
       "      <td>1.164906</td>\n",
       "      <td>0.081826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-18</th>\n",
       "      <td>0.072301</td>\n",
       "      <td>-0.449992</td>\n",
       "      <td>1.272230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-19</th>\n",
       "      <td>-0.310564</td>\n",
       "      <td>0.330637</td>\n",
       "      <td>0.183283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-20</th>\n",
       "      <td>-0.884466</td>\n",
       "      <td>-1.538738</td>\n",
       "      <td>-1.199347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-21</th>\n",
       "      <td>-1.368203</td>\n",
       "      <td>-0.300517</td>\n",
       "      <td>-2.301899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-22</th>\n",
       "      <td>-0.751779</td>\n",
       "      <td>0.905494</td>\n",
       "      <td>-1.756754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-23</th>\n",
       "      <td>-0.026914</td>\n",
       "      <td>-0.329971</td>\n",
       "      <td>-1.294168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-24</th>\n",
       "      <td>0.511469</td>\n",
       "      <td>0.101785</td>\n",
       "      <td>0.001063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-25</th>\n",
       "      <td>-0.498993</td>\n",
       "      <td>-0.328754</td>\n",
       "      <td>0.301644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002-09-26</th>\n",
       "      <td>0.564092</td>\n",
       "      <td>0.466081</td>\n",
       "      <td>-0.599783</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01  1.508512 -0.337924 -0.322681\n",
       "2000-01-02  0.546119 -0.569760 -0.475934\n",
       "2000-01-03  0.479077  0.141438 -1.540489\n",
       "2000-01-04 -0.829857 -0.823538  1.643723\n",
       "2000-01-05  0.255760  1.210986  0.931286\n",
       "2000-01-06  2.070021  1.276612 -0.115412\n",
       "2000-01-07 -0.473810  0.291923 -1.695630\n",
       "2000-01-08  0.602344 -1.268760  0.181028\n",
       "2000-01-09  1.004540 -0.616671 -0.362409\n",
       "2000-01-10  0.485239  0.971017 -0.087146\n",
       "2000-01-11  0.133297 -0.348421 -1.032877\n",
       "2000-01-12 -0.940945 -0.507634  0.575378\n",
       "2000-01-13  0.606164 -0.641187 -1.049049\n",
       "2000-01-14  0.646394 -0.504994 -0.112327\n",
       "2000-01-15 -0.347899  0.132691  0.400609\n",
       "2000-01-16  0.621119 -0.004572 -0.340887\n",
       "2000-01-17 -0.736580  0.399046  0.088845\n",
       "2000-01-18 -0.912036  0.840877 -0.074514\n",
       "2000-01-19  1.287786  0.779607 -0.067636\n",
       "2000-01-20 -0.599940  0.094742 -0.163426\n",
       "2000-01-21  0.935834 -0.144410 -0.085487\n",
       "2000-01-22  1.226581 -1.117103  0.246942\n",
       "2000-01-23  1.530435 -1.438474  0.542377\n",
       "2000-01-24 -0.838566 -0.324698  1.184812\n",
       "2000-01-25 -0.311246 -1.377314 -0.431623\n",
       "2000-01-26  0.801814  0.438467 -1.915434\n",
       "2000-01-27  0.815048 -1.009805  0.501989\n",
       "2000-01-28  0.897600  0.628330  0.475561\n",
       "2000-01-29  0.696502 -0.822199 -0.448787\n",
       "2000-01-30  0.471048 -1.841923  0.158116\n",
       "...              ...       ...       ...\n",
       "2002-08-28 -0.184911  1.282684  0.951569\n",
       "2002-08-29 -0.656969 -0.103318 -0.464619\n",
       "2002-08-30 -1.051274  0.722578 -0.795858\n",
       "2002-08-31  1.869913 -2.879455 -0.092262\n",
       "2002-09-01 -1.186259  0.897260 -0.770680\n",
       "2002-09-02  0.898835 -0.175212 -0.485235\n",
       "2002-09-03 -1.657890  0.588649  0.802933\n",
       "2002-09-04 -0.184906 -0.596059  0.759456\n",
       "2002-09-05  1.069876 -1.375438  0.821087\n",
       "2002-09-06 -0.721034 -0.424185  0.150196\n",
       "2002-09-07  0.048521  0.938075  0.420614\n",
       "2002-09-08  1.710716 -0.988647 -1.014892\n",
       "2002-09-09  0.051754  0.758752  0.068914\n",
       "2002-09-10 -0.404318 -1.102603 -0.641363\n",
       "2002-09-11 -0.711666 -0.850081  1.223433\n",
       "2002-09-12  0.105811  0.540562 -0.662475\n",
       "2002-09-13 -0.182622 -0.815966 -1.001960\n",
       "2002-09-14 -0.831867 -0.664711 -1.958620\n",
       "2002-09-15 -0.178157 -0.865264  0.403366\n",
       "2002-09-16  0.775322 -0.339516  0.419001\n",
       "2002-09-17  1.363952  1.164906  0.081826\n",
       "2002-09-18  0.072301 -0.449992  1.272230\n",
       "2002-09-19 -0.310564  0.330637  0.183283\n",
       "2002-09-20 -0.884466 -1.538738 -1.199347\n",
       "2002-09-21 -1.368203 -0.300517 -2.301899\n",
       "2002-09-22 -0.751779  0.905494 -1.756754\n",
       "2002-09-23 -0.026914 -0.329971 -1.294168\n",
       "2002-09-24  0.511469  0.101785  0.001063\n",
       "2002-09-25 -0.498993 -0.328754  0.301644\n",
       "2002-09-26  0.564092  0.466081 -0.599783\n",
       "\n",
       "[1000 rows x 3 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tsdf = pd.DataFrame(np.random.randn(1000, 3), columns=['A', 'B', 'C'],\n",
    "                   index=pd.date_range('1/1/2000', periods=1000))\n",
    "tsdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A   2001-07-13\n",
       "B   2002-05-25\n",
       "C   2001-11-26\n",
       "dtype: datetime64[ns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tsdf.apply(lambda x:x.idxmax())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "apply()方法当然支持接收其他参数了，比如下面的例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def subtract_and_divide(x, sub, divide=1):\n",
    "    return (x - sub)/divide"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-1.714087</td>\n",
       "      <td>-1.686787</td>\n",
       "      <td>-1.568106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>-1.465661</td>\n",
       "      <td>-0.951121</td>\n",
       "      <td>-1.828522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-1.997844</td>\n",
       "      <td>-2.030687</td>\n",
       "      <td>-1.833101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>-1.334175</td>\n",
       "      <td>-1.313068</td>\n",
       "      <td>-1.644887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>-1.560009</td>\n",
       "      <td>-1.851679</td>\n",
       "      <td>-1.780064</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>-1.755992</td>\n",
       "      <td>-1.907066</td>\n",
       "      <td>-1.007793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>-1.774335</td>\n",
       "      <td>-2.245656</td>\n",
       "      <td>-2.193020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-1.949898</td>\n",
       "      <td>-1.452840</td>\n",
       "      <td>-2.080566</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -1.714087 -1.686787 -1.568106\n",
       "2000-01-02 -1.465661 -0.951121 -1.828522\n",
       "2000-01-03 -1.997844 -2.030687 -1.833101\n",
       "2000-01-04 -1.334175 -1.313068 -1.644887\n",
       "2000-01-05 -1.560009 -1.851679 -1.780064\n",
       "2000-01-06 -1.755992 -1.907066 -1.007793\n",
       "2000-01-07 -1.774335 -2.245656 -2.193020\n",
       "2000-01-08 -1.949898 -1.452840 -2.080566"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(subtract_and_divide, args=(5,),divide=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "另一个有用的特性是对DataFrame对象传递Series方法，然后针对DF对象的每一列或每一行执行 Series内置的方法！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.142261</td>\n",
       "      <td>-0.060360</td>\n",
       "      <td>0.295683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.603017</td>\n",
       "      <td>2.146638</td>\n",
       "      <td>-0.485565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-0.993533</td>\n",
       "      <td>-1.092061</td>\n",
       "      <td>-0.499304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.997475</td>\n",
       "      <td>1.060795</td>\n",
       "      <td>0.065338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>0.319973</td>\n",
       "      <td>-0.555038</td>\n",
       "      <td>-0.340192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>-0.267975</td>\n",
       "      <td>-0.721199</td>\n",
       "      <td>1.976621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>-0.323005</td>\n",
       "      <td>-1.736969</td>\n",
       "      <td>-1.579061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-0.849693</td>\n",
       "      <td>0.641479</td>\n",
       "      <td>-1.241698</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.142261 -0.060360  0.295683\n",
       "2000-01-02  0.603017  2.146638 -0.485565\n",
       "2000-01-03 -0.993533 -1.092061 -0.499304\n",
       "2000-01-04  0.997475  1.060795  0.065338\n",
       "2000-01-05  0.319973 -0.555038 -0.340192\n",
       "2000-01-06 -0.267975 -0.721199  1.976621\n",
       "2000-01-07 -0.323005 -1.736969 -1.579061\n",
       "2000-01-08 -0.849693  0.641479 -1.241698"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.142261</td>\n",
       "      <td>-0.060360</td>\n",
       "      <td>0.295683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.603017</td>\n",
       "      <td>2.146638</td>\n",
       "      <td>-0.485565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-0.993533</td>\n",
       "      <td>-1.092061</td>\n",
       "      <td>-0.499304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.997475</td>\n",
       "      <td>1.060795</td>\n",
       "      <td>0.065338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>0.319973</td>\n",
       "      <td>-0.555038</td>\n",
       "      <td>-0.340192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>-0.267975</td>\n",
       "      <td>-0.721199</td>\n",
       "      <td>1.976621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>-0.323005</td>\n",
       "      <td>-1.736969</td>\n",
       "      <td>-1.579061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-0.849693</td>\n",
       "      <td>0.641479</td>\n",
       "      <td>-1.241698</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.142261 -0.060360  0.295683\n",
       "2000-01-02  0.603017  2.146638 -0.485565\n",
       "2000-01-03 -0.993533 -1.092061 -0.499304\n",
       "2000-01-04  0.997475  1.060795  0.065338\n",
       "2000-01-05  0.319973 -0.555038 -0.340192\n",
       "2000-01-06 -0.267975 -0.721199  1.976621\n",
       "2000-01-07 -0.323005 -1.736969 -1.579061\n",
       "2000-01-08 -0.849693  0.641479 -1.241698"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.apply(pd.Series.interpolate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 对单个元素应用Python方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "既然不是所有的方法都能被向量化(接收NumPy数组，返回另一个数组或者值),但是DataFrame内置的applymap()和Series的map()方法能够接收任意的接收一个值且返回一个值的Python方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>two</th>\n",
       "      <th>three</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>-0.539083</td>\n",
       "      <td>-2.681732</td>\n",
       "      <td>-0.183946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-0.824301</td>\n",
       "      <td>1.172048</td>\n",
       "      <td>0.387141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>-0.251208</td>\n",
       "      <td>-1.129691</td>\n",
       "      <td>0.937757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>0.554184</td>\n",
       "      <td>0.016947</td>\n",
       "      <td>0.755570</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one       two     three\n",
       "a -0.539083 -2.681732 -0.183946\n",
       "b -0.824301  1.172048  0.387141\n",
       "c -0.251208 -1.129691  0.937757\n",
       "d  0.554184  0.016947  0.755570"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4 = pd.DataFrame(np.random.randn(4, 3),index=['a','b','c','d'],columns=['one', 'two', 'three'])\n",
    "df4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "f = lambda x:len(str(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    15\n",
       "b    15\n",
       "c    15\n",
       "d    14\n",
       "Name: one, dtype: int64"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4['one'].map(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>two</th>\n",
       "      <th>three</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>15</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   one  two  three\n",
       "a   15   14     15\n",
       "b   15   13     14\n",
       "c   15   14     14\n",
       "d   14   15     14"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4.applymap(f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series.map()还有一个功能是模仿merge(), join()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(['six', 'seven', 'six', 'seven', 'six'], index=['a', 'b', 'c', 'd', 'e'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "t = pd.Series({'six':6., 'seven':7.})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a      six\n",
       "b    seven\n",
       "c      six\n",
       "d    seven\n",
       "e      six\n",
       "dtype: object"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "seven    7.0\n",
       "six      6.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    6.0\n",
       "b    7.0\n",
       "c    6.0\n",
       "d    7.0\n",
       "e    6.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.map(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## reindex和改变label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "reindex()是pandas中基本的数据对齐方法。其他所有依赖label对齐的方法基本都要靠reindex()实现。reindex(重新索引)意味着是沿着某条轴转换数据以匹配新设定的label。具体来说，reindex()做了三件事情：\n",
    "* 对数据进行排序以匹配新的labels\n",
    "* 如果新label对应的位置没有数据，插入缺失值NA\n",
    "* 可以指定调用fill填充数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "下面是一个简单的例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(np.random.randn(5), index=['a','b','c','d','e'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    0.735912\n",
       "b   -0.102690\n",
       "c    2.403671\n",
       "d   -0.472632\n",
       "e   -0.735369\n",
       "dtype: float64"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "e   -0.735369\n",
       "b   -0.102690\n",
       "f         NaN\n",
       "d   -0.472632\n",
       "dtype: float64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.reindex(['e', 'b', 'f', 'd'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "对于DataFrame来说，你可以同时改变列名和索引值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.142261</td>\n",
       "      <td>-0.060360</td>\n",
       "      <td>0.295683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.603017</td>\n",
       "      <td>2.146638</td>\n",
       "      <td>-0.485565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-0.993533</td>\n",
       "      <td>-1.092061</td>\n",
       "      <td>-0.499304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.997475</td>\n",
       "      <td>1.060795</td>\n",
       "      <td>0.065338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>0.319973</td>\n",
       "      <td>-0.555038</td>\n",
       "      <td>-0.340192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>-0.267975</td>\n",
       "      <td>-0.721199</td>\n",
       "      <td>1.976621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>-0.323005</td>\n",
       "      <td>-1.736969</td>\n",
       "      <td>-1.579061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-0.849693</td>\n",
       "      <td>0.641479</td>\n",
       "      <td>-1.241698</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.142261 -0.060360  0.295683\n",
       "2000-01-02  0.603017  2.146638 -0.485565\n",
       "2000-01-03 -0.993533 -1.092061 -0.499304\n",
       "2000-01-04  0.997475  1.060795  0.065338\n",
       "2000-01-05  0.319973 -0.555038 -0.340192\n",
       "2000-01-06 -0.267975 -0.721199  1.976621\n",
       "2000-01-07 -0.323005 -1.736969 -1.579061\n",
       "2000-01-08 -0.849693  0.641479 -1.241698"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "      <th>one</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   three  two  one\n",
       "c    NaN  NaN  NaN\n",
       "f    NaN  NaN  NaN\n",
       "b    NaN  NaN  NaN"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.reindex(index=['c', 'f', 'b'], columns=['three', 'two', 'one'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果只想改变列或者索引的label，DataFrame也提供了reindex_axis()方法，接收label和axis。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "rs = s.reindex(df.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-01   NaN\n",
       "2000-01-02   NaN\n",
       "2000-01-03   NaN\n",
       "2000-01-04   NaN\n",
       "2000-01-05   NaN\n",
       "2000-01-06   NaN\n",
       "2000-01-07   NaN\n",
       "2000-01-08   NaN\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rs.index is df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "上面一行代码顺便说明了Series的索引和DataFrame的索引是同一类的实例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 重新索引来和另一个对象对齐 reindex_like()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "你可能想传递一个对象，使得原来对象的label和传入的对象一样，使用reindex_like()即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df2 = pd.DataFrame(np.random.randn(3, 2),index=['a','b','c'],columns=['one', 'two'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>-0.207263</td>\n",
       "      <td>0.618381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>0.271959</td>\n",
       "      <td>-0.013542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.486730</td>\n",
       "      <td>1.308178</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one       two\n",
       "a -0.207263  0.618381\n",
       "b  0.271959 -0.013542\n",
       "c  0.486730  1.308178"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.142261</td>\n",
       "      <td>-0.060360</td>\n",
       "      <td>0.295683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.603017</td>\n",
       "      <td>2.146638</td>\n",
       "      <td>-0.485565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>-0.993533</td>\n",
       "      <td>-1.092061</td>\n",
       "      <td>-0.499304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.997475</td>\n",
       "      <td>1.060795</td>\n",
       "      <td>0.065338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>0.319973</td>\n",
       "      <td>-0.555038</td>\n",
       "      <td>-0.340192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>-0.267975</td>\n",
       "      <td>-0.721199</td>\n",
       "      <td>1.976621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>-0.323005</td>\n",
       "      <td>-1.736969</td>\n",
       "      <td>-1.579061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-0.849693</td>\n",
       "      <td>0.641479</td>\n",
       "      <td>-1.241698</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.142261 -0.060360  0.295683\n",
       "2000-01-02  0.603017  2.146638 -0.485565\n",
       "2000-01-03 -0.993533 -1.092061 -0.499304\n",
       "2000-01-04  0.997475  1.060795  0.065338\n",
       "2000-01-05  0.319973 -0.555038 -0.340192\n",
       "2000-01-06 -0.267975 -0.721199  1.976621\n",
       "2000-01-07 -0.323005 -1.736969 -1.579061\n",
       "2000-01-08 -0.849693  0.641479 -1.241698"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   one  two\n",
       "a  NaN  NaN\n",
       "b  NaN  NaN\n",
       "c  NaN  NaN"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.reindex_like(df2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 使用align() 是两个对象相互对齐"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "align()方法是让两个对象同事对齐的最快方法。它含有join参数，\n",
    "* join='outer': 取得两个对象的索引并集，这也是join的默认值。\n",
    "* join='left': 使用调用对象的索引值\n",
    "* join='right'：使用被调用对象的索引值\n",
    "* join='inner': 使用两个对象的索引交集\n",
    "\n",
    "align()方法返回一个元组，元素元素是重新索引的Series对象。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s1 = s[:4]\n",
    "s2 = s[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a   -1.131265\n",
       "b    0.267754\n",
       "c    0.679747\n",
       "d    0.185136\n",
       "dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b    0.267754\n",
       "c    0.679747\n",
       "d    0.185136\n",
       "e   -0.440251\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(a   -1.131265\n",
       " b    0.267754\n",
       " c    0.679747\n",
       " d    0.185136\n",
       " e         NaN\n",
       " dtype: float64, a         NaN\n",
       " b    0.267754\n",
       " c    0.679747\n",
       " d    0.185136\n",
       " e   -0.440251\n",
       " dtype: float64)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1.align(s2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(b    0.267754\n",
       " c    0.679747\n",
       " d    0.185136\n",
       " dtype: float64, b    0.267754\n",
       " c    0.679747\n",
       " d    0.185136\n",
       " dtype: float64)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1.align(s2, join='inner') #交集是 'b', 'c', 'd'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "对于DataFrame来说，join方法默认会应用到索引和列名。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.064020</td>\n",
       "      <td>1.089580</td>\n",
       "      <td>-1.007871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>-1.067990</td>\n",
       "      <td>0.075548</td>\n",
       "      <td>-1.326231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>0.813001</td>\n",
       "      <td>0.010123</td>\n",
       "      <td>0.729032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.283158</td>\n",
       "      <td>0.252955</td>\n",
       "      <td>-0.196908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>-1.481362</td>\n",
       "      <td>-0.773020</td>\n",
       "      <td>-0.177803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>1.760388</td>\n",
       "      <td>-1.276605</td>\n",
       "      <td>0.583953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>1.197909</td>\n",
       "      <td>0.013248</td>\n",
       "      <td>0.027281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>1.000680</td>\n",
       "      <td>-0.523045</td>\n",
       "      <td>0.821203</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.064020  1.089580 -1.007871\n",
       "2000-01-02 -1.067990  0.075548 -1.326231\n",
       "2000-01-03  0.813001  0.010123  0.729032\n",
       "2000-01-04  0.283158  0.252955 -0.196908\n",
       "2000-01-05 -1.481362 -0.773020 -0.177803\n",
       "2000-01-06  1.760388 -1.276605  0.583953\n",
       "2000-01-07  1.197909  0.013248  0.027281\n",
       "2000-01-08  1.000680 -0.523045  0.821203"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df2 = df.iloc[:5,:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.064020</td>\n",
       "      <td>1.089580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>-1.067990</td>\n",
       "      <td>0.075548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>0.813001</td>\n",
       "      <td>0.010123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.283158</td>\n",
       "      <td>0.252955</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>-1.481362</td>\n",
       "      <td>-0.773020</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B\n",
       "2000-01-01 -0.064020  1.089580\n",
       "2000-01-02 -1.067990  0.075548\n",
       "2000-01-03  0.813001  0.010123\n",
       "2000-01-04  0.283158  0.252955\n",
       "2000-01-05 -1.481362 -0.773020"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(                   A         B\n",
       " 2000-01-01 -0.064020  1.089580\n",
       " 2000-01-02 -1.067990  0.075548\n",
       " 2000-01-03  0.813001  0.010123\n",
       " 2000-01-04  0.283158  0.252955\n",
       " 2000-01-05 -1.481362 -0.773020,                    A         B\n",
       " 2000-01-01 -0.064020  1.089580\n",
       " 2000-01-02 -1.067990  0.075548\n",
       " 2000-01-03  0.813001  0.010123\n",
       " 2000-01-04  0.283158  0.252955\n",
       " 2000-01-05 -1.481362 -0.773020)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.align(df2, join='inner')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(                   A         B         C\n",
       " 2000-01-01 -0.064020  1.089580 -1.007871\n",
       " 2000-01-02 -1.067990  0.075548 -1.326231\n",
       " 2000-01-03  0.813001  0.010123  0.729032\n",
       " 2000-01-04  0.283158  0.252955 -0.196908\n",
       " 2000-01-05 -1.481362 -0.773020 -0.177803\n",
       " 2000-01-06  1.760388 -1.276605  0.583953\n",
       " 2000-01-07  1.197909  0.013248  0.027281\n",
       " 2000-01-08  1.000680 -0.523045  0.821203,                    A         B   C\n",
       " 2000-01-01 -0.064020  1.089580 NaN\n",
       " 2000-01-02 -1.067990  0.075548 NaN\n",
       " 2000-01-03  0.813001  0.010123 NaN\n",
       " 2000-01-04  0.283158  0.252955 NaN\n",
       " 2000-01-05 -1.481362 -0.773020 NaN\n",
       " 2000-01-06       NaN       NaN NaN\n",
       " 2000-01-07       NaN       NaN NaN\n",
       " 2000-01-08       NaN       NaN NaN)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.align(df2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "align()也含有一个axis参数，指定仅对于某一坐标轴进行对齐。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(                   A         B         C\n",
       " 2000-01-01 -0.064020  1.089580 -1.007871\n",
       " 2000-01-02 -1.067990  0.075548 -1.326231\n",
       " 2000-01-03  0.813001  0.010123  0.729032\n",
       " 2000-01-04  0.283158  0.252955 -0.196908\n",
       " 2000-01-05 -1.481362 -0.773020 -0.177803,                    A         B\n",
       " 2000-01-01 -0.064020  1.089580\n",
       " 2000-01-02 -1.067990  0.075548\n",
       " 2000-01-03  0.813001  0.010123\n",
       " 2000-01-04  0.283158  0.252955\n",
       " 2000-01-05 -1.481362 -0.773020)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.align(df2, join='inner', axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "DataFrame.align()同样能接收Series对象，此时axis指的是DataFrame对象的索引或列。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(                   A         B         C\n",
       " 2000-01-01 -0.064020  1.089580 -1.007871\n",
       " 2000-01-02 -1.067990  0.075548 -1.326231\n",
       " 2000-01-03  0.813001  0.010123  0.729032\n",
       " 2000-01-04  0.283158  0.252955 -0.196908\n",
       " 2000-01-05 -1.481362 -0.773020 -0.177803\n",
       " 2000-01-06  1.760388 -1.276605  0.583953\n",
       " 2000-01-07  1.197909  0.013248  0.027281\n",
       " 2000-01-08  1.000680 -0.523045  0.821203, A   -0.06402\n",
       " B    1.08958\n",
       " C        NaN\n",
       " Name: 2000-01-01 00:00:00, dtype: float64)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.align(df2.ix[0], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 重索引时顺便填充数值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "reindex()方法还有一个method参数，用于填充数值,method取值如下：\n",
    "* pad/ffill: 使用**后面的值**填充数值\n",
    "* bfill/backfill: 使用**前面的值**填充数值\n",
    "* nearest: 使用最近的索引值进行填充"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "以Series为例，看一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "rng = pd.date_range('1/3/2000', periods=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ts = pd.Series(np.random.randn(8), index=rng)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ts2 = ts[[0, 3, 6]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04    0.077131\n",
       "2000-01-05   -0.714099\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.942098\n",
       "2000-01-08   -0.242916\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10    0.671837\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-06   -0.109570\n",
       "2000-01-09   -0.733449\n",
       "dtype: float64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04         NaN\n",
       "2000-01-05         NaN\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07         NaN\n",
       "2000-01-08         NaN\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10         NaN\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04   -0.765910\n",
       "2000-01-05   -0.765910\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.109570\n",
       "2000-01-08   -0.109570\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10   -0.733449\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index, method='ffill') #索引小那一行的数值填充NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04   -0.109570\n",
       "2000-01-05   -0.109570\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.733449\n",
       "2000-01-08   -0.733449\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10         NaN\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index, method='bfill') #索引大的非NaN的数值填充NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04   -0.765910\n",
       "2000-01-05   -0.109570\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.109570\n",
       "2000-01-08   -0.733449\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10   -0.733449\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index, method='nearest')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "method参数要求索引必须是有序的：递增或递减。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "除了method='nearest',其他method取值也能用fillna()方法实现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04   -0.765910\n",
       "2000-01-05   -0.765910\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.109570\n",
       "2000-01-08   -0.109570\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10   -0.733449\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index).fillna(method='ffill')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "二者的区别是：如果索引不是有序的，reindex()会报错，而fillna()和interpolate()不会检查索引是否有序。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 重索引时 有条件地填充NaN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "limit和tolerance参数会对填充操作进行条件限制，通常限制填充的次数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04   -0.765910\n",
       "2000-01-05         NaN\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.109570\n",
       "2000-01-08         NaN\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10   -0.733449\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index, method='ffill', limit=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-06   -0.109570\n",
       "2000-01-09   -0.733449\n",
       "dtype: float64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04    0.077131\n",
       "2000-01-05   -0.714099\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.942098\n",
       "2000-01-08   -0.242916\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10    0.671837\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000-01-03   -0.765910\n",
       "2000-01-04   -0.765910\n",
       "2000-01-05         NaN\n",
       "2000-01-06   -0.109570\n",
       "2000-01-07   -0.109570\n",
       "2000-01-08         NaN\n",
       "2000-01-09   -0.733449\n",
       "2000-01-10   -0.733449\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts2.reindex(ts.index, method='ffill', tolerance='1 day')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 移除某些索引值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "和reindex()方法很相似的是drop()，用于移除索引的某些取值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>-0.064020</td>\n",
       "      <td>1.089580</td>\n",
       "      <td>-1.007871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>-1.067990</td>\n",
       "      <td>0.075548</td>\n",
       "      <td>-1.326231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>0.813001</td>\n",
       "      <td>0.010123</td>\n",
       "      <td>0.729032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.283158</td>\n",
       "      <td>0.252955</td>\n",
       "      <td>-0.196908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>-1.481362</td>\n",
       "      <td>-0.773020</td>\n",
       "      <td>-0.177803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>1.760388</td>\n",
       "      <td>-1.276605</td>\n",
       "      <td>0.583953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>1.197909</td>\n",
       "      <td>0.013248</td>\n",
       "      <td>0.027281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>1.000680</td>\n",
       "      <td>-0.523045</td>\n",
       "      <td>0.821203</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   A         B         C\n",
       "2000-01-01 -0.064020  1.089580 -1.007871\n",
       "2000-01-02 -1.067990  0.075548 -1.326231\n",
       "2000-01-03  0.813001  0.010123  0.729032\n",
       "2000-01-04  0.283158  0.252955 -0.196908\n",
       "2000-01-05 -1.481362 -0.773020 -0.177803\n",
       "2000-01-06  1.760388 -1.276605  0.583953\n",
       "2000-01-07  1.197909  0.013248  0.027281\n",
       "2000-01-08  1.000680 -0.523045  0.821203"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-01</th>\n",
       "      <td>1.089580</td>\n",
       "      <td>-1.007871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-02</th>\n",
       "      <td>0.075548</td>\n",
       "      <td>-1.326231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>0.010123</td>\n",
       "      <td>0.729032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>0.252955</td>\n",
       "      <td>-0.196908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>-0.773020</td>\n",
       "      <td>-0.177803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>-1.276605</td>\n",
       "      <td>0.583953</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>0.013248</td>\n",
       "      <td>0.027281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-08</th>\n",
       "      <td>-0.523045</td>\n",
       "      <td>0.821203</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   B         C\n",
       "2000-01-01  1.089580 -1.007871\n",
       "2000-01-02  0.075548 -1.326231\n",
       "2000-01-03  0.010123  0.729032\n",
       "2000-01-04  0.252955 -0.196908\n",
       "2000-01-05 -0.773020 -0.177803\n",
       "2000-01-06 -1.276605  0.583953\n",
       "2000-01-07  0.013248  0.027281\n",
       "2000-01-08 -0.523045  0.821203"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop(['A'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 重命名索引值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "rename() 方法可以对索引值重新命名，命名方式可以是字典或Series，也可以是任意的方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a   -1.131265\n",
       "b    0.267754\n",
       "c    0.679747\n",
       "d    0.185136\n",
       "e   -0.440251\n",
       "dtype: float64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A   -1.131265\n",
       "B    0.267754\n",
       "C    0.679747\n",
       "D    0.185136\n",
       "E   -0.440251\n",
       "dtype: float64"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rename(str.upper)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "唯一的要求是传入的函数调用索引值时必须有一个返回值，如果你传入的是字典或Series，要求是索引值必须是其键值。这点很好理解。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>-0.074436</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.720569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>1.154073</td>\n",
       "      <td>-0.335651</td>\n",
       "      <td>0.238351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.147157</td>\n",
       "      <td>0.848397</td>\n",
       "      <td>0.827290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0.699900</td>\n",
       "      <td>-1.079261</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        one     three       two\n",
       "a -0.074436       NaN -1.720569\n",
       "b  1.154073 -0.335651  0.238351\n",
       "c  0.147157  0.848397  0.827290\n",
       "d       NaN  0.699900 -1.079261"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({'one' : pd.Series(np.random.randn(3), index=['a', 'b', 'c']),\n",
    "                   'two' : pd.Series(np.random.randn(4), index=['a', 'b', 'c', 'd']),\n",
    "                    'three' : pd.Series(np.random.randn(3), index=['b', 'c', 'd'])})\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>foo</th>\n",
       "      <th>three</th>\n",
       "      <th>bar</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>apple</th>\n",
       "      <td>-0.074436</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.720569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>banana</th>\n",
       "      <td>1.154073</td>\n",
       "      <td>-0.335651</td>\n",
       "      <td>0.238351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>0.147157</td>\n",
       "      <td>0.848397</td>\n",
       "      <td>0.827290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>durian</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0.699900</td>\n",
       "      <td>-1.079261</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             foo     three       bar\n",
       "apple  -0.074436       NaN -1.720569\n",
       "banana  1.154073 -0.335651  0.238351\n",
       "c       0.147157  0.848397  0.827290\n",
       "durian       NaN  0.699900 -1.079261"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.rename(columns={'one' : 'foo', 'two' : 'bar'},\n",
    "          index={'a' : 'apple', 'b' : 'banana', 'd' : 'durian'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "默认情况下修改的仅仅是副本，如果想对原对象索引值修改，inplace=True."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "在0.18.0版本中，rename方法也能修改Series.name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a   -1.131265\n",
       "b    0.267754\n",
       "c    0.679747\n",
       "d    0.185136\n",
       "e   -0.440251\n",
       "Name: sclar-name, dtype: float64"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rename('sclar-name')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 迭代操作 Iteration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas中迭代操作依赖于具体的对象。迭代Series对象时类似迭代数组,产生的是值，迭代DataFrame或Panel对象时类似迭代字典的键值。\n",
    "\n",
    "一句话，(for i in object)产生:\n",
    "* **Series**: 值\n",
    "* **DataFrame**: 列名\n",
    "* **Panel**: item名"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "看一下例子吧："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col1</th>\n",
       "      <th>col2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0.172851</td>\n",
       "      <td>-0.382805</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>-0.066942</td>\n",
       "      <td>0.672660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>-2.091001</td>\n",
       "      <td>0.510819</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       col1      col2\n",
       "a  0.172851 -0.382805\n",
       "b -0.066942  0.672660\n",
       "c -2.091001  0.510819"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({'col1' : np.random.randn(3), 'col2' : np.random.randn(3)},\n",
    "                  index=['a', 'b', 'c'])\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "col1\n",
      "col2\n"
     ]
    }
   ],
   "source": [
    "for col in df: #产生的是列名\n",
    "    print col"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas对象也有类似字典的iteritems()方法来迭代(key, value)。\n",
    "\n",
    "为了每一行迭代DataFrame对象，有两种方法：\n",
    "* **iterrows()**: 按照行来迭代(index, Series)。会把每一行转为Series对象。\n",
    "* **itertuples()**: 按照行来迭代namedtuples. 这种方法比iterrows()快，大多数情况下推荐使用此方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**警告**：\n",
    "\n",
    "迭代pandas对象通常会比较慢。所以尽量避免迭代操作，可以用以下方法替换迭代：\n",
    "* 数据结构的内置方法，索引或numpy方法等\n",
    "* apply()\n",
    "* 使用cython写内循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**警告**：\n",
    "\n",
    "**当迭代进行时永远不要有修改操作**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## iteritems()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "类似字典的接口，iteritems()对键值对进行迭代操作：\n",
    "* Series: (index, scalar value)\n",
    "* DataFrame: (column, Series)\n",
    "* Panel: (item, DataFrame)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "看一下例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "col1 a    0.172851\n",
      "b   -0.066942\n",
      "c   -2.091001\n",
      "Name: col1, dtype: float64\n",
      "col2 a   -0.382805\n",
      "b    0.672660\n",
      "c    0.510819\n",
      "Name: col2, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "for item, frame in df.iteritems():\n",
    "    print item, frame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### iterrows()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "iterrows()方法用于迭代DataFrame的每一行，返回的是索引和Series的迭代器，但要注意Series的dtype可能和原来每一行的dtype不同。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a col1    0.172851\n",
      "col2   -0.382805\n",
      "Name: a, dtype: float64\n",
      "b col1   -0.066942\n",
      "col2    0.672660\n",
      "Name: b, dtype: float64\n",
      "c col1   -2.091001\n",
      "col2    0.510819\n",
      "Name: c, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "for row_index, row in df.iterrows():\n",
    "    print  row_index, row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### itertuples()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "itertuples()方法迭代DataFrame每一行，返回的是namedtuple。因为返回的不是Series，所以会保留DataFrame中值的dtype。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pandas(Index='a', col1=0.17285076817271622, col2=-0.38280463605747844)\n",
      "Pandas(Index='b', col1=-0.066941511967637243, col2=0.67266041180134961)\n",
      "Pandas(Index='c', col1=-2.0910005480409719, col2=0.51081932119946261)\n"
     ]
    }
   ],
   "source": [
    "for row in df.itertuples():\n",
    "    print row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## .dt 访问器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series对象如果索引是datetime/period,可以用自带的.dt访问器返回日期、小时、分钟。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(pd.date_range('20160101 09:10:12', periods=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2016-01-01 09:10:12\n",
       "1   2016-01-02 09:10:12\n",
       "2   2016-01-03 09:10:12\n",
       "3   2016-01-04 09:10:12\n",
       "dtype: datetime64[ns]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    9\n",
       "1    9\n",
       "2    9\n",
       "3    9\n",
       "dtype: int64"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.dt.hour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    12\n",
       "1    12\n",
       "2    12\n",
       "3    12\n",
       "dtype: int64"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.dt.second"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    2\n",
       "2    3\n",
       "3    4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.dt.day"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "改变时间的格式也很方便，Series.dt.strftime()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(pd.date_range('20130101', periods=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2013-01-01\n",
       "1   2013-01-02\n",
       "2   2013-01-03\n",
       "3   2013-01-04\n",
       "dtype: datetime64[ns]"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2013/01/01\n",
       "1    2013/01/02\n",
       "2    2013/01/03\n",
       "3    2013/01/04\n",
       "dtype: object"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.dt.strftime('%Y/%m/%d')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " ## 字符串处理方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series带有一系列的字符串处理, 默认不对NaN处理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s = pd.Series(['A', 'B', 'C', 'Aaba', 'Baca', np.nan, 'CABA', 'dog', 'cat'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       A\n",
       "1       B\n",
       "2       C\n",
       "3    Aaba\n",
       "4    Baca\n",
       "5     NaN\n",
       "6    CABA\n",
       "7     dog\n",
       "8     cat\n",
       "dtype: object"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       a\n",
       "1       b\n",
       "2       c\n",
       "3    aaba\n",
       "4    baca\n",
       "5     NaN\n",
       "6    caba\n",
       "7     dog\n",
       "8     cat\n",
       "dtype: object"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.str.lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 排序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "排序方法可以分为两大类： 按照实际的值排序和按照label排序。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 按照index排序 sort_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series.sort_index(), DataFrame.sort_index(), 参数是ascending, axis。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "      <th>one</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   three  two  one\n",
       "a    NaN  NaN  NaN\n",
       "d    NaN  NaN  NaN\n",
       "c    NaN  NaN  NaN\n",
       "b    NaN  NaN  NaN"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unsorted_df = df.reindex(index=['a', 'd', 'c', 'b'],\n",
    "                         columns=['three', 'two', 'one'])\n",
    "unsorted_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "      <th>one</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   three  two  one\n",
       "a    NaN  NaN  NaN\n",
       "b    NaN  NaN  NaN\n",
       "c    NaN  NaN  NaN\n",
       "d    NaN  NaN  NaN"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unsorted_df.sort_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "      <th>one</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   three  two  one\n",
       "d    NaN  NaN  NaN\n",
       "c    NaN  NaN  NaN\n",
       "b    NaN  NaN  NaN\n",
       "a    NaN  NaN  NaN"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unsorted_df.sort_index(ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   one  three  two\n",
       "a  NaN    NaN  NaN\n",
       "d  NaN    NaN  NaN\n",
       "c  NaN    NaN  NaN\n",
       "b  NaN    NaN  NaN"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unsorted_df.sort_index(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a   NaN\n",
       "b   NaN\n",
       "c   NaN\n",
       "d   NaN\n",
       "Name: three, dtype: float64"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unsorted_df['three'].sort_index() # Series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 按照值排序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series.sort_values(), DataFrame.sort_values()用于按照值进行排序，参数有by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   one  three  two\n",
       "0    2      5    1\n",
       "1    1      4    3\n",
       "2    1      3    2\n",
       "3    1      2    4"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = pd.DataFrame({'one':[2,1,1,1],'two':[1,3,2,4],'three':[5,4,3,2]})\n",
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   one  three  two\n",
       "0    2      5    1\n",
       "2    1      3    2\n",
       "1    1      4    3\n",
       "3    1      2    4"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.sort_values(by='two')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>three</th>\n",
       "      <th>two</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   one  three  two\n",
       "2    1      3    2\n",
       "1    1      4    3\n",
       "3    1      2    4\n",
       "0    2      5    1"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.sort_values(by=['one', 'two'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "通过na_position参数处理NA值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "s[2] = np.nan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       A\n",
       "3    Aaba\n",
       "1       B\n",
       "4    Baca\n",
       "6    CABA\n",
       "8     cat\n",
       "7     dog\n",
       "2     NaN\n",
       "5     NaN\n",
       "dtype: object"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.sort_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2     NaN\n",
       "5     NaN\n",
       "0       A\n",
       "3    Aaba\n",
       "1       B\n",
       "4    Baca\n",
       "6    CABA\n",
       "8     cat\n",
       "7     dog\n",
       "dtype: object"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.sort_values(na_position='first') #将NA放在前面"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### searchsorted()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series.searchsorted()类似numpy.ndarray.searchsorted()。 找到元素在排好序后的位置(下标)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ser = pd.Series([1,2,3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 3], dtype=int64)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ser.searchsorted([0, 3]) #元素0下标是0， 元素3下标是2。注意不同元素之间是独立的，所以元素3的位置是2而不是插入0后的3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 3], dtype=int64)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ser.searchsorted([0, 4]) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 3], dtype=int64)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ser.searchsorted([1, 3], side='right')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2], dtype=int64)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ser.searchsorted([1, 3], side='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ser = pd.Series([3, 1, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2], dtype=int64)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ser.searchsorted([0, 3], sorter=np.argsort(ser))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 最小/最大值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series有nsmallest() nlargest()方法能够返回最小或最大的n个值。如果Series对象很大，这两种方法会比先排序后使用head()方法快很多。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    6\n",
       "1    3\n",
       "2    8\n",
       "3    7\n",
       "4    5\n",
       "5    2\n",
       "6    4\n",
       "7    0\n",
       "8    1\n",
       "9    9\n",
       "dtype: int32"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = pd.Series(np.random.permutation(10))\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7    0\n",
       "8    1\n",
       "5    2\n",
       "1    3\n",
       "6    4\n",
       "4    5\n",
       "0    6\n",
       "3    7\n",
       "2    8\n",
       "9    9\n",
       "dtype: int32"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.sort_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7    0\n",
       "8    1\n",
       "5    2\n",
       "dtype: int32"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.nsmallest(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9    9\n",
       "2    8\n",
       "3    7\n",
       "dtype: int32"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.nlargest(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "从v0.17.0开始，DataFrame也有了以上两个方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'a': [-2, -1, 1, 10, 8, 11, -1],\n",
    "                      'b': list('abdceff'),\n",
    "                       'c': [1.0, 2.0, 4.0, 3.2, np.nan, 3.0, 4.0]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-2</td>\n",
       "      <td>a</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1</td>\n",
       "      <td>b</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>d</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>c</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>11</td>\n",
       "      <td>f</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-1</td>\n",
       "      <td>f</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    a  b    c\n",
       "0  -2  a  1.0\n",
       "1  -1  b  2.0\n",
       "2   1  d  4.0\n",
       "3  10  c  3.2\n",
       "4   8  e  NaN\n",
       "5  11  f  3.0\n",
       "6  -1  f  4.0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>11</td>\n",
       "      <td>f</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>c</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>d</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1</td>\n",
       "      <td>b</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    a  b    c\n",
       "5  11  f  3.0\n",
       "3  10  c  3.2\n",
       "4   8  e  NaN\n",
       "2   1  d  4.0\n",
       "1  -1  b  2.0"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nlargest(5, 'a') #列 'a'最大的3个值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>11</td>\n",
       "      <td>f</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>c</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>d</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1</td>\n",
       "      <td>b</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    a  b    c\n",
       "5  11  f  3.0\n",
       "3  10  c  3.2\n",
       "4   8  e  NaN\n",
       "2   1  d  4.0\n",
       "1  -1  b  2.0"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nlargest(5, ['a', 'c'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-2</td>\n",
       "      <td>a</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1</td>\n",
       "      <td>b</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-1</td>\n",
       "      <td>f</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   a  b    c\n",
       "0 -2  a  1.0\n",
       "1 -1  b  2.0\n",
       "6 -1  f  4.0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nsmallest(3, 'a')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-2</td>\n",
       "      <td>a</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1</td>\n",
       "      <td>b</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-1</td>\n",
       "      <td>f</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>d</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   a  b    c\n",
       "0 -2  a  1.0\n",
       "1 -1  b  2.0\n",
       "6 -1  f  4.0\n",
       "2  1  d  4.0\n",
       "4  8  e  NaN"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nsmallest(5, ['a', 'c'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 多索引列 排序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果一列是多索引，你必须指定全部每一级的索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df1 = pd.DataFrame({'one':[2,1,1,1],'two':[1,3,2,4],'three':[5,4,3,2]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df1.columns = pd.MultiIndex.from_tuples([('a','one'),('a','two'),('b','three')])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">a</th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>two</th>\n",
       "      <th>three</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    a         b\n",
       "  one two three\n",
       "0   2   5     1\n",
       "1   1   4     3\n",
       "2   1   3     2\n",
       "3   1   2     4"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">a</th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>one</th>\n",
       "      <th>two</th>\n",
       "      <th>three</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    a         b\n",
       "  one two three\n",
       "3   1   2     4\n",
       "2   1   3     2\n",
       "1   1   4     3\n",
       "0   2   5     1"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.sort_values(by=('a','two'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 复制"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "copy()方法复制数据结构的值并返回一个新的对象。记住**复制操作不到万不得已不使用**。 比如，改变DataFrame对象值的几种方法：\n",
    "* inserting, deleting, modifying a column\n",
    "* 为索引、列 赋值\n",
    "* 对于同构数据，直接使用values属性修改值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**几乎所有的方法都不对原对象进行直接修改，而是返回修改后的一个新对象！如果原对象数据被修改，肯定是你显示指定的修改操作。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " ## dtypes属性"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "pandas对象的主要数据类型包括: float, int, bool, datetime64[ns], datetime64[ns, tz], timedelta[ns], category, object. 除此之外还有更具体的说明存储比特数的数据类型比如int64, int32。\n",
    "\n",
    "DataFrame的dtypes属性返回一个Series对象，Series值是DF中每一列的数据类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame(dict(A = np.random.rand(3),\n",
    "                            B = 1,\n",
    "                           C = 'foo',\n",
    "                            D = pd.Timestamp('20010102'),\n",
    "                           E = pd.Series([1.0]*3).astype('float32'),\n",
    "                                        F = False,\n",
    "                                       G = pd.Series([1]*3,dtype='int8')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>E</th>\n",
       "      <th>F</th>\n",
       "      <th>G</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.223911</td>\n",
       "      <td>1</td>\n",
       "      <td>foo</td>\n",
       "      <td>2001-01-02</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.306255</td>\n",
       "      <td>1</td>\n",
       "      <td>foo</td>\n",
       "      <td>2001-01-02</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.435360</td>\n",
       "      <td>1</td>\n",
       "      <td>foo</td>\n",
       "      <td>2001-01-02</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A  B    C          D    E      F  G\n",
       "0  0.223911  1  foo 2001-01-02  1.0  False  1\n",
       "1  0.306255  1  foo 2001-01-02  1.0  False  1\n",
       "2  0.435360  1  foo 2001-01-02  1.0  False  1"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A           float64\n",
       "B             int64\n",
       "C            object\n",
       "D    datetime64[ns]\n",
       "E           float32\n",
       "F              bool\n",
       "G              int8\n",
       "dtype: object"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Series同样有dtypes属性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['A'].dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果pandas对象的一列中有多种数据类型，dtype返回的是能兼容所有数据类型的类型，object范围最大的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1.0\n",
       "1    2.0\n",
       "2    3.0\n",
       "3    4.0\n",
       "4    5.0\n",
       "5    6.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([1,2,3,4,5,6.])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      1\n",
       "1      2\n",
       "2      3\n",
       "3      6\n",
       "4    foo\n",
       "dtype: object"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([1,2,3,6.,'foo'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "get_dtype_counts()方法返回DataFrame中每一种数据类型的列数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bool              1\n",
       "datetime64[ns]    1\n",
       "float32           1\n",
       "float64           1\n",
       "int64             1\n",
       "int8              1\n",
       "object            1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.get_dtype_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "数值数据类型可以在ndarray，Series和DataFrame中传播。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df1 = pd.DataFrame(np.random.randn(8, 1), columns=['A'], dtype='float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.325100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.052396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.866763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.647942</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.518647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.982795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.068309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2.034926</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A\n",
       "0  0.325100\n",
       "1 -1.052396\n",
       "2  1.866763\n",
       "3 -0.647942\n",
       "4 -1.518647\n",
       "5  0.982795\n",
       "6 -0.068309\n",
       "7  2.034926"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "dtype: object"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.427979</td>\n",
       "      <td>0.280275</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.701172</td>\n",
       "      <td>-0.456572</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.558105</td>\n",
       "      <td>-0.248928</td>\n",
       "      <td>255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.361572</td>\n",
       "      <td>0.746420</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.667969</td>\n",
       "      <td>0.722280</td>\n",
       "      <td>255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-1.436523</td>\n",
       "      <td>-0.864915</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.147949</td>\n",
       "      <td>1.620295</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.075989</td>\n",
       "      <td>-0.527997</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A         B    C\n",
       "0 -0.427979  0.280275    0\n",
       "1 -0.701172 -0.456572    0\n",
       "2 -0.558105 -0.248928  255\n",
       "3 -0.361572  0.746420    0\n",
       "4 -1.667969  0.722280  255\n",
       "5 -1.436523 -0.864915    0\n",
       "6  0.147949  1.620295    0\n",
       "7  0.075989 -0.527997    0"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.DataFrame(dict( A = pd.Series(np.random.randn(8), dtype='float16'),\n",
    "                         B = pd.Series(np.random.randn(8)),\n",
    "                         C = pd.Series(np.array(np.random.randn(8), dtype='uint8')) )) #这里是float16， uint8\n",
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float16\n",
       "B    float64\n",
       "C      uint8\n",
       "dtype: object"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 数据类型的默认值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "整型的默认值蕾西int64，浮点型的默认类型是float64，和你用的是32位还是64位的系统无关。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([1, 2], columns=['a']).dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame({'a': [1, 2]}).dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame({'a': 1}, index=list(range(2))).dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Numpy中数值的具体类型则要依赖于平台。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "frame = pd.DataFrame(np.array([1, 2])) #如果是在32位系统，数据类型int32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### upcasting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "不同类型结合时会upcast，即得到更通用的类型，看例子吧："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "dtype: object"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float16\n",
       "B    float64\n",
       "C      uint8\n",
       "dtype: object"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float64\n",
       "C    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.reindex_like(df2).fillna(value=0.0).dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df3 = df1.reindex_like(df2).fillna(value=0.0) + df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float64\n",
       "C    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### astype()方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "使用astype()显示的进行转型。默认会返回原对象的副本，即使数据类型不变。当然可以传递copy=False参数直接对原对象转型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.102879</td>\n",
       "      <td>0.280275</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.753568</td>\n",
       "      <td>-0.456572</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.308658</td>\n",
       "      <td>-0.248928</td>\n",
       "      <td>255.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.009514</td>\n",
       "      <td>0.746420</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-3.186615</td>\n",
       "      <td>0.722280</td>\n",
       "      <td>255.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.453728</td>\n",
       "      <td>-0.864915</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.079640</td>\n",
       "      <td>1.620295</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2.110915</td>\n",
       "      <td>-0.527997</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A         B      C\n",
       "0 -0.102879  0.280275    0.0\n",
       "1 -1.753568 -0.456572    0.0\n",
       "2  1.308658 -0.248928  255.0\n",
       "3 -1.009514  0.746420    0.0\n",
       "4 -3.186615  0.722280  255.0\n",
       "5 -0.453728 -0.864915    0.0\n",
       "6  0.079640  1.620295    0.0\n",
       "7  2.110915 -0.527997    0.0"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float64\n",
       "C    float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float32\n",
       "C    float32\n",
       "dtype: object"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.astype('float32').dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### 对object类型进行转型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "convert_objects()方法能对object类型进行转型。如果想转为数字，参数是convert_numeric=True。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df3['D'] = '1.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df3['E'] = '1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "      <th>E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.102879</td>\n",
       "      <td>0.280275</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.753568</td>\n",
       "      <td>-0.456572</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.308658</td>\n",
       "      <td>-0.248928</td>\n",
       "      <td>255.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.009514</td>\n",
       "      <td>0.746420</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-3.186615</td>\n",
       "      <td>0.722280</td>\n",
       "      <td>255.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.453728</td>\n",
       "      <td>-0.864915</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.079640</td>\n",
       "      <td>1.620295</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2.110915</td>\n",
       "      <td>-0.527997</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          A         B      C   D  E\n",
       "0 -0.102879  0.280275    0.0  1.  1\n",
       "1 -1.753568 -0.456572    0.0  1.  1\n",
       "2  1.308658 -0.248928  255.0  1.  1\n",
       "3 -1.009514  0.746420    0.0  1.  1\n",
       "4 -3.186615  0.722280  255.0  1.  1\n",
       "5 -0.453728 -0.864915    0.0  1.  1\n",
       "6  0.079640  1.620295    0.0  1.  1\n",
       "7  2.110915 -0.527997    0.0  1.  1"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float64\n",
       "C    float64\n",
       "D     object\n",
       "E     object\n",
       "dtype: object"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.dtypes #现在'D' 'E'两列都是object类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\python27\\lib\\site-packages\\ipykernel\\__main__.py:1: FutureWarning: convert_objects is deprecated.  Use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float64\n",
       "C    float64\n",
       "D    float64\n",
       "E      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.convert_objects(convert_numeric=True).dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df3['D'] = df3['D'].astype('float16')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df3['E'] = df3['E'].astype('int32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    float32\n",
       "B    float64\n",
       "C    float64\n",
       "D    float16\n",
       "E      int32\n",
       "dtype: object"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 基于dtype 选择列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "select_dtypes()方法实现了基于列dtype的构造子集方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'string': list('abc'),\n",
    "                    'int64': list(range(1, 4)),\n",
    "                    'uint8': np.arange(3, 6).astype('u1'),\n",
    "                    'float64': np.arange(4.0, 7.0),\n",
    "                    'bool1': [True, False, True],\n",
    "                    'bool2': [False, True, False],\n",
    "                    'dates': pd.date_range('now', periods=3).values,\n",
    "                    'category': pd.Series(list(\"ABC\")).astype('category')})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bool1</th>\n",
       "      <th>bool2</th>\n",
       "      <th>category</th>\n",
       "      <th>dates</th>\n",
       "      <th>float64</th>\n",
       "      <th>int64</th>\n",
       "      <th>string</th>\n",
       "      <th>uint8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>A</td>\n",
       "      <td>2016-04-20 10:26:18.089</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>B</td>\n",
       "      <td>2016-04-21 10:26:18.089</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2</td>\n",
       "      <td>b</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>C</td>\n",
       "      <td>2016-04-22 10:26:18.089</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3</td>\n",
       "      <td>c</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bool1  bool2 category                   dates  float64  int64 string  uint8\n",
       "0   True  False        A 2016-04-20 10:26:18.089      4.0      1      a      3\n",
       "1  False   True        B 2016-04-21 10:26:18.089      5.0      2      b      4\n",
       "2   True  False        C 2016-04-22 10:26:18.089      6.0      3      c      5"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df['tdeltas'] = df.dates.diff()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df['uint64'] = np.arange(3, 6).astype('u8')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df['other_dates'] = pd.date_range('20130101', periods=3).values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "df['tz_aware_dates'] = pd.date_range('20130101', periods=3, tz='US/Eastern')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bool1</th>\n",
       "      <th>bool2</th>\n",
       "      <th>category</th>\n",
       "      <th>dates</th>\n",
       "      <th>float64</th>\n",
       "      <th>int64</th>\n",
       "      <th>string</th>\n",
       "      <th>uint8</th>\n",
       "      <th>tdeltas</th>\n",
       "      <th>uint64</th>\n",
       "      <th>other_dates</th>\n",
       "      <th>tz_aware_dates</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>A</td>\n",
       "      <td>2016-04-20 10:26:18.089</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>3</td>\n",
       "      <td>NaT</td>\n",
       "      <td>3</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>2013-01-01 00:00:00-05:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>B</td>\n",
       "      <td>2016-04-21 10:26:18.089</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2</td>\n",
       "      <td>b</td>\n",
       "      <td>4</td>\n",
       "      <td>1 days</td>\n",
       "      <td>4</td>\n",
       "      <td>2013-01-02</td>\n",
       "      <td>2013-01-02 00:00:00-05:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>C</td>\n",
       "      <td>2016-04-22 10:26:18.089</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3</td>\n",
       "      <td>c</td>\n",
       "      <td>5</td>\n",
       "      <td>1 days</td>\n",
       "      <td>5</td>\n",
       "      <td>2013-01-03</td>\n",
       "      <td>2013-01-03 00:00:00-05:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bool1  bool2 category                   dates  float64  int64 string  \\\n",
       "0   True  False        A 2016-04-20 10:26:18.089      4.0      1      a   \n",
       "1  False   True        B 2016-04-21 10:26:18.089      5.0      2      b   \n",
       "2   True  False        C 2016-04-22 10:26:18.089      6.0      3      c   \n",
       "\n",
       "   uint8  tdeltas  uint64 other_dates            tz_aware_dates  \n",
       "0      3      NaT       3  2013-01-01 2013-01-01 00:00:00-05:00  \n",
       "1      4   1 days       4  2013-01-02 2013-01-02 00:00:00-05:00  \n",
       "2      5   1 days       5  2013-01-03 2013-01-03 00:00:00-05:00  "
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bool1                                   bool\n",
       "bool2                                   bool\n",
       "category                            category\n",
       "dates                         datetime64[ns]\n",
       "float64                              float64\n",
       "int64                                  int64\n",
       "string                                object\n",
       "uint8                                  uint8\n",
       "tdeltas                      timedelta64[ns]\n",
       "uint64                                uint64\n",
       "other_dates                   datetime64[ns]\n",
       "tz_aware_dates    datetime64[ns, US/Eastern]\n",
       "dtype: object"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "select_dtypes()有两个参数：include, exclude。含义是要选择的列的dtype和不选择列的dtype。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bool1</th>\n",
       "      <th>bool2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bool1  bool2\n",
       "0   True  False\n",
       "1  False   True\n",
       "2   True  False"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.select_dtypes(include=[bool])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bool1</th>\n",
       "      <th>bool2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bool1  bool2\n",
       "0   True  False\n",
       "1  False   True\n",
       "2   True  False"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.select_dtypes(include=['bool'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bool1                                   bool\n",
       "bool2                                   bool\n",
       "category                            category\n",
       "dates                         datetime64[ns]\n",
       "float64                              float64\n",
       "int64                                  int64\n",
       "string                                object\n",
       "uint8                                  uint8\n",
       "tdeltas                      timedelta64[ns]\n",
       "uint64                                uint64\n",
       "other_dates                   datetime64[ns]\n",
       "tz_aware_dates    datetime64[ns, US/Eastern]\n",
       "dtype: object"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bool1</th>\n",
       "      <th>bool2</th>\n",
       "      <th>float64</th>\n",
       "      <th>int64</th>\n",
       "      <th>tdeltas</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1 days</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1 days</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bool1  bool2  float64  int64  tdeltas\n",
       "0   True  False      4.0      1      NaT\n",
       "1  False   True      5.0      2   1 days\n",
       "2   True  False      6.0      3   1 days"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.select_dtypes(include=['number', 'bool'], exclude=['unsignedinteger'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果要选择字符串类型的列，必须使用object类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>string</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  string\n",
       "0      a\n",
       "1      b\n",
       "2      c"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.select_dtypes(include=['object'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "如果想要知道某种数据类型的所有子类型，比如numpy.number类型，你可以定义如下的方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def subdtypes(dtype):\n",
    "        subs = dtype.__subclasses__()\n",
    "        if not subs:\n",
    "            return dtype\n",
    "        return [dtype, [subdtypes(dt) for dt in subs]]\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[numpy.generic,\n",
       " [[numpy.number,\n",
       "   [[numpy.integer,\n",
       "     [[numpy.signedinteger,\n",
       "       [numpy.int8,\n",
       "        numpy.int16,\n",
       "        numpy.int32,\n",
       "        numpy.int32,\n",
       "        numpy.int64,\n",
       "        numpy.timedelta64]],\n",
       "      [numpy.unsignedinteger,\n",
       "       [numpy.uint8,\n",
       "        numpy.uint16,\n",
       "        numpy.uint32,\n",
       "        numpy.uint32,\n",
       "        numpy.uint64]]]],\n",
       "    [numpy.inexact,\n",
       "     [[numpy.floating,\n",
       "       [numpy.float16, numpy.float32, numpy.float64, numpy.float64]],\n",
       "      [numpy.complexfloating,\n",
       "       [numpy.complex64, numpy.complex128, numpy.complex128]]]]]],\n",
       "  [numpy.flexible,\n",
       "   [[numpy.character, [numpy.string_, numpy.unicode_]],\n",
       "    [numpy.void, [numpy.record]]]],\n",
       "  numpy.bool_,\n",
       "  numpy.datetime64,\n",
       "  numpy.object_]]"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdtypes(np.generic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}