Pandas

Creating, Reading and Writing

There are two core objects in pandas: the DataFrame and the Series.

pd.DataFrame({'Bob': ['I liked it.', 'It was awful.'], 'Sue': ['Pretty good.', 'Bland.']}, index=['Product A', 'Product B']) pd.Series([30, 35, 40], index=['2015 Sales', '2016 Sales', '2017 Sales'], name='Product A')

Reading data

wine_reviews = pd.read_csv("../input/wine-reviews/winemag-data-130k-v2.csv", index_col=0) wine_reviews.head()

index_col=0：使用数据集自带索引

wine_reviews.shape

Indexing, Selecting & Assigning

reviews.country reviews['country']

Index-based selection

reviews.iloc[:, 0] reviews.iloc[[0, 1, 2], 0] reviews.iloc[-5:] # 最后五个元素

Label-based selection

reviews.loc[0, 'country'] reviews.loc[:, ['taster_name', 'taster_twitter_handle', 'points']] df.loc['Apples':'Potatoes']

注意iloc是左闭右开，loc是左闭右闭

Manipulating the index

reviews.set_index("title")

Conditional selection

reviews.country == 'Italy' # This operation produced a Series of True/False booleans based on the country of each record. reviews.loc[(reviews.country == 'Italy') & (reviews.points >= 90)] reviews.loc[(reviews.country == 'Italy') | (reviews.points >= 90)] reviews.loc[reviews.country.isin(['Italy', 'France'])] reviews.loc[reviews.price.notnull()] # 常用于筛选缺失值，还有一个isnull()

Assigning data

reviews['critic'] = 'everyone' reviews['index_backwards'] = range(len(reviews), 0, -1) 0 129971 1 129970 ... 129969 2 129970 1 Name: index_backwards, Length: 129971, dtype: int64

统计信息与map操作

Summary functions

reviews.points.describe() count 129971.000000 mean 88.447138 ... 75% 91.000000 max 100.000000 Name: points, Length: 8, dtype: float64 reviews.taster_name.describe() count 103727 unique 19 top Roger Voss freq 25514 Name: taster_name, dtype: object reviews.points.mean() reviews.taster_name.unique() reviews.taster_name.value_counts()

map和apply map返回series，apply返回dataframe

reviews.points.map(lambda p: p - review_points_mean) def remean_points(row): row.points = row.points - review_points_mean return row reviews.apply(remean_points, axis='columns')

有时也可以直接操作

reviews.points - review_points_mean reviews.country + " - " + reviews.region_1

Grouping and Sorting

Groupwise analysis

reviews.groupby('points').price.min() reviews.groupby('winery').apply(lambda df: df.title.iloc[0]) reviews.groupby(['country']).price.agg([len, min, max])

Multi-indexes

countries_reviewed = reviews.groupby(['country', 'province']).description.agg([len]) countries_reviewed.reset_index() #复位为单索引结构

Sorting sort_values() defaults to an ascending sort, where the lowest values go first.

countries_reviewed.sort_values(by='len', ascending=False) countries_reviewed.sort_values(by=['country', 'len'])

To sort by index values, use the companion method sort_index(). This method has the same arguments and default order:

countries_reviewed.sort_index()

Data Types and Missing Values

The data type for a column in a DataFrame or a Series is known as the dtype.

reviews.price.dtype reviews.dtypes

格式转换

reviews.points.astype('float64')

Missing data

reviews[pd.isnull(reviews.country)] reviews.region_2.fillna("Unknown") reviews.taster_twitter_handle.replace("@kerinokeefe", "@kerino")