This post presents a solution to Dunder Data Challenge #5 — Keeping Values Within the Interquartile Range.
All challenges may be worked in a Jupyter Notebook right now thanks to Binder (mybinder.org).
We begin by finding the first and third quartiles of each stock using the
quantile method. This is an aggregation which returns a single value for each column by default. Set the first parameter,
q to a float between 0 and 1 to represent the quantile. Below, we create two variables to hold the first and third quartiles (also known as the 25th and 75th percentiles) and output their results to the screen.
import pandas as pd
stocks = pd.read_csv('../data/stocks10.csv', index_col='date',
>>> lower = stocks.quantile(.25)
>>> upper = stocks.quantile(.75)
Name: 0.25, dtype: float64
Name: 0.75, dtype: float64
We now use the
clip method which trims values in a DataFrame at the given threshold. It has two parameters
upper which can either be a single value or a sequence of values. We set each parameter to the Series containing the appropriate quartile. The
clip method requires that we use the
axis parameter to inform pandas which direction to align the given sequence. We align with the columns.
stocks_final = stocks.clip(lower, upper, axis='columns')
Let’s verify that each column contains the correct values by taking the min and max of each one.
Using one line of code, we can pass the Series containing the quartiles directly to the
This is just for fun, but you can pass the
quantile method a list to return multiple quantiles on each column.
pandas default iteration is over the column names. But, numpy defaults its iteration by row. We can use this knowledge to unpack each of the first two rows as the first two parameters in the
clip method after using the
values attribute to get the numpy array from the DataFrame.
stocks.clip(*stocks.quantile([.25, .75]).values, axis='columns')
If you are looking to completely master the pandas library and become a trusted expert for doing data science work, check out my book Master Data Analysis with Python. It comes with over 300 exercises with detailed solutions covering the pandas library in-depth.