Pandas DataFrame shift() Method – Be on the Right Side of Change (original) (raw)
Preparation
Before any data manipulation can occur, two (2) new libraries will require installation.
- The Pandas library enables access to/from a DataFrame.
- The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.
To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.
$ pip install pandas
Hit the <Enter> key on the keyboard to start the installation process.
$ pip install numpy
Hit the <Enter> key on the keyboard to start the installation process.
If the installations were successful, a message displays in the terminal indicating the same.
FeFeel free to view the PyCharm installation guide for the required libraries.
Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.
import pandas as pd import numpy
The shift() moves the index by a select number of period(s) with an option of setting the time-frequency.
The syntax for this method is as follows:
DataFrame.shift(periods=1, freq=None, axis=0, fill_value=NoDefault.no_default)
| periods | This parameter is the number of periods to shift (positive/negative). |
|---|---|
| freq | Click here to view the frequencies, or navigate to an IDE and run: print(pd.tseries.offsets.__all__) |
| axis | If zero (0) or index is selected, apply to each column. Default is 0 (column). If zero (1) or columns, apply to each row. |
| fill_value | This parameter is the fill value of new missing values. The default value depends on dtype. – Numeric: np.nan. – Datetime/timedelta/period: NaT. – Extension dtypes: self.dtype.na_value. |
This example generates seven (5) random numbers for three (3) daily samples. Running this code shifts the data by one (1) index. The shifted data replaces with the NaN value.
df = pd.DataFrame({'Sample-1': list(np.random.randint(0,100,size=5)), 'Sample-2': list(np.random.randint(0,100,size=5)), 'Sample-3': list(np.random.randint(0,100,size=5))}, index=pd.date_range('2020-01-01', '2020-01-05')) print(df)
result1 = df.shift(periods=1) print(result1)
result2 = df.shift(periods=1, fill_value=0) print(result2)
- Line [1] does the following:
- An index creates based on the start date for five (5) days.
- The frequency changes to
'D'(Daily Frequency). - The output saves to
idx. - Create a DataFrame with five (5) random integers for three (3) samples.
- The index creates based on a specified date range.
- The output saves to
df.
- Line [2] outputs the DataFrame to the terminal.
- Line [3] shifts the data by one (1) period. The first-row data replaces with NaN values. The output saves to
result1. - Line [4] outputs
result1to the terminal. - Line [5] shifts the data by one (1) period and sets the fill value to zero (0). The output saves to
result2. - Line [6] outputs
result2to the terminal.
Output
df
| | Sample-1 | Sample-2 | Sample-3 | | | ---------- | -------- | -------- | -- | | 2020-01-01 | 18 | 85 | 15 | | 2020-01-02 | 27 | 66 | 4 | | 2020-01-03 | 78 | 68 | 5 | | 2020-01-04 | 6 | 77 | 18 | | 2020-01-05 | 94 | 20 | 82 |
**result1**
| | Sample-1 | Sample-2 | Sample-3 | | | ---------- | -------- | -------- | ---- | | 2020-01-01 | NaN | NaN | NaN | | 2020-01-02 | 18 .0 | 85.0 | 15.0 | | 2020-01-03 | 27 .0 | 66.0 | 4.0 | | 2020-01-04 | 78.0 | 68 .0 | 5.0 | | 2020-01-05 | 6 .0 | 77.0 | 18.0 |
The values in the first row now display NaN values.
The last row from the original DataFrame (df) does not display.
**result2**
| | Sample-1 | Sample-2 | Sample-3 | | | ---------- | -------- | -------- | ---- | | 2020-01-01 | 0 | 0 | 0 | | 2020-01-02 | 18 .0 | 85.0 | 15.0 | | 2020-01-03 | 27 .0 | 66.0 | 4.0 | | 2020-01-04 | 78.0 | 68 .0 | 5.0 | | 2020-01-05 | 6 .0 | 77.0 | 18.0 |
The NaN values from result1 change to zero (0).
The last row from the original DataFrame (df) does not display.
More Pandas DataFrame Methods
Feel free to learn more about the previous and next pandas DataFrame methods (alphabetically) here:
Also, check out the full cheat sheet overview of all Pandas DataFrame methods.