Pandas DataFrame dropna() Method – Be on the Right Side of Change (original) (raw)

Preparation

Before any data manipulation can occur, two (2) new libraries will require installation.

The Pandas library enables access to/from a DataFrame.
The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.

To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.

$ pip install pandas

Hit the <Enter> key on the keyboard to start the installation process.

$ pip install numpy

Hit the <Enter> key on the keyboard to start the installation process.

If the installations were successful, a message displays in the terminal indicating the same.

Feel free to view the PyCharm installation guide for the required libraries.

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import pandas as pd import numpy as np

The dropna() method removes missing data from a DataFrame/Series.

The syntax for this method is as follows:

DataFrame.dropna(axis=0, how='any', thresh=None, subset=None, inplace=False)

axis	If zero (0) or index is selected, apply to each column. Default 0.If one (1) apply to each row.
how	Determines when a row/column removes from the DataFrame. The available options are: Any: if any NA values, drop row/column. All: if all NA values, then drop row/column.
thresh	This parameter requires that there many Non-NA values.
subset	This subset is the label(s) along the other axis to include. Must be in an array-like format and contain a list of columns in the subset.
inplace	If set to True, the changes apply to the original DataFrame/Series. If False, the changes apply to a new DataFrame/Series. By default, False.

Note: A list of a few possible empty values are:

'NaN'
pd.NaN
np.nan
None
NaT

In this example, the DataFrame contains some missing data. Therefore, this code will attempt to remove the rows that contain these values.

df = pd.DataFrame({'Data-1': [np.nan, 11, 12], 'Data-2': [13, 14, pd.NaT], 'Data-3': [None, 15, 16]}, index=['Row-1', 'Row-2', 'Row-3']) print(df)

result = df.dropna() print(result)

Line [1] creates a dictionary of lists and saves it to df.
Line [2] outputs the DataFrame to the terminal.
Line [3] removes the rows containing missing values. This output saves to the result variable.
Line [4] outputs the result to the terminal.

Output

| | Data-1 | Data-2 | Data-3 | | | -------- | ------ | ------ | ---- | | Row-1 | NaN | 13.0 | NaN | | Row-2 | 11.0 | 14.0 | 15.0 | | Row-3 | 12.0 | NaT | 16.0 |

result

| | Data-1 | Data-2 | Data-3 | | | -------- | ------ | ------ | ---- | | Row-2 | 11.0 | 14.0 | 15.0 |

💡 Note: Row-2 is the only row that contains valid data and the only row left after applying the dropna() method.

More Pandas DataFrame Methods

Feel free to learn more about the previous and next pandas DataFrame methods (alphabetically) here:

Also, check out the full cheat sheet overview of all Pandas DataFrame methods.