Pandas DataFrame isna() and isnull() Method – Be on the Right Side of Change (original) (raw)

Preparation

Before any data manipulation can occur, two (2) new libraries will require installation.

The Pandas library enables access to/from a DataFrame.
The NumPy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.

To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.

$ pip install pandas

Hit the <Enter> key on the keyboard to start the installation process.

$ pip install numpy

Hit the <Enter> key on the keyboard to start the installation process.

If the installations were successful, a message displays in the terminal indicating the same.

Feel free to view the PyCharm installation guide for the required libraries.

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import pandas as pd import numpy as np

The DataFrame isna() and isnull() methods return Boolean (True/False) values in the same shape as the DataFrame/Series passed. If any empty values are of the following type, they will resolve to True.

None
NaN
NaT
NA

All other values (valid data) will resolve to False.

💡 Note: Any empty strings or numpy.inf are not considered empty unless use_inf_as_na is set to True.

The syntax for these methods is as follows:

DataFrame.isna() DataFrame.isnull()

Parameters – These methods contain no parameters.

For this example, three (3) temperatures over three (3) days for Anchorage, Alaska, save to a DataFrame. Unfortunately, some temperatures did not accurately record.

The code below returns a new DataFrame containing True values in the same position as the missing temperatures and False in the remainder.

df_temps = pd.DataFrame({'Day-1': [np.nan, 11, 12], 'Day-2': [13, 14, pd.NaT], 'Day-3': [None, 15, 16]}, index=['Morning', 'Noon', 'Evening']) print(df_temps)

result = df_temps.isna() print(result)

Line [1] creates a dictionary of lists and saves it to df_temps.
Line [2] outputs the DataFrame to the terminal.
Line [3] uses isna() to set the empty values (np.nan, pd.NaT, None) to True and the remainder (valid values) to False. This output saves to the result variable.
Line [4] outputs the result to the terminal.

Output

original df_temps

| | Day-1 | Day-2 | Day-3 | | | ------- | ------- | ------- | ------- | | Morning | NaN | 13 | NaN | | Noon | 11.0 | 14 | 15.0 | | Evening | 12.0 | NaT | 16.0 |

result

| | Day-1 | Day-2 | Day-3 | | | ------- | -------- | -------- | -------- | | Morning | True | False | True | | Noon | False | False | False | | Evening | False | True | False |

df_temps = pd.DataFrame({'Day-1': [np.nan, 11, 12], 'Day-2': [13, 14, pd.NaT], 'Day-3': [None, 15, 16]}, index=['Morning', 'Noon', 'Evening']) print(df_temps)

result = df_temps.isnull() print(result)

Line [1] creates a dictionary of lists and saves it to df_temps.
Line [2] outputs the DataFrame to the terminal.
Line [3] uses isnull() to set the empty values (np.nan, pd.NaT, None) to True and the remainder (valid values) to False. This output saves to the result variable.
Line [4] outputs the result to the terminal.

Output

original df_temps

| | Day-1 | Day-2 | Day-3 | | | ------- | ------- | ------- | ------- | | Morning | NaN | 13 | NaN | | Noon | 11.0 | 14 | 15.0 | | Evening | 12.0 | NaT | 16.0 |

result

| | Day-1 | Day-2 | Day-3 | | | ------- | -------- | -------- | -------- | | Morning | True | False | True | | Noon | False | False | False | | Evening | False | True | False |

💡 Note: The isnull() method is an alias of the isna() method. The output from both examples is identical.

More Pandas DataFrame Methods

Feel free to learn more about the previous and next pandas DataFrame methods (alphabetically) here:

Also, check out the full cheat sheet overview of all Pandas DataFrame methods.