Leveraging Pandas notnull() to Identify Valid Values

‘notnull’ method is used to check for missing values in a DataFrame.  It return a Boolean mask, where the value is False for the column with missing value and True for the column with some value.

 

Let’s experiment with below data set.

 

Example

       Name  Age       City  Gender  Percentage
0   Krishna   34  Bangalore    Male         NaN
1     Sailu   35       None  Female        76.0
2      Joel   29  Hyderabad    Male        67.0
3     Chamu   35    Chennai  Female       100.0
4  Jitendra   52  Bangalore    Male         NaN
5       Raj   34       None    Male        89.0

 

Example 1: Get all the persons whose City has some information.

city_info_presented = df['City'].notnull()
city_info_presented_rows = df[city_info_presented]

 

Example 2: Get all the persons whose Percentage has some information.

percentage_info_presented = df['Percentage'].notnull()
percentage_info_presented_rows = df[percentage_info_presented]

not_null.py

import pandas as pd
import numpy as np

# Create a sample DataFrame
data = {'Name': ['Krishna', 'Sailu', 'Joel', 'Chamu', 'Jitendra', "Raj"],
        'Age': [34, 35, 29, 35, 52, 34],
        'City': ['Bangalore', None, 'Hyderabad', 'Chennai', 'Bangalore', None],
        'Gender': ['Male', 'Female', 'Male', 'Female', 'Male', 'Male'],
        'Percentage': [np.nan, 76, 67, 100, np.nan, 89]}

df = pd.DataFrame(data)
print('Original DataFrame')
print(df)

print('\nGet all the persons whose City has some information')
city_info_presented = df['City'].notnull()
city_info_presented_rows = df[city_info_presented]
print(city_info_presented_rows)

print('\nGet all the persons whose Percentage has some information')
percentage_info_presented = df['Percentage'].notnull()
percentage_info_presented_rows = df[percentage_info_presented]
print(percentage_info_presented_rows)

Output

Original DataFrame
       Name  Age       City  Gender  Percentage
0   Krishna   34  Bangalore    Male         NaN
1     Sailu   35       None  Female        76.0
2      Joel   29  Hyderabad    Male        67.0
3     Chamu   35    Chennai  Female       100.0
4  Jitendra   52  Bangalore    Male         NaN
5       Raj   34       None    Male        89.0

Get all the persons whose City has some informaiton
       Name  Age       City  Gender  Percentage
0   Krishna   34  Bangalore    Male         NaN
2      Joel   29  Hyderabad    Male        67.0
3     Chamu   35    Chennai  Female       100.0
4  Jitendra   52  Bangalore    Male         NaN

Get all the persons whose Percentage has some information
    Name  Age       City  Gender  Percentage
1  Sailu   35       None  Female        76.0
2   Joel   29  Hyderabad    Male        67.0
3  Chamu   35    Chennai  Female       100.0
5    Raj   34       None    Male        89.0

Previous                                                 Next                                                 Home