Pandas: Sort a multi-indexed DataFrame

Using sort_index method, we can sort a multi indexed data frame.

Let’s experiment with the below data set.

    Year  Quarter  Sales       City
0  2022        1    100  Bangalore
1  2022        3    150  Bangalore
2  2020        2    115  Bangalore
3  2021        3    120  Hyderabad
4  2021        1    180  Hyderabad
5  2021        2     90  Hyderabad
6  2020        1    130    Chennai
7  2022        2    160    Chennai

 

Set the Year and Quarter as multi index columns

df.set_index(['Year', 'Quarter'], inplace=True)

Above snippet convert the data set like below.

               Sales       City
Year Quarter                  
2022 1          100  Bangalore
     3          150  Bangalore
2020 2          115  Bangalore
2021 3          120  Hyderabad
     1          180  Hyderabad
     2           90  Hyderabad
2020 1          130    Chennai
2022 2          160    Chennai

Sort the data by ascending order of index columns

new_df = df.sort_index()

 

new_df points to below data set.

               Sales       City
Year Quarter                  
2020 1          130    Chennai
     2          115  Bangalore
2021 1          180  Hyderabad
     2           90  Hyderabad
     3          120  Hyderabad
2022 1          100  Bangalore
     2          160    Chennai
     3          150  Bangalore

 

Sort the data by descending order of index columns

By setting the ascending argument to False, we can sort the data by ascending order of index columns.

new_df = df.sort_index(ascending=False)

 ‘new_df’ point to below data set.

               Sales       City
Year Quarter                  
2022 3          150  Bangalore
     2          160    Chennai
     1          100  Bangalore
2021 3          120  Hyderabad
     2           90  Hyderabad
     1          180  Hyderabad
2020 2          115  Bangalore
     1          130    Chennai

 

Sort the data by ascending order of Year and descending order of Quarter

By passing the list of Booleans to the ascending argument, we can achieve this.

new_df = df.sort_index(ascending=[True,False])

 

List of booleans that we passed to ascending argument are in the order of the multi index columns. In this case, True mapped to Year column and False mapped to the ‘Quarter’ column.

 

‘new_df’ point to below data set.

               Sales       City
Year Quarter                  
2020 2          115  Bangalore
     1          130    Chennai
2021 3          120  Hyderabad
     2           90  Hyderabad
     1          180  Hyderabad
2022 3          150  Bangalore
     2          160    Chennai
1	100  Bangalore

 

Sort the data by descending order of Year and ascending order of Quarter

By passing the list of Booleans to the ascending argument, we can achieve this.

 

new_df = df.sort_index(ascending=[False,True])

 

‘new_df’ point to below data set.

               Sales       City
Year Quarter                  
2022 1          100  Bangalore
     2          160    Chennai
     3          150  Bangalore
2021 1          180  Hyderabad
     2           90  Hyderabad
     3          120  Hyderabad
2020 1          130    Chennai
     2          115  Bangalore

 

Find the below working application.

 

sort_multi_index_data.py

import pandas as pd

# Create a sample DataFrame
data = {'Year': [2022, 2022, 2020, 2021, 2021, 2021, 2020, 2022],
        'Quarter': [1, 3, 2, 3, 1, 2, 1, 2],
        'Sales': [100, 150, 115, 120, 180, 90, 130, 160],
        'City': ['Bangalore', 'Bangalore', 'Bangalore', 'Hyderabad', 'Hyderabad', 'Hyderabad', 'Chennai', 'Chennai']
        }
df = pd.DataFrame(data)
print('Original DataFrame\n', df)

# Set Year and Quarter as indexes
df.set_index(['Year', 'Quarter'], inplace=True)
print('\nAfter setting index columns Year and Quarter\n',df)

# Sort the data by ascending order of the Year and Quarter columns
new_df = df.sort_index()
print('\nSort the data set by ascending order of index columns\n',new_df)

# Sort the data by descending order of the Year and Quarter columns
new_df = df.sort_index(ascending=False)
print('\nSort the data set by descending order of index columns\n',new_df)

# Sort the data by ascending order of the Year and descending order Quarter columns
new_df = df.sort_index(ascending=[True,False])
print('\nSort the data by ascending order of the Year and descending order Quarter columns\n',new_df)

# Sort the data by descending order of the Year and ascending order Quarter columns
new_df = df.sort_index(ascending=[False, True])
print('\nSort the data by descending order of the Year and ascending order Quarter columns\n',new_df)

 

Output

Original DataFrame
    Year  Quarter  Sales       City
0  2022        1    100  Bangalore
1  2022        3    150  Bangalore
2  2020        2    115  Bangalore
3  2021        3    120  Hyderabad
4  2021        1    180  Hyderabad
5  2021        2     90  Hyderabad
6  2020        1    130    Chennai
7  2022        2    160    Chennai

After setting index columns Year and Quarter
               Sales       City
Year Quarter                  
2022 1          100  Bangalore
     3          150  Bangalore
2020 2          115  Bangalore
2021 3          120  Hyderabad
     1          180  Hyderabad
     2           90  Hyderabad
2020 1          130    Chennai
2022 2          160    Chennai

Sort the data set by ascending order of index columns
               Sales       City
Year Quarter                  
2020 1          130    Chennai
     2          115  Bangalore
2021 1          180  Hyderabad
     2           90  Hyderabad
     3          120  Hyderabad
2022 1          100  Bangalore
     2          160    Chennai
     3          150  Bangalore

Sort the data set by descending order of index columns
               Sales       City
Year Quarter                  
2022 3          150  Bangalore
     2          160    Chennai
     1          100  Bangalore
2021 3          120  Hyderabad
     2           90  Hyderabad
     1          180  Hyderabad
2020 2          115  Bangalore
     1          130    Chennai

Sort the data by ascending order of the Year and descending order Quarter columns
               Sales       City
Year Quarter                  
2020 2          115  Bangalore
     1          130    Chennai
2021 3          120  Hyderabad
     2           90  Hyderabad
     1          180  Hyderabad
2022 3          150  Bangalore
     2          160    Chennai
     1          100  Bangalore

Sort the data by descending order of the Year and ascending order Quarter columns
               Sales       City
Year Quarter                  
2022 1          100  Bangalore
     2          160    Chennai
     3          150  Bangalore
2021 1          180  Hyderabad
     2           90  Hyderabad
     3          120  Hyderabad
2020 1          130    Chennai
     2          115  Bangalore

 

Sort at given index level

By specifying ‘level’ argument, we can sort the data frame at specific index level.

 

Example 1: Sort the data set by ascending order of Year column.

 

We can specify the index level to sort by specify the level index or level label.

new_df = df.sort_index(level=0)
new_df = df.sort_index(level='Year')

 

Example 2: Sort the data by descending order of the Quarter.

new_df = df.sort_index(level=1, ascending=False)
new_df = df.sort_index(level='Quarter', ascending=False)

 

Find the below working application.

 

sort_multi_index_specific_level.py

import pandas as pd

# Create a sample DataFrame
data = {'Year': [2022, 2022, 2020, 2021, 2021, 2021, 2020, 2022],
        'Quarter': [1, 3, 2, 3, 1, 2, 1, 2],
        'Sales': [100, 150, 115, 120, 180, 90, 130, 160],
        'City': ['Bangalore', 'Bangalore', 'Bangalore', 'Hyderabad', 'Hyderabad', 'Hyderabad', 'Chennai', 'Chennai']
        }
df = pd.DataFrame(data)
print('Original DataFrame\n', df)

# Set Year and Quarter as indexes
df.set_index(['Year', 'Quarter'], inplace=True)
print('\nAfter setting index columns Year and Quarter\n',df)

# Sort the data set by ascending order of Year column
new_df = df.sort_index(level=0)
print('\nSort the data set by ascending order of Year column\n',new_df)

new_df = df.sort_index(level='Year')
print('\nSort the data set by ascending order of Year column\n',new_df)

# Sort the data by descending order of the Quarter
new_df = df.sort_index(level=1, ascending=False)
print('\nSort the data by descending order of the Quarter\n',new_df)

new_df = df.sort_index(level='Quarter', ascending=False)
print('\nSort the data by descending order of the Quarter\n',new_df)

 

Output

Original DataFrame
    Year  Quarter  Sales       City
0  2022        1    100  Bangalore
1  2022        3    150  Bangalore
2  2020        2    115  Bangalore
3  2021        3    120  Hyderabad
4  2021        1    180  Hyderabad
5  2021        2     90  Hyderabad
6  2020        1    130    Chennai
7  2022        2    160    Chennai

After setting index columns Year and Quarter
               Sales       City
Year Quarter                  
2022 1          100  Bangalore
     3          150  Bangalore
2020 2          115  Bangalore
2021 3          120  Hyderabad
     1          180  Hyderabad
     2           90  Hyderabad
2020 1          130    Chennai
2022 2          160    Chennai

Sort the data set by ascending order of Year column
               Sales       City
Year Quarter                  
2020 1          130    Chennai
     2          115  Bangalore
2021 1          180  Hyderabad
     2           90  Hyderabad
     3          120  Hyderabad
2022 1          100  Bangalore
     2          160    Chennai
     3          150  Bangalore

Sort the data set by ascending order of Year column
               Sales       City
Year Quarter                  
2020 1          130    Chennai
     2          115  Bangalore
2021 1          180  Hyderabad
     2           90  Hyderabad
     3          120  Hyderabad
2022 1          100  Bangalore
     2          160    Chennai
     3          150  Bangalore

Sort the data by descending order of the Quarter
               Sales       City
Year Quarter                  
2022 3          150  Bangalore
2021 3          120  Hyderabad
2022 2          160    Chennai
2021 2           90  Hyderabad
2020 2          115  Bangalore
2022 1          100  Bangalore
2021 1          180  Hyderabad
2020 1          130    Chennai

Sort the data by descending order of the Quarter
               Sales       City
Year Quarter                  
2022 3          150  Bangalore
2021 3          120  Hyderabad
2022 2          160    Chennai
2021 2           90  Hyderabad
2020 2          115  Bangalore
2022 1          100  Bangalore
2021 1          180  Hyderabad
2020 1          130    Chennai

 

 

Previous                                                 Next                                                 Home

OpenCV: recognize faces

In this post, I am going to explain how to recognize faces using OpenCV built-in face recognizer.

 

You can download this example from below link.

https://github.com/harikrishna553/python/tree/main/opencv/1.images/face_recognition

 

 

Train the model with images and respective labels

I am using haar cascade face recognizer to detect the faces in given images.

 

haar_cascade = cv.CascadeClassifier('haarcascade_frontalface_default.xml')

 

Project hierarchy looks like below.

 

I collected some images of Mahendra singh Dhoni, Virender Sehwag and kept them in ‘Dhoni’ and ‘Sehwag’ folder.

 

We need to loop through the images one by one, extract the face coordinates and map the labels to them.

 

Following snippet do the above one.

peoples = ['Dhoni', 'Sehwag']

features = []
labels = []

def populate_face_rect_and_label():
    for person in peoples:
        path = os.path.join('.', person)
        label = peoples.index(person)

        for image in os.listdir(path):
            img_path = os.path.join(path, image)

            image_array = cv.imread(img_path)
            gray_scaled_image = cv.cvtColor(image_array, cv.COLOR_BGR2GRAY)
            faces_rectangle = haar_cascade.detectMultiScale(gray_scaled_image, scaleFactor=1.1, minNeighbors=7)
            for (a, b, c, d) in faces_rectangle:
                face_array = gray_scaled_image[b:b+d, a:a+c]
                features.append(face_array)
                labels.append(label)

Once face coordinates and their respective labels are identified, we need to train the model.

face_recognizer = cv.face_LBPHFaceRecognizer.create()  # Use create() to create the recognizer

features = np.array(features, dtype='object')
labels = np.array(labels, dtype='int')  # Use 'int' instead of 'uint8'

face_recognizer.train(features, labels)

Test the image with trained model

Extract the face coordinates from trained model.

image_to_test = cv.imread('img_to_test.png')
gray_scale_image_to_test = cv.cvtColor(image_to_test, cv.COLOR_BGR2GRAY)
face_rect = haar_cascade.detectMultiScale(gray_scale_image_to_test, scaleFactor=1.3, minNeighbors=5)

For each recognized face in the test image, extract the face coordinates, and predict the label using face recognizer predict method

for (a, b, c, d) in face_rect:
    face = gray_scale_image_to_test[b:b+d, a:a+c]
    label, confidence = face_recognizer.predict(face)
    print(f'Label {peoples[label]} with confidence {confidence}')
    cv.putText(image_to_test, str(f'Label {peoples[label]} with confidence {confidence}'), (50, 50), cv.FONT_HERSHEY_SIMPLEX, 1.0, (255, 0, 255), thickness=4)
    cv.rectangle(image_to_test, (a, b), (a+c, b+d), color=(0, 255, 0), thickness=3)

Find the below working application.

 

face_recognizer.py

import os
import cv2 as cv
import numpy as np

haar_cascade = cv.CascadeClassifier('haarcascade_frontalface_default.xml')

peoples = ['Dhoni', 'Sehwag']

features = []
labels = []

def populate_face_rect_and_label():
    for person in peoples:
        path = os.path.join('.', person)
        label = peoples.index(person)

        for image in os.listdir(path):
            img_path = os.path.join(path, image)

            image_array = cv.imread(img_path)
            gray_scaled_image = cv.cvtColor(image_array, cv.COLOR_BGR2GRAY)
            faces_rectangle = haar_cascade.detectMultiScale(gray_scaled_image, scaleFactor=1.1, minNeighbors=7)
            for (a, b, c, d) in faces_rectangle:
                face_array = gray_scaled_image[b:b+d, a:a+c]
                features.append(face_array)
                labels.append(label)

populate_face_rect_and_label()

face_recognizer = cv.face_LBPHFaceRecognizer.create()  # Use create() to create the recognizer

features = np.array(features, dtype='object')
labels = np.array(labels, dtype='int')  # Use 'int' instead of 'uint8'

face_recognizer.train(features, labels)
# face_recognizer.save('trained_face_recognizer.yml')  # Save the trained model

image_to_test = cv.imread('img_to_test.png')
gray_scale_image_to_test = cv.cvtColor(image_to_test, cv.COLOR_BGR2GRAY)
face_rect = haar_cascade.detectMultiScale(gray_scale_image_to_test, scaleFactor=1.3, minNeighbors=5)

for (a, b, c, d) in face_rect:
    face = gray_scale_image_to_test[b:b+d, a:a+c]
    label, confidence = face_recognizer.predict(face)
    print(f'Label {peoples[label]} with confidence {confidence}')
    cv.putText(image_to_test, str(f'Label {peoples[label]} with confidence {confidence}'), (50, 50), cv.FONT_HERSHEY_SIMPLEX, 1.0, (255, 0, 255), thickness=4)
    cv.rectangle(image_to_test, (a, b), (a+c, b+d), color=(0, 255, 0), thickness=3)

cv.imshow('Face Recognition', image_to_test)
cv.waitKey(0)
cv.destroyAllWindows()

#print(f'Features length: {len(features)}')
#print(f'Labels length: {len(labels)}')

Previous                                                    Next                                                    Home

Reading videos in OpenCV

In this post, I am going to explain how to read videos in OpenCV. Videos in OpenCV are read frame by frame. Follow below step-by-step procedure to read the video.

 

Step 1: Import opencv library.

import cv2 as cv

 

‘cv’ is an alias name given the cv2 module. This alias is used to reference the OpenCV functions and classes throughout the code

 

Step 2: Get an instance of video capture object.

video_capture = cv.VideoCapture('videos/bird.mp4')

 

Using video_capture object, we can read frames from the video, process the frames, and perform various video processing tasks like object tracking, detection, or any other video-related operations.

 

Step 3: Read the frames continuously from the video and display them in a window.

while True:
    # Read the video frame by frame
    # is_true says whether the frame is successfully read or not
    is_true, frame = video_capture.read()

    if is_true == False:
        break

    # Show the frame
    cv.imshow('video', frame)

    # If the letter x is pressed then come out of video
    if (cv.waitKey(20) & 0xFF) == ord('x'):
        break

 

is_true, frame = video_capture.read()

Above statement read single frame from a video source. ‘video_capture.read’ method read single from the given video and return two values. First value ‘is_true’ indicates whether the frame was successfully read or not. Second value ‘frame’ points to the actual frame data that was read from the video file.

 

cv.imshow('bird video', frame)

 

'imshow' method displays the image in a new window. First argument 'bird video' specifies the title of the window in which the frame is displayed. Second argument ‘frame’ specifies the data that you want to display.

 

if (cv.waitKey(20) & 0xFF) == ord('x'):

    break

Above snippet waits for a key press event with a 20-millisecond timeout, checks if the key pressed is 'x', and if it is, it breaks out of the loop.

 

Step 4: Release the video capture resource.

video_capture.release()

 

Once we are done with video file processing, it is important to release the resources properly. Above statement release the resources associated with video capture object.

 

Step 5: Close all the currently open windows.

cv.destroyAllWindows()

 

When you want to close all OpenCV windows and exit your application gracefully,  use destroyAllWindows method. Calling ‘destroyAllWindows’ method prevent resource leakage issues.

 

Find the below working application.

 

read_video.py

import cv2 as cv

video_capture = cv.VideoCapture('videos/bird.mp4')

while True:
    # Read the video frame by frame
    # is_true says whether the frame is successfully read or not
    is_true, frame = video_capture.read()

    if is_true == False:
        break

    # Show the frame
    cv.imshow('bird video', frame)

    # If the letter x is pressed then come out of video
    if (cv.waitKey(20) & 0xFF) == ord('x'):
        break

video_capture.release()

# Close the OpenCV windows
cv.destroyAllWindows()

 

 

Previous                                                    Next                                                    Home

Resize the video in OpenCV

We can resize the video by resizing the video frame by frame.

Following snippet resize the frame or image.

def resize_frame(frame, scale_width=0.5, scale_height=0.5):
    width = int(frame.shape[1] * scale_width)
    height = int(frame.shape[0] * scale_height)
    new_dimensions = (width, height)
    return cv.resize(frame, new_dimensions, interpolation=cv.INTER_AREA)

Find the below working application.

 

resize_video.py

import cv2 as cv

def resize_frame(frame, scale_width=0.5, scale_height=0.5):
    width = int(frame.shape[1] * scale_width)
    height = int(frame.shape[0] * scale_height)
    new_dimensions = (width, height)
    return cv.resize(frame, new_dimensions, interpolation=cv.INTER_AREA)

video_capture = cv.VideoCapture('videos/bird.mp4')

while True:
    # Read the video frame by frame
    # is_true says whether the frame is successfully read or not
    is_true, frame = video_capture.read()

    # You can skip this if you want
    if is_true == False:
        break

    frame = resize_frame(frame, 1.4, 0.7)
    # Show the frame
    cv.imshow('bird video', frame)

    # If the letter x is pressed then come out of video
    if (cv.waitKey(20) & 0xFF) == ord('x'):
        break

video_capture.release()

# Close the OpenCV windows
cv.destroyAllWindows()

How to resize the live video?

Above example works for an existing video. But what about a video stream or capturing video from a device (like a webcam). Following lines are used to set the desired frame width and height for video capture.

 

video_capture.set(3, width)

video_capture.set(4, height)

 

First line sets the width property of the video capture object to the value specified in the width variable. Second line sets the height property of the video capture object to the value specified in the height variable.

Previous                                                    Next                                                    Home