AI Volume Control Using Python and OpenCV

Description:-

Gesture recognition is a technology that enables machines to interpret and understand human gestures through machine vision, image processing, and computer vision techniques. This technology can recognize hand gestures, facial expressions, and other movements. It can be used in virtual reality applications, gaming, and robotics.

Gesture recognition can improve the user experience in various applications, allowing for a more natural way of interacting with a computer system. It can also be used in security systems, allowing for more accurate and secure identification of people.

NumPy is a Python programming language library that adds support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.

What is MediaPipe?

Mediapipe is an open-source machine learning library of Google, which has some solutions for face recognition and gesture recognition and provides encapsulation of Python, js, and other languages. MediaPipe Hands is a high-fidelity hand and finger tracking solution. It uses machine learning (ML) to infer 21 key 3D hand information from just one frame. We can use it to extract the coordinates of the key points of the hand.

Hand Landmark Model Bundle

The hand landmark model bundle detects the key point localization of 21 hand-knuckle coordinates within the detected hand regions. The model was trained on approximately 30K real-world images and several rendered synthetic hand models imposed over various backgrounds. The definition of the 21 landmarks is given below.

The aim of this tutorial is to help you understand how to develop an interface that will capture human hand gestures dynamically and control the volume level.

Working Principle

We use the camera of our device for this project. The camera detects our hand with points in it so that it can see the distance between our thumb fingertip and the index fingertip. The distance between points 4 and 8 is directly proportional to the volume of the device.

Methodology/Approach

• Detect hand landmarks
• Calculate the distance between thumb tip and index fingertip.
• Map the distance of the thumb tip and index fingertip with volume range. In this case, the distance between the thumb tip and index fingertip was within the range of 30 – 350 and the volume range was from -63.5 – 0.0.
• In order to exit, press ‘Spacebar

Guide to Control Volume Using Hand Gesture

Step 1. Install media pipe library.

Step 2. Import necessary libraries. Also, initialize the mediapipe holistic function.

Step 3. Initialize the webcam capture function.

Step 4. Define mediapipe holistic function

Create an instance of the hand’s module provided by Mediapipe followed by an instance of Mediapipe’s drawing utility. The drawing utility helps to draw those 21 landmarks and the lines connecting those landmarks on your image or frame(which you have noticed in the above video).

Step 5. Real time hand detection

Now, read the webcam value in a while function. Also, convert the image from BGR to RGB.

Step 6. Find the location of 21 landmarks on the hand.

Note: This is the new id of the hand connection.

Step 7. In the output, you can see the decimal values. Convert these decimal values into pixels.

Output:-

Step 8. Now, start writing the class to detect hand tip.

Note:

max_num_hands: The number of hands that you want Mediapipe to detect. Mediapipe will return an array of hands and each element of the array(or a hand) would in turn have its 21 landmark points

min_detection_confidence, min_tracking_confidence: When the Mediapipe is first started, it detects the hands. After that, it tries to track the hands as detecting is more time-consuming than tracking. If the tracking confidence goes down the specified value then again it switches back to detection.

All these parameters are needed by the Hands() class, so we pass them to the Hands class in the next line.

Step 9: Define findHands function to find the hand landmark in the image.

Function Arguments: In case, the image has multiple hands, our function would return landmarks for only the specified hand number. A Boolean parameter draw decides if we want the Medapipe to draw those landmarks on our image. The next line does everything. This small line actually is doing a lot behind the scenes and is getting all the landmarks for you.

Step 10: Now, make a function to find the location of hand landmarks.

Step 11. Define the main() function for call class method functions.

Note: Now, our hand landmarks detection part is complete. Check how can you control the computer volume through the index finger tip and thumb tip.

Volume Controller:.Before we write any of our custom code, we need to install an external python package – pycaw. This library will handle the controlling of our system volume.

Steps to Control the Volume with Hand Gestures

Step 1. Make a new Python file & import the necessary libraries.

Step 2. Initialize the webcam capture.

Step 3. Call hand_tracking class

Now, call the hand detector from hand_tracking class with 0.7 with one hand

Step 4. Control audio utilities

Now, define a function to control audio utilities of the computer to interface computer volume with distance b/w index tip and thumb tip.

Step 5. Define some variables related to the volume bar.

Step 6. Read the webcam and filter the fingers by bbox area.

Step 7. Find the distance b/w the thumb and index finger using the findDistance function.

Step 8. Now, set the volume.

Step 9. Draw the rectangle shape of the volume bar and put this shape on the webcam.

Output:-

Conclusion :-

Wrapping up, the tutorial here definitely provided you with insight into creating a hand gesture volume controller. So, next time you are enjoying music while working and want to control the volume of the music. If you are working while listening to your favourite music, you can easily control the volume level of your music with just a gesture of your hand.