Introduction to On-Device AI – Free AI Course (Part 1)

To keep pace with AI’s breakneck speed, software developers are constantly conjuring up innovative ways to ramp up their app game.

This article series is tailored to dive deep into the domain of on-device AI. We’ll be focusing on deploying AI models directly on local devices such as smartphones, tablets, and IoT devices.

Following this series lets you understand how to implement, optimize, and utilize on-device AI to enhance your projects.

Scope and Objectives of the On-Device AI Series

This series covers various aspects of on-device AI, encompassing fundamental principles, advanced techniques, practical applications, and real-world case studies. Our objectives include:

• Fundamentals of On-Device AI: We introduce the basics of on-device AI. We’ll explain its significance and the benefits it brings over cloud-based AI solutions.
• Model Optimization Techniques: We delve into strategies for optimizing AI models for on-device deployment. We’ll then ensure they run efficiently without compromising performance.
• Practical Applications: We provide hands-on examples and tutorials to demonstrate how on-device AI can be applied in different scenarios.
• Device-Specific Considerations: We discuss the unique challenges and considerations when deploying AI models on various devices. This includes smartphones, tablets, and IoT gadgets.
• Iterative Model Refinement: We explore methodologies for refining AI models through continuous evaluation and adjustment. This is to enhance their accuracy and efficiency on specific devices.

The primary goal of this series is to empower you with the knowledge and skills needed to deploy and optimize AI models on local devices effectively.

By the end of this series, you should feel confident in your ability to leverage on-device AI. Also, you’ll be able to create responsive, privacy-preserving, and efficient AI-driven applications.

Whether you’re just starting with on-device AI or looking to refine your existing skills, this series has something to offer for every level of expertise in AI development.

Overview of On-Device AI

On-device AI empowers the deployment and execution of artificial intelligence models directly on local devices like smartphones, tablets, and IoT devices.

This approach offers enhanced privacy, reduced latency, offline functionality, and improved bandwidth efficiency by processing data locally.

These benefits are especially critical in environments with limited or unreliable internet connectivity. They enable AI tasks to be performed without dependence on external networks.

This can be especially advantageous for use cases such as:

• Real-time sensor data processing
• Natural language processing
• Image recognition
• and, Personalized recommendations.

Importance and Benefits of On-Device AI

• Privacy: Data remains on the device. This reduces exposure to privacy breaches since it doesn’t need to be transmitted over the internet.
• Latency: Immediate processing allows for real-time responses without the delays caused by data transmission to and from a server.
• Offline Functionality: AI capabilities are available without an internet connection. This makes applications more reliable in areas with poor or no connectivity.
• Bandwidth: Reduces the need for data transmission. This saves bandwidth and reduces costs, especially in scenarios involving large data volumes.

Applications of On-Device AI

• Real-time Object Detection: Detecting objects like faces, QR codes, and various items in real-time.
• Speech Recognition: Converting spoken language into text locally improves response time and privacy.
• Pose Estimation: Analyzing and predicting human poses in real-time video streams is useful for fitness apps and interactive gaming.
• Image Generation: Creating images from text descriptions, and enhancing creative applications.
• Super Resolution: Enhancing the resolution of images, and improving the quality of visuals.
• Image Segmentation: Dividing an image into segments for detailed analysis is crucial for various applications like medical imaging and autonomous driving.

Overview of Image Segmentation

Image segmentation is the process of partitioning an image into multiple segments or regions. The aim is to simplify representation and make analysis easier.

It plays a crucial role in various computer vision tasks by providing detailed information about the structure and content of an image.

Different Types of Image Segmentation

• Semantic Segmentation: Assigns a class label to each pixel in an image (e.g., road, tree, sidewalk), focusing on identifying categories.
• Instance Segmentation: Differentiating between individual instances of the same class (e.g., identifying each person separately) provides more detailed information.

Applications of Image Segmentation in Real-Time

• Autonomous Driving: Identifying and classifying road elements, vehicles, and pedestrians to enhance safety and navigation.
• Image Editing: Applying effects to specific parts of an image, improving creative and professional workflows.
• Video Conferencing: Blurring backgrounds while focusing on the speaker, enhancing privacy and visual clarity.
• Agriculture: Analyzing drone-captured images for crop and land management, optimizing agricultural practices.

Building a Segmentation Model

Step 1: Importing Necessary Libraries

First, we need to import the essential libraries required for model building and deployment.

# Import the necessary libraries for model building and deployment
from qai_hub_models.models.ffnet_40s import Model  # Import FFNet model
from torchinfo import summary  # Library to print model summary

Step 2: Loading Pre-Trained Weights

Load the pre-trained FFNet model with pre-trained weights and define the input shape for the model.

# Load the pre-trained FFNet model with pre-trained weights
model = Model.from_pretrained()
input_shape = (1, 3, 1024, 2048)  # Define the input shape for the model

# Print the model summary including the number of parameters and multiply-add operations
stats = summary(model, input_size=input_shape, col_names=["num_params", "mult_adds"])
print(stats)

Step 3: Exploring Variations of FFNet

Explore different variants of the FFNet model, both high-resolution and low-resolution versions.

# High-resolution variants of FFNet models
from qai_hub_models.models.ffnet_40s import Model  # Import FFNet 40s model
# from qai_hub_models.models.ffnet_54s import Model  # Uncomment to import FFNet 54s model
# from qai_hub_models.models.ffnet_78s import Model  # Uncomment to import FFNet 78s model

# Low-resolution variants of FFNet models
low_res_input_shape = (1, 3, 512, 1024)  # Define input shape for low-resolution models
# from qai_hub_models.models.ffnet_78s_lowres import Model  # Uncomment to import FFNet 78s low-res model
# from qai_hub_models.models.ffnet_122ns_lowres import Model  # Uncomment to import FFNet 122ns low-res model

# Load the selected model with pre-trained weights
model = Model.from_pretrained()
# Print the model summary for the selected input shape
stats = summary(model, input_size=input_shape, col_names=["num_params", "mult_adds"])
print(stats)

Step 4: Setting Up AI Hub for Device-in-the-Loop Deployment

Configure QAI Hub with the necessary API token for deploying the model.

# Import the QAI Hub library for deployment
import qai_hub
from utils import get_ai_hub_api_token  # Utility function to get AI Hub API token

# Configure QAI Hub with the API token
ai_hub_api_token = get_ai_hub_api_token()
!qai-hub configure --api_token $ai_hub_api_token

Step 5: Running the Model on a Real Smartphone

Select a random device from the list of supported devices and print the selected device.

# Select a random device from the list of supported devices
devices = [
    "Samsung Galaxy S22 Ultra 5G",
    "Samsung Galaxy S22 5G",
    "Samsung Galaxy S22+ 5G",
    "Samsung Galaxy Tab S8",
    "Xiaomi 12",
    "Xiaomi 12 Pro",
    "Samsung Galaxy S22 5G",
    "Samsung Galaxy S23",
    "Samsung Galaxy S23+",
    "Samsung Galaxy S23 Ultra",
    "Samsung Galaxy S24",
    "Samsung Galaxy S24 Ultra",
    "Samsung Galaxy S24+",
]

import random  # Import random module to randomly select a device
selected_device = random.choice(devices)  # Select a device randomly
print(selected_device)  # Print the selected device

Step 6: On-Device Demonstration

Run the FFNet model demonstration on the selected device.

# Run the FFNet model demonstration on the selected device
%run -m qai_hub_models.models.ffnet_40s.demo -- --device "$selected_device" --on-device

Final Thoughts

Deploying AI models on devices opens up numerous possibilities for real-time applications across various industries. By leveraging on-device AI, we can achieve higher privacy, lower latency, and enhanced user experiences.

The FFNet model is highly efficient and performing. It serves as an excellent example of how sophisticated AI models can be deployed on local devices.

In the next article, we’ll delve into Preparing for On-Device Deployment. We’ll explore the key considerations and steps that ensures the AI models are optimized and ready for efficient deployment on various devices.

This will include best practices for:

• Model optimization
• Compatibility checks
• and, Performance testing

We’ll ensure a seamless and effective on-device AI experience! Stay tuned for insights on how to maximize the potential of your AI applications in real-world scenarios!