In the realm of computer vision, object detection plays a crucial role in identifying and locating objects within an image or video. This blog delves into the exciting world of serverless computing and explores how AWS Lambda can be harnessed to execute object detection tasks using the powerful YOLO model, implemented in .NET.

Why Serverless and AWS Lambda?

Serverless computing offers a compellign paradigm shift, allowing developers to focus on application logic without the burden of server management. AWS Lambda a serverless compute service by Amazon Web Services (AWS), executes code in response to events. This makes it a perfect fit for object detection tasks, where the workload spikes with incoming images.

YOLO: The Object Detection Powerhouse

You Only Look Once (YOLO) is a family of real-time object detection algorithms renowned for their speed and accuracy. Pre-trained YOLO models are readily available, making it easier to get started with object detection.

Building the Serverless Object Detection Pipeline

Here's a breakdown of the key steps involved in constructing our serverless object detection pipeline:

Pre-requisites:

An AWS account with necessary permissions
A pre-trained YOLO model (compatible with .NET) downloaded and stored in an S3 bucket
Familiarity with C# and AWS Lambda development

Setting up the Lambda Fuction:

AWS Amazon.Lambda.Templates .NET project templates

To generate Lambda function code, use the Amaz on.Lambda.Templates NuGet package. To install this template package, run the following command:

dotnet new install Amazon.Lambda.Templates

AWS Amazon.Lambda.Tools .NET Global CLI tools

To create your Lambda functions, you use the Amazon.Lambda.Tools .NET Global Tools ex tension. To install Amazon.Lambda.Tools, run the following command:

dotnet tool install -g Amazon.Lambda.Tools

Creating .NET projects using the .NET CLI

dotnet new lambda.EmptyFunction --name ObjectDetectionLambda

This command creates the following files and directories in your project directory.

└── ObjectDetectionLambda
    ├── src
    │   └── ObjectDetectionLambda
    │       ├── Function.cs
    │       ├── Readme.md
    │       ├── aws-lambda-tools-defaults.json
    │       └── ObjectDetectionLambda.csproj
    └── test
        └── ObjectDetectionLambda.Tests
            ├── FunctionTest.cs
            └── ObjectDetectionLambda.Tests.csproj

Under the src/ObjectDetectionLambda directory, examine the following files:

aws-lambda-tools-defaults.json: This is where you specify the command line options when deploying your Lambda function. For example:

    "profile" : "default",
    "region" : "us-east-1",
    "configuration" : "Release",
    "function-architecture": "x86_64",
    "function-runtime":"dotnet8",
    "function-memory-size" : 256,
    "function-timeout" : 30,
    "function-handler" : "ObjectDetectionLambda::ObjectDetectionLambda.Function::FunctionHandler"

Function.cs: Your Lambda handler function code. It's a C# template that includes the default Amazon.Lambda.Core library and a default LambdaSerializer attribute. For more information on serialization requirements and options, see Serialization in Lambda functions. It also includes a sample function that you can edit to apply your Lambda function code.

  using Amazon.Lambda.Core;

  // Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
  [assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

  namespace ObjectDetectionLambda;

  public class Function
  {

      /// <summary>
      /// A simple function that takes a string and does a ToUpper
      /// </summary≫
      /// <param name="input"></param>
      /// <param name="context"></param>
      /// <returns></returns>
      public string FunctionHandler(string input, ILambdaContext context)
      {
          return input.ToUpper();
      }
  }

ObjectDetectionLambda.csproj: An MSBuild file that lists the files and assemblies that comprise your application.

  <Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
      <TargetFramework>net8.0</TargetFramework>
      <ImplicitUsings>enable</ImplicitUsings>
      <Nullable>enable</Nullable>
      <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
      <AWSProjectType>Lambda</AWSProjectType>
      <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
      <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
      <!-- Generate ready to run images during publishing to improve cold start time. -->
      <PublishReadyToRun>true</PublishReadyToRun>
    </PropertyGroup>
    <ItemGroup>
      <PackageReference Include="Amazon.Lambda.Core" Version="2.2.0" />
      <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.4.0" />
    </ItemGroup>
  </Project>

Readme: Use this file to document your Lambda function.

Under the ObjectDetectionLambda/test directory, examine the following files:

ObjectDetectionLambda.Tests.csproj: As noted previously, this is an MSBuild file that lists the files and assemblies that comprise your test project. Note also that it includes the Amazon.Lambda.Core library, so you can seamlessly integrate any Lambda templates required to test your function.

Install the required nuget packages:

dotnet add package YoloV8

Develop function code to:

Read the image data received from the Lambda event
Load the pretrained YOLO model from the S3 bucket.
Pre-process the image for model input
Perform object detection using the YOLO model
Post-process the results and format the detected objects (bounding boxes, class labels, confidence scores).
Return the processed object detection results in the Lambda fucntion response
Use the results to plot bounding boxes on the image
Save the image in S3

```csharp using Amazon.Lambda.Core; using Amazon.Lambda.S3Events; using Amazon.S3; using Amazon.S3.Model; using Newtonsoft.Json; using Compunet.YoloV8; using Compunet.YoloV8.Plotting; using SixLabors.ImageSharp; using SixLabors.Fonts;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class. [assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace ObjectDetectionLambda;

public class Function { IAmazonS3 S3Client { get; set;}

public Function() { S3Client = new AmazonS3Client(); }

public Function(IAmazonS3 s3Client) { S3Client = s3Client; }

///
/// A simple function that takes a string and does a ToUpper ///
/// The event for the Lambda function handler to process. /// The ILambdaContext that provides methods for logging and describing the Lambda environment. /// public async Task FunctionHandler(S3Event evnt, ILambdaContext context) { context.Logger.LogLine($"Received S3 event: {evnt}");

var eventRecords=evnt.Records??new List();

foreach (var record in evnt.Records) { var s3Event=record.S3; if(s3Event==null) { context.Logger.LogLine("S3 event is null"); continue; } try{ var bucket = record.S3.Bucket.Name; var key = record.S3.Object.Key; var response = await S3Client.GetObjectMetadataAsync(s3Event.Bucket.Name, s3Event.Object.Key); context.Logger.LogLine($"Object {key} in bucket {bucket} is {response.ContentLength} bytes");

//get image from s3 var image = await S3Client.GetObjectAsync(s3Event.Bucket.Name, s3Event.Object.Key); var imageStream = image.ResponseStream; //convert the image to a byte array var imageData = new BinaryReader(imageStream).ReadBytes((int)imageStream.Length);

using var predictor=YoloV8Predictor.Create("/opt/yolov8s.onnx");

var result =await predictor.DetectAsync(imageData);

context.Logger.LogLine($"Detected {result}");

using var imageResult = Image.Load(imageData);

FontCollection collection = new(); FontFamily family = collection.Add("/opt/font/CONSOLA.TTF"); Font font = family.CreateFont(16, FontStyle.Italic);

// Create an instance of DetectionPlottingOptions and set the FontFamily property var plottingOptions = new DetectionPlottingOptions { FontFamily = family };

using var plotted=await result.PlotImageAsync(imageResult,plottingOptions);

// Save the processed image to another S3 folder using (var ms = new MemoryStream()) { plotted.SaveAsJpeg(ms); ms.Position = 0;

var putRequest = new PutObjectRequest { BucketName = s3Event.Bucket.Name, Key = $"inferenced_image/{s3Event.Object.Key}", InputStream = ms, ContentType = "image/jpeg" }; var putResponse=await this.S3Client.PutObjectAsync(putRequest); context.Logger.LogLine($"Uploaded image to S3 with status code {putResponse.HttpStatusCode}"); } } catch(Exception ex) { context.Logger.LogLine($"Error processing object from S3: {ex.Message}"); throw; }

} } }



# Deployment and Testing:

To build your deployment package and deploy it to Lambda, use `Amazon.Lambda.Tools` CLI tools. To deploy the function from the files created in the previous steps.

Navigate into the folder containing the function's `.csproj file`.

```bash
cd ObjectDetectionLambda/src/ObjectDetecitonLambda

Run the following command to deploy the code to Lambda as a .zip deployment package.

dotnet lambda deploy-function ObjectDetectionLambda

During the deployment, the wizard asks you to select a Lambda execution role. For this example, select the lambda_basic_role.

Create an S3 bucket with the following structure:

Triggering the Lambda function:

Consider triggering the Lambda fuction via other AWS services like S3 object upload events, enabling automated object detection on uploaded images.

After adding S3 bucket events as trigger for the Lambda, the bucket should have event notification as shown below

Include the necessary roles and test the project by adding and image to the raw_image folder in the S3 bucket

The result is as shown below:

Additional Considerations

Model Optimization: Since Lambda functions have limitations on memory and execution time, explore model optimization techniques like quantization to reduce model size and improve inference speed.
Error Handling and Logging: Implement robust error handling mechanisms within the Lambda function to gracefully manage potential issues like invalid image data or model loading failures. Utilize AWS CloudWatch for logging function execution and errors.
Security: Secure your Lambda function by enforcing proper IAM policies and access controls.

Conclusion

By leveraging AWS Lambda and the power of YOLO models, you can create a scalable and cost-effective serverless object detection solution. This blog has provided a foundational roadmap to embark on this exciting journey. Remember to explore the vast resources available online for specific YOLO frameworks and .NET development on AWS Lambda.

Fanuel Conrad's Blog