Download
Introduction to Intel® Perceptual Computing SDK: HelloCam [PDF 305 KB]
When you want to learn a new programming language or SDK, the first step is usually is to write a program called "Hello World!" Here, we are talking about how to use the web cam features of the Intel® Perceptual Computing SDK, so we will call our first program "Hello Cam!"
But first, a bit of architecture. The following figure shows the architectural model of the Intel Perceptual Computing SDK:
Figure 1. Intel Perceptual Computing SDK Architecture
In this article we will build an example using the functionality provided by .NET porting of Perceptual SDK (shown in red in Figure 1).
In particular, we will use the classes inside libpxcclr.dll, which is located in the folders:
$(PCSDK_DIR)binwin32 $(PCSDK_DIR)binx64
This library is a wrapper for the C++ classes and functionalities by exposing them to the .NET world (both C# and VB.NET).
The Windows Form project
To begin, we open Visual Studio* 2012 and create a new Windows Forms project as shown in the following figure.
Once the project is created, we need to add a reference to the dll using the command "Add Reference ..."
In this project, we will use the features made available by the class UtilMPipeline. This class works as a bridge between the .NET world and class UtilPipeline (written in C++ and documented here http://software.intel.com/sites/landingpage/perceptual_computing/documentation/html/utilpipeline.html) . This class is the easiest way to access the functionalities offered by the SDK.
In this sample, we will just acquire images from the web cam and show them in our GUI. In particular, we implement our own class that extends the UtilMPipeline.
Public Class ImageCamPipeline
Inherits UtilMPipeline
Public Sub New()
MyBase.New()
EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_RGB24, 640, 480)
End Sub
End Class
The class constructor enables us to use the web cam module of the base class (in our example the image format is RGB 24 bit and the size is 640 by 480 pixels) called the EnableImage method. We will see later the other possibilities offered by EnableImage method and by the enumeration PXCMImage.ColorFormat.
First of all, we can note that all "managed" classes (.NET world) have the letter M in between the prefix that indicates the area of use and a suffix that indicates the functionalities. For example, the PXCMImage is the .NET class that implements the interface PXCImage of the Perceptual Computing framework.
To manage the image flow from the web cam and show them to our GUI, we need to override the OnImage method of UtilMPipeline.
Public Overrides Sub OnImage(image As PXCMImage)
' retrieve the active session
Dim session = Me.QuerySession()
If session IsNot Nothing Then
Dim bitmap As Bitmap
' retrieve bitmap from webcam
Dim pcmStatus = image.QueryBitmap(session, lastProcessedBitmap)
If pcmStatus = pxcmStatus.PXCM_STATUS_NO_ERROR Then
' Create a new bitmap and fire event to GUI
bitmap = New Bitmap(lastProcessedBitmap)
RaiseEvent ImageCaptured(Me, New ImageCapturedEventArgs() With {.Image = bitmap,
.TimeStamp = DateTime.Now,
.FPS = CalculateFPS(.TimeStamp)})
End If
End If
End Sub
ImageCamPipeline fires the ImageCaptured event when a new image is ready and the GUI can manage the event to show the image.
Public Event ImageCaptured(sender As Object, e As ImageCapturedEventArgs)
In our sample, the event argument contains the image, the capture timestamp (date and time) and the instantaneous frame rate (calculated as the difference between the timestamps of two consecutive images).
Public Class ImageCapturedEventArgs
Inherits EventArgs
Public Property Image As Bitmap
Public Property TimeStamp As DateTime
Public Property FPS As Integer?
End Class
The instantaneous frame rate is calculated from the difference of timestamps of two consecutive images using the following function:
Private Function CalculateFPS(currentTimeStamp As Date) As Integer?
Dim fps As Integer? = Nothing
If LastImageTimestamp.HasValue Then
fps = CInt(Math.Floor(1000D / (currentTimeStamp - LastImageTimestamp.Value).TotalMilliseconds))
End If
Me.LastImageTimestamp = currentTimeStamp
Return fps
End Function
Once we have created the class that interacts with the SDK functionality, we can begin to create the interface that will display the images and the frame rate.
The following figure shows the Visual Studio designer for our form:
The code behind of the form is quite simple:
Public Class Form1
Private pipeline As ImageCamPipeline
Private Sub Form1_FormClosing(sender As Object, e As FormClosingEventArgs) Handles Me.FormClosing
pipeline.Dispose()
End Sub
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles Me.Load
pipeline = New ImageCamPipeline
AddHandler pipeline.ImageCaptured, AddressOf ImageCapturedHandler
pipeline.LoopFrames()
End Sub
Private Sub ImageCapturedHandler(sender As Object, e As ImageCapturedEventArgs)
UpdateInterface(e)
End Sub
Private Delegate Sub UpdateInterfaceDelegate(e As ImageCapturedEventArgs)
Public Sub UpdateInterface(e As ImageCapturedEventArgs)
If Me.InvokeRequired Then
Me.Invoke(New UpdateInterfaceDelegate(AddressOf UpdateInterface), e)
Else
pctImage.Image = e.Image
If e.FPS.HasValue Then
lblFPS.Text = String.Format("{0} fps", e.FPS.Value)
Else
lblFPS.Text = String.Empty
End If
End If
End Sub
End Class
The form has an instance of ImageCamPipeline that is created in the load form event. Once we have created the ImageCamPipeline's instance, we add the event handler for ImageCaptured and finally we call the LoopFrames method to start the image capture.
The LoopFrames method, as we can read in the official documentation (http://software.intel.com/sites/landingpage/perceptual_computing/documentation/html/loopframes_util pipeline.html), starts a loop that acquires and releases frames. Reading the official documentation, we see that the pseudo-code executed by the method is as follows:
if (!Init()) return false;
for (;;) {
// some device hot-plug code omitted.
if (!AcquireFrame(true)) break;
if (!ReleaseFrame()) break;
}
Close();
return true
This class performs three main steps. It:
- is initialized (Init method)
- runs an endless loop in which the frame is retrieved from the cam and, subsequently, released (AcquireFrame and ReleaseFrame methods)
- closes the connection with the device (Close method)
LoopFrames blocks the thread that calls it, and in our interface, this means that the GUI is unable to start because the LoopFrames method is called in the Load event handler that takes place in UI main thread.
To solve the problem, we can override the LoopFrames method so that the LoopFrames method of the base class is run in a new thread. The new thread won't block the main thread and our UI can start.
Public Overrides Function LoopFrames() As Boolean
If LoopFramesThread Is Nothing Then
LoopFramesThread = New Thread(New ThreadStart(AddressOf LoopFramesCode))
LoopFramesThread.Start()
Return True
Else
Return False
End If
End Function
Private LoopFramesThread As Thread
Private Sub LoopFramesCode()
Try
RaiseEvent PipelineStarted(Me, EventArgs.Empty)
Dim result = MyBase.LoopFrames()
If Not result Then RaiseEvent PipelineError(Me, EventArgs.Empty)
Catch ex As Exception
RaiseEvent PipelineStopped(Me, EventArgs.Empty)
End Try
End Sub
We use Try...Catch in the method to manage the thread abort that may occur when we close the application.
The PipelineStarted, PipelineError e PipelineStopped events allow us to notify the UI when the pipeline starts if an error occurs and the pipeline is stopped.
We might have an error if we try to use a feature not available on the web cam that we are using. For example, if we try to take a depth image and we are not using a Creative* Interactive Gesture Camera.
Redefining the LoopFrames method, our pipeline is multi-threaded and the GUI is fluid and responsive. As the LoopFrames method is performed in a secondary thread, the ImageCaptured event, raised from the pipeline, occurs in a thread that is not the UI main thread.
Since Windows Forms are STAs (single-threaded apartments), we are forced to update our interface using the InvokeRequired property plus the Invoke method (UpdateInterface method). The sample is quite simple, but it allows you to see how the Perceptual Computing SDK works.
One of the interesting features of the Perceptual Computing SDK is that certain features will work even if you are not using a Creative Cam. For example, if you retrieve RGB images from the web cam, you can run the code also using your PC’s web cam.
With a Creative Cam you can retrieve other types of images (such as Depth image) and, in general, increase the performance. For example, the following figure shows a comparison between the integrated web cam on my Ultrabook™ system and Creative Cam in terms of definition and, especially, of instantaneous frame rate.
On a Creative Cam, we can retrieve the Depth image simply by changing the pipeline constructor:
Public Sub New()
MyBase.New()
EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_DEPTH, 320, 240)
End Sub
The following figure shows the depth image:
The set of values for the PCXMImage.ColorFormat enumeration is available at http://software.intel.com/sites/landingpage/perceptual_computing/documentation/html/index.html?pxcimagecolorformat.html.
Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2013 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.