Cognitive Service Emotion API In UWP With Azure, XAML And C#

Vijayaragavan S
8y
6.6k
0
2

Article

Before reading this article, please go through the following articles.

Microsoft Cognitive Services let you build apps with powerful algorithms, using just a few lines of code. They work across devices and platforms (such as iOS, Android, and Windows), keep improving, and are easy to set up.

The emotions detected are anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. These emotions are understood to be cross-culturally and universally communicated with particular facial expressions.

Reading this article, you can learn how to create Cognitive Service Emotion API in Universal Windows Apps development with Azure, XAML, and Visual C#. Here, input is a captured image and Emotion API returns different emotions like anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise.

The following important tools are required for developing UWP.

Windows 10 (Recommended).
Visual Studio 2015 Community Edition (It is a free software available online).
Emotion API Key (How To Create Microsoft Cognitive Service Emotion API On Azure Portals).

Now, we can discuss step by step app development.

Step 1

Open Visual Studio 2015. Go to Start -> New Project-> select Universal (under Visual C# -> Windows) -> Blank App -> give your app a suitable name (UWPCoginitiveEmotion) -> OK.

After choosing the target and minimum platform versions that your app will support, the Project creates App.xaml and MainPage.xaml.

Add TextBlock control and change the name and text property for the title.

Step 2

Add Image control and set the name property for storing the captured image.

UWPCoginitiveEmotion

Add a Button control and set the name for taking a photo from stream to image control.

Add a Button control and set the name as Get Emotions.

Add the Click event method for both the button controls.

Add a Textblock control and set the name and text properties for Emotion result.

UWPCoginitiveEmotion

Step 3

Open (double click) the file MainPage.xaml in the Solution Explorer and add the following references in the project.

Microsoft.ProjectOxford.Face
Newtonsoft.Json

For adding Microsoft.ProjectOxford.Face Reference, right click your project (UWPCoginitiveEmotion) and select "Manage NuGet Packages".

Choose "Browse" and search Microsoft.ProjectOxford.Face. Select the package and install it.

Reference is added to your project.

For adding Newtonsoft.Json Reference, choose browse and search Newtonsoft.Json, select the package and install it.

UWPCoginitiveEmotion

Reference added to your project.

Note

Automatically, the following code will be generated in XAML code view, while we are done in the design view.

<Page x:Class="UWPCoginitiveEmotion.MainPage" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:local="using:UWPCoginitiveEmotion" xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="d">
<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<TextBlock x:Name="tblTitle" HorizontalAlignment="Left" Margin="255,98,0,0" TextWrapping="Wrap" Text="UWP Cognitive Service Emotion API Demo" VerticalAlignment="Top" FontSize="36" FontWeight="Bold" />
<Image x:Name="imgEmotion" HorizontalAlignment="Left" Height="304" Margin="408,224,0,0" VerticalAlignment="Top" Width="310" />
<Button x:Name="btnTakePhoto" Content="Take Photo" HorizontalAlignment="Left" Margin="172,548,0,0" VerticalAlignment="Top" FontSize="20" FontWeight="Bold" Click="btnTakePhoto_Click" />
<Button x:Name="btnEmotions" Content="Get Emotions" HorizontalAlignment="Left" Margin="810,548,0,0" VerticalAlignment="Top" Height="39" Width="227" FontSize="20" FontWeight="Bold" Click="btnEmotions_Click" />
</Grid>
</Page>

Step 4

Add the following namespaces in Mainpage.xaml.cs for capturing a photo and getting emotions.

using Windows.Media.Capture;
using Windows.Storage;
using Windows.Graphics.Imaging;
using Windows.UI.Xaml.Media.Imaging;
using Windows.Storage.Streams;
using Microsoft.ProjectOxford.Emotion;
using Microsoft.ProjectOxford.Emotion.Contract;

Step 5

Add the EmotionServiceClient key by using Azure service and generate it (For more information, please refer to the article How To Create Microsoft Cognitive Service Emotion API On Azure Portal).

Add the following code with Azure generated key for EmotionServiceClient.

const string APIKey = "<Your Key>";
EmotionServiceClient emotionserviceclient = new EmotionServiceClient(APIKey);
Add the code
for Emotion Details from the captured image
CameraCaptureUI captureUI = new CameraCaptureUI();
StorageFile photo;
IRandomAccessStream imageStream;
const string APIKey = "<Your Key>";
EmotionServiceClient emotionserviceclient = new EmotionServiceClient(APIKey);
Emotion[] emotionresult;
public MainPage() {
this.InitializeComponent();
captureUI.PhotoSettings.Format = CameraCaptureUIPhotoFormat.Jpeg;
captureUI.PhotoSettings.CroppedSizeInPixels = new Size(200, 200);
}
private async void btnTakePhoto_Click(object sender, RoutedEventArgs e) {
try {
photo = await captureUI.CaptureFileAsync(CameraCaptureUIMode.Photo);
if (photo == null) {
return;
} else {
imageStream = await photo.OpenAsync(FileAccessMode.Read);
BitmapDecoder decoder = await BitmapDecoder.CreateAsync(imageStream);
SoftwareBitmap softwarebitmap = await decoder.GetSoftwareBitmapAsync();
SoftwareBitmap softwarebitmapBGRB = SoftwareBitmap.Convert(softwarebitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
SoftwareBitmapSource bitmapsource = new SoftwareBitmapSource();
await bitmapsource.SetBitmapAsync(softwarebitmapBGRB);
imgEmotion.Source = bitmapsource;
}
} catch {
tblEmotion.Text = "error taking Photo";
}
}
private async void btnEmotions_Click(object sender, RoutedEventArgs e) {
try {
emotionresult = await emotionserviceclient.RecognizeAsync(imageStream.AsStream());
if (emotionresult != null) {
Scores score = emotionresult[0].Scores;
tblEmotion.Text = "Your Emotion are : \n" +
"Happiness: " + score.Happiness + "\n" +
"Sadness: " + score.Sadness + "\n" +
"Surprise: " + score.Surprise + "\n" +
"Neutral: " + score.Neutral + "\n" +
"Anger: " + score.Anger + "\n" +
"Contempt: " + score.Contempt + "\n" +
"Disgust: " + score.Disgust + "\n" +
"Fear: " + score.Fear + "\n";
}
} catch {
tblEmotion.Text = "Error Returning the emotion";
}
}