Integrating Cortana in your app is basically a four step process.
1. Define Voice Command xml file (VCD file)
2. Install the voice commands defined in VCD file
3. Handle OnActivated method to process voice commands
4. Add Microphone Capability in Package.appxmanifest
Let’s define our VCD file as following:
<?xml version="1.0" encoding="utf-8"?>
<!-- Be sure to use the new v1.1 namespace to utilize the new PhraseTopic feature
-->
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.2">
<!-- The CommandSet Name is used to programmatically access the CommandSet -->
<CommandSet xml:lang="en-us" Name="SetColorCommandSet_en-us">
<!-- The CommandPrefix provides an alternative to your full app name for invocation
-->
<CommandPrefix>Set color</CommandPrefix>
<!-- The CommandSet Example appears in the global help alongside your app
name -->
<Example> Set color red</Example>
<Command Name="ShowCommand">
<Example> Set color Blue </Example>
<ListenFor> {dictatedShowTerms} </ListenFor>
<Feedback> Setting color ... </Feedback>
<Navigate Target="MainPage.xaml" />
</Command>
<Command Name="NaturalLanguageCommand">
<Example> Show me red color</Example>
<ListenFor> {naturalLanguage} </ListenFor>
<Feedback> Showing color... </Feedback>
<Navigate Target="MainPage.xaml" />
</Command>
<PhraseTopic Label="dictatedShowTerms" Scenario="Search">
<Subject> Blue </Subject>
<Subject> Black </Subject>
<Subject> Green </Subject>
<Subject> Red </Subject>
<Subject> Yellow </Subject>
</PhraseTopic>
<PhraseTopic Label="naturalLanguage" Scenario="Natural Language">
<Subject> I want to see Blue color</Subject>
<Subject> Show me Black color</Subject>
<Subject> What about Green color!</Subject>
<Subject> Show Red </Subject>
<Subject> Paint Yellow </Subject>
</PhraseTopic>
</CommandSet>
</VoiceCommands>
Now let’s understand the important elements of this file.
CommandSet
xml:lang="en-us" is how you define culture for the command. one important
thing to notice is that your language, region settings and Speech language much
match to the culture you have defined in VCD file for the command set. For this
article, I have used en-US as the culture and that means that my default language
is set to English (United States).
CommandPrefix
The command prefix is a phrase or set of words that you use to invoke the app. Note
that every command you give must start with this.
Show command
The show command is an example of a normal command you define for Cortana to understand
what to dictate and pass to the application. The ListenFor defines what to listen
for. Whatever you give in curly braces is a PhraseTopic that you can define for
the command. In our case it is dictatedShowTerms and there I have just list down
names of few colors. Note the Scenario is set to Search. You can also define
optional words in the squre braces for ListenFor like following:
<ListenFor> [Show me] {dictatedShowTerms} </ListenFor>
Natural Language command
The natural language as the name suggest, is where the fun begins with Cortana! You
can define natural speaking way of getting voice commands such as “I want to
see Blue color”, “Show red” etc. Note that we convey to treat this PhraseTopic
as natural language by setting the Scenario to Natural Language enum value.
Feedback
Feedback will define what Cortana will talk back to the user upon successful detection
of the voice command.
Now it’s time to write some code to install the VCD file. For this article, I have
used Blank universal windows project template. In the App.xaml.cs file, I have
following method defined:
private async Task InstallVoiceCommandsAsync()
{
var storageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///VoiceCommands.xml"));
await
Windows.ApplicationModel.VoiceCommands.VoiceCommandDefinitionManager
.InstallCommandDefinitionsFromStorageFileAsync(storageFile);
}
This method simply gets file from the package and install it using InstallCommandDefinitionsFromStorageFileAsync
method. We are calling this method in OnLaunched method as following:
protected override async void OnLaunched(LaunchActivatedEventArgs e)
{
……
……
await InstallVoiceCommandsAsync();
}
To process the voice command outputs, we need to define our OnActivated method as
following:
protected override void OnActivated(IActivatedEventArgs args)
{
base.OnActivated(args);
rootFrame = Window.Current.Content as Frame;
// Do not repeat app initialization when the Window already has content,
// just ensure that the window is active
if (rootFrame == null)
{
// Create a Frame to act as the navigation context and navigate to the first page
rootFrame = new Frame();
}
// Place the frame in the current Window
Window.Current.Content = rootFrame;
if (args.Kind == ActivationKind.VoiceCommand)
{
var commandArgs = args as VoiceCommandActivatedEventArgs;
if (commandArgs != null)
{
SpeechRecognitionResult speechRecognitionResult = commandArgs.Result;
var voiceCommandName = speechRecognitionResult.RulePath[0];
var textSpoken = speechRecognitionResult.Text;
switch (voiceCommandName)
{
case "NaturalLanguageCommand":
case "ShowCommand":
if (textSpoken.ToLower().Contains("black"))
{
rootFrame.Navigate(typeof (MainPage), "Black");
}
if (textSpoken.ToLower().Contains("blue"))
{
rootFrame.Navigate(typeof (MainPage), "Blue");
}
if (textSpoken.ToLower().Contains("red"))
{
rootFrame.Navigate(typeof (MainPage), "Red");
}
if (textSpoken.ToLower().Contains("green"))
{
rootFrame.Navigate(typeof (MainPage), "Green");
}
if (textSpoken.ToLower().Contains("yellow"))
{
rootFrame.Navigate(typeof (MainPage), "Yellow");
}
break;
default:
rootFrame.Navigate(typeof (MainPage), "All");
break;
}
}
}
Window.Current.Activate();
}
Here the code is pretty much self-explanatory. We process arguments and based on
the words we have got in result text, we navigate to MainPage with the color
name. Based on this argument, the MainPage will show the color by filling up
the rectangle that we have defined in MainPage.xaml. You can download the code
supplied at the end of this article to see that part of code. Now it’s time to
run our Universal app with Cortana!
Once we run the app for the first time, the vcd will get installed. After that, we
can ask Cortana “What can I say” and it will give us list of all the voice commands
available across the applications. In that we can see our application as following:
Now, we can simply speak “Set color red”. If Cortana understands what we spoke, we
will get dialog similar to following:
We can test our natural language command by speaking “Set color, Show me yellow color”.
Upon successful detection of the command, we will see dialog similar to the following:
You can download source code here:
https://www.dropbox.com/s/b5ievzkowjbkzhe/MyControl.zip?dl=0