Recently I wrote about creating AI models using the Azure Custom Vision Service. In these posts I looked at creating and training models, running them online, then finally exporting the models to run on iOS using CoreML and Android using TensorFlow.
Recently Microsoft announced another way to export models - as ONNX models that can be run using Windows ML. Like with CoreML and TensorFlow, these are models that can be run on-device, taking advantage of the power of the devices GPU instead of needing to be run in the cloud.
ONNX models can be exported in the same way as CoreML and TensorFlow - select you iteration, click the Export button to generate the model, then the Doanload button to download it.
To use this model you need a UWP app targeting Build 17110 or higher of the Windows SDK (as this version is the first one containing the Windows ML API). Create a new app, then drag your downloaded model into the
Assets folder. When you add the model, some wrapper classes will be created inside a file in the root directory of your app with a really weird name - they will all start with the Id of your model converted into a format that is C# friendly. The three classes that will be created are for inputs, outputs and running the model - and will end with
It would make sense to rename them, so rename the file to something sensible that reflects your model, then rename the classes as well to
ModelInputis the input to the model and contains a single property called
datawhich is a
VideoFrame. You can populate this with a capture from the camera or a frame from a video feed.
ModelOutputis the outputs of the model as a dictionary of strings to floats. The dictionary is auto-populated with the tags from the model, and when the model is run this will be updated to set the percentages for each tag.
Modelis the class that evaluates the model, binding the inputs and outputs and executing the model.
To create a model, use the static
CreateModel method on
Model, passing in the
<model>.onnx file as a
StorageFile, loaded like this:
var file = await StorageFile.GetFileFromApplicationUriAsync(new Uri($"ms-appx:///Assets/<model>.onnx"));
To run the model, capture a
VideoFrame, set it on an instance of
ModelInput, then pass that to the
EvaluateAsync method on the model. This will return a
ModelOutput with the probabilities set.
You can find a sample of this in the GitHub Azure Samples repo. You can also learn more by checking out the docs.