Should You Use AI Builder to Enhance Apps and Flows?

Are you wondering if AI Builder is up to enhancing apps and workflows? Get all the details in this clip from my Pluralsight course!

Start the learning path

Video Summary

Here are the key points from the video:

  • Introduction to AI Builder: The AI Builder is a Power Platform capability that enhances business performance by automating processes and predicting outcomes. It offers a point-and-click experience, making it accessible without coding or data science skills.
  • AI Models Overview: AI Builder includes both pre-built and custom AI models. Pre-built models like business card reader, sentiment analysis, and text recognition are ready to use, while custom models like form processing and object detection require training with specific data.
  • Using Pre-built Models: Pre-built AI models can be easily integrated into Power Automate and Power Apps. Microsoft provides samples and testing options, such as trying out sentiment analysis with custom text to see the results.
  • Training Custom Models: Custom AI models need to be trained with sample data. For example, the object detection model requires at least 15 images per object, but more data improves accuracy. Training ensures the model performs well with real-world data.
  • Demo and Testing: The demo showed how to use AI Builder in a Power Automate flow, like sending an email when a negative tweet is detected. It emphasized the importance of having enough sample data for training and testing AI models effectively.

For more information, read the transcript blog below, or watch the video above!

Video Transcript

Let’s start by doing an introduction to the AI Builder. Let me begin with the introduction directly from Microsoft. The AI Builder is a Power Platform capability that helps you improve business performance by automating processes and predicting outcomes. The AI Builder is a turnkey solution that brings the power of artificial intelligence through a point-and-click experience. If we go beyond the definition, the AI Builder allows you to add intelligence to your applications without coding or data science skills required. The AI Builder can be leveraged in both Power Apps and Power Automate. It’s important to note that the AI Builder is an additional license in the Power Platform, so it’s not included by default. Licensing will change in the future, so always check with your Microsoft licensing professional for the current cost.

In the AI Builder, the artificial intelligence is based on different AI model types that you choose from based on your business needs. An AI model is basically an algorithm that is trained using machine learning to do a specific task. AI models are very specific in what they can do, depending on how you have trained them. The AI Builder contains multiple pre-built AI models and also some that you can create and train on your own. If we do a quick overview of some of the pre-built AI models, we have the business card reader, which allows you to automatically process business card information. We also have an entity extraction model, a keyphrase extraction model, a language detection model, a sentiment analysis model, and a text recognition model, which allows you to extract text from images.

We also have four types of custom AI models, which are a bit generic and need you to train them for a specific task with your specific content. We have a form processing model, which allows you to read documents and process and save important information from that document. We have an object detection model, which allows you to recognize objects in images as well as count them. We have a text classification model, which allows you to categorize text by meaning. Finally, we have a prediction model, which allows you to predict whether something will happen.

Now that we have seen the models, what are some examples we could use them for? Let’s take the form processing model. We could train it to process invoices sent by email, get them approved with Power Automate, and save important information in the payment system. With the object detection model, we could implement a Power App where a field worker could take a picture of a shelf and let the AI Builder calculate the inventory and products in the picture. With the sentiment analysis model, we could use Power Automate to get all of the Twitter mentions of our brand, filter out the ones that have a negative sentiment, and send them to the marketing manager for personal follow-ups.

Now that we know what the AI models are and we looked at some ideas on how they could be used, how do we actually integrate them into our applications? For the pre-built AI models, most of the work is actually done for you as the model is pre-built and ready. You can just use it inside Power Automate and Power Apps. For pre-built models, Microsoft provides samples in either Power Automate, Power Apps, or both, so you can take a look at an example to see how it’s built with Microsoft’s best practices and then get inspired to build your own. For some models, you can even test them before deploying, such as the sentiment analysis one. You can pick from a pre-built Microsoft text or add your own text and see what the result would be.

With custom AI models, it will take a bit more time as the model must be trained before it can be used. To train a model, you will need to provide it with enough sample data so it can learn how to interpret it. Each type of AI model has its own training requirements. For example, for the object detection model, you need to provide a minimum of 15 images of each object, but the recommendation is 50 images of each object. One thing is for sure: the more sample data you provide, the more trained the model is, and the better results you will have.

Now that we have seen the theory, let’s go over to the lab environment and explore the AI Builder, as well as see how we can use a pre-built AI model in a Power Automate flow. I am now back in the live environment. Let me open up the browser here where I’m back in the Power Apps maker portal. On the left side, I will go under the AI Builder category and then under “Build.” You will see that the first thing we see on our screen are the custom AI models, the ones that we actually need to train. These include category classification, entity extraction, form processing, object detection, and prediction. You can click on each one of them, and then you have examples over here. For example, if I go back to object detection, I see examples that really show me what this model can achieve, such as identifying products from an image.

Now let me close this and scroll down a bit lower. Here, I will see all of the pre-built AI models, the ones I can use right away. They are in the “Get straight to productivity” category. At the time of recording this course in October 2021, there were 11 different pre-built AI models, but Microsoft might add some new ones in the future. Let’s go and test one of them. I’ll go into one that’s really easy to show, which is the sentiment analysis model. One of the cool things about many pre-built AI models is that I can go and just try it out. I can either pick one of the Microsoft pre-built sentences or add my own. For example, let’s put “Vlad is the best” as a test and analyze the text. This is positive, as you see because it’s a positive sentence. But if I say “The website is slow,” this will probably be a negative one because the website being slow is not a good thing. You can really put in your text, test it out, and see what the result would be.

At the bottom, you see for the sentiment analysis model, we actually have the option to use it in a flow or an app. When we click on “Use in a flow,” for example, it will bring us to Power Automate and show us all of the different templates that have this AI Builder model. Let’s take a look at a really easy one: send an Outlook email when a negative tweet is posted as per AI Builder. Let’s click on it. It will show me what the trigger is. It will use Dataverse and Outlook. Let’s click on “Continue,” and what this will do is allow me to create a very quick flow where it will search for all of the text with the Global Mantics hashtag, take a look at the language, and if the sentiment is negative, it will send an email to the person I chose, saying, “Hey, there’s a tweet that’s negative.”

If you want to try out the AI Builder, it’s super easy to try it out yourself using the pre-built AI models. If you want something more advanced, such as form processing, you will need to train the model first. For this one, you’ll need at least five documents with the same layout to train it, but I usually recommend having at least 10-15 examples because the more data you give it, the better the AI model will be trained. Now, I know that not all of us have all of those examples ready, especially for a lab environment. So, let’s end this demo in which we have explored the AI Builder and how you can easily test it out in a Power Automate flow. Let’s go back to the slides and learn how you can get sample data to test out the AI Builder in your own environment without necessarily using production data.