Is SharePoint Content AI Worth It? Real Use Cases + Free Assessment Tool

Is SharePoint Content AI actually worth the investment? In this episode of the Ultimate SharePoint Content AI Deep Dive Series, we break down the real-world ROI of Content AI with practical scenarios, customer stories, and Microsoft’s free assessment tool to help you decide.

Joined again by fellow MVPs Gokan Ozcifci and Drew Madelung, we cover:
✔️ What kinds of problems is SharePoint Content AI best suited to solve
✔️ When it’s not the right tool for the job
✔️ Real customer use cases (including manufacturing + invoice automation)
✔️ Why “pay-as-you-go” pricing doesn’t mean unpredictable costs
✔️ How to use the Microsoft 365 Assessment Tool to identify AI opportunities

🛠️ Learn more about the Microsoft 365 Assessment Tool
📘 Download the free SharePoint Content AI eBook
🎥 Watch the full SharePoint Content AI series

Watch my 100+ courses on Pluralsight

Video Summary

• It’s All About the Use Case
  SharePoint Content AI isn’t a one-size-fits-all solution. Its real value shines when you apply it to specific business problems—like automating document processing, extracting metadata, or assembling content. If you’ve got a clear scenario, Content AI can deliver serious ROI.
• Start with the Microsoft 365 Assessment Tool
  Before diving in, use the free M365 Assessment Tool to scan your environment. It helps identify where Content AI can make the biggest impact, like libraries with custom columns or content types. It’s a great way to find those hidden gems in your tenant.
• Real-World Wins Make It Click
  From replacing outdated invoicing systems to automating document templates, real customer stories show how Content AI can solve long-standing problems quickly and efficiently, especially when paired with pre-built models.
• Training and Adoption Are Key
  Even the best tools need champions. To get the most out of Content AI, you’ll need to invest in training, build internal knowledge, and create adoption campaigns. It’s not just about the tech—it’s about empowering your people.
• Measure Value Through Feedback, Not Just Numbers
  ROI can be tricky to quantify, so use surveys and feedback loops. Ask your users: Has this improved your process? That qualitative insight is often the best indicator of success.

For more information, read the transcript blog below, or watch the video above!

Transcript

Hello everyone, and welcome back to the Ultimate SharePoint Content AI Deep Dive series. Before we deep dive into all the amazing features in future episodes, we really wanted to reserve this first episode to ask probably one of the biggest questions that your decision-makers will have: Is it even worth it?

In this episode, we will spend time looking at multiple scenarios, real-life examples, and even how to use the Microsoft 365 Assessment Tool to discover what processes you could optimize in your Microsoft 365 tenant. In this video, I’m joined again by both Drew Madelung and Gokan Ozcifci, who will be able to share their real-life experiences about SharePoint Content AI. Welcome back to both of you.

Thank you, Vlad. Thank you, and looking forward to talking about Content AI and how it affects your value in your organization. So, starting off thinking about SharePoint content, we are thinking about SharePoint and its about the files and documents that you have in your organization. It’s the files in Teams, the files in SharePoint, the files in OneDrive—they are all valuable information that somehow supports your business in some way. Content AI is a set of features that empowers SharePoint and brings that data value to life.

Now, when you’re thinking about this, SharePoint Content AI is a solution that will solve specific problems. So when you’re looking at deploying and managing Content AI and the different feature sets you’ll see throughout this series, ask this question to yourself: Do I have a problem in my organization or scenario that I would benefit from using these solutions? Some of these solutions have some very complex scenarios, and some of them have some very basic scenarios.

When you’re looking at the business value, Gokan, what do you think? When you’re looking at the business value of Content AI, whenever you extract information with Content AI and put it in your document library, it becomes part of SharePoint. As you said, it’s part of SharePoint, and whenever that data is available for your end users, you can go every side within that diagram and you can use that data to build either low-code applications or pro-code applications, even use that into PnP Search or Microsoft Search, make that data ready for Copilot, or even apply some DLP and labeling from a compliance perspective.

The business value is so high that whenever the data is in SharePoint, you can go anywhere and use and consume that data. Customers just love the fact that stuff they have been forgetting for years comes into SharePoint, and they can use that as a solid foundation to go further. It has no price for them. And that’s what we try to explain with that image—that once it’s available in SharePoint, it can go up anywhere.

Thinking about that, that’s a lot of information, right? You could have millions or hundreds of millions of documents that you might want to extract information from. You might be doing OCR, or you might want to make new documents. Each of those has very specific scenarios that you might have to identify within your organization. And with that, that will lead to some of those challenges.

The first challenge you’re going to think about is all of the solutions, and the value you’re going to have is based on a pay-as-you-go solution. So you need to think about the cost that this is going to take, depending on the type of solution that you’re working on. As you think about the series, each of these will go through what the prices are, what the costs can be, and how to set up pay-as-you-go in the future. But keep those in mind as you’re thinking about: Is SharePoint Content AI worth it? It’s going to be based on the scenarios that you build.

When you deploy this, it will require training. You’ll see in the upcoming videos that there can be some very complex topics in the model generation. It is not something you can simply set. Some of them are easier than others, but it does require end-user training, which means that to get the right value, you have to educate those makers, build champion programs, build the right knowledge in your organization to take advantage of these Content AI solutions—or else it’s just a solution without a problem.

It is not fit for all unstructured data. The scenarios you’re looking for are going to be specific. When you get into content extraction, autofill, using these generative AI solutions, you’re not going to use that in OneDrive. There’s probably not going to be scenarios that you need to just migrate a file share and do it on everything. It’s not necessarily the right use case for Content AI and content extraction. So think of the specific use cases in your world.

Once you start running these, we are in an interesting time now of trust. Do you trust the AI solution? If we’re extracting information, do you trust that data that’s being pulled out? We’re in a “trust but verify” phase. It depends on your scenario—how much you want to trust an LLM to validate invoice costs. Do I make approval based on an LLM extracting that information?

And with that, we don’t know where to start. You can say, “I want to do better content. I want to do better document processing.” That’s great. That’s a great conversation. But how do you even know where to start in your organization?

When you think about this, trying to find that value and where to start, you’re going to look at the higher-value, complexity scenarios that content brings around content extraction and autofill. Those are going to be the core two of: Is SharePoint Content AI worth it? Look at those two. That will be, in our opinion, the highest path to providing value. They will have a higher cost of consumption, but they will give you more value than the other solutions that are there, because it takes more time to set them up.

When you look at things like document translation, annotations, the PDF merge and extract functions—they’re just part of the solution. As they become more complex, you need to spend more time to find that value. And to do that in your organization, you want to know where to start.

One of the things that I talked about at the beginning is that there is a Microsoft 365 Assessment Tool available for you today to scan and look at data in your organization. So instead of going through slides, I’m going to go straight into it and let’s talk about what the M365 Assessment Tool looks like. And it’s free. It is a free tool that you can get out there today.

To start, one of the things you can look at is in the Learn documentation, there is a section under the Syntex section. The materials you’re going to see are still brand new with Syntex. As we talked about in the first video, it has been an evolution, and some of the documentation materials are still not up to date with where we are positioning the content practice or the Content AI solutions. You can start in Learn, and you can see that this does provide documentation around a Syntex assessment tool.

What you’ll do is go to a separate location to grab the documentation, download the tool, decide on an authentication method, configure permissions to run the scan, and then run the Syntex assessment—this becomes the M365 assessment. First, when you download the tool, you’re going to go to a site called the M365 Assessment Tool site. Very specific, with a link to download the tool. They’re as creative as I—it’s incredibly creative.

You will see, as you look through this, that this tool has been enhanced over the years. It was originally built for the Workflow 2013 deprecation to scan your environments and InfoPath services as you’re going through the deprecation. It has since been added into a role for Syntex adoption, our world—Content AI adoption—to figure out where your organization would benefit from that.

Once you download the tool, it’s just an EXE that you download. That’s all it is. Once you download it, you need to configure authentication. Since this is a Graph-based tool that will scan, you will set up an Entra ID. All of the documentation includes how to set everything up. It includes the PowerShell cmdlet to run and register the app. It will require Sites.FullControl.All, as expected, because we’ll be scanning our SharePoint sites. So this is a highly privileged app. I recommend building one just for the scan. It’s fine—you can always delete it after the fact.

Should you be a global admin or maybe an enterprise app admin? What is the minimum role to set this up? A standard user can create the app registration, but to grant permission to the app, you will have to be a global administrator. These are application-level permissions because this app will have access to the entire tenant to scan every SharePoint site.

In my example, here’s my Microsoft Assessment Tool application. I use this for some other scenarios, so I have granted some other permissions to support it. But you’ll see I have Sites. The FullControl application is delegated, depending on how I want to run it. You can run it as a delegated or as an application. From an authentication perspective, it will be a certificate. You want to run a standard Entra app configuration, create a self-signed cert, put it on the device that you want to run it on, upload the certificate into the application, and then the M365 Assessment Tool will use that certificate and authentication to run the assessment.

Now, what it looks like when you run it—great, I’ve now created the app, downloaded the tool, and all we need to do is open the application. You’ll see it has this nice little GUI for us. It looks like Star Wars—it looks great. What I’m going to do is start an assessment, and I’ll also go to one that I’ve already run because it will take some time to run in your tenant. If you have a very large tenant, I would expect this to take quite a bit of time—many hours—with the potential for throttling, because this will use that application. It will finish; it runs in the background. You can close the application.

So, what I do if I want to start something is run a command that simply says I want to start the assessment. The mode is going to be Syntex—we’re not doing the workflow or InfoPath; we’re doing a Syntex assessment. The auth mode is going to be application-specific, so I’m using app permissions. My tenant is “timely.” I have my application ID, which is that Entra ID. I have my cert path—I did a self-signed cert and put it on my machine. This is my path to that cert, and I’m running a Syntex full assessment.

When I execute this, it’s going to connect. You’ll see it start the M365 assessment, and it will begin scanning. You can see my tenant—it started. That’s done. It’s now running. I can check the status and see it running live in my tenant, scanning this information. This is one of my lab environments. We have 337 sites there. You’ll see it slowly go through this content.

Once it’s done, I’m going to see the results of one. I could say “list,” and you’ll see I have one that was finished and one that was running. When one is finished, one sees the start time and the progress. What happens when you’re done is it exports a CSV. You can export a CSV of all the sites, but what it gives you is a Power BI report.

At the beginning here, we have a Syntex assessment report, which is our Content AI assessment report (just pretend that says Content AI). You’ll see we have a large collection of pages and reports. These are used to identify locations in your tenant that have potential uses for Content AI. This can include things like libraries with custom columns. If people are creating custom columns, they’re probably doing something more advanced—a more advanced power user.

What I love is libraries with custom content types. If a power user is making a custom content type, they know what they’re doing—hopefully. If not, this might open up another can of worms for them. But if you think about it, if I have 100,000 SharePoint sites, I could probably get rid of 98% of them by simply looking at something like this and saying, “These are areas of the business where someone has done something beyond standard SharePoint user administration.” That might be useful for the Content AI features we’ll be talking about in the rest of the series.

You will need to talk to these people—do some type of conversation with them to say, “Let’s find out if this will be beneficial for you.” Because answering “Is it worth it?” means answering whether Content AI will be enhanced by using those sites. Then get feedback from them—sites that have custom content types or are very large might be doing transactional processing. They might want to use Content AI. You might have pre-built model candidates based on the names.

We have a video on pre-built models. You have a scan here of libraries where the name matches pre-built models, to know if this might be a prime location. You can go talk to them and educate them on what pre-built models exist. And if you are starting to run models out there, you can see where models are. You can start to use it to track a little bit of information, to say maybe you’ve turned it on, and you have “rogue” Content AI. There’s nothing necessarily wrong with that. You could turn it on, and people start using it. Then you can talk to them and say, “What are you doing in that location? Let’s talk more about the models you’ve built.”

What do you think about this, Gokan?

Pretty amazing. I’m pretty sure a lot of organizations can profit from that and learn about themselves. I’m pretty sure 99% of organizations have no clue who built what, and that tool gives you phenomenal insights. And the last part you showed about AI models—that’s perfect. You can go and talk to them, try to understand why they’ve been building that, the purpose of it, and maybe empower them, help them, promote them. I think it’s phenomenal—honestly, that’s so cool.

So when you’re asking the question, “Is SharePoint Content AI worth it?”—yes, for sure. I don’t think we’d be doing the series if we didn’t believe in it. But there’s more to it, right, Gokan?

Yes, there’s more than simply saying it’s worth it. It takes a little more time. It needs use cases, if you ask me. Within this video series, we’re going to show you a lot of features and how they work. But you’ll never be able to sell that to your customer if there are no use cases attached to it.

Let me give you a few examples of how I could sell Content AI to my customers. One of them, which is the biggest manufacturing company in the Benelux region in Europe, just used macros to create documents with prefilled metadata coming from somewhere. They contacted me and said, “Hey, we heard about Copilot. Can Copilot help us?” I said, “Look, that scenario is exactly what Content AI can do for you with content assembly.” I just have a template of my document, and I just need to fill in a few words coming from a list or managed metadata or other data, and then print that invoice or contract out. They were just mind-blown at how easy that was done with Content AI.

Another example is my customer in the French region who was using Epicor. I’ve had experience with it. It was an invoicing tool they’d been using for the last 10 years. They had no contract anymore, no support anymore. They didn’t know what to do. They contacted me and said, “Hey, can you help us with transforming that into a Power App?” I said, “Yes, we could—but it’s going to cost you a million dollars because it’s too much. Or we can just go for Content AI and build a model that can extract that information for you.” And you know what? We used a pre-built model because all the information they needed came from a pre-built model. So in a few clicks, they were simply amazed. That’s how I could sell Content AI—because I had a use case, there was a problem, and Content AI could help them. We got a fast result. That’s how you could sell the Content AI part in the organization.

So you’re saying content extraction and content assembly require scenarios?

Yes, there are some that don’t. Those are the core ones when you’re looking at “Is SharePoint Content AI worth it?” The answer is yes, when you find those scenarios for the higher-value ones. There are still Content AI features that just work. You want to work with content query, do PDF management, do some taxonomy work—some very basic stuff that’s there.

You do need to have training for this. When you worked with those customers, you had to teach them what content assembly was. You had to educate them on what models are. That’s what we hope to do in this series—we’re going to educate you around those potential technologies that exist and how to use them. But you need to take them into your business. You need to take what we’re sharing with you and build your own training and adoption campaigns in your organization, or link them to our playlist and send them all of this stuff.

Think about this: it is hard to measure ROI. You’re going to have costs against this. You’re not necessarily going to be able to build ROI directly on the Content AI usage that exists. We recommend using things like surveys and feedback. Take that assessment tool after things have happened—you’ve deployed Content AI, you’ve deployed these features—and come back and say, “Let’s talk with them again. Have we improved that invoice process?” That can bring you back to say that the value we’re spending and that consumption is worth it.

And like you said, I think the first step now that we’ve decided it’s worth it is to set up pay-as-you-go, which is a bit of the foundation of it all. Even if you do not go forward, after you set up pay-as-you-go, some features just start working.

So in the next episode of the series, that’s exactly what we are going to cover. You should see the link to it appear on your screen right now. We hope you’re going to go watch it, and thank you so much to Drew and Gokan for sharing your real-life experience and examples on this one. We hope to see you all in the next episode of this series.