Customizing Microsoft 365 Copilot: Concepts & Terms You NEED to Know

Confused about Copilot terms like agents, declarative agents, and custom engine agents? 🤔 You’re not alone! In this episode of The Ultimate Guide to Customizing Microsoft 365 Copilot, Vlad and Sebastien Levert break down everything you need to know before building your own agents.

Learn how Copilot works, where customization opportunities lie, and the differences between declarative and custom engine agents. From foundational terms to advanced use cases, this video provides the clarity you need to start creating purpose-driven agents for your organization.

Watch my 90+ courses on Pluralsight (opens in a new tab)

Video Summary

• Customization Opportunities: Microsoft 365 Copilot allows extensive customization through agents, which can be tailored to specific organizational needs, from simple retrieval tasks to complex workflows.
• Types of Agents: There are two main types of agents – declarative agents and custom engine agents. Declarative agents use Microsoft’s foundational models and orchestrators, while custom engine agents allow for more tailored solutions using external models and data sources.
• Building Agents: Agents can be built using Copilot Studio for a more user-friendly experience or Visual Studio Code for more control and integration with software development practices.
• Skills and Capabilities: Agents can be equipped with various skills and capabilities, such as data retrieval, task automation, and integration with external systems. These can be pre-programmed or selected from a list of available options.
• User Experience: The user experience remains consistent across Microsoft 365 applications, ensuring that users can interact with agents seamlessly, regardless of the underlying complexity.

For more information, read the transcript blog below, or watch the video above!

Video Transcript

Are you confused about the difference between Copilot chatbox agents, declarative agents, custom engine agents, and all of the other terms that Microsoft uses when they talk about customizing Copilot? Well, you’re at the right spot because in the second episode of The Ultimate Guide to Customizing Microsoft 365 Copilot series, we will cover all the terms and concepts that you need to know before you build your own agents. In this series, I’m joined by Sebastian Levert, an amazing developer and principal PM manager at Microsoft on the Copilot DevX team. Seb, welcome back!

“Well, thanks for having me again, Vlad. I’m excited to be here and I’m excited also to kind of cover all these base terms today. I think there’s a lot to learn and let’s dive into this.”

“There’s a lot to learn and Seb, in this episode, you’re the boss because you’ll need to teach me and everyone watching at the same time that important foundation of what terms and concepts that we need to know to customize Copilot. So from now on, take it away.”

“Awesome. Let’s start at the top. Let’s start thinking about what we want to do here and it’s really to customize the experience of Microsoft 365 Copilot. How do we do that? Well, first we need to understand what is available from a customization opportunity, right? We need to understand where you can put your own brain inside Microsoft 365 Copilot. So we’re going to start at the top and we’re going to talk about Microsoft 365 Copilot and how it’s actually architected to enable you to bring more into this. For sure, you can use today Microsoft 365 as a user and you have access. The first thing we brought in was access to the web. So you can ask a question, you can prompt Copilot and say, ‘Hey, a good example and I think you’re all going to love this one is what’s the best poutine recipe that is out there?’ We might not hide exactly where we’re from with that. We’re going to let you guess, but you might want to ask it and sometimes it can be useful when you’re, I don’t know, you’re working in a recipe book corporation. But it’s very limited. It’s limited to the web. It’s limited to you can decide which website you might want to use, but that’s kind of locked there.”

“Then afterwards, you have data in Microsoft 365, right? That data is your organizational data, your intranet, your policies, your work documents, your presentations, your budgets, your OKRs, your objectives, your one-pagers, your product specs, and your library content. Any type of content is available either from a team standpoint or at a personal level. But also at your personal level, right? How many documents do you have in your OneDrive?”

“Many, definitely too many.”

“And all of Microsoft 365 Copilot can be grounded. And when we say grounded, it means that you can use the data that is available in the Microsoft Graph to help with summarizing and answering natural language questions. So when you say, ‘Why did we overspend on X category of items in the last fiscal?’ it’s going to go and find the nearest and the best content that is available to you to generate the answer back to you to make sure that you always have the best answer possible. It will always also add some citations. So it’s going to tell you this data is coming from this document. It gives you a little bit more trust in the generative AI world, right? Because you know what that data is. Where was this just generated by AI and now you’re just trying to…”

“And that’s so useful because sometimes I’m like, I want to learn more. Like, where did this come from? I feel like all the info from this document, I need to read the whole thing. So it really helps me know, okay, I need to go read that document. I missed it.”

“Exactly. And that’s the role of AI, right? The role of AI, yes, it is to generate beautiful answers but also to help you find the right thing so you can continue delving into that specific document. Boom, you’re into that Word doc. Boom, you’re into that presentation. Then afterwards, M365 Copilot lives in so many places. It lives as an app. It lives inside Microsoft Teams. It lives in all the Office tools, in Word, Excel, PowerPoint, Outlook, name it. It lives in Edge. It lives in Windows. So how can we bring all of that together to make sure that when you build something, it’s available throughout all of the capabilities we have inside Microsoft 365? Well, this is your opportunity right there to create what we call agents. Agents are small pieces of software that you don’t need to know how to code to create, actually, that comes in and bring more context to Microsoft 365 Copilot to ensure that the answers that are being generated for you in the context of M365 Copilot are grounded and scoped to what you really care about. Sometimes you ask a question and you really want to be super specific on what you’re looking forward to. Whatever you write in the prompt, you might not be able to get the right level of detail. But here with agents, you have control over all of that. So that’s what we’re bringing to the table. You have the opportunity to build agents that live in your enterprise. They’re all user-personalized based on who you are, what you have access to, what are patterns that you have with that data, but also are fully enterprise managed, meaning that admins can decide to deploy an agent to everybody or just to a group of people and so on and so forth.”

“Awesome. When you think about the customization opportunity of agents, well, what does it mean for a maker, for an information worker, for a developer? What does it mean for them? Well, first, it means that they’re going to build on top of a trusted platform by all of our customers and partners. We’re not reinventing a new way to distribute apps. We’re not reinventing a new way to govern these agents. These are all the same things you’ve been doing in Microsoft 365 for a long time. Then you have the opportunity to build on top of our AI platform. That’s key. You’re investing in that AI platform as a customer. Now you can build on top of it the same way you can build on top of SharePoint, the same way you can build on top of Microsoft Teams, the same way you can build on top of the Office apps. Now Copilot just becomes a new surface that you can build on top of. But sometimes there’s always a but. Yeah, there’s always a but. Sometimes there are things that you believe in into our platform, but sometimes you want to bring that might be more tailored to your own organization. So maybe for some specific use cases, you want to use your own large language model or you want to use your own databases or you want to bring your own AI stack that is connected to other of your some other products that you’re also working on. Well, you can also do that with this opportunity. So it’s either you build on top of ours or you bring yours to participate in the ecosystem with ours. Users access all of their systems in one familiar experience. We don’t want to have another app. You have already like you have enough tabs open in your browser today. Don’t add another one. And that’s why everything now is in the M365 Copilot app from your docs to the experience of chatting with M365 Copilot to the rest of the ecosystem. Everything is in one single location. And that’s very important for users because you don’t want to retrain people and say, hey, this agent or this tool, you have this UI, this tool, you have this UI. They lose productivity. So if you can bring it all in one UI, one experience, they only have to learn once.”

“Exactly. And the beauty here is in the future, the agents you’re going to bring to that platform will also be part of all of these experiences. So it’s again, you don’t have to build an app to do something. You just need to bring an agent that’s going to participate in the rest of the ecosystem. Think about it. Everything is Team-based. Like you’re spending, I spend my day in Teams. I chat, I have channels, I have conversations with all of them. And Copilot in general is part of Teams in so many different ways. So you also have access to all of these active Teams users today. And it’s great since as time of the recording, this week, even a vast majority of the Teams users today have access to a limited capability of Microsoft 365 Copilot as part of their current entitlements with their Teams and M365 licenses. So that’s great. Now it does not necessarily require the full-blown M365 Copilot license where you can get a part of the value. And if you want, you can either go on pay as you go for some of your users or you can go and add everybody. So now what we love to call the total addressable market or the ecosystem you can really tap into is more than 320 million users with the agents you’re building. So if you build a great agent, think about the impact that it can make for your organization.”

“And that’s amazing for organizations too because I know that a lot of people are scared. Okay, what if I buy Copilot but nobody’s going to use it? Now at least you can start in that pay-as-you-go, enable it for the whole company and then see maybe for this department we’re going to pay the full thing for all the other departments. Pay as you go is cheaper and you can choose that flexible option that is better for your organization, which is amazing.”

“Exactly. It brings back the choice in the hands of the organization. Yes, not everyone will work in every single tier, but that’s probably how you want to think about it. We want to give some simple AI tools for parts of the workforce, but for the information workers who are spending time and days inside documents, maybe you want to give them the entire suite, and that’s how we need to think about it. Talking about documents, one of the beauties of building agents and living on top of the AI platform that we provide is the fact that you can now summarize and act on all of your M365 data. How many documents do you have? I asked you earlier how many documents you have in OneDrive—a lot. But then, how many documents do you have organization-wide that you see in transit every day through emails, through Teams messages, through sharing links? They’re everywhere, and the beauty here is that you will now be able to find the needle in the haystack by being able to generate great summaries of M365 data.”

“Finally, if you are a partner, an ISV, or somebody that builds software for a living, you are going to be able to grow your customer base in our store ecosystem. But what is really, really cool, if you think about it from more of a user point of view, is that tomorrow I build a great agent—an agent, for example, that helps me write one-pagers. I give very strict instructions to an agent, saying how I want things to be, and what are two or three or four great examples of one-pagers. Then from there, I build an agent, but then I don’t want to keep it for myself. I want to share it with you, I want to share it with my team, I want to share it with the entire organization. So as a maker, and an information worker, you can also grow your customer base by sharing and making sure that your agent is seen and used by everybody in the organization. So for us, we really see that as a tremendous opportunity for ISVs and partners, but also internally to almost do some entrepreneurship within the organization. And that’s really where we’re already seeing these AI champions in organizations building tooling agents for others to succeed even more in that AI world.”

“That’s amazing. So a bunch of our partners are already part of it. There’s tons of them, there’s even more than that. We really invite you to go to our store, click on the agents category in the store, and start looking at what’s available right there. We do provide, as Microsoft, a couple of existing agents so you can try them out. There’s the prompt coach, there’s the career coach, brainstorm. I started using some of them, they’re pretty cool, they’re actually really cool, and you’re going to see they’re not that hard to build, which is really surprising. So, and we’re going to come to that later, to how we build, but let’s continue on the what and kind of why of all that. So building agents enables you to tailor your organization or your processes to basically any type of job. We can start with simple work—it’s becoming simple, but a couple of months ago it was not that simple. So I would say a simple scenario would be for retrieval: “Hey Copilot, I was off last week, can you give me the top five documents that I need to review right now because they were part of heated conversations, meetings that I was a part of, and people were sharing these documents a lot.” Copilot will understand your query, go on, ground the answer in that data, reason over it, summarize it, and might even answer the questions. Like, maybe you can start talking with these documents, as those are the things that, in a way, you can do today with Copilot chat or with OneDrive Copilot, things like that. Those are a lot of the stuff that kind of comes in with M365 Copilot.”

“Absolutely, but here the beauty is now you give the opportunity to the user to be a lot more simplistic in how they talk with the agent. You don’t need to give the magic to the user on, “Okay, so this is how you trigger the right way Copilot to give you the precise information you need.” No, this is all in the agent. The agent knows about all of these rules, all of these tricks that it needs to play, so it’s really helping to simplify the user experience. You don’t need to be a prompt engineer anymore. Exactly, I kind of hate that term because I feel like Copilot should make our life easier, and then you look on LinkedIn and everybody’s saying, “This is the prompt you should try today,” and it’s like four pages long. “You are a marketing manager that blah blah blah blah blah,” and then it becomes like this, and nobody wants to write that. Exactly, so we want to leverage agents to simplify the work. But what is cool is that you can build an agent to become a prompt engineer. The prompt coach is a great example of that. So it’s all things that are super useful as a retrieval scenario. Then maybe you want to say, “Okay, based on these documents, I need to create a PO, a purchase order, in my own line of business system.” Okay, so based on that, I asked the questions, and then one of the documents was like, “We absolutely need to create a PO for purchasing a bunch of new components for our data center.” Okay, now I need to leave Copilot. Nobody wants to leave Copilot, so why wouldn’t you just say, “Based on the item list that lives in that Excel file, please create a PO that automatically lists the items,” and so on and so forth. So you can now also have Copilot perform tasks on your behalf. That’s where we bring more complexity to the flow. So that’s where there’s a huge opportunity, we think, for people to really go through the task habit inside Microsoft 365.”

“And the more we grow in Copilot, the more we’re going to see autonomous agents—autonomous agents that are all operating independently but that are holistically planning how to get the thing done that the user is asking. I can say, for example, “Go do this, go get me these documents, find the most important items, create a PO out of it, send an email to my customer, and then finally do this.” And then you could actually have agents that are specialized—specialized in data retrieval, specialized in PO generation, specialized in email drafting—and then they’re going to orchestrate all of that together as a swarm, as a team of agents. They become kind of like a team that knows what each one of them is good at, and hopefully, they work together to get the best out of each one to get you the result. That’s exactly the goal, and that’s really where with M365 Copilot we are going. So you can see that you can go from very simple to very advanced scenarios, and we really encourage you to build purpose-led agents, meaning that when you build an agent, don’t build an agent that does it all. Go very specific at what you want the agent to do, so then when autonomy becomes really available throughout all the product sites, all of them will become experts in their own thing, and they’re all going to collaborate with each other. You have the opportunity to build agents for any job. Start now, that’s already a job. Getting there, getting there, getting there.”

“So, how is that built? Well, first, an agent is built on what we call a foundational model. The foundational model here is the large language model—this includes GPT-4, GPT-40, and others. Exactly. I love to call it the “AI magic” that lives at the bottom of the stack. Everything goes through that model to understand intent, generate answers, and so on. But there’s a lot of logic surrounding it.”

“There’s a lot that ensures the LLM is used at the right moment and that we send it the right prompts. Internally, there is something called an orchestrator, which is what Microsoft 365 Copilot provides. It automatically connects to the foundational model. Then, the orchestrator connects to knowledge, which consists of two things: instructions and grounding. Instructions are what you provide to the agent—what it should and should not do, how it should answer, and the tone of the response (e.g., formal, funny, or even like a pirate).”

“Grounding refers to the data within M365, which is essential because it allows Microsoft 365 Copilot to understand the data that an organization is building. After that, there are skills. Skills include actions that Copilot can take, such as calling an external system to create a purchase order or retrieving real-time data that doesn’t live inside an M365 tenant—for example, stock prices or a list of S&P 500 companies.”

“Capabilities are another aspect of skills. These are built by Microsoft and include features like code generation, image generation, and more. For instance, a data analyst might use Copilot’s code interpreter to generate a graph from an Excel spreadsheet or a live data feed. Image generation has been a game-changer in AI, allowing users to prompt and generate images that look quite realistic. Soon, video generation will also be available.”

“We also have triggers. Right now, we offer two types: scheduled prompts and user typing. A user typing a query into Copilot chat and selecting an agent is technically a trigger. More complex workflows are also on the way.”

“Now, going back to skills: capabilities are built into the orchestrator by Microsoft, and users can select them from a list, such as a code generator or an image generator. Actions, on the other hand, must be preprogrammed into an agent. Users define a set of actions an agent can take—such as reading or writing to a database—by specifying APIs the agent can call. Copilot will not generate its own actions; they must be configured.”

“For example, a software developer might want to determine the most critical task to work on. Their task management system might be outside of M365, in platforms like GitHub Issues, Azure DevOps, or Jira. In this case, the user could give Copilot access to read the task list, summarize it, infer the most critical next task, and return it with a citation linking back to the original issue.”

“Triggers give users options such as user input, another agent calling the agent, or scheduled execution. Workflows, meanwhile, will allow agents to integrate with Power Automate or Azure Logic Apps, enabling automation beyond what’s available today.”

“Looking at the user experience, it starts with where the user is. They might be in Microsoft 365 Copilot, in Word, in PowerPoint, in Outlook, or in Teams. The UI is already built, but in the future, users will have the ability to build their own experiences if needed.”

“When we say “Copilot is the UI for AI,” we mean that the user experience remains consistent, but everything under it—logic, actions, and capabilities—can be modified. There are two ways to build on M365 Copilot. The first is customizing the core capabilities of Copilot within M365, which is powerful yet simple. This aligns with the Power Platform philosophy—allowing users to build impactful solutions with minimal effort. Tools like Copilot Studio, Power Automate, Power Apps, and Power Pages empower end users to build without extensive coding.”

“The second way is to build a custom engine. Think of it like a car—you might love the body of your car but want to replace the engine with something tailored to your needs, like a track-ready engine. Similarly, users can build custom engine agents by replacing the default orchestrator, perhaps using LangChain, Semantic Kernel, or a homegrown solution. They can also bring their own foundational models—such as OpenAI, Azure OpenAI, or an industry-specific LLM, like one trained for medical applications. They can even integrate custom knowledge sources beyond M365 data.”

“With custom engine agents, users can fully control the experience while still using Copilot Chat or Teams as the front-end UI. It’s the same “UI for AI,” but everything under the surface is customizable.”

“At the end of the day, Microsoft 365 Copilot ensures a consistent look and feel, no matter how an agent is built. Users can take the easier route and use Copilot Studio, leveraging built-in models, orchestrators, and Power Platform connectors. Or they can go deeper into custom development with Visual Studio Code, writing their own logic and integrations from scratch.”

“Looking ahead, autonomy will be a key feature. For instance, if users frequently ask for steps A, B, C, and D in sequence, the system could eventually suggest automating the entire workflow. While autonomy isn’t fully rolled out yet, it’s coming soon.”

“As we left our series open-ended, we’re going to come back when we have more to say. Finally, the user experience (UX) starts with where you are and what you are in. Maybe you’re in M365 Copilot, and you select the agent, or maybe you are in Word, PowerPoint, Outlook, or Teams. All of that is an experience that you’re going to build over time. A lot of it is already built for you, but in the future, you’ll have the ability to build your own user experience if you want to. When SAA says Copilot is the UI for AI, he means that the user experience is the same, but you can modify everything under that user experience. That’s exactly the case.”

“How do we do that? Well, we have two options. The first option is to build on M365 Copilot, customizing the core capability of M365 Copilot. It’s powerful but simple at the same time. The powerful simplicity here really strikes me because it tells me I can build something powerful in just a few clicks, which is kind of the promise of what the Power Platform is. Copilot Studio, Power Automate, Power Apps, and Power Pages are experiences that empower the end user, the maker. It’s actually pretty simple to achieve something really powerful.”

“You also have the opportunity to build your own custom engine. Think about a car: you might like the body of your car but want to change the engine because you need it to go on the track. You can remove the engine and replace it with another one that you built. The car will still run, and it will look the same, but the experience will be more tailored to your needs. With custom engine agents, you have the opportunity to do that in many ways. You can start with Copilot Studio or Agent Builder, which are end-user maker experiences, and then move on to Visual Studio Code if you need more control. If you are a software shop and want source control management or to go through the software lifecycle, you can do that with Visual Studio Code. There is virtually no difference between what you build in Copilot Studio and what you can build in Visual Studio Code.”

“For the end user, they’re just using agents. The agent doesn’t give you a hint of how it was created; it just provides the capabilities that the user needs. That’s amazing. These two things are very specific. First, the declarative agents are where the user experience, the orchestrator, and the foundation model are all provided for you. You don’t need to customize any of that. The user experience is the chat experience, the orchestrator is Microsoft 365 Copilot, and the foundational models are your GPT series models. You can bring instructions and grounding as knowledge, and you can also bring action and capabilities as skills. That’s how you build a declarative agent, either through Agent Builder, Copilot Studio, or Visual Studio Code. The same capabilities are there; there are just different authoring opportunities for you.”

“If you want to go deeper, you can do custom engine agents, which are also available through Copilot Studio or Visual Studio Code. Here, you’re at a restaurant with a big menu of things, and you decide how you want to order. In the end, you’re going to buy a meal, but you don’t know exactly how much it’s going to cost or how much you’re going to get. You bring your own orchestrator, maybe using LLM Chain or Semantic Kernel, or your own homegrown orchestrator. For the foundational model, you might use OpenAI, Azure OpenAI, or an industry-tailored LLM to your needs. If you’re in the medical business, you might want a medically trained LLM. You bring your own knowledge from various sources and provide your own instructions. This way, you create an agent that people can consume from Teams or Copilot Chat, but you customize 100% of what they consume. The UI for AI remains the same, but everything underneath is built by you.”

“In the end, it will always look like Microsoft 365 Copilot, but all the internals are on you. There are easier ways to build using Copilot Studio, where you use models, orchestrators, and actions from the Power Platform connectors. You can ground into an Azure Blob Storage or go into code and build it all yourself. If you want autonomy, you will need to go custom engine, but both options will be available. Agent-to-agent communication is part of M365 Copilot, the UI for AI. You can have a simple agent for summarizing key takeaways from meetings and a specialized agent for creating patient notes based on transcripts, using industry-standard LLMs.”

“Your declarative agent will be orchestrated in a flow of agents, starting from a declarative agent to a custom engine agent. The user never really knows the difference, which is the beauty of it. The user doesn’t need to know the details. These are the two types of agents we’re going to build. You need to transform the way your customers interact with agents. It changes the mentality and habits, but you have the opportunity to make them personal and powerful. Copilot becomes more than just document fetching; it becomes the full-blown app platform you’ve always dreamed of. Everyone in IT has always wanted a single place for all their apps, and Copilot can be that single landing spot.”

“That’s what I wanted to cover today: the terms, concepts, and how to customize agents. In this series, we’ll dive deeper into what agents are and give you a glimpse of what you can do today. If you want to get started, you can go to aka.ms/buildagents or stay within our series. We’ll show you everything there. For official docs, visit aka.ms/buildagent. I hope you enjoyed this video and found it valuable. Make sure to check out the next episode in the series and hit the subscribe button to get notified of new episodes. See you in the next one. Cheers!”