How Do I Build Microsoft 365 Copilot Agents with Code? [DEMO]

We covered how to do it without code, now it’s time to open up Visual Studio and build Microsoft 365 Copilot Agents using CODE! If you’re looking for how to build Copilot Agents without code, click here.

Watch my 90+ courses on Pluralsight (opens in a new tab)

Video Summary

• Action Capabilities in Copilot: We’ve moved beyond just retrieving information with Copilot. Now, we can perform actions directly within the Copilot UI, enhancing productivity without switching contexts.
• Pro-Code and No-Code Experiences: While this demo focused on a pro-code experience using Visual Studio Code, similar capabilities will be available in Agent Builder for no-code users, making it accessible to everyone.
• Integration with APIs: By connecting to APIs, like the GitHub API, we can automate tasks such as assigning issues. This integration is seamless and transparent, showing exactly what data is being shared.
• Power Platform Connectors: With over a thousand connectors available in Power Platform, you can often find pre-built solutions to integrate with your systems, reducing the need to build custom APIs from scratch.
• Future Updates and Resources: We’ll continue to update the series as Microsoft evolves Copilot. All the code used in the demos will be made available, and you can connect with Seb for more insights and updates.

For more information, read the transcript blog below, or watch the video above! Or if you’re looking for the intro to the series to start from the beginning, click here!

Video Transcript

Copilot agents are all the buzz right now, and while Microsoft offers a ton of different ways to create agents without coding, sometimes you need to open up Visual Studio Code to really take Copilot to the next level. Welcome back to The Ultimate Guide to Customizing Microsoft 365 Copilot series. I’m joined again by Sebastien Levert, Principal PM Manager at Microsoft and an amazing developer. Seb, how are you doing?

I’m great. We got to open a little bit of VS Code earlier. Now we’re going to really make cool things today.

Now we’ll make more VS Code, yes. Okay, so Seb, this is the last episode, at least for the initial release of our series, and we covered everything from terms and concepts to no-code options to using code to add more knowledge to our tenant, to the graph, to Copilot. What’s left to learn? What are we going to learn in this video?

So the one thing that I feel is left to learn is twofold. The first thing is how could I do all of that end-to-end as a dev, right? How can I do that as a dev, and how can I bring the same functionality and capabilities that are available in Agent Builder but for my development build solution, especially when you want to redeploy it from QA to prod or create a product that you’re going to deploy to all your clients? You can’t just give people the connector and tell them, “Yeah, yeah, go in Agent Builder and have fun.”

Exactly. We want to make sure to create repeatable scenarios, want to create repeatable deployments. We want to be able to leverage lots of the source control systems that your organization is using. You want to also probably be this version of the truth of agents. You want to deploy them and make them available for your organization. So all of that is what we’re going to cover today with building agents with code.

Awesome. Let’s get started.

Let’s do that. So in the past, again, M365 Copilot drives the UX, drives the orchestration, drives the foundational model. We covered a little bit of knowledge with graph connectors and SharePoint grounding, but today what we’re going to go a little bit deeper into is bringing actions. So not only are we going to be a retrieval tool, but now we’re going to be able to act on that data.

Cool. Do you remember the GitHub issues?

Yeah.

Earlier you said, “Seb, you’re done with your issues,” but that wasn’t exactly true. We still had a couple of issues open. They were not assigned to me, but I know someone who can help me. So today we’re going to even assign a task directly from Copilot inside GitHub, and we’re going to do that by bringing skills and actions right in there. So we’re going to really make it not only a retrieval tool but we’re going to make it part of your business processes. So you know me, done with slides. Let’s go to the demo.

So I’m here. I’m in Copilot. We still have our GitHub issues agent created earlier, the PO agent that we created earlier in the series, but now what I want to focus on is starting from code entirely. That means that we can start right here, dev starts in one single place, and it’s in the IDE. So let me go straight into Visual Studio Code where I already opened up. Let me do that, and then I opened up Teams Toolkit. Teams Toolkit is the tool that brings the capability of building every type of functionality around Teams and Copilot together. So you can create a new app, you can view some samples, and some of the code that we’ve shown during this session is available there. You can also follow our guided tutorial, building a notification bot in Teams or building a declarative agent, which is exactly what we’re going to build today, but we’re not going to use the guided experience for now.

So how do we start? Well, it’s very simple. We create a new app, and here you have the ability to create a ton of different artifacts that live in the Teams and M365 ecosystem. A declarative agent or a custom engine agent, which are two ways of doing AI agents, or a bot or a tab or a message extension or whatnot.

So the Teams Toolkit really started off as the Teams Toolkit but more like your M365 Dev Toolkit today. It’s really whatever you want to do in M365, Teams Toolkit is still your starting point.

Absolutely. So we’re going to start, and in our case, we want to build a declarative agent. Awesome. In this very specific case, I want to take advantage of all the foundational models, the orchestration, and the UX. So what does it mean? It means that I want to use a declarative agent, not a custom engine agent, which is absolutely a logical choice if you want to go, I would say, low code or less code. If you want to go and you’re really building a very strong AI-driven app, maybe a custom engine agent will be for you, but for now, we’re going to start with a declarative agent. And here it’s going to ask you one thing: do you need action or do you not need action? What is an action? An action is a way to interact with external data to bring a skill or a capability in to be able to act on data. In our case, just to get started, we’re going to say no action just to see what it means to build the most basic agent we can. I’m going to take no action. I’m going to use the default folder. I’m going to call it a demo. Awesome. Let’s do it. And there we go. After it’s already done, my agent is done. That was quick, right? I don’t know why you build so much as a consultant when it’s already done. I’m joking.

So here I have an overview of a declarative agent, you can follow this guide to understand more and more, but I want to show you. So what did we do? We go to Teams Toolkit, and we click on provision. The first time you’re going to hit provision, the provision means I want to add this agent to Copilot, but I don’t want to distribute it yet to my organization. I want it to be available just for myself. So I’m going to hit provision, and it’s going to go through five different steps, and it’s really pretty quick. It creates a Teams application, and then afterwards, it does a bunch of validation. It validates everything. It brings some variables back inside the project to make sure that the next time you run, we’re updating the same agent. And there we go. So quick admin question for you.

Yes. So does it need side loading enabled? Yes, for your user, at least from a Teams policy. You can enable it for all your devs if you want to, but you need side loading enabled to enable this experience. Exactly. And that’s why we recommend to either turning on side loading or leveraging a development environment, a dev tenant part of the dev program, or any of the tenants that you’re using for development inside your organization. Awesome. So now let me go to Copilot, and look at that. I didn’t do anything. It automatically refreshed. Now I have a demo. I’m going to go here, and I’m going to say hello, and it’s going to do one thing. It’s going to say, “Thanks for using the toolkit to create your declarative agent. Hello, how can I assist you today?” And the reason why it’s the only thing it does is because it’s actually the only thing it does, but we haven’t told it anything yet. Exactly. So let me go and show you what it looks like from a capability standpoint. So first, I have a manifest file. Let me close this one up. The manifest file is basically the same as a Teams manifest file that you might have seen in the past, and where it becomes a little bit different is here, where we have a Copilot agent and a declarative agent. So here you say, “I have a declarative agent. Its ID is this one, and the file that represents its capabilities is this one.” Could I deploy more than one agent inside an app? Not as of today. Okay. So it’s built to be flexible for the future, but today it’s one-to-one, the same way with bots. It’s exactly the same way it’s structured. It’s an array where you have a single item, but it’s something we’re thinking about in the future. That’s why it’s like that. Then afterwards, go to my declarative agents. There’s a schema, a version.

Okay. There’s a demo. There’s a name, sorry, a description, and a set of instructions. And here, are the instructions, you’re in dev mode, right? So you can really have great instructions with the tooling that you’re used to. So here, in that case, we have the instruction.txt file, and now you have it here. So that’s where it took that from. You start every response with that. Exactly. Exactly. You should start every response with, “Thanks for using the toolkit to do that,” and then answer the question and help the user. Okay. That’s probably something you want to remove before you ship it, or not. It depends. It depends on how you want to promote the Teams toolkit. But here, that’s really where you put in that value, which is great because it’s in your IDE. You’re going to be able to do formatting in there. You’re going to be able to do markdown. You’re going to be able to do a lot of really interesting things with that file. So let me close that one and go here.

Now, you remember when you’re in Agent Builder, you have these buttons and these checkboxes and these pickers. You have all of that in Teams Toolkit or in these files, but it’s for coders. It’s for devs. So let me show you, for example, you have capabilities. I’m going to go and enable some capabilities here. The first capability I can enable, let me create an object here, sorry for that, is GitHub Copilot, it’s all GitHub Copilot helping me here. So I have a name, and I can do a web search. And web search, which is really interesting, is in the Teams Toolkit version, it’s going to come soon to Agent Builder, but you have the option to have a series of sites. And then here I can actually say which URLs are available. Here would be very meta to use bing.com, but here I could use, for example, Vlad Talks Tech. And now it’s going to ground the content in that thing. It’s not going to ingest it, it’s going to do it at runtime. It’s going to send a query to Bing. Bing will take the query in, will give you back a set of results, and then Copilot will do whatever it wants.

This could replace what we did when we created a built-in graph connector, but also this would not consume my graph 50 million items, right?

Exactly, you’re absolutely right. Then other things we can do here, I’m going to call it a name, sorry for that, I’m going to use a name, and I can do a code interpreter. That’s the checkbox we have in Agent Builder, exactly. And that’s it, I just enabled it. Exactly the same thing for graphic art. Oh, sorry, not a graphic connector, but a graphic art. Same thing for graphic art. So graphic art is the visualizer, if I remember correctly. It is the one that generates images. Then here, name, I can do OneDrive and SharePoint. So here I can do OneDrive and SharePoint. I can generate either site items by SharePoint IDs, so if you know the specific IDs you’re looking for, you can use them, or by URL.

Any best practices people should use?

SharePoint IDs are definitely the ones that are going to give you the best answer back, but SharePoint IDs are not user-friendly. So we’re working on new capabilities to enable the SharePoint IDs without you putting the SharePoint IDs in. But right now, I would say from a developer experience, the best is the items by URL, which gives you a little bit more there. And here you can have as many as you want, so you can bring as many URLs as you want, either direct document, folder, library, or site.

Are you still limited to 20 like in the Agent Builder?

No, you are not. You are not limited to 20. 20 is just a maker experience limitation. That’s a good limitation to know. And here, if you don’t put anything, it’s going to ground on all of SharePoint.

Oh, wow, okay.

Which brings value on one side but also brings complexity. Same thing for graph connectors. You can do the same graph connectors, and now you can just go in and say connections. And here you can ground on, sorry, I need an object here, connection ID, and there would be GitHub issues. As you remember earlier, we used this one. Now that one would be grounded in the web, in code interpreter, in graphic art, in all of SharePoint, and in this very specific graph connector.

Wow.

But right now, so we are mixing a few things. We’re mixing knowledge sources like web OneDrive and SharePoint and the graph connector with capabilities, with skills, like graphic art and code interpretation. So they all still go under capabilities, so it’s not like the slide where it is separate. Yes, in code here, they’re one and the same. Exactly, because internally, they’re actually all the same. That from a maker standpoint, we want to make sure we separate them so it makes sense, but for devs, we want to make sure that they’re just the capability of searching for SharePoint content, the capability for finding content in graph connector, and so on and so forth. Then afterwards, you have your conversation starters. Then conversation starter is a title and a text. So as simple as that, then you can just put whatever you want there, and you’re going to have them. And finally, we have actions. And actions are really where you are adding capabilities from remote systems. But in this case, what I want to do, I’m going to jump, you know, cooking shows, you have the turkey in the oven, it’s really long, you need to add juice all the time.

Well, now I’m going to go to the cooked version of this so we can walk you through some of these things around actions. So let me go here, and let me go to, I have too many windows, to this one where I have an agent. Same thing, with very similar capabilities. I have an app package, and I have a declarative agent as part of it. If I go up, I’m going to find I have a GitHub agent. It’s an assistant for project management professionals with GitHub. I have a link to my instructions. I’m going to show you the instructions actually right now. And here you’re going to see it’s a lot more detailed. You’re an expert at helping users. Guidelines every time the user talks about issues, always refer to them as GitHub issues. When a user asks for a list, always use the, and this is a variable that we have, so it’s going to automatically be updated based on the build. Here, when it asks for a list of issues for a specific repository, always render them as a table with the columns empty, issue number, title, assigned to, and state. If not, render these on, depending on what you’re looking for, and so on and so forth. You must use the owner property from the connector data when requested for an API, so how to connect data from one system to the next. You’ll see why in just a second.

What are the wild cards or the asterisks you added there?

So those are markdown principles. A single asterisk is basically a bullet list. So it reads it as a set of guidelines, and here it’s bolded, so it’s going to understand that this is important. So Copilot can understand bolding, can understand that. So if you put something all in caps, even without the bolding, it would still understand that, hey, this is more important, but this is really important because there are both caps. So these are the instructions, so you can really go pretty insane. There’s a limit of 8,000 characters, and we have customers already hitting these limits because people are really putting a lot of juice in these instructions.

Wow.

Then afterwards, our conversation started.

Will it make your agent slower?

No, no, no, it should not. It’s really, really, really quick. So here I have a bunch of different conversation starters and a set of capabilities. So here is a graph connector with the connection ID we said earlier that we don’t want to change in that case because it makes it simpler if you want to change stuff. Web search, I’m grounding it into the help documentation of GitHub for issues. So maybe you want to know how to use the UI to generate projects. Well, you’re going to have the answer there. We’re going to use a code interpreter. We’re going to see why later on. And then we have an action, which is a plugin. It’s a GitHub API plugin. Let me show you the file that we have here before we go forward.

Will this code be available for everyone?

Yes, absolutely.

Great. So if any of you want to try it, which you should, make sure you check out the description below right now and download the code so you can open it up and follow along.

That’s it. So an API plugin. So when you create a project with Teams Toolkit, you have the ability to add an action, and you will be able to go through a really interesting flow. But let me show you what the flow would look like from here. So here I can go in and add an action, and now it’s going to ask you, “Hey, do you have an open API?” All of the actions are driven via an open API description. An open API description is a document that describes the shape of an API, all the endpoints that are available, what’s the data being returned, and so on. That way, Copilot can understand how to call it, but also what’s going to come back from it, so it’s going to be able to infer some things from the description. In this case, I’m going to say I’m going to add to this manifest here, and now I have the ability to say, “Hey, do I want to browse an existing open API I have on my desk or on my desktop, or do I want to search?” And here it’s really cool because we give you access to more than 3,500 public and free APIs.

So here I can go and say GitHub, and now it’s going to say, “Okay, I found the GitHub API.” Well, okay, well, I’m going to use this GitHub API because I don’t use any of their enterprise software or whatsoever. I want to use this one. What’s going to happen here is it’s going to load in memory the experience here, and it’s going to ask you, “Hey, these are all the endpoints that are available.” In our case, for example, I have a repo, I have an org ID, a repo ID, and I have issues. I should have issues. There we go. And I want to be able to issue numbers. I want to do a patch. I want to be able to update. Okay, a patch, an issue-based. I’m just going to add this one. So I’m just basically shopping right now. I’m shopping for APIs, and when you’re ready, you just need to generate this thing here. And it’s going to automatically generate that plugin for you. How cool is that?

It’s pretty cool. I like the shopping aspect of it and then just adding everything.

Exactly. So for now, I’m just going to cancel, and the reason why is I already generated it in the past. So let me hit cancel. I’m going to go back here, and now actually it generated some stuff already, but I’m just going to remove these things so it doesn’t get confused. Exactly. So I already generated it, called the GitHub API plugin. Now I’m going to go here, and here you’re going to see that it highlights two functions: the add assignees and the issues update. So now I can add an assignee to an issue, and I can update an issue. And I can see that it generated for me that very tricky long open API. Oh my God, like this is the part you don’t want to get into. You don’t want to play with that. You don’t want to play with that. So this is there, but you don’t touch it. Exactly. You don’t want to touch it. We generated it for you, just ignore that part.

Wow, okay.

Because it’s a really long format, and it’s a very dev-y one, but it’s also not great at editing. The second thing it does is it’s going to also automatically get authentication in. So it’s going to automatically say, “I want to have OAuth for that API.” So it means that it’s going to do a full OAuth flow between Copilot and this API, and it’s going to have a reference ID. And this reference ID, the first time you’re going to run this, Teams Toolkit will prompt you, “What is your ID? What is your secret? What is this? What is that?” And it’s going to create it for you. You can also go directly to dev.teams.microsoft.com, and you’re going to be able to actually register new by using the tools here. You’re going to be able to register a new one. So if you don’t want to go through the automated way, you can go through the portal, or you can use the automated way directly in Teams Toolkit.

Because when you want to assign something or update something, you need to prove who you are before you can actually do it.

Well, because it’s an actual change.

Exactly. And then that’s it. That’s what it is.

Can I have it so right now the way you have it, everything will run under that secret that you created?

No, it’s a secret to create an OAuth registration. So it’s really OAuth, so it’s on your behalf. So every time a user starts using this agent, the first time they use it, they’re going to have to authenticate.

Exactly.

Go to GitHub.

Exactly.

Okay.

Exactly. So now that’s what we have here. I go back to Teams Toolkit, and I hit provision. Now what’s going to happen is it’s going to deploy this agent.

I like that it started at step four out of five because it skips already.

Exactly. It was already there, and now it’s all executed successfully. Let me go here, and let me load that one in. I’m going to reload, and here I’m going to find the GitHub agent.

The one with the nice logo.

The one with the nice logo. And now I can do, for example, best practices. What are some of the best practices in issues management using GitHub? So now it’s going to say, “Oh, wait a second. I’m looking for web stuff.” Probably web stuff. I’m grounded in the web, but I’m only grounded in that content that is coming from docs.github.com. So it’s a very narrow view of it.

That’s amazing. So if you’re building either for yourself or for your customers, you’re going to be able to ground your customers in your own support knowledge base if it’s public.

And you see show plugin developer info because you provisioned it, you sideloaded it, so it knows, “Hey, you’re a dev. You’re doing this for you.”

Exactly. But I’m going to show you what really cool thing is happening there in a couple of minutes. Then afterwards, let me go back. Another conversation. So here I can say, “List the latest issues from the repo.” It’s called GitHub agent. And now it’s going to say, “Wait a second. I know issues. Issues to be GitHub. GitHub means graph connector. I’m going to go graph connector. We’re going to use the connector we created in the previous one.” And now there you go. I just found that very nice table.

It’s better than the one we showed earlier.

Way cleaner, way better.

Exactly. And now I can say, “Well, it’s all there. I could ask which one should I prioritize first, but okay, we’ve already done that demo, so not that impressive.” But the demo we didn’t do is, “Hey, I want to assign the API endpoint. I want to send it to Vlad because Vlad is the one that should be working on that.” So let me do that. Assign the API endpoint issue to Vlad Catrinescu.

So if you don’t want issues assigned to you, make your GitHub username tough.

Exactly.

Or get a complicated name like me.

Exactly. So what’s going to happen here is it’s going to know assigning an issue is an endpoint that I referenced in my action. So now Copilot will know how to call, where to call, how to authenticate, and do all of that. And it’s going to, even before it calls it, it’s going to give you a really cool thing. It’s going to tell you, “Hey, by the way, I’m going to share that data with GitHub.” So from a data transparency standpoint, it’s really interesting. I’m asking for four parameters. The owner is Sebastien Levert. The owner is the owner repo, not the owner of the task. The repo is a GitHub agent. The issue number is six. Look at that, six. And the assignee is now Vlad Catrinescu.

So this is a dev thing, or will it always be shown?

It’s always going to be shown. And the reason why is you want to make it extremely transparent for people to understand what gets transferred across the wire. We can let Copilot know, “I don’t want to have anything to do with that.” We can say it’s not consequential, but when it’s consequential, we really want you to put that. And now I’m going to hit confirm. I’m not going to get a sign-in button because I’m already signed in. The first time you’re going to say, “Wait a second, before you actually log in, before you can do that, we need to log you in. You’re going to log into your GitHub account. We’re going to do that, and it’s going to work.” Then I’m going to hit confirm. And now the GitHub agent should be working on… Oh, actually, look, I might have created a new version of that. So it actually asked me to sign in. Perfect for our demo. So I’m going to sign in. It’s going to pop up on a screen. It’s clever enough to know a lightweight version of SSO. And now it should go on and assign the issue. So what did it do? How do I know what it is called how it is called and how it found the right endpoint? That’s where the plugin dev info is interesting. So now it’s going to tell you, “We found one plugin. It’s a GitHub plugin. In the plugin, we found two matched functions. We found the issues update and the add assignees. We decided to select the issues and add assignees based on their needs. And then afterwards, here’s how we called it.” And now you can actually open this and see the entire endpoint. So it did a post on this with this endpoint, with this body, and it got a 200 back. 200 means success, means that Copilot can return back and say all of that to us, that it actually succeeded.

That’s amazing. So you really have a bit of transparency into how did it get to that answer.

Exactly. And we’re really working on improving these experiences to make it even more valuable for things outside of the actions realm, for how it ground on SharePoint, how it ground on the graph, what these capabilities were used. So we’re working on that as we speak. But what I wanted to show is that as devs, you have a great opportunity to bring that really rich experience. And now just to prove that we’re not lying, there’s one here.

Exactly. The API endpoint now I just assigned to you and myself. I added you right there.

And there you go.

That’s amazing. And that’s really how simple it is to bring content from graph connectors and act upon it using an API, but all of that using our Teams Toolkit capabilities that we built.

That’s amazing, the fact that you can add actions. And I think that’s where a lot of the value is. And it kind of reminds me of the terms and concepts when you introduced that slide about agents where retrieval is kind of step one. But now we’re at step two where we can make it do actions and do things for us without ever quitting that Copilot UI.

Exactly. Something I want to be very, very mindful of is this experience is not a pro-code experience. It’s not necessarily a pro-code experience. We’re also bringing that capability inside Agent Builder. Connected to, again, an open API description, you’re going to be able to do that as a maker. And in Copilot Studio, you also have the opportunity to connect to Power Platform connectors to do something similar to approve, to get to the same thing. Where we believe that there’s a high value here, we connected to the GitHub API, but maybe you need to build your own API to connect to your internal systems. And we really think that there is also how pro devs will have a great experience with it.

Definitely, and like you said, Power Platform has over a thousand connectors, so definitely before you build your own, see if you can use one of those and not have to maintain it. But every company will need to build their own eventually. Unless you’re a very small company with not that many systems, most enterprises will need to get in here.

Exactly. So here we’re building with Visual Studio Code agent, and we hope you like it. We hope you’ll be able to dive into it. And as Vlad was saying, we’re going to make all the code available for everybody.

That’s awesome, Seb. Well, thank you so much for creating the series with me. You shared so much knowledge and really gave us a step-by-step guide on how to customize Copilot from the basics and understanding what an agent is and what skills, knowledge, and all those terms mean, all the way to creating an agent without going in the UI at all, just everything in Visual Studio.

For everyone, if you found this valuable, please go and connect with Seb. You have all his social media handles in the description below. And we will make sure to update the series as Microsoft adds different ways to customize Copilot or they rebrand something or change a logo or something, which might happen. Hopefully not, but it might happen. So for now, until we add new videos, make sure you check out some of the other YouTube videos from my channel that the YouTube algorithm thinks you’ll find interesting. They’ll be on your screen right now. And thank you so much again. Thank you, everyone, for watching, and we really hope you enjoyed it. So thank you. Cheers.