Quick guide to using the Conditional Access Optimisation Agent

This blog is a high level walk-through the CAO Agent, showing the fundamentals of the product, demonstrating the capabilities and value it can provide organisations. Please ensure you check Microsoft's documentation for a more detailed dive around setting up Security Copilot and other features such as chatting with the agent which aren't covered here.

Firstly what is the Conditional Access Optimisation Agent?

In a nutshell, the agent helps administrators ensure that users and applications are protected by conditional access policies, based on best practices and Zero Trust.

Do I need it?

I'd say try it first and see if it flags up and gaps in your environment.

Is it free?

Suuuuurrreee!!!

No, it's not.

You'll need to setup a new workspace within the Microsoft Security Copilot portal and assign SCU's (security compute units) accordingly.

Each time the agent is run, it uses 0.3 SCU (1 SCU is $4). Out of the box it's configured to run every 24 hours, but you can turn that off and run it manually.

Setting up the Conditional Access Optimisation Agent

Microsoft have thankfully made this process really straight forward, so you'll be up and running in no time.

Note: You'll need to set the agent up with an assigned identity, but for this demo, I'm just using a GA account.

Navigate to the Security Copilot portal https://securitycopilot.microsoft.com/ and select "Agents" from the left hand menu.

You'll see all the agents currently available within your tenant, so find the CAO Agent and click the Set up button.

On the next window, click Set up in Entra which takes you automatically to the Entra Portal.

Click the View details button.

I'd recommend leaving the option "Allow agent to make new policies in report only mode by itself" ticked.

To run the agent, click the Start Agent button.

After a minute or so, you'll see a really useful dashboard showing an overview of findings. Now this being a non prod test tenant, it doesn't contain many identities, plus the Security Defaults are enabled, hence there are recommendations to enable some of the more basic policies.

To see more information of each recommendation, click the Review Suggestion button(s) accordingly which will launch a new blade.

If you have Security Defaults still enabled, you'll get the following error when clicking the Turn on policy button.

To disable Security Defaults, go to the Azure Portal > Microsoft Entra ID > Properties > Manage Security defaults (near the bottom of the main page), then change the drop down menu option to Disabled.

Return to the Entra portal and CAO Agent. On the overview tab, select one of the policies by clicking Review Selection. For this demo I've chosen Block legacy authentication.

Now you have several considerations here.

1. Do you want to make any changes to the policy before deploying

2. Be aware the policy is in report only mode (this is sensible to ensure you're not implementing anything that may break prod)

3. The account you're using will be excluded from policy enforcement. I would suggest changing that to exclude your Break Glass accounts.

Changes can be made by clicking the Edit button at the top of the screen which will take you to the policy editor.

Once enabled, the blade will change its status to On.

What Are the Other Options?

Going from left to right, select Activities to get a view of costs associated with running the Agent, along with date, time and duration.

Suggestions is just another way to see the recommended Conditional Access Policies the agent has come up with.

Settings allows you to make changes and show additional information including

• disabling the automatic analysis every 24 hours

• Whether you want to scan Users / Applications or not

• Allowing the agent to autonomously create new policies in report mode

• Allow the agent to send Teams messages and to who

• Whether you want to allow the agent to do a low risk approach to deploying CA Policies

• Details on the connected account

• RBAC

• ServiceNow ticketing connectivity

• Custom Instructions (optional)

More info available here https://learn.microsoft.com/en-gb/entra/security-copilot/conditional-access-agent-optimization#settings

Now I've just started my journey with the CAO Agent, so there's still lots to learn, but I'm liking what I've seen so far, and I think this sets a good precedence for things to come.

As I continue with testing, I'm keen to see whether new policy complexities will come up along with Intune integration. So perhaps there might be a follow up blog at a later date to see how I've got on with it.

I'd also like the option to configure the run schedule at different intervals, perhaps every few days or once a week?

However, I'd definitely recommend companies set this up and at least try it to see if there's anything being flagged up that would help further secure their tenants.

If you're already a user or considering using the CAO Agent, I'd be keen to hear your thoughts too!