How to analyze Conditional Access Policies with ‘Report Only’ Mode?

Conditional Access Polices can be setup in 3 main modes. On/ Off/ Report Only.

On and Off modes are self explanatory where “Report Only” mode needs additional work. This post will go in detail on how to use the Report Only mode before you actually switch to ON.

Read more about Conditional Access Policies

Azure AD Conditional Access Policies 101

Why “Report Only”

This setting is ideal when you don’t want to switch ON the CA Policy without 100% sure about the results. Will there be a disruption to the work? Will the users get blocked? Will they be prompted for the MFA challenge?

In a situation like that, you can set the policy to Report Mode and then start inspection. Best to run the policy in the report only mode for few days and collect data to understand the policy’s behavior.

Set the policy to “Report Only” mode

Once you created the policy, set the state to “Report Only” mode. This will not activate the policy, but will monitor the conditions.

---

1. Create the Log Analytics Workspace

You must have an Azure subscription in order to create the Log Analytics Workspace

• Create a new resource in your resource group. Search for Log Analytics Workspace

• Hit Create

• Complete the creation

---

2. Connect the Log Analytics Workspace to Azure AD

• Go to the Azure AD portal > Go to Azure Active Directory > select Diagnostics Settings under Monitoring section

• Click on Add diagnostic setting
• Select the option Send to Log Analytics Workspace
• Select the Subscription and the Log Analytics Workspace and select the options under the Log

---

3. Read Reports

Once the Log Analytics Workspace has been connected to the Azure AD to send data to it,

Go to the Azure AD Portal > All Services > Azure AD Conditional Access > Insights and Reporting

Go to the Impact Summary Section and it will give you the repots in detail

Optional

• Set the Auto Refresh to ON and the desired interval so you’ll get more realistic data

Workbook Gallery

Now that the Azure AD is connected to the Log Analytics workspace, if you navigate to the Azure AD > Workbooks under Monitoring, you can see more templates where you can use to run more reports

Final Thoughts

This is always a good idea to understand how your CA Policies will apply to the users and devices when you apply them. Best to run this in Report Only mode for few days to get a good view on the CA Policy end state.