Collecting logs from Microsoft 365

TeskaLabs LogMan.io can collect logs from Microsoft 365, formerly Microsoft Office 365.

There are following classes of Microsoft 365 logs:

1) Audit logs: They contain information about various user, admin, system, and policy actions and events from Azure Active Directory, Exchange and SharePoint.

2) Message Trace: It provides an ability to gain an insight into the e-mail traffic passing thru Microsoft Office 365 Exchange mail server.

Enable auditing of Microsoft 365

By default, audit logging is enabled for Microsoft 365 and Office 365 enterprise organizations. However, when setting up logging of a Microsoft 365 or Office 365 organization, you should verify the auditing status of Microsoft Office 365.

1) Go to https://compliance.microsoft.com/ and sign in.

2) In the left navigation pane of the Microsoft 365 compliance center, click Audit.

3) Click the Start recording user and admin activity banner.

It may take up to 60 minutes for the change to take effect.

For more details, see Turn auditing on or off.

Configuration of Microsoft 365

Before you can collect logs from Microsoft 365, you must configure Microsoft 365. Be aware that configuration takes a significant amount of time.

1) Setup a subscription to Microsoft 365 and a subscription to Azure.

You need a subscription to Microsoft 365 and a subscription to Azure that has been associated with your Microsoft 365 subscription. You can use trial subscriptions to both Microsoft 365 and Azure to get started.
For more details, see Welcome to the Office 365 Developer Program.

2) Register your instance of TeskaLabs LogMan.io in Azure AD.

It allows you to establish an identity for TeskaLabs LogMan.io and assign specific permissions it needs to collect logs from Microsoft 365 API.

Sign in to the Azure portal, using the credential from your subscription to Microsoft 365 you wish to use.

3) Navigate to Azure Active Directory.

4) On the Azure Active Directory page, select "App registrations" (1), and then select "New registration" (2).

5) Fill the registration form for TeskaLabs LogMan.io application

Name: "TeskaLabs LogMan.io"
Supported account types: "Account in this organizational directory only"
Redirect URL: None

Press "Register" to complete the process.

6) Collect essential informations

Store following informations from the registered application page at Azure Portal:

Application (client) ID
Directory (tenant) ID

7) Create a client secret

The client secret is used for the safe authorization and access of TeskaLabs LogMan.io.

After the page for your app is displayed, select Certificates & secrets (1) in the left pane. Then select "Client secrets" tab (2). On this tab, create new client secrets (3).

8) Fill in the information about a new client secret

Description: "TeskaLabs LogMan.io Client Secret"
Expires: 24 months

Press "Add" to continue.

9) Click the clipboard icon to copy the client secret value to the clipboard.

Store the Value (not the Secret ID) for a configuration of TeskaLabs LogMan.io.

10) Specify the permissions for TeskaLabs LogMan.io to access the Microsoft 365 Management APIs

Go to App registrations > All applications in the Azure Portal and select "TeskaLabs LogMan.io".

11) Select API Permissions (1) in the left pane and then click Add a permission (2).

12) On the Microsoft APIs tab, select Microsoft 365 Management APIs.

13) On the flyout page, select the all types of permissions

Delegated permissions
ActivityFeed.Read
ActivityFeed.ReadDlp
ServiceHealth.Read
Application permissions
ActivityFeed.Read
ActivityFeed.ReadDlp
ServiceHealth.Read

Click "Add permissions" to finish.

14) Add "Microsoft Graph" permissions.

Delegated permissions
AuditLog.Read.All
Application permissions
AuditLog.Read.All

Select "Microsoft Graph", "Delegated permissions", then seek and select "AuditLog.Read.All" in "Audit Log".

Then select again "Microsoft Graph", "Application permissions" then seek and select "AuditLog.Read.All" in "Audit Log".

15) Add "Office 365 Exchange online" permissions for collecting Message Trace reports.

Click on "Add a permission" again.
Then go to "APIs my organization uses".
Type "Office 365 Exchange Online" to search bar.
Finally select "Office 365 Exchange Online" entry.

Select "Application permissions".
Type "ReportingWebService" into a search bar.
Check the "ReportingWebService.Read.All" select box.
Finally click on "Add permissions" button.

16) Grant admin consent

17) Navigate to Azure Active Directory

18) Navigate to Roles and administrators

19) Assign TeskaLabs LogMan.io to Global Reader role

Type "Global Reader" into a search bar.
Then click on "Global Reader" entry.

Select "Add assignments".
Type "TeskaLabs LogMan.io" into a search bar. Alternatively use "Application (client) ID" from previous steps.
Select "TeskaLabs LogMan.io" entry, the entry will appear in "Selected items". Hit "Add" button.

Congratulations! Your Microsoft 365 is now ready for an log collection.

Configuration of TeskaLabs LogMan.io

Connection

It is first needed to set up a connection for periodic OAuth token renewal:

connection:MSOffice365:MSOffice365Connection:
  client_id:  # Application (client) ID from Azure Portal
  tenant_id:  # Directory (tenant) ID from Azure Portal
  client_secret:  # Client secret value from Azure Portal
  resources:  # (optional) resource to get data from separated by comma (,) (default: https://manage.office.com,https://outlook.office365.com)

Client ID, tenant ID and client secret have to always be specified.

Input from Audit

Configuration options to set up the connection and queries:

connection:  # ID of the MSOffice365 connection 
content_type:  # (optional) Content type of obtained logs (default: Audit.AzureActiveDirectory Audit.Exchange Audit.SharePoint Audit.General)
resource:  # (optional) resource to get data from (default: https://manage.office.com)

output:  # Which output to send the incoming events to

encoding:  # (optional) Charset encoding of the server response bulk content (default: utf-8)
last_value_storage:  # (optional) Persistent storage for the current last value (default: ./var/last_value_storage)

Example

input:MSOffice365Source:MyMSOffice365Source:
connection: MSOffice365Connection
...

Input from Message Trace

Configuration options to set up the source of data of MS Office Message Trace:

connection:  # ID of the MSOffice365 connection

output:  # Which output to send the incoming events to

resource:  # (optional) resource to get data from (default: https://outlook.office365.com)
encoding:  # (optional) Charset encoding of the server response bulk content (default: utf-8)
last_value_storage:  # (optional) Persistent storage for the current last value (default: ./var/last_value_storage)
refresh:  # (optional) The refresh interval in seconds to obtain messages from the API (default: 600)

Example

input:MSOffice365MessageTraceSource:MSOffice365MessageTraceSource:
connection: MSOffice365Connection
...

Refresh of the client secret

The client secret will expire after 24 months and it has to be periodically recreated.

1) Navigate to Azure Active Directory.

2) Go to "App registrations" and select "TeskaLabs LogMan.io".

3) Create a new client secret.

Go to "Certificates & secrets".
Hit "New client secret" in "Client secrets" tab.
Fill "TeskaLabs LogMan.io Client Secret 2" in the Description. Use increasing numbers for new client secrets.
Select "730 days (24 mothns)" expiration. Hit "Add" button.

4) Reconfigure TeskaLabs LogMan.io to use new client secrets.

5) Delete the old client secret.

Microsoft 365 Attributes explained

Attribute	Description	Values as an example	Notes	Full list (ext)
o365.audit.ActorContextId	ID of the user or service account that performed the action.	571c8d2c-1ae2-486d-a17c-81bf54cbaa15
o365.audit.ApplicationId	Application identifier (unique letter+number string)	89bee1f7-5e6e-4d8a-9f3d-ecd601259da7
o365.audit.AzureActiveDirectoryEventType	The type of Azure Active Directory event. The following values indicate the type of event.	0 - Indicates an account login event. 1 - Indicates an Azure application security event.
o365.audit.DeviceProperties	Source device properties such as OS, browser type etc.	Name:"OS" Value:"Linux" } {2 items Name:"BrowserType" Value:"Firefox" } {2 items Name:"IsCompliantAndManaged" Value:"False" } {2 items Name:"SessionId" Value:"e94ad17c-354f-4009-a9ee-34900770e997"	Parcing of these properties is still in progress
o365.audit.ErrorNumber	An error code string that can be used to classify types of errors that occur, and should be used to react to errors.	0, 50140, 501314 ...		https://learn.microsoft.com/en-us/azure/active-directory/develop/reference-aadsts-error-codes
o365.audit.ExtraProperties	Not defined yet	//
o365.audit.FileSizeBytes	FIle size in bytes	23301
o365.audit.InterSystemsId	Unique inter system ID string	acc33436-ee63-4d81-b6ee-544998a1c7d9
o365.audit.IntraSystemId	Unique intra system ID string	01dd20c0-edb9-4aaa-a51b-2bf38e1a8900
o365.audit.ItemName	Unique item name	b1379a75-ce5e-4fa3-80c6-89bb39bf646c
o365.audit.LogonError	Error message displayed after failed login	InvalidUserNameOrPassword, TriggerBrowserCapabilitiesInterrupt, InvalidPasswordExpiredPassword
o365.audit.ObjectId	URL path to accesed file	https://telescopetest.sharepoint.com/sites/Shared Documents/Docs/o365 - logs.xlsx
o365.audit.RecordType	The type of operation indicated by the record. This property indicates the service or feature that the operation was triggered in.	6		https://learn.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema#auditlogrecordtype
o365.audit.ResultStatus	Triggered response	Success, Fail
o365.audit.SourceFileExtension	Accessed file extension (format type).	.xlsx, .pdf, .doc etc.
o365.audit.SourceFileName	Name of file user accessed	"o365.attributesexplained.xlsx"
o365.audit.SupportTicketId	ID of the potential Support ticket, after user opened a support request in Azure Active Directory.	//	The customer support ticket ID for the action in "act-on-behalf-of" situations.
o365.audit.TargetContextId	The GUID of the organization that the targeted user belongs to.	571c8d2c-1ae2-486d-a17c-81bf54cbaa15
o365.audit.UserKey	An alternative ID for the user identified in the UserID property. For example, this property is populated with the passport unique ID (PUID) for events performed by users in SharePoint. This property also might specify the same value as the UserID property for events occurring in other services and events performed by system accounts.	i:0h.f\|membership\|1003200224fe6604@live.com
o365.audit.UserType	The type of user that performed the operation. The following values indicate the user type.	0 - A regular user. 2 - An administrator in your Microsoft 365 organization.1 3 - A Microsoft datacenter administrator or datacenter system account. 4 - A system account. 5 - An application. 6 - A service principal. 7 - A custom policy. 8 - A system policy.
o365.audit.Version	Indicates the version number of the activity (identified by the Operation property) that's logged.	1
o365.audit.Workload	The Microsoft 365 service where the activity occurred.	AzureActiveDirectory
o365.message.id	This is the Internet message ID (also known as the Client ID) found in the message header in the Message-ID: header field.	08f1e0f6806a47b4ac103961109ae6ef@server.domain	This ID should be unique; however, not all sending mail systems behave the same way. As a result, there's a possibility that you may get results for multiple messages when querying upon a single Message ID.
o365.message.index	Value of MessageTrace Index	1, 2, 3 ...
o365.message.size	Size of the sent/received message in bytes.	33489
o365.message.status	Following action after sending the message.	Delivered, FilteredAsSpam, Expanded		https://learn.microsoft.com/en-us/exchange/monitoring/trace-an-email-message/run-a-message-trace-and-view-results
o365.message.subject	Message subject; can be written uniquely.	"Binding Offer Letter for Ms. Smith"