Collecting logs from Microsoft 365¶
TeskaLabs LogMan.io can collect logs from Microsoft 365, formerly Microsoft Office 365.
There are following classes of Microsoft 365 logs:
-
Audit logs: They contain information about various user, admin, system, and policy actions and events from Azure Active Directory, Exchange and SharePoint.
-
Message Trace: It provides an ability to gain an insight into the e-mail traffic passing thru Microsoft Office 365 Exchange mail server.
Enable auditing of Microsoft 365¶
By default, audit logging is enabled for Microsoft 365 and Office 365 enterprise organizations. However, when setting up logging of a Microsoft 365 or Office 365 organization, you should verify the auditing status of Microsoft Office 365.
1) Go to https://compliance.microsoft.com/ and sign in
2) In the left navigation pane of the Microsoft 365 compliance center, click Audit
3) Click the Start recording user and admin activity banner
It may take up to 60 minutes for the change to take effect.
For more details, see Turn auditing on or off.
Configuration of Microsoft 365¶
Before you can collect logs from Microsoft 365, you must configure Microsoft 365. Be aware that configuration takes a significant amount of time.
1) Setup a subscription to Microsoft 365 and a subscription to Azure
You need a subscription to Microsoft 365 and a subscription to Azure that has been associated with your Microsoft 365 subscription.
You can use trial subscriptions to both Microsoft 365 and Azure to get started.
For more details, see Welcome to the Office 365 Developer Program.
2) Register your TeskaLabs LogMan.io collector in Azure AD
It allows you to establish an identity for TeskaLabs LogMan.io and assign specific permissions it needs to collect logs from Microsoft 365 API.
Sign in to the Azure portal, using the credential from your subscription to Microsoft 365 you wish to use.
3) Navigate to Azure Active Directory
4) On the Azure Active Directory page, select "App registrations" (1), and then select "New registration" (2)
5) Fill the registration form for TeskaLabs LogMan.io application
- Name: "TeskaLabs LogMan.io"
- Supported account types: "Account in this organizational directory only"
- Redirect URL: None
Press "Register" to complete the process.
6) Collect essential informations
Store following informations from the registered application page at Azure Portal:
- Application (client) ID aka
client_id - Directory (tenant) ID aka
tenant_id
7) Create a client secret
The client secret is used for the safe authorization and access of TeskaLabs LogMan.io.
After the page for your app is displayed, select Certificates & secrets (1) in the left pane. Then select "Client secrets" tab (2). On this tab, create new client secrets (3).
8) Fill in the information about a new client secret
- Description: "TeskaLabs LogMan.io Client Secret"
- Expires: 24 months
Press "Add" to continue.
9) Click the clipboard icon to copy the client secret value to the clipboard
Store the Value (not the Secret ID) for a configuration of TeskaLabs LogMan.io, it will be used as client_secret.
10) Specify the permissions for TeskaLabs LogMan.io to access the Microsoft 365 Management APIs
Go to App registrations > All applications in the Azure Portal and select "TeskaLabs LogMan.io".
11) Select API Permissions (1) in the left pane and then click Add a permission (2)
12) On the Microsoft APIs tab, select Microsoft 365 Management APIs
13) On the flyout page, select the all types of permissions
- Delegated permissions
ActivityFeed.ReadActivityFeed.ReadDlpServiceHealth.Read
- Application permissions
ActivityFeed.ReadActivityFeed.ReadDlpServiceHealth.Read
Click "Add permissions" to finish.
14) Add "Microsoft Graph" permissions
- Delegated permissions
AuditLog.Read.All
- Application permissions
AuditLog.Read.All
Select "Microsoft Graph", "Delegated permissions", then seek and select "AuditLog.Read.All" in "Audit Log".
Then select again "Microsoft Graph", "Application permissions" then seek and select "AuditLog.Read.All" in "Audit Log".
15) Add "Office 365 Exchange online" permissions for collecting Message Trace reports
Click on "Add a permission" again.
Then go to "APIs my organization uses".
Type "Office 365 Exchange Online" to search bar.
Finally select "Office 365 Exchange Online" entry.
Select "Application permissions".
Type "ReportingWebService" into a search bar.
Check the "ReportingWebService.Read.All" select box.
Finally click on "Add permissions" button.
16) Grant admin consent
17) Navigate to Azure Active Directory
18) Navigate to Roles and administrators
19) Assign TeskaLabs LogMan.io to Global Reader role
Type "Global Reader" into a search bar.
Then click on "Global Reader" entry.
Select "Add assignments".
Type "TeskaLabs LogMan.io" into a search bar. Alternatively use "Application (client) ID" from previous steps.
Select "TeskaLabs LogMan.io" entry, the entry will appear in "Selected items".
Hit "Add" button.
Congratulations! Your Microsoft 365 is now ready for an log collection.
Configuration of TeskaLabs LogMan.io¶
Example¶
connection:MSOffice365:MSOffice365Connection:
client_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
tenant_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
client_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Collect Microsoft 365 Audit.General
input:MSOffice365:MSOffice365Source1:
connection: MSOffice365Connection
content_type: Audit.General
output: ms-office365-01
# Collect Microsoft 365 Audit.SharePoint
input:MSOffice365:MSOffice365Source2:
connection: MSOffice365Connection
content_type: Audit.SharePoint
output: ms-office365-01
# Collect Microsoft 365 Audit.Exchange
input:MSOffice365:MSOffice365Source3:
connection: MSOffice365Connection
content_type: Audit.Exchange
output: ms-office365-01
# Collect Microsoft 365 Audit.AzureActiveDirectory
input:MSOffice365:MSOffice365Source4:
connection: MSOffice365Connection
content_type: Audit.AzureActiveDirectory
output: ms-office365-01
# Collect Microsoft 365 DLP.All
input:MSOffice365:MSOffice365Source5:
connection: MSOffice365Connection
content_type: DLP.All
output: ms-office365-01
output:XXXXXX:ms-office365-01: {}
# Collect Microsoft 365 Message Trace logs
input:MSOffice365MessageTraceSource:MSOffice365MTSource1:
connection: MSOffice365Connection
output: ms-office365-message-trace-01
output:XXXXXX:ms-office365-message-trace-01: {}
Connection¶
The connection to Microsoft 365 must be configured first in the connection:MSOffice365:... section.
connection:MSOffice365:MSOffice365Connection:
client_id: # Application (client) ID from Azure Portal
tenant_id: # Directory (tenant) ID from Azure Portal
client_secret: # Client secret value from Azure Portal
resources: # (optional) resource to get data from separated by comma (,) (default: https://manage.office.com,https://outlook.office365.com)
Danger
Fields client id, tenant_id and client secret MUST be specified for a successful connection to Microsoft 365.
Collecting from Microsoft 365 activity logs¶
Configuration options to set up the collection fot the Auditing logs (Audit.AzureActiveDirectory, Audit.SharePoint, Audit.Exchange, Audit.General and DLP.All):
input:MSOffice365:MSOffice365Source1:
connection: # ID of the MSOffice365 connection
output: # Which output to send the incoming events to
content_type: # (optional but advised) Content type of obtained logs (default: Audit.AzureActiveDirectory Audit.SharePoint Audit.Exchange Audit.General DLP.All)
refresh: # (optional) The refresh interval in seconds to obtain messages from the API (default: 600)
last_value_storage: # (optional) Persistent storage for the current last value (default: ./var/last_value_storage)
Collecting from Microfost 365 Message Trace¶
Configuration options to set up the source of data of Microsoft 365 Message Trace:
input:MSOffice365MessageTraceSource:MSOffice365MessageTraceSource1:
connection: # ID of the MSOffice365 connection
output: # Which output to send the incoming events to
refresh: # (optional) The refresh interval in seconds to obtain messages from the API (default: 600)
last_value_storage: # (optional) Persistent storage for the current last value (default: ./var/last_value_storage)
Refresh of the client secret¶
The client secret will expire after 24 months and it has to be periodically recreated.
1) Navigate to Azure Active Directory.
2) Go to "App registrations" and select "TeskaLabs LogMan.io".
3) Create a new client secret.
Go to "Certificates & secrets".
Hit "New client secret" in "Client secrets" tab.
Fill "TeskaLabs LogMan.io Client Secret 2" in the Description. Use increasing numbers for new client secrets.
Select "730 days (24 mothns)" expiration.
Hit "Add" button.
4) Reconfigure TeskaLabs LogMan.io to use new client secrets.
5) Delete the old client secret.
Microsoft 365 Attributes explained¶
| Attribute | Description | Values as an example | Notes | Full list (ext) | |
|---|---|---|---|---|---|
| o365.audit.ActorContextId | ID of the user or service account that performed the action. | 571c8d2c-1ae2-486d-a17c-81bf54cbaa15 | |||
| o365.audit.ApplicationId | Application identifier (unique letter+number string) | 89bee1f7-5e6e-4d8a-9f3d-ecd601259da7 | |||
| o365.audit.AzureActiveDirectoryEventType | The type of Azure Active Directory event. The following values indicate the type of event. | 0 - Indicates an account login event. 1 - Indicates an Azure application security event. |
|||
| o365.audit.DeviceProperties | Source device properties such as OS, browser type etc. | Name:"OS" Value:"Linux" } {2 items Name:"BrowserType" Value:"Firefox" } {2 items Name:"IsCompliantAndManaged" Value:"False" } {2 items Name:"SessionId" Value:"e94ad17c-354f-4009-a9ee-34900770e997" |
Parcing of these properties is still in progress | ||
| o365.audit.ErrorNumber | An error code string that can be used to classify types of errors that occur, and should be used to react to errors. | 0, 50140, 501314 ... | https://learn.microsoft.com/en-us/azure/active-directory/develop/reference-aadsts-error-codes | ||
| o365.audit.ExtraProperties | Not defined yet | // | |||
| o365.audit.FileSizeBytes | FIle size in bytes | 23301 | |||
| o365.audit.InterSystemsId | Unique inter system ID string | acc33436-ee63-4d81-b6ee-544998a1c7d9 | |||
| o365.audit.IntraSystemId | Unique intra system ID string | 01dd20c0-edb9-4aaa-a51b-2bf38e1a8900 | |||
| o365.audit.ItemName | Unique item name | b1379a75-ce5e-4fa3-80c6-89bb39bf646c | |||
| o365.audit.LogonError | Error message displayed after failed login | InvalidUserNameOrPassword, TriggerBrowserCapabilitiesInterrupt, InvalidPasswordExpiredPassword | |||
| o365.audit.ObjectId | URL path to accesed file | https://telescopetest.sharepoint.com/sites/Shared Documents/Docs/o365 - logs.xlsx | |||
| o365.audit.RecordType | The type of operation indicated by the record. This property indicates the service or feature that the operation was triggered in. | 6 | https://learn.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema#auditlogrecordtype | ||
| o365.audit.ResultStatus | Triggered response | Success, Fail | |||
| o365.audit.SourceFileExtension | Accessed file extension (format type). | .xlsx, .pdf, .doc etc. | |||
| o365.audit.SourceFileName | Name of file user accessed | "o365.attributesexplained.xlsx" | |||
| o365.audit.SupportTicketId | ID of the potential Support ticket, after user opened a support request in Azure Active Directory. | // | The customer support ticket ID for the action in "act-on-behalf-of" situations. | ||
| o365.audit.TargetContextId | The GUID of the organization that the targeted user belongs to. | 571c8d2c-1ae2-486d-a17c-81bf54cbaa15 | |||
| o365.audit.UserKey | An alternative ID for the user identified in the UserID property. For example, this property is populated with the passport unique ID (PUID) for events performed by users in SharePoint. This property also might specify the same value as the UserID property for events occurring in other services and events performed by system accounts. | i:0h.f|membership|1003200224fe6604@live.com | |||
| o365.audit.UserType | The type of user that performed the operation. The following values indicate the user type. | 0 - A regular user. 2 - An administrator in your Microsoft 365 organization.1 3 - A Microsoft datacenter administrator or datacenter system account. 4 - A system account. 5 - An application. 6 - A service principal. 7 - A custom policy. 8 - A system policy. |
|||
| o365.audit.Version | Indicates the version number of the activity (identified by the Operation property) that's logged. | 1 | |||
| o365.audit.Workload | The Microsoft 365 service where the activity occurred. | AzureActiveDirectory | |||
| o365.message.id | This is the Internet message ID (also known as the Client ID) found in the message header in the Message-ID: header field. | 08f1e0f6806a47b4ac103961109ae6ef@server.domain | This ID should be unique; however, not all sending mail systems behave the same way. As a result, there's a possibility that you may get results for multiple messages when querying upon a single Message ID. | ||
| o365.message.index | Value of MessageTrace Index | 1, 2, 3 ... | |||
| o365.message.size | Size of the sent/received message in bytes. | 33489 | |||
| o365.message.status | Following action after sending the message. | Delivered, FilteredAsSpam, Expanded | https://learn.microsoft.com/en-us/exchange/monitoring/trace-an-email-message/run-a-message-trace-and-view-results | ||
| o365.message.subject | Message subject; can be written uniquely. | "Binding Offer Letter for Ms. Smith" | |||