EventTracker v8.x and above Publication Date: March 5, 2017
Abstract This guide provides instructions to configure Office 365 to generate logs for critical events. Once EventTracker is configured to collect and parse these logs, dashboard and reports can be configured to monitor Office 365 usage. Scope The configurations detailed in this guide are consistent with EventTracker Enterprise version 7.x and later, and Microsoft Office 365. Audience IT Admins, Office 365 administrators and EventTracker users who wish to forward logs to EventTracker Manager and monitor events using Event Tracker Enterprise. The information contained in this document represents the current view of EventTracker. on the issues discussed as of the date of publication. Because EventTracker must respond to changing market conditions, it should not be interpreted to be a commitment on the part of EventTracker, and EventTracker cannot guarantee the accuracy of any information presented after the date of publication. This document is for informational purposes only. EventTracker MAKES NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, this paper may be freely distributed without permission from EventTracker, if its content is unaltered, nothing is added to the content and credit to EventTracker is provided. EventTracker may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from EventTracker, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. The example companies, organizations, products, people and events depicted herein are fictitious. No association with any real company, organization, product, person or event is intended or should be inferred. 2018 EventTracker Security LLC. All rights reserved. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. 1
Table of Contents Abstract... 1 Scope... 1 Audience... 1 Overview... 3 Prerequisites... 3 Configure Office 365 to forward logs to EventTracker... 3 Assign Report Reader Permission to an Office 365 User... 6 Register Application with your Azure Active Directory Tenant... 9 To find your Office 365 tenant ID in the Azure AD portal... 13 EventTracker Knowledge Pack (KP)... 13 Alert... 13 Reports... 13 Dashboards... 15 Import Knowledge Pack into EventTracker... 16 Import Knowledge Objects... 16 Import Category... 18 Import Tokens... 19 Import Flex Reports... 21 Verify Knowledge Pack in EventTracker... 23 Verify Knowledge Object... 23 Verify Category... 23 Verify Token Values... 24 Verify Flex Reports... 25 Create Dashboards in EventTracker... 26 Schedule Reports... 26 Create Dashlets... 28 Import Dashlet... 32 Sample Reports... 35 Sample Dashboards... 38 2
Overview EventTracker Knowledge pack for Office 365 captures important and critical activities in Exchange, Azure Active Directory, SharePoint, OneDrive and Skype. Monitoring these activities are critical from a security aspect and is required for compliance and operational reasons. The dashboards, reports will help you in getting deeper insights to analyze various security use cases like login activities from different country, changes in user permission, malicious file detection in SharePoint and OneDrive, spam and malicious email detection and mailbox auditing. EventTracker detects and alerts a spoofed email from the received emails. EventTracker helps you to monitor day to day activities of Office 365 Exchange like mailbox usage, summary of mail traffic, stale mailbox information and files uploaded/downloaded from SharePoint etc. Prerequisites EventTracker v8.x or above should be installed. PowerShell 5.0 should be installed on EventTracker Manager/Agent machine where Office 365 integrator application is running. User should have administrative privilege on EventTracker Manager/Agent machine. Office 365 service account details who have Report reader permission. Instructions are mentioned here App registered in Azure AD with Microsoft graph API permission. Instructions are mentioned here Configure Office 365 to forward logs to EventTracker 1. Contact EventTracker support for office365 Report pack. 2. Download the file on EventTracker Server or any other system having EventTracker Agent. 3. Save Office365Integrator.zip. (Here we are using d:\office365integrator\ as example). 4. Extract and run executable file Office 365 Integrator.exe. 5. After launching integrator, it will check for EventTracker Agent and PowerShell 5.0. If both are available, then integrator will allow you to configure office 365. 3
Figure 1 Else you must install EventTracker agent as well as PowerShell 5.0 in the machine. Figure 2 6. Fill Office 365 service account details who have Report reader permission. Service account with administrative access is not required to fetch the logs and a normal service account with Report reader permission would suffice. For creating a service account with Report reader permissions, please follow the instruction mentioned here. 7. If you wish to fetch office 365 usage statistics logs like mailbox, SharePoint, OneDrive, skype usage, etc., please check Fetch mailbox, SharePoint, OneDrive and Skype usage statistics option and fill the details of the app registered in Azure AD with Microsoft graph API permission. If user doesn t have app registered in Azure AD, please follow the instruction mentioned here. 4
8. Provide the tenant ID for the enterprise. Please follow the instruction mentioned here, if tenant ID is not known. 9. After filling all details, please select OK button and check if the following task is created in task Scheduler. Figure 3 10. Also Verify LFM (Log file monitor) is created in EventTracker Agent Configuration. 5
Figure 4 Assign Report Reader Permission to an Office 365 User For creating Office 365 service account with Report reader role permission, please follow below procedure. This procedure should be carried out by a user having Administrator rights in Office365. 1. Click here to go to the Office 365 admin center. 2. Go to the Office 365 admin center by selecting the app launcher icon Office 365 app launcher icon in the upper-left and choosing Admin. 6
Figure 5 3. On the left, select ACTIVE USERS and then select the + sign to Add new users. 7
Figure 6 4. On the Create new user account, populate the necessary fields. 5. Uncheck the box for Make this user change their password with Outlook on the web on next login. and click on Create. 6. In the Admin center, select Users. 7. On the Active user s page, choose the user whose administrator role you want to change. The properties page for the user opens. 8. Next to Roles, choose Edit. If you don't see the Edit button, then you don't have global admin permissions and can't assign admin roles to other people. Ask a global admin in your business to assign roles for you. In a small business, the business owner (the person who purchased Office 365) is a global admin. In a large business, key people in the IT department are global admins. Figure 7 8
9. Choose the Edit button next to Roles. 10. Choose Report Reader roles and Save it. Register Application with your Azure Active Directory Tenant If Application has not been registered in Azure AD, please follow the below procedure. This procedure should be carried out by a user having Administrator rights in Office365. For granting permissions user should be having administrator privileges. 1. Sign in to the Azure portal. 2. If your account gives you access to more than one, click your account in the top right corner, and set your portal session to the desired Azure AD tenant. 3. In the left-hand navigation pane, click the Azure Active Directory service, click App registrations, and click New application registration. Figure 8 4. When the Create page appears, enter your application's registration information: Name: Enter a meaningful application name Application type: Select Web app / API Sign-On URL: For "Web app / API" applications, provide the http://localhost. 9
Figure 9 10
5. When finished, click Create. Azure AD assigns a unique Application ID to your application, and you are taken to your application's main registration page. Please note down Application ID. 6. To add permission(s) to access resource APIs from your client Click the Required Permissions section on the Settings page. Click the Add button. Click Select an API to select the type of resources you want to pick from. Browse through the list of available APIs or use the search box to select from the available resource applications in your directory that expose a web API. Click the resource you are interested in, then click on Select. You are taken to the Enable Access page. Select the Application Permissions and/or Delegated Permissions your application needs when accessing the API. Figure 10 7. After Adding Application, please add required permissions for Microsoft Graph and grant permissions for it. For granting permissions, user(s) with Administrator privileges is required. 11
Figure 11 8. You are taken to the application's main registration page, which opens the Settings page for the application. To add a secret key for your web application's credentials: Click the Keys section on the Settings page. Add a description for your key. Select Never expires duration. Click Save. The right-most column will contain the key value, after you save the configuration changes. Be sure to copy the key for use in your client application code, as it is not accessible once you leave this page. Figure 12 9. Please note down Application ID and Client Secret after registering App. 12
To find your Office 365 tenant ID in the Azure AD portal 1. Sign in to the Azure portal. 2. In the Microsoft Azure portal, click Azure Active Directory. 3. Under Manage, click Properties. The tenant ID is shown in the Directory ID box. Figure 13 EventTracker Knowledge Pack (KP) Once logs are received in EventTracker; category, alert, reports and dashboards can be configured in EventTracker. The following Knowledge Packs are available in EventTracker v7 and later to support Office 365 monitoring: Alert Office 365 - Exchange Spam Mail Traffic Details: This alert will generate when spam mail is received on office 365 exchange server. Office 365 Exchange Malware Detected: This alert will generate when malware is detected by office 365 exchange server. Office 365 Exchange Spoofed Mail Detected: This alert will generate when the sender of mail is spoofed. Reports Office 365 - Exchange Mail Traffic Details: This report provides information related to total mail traffic. 13
Office 365 - Exchange Inbound Mail Traffic Details: This report provides information related to inbound mail traffic. Office 365 - Exchange Outbound Mail Traffic Details: This report provides information related to outbound mail traffic. Office 365 - Exchange Spam Mail Traffic Details: This report provides information related to spam mail traffic. Office 365 - Exchange Malware Traffic Details: This report provides information related to malware containing mail traffic. Office 365 - Exchange Mailbox Transport Rule Traffic Details: This report provides information related to mail traffic matched by transport rule. Office 365 - Exchange Message Trace Details: This report provides information related to mails sent and received by various UPN s. Office 365 - Email activity counts: This report provides information related to email activity (mail sent, received, etc.) happened in last one week. Office 365 - Email app usage user counts: This report provides information related to app used to access office 365 exchange in last one week Office 365 - Email app usage user details: This report provides information related to app used by user to access office 365 exchange mail. Office 365 - Email app user counts: This report provides statistics related to app used by user for accessing mail. Office 365 - Mailbox usage details: This report provides information related to usage of mailbox. Office 365 - Mailbox usage mailbox counts: This report provides statistics information related to usage of mailbox. Office 365 - Mailbox usage quota status: This report provides information related to mailbox quota usage. Office 365 - Mailbox storage usage: This report provides information related to usage of storage provided to user in office 365 exchange. Office 365 - Activation counts: This report provides information related to licenses activation in office 365. Office 365 - Activation user counts: This report provides statistics related to user for which license is activated. Office 365 - Active user counts: This report provides information related to active user in office 365. Office 365 - OneDrive activity file counts: This report provides statistics information related to file used in OneDrive by user. Office 365 - OneDrive activity user counts: This report provides statistics information related to activities occurred in OneDrive by user. 14
Office 365 - OneDrive usage account details: This report provides detail information about usage of OneDrive. Office 365 - OneDrive usage storage: This report provides information related to usage of storage provided in office 365. Office 365 - SharePoint activity pages: This report provides information about activity happened for a page in SharePoint. Office 365 - SharePoint activity user details: This report provides information about user activities in SharePoint. Office 365 - SharePoint site usage file counts: This report provides information about usage of files in SharePoint sites. Office 365 - SharePoint site page usage: This report provides information about page usage in SharePoint sites. Office 365 - SharePoint site storage usage: This report provides information about usage of storage provided to SharePoint sites. Office 365 - Skype for business activity counts: This report provides statistics information related to skype for business activities. Office 365 - Skype for business activity user counts: This report provides statistics information about user of skype for business. Office 365 - Skype for business device usage distribution user counts: This report provides statistics information about device usage for skype for business. Office 365 - Skype for business peer to peer activity counts: This report provides statistics information about peer to peer activities of skype for business. Office 365 - Skype for business peer to peer activity user counts: This report provides statistics information for a user of skype for business. Office 365 - Unified audit details: This report provides details information about audit events generated for Azure Active Directory, OneDrive, SharePoint, Skype for business, etc. Dashboards Exchange Top Mail Today: This dashlet displays top mail users for each day. Exchange Accounts Created Last Week: This dashlet displays total accounts created for every week. Exchange Active Users Last Week: This dashlet displays total active users for every week. Exchange Outbound Mail Count Last Week: This dashlet displays total outbound mail count for every week. Exchange Top Inactive Users Last Week: This dashlet displays total inactive users for every week. Exchange Top Mail Size Today: This dashlet displays top mail sizes for each day. Exchange Top Mail User Today: This dashlet displays top mail users for each day. Exchange Top Spammers Today: This dashlet displays top spam mail senders for each day. 15
Exchange OS Usage: This dashlet displays operating systems used for connection. Exchange Browser Usage: This dashlet displays browsers used for connection. Exchange Client Usage: This dashlet displays clients used for connection. Exchange Mailbox Space Usage: This dashlet displays mailbox storage used per UPN. Import Knowledge Pack into EventTracker 1. Launch EventTracker Control Panel. 2. Double click Export/Import Utility, and then click the Import tab. 3. Import Tokens/Flex Reports as given below. Import Knowledge Objects Figure 14 1. Click Knowledge objects under Admin option in the EventTracker manager page. 2. Locate the file named KO_Office365 Exchange.etko. 16
Figure 15 3. Now select all the check box and then click on Import option. 17
Figure 16 4. Knowledge objects are now imported successfully. Import Category Figure 17 1. Click Category option, and then click the browse button. 18
Figure 18 2. Locate.iscat file, and then click the Open button. 3. To import categories, click the Import button. EventTracker displays success message. 4. Click OK, and then click the Close button. Import Tokens Figure 19 Click Token Value option, and then click the browse Locate O365.istoken file, and then click the Open button. button. 19
To import token value, click the Import button. EventTracker displays success message. Figure 20 Click OK, and then click the Close button. Figure 21 20
Import Flex Reports 1. Click Reports option, and select new (.etcrx) from the option. Figure 22 2. Locate the file named FlexReports_Office365 Exchange.etcrx, and select all the check box. 21
Figure 23 3. Click the Import button to import the reports. EventTracker displays success message. Figure 24 22
Verify Knowledge Pack in EventTracker Verify Knowledge Object 1. Logon to EventTracker Enterprise. 2. Click the Admin menu, and then click Knowledge Object. 3. In Knowledge Object Group Tree to view imported knowledge object, scroll down and click Office 365 group folder. Knowledge Object are displayed in the pane. Verify Category Figure 25 1. Logon to EventTracker Enterprise. 2. Click the Admin menu, and then click Category. 3. In Category Group Tree to view imported category, scroll down and click Office 365 group folder. Category are displayed in the pane. 23
Verify Token Values Figure 26 1. Logon to EventTracker Enterprise. 2. Click the Admin menu, and then click Parsing Rules. 3. In Token Value Group Tree to view imported token values, scroll down and click Office 365 group folder. Token values are displayed in the token value pane. 24
Verify Flex Reports Figure 27 1. Logon to EventTracker Enterprise. 2. Click the Reports menu, and then Configuration. 3. Select Defined in report type. 4. In Report Groups Tree to view imported Scheduled Reports, scroll down and click Office 365 group folder. Reports are displayed in the Reports configuration pane. 25
Figure 28 Create Dashboards in EventTracker In case of EventTracker 9.0 and later, we recommend importing dashlet for Office 365. Schedule Reports 1. Open EventTracker in browser and logon. 26
Figure 29 2. Navigate to Reports>Configuration. Figure 30 3. Select Office 365 in report groups. Check Defined dialog box. 27
4. Click on schedule to plan a report for later execution. Figure 31 5. Choose appropriate time for report execution and in Step 8 check Persist data in Eventvault explorer box. Figure 32 6. Check column names to persist using PERSIST checkboxes beside them. Choose suitable Retention period. 7. Proceed to next step and click Schedule button. 8. Wait for scheduled time or generate report manually. Create Dashlets 1. EventTracker 8 and later is required to configure flex dashboard. 28
2. Open EventTracker in browser and logon. Figure 33 3. Navigate to Dashboard>Flex. Flex Dashboard pane is shown. 4. Click to add a new dashboard. Flex Dashboard configuration pane is shown. Figure 34 29
Figure 35 5. Fill fitting title and description and click Save button. 6. Click to configure a new flex dashlet. Widget configuration pane is shown. Figure 36 7. Locate earlier scheduled report in Data Source dropdown. 30
8. Select Chart Type from dropdown. 9. Select extent of data to be displayed in Duration dropdown. 10. Select computation type in Value Field Setting dropdown. 11. Select evaluation duration in As Of dropdown. 12. Select comparable values in X Axis with suitable label. 13. Select numeric values in Y Axis with suitable label. 14. Select comparable sequence in Legend. 15. Click Test button to evaluate. Evaluated chart is shown. 16. If satisfied, click Configure button. Figure 37 31
Figure 38 17. Click customize to locate and choose created dashlet. 18. Click to add dashlet to earlier created dashboard. Note: In case of EventTracker 9.0 and later, we don t need to create dashlet. We can import dashlet using EventTracker dashboard Console. Import Dashlet In EventTracker 9.0, we have added new feature which will help to import/export of dashlet. Following is the procedure to do that: 1. Login into EventTracker Enterprise Web console. Figure 39 32
2. Go to My Dashboard option. 3. Click on import button and select.etwd File. Figure 39 Figure 40 33
Figure 41 4. Click upload and select Dashboard which you want to import. 34
Figure 42 5. Click on Import button. It will upload all selected dashboard. Sample Reports 1. Office 365 - Exchange OS Usage Details 2. Office 365 - Exchange Browser Usage Details Figure 43 Figure 44 35
3. Office 365 - Exchange Client Usage Details: Figure 45 4. Office 365 - Exchange Spam Mail Traffic Details 5. Office 365 - Exchange Message Trace Details Figure 46 36
Figure 47 6. Office 365 - Exchange Inactive Mail User Details Figure 48 37
Sample Dashboards 1. Office 365 Exchange Top spam mail by sender Figure 49 38
2. Office 365 Exchange Top Spam mail by Recipient Figure 50 3. Office 365 Exchange Malicious Email by Threat Name Figure 51 39
4. Office 365 Exchange Malicious Email by Sender Figure 52 5. Office 365 Exchange Malicious Email by Recipient Figure 53 40
6. Office 365 Exchange Admin Activities by Operation 7. Office 365 Exchange Admin Activities by User Figure 54 Figure 55 41
8. Office 365 Exchange Activities by User Type Figure 56 42
9. Office 365 Azure Active Directory Login failed by Reason 10. Office 365 Azure Active Directory login by user Figure 57 Figure 58 43
11. Office 365 - Azure Active Directory Login by Status Figure 59 12. Office 365 Azure Active Directory Login failed by Country Figure 60 44
13. Office 365 Azure Active Directory Login Activities by Client IP 14. Office 365 Azure Active Directory Events Figure 61 Figure62 45
15. Office 365 SharePoint Activities by Operation 16. Office 365 SharePoint Activities by User Figure 62 Figure 63 46
17. Office 365 SharePoint Activities by User Agent 18. Office 365 SharePoint Activities by File Type Figure 64 Figure 65 47
19. Office 365 SharePoint Activities by File Extension 20. Office 365 OneDrive Activities Figure 66 Figure 67 48
21. Office 365 OneDrive Activities by Operation Figure 68 22. Office 365 OneDrive Activities by User Figure 69 49
23. Office 365 OneDrive Activities by User Agent Figure 70 24. Office 356 OneDrive Activities by Resource Figure 71 50
25. Office 365 OneDrive Activities by File Extension 26. Office 365 Exchange Top Sender Figure 72 Figure 73 51
27. Office 365 Exchange Top Recipient Figure 74 28. Office 365 SharePoint Activities Figure 75 52