METADATA FRAMEWORK 6.3. Probe Configuration

Transcription

1 METADATA FRAMEWORK 6.3 Probe Configuration

2 Publishing Information Software version Document version 34 Publication date May 22, 2017 Copyright Varonis Systems Inc. All rights reserved. This information shall only be used in conjunction with services contracted for with Varonis Systems, Inc. and shall not be used to the detriment of Varonis Systems, Inc. in any manner. User agrees not to copy, reproduce, sell, license, or transfer this information without prior written consent of Varonis Systems, Inc. Other brands and products are trademarks of their respective holders.

3 1 INTRODUCTION This document describes the configuration options available for the CIFS filter and Probe. Most of the configuration parameters described below are preconfigured to optimal default values and should not be changed unless there is specific need to do so. Important: A number of configuration parameters are duplicated across various sections of the configuration files. These parameters are presented only once in the document; duplicates are not presented. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 1

4 2 PROBE SERVICES The Probe has two services: Varonis IDU Monitor - Monitors the Probe service as well as other environment parameters as described below, and starts or stops the Probe as needed. Varonis IDU Probe - The Probe service itself. The Probe service connects to the file servers and collects file access events. To manually install the probe services, use the following commands: VrnsMonSvc.exe /service VrnsProbeSvc.exe /service To manually uninstall the services use the following commands: VrnsMonSvc.exe /unregserver VrnsProbeSvc.exe /unregserver METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 2

5 3 PROBE SERVICE CONFIGURATION FILES The Probe service configuration files provide a very high level of granularity in controlling the Probe's behavior. All the files described below are located in the Program Files\Varonis \Probe folder. These files contain the configuration parameters in XML format, which can be edited manually. If a change is made, the Probe must be restarted for the change to take effect. data.config File This file is used to specify the connection parameters the Probe uses to connect to the database. Provider - Determines the database provider used by the Probe. This value must never be changed from the default set value. ConnectionString - The encrypted database connection string. This value must not be changed. If an unencrypted connection string is used the Probe prints a warning in the Event Log. logging.config File This file is used to set event log parameters: LoggingLevel - The detail level of the events written to the Event Log. The default and highest value is 4, which logs events at a debug level. The logging level may be set to the following: 0 - None 1 - Error 2 - Warning 3 - Info 4 - Debug LogMaxSize - Sets the size of the Varonis event log (default is MB). smtp.config File This file contains the SMTP parameters used for sending notifications. The information in this file is overwritten every time the Probe is restarted, based on the information stored in the database. Therefore, changes made to this file have no effect and are overwritten the next time the Probe is restarted. Use the Configuration window in the UI to make changes to the SMTP parameters. VrnsProbeSvc.exe.user.config File This file is provided out of the box as VrnsProbeSvc.exe.user.config.template. To use it, it must be renamed to VrnsProbeSvc.exe.user.config. The file enables overwriting the configuration determined by VrnsProbeSvc.exe.varonis.config. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 3

6 VrnsProbeSvc.exe.varonis.config File This file is provided out of the box as VrnsProbeSvc.exe.user.config.template. To use it, it must be renamed to VrnsProbeSvc.exe.user.config. The file enables overwriting the configuration determined by VrnsProbeSvc.exe.varonis.config. Logging Section File - Indicates other files from which to receive parameters. By default, this parameter points to logging.config, smtp.config and VrnsProbeSvc.exe.user.config. TraceLoggerEnabled - Enables or disables trace and message logging (0 - disabled by default). Do not enable unless instructed to do so by Engineering. LoggingLevel - Similar to the parameter in logging.config. LoggingProvider - Determines the provider that is used to log information. By default, this is set to use the Event Viewer. It can be configured to use a log file instead by changing 'EventVwr' to 'File' as the first parameter. The second parameter determines the DLL that is used to log events. LogFilterThreshold - Sets the threshold to filter messages by error code (by default 5). SaveDataFromPreviousRun - Determines whether to keep the files from the last run (for logging to file only; default is 1). MaxNumOfFiles - Number of files to keep (for logging to file only; default is 7). FileMaxSize - Maximum file size (for logging to file only; default 10 MB). Trace Logging The Trace Logging and Message Logging sections are internal sections used by the core and must not be edited unless some special issue occurs. They are enabled by the TraceLoggerEnabled flag. FileName - Determines the name of the file in which to write special tracing (default - trace.varonis.log). LoggingLevel - Similar to the parameter in logging.config. Message Logging The Trace Logging and Message Logging sections are internal sections used by the core and must not be edited unless some special issue occurs. They are enabled by the TraceLoggerEnabled flag. FileName - Determines the name of the file in which to write special tracing (default - message.varonis.log). LoggingLevel - Similar to the parameter in logging.config. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 4

7 Database Section File - Indicates which files are used to set up the database connection string. By default it is set to use the data.config file described above. ConnectionString - Determines the format of a valid connection string (this parameter must not be edited). Build Section This section contains information about the build. The section is rarely used, but must not be edited. ExplorerSDK Section This section contains information about an internal tool used by the Probe. It must not be edited. Probe Section This section contains most of the Probe's operating parameters, including connection strategy and timeouts when the Probe tries to connect or reconnect to a file server. File - Indicates other files from which to receive parameters. By default, this parameter points to VrnsProbeSvc.exe.user.config. servicename - The name of the Probe service (can be used in net start/stop commands or batch files). displayname - The name of the service as it appears in the Services Manager list. bypassmonitor - When set to 1, the Probe service starts even if no monitor service is present. Default is 0. This value should not be changed during normal operation. monitorservice - The name of the monitor service the Probe communicates with and sends keep-alive ping messages to. DAL - Indicates how internal communicate with the database is achieved. This parameter must not be edited. Version - The version number of the Probe. bulkinsertenable - Enables performing a bulk insert of events to the database. Use of this parameter increases overall performance. Default is enabled. bulkinsertqueuesize - Indicates the maximum size of the queue the Probe can maintain before it must send events to the database. bulkinsertqueuetimeout - Indicates the maximum timeout for which the Probe can maintain events in the queue before it must send them to the database. connectverify - The time period after which the Probe checks all the connections and tries to reconnect to any disconnected file server based on the logic above. The effect is identical to a Probe restart, with regard to file server reconnection. KeppAliveInterval Indicates the time interval at which the monitor thread checks various activity issues. CheckAliveDBConnection Indicates the time interval at which the database connection is checked to ensure it is active. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 5

8 CheckAliveWorkerActivityInterval Indicates the time interval at which the threads are checked to ensure they are still responding. WaitAfterFilersConnect Indicates the maximum timeout that the Probe waits after connecting to a file server. MaximumEventPathLength Indicates the maximum number of characters the Probe allows in path names. MaximumEventFileNameLength Indicates the maximum number of characters the Probe allows in file names. Filer Section This section determines the configuration of all file servers: threadpoolsize - The size of the thread pool. This parameter must not be changed from the default value of 10 unless you are instructed otherwise by Engineering. enablecache - The Probe can cache events before they are written to the database, so that if the same event is repeated several times within a minute it is written only once to the database. Caching events lowers the load on the database, and should be enabled unless you are instructed otherwise by engineering. cachesize - The size of the cache when it is enabled. queuesize - The size of the queue used by the Probe to store events from each file server when the database is overloaded and there is a risk of losing events. The events are held in the queue until the database becomes available and they can be written. Connection Parameters The Probe uses the following parameters to determine how it connects to a file server if it was disconnected. The monitor service includes built restart logic, which is activated only when the Probe must be shut down due to an error or overload conditions. Under normal operating conditions, the monitor and Probe services do not restart. When a file server is disconnected, the Probe attempts to reconnect a number of times up to the value specified by the RetryMax parameter, waiting for the amount of time specified by RetryWait between reconnect attempts. If a connection is established but not sustained, the Probe considers the attempt as failed and continues the count toward RetryMax. This logic prevents unstable connections from keeping the Probe in a constant reconnect mode. The value specified by RetryReset determines how long a connection needs to remain stable to reset the retry counter. retryreset - Determines how long a connection needs to exist before the retry counter can be reset. retrywait - The time to wait before attempting to reconnect to a file server. retrymax - The maximum number of reconnect attempts before the Probe stops attempting to connect (until the value specified by connectverify has elapsed). METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 6

9 lostcountinterval - The number of events that are lost by the Probe (that is, not written to the database) before an error message is logged. usewnetapi - The WNet API provides better compatibility when connecting to Windows 2000 servers and replaces the impersonation method used in prior versions. Do not edit this parameter unless instructed to do so by Engineering. Common Configuration EnableLocationFiltering - Enables filtering folders for which events are not collected (enabled by default). EnablePatternFilter Enables the pattern filter mechanism. This mechanism was created to enable filtering temporary files and operations that are detected as part of a short sequence of events. For example, files which were removed immediately after creation are temporary and need not be reported. If a folder was created and then immediately renamed, a custom Create Folder event containing its final name should be reported, because it is a standard pattern of events occurring during folder creation in Windows Explorer. The default is 1 (mechanism enabled). MaxPatternLength - Defines the maximum length for one pattern to be evaluated. MaxPatternTimestampMs - Defines the maximum pattern time length (timestamp in milliseconds). EnableFilenameExtentionFiltering Enables filtering based on file extension for which events need not be gathered (default is Disabled). QueueSizeMin - The number of events that trigger the Probe to connect to the file server and collect events. If the CIFS filter on the file server side has fewer events, the Probe connects only after the time specified by QueueWaitTime has passed. However, if the file server is heavily loaded, the Probe may connect more frequently if the file server has collected the number of events specified by QueueSizeMin. QueueWaitTime - The interval the Probe uses to connect to the file server and collect events. This is set by default to 60 seconds (keep this in mind when demonstrating events capture by the Probe, since events are not displayed immediately in the log screen). The value is in milliseconds. RpcTimeOutFactor - The amount of time to add to the queue wait time to configure the required timeout (default is 3 minutes). [long]requirerpcauth Indicates whether authentication is required. 0 = no authentication, 1 = RPC authentication, 2 = RPC authentication and encryption. [long]pull_event_wait - Indicates the time to wait if the last time the Probe connected to the file server there actually were events to be collected. [long]pull_noevent_wait - Indicates the time to wait if the last time the Probe connected to the file server there were no events to be collected. LogBufSize - The size of the log buffer the file server allocates to store events when they are read from the event log. This is the initial size of the buffer. If the buffer is filled, the Probe doubles the size of the buffer until it reaches the value specified by LogBufSizeMax. LogBufSizeMax - The maximum size of the log buffer the Probe may allocate for pulling audit log events from the EMC machine. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 7

10 Proxy File Server Parameters ProxyHost - The host name of the proxy file server (by default, this parameter is not used). [long]proxyport - The port of the proxy file server (by default, this parameter is not used). ProxyUserName - The user name of the proxy file server (by default, this parameter is not used). ProxyPassword - The password of the proxy file server (by default, this parameter is not used). Windows File Server Parameters These parameters can be applied to a specific file server rather than all file servers by adding the file server ID at the end of the parameter name. For example, QueueWaitTime101 applies the value only to file server 101 and not to all file servers. disableauth - When this parameter is set to 0 on the Probe and the file server, no authentication is required for the Probe to connect to the file server and collect events. While this option is not currently in use, it is intended to be used only when a problem is encountered with the connection between the Probe and the file server. NetApp File Server Parameters [long]minimumsizefor100precent - Indicates the minimum file size that must be read 100 percent to mark a file as read. [long]minimumsizefor50precent - Indicates the minimum file size that must be read at least 50 percent to mark a file as read. fileops - Provides a mask that allows only specific operations to be monitored. Each operation is represented by a single bit in the mask. The list of operations can be found in the FILE_OPERATIONS table in the Probe database. The default value is 0, which captures all the events supported by the OnTap version running on the file server. Similar to the Windows parameters, fileops can be set to apply to a specific file server rather than all file servers by adding the file server ID at the end of the parameter name (for example, fileops101, fileops5, etc.). [long]cifs_fpolicy_async - Determines whether the Fpolicy functions in asynchronous mode. 0 = asynchronous mode disabled, 1 = enabled, 2 = auto-detect the best mode for the file server (default 2). [long]nfs_fpolicy_fullpath - Enables easily finding the full path of a file. [long]cifs_fpolicy_version - Indicates the version of the FPolicy. This is relevant only for proxy mode and only if auto-discovery cannot be used. 0 - invalid value. [long]fpolicy_version_auto - If enabled, the version might be changed according to best practice for the NetApp file server. If disabled, the value of CIFS_Fpolicy_Version is the only value that is used. [long]disablesetsecevents - Disables Set Security events. [long]fpolicy_enabled_protocols - Indicates supported protocols: 1 = CIFS, 2 = NFS, 3 = CIFS and NFS. [long]open_read_op_selector - Indicates the operation that is used (bitwise) as "OPEN": 1 = OPEN, 2 = READ, 4 = WRITE, 0x8000 enables logic read (false positive handle). METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 8

11 EMC Celerra File Server Parameters HandleCacheSize - The size of the internal EMC Probe handle cache, used to resolve handle events from the EMC Audit log to object (file/dir) names. This parameter must not be changed without direct instructions from Engineering. VerifyLogSettings - Determines whether the audit log settings on the EMC machine are verified (using remote registry connectivity) when the Probe starts. LogRetention - Sets the value of the EMC security log's Retention key to 0, which instructs the log to overwrite events as needed. LogFileSize - The size of the security event log on the EMC machine. LogFilePath - The path to access the log file on the file server. Not currently in use. [long]fix_cellera_delete_file_op - If a directory containing a dot (.) in it its file name is deleted, this parameter determines it was an EMC error and replaces the event with a file event. EMC CEPA Parameters [long]cepapathresolution - Determines how to resolve paths from CEPA. 0 = Detect a virtual data mover, 1 = Assume data mover, 2 = Assume virtual data mover. Active Directory Parameters [long]enablecache - Enable caching (default = 0). Hitachi NAS File Server Parameters [long]pathlookupfailuretimeout - Timeout if path resolution fails, in seconds. If path resolution failed, no path resolution is performed during the defined timeout. Disabled = 0. [long]pathlookupvaliditytime - Time for which the resolved path is valid, in seconds. 0 = Always. [long]collectcreateevents - Defines whether Create events are collected. 0 = Not collected, 1 = Collected. [long]collectrenameevents - Defines whether Rename events are collected. 0 = Not collected, 1 = Collected. [long]suppressrpcerrorlimit - Counter that suppresses RPC errors upon reconnection. [long]retryresetbluearc - Overwrite retryreset (see general section above). If value > 0, the mechanism is reset (since retryresetbluearc < retrywaitbluearc X retrymaxbluearc). Default = 10 seconds. [long]retrywaitbluearc - Overwrite retrywait (see general section above) if value > 0. Default = 0.5 minutes. [long]retrymaxbluearc - Overwrite retrymax (see general section above) if value > 0. [long]autoreopenlogwait Automatically reopen the event viewer log if no new event has arrived for some time (in milliseconds). 0 = Disabled. Unix File Server Parameters [long]unixqueuesizemin - The number of events that trigger the Probe to connect to the file server and collect events. If the CIFS filter on the file server side has fewer events, the METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 9

12 Probe connects only after the time specified by QueueWaitTime has passed. However, if the file server is heavily loaded, the Probe may connect more frequently if the file server has collected the number of events specified by QueueSizeMin. [long]unixqueuetimeout - Indicates the maximum timeout for which the Probe can maintain events in the queue before it must send them to the database. [long]unixrpctimeoutconstant - The amount of time to add to the queue wait time to configure the required timeout (default is 30 seconds). [long]unixprobeidletime - Determines the Probe's allowed idle time. [long]rpcfilerprognum - The program number in the Create Port function. [long]probe_rpc_port - The RPC port in use for the Probe. SharePoint File Server Parameters [long]delay_before_handle_events - The time to wait (in milliseconds) before inserting events. [long]intervalfordeletingeventsminutes The interval (in minutes) at which events are deleted. 0 = Deletion timing not in use. [long]offsetfordeletingeventsminutes - 0 = Deletes events older than [last received event in the Probe - offset]. [long]hour24fordeletingevents - If events are deleted at a specific time, this parameter sets the hour at which events are deleted (value must be >= 0 <=23). [long]minfordeletingevents - If events are deleted at a specific time, this parameter sets the minutes at which events are deleted (value must be >=0 <=59). [long]leaveeventcopy Set to leave a copy of events on SharePoint after the Varonis agent extracts them. [long]objectmodelevents - Extract events which are associated with an object model (that is, events that are not created by SharePoint itself). [long]logdelay - Extract events from older dates, no sooner than this delay (in milliseconds). [long]activateremoteagent - Since the SharePoint monitor activates the agent, this parameter is not normally in use. [long]auditcheckinterval - Interval (in minutes) at which the audit check is performed. [long]errorfilterreset - The interval (in milliseconds) at which error filtering behavior is reset. [long]stackinerrorreport - Set to include a call stack in the error message, as well as how many are allowed. [long]errorfilterthreashold - Number of messages to be reported (the count is periodically cleared). [long]siteinerrorthreashold - The number of faulting sites to show in each error. Redundant sites are discarded. SP_IsLegacyAuditMethod - Determines whether Legacy mode is used (to work with the SharePoint database) or New mode is used (to work with CIFSQueue). METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 10

13 Exchange Storage Group Parameters [long]enablelocationfiltering - Enables filtering folders for which events are not collected (enabled by default). EnableFilenameExtentionFiltering Enables filtering based on file extension for which events need not be gathered (default is Disabled). Resource Section The Probe enables definition of more than one resource to be monitored per file server. For example, both CAS and STORE might be defined on the same Exchange machine. Most of the configuration parameters in the Resource section have the same meaning as their counterparts in the Filers section. Filers Section The Filers section of the file indicates which DLL the Probe uses for each file server type. Do not change this section unless instructed to do so by Engineering. It is included for future expansion and flexibility. Each file server type is listed in this section along with the DLL the Probe uses to access the file server and collect events. AutoLoader Section The AutoLoader is a core internal DLL that provides some Unix functionality for the Probe. This section is rarely in use and will likely be deprecated in the future. Do not make any changes to this section. VrnsMonSvc.exe.config File This file controls the monitor service. It can be modified to determine how the Probe service and resources on the server are monitored and what action to take when a resource anomaly is detected. Logging and Database Sections These sections are identical to the sections found in the VrnsProbeSvc.exe.config file. See the Logging and Database sections above for information on these sections. VrnsMonSvc.exe.varonis.config File This is the Monitor configuration file. The general file structure is similar to that of VrnsProbeSvc.exe.varonis.config and is overwritten by VrnsMonSvc.exe.user.config. New paramaters (compared to VrnsMonSvc.exe.user.config): 1. servicename - The name of the monitor service (can be used in net start/stop commands or batch files). 2. displayname - The name of the monitor service as it appears in the Services Manager list. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 11

14 3. setrecoveryopt - Toggles the use of the service recovery option (available under Service Properties on the Recovery tab). 4. recoverysleep - Sets the recovery parameter that determines the time to wait before attempting to restart the service (in minutes). The value in the file is written to the service properties. 5. monitorwait - The ping period of monitored services in milliseconds. 6. pingtimeout - The time that must elapse with no service ping before the monitor shuts down all services. Services Section The Services section lists all the services that the monitor starts or stops along with itself. By default it contains only the Probe service (VrnsProbeSvc.exe). More services may be added in the future if necessary. Checks Section The Checks section lists the various services and subsystems that the monitor service watches. The first part lists all the monitored components and allows monitoring to be turned on or off based on the value of each key. The default list includes the following components: SystemCpu ProbeSvcRun ProbeCpu ProbeMem The next sections list the specific monitoring parameters for each component. Some parameters are used by all the components while others are specific to only one. Common parameters include: Check - The name of the component to be monitored. The value must be one of the values listed at the beginning of the section. Threshold - The value of this parameter may refer to different measurements, depending on the component. For CPU components (overall CPU and Probe service CPU), it determines the CPU percent threshold. For Probe memory, it determines the memory consumption threshold. For the Probe service itself, it merely indicates that the process must run (a value of 0 means the process is not running). The interpretation of the value is handled by the DLL listed in the Plugins part of this section. Deviation - The time (in milliseconds) that the threshold is crossed before an alert is triggered. Other parameters may contain the name of the monitored service or process. In addition to the default set of components, other services or processes can be added by using the same format that is used to configure monitoring of the Probe service and process (any process may be added, even processes not directly related to DatAdvantage if there is a reason for monitoring them). METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 12

15 Plugins Section This section lists all the monitored components and the DLL used for monitoring each component. Currently all of the components are monitored by vrnschk.dll. However, this architecture allows the functionality to be extended easily in the future by adding other DLLs or replacing the vrnschk.dll with another. METADATA FRAMEWORK 6.3 PROBE CONFIGURATION 13