ENABLING RPC OVER HTTPS CONNECTIONS TO M-FILES SERVER

Similar documents
RPC Over HTTP Install Windows Server 2003 Configure your Exchange 2003 front-end server as an RPC Proxy server

VMware Horizon View Deployment

On-demand target, up and running

Documentation. nfront Web Password Change. Version nfront Security. All Rights Reserved.

Configure the IM and Presence Service to Integrate with the Microsoft Exchange Server

Configuring Remote Access using the RDS Gateway

VMware Identity Manager Connector Installation and Configuration (Legacy Mode)

Workspace ONE UEM Certificate Authentication for EAS with ADCS. VMware Workspace ONE UEM 1902

Bomgar Vault Server Installation Guide

How to Configure Authentication and Access Control (AAA)

Installing and Configuring VMware Identity Manager Connector (Windows) OCT 2018 VMware Identity Manager VMware Identity Manager 3.

Advanced option settings on the command line. Set the interface and ports for the OpenVPN daemons

Version Installation Guide. 1 Bocada Installation Guide

BlackBerry UEM Configuration Guide

Configuration Guide. BlackBerry UEM. Version 12.9

Installing and Configuring vcloud Connector

Link Platform Manual. Version 5.0 Release Jan 2017

Configuration Guide. BlackBerry UEM Cloud

Remote Desktop Services Deployment Guide

Remote Desktop Services. Deployment Guide

Android Mobile Single Sign-On to VMware Workspace ONE. SEP 2018 VMware Workspace ONE VMware Identity Manager VMware Identity Manager 3.

BIG-IP Access Policy Manager : Secure Web Gateway. Version 13.0

Deploying F5 with Microsoft Remote Desktop Services

Managing Certificates

10ZiG Manager Cloud Setup Guide

Configuring Cisco Unified MeetingPlace Web Conferencing Security Features

Enforced Client Policy & Reporting Server (EPRS) 2.3. Administration Guide

How to Set Up External CA VPN Certificates

Configuration Guide. BlackBerry UEM. Version 12.7 Maintenance Release 2

Installing and Configuring vcenter Multi-Hypervisor Manager

Installing and Configuring vcloud Connector

LANDesk Management Gateway. Users Guide to Using the Management Gateway 4.2 and prior versions

Security in Bomgar Remote Support

Deploying the BIG-IP System v11 with Microsoft Exchange 2010 and 2013 Client Access Servers

Microsoft ISA 2006 Integration. Microsoft Internet Security and Acceleration Server (ISA) Integration Notes Introduction

Verify that your operating environment meets all hardware and software requirements. For detailed requirements

VMware Content Gateway to Unified Access Gateway Migration Guide

How to configure the UTM Web Application Firewall for Microsoft Remote Desktop Gateway connectivity

This PDF Document was generated for free by the Aloaha PDF Suite If you want to learn how to make your own PDF Documents visit:

LifeSize Control Installation Guide

HP Instant Support Enterprise Edition (ISEE) Security overview

Deploying the BIG-IP System v11 with Microsoft Exchange 2010 and 2013 Client Access Servers

Ubiquity Server Manual

Security in the Privileged Remote Access Appliance

About This Document 3. Overview 3. System Requirements 3. Installation & Setup 4

HPE Security Fortify WebInspect Enterprise Software Version: Windows operating systems. Installation and Implementation Guide

Step-by-step installation guide for monitoring untrusted servers using Operations Manager

Configuring F5 for SSL Intercept

How to Configure SSL Interception in the Firewall

Installing AX Server with PostgreSQL (multi-server)

Installation Guide Worksoft Certify Execution Suite

Secure Web Appliance. SSL Intercept

M-FILES SETUP: ADVANCED USER'S GUIDE

Message Networking 5.2 Administration print guide

AUTODESK DATA MANAGEMENT SERVER. Advanced Configuration Guide for Autodesk data management server

RED IM Integration with Bomgar Privileged Access

Cisco Expressway with Jabber Guest

Web Applications Installation. version

Dameware ADMINISTRATOR GUIDE. Version Last Updated: October 18, 2017

ENTRUST CONNECTOR Installation and Configuration Guide Version April 21, 2017

Prophet 21 Middleware Installation Guide. version 12.16

VMware Enterprise Systems Connector Installation and Configuration. JULY 2018 VMware Identity Manager 3.2 VMware Identity Manager VMware AirWatch 9.

Cisco TelePresence Conductor with Cisco Unified Communications Manager

Installation Guide for Pulse on Windows Server 2012

PCI DSS Compliance. White Paper Parallels Remote Application Server

NBC-IG Installation Guide. Version 7.2

Platform Compatibility... 1 Enhancements... 2 Known Issues... 3 Upgrading SonicOS Enhanced Image Procedures... 3 Related Technical Documentation...

VII. Corente Services SSL Client

Click Studios. Passwordstate. Installation Instructions

VI. Corente Services Client

Installation on Windows Server 2008

Novell Access Manager

VMware Identity Manager Cloud Deployment. Modified on 01 OCT 2017 VMware Identity Manager

Managing External Identity Sources

VMware Identity Manager Cloud Deployment. DEC 2017 VMware AirWatch 9.2 VMware Identity Manager

Policy Manager for IBM WebSphere DataPower 7.2: Configuration Guide

Administrator's Guide

System Administration

Fireware-Essentials. Number: Fireware Essentials Passing Score: 800 Time Limit: 120 min File Version: 7.

WatchGuard XCS and Outlook Web Access 2013

IFS TOUCH APPS SERVER INSTALLATION GUIDE

Exchange Pro 4.4. User Guide. March 2017

Sophos Mobile as a Service

Integrating AirWatch and VMware Identity Manager

Implementing Messaging Security for Exchange Server Clients

Configuring a Windows Server 2008 Applications Infrastructure

Entrust Connector (econnector) Venafi Trust Protection Platform

Getting Started with VMware View View 3.1

DEPLOYMENT GUIDE Version 1.2. Deploying the BIG-IP System v10 with Microsoft IIS 7.0 and 7.5

Deploying F5 with Microsoft Active Directory Federation Services

Internet Script Editor

Sophos Mobile Control SaaS startup guide. Product version: 7

Status Web Evaluator s Guide Software Pursuits, Inc.

VMware Workspace ONE UEM VMware AirWatch Cloud Connector

Step-by-Step Configuration

How to Set Up VPN Certificates

Deploying F5 with Microsoft Active Directory Federation Services

Webthority can provide single sign-on to web applications using one of the following authentication methods:

Administrator's Guide

Read the following information carefully, before you begin an upgrade.

Transcription:

M-FILES CORPORATION ENABLING RPC OVER HTTPS CONNECTIONS TO M-FILES SERVER LAST UPDATED DECEMBER 13, 2017 VERSION 2.9

Contents 1. Overview... 3 1.1 Prerequisites... 3 2. Network Layout... 4 2.1 Separate Proxy Server and M-Files Server... 4 2.2 Single Server... 5 2.3 Data Security of "RPC over HTTP with SSL"... 6 3. Configuring the Servers... 7 3.1 Proxy Server... 7 3.2 M-Files Server... 11 4. Configuring the Servers in a Multitenant Cloud Environment... 12 5. Configuring M-Files Desktop Computers... 12 5.1 Disable HTTPS Encryption... 13 6. Troubleshooting... 14 6.1 General Troubleshooting... 14 6.2 Testing RPC Proxy Settings... 15 6.3 Disabling Compression... 17 6.4 Client Trust for SSL Certificate... 17 6.5 Copying Configuration from a Working Computer... 17 6.6 SSL Offloading... 17 6.7 "Not enough storage is available to complete this operation"... 18 7. Change History... 18

1. Overview By default, M-Files Desktop communicates with M-Files Server by using the Remote Procedure Call (RPC) protocol (TCP/IP, port 2266). This is typically the preferred means of communication within the company's internal network as it requires no additional configuration steps. In M-Files 10.2 and later, users can enable encryption for the RPC communication between M-Files Desktop and M-Files Server by turning on the "Encrypted connection" option in M-Files Desktop Settings (formerly M-Files Client Settings). In some situations, it is desirable to enable M-Files Desktop to communicate with M-Files Server via the HTTPS protocol instead of RPC. This is especially useful if clients are connecting from outside the company's internal network. HTTPS connections are always encrypted and are typically not blocked in hotel networks or other public networks. This document provides instructions for enabling "RPC over HTTP with SSL" communication between M-Files Desktop and M-Files Server. In this configuration, all traffic from M-Files Desktop is encrypted and tunneled through TCP port 443. See the document Protecting Data in Transit with Encryption in M-Files for information on choosing between "RPC Encryption" and "RPC over HTTP with SSL". 1.1 Prerequisites Please make sure your environment meets these requirements before moving forward. 1.1.1 Client Computer Requirements Make sure the M-Files software installed on client computers meets these minimum requirements: COMPONENT M-Files Desktop REQUIREMENT M-Files 10.0 Service Release 3 (10.0.3911.85) or later. Compatibility with older client versions requires additional configuration tasks to be carried out on the server. For instructions for older M-Files Desktop versions, see this document. 1.1.2 Server Computer Requirements Make sure the proxy server meets the following minimum requirements: COMPONENT Operating system REQUIREMENT Windows Server 2003 Service Pack 2 or later. Windows Server 2008 R2 or later is recommended. Make sure the M-Files server (application server) meets the following minimum requirements: 3

COMPONENT Operating system M-Files Server REQUIREMENT Windows Server 2003 Service Pack 2 or later. Windows Server 2008 R2 or later is recommended. M-Files 9.0 or later. 2. Network Layout The HTTPS communication between M-Files Desktop and M-Files Server is based on the use of the RPC over HTTP protocol with SSL/TLS encryption. IIS (Internet Information Services) runs a component called RPC over HTTP Proxy that receives HTTPS traffic from the client and forwards it to M-Files Server as RPC calls. M-Files Server authenticates the user with the user's credentials. On the server side, there are two server roles in this setup: The server on which IIS is running is referred to as the proxy server. The server on which the M-Files Server software is running is referred to as the M-Files server. If you do not wish to separate IIS and M-Files Server to different servers, you can set up a single server that has both roles. Note: The RPC over HTTP Proxy component requires port 443 for its own use, and the port cannot be changed. This means that the proxy server cannot use port 443 for any other service. 2.1 Separate Proxy Server and M-Files Server The most common reason for separating the proxy server and M-Files server roles to separate servers is the additional security that this kind of isolation brings, especially if the proxy server is exposed to the public Internet. From the performance point of view, the roles do not typically need to be separated. Image 1 shows the network layout with separate proxy server and M-Files server computers. 4

Proxy Server in DMZ Public Internet User HTTPS (TCP port 443) DMZ Internal Network Firewall Software: - IIS with RPC over HTTP Proxy - M-Files Web Access (optional) RPC (TCP ports 2266 and 4466) RPC (TCP port 2266) Desktop user Proxy Server (dnsalias.domain.com) HTTPS (TCP port 443) M-Files Server (dnsalias.domain.local) User M-Files Server authenticates the user with the user s credentials. Image 1: Network layout with a separate proxy server in DMZ. If you set up a proxy server in addition to the actual M-Files Server as shown in Image 1, you will typically have two separate DNS names that eventually lead to the same M-Files Server (for example, dnsalias.domain.com and dnsalias.domain.local). To avoid confusion, you should use only one of those DNS names on any single client device. For example, in M-Files Desktop, do not configure two connections that point to the same vault by using different DNS aliases. 2.2 Single Server If the organization does not have a DMZ area in its network or does not want to set up a separate proxy server for other reasons, the role of the proxy server and the M-Files server can be combined (see Image 2). 5

No DMZ Public Internet User HTTPS (TCP port 443) Internal Network Firewall Software: - IIS with RPC over HTTP Proxy - M-Files Web Access (optional) - M-Files Server RPC (TCP port 2266) Desktop user M-Files Server (dnsalias.domain.com) HTTPS (TCP port 443) M-Files Server authenticates the user with the user s credentials. User Image 2: Network layout without a separate proxy server. Using a separate proxy server and placing it in the DMZ area of the network as described in section 2.1 provides additional isolation for M-Files Server and is the recommended approach. 2.3 Data Security of "RPC over HTTP with SSL" When using RPC over HTTP with SSL, the HTTPS communication between M-Files Desktop and the proxy server is protected by SSL/TLS encryption. Encryption of this traffic is critically important because in many usage scenarios, this traffic travels over the Internet. On the proxy server, IIS runs a component called RPC over HTTP Proxy that receives HTTPS traffic from the client and forwards it to M-Files Server as RPC calls. The SSL/TLS encryption is decrypted on the IIS level. The RPC communication between IIS and M-Files Server is not encrypted. This portion of the traffic travels within an organization's server network and behind a firewall. The RPC over HTTP Proxy component in IIS can forward calls only to computers and ports that are specified in the ValidPorts registry entry on the proxy server. By specifying only the M-Files server computer and the port 4466 in the ValidPorts entry, you can ensure that clients cannot attempt to communicate with any other RPC servers via the RPC over HTTP Proxy component. In addition, before forwarding traffic to a given port on a target RPC server, the RPC over HTTP Proxy component performs a special packet exchange with the RPC server listening on that port to verify it is willing to accept requests over HTTP. RPC servers cannot accept RPC over HTTP calls unless they specifically requested RPC to listen on RPC over HTTP by specifying the "ncacn_http" protocol sequence. This behavior provides additional protection for RPC servers that listen on a port that is listed in the ValidPorts registry entry on the proxy server: unless the RPC server has specifically requested to listen on RPC over HTTP, it will not receive calls originating from outside the firewall. 6

Optionally, you can strengthen the security of the M-Files system by enforcing the use of pre-shared key authentication on M-Files clients. See the documents Protecting Data in Transit with Encryption in M-Files and Securing Access to M-Files Vaults with a Pre-Shared Key for more information. Additionally, M-Files is compatible with SAML v2.0 compliant identity providers that support multi-factor authentication. See the document Deploying SAML v2.0 for M-Files Authentication for more information. 3. Configuring the Servers The server on which IIS is running is referred to as the proxy server. The server on which the M-Files Server software is running is referred to as the M-Files server. If you do not wish to separate IIS and M-Files Server to different servers, you can set up a single server that has both roles. Note: If you want to use M-Files Web and RPC over HTTPS on the same server, it is recommended to first configure M-Files Web and only then the RPC over HTTPS connection. For instructions on setting up M-Files Web on a proxy server, see How to Configure M-Files Web on a Separate Server Computer. Follow the steps below to install and configure the required software components on the servers. 3.1 Proxy Server The proxy server runs IIS. IIS receives the HTTPS traffic from the clients and converts it to RPC traffic for M-Files Server. Follow the steps below to install and configure IIS and the RPC over HTTP Proxy component on the proxy server. Note: If the proxy server is running multiple sites, the RPC over HTTP Proxy component must be added for the M-Files Web site. If M-Files Web is not the default web site of the proxy server, the following registry key needs to be added before installing the RPC over HTTP Proxy component: Key name: HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\RpcProxy Value name: WebSite Value type: REG_SZ Value data: <The name of your M-Files Web site> Note: The RPC over HTTP Proxy component requires port 443 for its own use, and the port cannot be changed. This means that the proxy server cannot use port 443 for any other service. 3.1.1 Install IIS and RPC over HTTP Proxy If the proxy server is running Windows Server 2003, follow these steps: 1. Select Start > Control Panel > Add or Remove Programs > Add/Remove Windows Components. 2. In the Application Server group, enable Internet Information Services (IIS). 3. In the Networking Services group, enable RPC over HTTP Proxy. If the proxy server is running Windows Server 2008 or later, follow these steps: 7

1. In Server Manager, under Roles, click Add Roles and add the Web Server (IIS) server role if not yet present. 2. In Server Manager, under Features, click Add Features, and add the RPC over HTTP Proxy feature. Accept the adding of required role services if prompted. Finally, verify that the M-Files Web site exists, uses port 80, and is running. 3.1.2 Configure the RPC Proxy After installing IIS and RPC over HTTP Proxy, configure the following registry values on the proxy server: Key HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\RpcProxy Value name Value type AllowAnonymous REG_DWORD Value data 1 Value name Value type Remarks ValidPorts REG_SZ If you use RPC over HTTP Proxy only for the purpose of enabling RPC over HTTPS connections to M-Files Server, you can replace any existing ValidPorts string value (for example, servername:100-5000) with dnsalias.domain.com:4466. This provides maximum security because RPC over HTTP Proxy will not forward RPC calls to any other computer or port. If you use RPC over HTTP Proxy for other, non-m-files purposes as well, you should append ;dnsalias.domain.com:4466 at the end of the other ValidPorts settings that you need. Value data dnsalias.domain.com:4466 Above, dnsalias.domain.com is the fully qualified DNS name that the clients use to connect to M-Files Server. This is typically the DNS name of the proxy server or a separately created DNS alias such as "mf.domain.com". 4466 is the default port used by M-Files Server for receiving RPC calls from IIS. The purpose of the ValidPorts entry is to enable IIS to forward the received traffic as RPC calls to M-Files Server to port 4466. IIS will only forward traffic to targets for which an exactly matching server name and port number are found in the ValidPorts entry. If the ValidPorts entry is missing or incorrect, the traffic will stop at IIS. Note: When the proxy server and M-Files server are separate servers, you must create a new DNS alias such as mf.domain.com and configure it to point to the proxy server's IP address in the DNS system. On the proxy server, this DNS name will then be mapped to the M-Files server's IP address by using a HOSTS file entry (see below). Do not use the proxy server's own DNS name. Using a separate DNS alias is required because the Windows Server operating system may refuse to read an IP address override from the HOSTS file for the server's own DNS name. 8

3.1.3 Modify the HOSTS file In the proxy server's HOSTS file in C:\Windows\System32\drivers\etc, map the fully qualified DNS name that the clients use for connecting to the proxy server to the IP address of the M-Files server. IIS will forward the RPC calls to M-Files Server based on the IP address specified in this entry. If the M-Files server is the same computer as the proxy server, use 127.0.0.1 as the IP address. For example: 127.0.0.1 dnsalias.domain.com # Map the DNS name that M-Files clients use to the IP address of M-Files Server If the M-Files server is a separate server, use the IP address of the M-Files server. For example: 10.0.0.124 dnsalias.domain.com # Map the DNS name that M-Files clients use to the IP address of M-Files Server Note: IIS will forward the RPC calls to the same DNS name that the clients used for connecting to the proxy server. Thus, the DNS name that the clients use in their vault connection settings must resolve to the IP address of the proxy server on the client computers. However, on the proxy server itself, this DNS name must resolve to the IP address of the M-Files server. This is achieved by mapping the DNS name to the proxy server's IP address on the DNS servers, and overriding this on the proxy server by using the HOSTS file to map the same DNS name to the M-Files server's IP address instead. For example, in the scenario that is shown in Image 1, the entry added to the HOSTS file on the proxy server should have the DNS name of the proxy server (dnsalias.domain.com) but the IP address of the M-Files server (that is, the IP address of dnsalias.domain.local, NOT the IP address of dnsalias.domain.com). 3.1.4 Install an SSL Certificate Get and install an SSL certificate for the website where the RPC proxy component is installed in IIS on the proxy server. Remember to add an HTTPS binding to the website in IIS and configure it to use the SSL certificate you acquired. You can get the certificate at http://www.ssls.com, for example. An SSL certificate is required for encrypted HTTPS traffic. The client computers must trust the SSL certificate of the server. If you use an official SSL certificate this is true by default. Note: Always use an official SSL certificate issued by a trusted Certification Authority (CA). Using a self-issued SSL certificate is not recommended because it adds a lot of complexity to the configuration of client computers. You would need to ensure that all client computers trust the self-issued SSL certificate both in the user's context and in the Local System account's context in which the M-Files Desktop service is running. If any part of the certificate trust configuration is incorrect, clients will fail to connect and will only report a generic network error. For this reason, using an official SSL certificate from, for example, http://www.ssls.com is highly recommended. If you use a self-issued SSL certificate, you must install the issuing CA's Root CA Certificate on each client computer. The root certificate must be available both in the user's context as well as in the computer account's context. See the following article on how to configure Computer Account level certificates: http://msdn.microsoft.com/en-us/library/ms788967(v=vs.110).aspx. 9

3.1.5 Configure the RPC Virtual Directory In IIS on the proxy server, inside the website where the RPC proxy component is installed, configure the RPC virtual directory's Authentication settings: 1. Allow Anonymous Authentication (typically disabled by default). 2. Allow Basic Authentication (typically already enabled). 3.1.6 Configure the RPC Proxy Website In IIS on the proxy server, make sure that the following Authentication setting is enabled for the website where the RPC proxy component is installed: Allow Anonymous Authentication (typically enabled by default) 3.1.7 Add Handler Mapping (IIS 7 only) If the proxy server is running IIS version 7, it may be necessary to manually create a handler mapping for the RPC Proxy DLL. Without this, IIS 7 may block requests to the DLL and will fail. Adding this setting is recommended only once you have first completed all other steps and determined that the connection is not working properly. To add a handler mapping manually, select the website where the RPC proxy component is installed, go to Handler Mappings, click the Add Script Map link on the right and enter the following values: Image 3: Edit Script Map in IIS. 3.1.8 Configure Firewall In the proxy server's firewall, allow incoming traffic to TCP port 443 (HTTPS port) to enable clients to communicate with the server computer via HTTPS. You can disable all other incoming traffic (except typically Remote Desktop, which is used for managing the server). You can also disable plain HTTP (TCP port 80). 10

3.1.9 Restart IIS On the proxy server, restart IIS for the above settings to take effect. Recommended: Disable SSL 2.0, SSL 3.0 and RC4 on the proxy server to improve security. Disabling SSL 2.0 and SSL 3.0 ensures that clients can only connect using TLS 1.0 or newer. Disabling RC4 ciphers is recommended to avoid using unsecure ciphers during the encryption. To disable SSL 2.0, SSL 3.0 and RC4, add the registry values of the attached file Disable SSLv2 and SSLv3.reg.txt on the proxy server and restart the server computer. Note: Restarting IIS is not sufficient for the disabling of SSL 2.0, SSL 3.0 and RC4 to become effective. A server restart is required for these settings to take effect. 3.2 M-Files Server The M-Files server computer runs the M-Files Server software. M-Files Server receives RPC calls from IIS on the proxy server. This document assumes that the M-Files Server software has already been installed on the M-Files server. Follow the steps below to configure M-Files Server to accept RPC over HTTP traffic. 3.2.1 Enable RPC over HTTP Traffic On the M-Files server, add the following registry value to enable M-Files Server to accept RPC over HTTP traffic: Key HKEY_LOCAL_MACHINE\Software\Motive\M-Files\<version>\Server\MFServer Value name Value type EnableRPCOverHTTP REG_DWORD Value data 1 Note: Verify that the value name does not contain a trailing space. 3.2.2 Configure Firewall In the M-Files server's firewall, allow incoming traffic to TCP port 4466 to enable M-Files Server to receive RPC calls from IIS on the proxy server. Additionally, you should allow incoming traffic to TCP port 2266 in the M-Files server's firewall if M-Files Desktop users may be connecting to M-Files Server also using the default RPC protocol (TCP/IP, port 2266) or if the proxy server is also running M-Files Web. Communication between M-Files Web and M-Files Server uses the default RPC protocol (TCP/IP, port 2266) if M-Files Web is running on a separate server. 11

3.2.3 Restart M-Files Server Restart the M-Files Server service for the above settings to take effect. 4. Configuring the Servers in a Multitenant Cloud Environment If you wish to enable RPC over HTTPS connections to M-Files Server in a multitenant cloud environment, configure the servers as instructed in Configuring the Servers above, with the following exceptions: Get a wildcard certificate for *.domain.com and install it on the proxy server. See Install an SSL Certificate for more information. On the proxy server, set the following value for the ValidPorts registry entry: *.domain.com:4466 See Configure the RPC Proxy for more information. For each tenant subdomain, create a public DNS entry that is mapped to the public IP address of the proxy server: customera.domain.com 123.123.123.123 customerb.domain.com 123.123.123.123 [ ] customern.domain.com 123.123.123.123 Modify the HOSTS file on the proxy server and map the DNS alias of each tenant to the internal IP address of the M-Files server: customera.domain.com 10.0.0.10 customerb.domain.com 10.0.0.10 [ ] customern.domain.com 10.0.0.10 See Modify the HOSTS file for more information. Or, if M-Files Server and IIS are running on the same server and there is no separate proxy server, modify the HOSTS file so that the DNS alias of each tenant is mapped to the IP address 127.0.0.1: customera.domain.com 127.0.0.1 customerb.domain.com 127.0.0.1 [ ] customern.domain.com 127.0.0.1 See Modify the HOSTS file for more information. 5. Configuring M-Files Desktop Computers Open M-Files Desktop Settings and add or edit a document vault connection. Select HTTPS as the protocol and set port number to 4466. 12

Note: The port number 4466 is used only for traffic between the proxy server and the M-Files server computer. All traffic from M-Files Desktop to the proxy server is encrypted and tunneled via TCP port 443 (HTTPS). On the proxy server, only the port 443 (HTTPS) needs to be opened in the firewall. See Image 4 for sample settings for the document vault connection in M-Files Desktop Settings: Image 4: Sample settings in M-Files Desktop Settings. 5.1 Disable HTTPS Encryption In some cases it is desirable to disable the use of SSL/TLS encryption between M-Files Desktop and the proxy server and instead use RPC over HTTP without encryption, that is, to use plain HTTP instead of HTTPS. Without SSL/TLS encryption, the traffic from M-Files Desktop towards the proxy server will be directed to the standard HTTP port (TCP port 80). You should disable the use of SSL/TLS encryption only if you have secured the communication by some other means. 13

To disable SSL/TLS encryption, add the following registry value on all client computers: Key HKEY_LOCAL_MACHINE\Software\Motive\M-Files Value name Value type EnableSSL REG_DWORD Value data 0 Default value 1 After changing the setting, restart the M-Files Desktop service. On the server, verify that IIS settings for the site where the RPC proxy component is installed as well as for the RPC virtual directory allow non-ssl traffic and that the default HTTP binding (port 80) is enabled. Also, ensure that the firewall on the proxy server allows incoming traffic to TCP port 80. When SSL/TLS encryption is not used, you do not need to install an SSL certificate in IIS on the proxy server. 6. Troubleshooting 6.1 General Troubleshooting The configuration of RPC over HTTP is fairly complex, and if any part of the configuration is incorrect, the connection from the client computer to the server is likely to fail. If the connection is not working and the error message does not immediately reveal the cause of the problem, you should first review all the settings carefully and double-check that the settings have been configured properly on the proxy server and the M-Files server as applicable. Error messages that refer to "network problems preventing communication with the server" indicate that something is preventing the communication from reaching the M-Files Server service. Potential causes include: Firewall not allowing incoming HTTPS traffic on the proxy server. IIS not running on the proxy server. The website where the RPC proxy component is installed not started in IIS on the proxy server. The website where the RPC proxy component is installed not configured for the default bindings in IIS on the proxy server (https, port 443). SSL certificate not installed in IIS on the proxy server. SSL certificate expired or otherwise not valid. SSL certificate not issued by a trusted Certification Authority (CA). Rpc and/or RpcWithCert virtual directories missing from IIS on the proxy server (RPC over HTTP Proxy not properly installed). ValidPorts registry value not properly configured on the proxy server or refers to a different DNS name or port number than what the clients use for connecting to the server. HOSTS file entry not added on the proxy server. HOSTS file entry referring to a different DNS name than what the clients use for connecting to the server. 14

HOSTS file entry mapping to a different IP address than the IP address of the computer that is running the M-Files Server software. IIS has not been restarted after changing the settings. Firewall not allowing incoming TCP traffic to port 4466 on the M-Files server. EnableRPCOverHTTP registry value not properly configured on the M-Files server. M-Files Server has not been restarted after changing the settings. Spelling errors in registry settings or trailing spaces in registry setting names, for example, "EnableRPCOverHTTPS" with an extra S or "EnableRPCOverHTTP " with an extra trailing space instead of the correct spelling "EnableRPCOverHTTP". You can export the registry branch and review the exported REG file's content to make it easier to notice especially any trailing spaces. An "Access denied" error message may indicate that some of the authentication-related settings are not correct. Potential causes include: Anonymous Authentication not enabled for the RPC virtual directory under the website where the RPC proxy component is installed in IIS on the proxy server. The AllowAnonymous registry value not properly configured on the proxy server. IIS has not been restarted after changing the settings. The following sections describe error conditions that may be encountered under special circumstances. 6.2 Testing RPC Proxy Settings You can test the RPC Proxy by pointing your Web browser to the URL https://<proxyserverurl>/rpc/rpcproxy.dll, where <proxyserverurl> is the fully qualified domain name (FQDN) of the proxy server, for example, dnsalias.domain.com. The page should ask for credentials. Enter Windows credentials which have permission to log on to Windows on the proxy server and press OK. A blank page as a result means that the RPC Proxy appears to be working OK. If instead of a blank page you receive an HTTP error page, this may indicate that the RPC Proxy is not correctly configured. Some known errors are listed below. If there are other errors (for example plain "Access Denied" text on the page), the test is inconclusive and you should check for other possible error cases. It may be necessary to open the URL by using a web browser on the proxy server itself to get the detailed error code. However, this can work properly only if the proxy server and the M-Files server are the same computer. If they are separate servers, then opening the above mentioned URL locally on the proxy server will fail because the HOSTS file entry causes the DNS alias to be mapped to the IP address of the M-Files server, which does not have the RPC Proxy. This can be worked around by temporarily modifying the HOSTS file entry to map the DNS alias to 127.0.0.1. After the test, the HOSTS file entry must be restored to map the DNS alias to the IP address of the M-Files server. Some known errors are listed below. 15

6.2.1 Error 401.1 Unauthorized This error appears at least when you press Cancel in the credentials dialog. Try closing the browser, and then entering valid Windows credentials to the dialog. 6.2.2 Error 500.19: Internal Server Error Make sure that DefaultAppPool -> Advanced Settings -> Enable 32-bit Applications is False. Restart IIS. 6.2.3 Error 404: Not found Make sure that RPC Proxy Server Extension ("%windir%\system32\rpcproxy\rpcproxy.dll") is allowed in the ISAPI and CGI restrictions configuration in IIS settings. 6.2.4 Error 405: Method Not Allowed Check Handler Mappings: ISAPI-dll should be enabled. If it isn't, select it, edit feature permissions, and add Execute. 6.2.5 Error 500.0: Internal Server Error Error Code: 0x8007007f There is a problem with the resource you are looking for, so it cannot be displayed. This may occur if the default website has been deleted and RPC has been manually added to another site. By default, IIS tries to look up RPC from the default website instance (with ID 1). To solve the problem, add the following registry value on the proxy server: Key HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\RpcProxy Value name Value type Value data Website REG_SZ Website name (for example, "Default Web Site") Additional resolution suggestions: http://support.microsoft.com/kb/942031. 6.2.6 Requests Hang until IIS is Restarted This is probably caused by the fact that <serverruntime uploadreadaheadsize="0"/> is missing from the following section in C:\Windows\System32\inetsrv\config\ApplicationHost.config: <location path="<the name of your website>/rpc"> <system.webserver> <serverruntime uploadreadaheadsize="0"/> </system.webserver> </location> Add the value and restart IIS. 16

6.3 Disabling Compression Some sources, such as http://forums.iis.net/p/1149768/1871631.aspx, suggest disabling compression may solve some problems, at least HTTP error 500.19 which references DynamicCompressionModule. Disable the compression with the following command-line parameters: %windir%\system32\inetsrv\appcmd.exe list config - section:system.webserver/httpcompression %windir%\system32\inetsrv\appcmd.exe set config -section:system.webserver/httpcompression /-[name='xpress'] To re-enable compression: %windir%\system32\inetsrv\appcmd.exe set config -section:system.webserver/httpcompression /+[name='xpress',dostaticcompression='false',dll='%windir%\system32\inetsrv\suscomp.dll'] 6.4 Client Trust for SSL Certificate Because the M-Files Desktop software uses the Local System account, the trust for the SSL certificate has to be defined for the Computer Account on the client computer. If the SSL certificate trust is OK on the user account level but missing on the Computer Account level, testing the connection from the M-Files Desktop Settings tool succeeds but navigating to the virtual M-Files drive in Windows Explorer displays a generic "network problems preventing communication with the server" error. See the following page on how to configure Computer Account level certificates: http://msdn.microsoft.com/en-us/library/ms788967(v=vs.110).aspx 6.5 Copying Configuration from a Working Computer If the fixes suggested above do not help resolve the issue, you can try to copy a configuration from a working IIS web server as suggested in http://blogs.msdn.com/b/saurabh_singh/archive/2008/08/30/troubleshooting-ts-gateway-connectivity-onwindows-2008-iis-7-0.aspx. Open C:\Windows\System32\inetsrv\config\ApplicationHost.config from a working installation. A sample file is also attached to this documentation. Take a backup of the same configuration file in the problematic installation. Replace the problematic installation's <application> and <location> tag contents with ones from the working configuration. Restart IIS. 6.6 SSL Offloading In case SSL offloading is performed in your network, ensure that the offloader does not decrypt any traffic towards M-Files. If the traffic is decrypted however, M-Files may report the RPC server to be unreachable. 17

6.7 "Not enough storage is available to complete this operation" Symptom: M-Files Desktop or M-Files Admin reports "Not enough storage is available to complete this operation" when connecting to M-Files Server Solution: Remove the following registry entries: "HKLM:\Software\Policies\Microsoft\Windows NT\DCOM").MachineAccessRestriction "HKLM:\Software\Policies\Microsoft\Windows NT\DCOM").MachineLaunchRestriction 7. Change History The table below describes the essential changes by document version. VERSION DATE ESSENTIAL CHANGES 1.0 2015-01-24 Initial version. 2.0 Added additional information about data security (chapter 2.3). 2.1 SSL certificate instructions in chapter 4.1.4 clarified. 2.2 Recommendations in chapter 4.1.8 clarified. 2.3 Note about the proxy server running multiple sites added to chapter 0. 2.4 Note about the configuration order of M-Files Web and RPC over HTTPS added to chapter 3. 2.5 Added troubleshooting for "Not enough storage..." error to chapter 6.7. 2.6 Added information on enabling anonymous authentication for the RPC Proxy website in IIS (chapter 3.1.6). 2.7 2017-09-08 Added instructions for completing the configuration in a multitenant cloud environment (chapter 4). 2.8 2017-11-13 Added a note about the fact that RPC over HTTP Proxy requires the port 443 to its own use (chapters 2 and 3.1). 2.9 2017-12-13 Small fixes. Changed a number of references to "Default Web Site" to instead refer to the website where the RPC proxy component is installed. 18