Authentication via Active Directory and LDAP Overview The LDAP and Active Directory authenticators available in Datameer provide remote authentication services for Datameer users. Administrators can configure Datameer to use their existing LDAP or Active Directory systems as the authenticator of record allowing for centralized management of user accounts and credentials outside of Datameer. Users can authenticate with Datameer using existing credentials which are verified against the remote system on every login. If the remote system no longer sanctions the user, access to Datameer is denied. This simplifies Datameer administration and allows end-users to use familiar single sign-on credentials when accessing Datameer. Overview Users and Groups Configuration Active Directory Advanced Configuration Configuration (As of Datameer version 6.3) Importing Users/Groups Advanced Topics Cache settings Enabling and disabling remote authentication Skipping users Using a default LDAP group Limiting LDAP user results Use a more restrictive search base Add constraints to the user definition Set a custom MaxResults setting for the Datameer LDAP user Users and Groups When using the LDAP Authenticator, credentials are authenticated at the time of login directly against the remote service. In order for Datameer to manage authorization and object permissions, a Datameer user entity is required to represent the remote user. It useful to think of users being imported into Datameer from LDAP with the remote server always ensuring that users have valid, active credentials. For details on how to add users from LDAP, see Importing Users. When a user is imported into Datameer, a user entity is created using the remote unique identifier as the Datameer username and populated with other account details, i.e email address. During import, Datameer creates groups based on the groups contained in the LDAP directory. This allows the use of existing LDAP groups for Datameer access control as the group memberships are mirrored across systems. The users' group memberships and user details are updated on every login to ensure that any changes to authorization policies in LDAP are reflected in every Datameer session. Since we rely on LDAP groups when the LDAP Authenticator is in use, there is no facility to create groups within Datameer*. Datameer can handle at maximum 500 LDAP groups. * See Using a Default LDAP Group for the one exception to this rule. Transforming user names into lowercase characters Many times an all lowercase username is required for proper Unix group authentication. Instead of needing to change all user names to lowercase characters, Datameer has the following property file to transform a username into all lowercase before passing it to the hadoop client API(s) to account for this issue. hadoop.security.auth_to_local=rule:[1:$1@$0](.*@ec2.internal)s/(.*)@ec2. INTERNAL/$1/L RULE:[2:$1@$0](.*@EC2.INTERNAL)s/(.*)@EC2.INTERNAL/$1/L DEFAULT To enable this feature, add the above property file to the custom Hadoop properties. Apache hadoop.security.auth_to_local documentation. Configuration
Warning We strongly suggest that you have superuser mode enabled and that you have the password handy when making changes to the Datameer authentication system. This helps to prevent being locked out of the system should something go wrong. Make sure the deployment property das.superuser.enabled is true and that you are aware of the values of das.superuser.username and das.superuser.password. You need to set the following in <Datameer Install Folder>/conf/live.properties. (If you are using a custom deploy mode via DAS_DEPLOY_MODE then use the properties file for that mode, live is the default): # You can set a super user account here that has the ADMIN role. This can be used to login even if # the external authentication service doesn't work das.superuser.enabled=true das.superuser.username=superuser das.superuser.password=password LDAPS If you plan on accessing your LDAP service over SSL, read Configuring Secure LDAP (LDAPS) before continuing. To configure Datameer to use your LDAP or ActiveDirectory service, click the Admin tab, click Authentication on the left menu and click Edit. This presents you with the option to choose between the Internal Datameer User Management or a Remote Authentication System. Select Remot e and pick ActiveDirectory/ LDAP from the drop down list: Next, you see the configuration settings for you remote authentication service:
The configuration settings are described below: Setting Description Server URL* The LDAP connection string used to access your server. This should be in the form ldap://server:port. See Configuring Secure LDAP (LDAPS) for details on connecting via LDAPS. Port numbers: LDAP/S - 389/636 or Active Directory - 3268 User The user account Datameer uses to connect to the remote service in order to perform authentication. For many LDAP services, this is a fully distinguished name of a user, for ActiveDirectory, this is username@domain.com Password The LDAP password for the user specified above Search Base* This is the LDAP location used as a search base. Datameer bases its LDAP queries from here only searching below this point, so it is a good way to partition the users who are available to Datameer. You can only specify one search base. In Active Directory it is possible to limit Datameer to users only contained in a specific domain or organizational unit using this setting. See Limiting LDAP User Results. Pagination Control Use pagination control (PC) to increase performance of requests with large numbers of results, this limits the number of result objects per page. Virtual Group Allows the option to create a group that is comprised of all verified users not assigned to a specific group in the authenticator. Active Directory Choosing the Active Directory service type configures the advanced query options with settings appropriate for most Active Directory installations. Also, a different strategy is used to list a user's groups which is required by Active Directory. If the default settings aren't appropriate for your installation they can be changed in the Query Options section. See Advanced Configuration. Using Active Directory authentication for access to networked servers/services, like FTP/SFTP/SSH must ensure the primary group of the user doesn't contains a space. (E.g., The group "domain users" would fail. Change the primary group of the user to the single word "domain_users").
Advanced Configuration To refine your authenticator configuration you can customize the settings Datameer uses when communicating with the remote service. Expand the Query Options form: Setting Description User Definition* The filter expression used when Datameer queries the remote system for available users. See Limiting LDAP User Results User's Group Membership Constraints Username Attribute* Limit queries to only the users who belong to the specified groups. One distinguished name per line. The attribute that is mapped to the Datameer username. This must be unique across all users. For Active Directory this is the samaccountname, for other LDAP providers it varies: uid, cn, username, etc. Email Attribute* The attribute that is mapped to the Datameer user's email address. Most systems use 'mail'.
Group Name Attribute* The attribute that is used as the group name in Datameer. Group Definition Group Membership Attribute Group Search Base Impersonation Attribute The filter query used when searching for object groups. The attribute used to determine a groups members. The search base to use when finding a User's groups, if different than the user search base. The Unix impersonation name to send to Hadoop, separate from the login name. Setting Include These Groups Exclude These Groups Ignore Case Description Specify a series regex filters, one per line and Datameer includes group names that match, unless they also match an exclude filter. Specify a series regex filters, one per line and Datameer excludes group names that match. Perform a case insensitive match when filtering group names. Group Membership Attribute and Group Filter only apply to 'Other LDAP' installations as Active Directory installations use a different group listing strategy Configuration (As of Datameer version 6.3) Datameer has improved connecting and using authentication services. Connections to multiple LDAP servers is supported. Nested groups on an authentication server is supported. To configure Datameer to use your LDAP or Active Directory service, select the Admin tab, choose Authentication from the menu on the left of the screen, and click the Edit button. Select Remote Authentication System from the drop down list and then select ActiveDirectory/LDAP fro m the drop-down list under the Authentication System heading.
The authentication configuration settings for Active Directory and LDAP are displayed. The default user values can be set when accessing multiple LDAP servers as the same user. If a default user/password is set, check the box under Server Connections that the default user values should be used to authenticate. Enter the server configuration settings. Multiple LDAP server can be added. Click the Add Server Connection button after all values have been entered. Datameer validates the server and credentials and then provides a blank Server Connection settings form for additional servers.
Setting Description Server URL* The LDAP connection string used to access your server. This should be in the form ldap://server:port. See Configuring Secure LDAP (LDAPS) for details on connecting via LDAPS. Port numbers: LDAP/S - 389/636 or Active Directory - 3268 Use Default User User Select this box if the default user/password above should be used to authenticate with this server. The user account Datameer will use to connect to the remote service in order to perform authentication. For many LDAP services this will be a fully distinguished name of a user, for ActiveDirectory, this is username@domain.com Password The LDAP password for the user specified above User Definition* Group Definition Search Base* The filter expression used when Datameer queries the remote system for available users. See Limiting LDAP User Results The filter expression used when Datameer queries the remote system for available groups. This is the LDAP location used as a search base. Datameer bases its LDAP queries from here only searching below this point, so it is a good way to partition the users who are available to Datameer. You can only specify one search base. In Active Directory it is possible to limit Datameer to users only contained in a specific domain or organizational unit using this setting. See Limiting LDAP User Results. Pagination Control Use pagination control (PC) to increase performance of requests with large numbers of results, this limits the number of result objects per page.
User and group query options. Setting Username Attribute* Email Attribute* Description The attribute that will be mapped to the Datameer username. This must be unique across all users. For Active Directory this is the samaccountname, for other LDAP providers it varies: uid, cn, username, etc. The attribute that is mapped to the Datameer user's email address. Most systems use 'mail'. Impersonation Attribute Group Name Attribute* Group Membership Attribute Virtual Group Group Search Base The Unix impersonation name to send to Hadoop, separate from the login name. The attribute that will be used as the group name in Datameer. The attribute used to determine group members. Allows the option to create a group that will be comprised of all verified users not assigned to a specific group in the authenticator. The search base to use when finding a User's groups, if different than the user search base.
Nested Groups User's Group Membership Constraints Groups within groups. By default, a users is only recognized as being in their immediate group. When the nested groups feature is checked, a user that is a member of a group within a group inherits permissions of all groups in which their group is contained. Limit queries to only the users who belong to the specified groups. One distinguished name per line. Group filters. Setting Include These Groups Exclude These Groups Ignore Case Description Specify a series regex filters, one per line and Datameer will include group names that match, unless they also match an exclude filter. Specify a series regex filters, one per line and Datameer will exclude group names that match. Perform a case insensitive match when filtering group names. Importing Users/Groups Once you have successfully configured your LDAP connection, you can now import users into Datameer, enabling users to access the system. Because AD/LDAP is active, Datameer's internal authentication method is disabled. Therefore, admins must create all groups needed in the AD and add the users there before importing them. To import, click Users from the Admin menu. From the Add Users and Groups from External Authenticator a list is populated with users/groups from the LDAP server. The search bar looks for a string contained in the User Name, email, or Group Name. Select the user/group, assign a role, and click Add. Each user must have a role. Once a user/group has been added, the name is greyed out from the list. That user/group has access to Datameer once the cache has been refreshed. In order to have access to Group Authentication tab, the Datameer license must support unlimited users. Users/Groups that have been added from the LDAP server is displayed and can be edited. Highlight multiple users/groups to to perform bulk role updates or deletions. It is also possible to bulk import users from your LDAP authenticator.
User Details Read-Only The username and email address aren't editable as they are provided bythe LDAP service. Only Enable/Disable, Roles, and comments are editable from this screen. Advanced Topics Cache settings Datameer loads the entire set of LDAP users into a cache that is periodically refreshed from LDAP. This offers tremendous performance benefits and enables features like search and browse in the UI. By default, Datameer refreshes this cache every hour. This means that modification to LDAP data might not be reflected until the cache is refreshed. This doesn't impact authentication credentials or removal of users from LDAP. Credential checks are done against the live service, not cached data. Also user data populated at login reflects up-to-date information. The cached data only impacts the importing of users and listing of available users To change the cache refresh settings you can set the system property ldap.authenticator.available.users.refresh.interval.min utes to whatever minute value you desire. For most installations, the 60 minute setting makes sense. Enabling and disabling remote authentication While setting up the system or when doing maintenance, you might find it necessary to switch off the LDAP Authenticator. When the LDAP Authenticator is disabled, all user's which were imported from LDAP are automatically disabled and the cache is dropped. When the authenticator is re-enabled, all users are then re-enabled and ready for use. You don't have to re-import any users. Also note that the cache has been re-built.
Skipping users Datameer builds a list of all available users for import based on the authenticator configuration. During this process, Datameer drops users from this list if: The user doesn't have a username The user doesn't have an email address The user isn't a member of a group (see Using a Default LDAP Group for a workaround) If you don't see users that you are expecting available for import to Datameer, check your logs and look for WARN level message such as: LdapUser has empty email address, will be filtered from available users : LdapUser has empty username, will be filtered out from available users : LdapUser has no groups and default group is disabled, will be filtered from available users : Seeing these types of messages often signifies a misconfiguration. Using a default LDAP group By default when constructing the list of users available for import, Datameer skips any users that aren't members of a group in LDAP. This behavior is desirable since this is most likely a mis-configuration and Datameer requires user to belong to a group. However, it is possible that your LDAP users haven't been assigned to a group or that for some reason the Datameer authenticator can't resolve any. In this case, you can set a system property - ldap.authenticator.use.default.group=true - creating a default group for imported LDAP users. The group DAP_USERS contains every user not assigned to another group, be aware of this when setting up group permissions. L Limiting LDAP user results Many LDAP providers, Active Directory included, limit the number of results returned in most searches. When this limit it is reached, no more results are sent to the client in a single request. This is problematic, especially if you don't have the ability to configure the LDAP service to allow more results. If your LDAP installation is sufficiently large and your service is configured to limit results in this way, Datameer might fail to load any users at all. To get around this issue, Datameer offers several configuration options: Use a more restrictive search base In the Query Options section of the Authenticator configuration screen, you can define a search base that limits the scope of the LDAP query. For example, if we have a search base for a Datameer LDAP server DC=datameer,DC=local and it contains 3,000 users which is over the limit of 1,000 results, we can refine it to include only the organizational unit we want, OU=BarUnit,DC=Datameer,DC=local which contains only 1,000 users. Now we can successfully import users from the organizational unit BarUnit and below in the LDAP tree. Add constraints to the user definition The Query Options section also contains a field for configuring a User Filter. This is analogous to an SQL WHERE clause used when Datameer queries for LDAP users. You can filter any attribute associated with a user in your LDAP server. As an example, we could filter users based on their membership in either the 'Finance' or 'IT' department. This would make only users matching the filter available for import into Datameer. The filter is defined using normal LDAP query syntax: "LDAP Queries" The above query says, "Find objects which are of objectclass 'person' and have a department value of either 'Finance' or 'IT'". For more
details about LDAP query syntax, go here. Set a custom MaxResults setting for the Datameer LDAP user If the Datameer LDAP needs to list large numbers of users, a user specific setting can be applied in most LDAP providers. This would allow this specific user to exceed the default query result settings for this domain. In Active Directory, this is achieved by setting a MaxResults value for the Datameer LDAP user.