Configuring Kerberos for Hadoop Distributed File System (HDFS) Transfers

Aspera Enterprise Server can be configured to use Kerberos for Hadoop Distributed File System (HDFS) transfers by creating a new Kerberos principal and editing the hdfs.properties configuration file, as described below.

On the Kerberos key distribution center (KDC), create a new principal.
The principal must have a name with the pattern: name/host@domain. For example, if the user is trap, the host is hadoop-node.aspera.us, and the domain is ASPERA.US, then the principal name is: trap/hadoop-node.aspera.us@ASPERA.US.
Configure HDFS for the user.
Open /etc/hadoop/conf/core-site.xml. Enable the principal to impersonate users and add the Enterprise Server IP address to the list of hosts allowed to connect to the NameNode. If the user is trap and the host is hadoop-node.aspera.us, the configuration is as follows:
```
<property>
    <name>hadoop.proxyuser.trap.groups</name>
    <value>*</value>
</property>
   
<property>
    <name>hadoop.proxyuser.trap.hosts</name>
    <value>hadoop-node.aspera.us, ip_address</value>
</property>
```
Save your changes and close the file.
Restart HDFS to activate your changes.
Export a keytab file for this principal and copy it to:
/opt/aspera/etc/trapd/hdfs.keytab

Configure Trapd for Kerberos in hdfs.properties.

Open /opt/aspera/etc/trapd/hdfs.properties and find the following section:

# The authentication method to use. Valid values are SIMPLE | KERBEROS
# Default is 'SIMPLE'
#aspera.hdfs.security.method = SIMPLE

# The path to the keytab file to use when Kerberos is enabled.
# Default: /opt/aspera/etc/trapd/hdfs.keytab
#aspera.hdfs.security.key-tab.file = /opt/aspera/etc/trapd/hdfs.keytab

# The Kerberos principal to use to access the name node.
# Example: trap/hadoop-node.aspera.us@ASPERA.US
# Default: none.
#aspera.hdfs.security.principal =

To enable Kerberos,

Uncomment the line #aspera.hdfs.security.method = SIMPLE and change the value to KERBEROS:
```
aspera.hdfs.security.method = KERBEROS
```
Uncomment the line #aspera.hdfs.security.principal = and enter the name of the principle. Using the example principal, trap/hadoop-node.aspera.us@ASPERA.US:
```
aspera.hdfs.security.principal = trap/hadoop-node.aspera.us@ASPERA.US
```

Configure Kerberos.
Open /etc/krb5.conf and edit as necessary with the principal and domain set in hdfs.properties.
Restart Trapd to activate your changes.
```
# /etc/init.d/asperatrapd restart
```