Support Article

Missing HBase Java client classes

SA-49351

Summary



Error when performing test connectivity after installing third party JAR files to connect to Hadoop or Hbase.


Error Messages



KERBEROSclient principal:hbase/servername.com@devhadoop.servername.COM, masterPrincipalKey:hbase.master.kerberos.principal, masterPrincipalName:hbase/servername.com@devhadoop.servername.COM, keytabFile:/tmp/778c402ac2478ef2d0bab0d41b038641.keytab
[WebContainer : 6] [TABTHREAD3] [ ] [PegaRULES:07.10] (on.HadoopAuthenticationManager) WARN servername.com|server.com userid - Problem while updating authentication config, Previous config: none, new config: Authentication config, authType: KERBEROSclient principal:hbase/servername.com@devhadoop.servername.COM, masterPrincipalKey:hbase.master.kerberos.principal, masterPrincipalName:hbase/servername.com@devhadoop.servername.COM, keytabFile:/tmp/778c402ac2478ef2d0bab0d41b038641.keytab



Steps to Reproduce



Follow the procedure in the document to Import the following Hadoop or Hbase third party JAR files in the database and perform Test Connectivity:

https://pdn.pega.com/jar-file-dependencies-hbase-and-hdfs-data-sets



Root Cause



User has a HBase setup which requires additional parameters at the client side (Pega) for a successful connection. Hadoop cluster was configured with mapping rules 'hadoop.security.auth_to_local' and encrypted communication through the 'hbase.rpc.protection' setting. These features are not supported by Pega.


Resolution



Apply HFix-39868. This allows HBase Connection Customization. Following are the steps to customize HBase JAVA connection for custom-user Hadoop cluster setups

1. The given hotfix connection or client configuration reads its configuration from various places to provide the required flexibility on user installations.

2. Customization:

- Default Files
By importing the following JAR files to the Pega platform, default values for various keys on HBase connections can be defined.
  • core-default.xml
  • core-site.xml
  • hdfs-default.xml
  • hdfs-site.xml
  • hbase-default.xml
  • hbase-site.xml

It is possible to import those files by uploading simple jar files into Customer:06-01-01 ruleset similar to uploading Big Data Third Party Dependencies-Jars, which have META-INF/MANIFEST.MF files.

For example,
.zip hbase-site-v3.jar hbase-site.xml META-INF/MANIFEST.MF

- Guided Configuration
Values added by UI or values entered from UI must be merged in second layer.

- Dynamic System Settings (DSS)
To override values that are from a UI and/or Pega platform, the DSS can be defined.


hbase.client.meta.operation.timeout
hbase.client.operation.timeout
zookeeper.session.timeout
hbase.zookeeper.property.clientPort
hbase.zookeeper.quorum
zookeeper.recovery.retry
hbase.security.authentication
hbase.rpc.protection
hbase.master.kerberos.principal
hadoop.security.authentication
hbase.rpc.engine
hbase.regionserver.kerberos.principal
hadoop.security.auth_to_local


To override any of the keys defined above, the DSS keys must be defined with the prefix 'bigdata/hadoop/configuration/' on Pega-BigDataruleset 'prconfig.xml;.

It is also possible to override any key using prconfig.xml. Any key with prefix 'bigdata/hadoop/configuration/' will be copied on to configuration

For example,

<env name="bigdata/hadoop/configuration/hadoop.security.auth_to_local" value="RULE:[1:$1](.*)s/(.*)/$1/L RULE:[2:$1](.*)s/(.*)/$1/L DEFAULT"/>

- Debugging
Enable logger:com.pega.bigdata.hbase.client.internal.HBConfigurationImpl at debug level.



Published January 14, 2018 - Updated October 8, 2020

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.