Support Article
Missing HBase Java client classes
SA-49351
Summary
Error when performing test connectivity after installing third party JAR files to connect to Hadoop or Hbase.
Error Messages
KERBEROSclient principal:hbase/servername.[email protected], masterPrincipalKey:hbase.master.kerberos.principal, masterPrincipalName:hbase/servername.[email protected], keytabFile:/tmp/778c402ac2478ef2d0bab0d41b038641.keytab
[WebContainer : 6] [TABTHREAD3] [ ] [PegaRULES:07.10] (on.HadoopAuthenticationManager) WARN servername.com|server.com userid - Problem while updating authentication config, Previous config: none, new config: Authentication config, authType: KERBEROSclient principal:hbase/servername.[email protected], masterPrincipalKey:hbase.master.kerberos.principal, masterPrincipalName:hbase/servername.[email protected], keytabFile:/tmp/778c402ac2478ef2d0bab0d41b038641.keytab
Steps to Reproduce
Follow the procedure in the document to Import the following Hadoop or Hbase third party JAR files in the database and perform Test Connectivity:
https://pdn.pega.com/jar-file-dependencies-hbase-and-hdfs-data-sets
Root Cause
User has a HBase setup which requires additional parameters at the client side (Pega) for a successful connection. Hadoop cluster was configured with mapping rules 'hadoop.security.auth_to_local' and encrypted communication through the 'hbase.rpc.protection' setting. These features are not supported by Pega.
Resolution
Apply HFix-39868. This allows HBase Connection Customization. Following are the steps to customize HBase JAVA connection for custom-user Hadoop cluster setups
1. The given hotfix connection or client configuration reads its configuration from various places to provide the required flexibility on user installations.
2. Customization:
- Default Files
By importing the following JAR files to the Pega platform, default values for various keys on HBase connections can be defined.
- core-default.xml
- core-site.xml
- hdfs-default.xml
- hdfs-site.xml
- hbase-default.xml
- hbase-site.xml
It is possible to import those files by uploading simple jar files into Customer:06-01-01 ruleset similar to uploading Big Data Third Party Dependencies-Jars, which have META-INF/MANIFEST.MF files.
For example, .zip hbase-site-v3.jar hbase-site.xml META-INF/MANIFEST.MF
- Guided Configuration
Values added by UI or values entered from UI must be merged in second layer.
- Dynamic System Settings (DSS)
To override values that are from a UI and/or Pega platform, the DSS can be defined.
hbase.client.meta.operation.timeout
hbase.client.operation.timeout
zookeeper.session.timeout
hbase.zookeeper.property.clientPort
hbase.zookeeper.quorum
zookeeper.recovery.retry
hbase.security.authentication
hbase.rpc.protection
hbase.master.kerberos.principal
hadoop.security.authentication
hbase.rpc.engine
hbase.regionserver.kerberos.principal
hadoop.security.auth_to_local
To override any of the keys defined above, the DSS keys must be defined with the prefix 'bigdata/hadoop/configuration/' on Pega-BigData ruleset 'prconfig.xml;.
It is also possible to override any key using prconfig.xml. Any key with prefix 'bigdata/hadoop/configuration/' will be copied on to configuration
For example,
<env name="bigdata/hadoop/configuration/hadoop.security.auth_to_local" value="RULE:[1:$1](.*)s/(.*)/$1/L RULE:[2:$1](.*)s/(.*)/$1/L DEFAULT"/>
- Debugging
Enable logger:com.pega.bigdata.hbase.client.internal.HBConfigurationImpl at debug level.
Published January 14, 2018 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.