Support Article
Cannot access encrypted file from Hadoop
SA-39554
Summary
User is facing the error reported below while creating a Hadoop Distributed File System (HDFS) dataset and browse a file to test a data set.
From the errors, it appears to be that Pega is not connecting to the Hadoop Key Management Server (KMS).
However there is no way to set the host and port of the KMS server in the Hadoop configuration rule.
Error Messages
[ APP:01.01] (lient.Data_Admin_Hadoop.Action) ERROR <your hostname>|<your IP> <your Operator> - put-com.pega.bigdata.dataset.hdfs.HDFSFileClient@1b1fd95c
[ APP:01.01] (nager.Data_Admin_Hadoop.Action) ERROR <your hostname>|<your IP> <your Operator> - put-com.pega.bigdata.dataset.hdfs.parquet.ParquetManager@75831953
[ APP:01.01] (adoop.util.PerformanceAdvisory) DEBUG <your hostname>|<your IP> <your Operator> - Crypto codec org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec is not available.
[ APP:01.01] (adoop.util.PerformanceAdvisory) DEBUG <your hostname>|<your IP> <your Operator> - Using crypto codec org.apache.hadoop.crypto.JceAesCtrCryptoCodec.
[ APP:01.01] (Data_Admin_DataSet_HDFS.Action) ERROR <your hostname>|<your IP> <your Operator> - Exception while loading file preview:
java.io.IOException: No KeyProvider is configured, cannot access an encrypted file
at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1410)
at org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:1489)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:310)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:304)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:304)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:775)
at com.pega.bigdata.dataset.hdfs.HDFSFileClient$ReadFileContentHDFSOperation.readFromStream(HDFSFileClient.java:224)
at com.pega.bigdata.dataset.hdfs.HDFSFileClient$ReadFileContentHDFSOperation.perform(HDFSFileClient.java:196)
at com.pega.bigdata.dataset.hdfs.HDFSFileClient$ReadFileContentHDFSOperation.perform(HDFSFileClient.java:168)
at com.pega.bigdata.dataset.hdfs.HDFSFileClient$SingleHDFSOperation.perform(HDFSFileClient.java:1046)
at com.pega.bigdata.hdfs.HDFSExecutor.performOnCachedFileSystem(HDFSExecutor.java:150)
at com.pega.bigdata.hdfs.HDFSExecutor.performOperation(HDFSExecutor.java:89)
at com.pega.bigdata.dataset.hdfs.HDFSFileClient.getFileContent(HDFSFileClient.java:393)
at com.pegarules.generated.activity.ra_action_pzpreviewfile_874845c432dcce16eb4dce06edf20a1d.step7_circum0(ra_action_pzpreviewfile_874845c432dcce16eb4dce06edf20a1d.java:722)
at com.pegarules.generated.activity.ra_action_pzpreviewfile_874845c432dcce16eb4dce06edf20a1d.perform(ra_action_pzpreviewfile_874845c432dcce16eb4dce06edf20a1d.java:184)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3553)
at com.pegarules.generated.activity.ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.step1_circum0(ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.java:319)
at com.pegarules.generated.activity.ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.perform(ra_action_pzrunactionwrapper_056ec03796d8a13802f156886066036d.java:70)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3553)
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:646)
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAlt(PRThreadImpl.java:465)
at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.runActivities(HttpAPI.java:3203)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java:391)
at sun.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
...
Steps to Reproduce
Not Applicable
Root Cause
A defect in Pegasystems’ code or rules
Feature to add KMS details does not exists in Hadoop Configuration rule.
Resolution
Apply HFix-34914.
Published August 21, 2017 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.