Skip to main content

This content has been archived and is no longer being updated. Links may not function; however, this content may be relevant to outdated versions of the product.

Support Article

KafkaException occurs when sending messages to Kafka bus

SA-66833

Summary



The requirement is to use custom Kafka modules in Pega 7.3. Third-party Kafka library JAR files are referenced in prconfig.xml. 
KafkaException occurs when using the Pega application to send messages to a Kafka bus.



Error Messages



at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) ~[catalina.jar:?]
at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:319) ~[catalina.jar:?]
at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:88) ~[catalina.jar:?]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) ~[catalina.jar:?]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) ~[tomcat-coyote.jar:8.5.15-dev]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) ~[tomcat-coyote.jar:8.5.15-dev]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) ~[tomcat-coyote.jar:8.5.15-dev]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) ~[tomcat-coyote.jar:8.5.15-dev]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-coyote.jar:8.5.15-dev]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-util.jar:8.5.15-dev]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335) ~[kafka-clients-0.10.0.1.jar:?]
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188) ~[kafka-clients-0.10.0.1.jar:?]
at xx.yyy.pega.distribution.publisher.kafka.KafkaPublisher.initialize(KafkaPublisher.java:50) ~[pega-kakfa-client.jar:?]
at com.pegarules.generated.SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.SendKafkaMessage01_01_03(SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.java:109) ~[?:?]
... 74 more
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.clients.producer.internals.DefaultPartitioner is not an instance of org.apache.kafka.clients.producer.Partitioner
at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:203) ~[kafka-clients-0.10.0.1.jar:?]
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:225) ~[kafka-clients-0.10.0.1.jar:?]
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188) ~[kafka-clients-0.10.0.1.jar:?]
at xx.yyy.pega.distribution.publisher.kafka.KafkaPublisher.initialize(KafkaPublisher.java:50) ~[pega-kakfa-client.jar:?]
at com.pegarules.generated.SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.SendKafkaMessage01_01_03(SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.java:109) ~[?:?]
... 74 more
2017-10-02 15:11:35,479 [abcdefg] [STANDARD] [ ] [Fraud:01.02] (internal.mgmt.Executable) ERROR a_host|aa.bb.ccc.dd.ee an_operator - Exception
java.lang.RuntimeException: org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at com.pegarules.generated.SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.SendKafkaMessage01_01_03(SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.java:115) ~[?:?]
at com.pegarules.generated.SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.invoke(SendKafkaMessage_010103_BLkrxruaR6FmX4XnkjcT0Q.java:73) ~[?:?]
at com.pega.pegarules.generation.internal.library.LibraryRuntime.resolveAndinvokeFunctionViaReflection(LibraryRuntime.java:222) ~[prprivate.jar:?]
at com.pega.pegarules.generation.internal.library.LibraryRuntime.invokeLibraryRuntime(LibraryRuntime.java:119) ~[prprivate.jar:?]
at com.pega.pegarules.generation.internal.library.LibraryFunctionUtilityImpl.resolveMethodCall(LibraryFunctionUtilityImpl.java:2923) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.Executable.resolveMethodCall(Executable.java:11215) ~[prprivate.jar:?]
at com.pegarules.generated.activity.ra_action_sendkafkamessage_bc6005b35aafc2b77e21cb27f9391736.step2_circum0(ra_action_sendkafkamessage_bc6005b35aafc2b77e21cb27f9391736.java:220) ~[?:?]
at com.pegarules.generated.activity.ra_action_sendkafkamessage_bc6005b35aafc2b77e21cb27f9391736.perform(ra_action_sendkafkamessage_bc6005b35aafc2b77e21cb27f9391736.java:87) ~[?:?]
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3586) ~[prprivate.jar:?]
at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivity(Executable.java:10770) ~[prprivate.jar:?]
at com.pegarules.generated.activity.sh_action_sendinfotokafkaeventbus_6f374bd572afb974baf3bb96b6086c03.step5_circum0(sh_action_send



Steps to Reproduce

  1. Update prconfig.xml to include the third-party Kafka library JAR files and restart the server.

    <env name="compiler/defaultPaths" value="/data/share/sharedlayer/lib/apitrust-api-00.00.08.001.jar:/data/share/sharedlayer/lib/bcpkix-jdk15on-1.55.jar:/data/share/sharedlayer/lib/bcprov-jdk15on-1.55.jar:/data/share/sharedlayer/lib/hamcrest-core-1.3.jar:/data/share/sharedlayer/lib/commons-configuration-1.10.jar:/data/share/sharedlayer/lib/pega-kakfa-client.jar:/data/share/sharedlayer/lib/toolkit-kafka-serialization-1.11.0-SNAPSHOT.jar:/data/share/sharedlayer/lib/toolkit-encryption-1.11.0-SNAPSHOT.jar:/data/share/sharedlayer/lib/protobuf-java-2.6.1.jar:/data/share/sharedlayer/lib/commons-lang-2.6.jar:/data/share/sharedlayer/lib/lz4-1.3.0.jar:/data/share/sharedlayer/lib/snappy-java-1.1.2.6.jar:/data/share/sharedlayer/lib/slf4j-api-1.7.25.jar:/data/share/sharedlayer/lib/commons-lang3-3.6.jar:/data/share/sharedlayer/lib/jcl-over-slf4j-1.7.25.jar:/data/share/sharedlayer/lib/commons-codec-1.10.jar:/data/share/sharedlayer/lib/gson-2.8.2.jar:/data/share/sharedlayer/lib/kafka-clients-0.10.0.1.jar"/>
     
  2. Implement a custom connector to post a message to a Kafka queue.


Root Cause



Pega 7.3 has an inbuilt kafka-clients-0.10.0.1.jar library. The custom library causes a conflict with the inbuilt library.

Resolution



Perform the following local-change:

Disable the out-of-the-box kafka-clients-0.10.0.1.jar in the application as below:
  1. Shut down the servers.
  2. Backup the PegaRULES database.
  3. Run the below SQL Update:

    update pr_engineclasses  set pzcodeset='pega-enginecodenouse' where  pzjar = 'kafka-clients-0.10.0.1.jar';
    commit;

     
  4. Restart the servers.
 

 

Published December 13, 2018 - Updated October 8, 2020

Was this useful?

0% found this useful

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega Community has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice
Contact us