LinkedIn
Copied!

Table of Contents

Operating the Stream service

Version:

Only available versions of this content are shown in the dropdown

Learn how to do a rolling and full restart of your Stream service, to manually scale up and scale down the service, and to recover the service after a failure.

The Stream service can also be configured to run in External Mode. For more information on how to configure it, see Configuring External Kafka as a Stream service. If the Stream service is running in External Mode, there are no Stream Tier instances and none of the procedures mentioned below are required.

Rolling restart Stream Tier

The Stream tier can be "rehydrated" by restarting one instance at a time. Terminate each instance in a graceful fashion. It may take up to 10 minutes to shut down a Stream instance as the data ownership has to be handed over to the remaining nodes. Do not forcefully terminate instances during this period.

When an instance comes back online, wait for a grace period of 5 minutes before you restart the next instance. During the grace period, the Stream node replicates the data which might have been missed during the period that the instance was unreachable.

Full restart Stream Tier

In case a rolling restart is not possible for some reason, you can do a full restart.

Take the nodes out of service one instance at a time. Each instance has to be terminated in a graceful fashion. It make take up to 10 minutes to shut down a Stream instance as the data ownership has to be handed over to the remaining nodes. Do not forcefully terminate instances during this period.

Bring instances back up in reverse order.

Scaling up Stream Tier

Stream service does not support automatic scaling, however, a manual scale-up is possible.

  1. In the header of Dev Studio, click Configure Decisioning Infrastructure Services Stream .
  2. Ensure that all Stream instances are up and have the status NORMAL.
  3. Ensure that under-replicated and offline partitions are 0 for all instances in the list.
  4. If under-replicated and offline partitions are greater than 0, wait until they get to 0.
    Scaling up the Stream service
    scaling-up icon
  5. Start an additional Stream instance.
    You can add only one instance at a time. Repeat the procedure if you need to add more instances.

Scaling down Stream Tier

Stream service does not support automatic scaling, however a manual scale-down is possible:

  1. In the header of Dev Studio, click Configure Decisioning Infrastructure Services Stream .
  2. Select a node you would like to take out of the service, and click Execute Decommission .
  3. Wait until the instance is removed from the list.
  4. Terminate the node.

Full recovery Stream Tier

In case of a catastrophic failure, the following procedure can be followed to recover the Stream tier:

This procedure results in Kafka Data Loss.
  1. Bring all Stream service instances down.
  2. In the header of Dev Studio, click Configure Decisioning Infrastructure Services Stream .
  3. Decommission all Stream service instances.
  4. Truncate the Stream service tables in the Pega Platform data schema:
    • truncate table pr_data_stream_nodes
    • truncate table pr_data_stream_sessions
    • truncate table pr_data_stream_node_updates
  5. Bring the Stream instances back up one instance at a time.
  6. Delete Kafka-data directory from the Stream instances.
    This directory, by default is created under Tomcat folder.

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.