In order to support create, read, update, and delete (CRUD) operations on nodes, Charlatan DAO interface implementations use the Pega Platform database. Charlatan stores the node tree in the following table:
node parent ID
session which created the node
fields with other node properties
Charlatan node tree
Broker watches support
In order to support watches, all updates need to be stored in the database. The existing interface functionality can also be implemented by using Apache Ignite/Hazelcast, or other databases.
Once a broker sets watch on a specific node change, it should receive a one-time update, regardless of whether the change was performed by this or any other broker. Every broker pulls updates created by different brokers from the table and notifies its watches.
For every node CRUD operation, an entry in the following node updates table is created and stored:
event type, for example: Created, Deleted, ChildrenChanged, DataChanged
full node path
broker identifier that generated the change
The node updates table
Ephemeral nodes support
If a node was created as ephemeral, it should be removed once the broker is off. An ephemeral broker from a previous broker session prevents the alive broker from starting. Removal of ephemeral nodes triggers alive brokers to re-select the leader, if needed, and to start replication operations.
Remove ephemeral nodes when broker is off and notify the rest of the brokers about the updates.
The database is used to monitor broker session states. With every broker’s keepalive message, the last_seen timestamp of the session is updated. See the following pr_data_streamservice_sessions table:
broker session identifier
timestamp of the last ping
The broker sessions table
Every Charlatan server periodically monitors its stale sessions and other Charlatan servers' stale sessions. If a stale session (last_seen timestamp timed out) is found, the server invalidates the stale session; it removes ephemeral nodes created by that session and notifies all connected brokers that are interested in this event.
A Charlatan server can invalidate sessions created by itself and sessions created by a different Charlatan sever.
When a Charlatan server/Pega Platform node crashes, broker sessions are invalidated by running the Charlatan server. If there are no more running Charlatan servers, the sessions are invalidated during the Charlatan server startup.
When a Kafka broker crashes, the Charlatan server invalidates its session.