SA-52147

Summary

User frequently, but sporadically experience 'hangs' whilst using the Pega Marketing Portal; the only way found to restore service is to restart PRPC.

PRPC and the backend Oracle Database are hosted on a 'Private Cloud' (Microsoft Azure) ; both PRPC and the Database are located on the same host.

Error Messages

No error messages are seen directly from the Portal ; the Portal simply becomes unresponsive.

The logs do show the following errors however:

2018-01-31 13:56:42,000 [ PegaRULES-Batch-1] [ STANDARD] [ ] [ PegaMarketing:07.31] (dbms.JdbcConnectionManagerImpl) ERROR - Not returning connection 4 for database "externalmktdata" to the pool as it previously encountered the following error
User ID: System
Last SQL: SELECT "LOCKNAME" AS "LockName" , "LOCKDATETIME" AS "LockDateTime" from PEGA_MARKETING.mkt_paidmedia_lock WHERE "LOCKDATETIME" < ?
java.sql.SQLRecoverableException: IO Error: Connection timed out (Read failed)
[...]

Steps to Reproduce

Not Applicable: just normal use of Pega Marketing Portal is sufficient to witness the issue.

Root Cause

A defect or configuration issue in the operating environment.

Pega Marketing requires a "Database Rule" called 'ExternalMKTData' .

See Page 7:

https://pdn.pega.com/documents/pega-marketing-731-installation-guide The deployment was setup as per the guide; a JDBC URL was provided - and the 'Test Connection' button showed the connection was working.

However: the 'hostname' in the JDBC URL was actually using the internet-facing hostname* for the DB host; rather than a local hostname.
The upshot of this is that traffic from PRPC to the DB was subject to Firewall Rules.

It become apparent that the Firewall was dropping packets from PRPC; due (presumably) to time-outs of 'idle' connections or some other Firewall Rule.

(*As an aside - this is not a recommended configuration: it is a potential security-risk).

Resolution

Make the following change to the operating environment:

The 'ExternalMKTData' should reference a local IP / hostname (that is a IP that is only resolvable from within the DMZ, not from the Internet).

In this case; it was possible to use 'localhost' for this value - since the DB and PRPC were co-located.

To ensure that the 'Database Rule' is picked-up ; restart PRPC after the changes were made and saved.

The following sketch shows the 'before' and 'after' situations: