Migrating SMTP and IMAP email servers from Microsoft Outlook to Office 365
This article explains what you need to know if your Pega 8 application is configured to send and receive email messages as Pega Correspondence given recent changes to Pega code and rules. If you, as an application developer, copied (Save As) the Pega-provided activity @baseclass.SendEmailNotification in Pega-ProCom RuleSet v7 or earlier, you should read this article.
This article assumes that you are using Microsoft Exchange as your server and that you are migrating your email accounts from on-premise Microsoft Outlook to cloud-based Office 365.
Pega-ProCom SendEmailNotification changed to Pega-Integration SendEmailNotification
Inbound and Outbound Email Correspondence are Data Integration capabilities of the Pega Platform. The activity @baseclass.SendEmailNotification in the Pega-ProCom ruleset allows you to override the default setting and tailor it to your needs. In Pega Platform releases of recent years, the SendEmailNotification activity has changed several times to address new features and provide hotfixes to issues reported.
With Pega 8.1, the activity @baseclass.SendEmailNotification changed to the ruleset Pega-IntegrationEngine.
This change can prevent SMTP from working when authentication is required.
Factors to consider and potential issues
Pegasystems’ IT and Engineering teams have identified several factors and issues in their work migrating Pega-internal Microsoft Exchange servers from Microsoft Outlook to Office 365.
If Tomcat is your application server, be aware that, with Pega 8.1, Java Mail no longer requires the use of the mail.jar file to be in the Tomcat \lib folder.
In addition, one or more required changes are needed to use SMTP on Office 365 or other SMTP servers requiring authentication.
On-premise Microsoft Exchange servers are set up to use Port 25 with Anonymous credentials. This means that these servers do not allow passwords to be added to the Data-EmailAccount. This makes the email accounts insecure.
Furthermore, in previous Pega releases, you probably copied and tailored Pega-ProCom activity @baseclass.SendEmailNotification to meet business requirements for Inbound and Outbound Email messages within your applications. Changes made since the original version of Pega-ProCom activity @baseclass.SendEmailNotification, if not updated, can cause issues.
A particularly challenging root cause to isolate is a cryptic error seen in the logs. The error persistently suggests the firewall or the list of allowable applications as the core issue when these factors are not the cause of the problem.
Depending on your Email Account configuration, you might see errors in the logs like the following error:
2019-03-25 12:05:16,301 [11,13,16,19,5,7,8,9]] [ STANDARD] [ ] [ ] (.generated.pega_procom_default) ERROR - Unexpected exception caught while trying to send an email message.
com.sun.mail.smtp.SMTPSendFailedException: 530 5.7.57 SMTP; Client was not authenticated to send anonymous mail during MAIL FROM [host.server.dns.name.here]
Best practice: Use the Pega-IntegrationEngine ruleset version of the SendEmailNotification activity.
Work with your IT organization to migrate your organization’s mailboxes, hosted on Microsoft Exchange servers, from Microsoft Outlook to Office 365.
After your organization’s mailboxes have been migrated to Office 365, you can begin the migration of your SMTP and IMAP servers to Office 365, using one of the suggested approaches, and adjust settings for Office 365 servers.
Choose one of the following approaches and adjust settings for Office 365 servers.
This approach might be the quickest to implement, but it might result in unexpected issues.
Withdraw the overridden version of the Pega-ProCom ruleset @baseclass.SendEmailNotification activity in your application and replace it with the new Pega 8.1-provided Pega-IntegrationEngine ruleset version of the SendEmailNotification activity.
Assumption: This approach assumes that you started with the Pega-provided Pega-ProCom version of the activity SendEmailNotification and have customized it as you upgraded from one Pega release to newer releases.
To continue using your customized version of Pega-ProCom SendEmailNotification, make the following changes to the SendEmailNotification activity and Data-EmailAccount instances:
- In the SendEmailNotification activity record, make the following changes:
- Update the Java Step 6:
Change this string --
String retval = pega_procom_default.SendEmailMessage(myStepPage);
To this string --
String retval = pega_integrationengine_default.SendEmailMessage(myStepPage);
- Update the Parameters tab to include all the Parameters and Local Variables that your version does not have.
- Update Step 1 for the additional Local Variables added to the Parameters tab.
- Update Step 5 for the additional Parameters added to the Parameters tab.
- Update the Java Step 6:
- In Data-EmailAccount instances, the Sender section, make the following changes, as shown in the image below:
- Specify values for the Identity fields.
- Specify the following Connection values:
- For SMTP host, enter smtp.office365.com.
- For Port, enter 587.
- Do not select the check box Use SMTPS.
Selecting the Use SMTPS check box causes the system to attempt to connect with Port 465, which is invalid. This option will not work because the Port value provided in the Data-EmailAccount will be ignored, unless the port is opened in the firewall and the host is expecting traffic on that port. For more information about Port 465, see the mailgun blog, Which SMTP Port Should I Use? Understanding Ports 25, 465, & 587.
- Do not rely on the button to prove that you can connect and send email.
This only sends a “hello” to the server with the UserID and Password to validate if it is correct. When you send an email, you are logging in to the mailbox and then sending the message.
This is not the same as checking credentials.
If you see an error in the logs like the one shown above (Error), you might have the issue described above and need to make the changes of the previous steps.
Coordinate with your IT organization to configure the Windows Active Directory UserIDs and Office 365 User Principal Names (UPNs) to match your enterprise’s domain, for example, <userID>@<mydomain>.com.
Use the UPN in the Data-EmailAccount instances for both SMTP and IMAP when pointing to Office 365 servers.
Additionally, adjust other settings for Email Listeners that are affected by the migration from on-premise Microsoft Outlook to cloud-based Office 365.
When an email is replied to in a Pulse post, only the text of the reply is included in the Pulse Note itself on the work item. This makes readability much better and avoids duplicating long conversations that go back and forth in the application’s Pulse thread.
- Set Identity to Shared account type.
This setting for application-related email accounts reduces overall cost.
- Update the setting EwsEnabled to False.
- Update the setting ImapUseProtocolDefaults to False.
- Update the setting PopMessagesRetrievalMimeFormat to htmlandtextalternative.
This setting allows the use of HTML formatting for inbound emails to the application's Email Listener.
- Set Passwords to Never Expire for shared mailboxes.
If the passwords have not changed or changed periodically, now is the time to change them.
There are a few methods to test your SMTP settings, but the following steps are the quickest way:
- Navigate to landing page.
- Scroll down to the Notifications section.
- In the E-mail account field, enter the name of the Data-EmailAccount instance.
- In the E-mail addresses field, enter your email address.
- Click Save Settings.
- Scroll up to the three index choices: Rules, Data, and Work.
- Select All data and click the button to the right.
- Respond to the prompt to get a quick result:
- Is the number of documents less than 20,000? Accept the default choice All Classes in this index or choose Only Classes listed below and pick one that does not have a lot of rows to it.
- Is the number of documents greater than 20,000? Reduce your options of what to index.
What matters here is that the email messages are being sent, not that the index is working.
- Click .
- Click again to confirm the process.
- Wait for the process to finish and the Email Notification to be received.
- If you receive the email message, your changes work.
- If you do not receive the email message, review the logs to determine what caused the failure. Also verify that you have completed all steps of Approach 2 correctly.
After testing SMTP for outbound email, verify your changes for IMAP inbound email. Your test ensures that the application’s Email Listener data instances are specified correctly and that the Email Listeners are connecting, logging into the mailbox, and retrieving and processing the email messages as they arrive.
To test your IMAP settings for inbound email, follow these steps:
- In Pega-IntSvcs, the class Email Listener, create the Data-Admin-Connect-EmailListener rule and logic to process inbound email messages to your application:, the ruleset
In the EmailListener form, the Email Account section, add the Data-EmailAccount name. The application uses the credentials that you specify to process email coming in to the mailbox.
See the Help topics, About Email Listener data instances, particularly Email Listener form – Completing the Properties tab.
- Log in to Outlook.
- Add the mailbox to your client.
- Send one or more email messages of any format and layout to the mailbox.
- Check to see that the email messages came into the Outlook Inbox as expected.
- In Active Listeners, including the Email Listener that you created in Step 1. see the
- Check the Email Listeners date-time stamp of Last Access, the Errors, the Status, and the Actions that you can perform.
- If you see errors, perform the appropriate action (from the menu), review the log files, and correct the problem as required.