Thanks to everyone who joined our recent webinar and kept the chat buzzing with questions! Missed the session or want to watch it again? Catch the full recording here. Below is the full Q&A from the session, answered by our panel - Tihomir Petrovic, Madhuri Vasa, Goutham Mandadi, and Sachin Vellanki.
Test Automation Scope & Compatibility
- Can test automation cover UI-Kit or Theme/Cosmos apps? No - GenAI test automation is currently supported only for Constellation applications.
- What about hybrid apps (Constellation implementation layer, UI Toolkit/Theme-Cosmos built-on)? Yes - supported, provided the top-level end-user application is built on Constellation.
- Can I run application tests without Deployment Manager? Yes - download the full automation script bundle as a zip and run it locally or on any other CI/CD pipeline.
- Can custom components be tested? Not currently - automation covers out-of-the-box Pega base components; for custom components you'll need to write custom code or adapt the auto-generated test skeletons.
- Can we test external AI agents? No - this capability only tests the functionality of the Pega application itself.
- Is test automation available for Client-Managed Cloud, or only Pega Cloud? AI-assisted authoring is available on Client-Managed Cloud, but the automated execution infrastructure is currently exclusive to Pega Cloud 3.
- Is AI testing only available on Pega Cloud, not on-premise? Same answer as above - authoring works broadly, but automated execution is a Pega Cloud 3 capability today.
Execution, Integration & Tooling
- Can external automation tools connect to Pega and execute our test cases? Not yet natively - tests currently run through Deployment Manager or within the Pega environment. An API endpoint for external tools to trigger API tests is on the roadmap.
- Is integrating with Playwright an option? Yes - Playwright is the primary engine. Generated tests use Playwright and TypeScript, and Pega Cloud provides native infrastructure to run them without needing third-party orchestrators like Jenkins or Azure DevOps.
- Do I need to install Playwright locally to run downloaded tests? Yes, if running locally (dependencies are provided). Not needed if running through Deployment Manager, since Pega handles the infrastructure as a service.
- Can Infinity '26 scripts be executed on another test bed, like Bamboo? Yes - download the test package and run it in your own orchestrator, as long as the environment is configured correctly.
- Is there a centralized view to monitor test execution across multiple pipelines? Today, the Application Quality dashboard in Deployment Manager shows execution trends for Pega Unit tests. Application Test execution trends are on the roadmap for early 2027.
- Can we reuse existing Playwright tests inside the Pega test suite / Deployment Manager? Not currently - the infrastructure supports only out-of-the-box Application Tests. Tagging and suite-management features are coming in future releases; please raise this as product feedback in the meantime.
- Can we configure simulations for integrations in an application test? No, this is not currently supported.
GenAI & Maintenance
- Can AI generate tests for existing applications, not just new Blueprint apps? Yes - as long as the application is built on the Constellation architecture.
- Should we write explicit prompts to create pipelines in DM, or during Blueprint design? Discussed live during the session - please check the recording for the full nuance on prompt timing.
- How do we maintain test cases as screens, data, or flows change - can AI help? Yes - AI can update tests by reading the changes in a branch.
- Does regenerating a test automatically invalidate old ones, or do we handle cleanup? There's no automatic cleanup of test cases or test data today; this needs to be handled manually or via a utility activity.
- Do generated test artifacts (e.g., cases) get cleaned up automatically, like Pega Unit tests? Same as above - no automatic cleanup yet; manual or utility-activity-based cleanup is required.
- What are the known limitations and accuracy metrics of this capability? In the current release, the focus is on the case lifecycle - landing pages, external-only REST APIs, and custom components are not yet automated.
- Can we integrate external AI tools like Claude or OpenAI to generate tests directly in a Constellation app? Yes - tests are rules, so they can be generated using external AI tools via the Pega MCP protocol (a topic we may dedicate a future webinar to).
- Is Gen AI licensing required? Can we bring our own model (e.g., Claude via MCP)? Does this work on Client Cloud? Three-part answer: (1) it's part of the Gen AI bundle and requires a signed addendum; (2) yes, you can connect through Claude or another external harness and leverage Pega MCP to generate tests; (3) it's possible on Client Cloud - please confirm specifics with your account executive.
Pipelines, Blueprint & Deployment
- What are the prerequisites for getting pipelines created? None from a manual setup standpoint - the process is fully automated for users on Pega Cloud 3 using Deployment Manager as a service.
- If an app is extended with a new Blueprint for another case type, how is the existing pipeline handled? Deployment Manager creates a "Version 2" of each relevant pipeline.
- Does Blueprint support reusing existing enterprise assets (data models, integrations, APIs) instead of generating new ones? Discussed live during the session - please see the recording for the detailed walkthrough.
- How do you recommend teams deliver a Blueprint-generated solution iteratively? Treat Blueprint as the app baseline/foundation - the generated implementation plan becomes the bridge into iterative delivery, rather than treating the branch as one large delivery unit.
- Are there plans to bring automatic pipeline creation to customers who don't use Pega Cloud? No current plans for Client-Managed Cloud or on-premise - please raise this as product feedback.
- DX API endpoints are configured per customer service package - how do we reconfigure tests with less effort? Discussed live during the session - please see the recording for the recommended approach.
Reporting, Thresholds & Reliability
- How long are reports available, and can we adjust the pass threshold (e.g., accept 97%)? Configurable thresholds are on the roadmap; today a 100% pass rate is required, though privileged users can manually skip failed tasks. Service-level reports are kept for 7 days, while route-to-live evidence (screenshots/videos) remains until manually erased.
- What about load testing? These tests focus on application functionality and lifecycle, not load - load testing is handled separately with tools like JMeter or LoadRunner against a baseline.
- Is recording video/images an alternative to robotic testing? No - Playwright records execution videos and screenshots for UI testing, but robotic testing targets desktop applications specifically; they're distinct capabilities.
- Is "application test" the same as the former "scenario testing"? "Application test" is the new term for Constellation apps, covering both UI and API (DX API) testing. "Scenario Testing" is on a deprecation path since it doesn't support Constellation.
- What about headless applications (Pega as orchestrator only, no UI, heavy REST usage)? Focus on API tests and unit tests - API testing lets you functionally test REST services and connectors without a UI layer.
- We hit a runtime exception running application tests via Deployment Manager - should we open a support case? Yes, please open a support case.
- Do we need to configure authentication to run application tests? Yes - since tests rely on DX APIs, you'll need to configure OAuth authentication profiles and client credentials.
Using bots for testing, especially for compliance and repetitive tasks? Discussed live during the session - please see the recording for this topic.
