Creating and running an Extract rule

After you enable the built-in Business Intelligence Exchange (BIX) ruleset on Pega Platform, you can create and run Extract rules to extract specific class data from your application.

Create and open an Extract rule to run a manual BIX extraction process. To limit the properties included in your extract, use filter criteria to extract specific properties from a class. Filters can also exclude specific properties from an extraction and extract properties that have changed since the last extraction was run. Implementation of filter conditions can significantly increase BIX extraction performance. See step 7 and Creating and running an Extract: Filter Criteria to implement filters. For more information about BIX performance, see Optimizing BIX data extraction performance.
Note: Manual BIX extractions are not designed to be run on a production environment. To run extractions as a background process on a production environment after creating an Extract rule, see Using the Job Scheduler rule to extract data with BIX.
  1. In the header of Dev Studio, click Create > SysAdmin > Extract.

    Alternatively, in the navigation pane of Dev Studio, click App. Right-click the name of the application from which you want to extract class data, and then select Create > SysAdmin > Extract.

  2. In the Label field, enter a name for the Extract rule.
  3. In the Context field, select the application, class, and ruleset from which to extract data.
  4. Click Create and open.
  5. On the Definition tab, specify the output format, and then select properties to extract:
    • XML: Extract properties to an XML file according to top-level pages, page lists, and page groups with embedded XML tags.
      • For on-premises environments: The application extracts the XML file to an output directory defined in the File Specification tab in step 8.
      • For Pega Cloud environments: The application extracts the XML file to the Pega Cloud File Storage Repository.
    • Comma Separated Value (CSV): Extract properties to comma-separated values in separate text files according to top-level pages, pages lists, page groups, and value properties.
      • For on-premises environments: The application extracts CSV files to an output directory defined in the File Specification tab in step 8.
      • For Pega Cloud environments: The application extracts CSV files to the Pega Cloud File Storage Repository. The CSV format for Pega Cloud environments also adds a passwordprotection option below the format drop-down.
    • Database Schema: Extract properties directly into an existing relational database outside of the application. The extraction process forms top-level pages, page lists, page groups, and value properties from a selected class into separate tables on the database. Identify the external database instance on the File Specification tab in step 8. For more information about creating an external database instance, see Creating database instances for JDBC URLs.
    1. In Property, at the class level, click the properties icon.
    2. In the Properties Configuration Window, in the From Class, select the check box of the page lists, page groups, and single value properties to extract, then click Submit.
    3. After selecting a page list or page group to extract, click the arrow icon next to the class to display the selected page lists or page groups.
    4. Click the properties icon next to that page list or page group to further specify nested pages or single value properties to extract.
    5. For single value properties, format how the extracted output appears with the following options varying on output type:
      • For CSV extractions: Map single-value properties to a name in the CSV file. Do not duplicate names.
      • For Database Schema extractions: In To Table, provide a name for the extracted properties table in the external database for the extraction process. Do not use whitespace or duplicate names.
    6. In the Property Configurations form, click Submit. Return to step 5b as necessary. XML formats can extract all properties from a selected class with option Get all properties below the format drop-down.
  6. Click Save.
  7. Optional: On the Filter Criteria tab, select filter conditions using properties of the primary class. For more information, see Creating and running an Extract: Filter Criteria.
  8. In the File Specification tab, enter directory specifications for the output file for the following BIX extraction formats:
    Extraction directories by environment
    File Specification Pega Cloud Environments On-premises
    XML XML & CSV extractions place the BIX extraction files into the environment's Pega Cloud File Storage repository. Enter an XML file output directory and an XML output file name. Optionally, enter a XML schema output directory and XML schema file name. For more information about generating an XML schema files (XSD) for XML extractions, see Generating an XSD file for XML output.
    CSV Enter the output directory of the CSV file.
    Database Schema Enter an existing external database instance with read-only privileges using a JDBC URL. Enter an existing external database instance with read-only privileges using a JDBC URL.
    Extraction manifest file directories by environment
    File Specification Pega Cloud Environments On-premises
    XML XML and CSV extractions place the extraction files in the Pega Cloud File Storage repository for XML & CSV manifest files. Enter a directory local to the app server for XML & CSV manifest files.
    CSV
    Database Schema Extractions map directly to an external database instance with read-only privileges created with a JDBC URL.
    1. Click Test Connectivity to confirm local application server filepath validity for XML and CSV extractions (on-premises only) and database connectivity for database schema extractions.
    2. Optional: In Include check total property, enter a property to use as a checksum SQL function for the extraction process. This option is not available for high-throughput CSV extractions. For more information, see Optimizing BIX data extraction performance.
    3. Click Save to apply the directories and external database location that you select.
  9. In the Parameters tab, create parameters that populate the clipboard at runtime. You use these parameters to customize BIX extractions by specifying them in the Filter Criteria tab.
    1. Enter the Name and Description for the parameter.
    2. In Data Type, enter the value type of the parameter
    3. In Default value, enter the parameters default value.
    4. Click Save to apply parameters.
    5. To specify these parameters in a Filter Criteria, enter these parameters in the Filter Criteria tab. For details, see Creating and running an Extract: Filter Criteria.
  10. Optional: In the Pages & Classes tab, create pages that populate the clipboard at runtime. You use these Pages & Classes to customize data extractions by specifying them in the Filter Criteria tab.
    1. In Page name, identify a clipboard page with the format pyWorkPage.pyUserPage.
    2. In the Class field, select the class of the specified page.
    3. Click Save to apply the page and class.
    4. To specify these pages and classes in a Filter Criteria, enter these pages and classes in the Filter Criteria tab. For details, see Creating and running an Extract: Filter Criteria.
  11. In the Execution History tab, view a list of previously run BIX extractions by clicking on each row.
  12. To manually run a BIX extraction process, in the rule form toolbar, click Actions > Run.
  13. Enter a Unique Identifier for the BIX extraction. Extract files contain the unique identifier to reference the Extract rule that ran the process. For details, see Unique run identifier.
  14. For CSV & XML extractions, click Perform Extract to extract data from your application. Click the Execution History tab to view the extraction process status.
    1. For Database Schema extractions, click Generate schema DDL to generate a DDL file to create a formatted database table for the extraction process.
    2. In the Number of records to update at a time field, enter the number of records to insert to the target database in a batch.
    3. Click Perform Extract.
      • For cloud deployments, a URL appears for the DDL file.
      • For on-premises deployments, a directory field appears for the system to download a DDL file to a location on the application server.

        Share the DDL file with your database administrator so that the administrator can format the external database table for extractions.

    4. Optional: To run manual extractions to a database schema, repeat step 12 with the Generating schema DDL check box cleared. BIX extractions through the Job Scheduler for database schema formats are now available to run after implementing schema DDL for a target database.
  15. Repeat step 12 to run future manual extraction processes.
    • To run a BIX extraction process with additional filter criteria (such as when defining a filter for a specific table that is updated from the last extraction), repeat steps 7a to 7d, then return to step 12.
What to do next: Retrieve your BIX extract files from their default save location:
  • For Pega Cloud Services clients using XML or CSV formats, your BIX extract files are saved in the Pega Cloud Services File Storage repository. For more information, see Procedures for BIX extractions in Pega Cloud environments.
  • For on-premises clients using XML or CSV formats, your BIX extract files are saved in the local directory that you specify.
  • For database schema extractions, refer to your external database instance to view the extraction table data.