Close popover

Table of Contents

Creating and running an Extract rule

Version:

After you enable the built-in Business Intelligence Exchange (BIX) ruleset on Pega Platform, you can create and run Extract rules to extract specific class data from your application.

Create and open an Extract rule to run a manual BIX extraction process. To limit the properties included in your extract, use filter criteria to extract specific properties from a class. Filters can also exclude specific properties from an extraction and extract properties that have changed since the last extraction was run. Implementation of filter conditions can significantly increase BIX extraction performance. See step 7 and Creating and running an Extract: Filter Criteria to implement filters. For more information about BIX performance, see Optimizing BIX data extraction performance.

Manual BIX extractions are not designed to be run on a production environment. To run extractions as a background process on a production environment after creating an Extract rule, see Using the Job Scheduler rule to extract data with BIX.
  1. In the header of Dev Studio, click Create SysAdmin Extract.

    Alternatively, in the navigation pane of Dev Studio, click App. Right-click the name of the application from which you want to extract class data, and then select Create SysAdmin Extract.

  2. In the Label field, enter a name for the Extract rule.

  3. In the Context field, select the application, class, and ruleset from which to extract data.

  4. Click Create and open.

  5. On the Definition tab, specify the output format, and then select properties to extract:

    • XML: Extract properties to an XML file according to top-level pages, page lists, and page groups with embedded XML tags.
      • For on-premises environments: The application extracts the XML file to an output directory defined in the File Specification tab in step 8.
      • For Pega Cloud environments: The application extracts the XML file to the Pega Cloud File Storage Repository.
    • Comma Separated Value (CSV): Extract properties to comma-separated values in separate text files according to top-level pages, pages lists, page groups, and value properties.
      • For on-premises environments: The application extracts CSV files to an output directory defined in the File Specification tab in step 8.
      • For Pega Cloud environments: The application extracts CSV files to the Pega Cloud File Storage Repository. The CSV format for Pega Cloud environments also adds a passwordprotection option below the format drop-down.
    • Database Schema: Extract properties directly into an existing relational database outside of the application. The extraction process forms top-level pages, page lists, page groups, and value properties from a selected class into separate tables on the database. Identify the external database instance on the File Specification tab in step 8. For more information about creating an external database instance, see Creating external database instances with JDBC URLs.
    1. In Property, at the class level, click the properties icon.

    2. In the Properties Configuration Window, in the From Class, select the check box of the page lists, page groups, and single value properties to extract, then click Submit.

    3. After selecting a page list or page group to extract, click the arrow icon next to the class to display the selected page lists or page groups.

    4. Click the properties icon next to that page list or page group to further specify nested pages or single value properties to extract.

    5. For single value properties, format how the extracted output appears with the following options varying on output type:

      • For CSV extractions: Map single-value properties to a name in the CSV file. Do not duplicate names.
      • For Database Schema extractions: In To Table, provide a name for the extracted properties table in the external database for the extraction process. Do not use whitespace or duplicate names.
    6. In the Property Configurations form, click Submit. Return to step 5b as necessary. XML formats can extract all properties from a selected class with option Get all properties below the format drop-down.

  6. Click Save.

  7. Optional:

    On the Filter Criteria tab, select filter conditions using properties of the primary class. For more information, see Creating and running an Extract: Filter Criteria.

  8. In the File Specification tab, specify the directory details for the specific output file format you want to use for the extraction process:

    On-premises extraction directories and file naming options

    Output format Fields
    XML
    • XML Schema Directory - Specify a local directory in which to place the XML schema file.
    • XML Schema File Name - Enter a name for the XML schema files.
    • XML File Output Directory - Specify the target directory to place the XML output files.
    • XML Output File Name - Specify the target directory in which to place the XML output files. Pega Platform automatically adds .xml and .xsd extensions to the file name.

    Pega Platform supports appending the following wild-card formatting strings to the output filename:

    • %i - (RunIdentifier) Adds a unique run-identifier to the extract file.
      • Pega Platform generates a run-identifier consisting of the pzInsKey of the Extract rule followed by the sequence number which increments after each extraction process.
    • %s - Adds an automatically generated sequence number to each new extract. For example, BIXExtract1, BIXExtract2
    • %c - (ClassName) Adds the class to which Pega Platform applied the Extract rule (see step 3). For example, BIXExtract_PegaSample-Work
    • %r - Adds the name of the Extract rule. For example, BIXExtract_WeeklyExtract1
    • %d - (ExtractRunDate) Adds the date of the extract run.
    • %t - (ExtractRunDateTime) Adds the date and time of the extract run.

    For example, specifying the output file name BIXExtract_%s_%c_%r generates in the output file name as PegaExtract_1_PegaSample-Work_WeeklyExtract.

    Use %d and %t when running multiple extracts to get unique filenames and avoid Pega Platform overwriting previous extract files with the same name in the specified directory
    CSV Specify the target directory for the CSV extract files.
    Database Schema Specify a JDBC URL to an existing external database instance into which to insert the database schema.

    Pega Cloud extraction directories

    Output format Fields
    XML XML & CSV extractions place the extract files into your Pega Cloud File Storage environment repository.
    CSV
    Database Schema Specify a JDBC URL that links to an existing external target database instance with read-only privileges into which to insert the database schema.

    On-premises manifest extraction directory and file naming options

    Output format Fields
    XML
    • Manifest XML file output directory - Specify a local target directory to receive the XML manifest file.
    • Manifest XML output file name - Enter a name for the XML manifest files. Pega Platform automatically adds .xml and .xsd extensions to the file name.

    Pega Platform supports appending the following wild-card formatting strings to the output filename:

    • %i - (RunIdentifier) Adds a unique run-identifier to the extract file.
      • Pega Platform generates a run-identifier consisting of the pzInsKey of the Extract rule followed by the sequence number which increments after each extraction process.
    • %s - Adds automatically generated sequence number of each new extract. For example, BIXExtract1, BIXExtract2
    • %c - (ClassName) Adds the class to which Pega Platform applied the Extract rule (see step 3). For example, BIXExtract_PegaSample-Work
    • %r - Adds the name of the Extract rule. For example, BIXExtract_WeeklyExtract1
    • %d - (ExtractRunDate) Adds the date of the extract run.
    • %t - (ExtractRunDateTime) Adds the date and time of the extract run.

    For example, specifying the output file name BIXExtract_%s_%c_%r generates in the output file name as PegaExtract_1_PegaSample-Work_WeeklyExtract.

    Use %d and %t when running multiple extracts to get unique filenames and avoid Pega Platform overwriting previous extract files with the same name in the specified directory.
    CSV Specify the target directory for the CSV manifest file.
    Database Schema Specify a JDBC URL to an existing external database instance into which to insert the database schema.

    Pega Cloud manifest extraction directories

    Output format Fields
    XML XML and CSV extractions place the manifest files in the Pega Cloud File Storage repository.
    CSV
    Database Schema Specify a JDBC URL to an existing external database instance into which to insert the database schema.
    1. Click Test Connectivity to confirm local application server filepath validity for XML and CSV extractions (on-premises only) and database connectivity for database schema extractions.

    2. Optional:

      In Include check total property, enter a property to use as a checksum SQL function for the extraction process. This option is not available for high-throughput CSV extractions. For more information, see Optimizing BIX data extraction performance.

    3. Click Save to apply the directories and external database location that you select.

  9. In the Parameters tab, create parameters that populate the clipboard at runtime. You use these parameters to customize BIX extractions by specifying them in the Filter Criteria tab.

    1. Enter the Name and Description for the parameter.

    2. In Data Type, enter the value type of the parameter

    3. In Default value, enter the parameters default value.

    4. Click Save to apply parameters.

    5. To specify these parameters in a Filter Criteria, enter these parameters in the Filter Criteria tab. For details, see Creating and running an Extract: Filter Criteria.

  10. Optional:

    In the Pages & Classes tab, create pages that populate the clipboard at runtime. You use these Pages & Classes to customize data extractions by specifying them in the Filter Criteria tab.

    1. In Page name, identify a clipboard page with the format pyWorkPage.pyUserPage.

    2. In the Class field, select the class of the specified page.

    3. Click Save to apply the page and class.

    4. To specify these pages and classes in a Filter Criteria, enter these pages and classes in the Filter Criteria tab. For details, see Creating and running an Extract: Filter Criteria.

  11. In the Execution History tab, view a list of concurrently running and previously run BIX extractions by clicking on each row.

  12. To manually run a BIX extraction process, in the rule form toolbar, click Actions Run .

  13. In the Label to use for this run field, enter a unique identifier for this BIX extraction. Extract files contain the unique identifier to reference the Extract rule that ran the process. For details, see Unique run identifier.

  14. For XML and CSV extracts, click Perform Extract.

  15. For Database Schema extractions, perform the following tasks to generate a DDL for new extracts, changes to an existing extraction, or after a BIX upgrade:

    1. Click Generate schema DDL to generate a DDL file to create a formatted database table for the extraction process.

    2. If you chose to generate a manifest and store in a database schema, click Generate manifest schema DDL for the manifest table.

    3. Click Perform Extract.

      • For Pega Cloud deployments, a URL appears for the DDL file.
      • For on-premises deployments, a directory field appears for the system to download a DDL file to a location on the application server.
      • Share the DDL file with your database administrator so that the administrator can format the external database table for extractions.
  16. For Database Schema extractions, perform the following tasks to run the extraction after completing step 15:

    1. To run manual extractions to a database schema, repeat step 13a. with the Generating schema DDL and the Generate manifest schema DDL check box cleared. BIX extractions through the Job Scheduler for database schema formats are now available to run after implementing schema DDL for a target database.

    2. In the Number of records to update at a time field, enter the number of records to insert to the target database in a batch.

    3. Click Perform Extract.

  17. Repeat step 12 to 16 as applicable to run additional manual extraction processes.

    • To run a BIX extraction process with additional filter criteria (such as when defining a filter for a specific table that is updated from the last extraction), repeat steps 7a to 7d, then return to step 12.
Retrieve your BIX extract files from their default save location:
  • For Pega Cloud Services clients using XML or CSV formats, your BIX extract files are saved in the Pega Cloud Services File Storage repository. For more information, see Procedures for BIX extractions in Pega Cloud environments.
  • For on-premises clients using XML or CSV formats, your BIX extract files are saved in the local directory that you specify.
  • For database schema extractions, refer to your external database instance to view the extraction table data.

Suggest Edit

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.