Data is jumbled up when creating file-file repository
When generating an export from a dataset, data is incorrectly formatted. This occurs when some properties do not have data.
Data is written correctly when all properties contain data.
0 20180613T201558.001 GMT 0 23456789 23456789 0 0 0 Class-Customer 0 33 0 0 0 0 0 0 0 33 0
0 20180613T201558.956 GMT 0 2345678 2345678 0 0 0 Class-Customer 0 0 0 0 0 0 0 0 0 0 0
When properties do not have data, tabs are missing and the order of the properties is incorrect as in the below example.
Class-Customer 20180614T165953.105 GMT 33 0 3456789 34567890 0 0 0 0
Steps to Reproduce
This behavior is as per Pega product design.
Here’s the explanation for the reported behavior:
This occurs when running Decision Data Store (DDS) > File data flow when the target dataset is configured with CSV file type and property auto mapping.
In DDS, records exist which contain different sets of properties. Data is streamed to a CSV file due to which the header and order of properties must be determined. This is done based on the first record. If the first record emerging from the DDS does not have a property set, which subsequent records have, then that record will not be present in the destination file.
Perform either of the following options for a deterministic order and set of properties:
- Configure mappings manually in the destination dataset.
- Use JSON files for output as it is a schema-less format.