Request: Easy method for analyzing different profiling results

[dancamper]

Satisfy this scenario: Profiling is used to analyze new data that will be ingested. Profiling results are saved as a logical file. Then, a new batch of data arrives and is profiled. The new method should compare the new profiling results with the old and output a summary of any differences.

The end goal is to highlight significant differences between the two profiles, which could indicate a significant or unexpected change in the incoming data stream.

[RichardTaylorHPCC]

JOIN the two using ROWDIFF in the TRANSFORM would be a start :)

[dcamper]

This is probably more appropriate as a stand-alone function rather than incorporated into Profile(), as it is file-based rather than field-based.