Overview
From goodtabes@3 (now early alpha) it will have a function like system.create_check('baseline/integrity/etc') returning a check object with check.validate_headers/row/table available. The amount of checks have been drastically reduces (only 2 core checks and 6 advanced).
So we will be able to integrate it into the validate processor like (maybe adding core checks baseline/integrity by default):
Flow(
validate(extra_checks=['baseline', 'integrity', ('blacklisted-value', {'fieldName': 'name'}), ...]
)
As an output, I think we just can add the errors list to a resource descriptor with a list of found errors.
Initially dataflows-goodtables integration is a @cschloer's idea but, for now, I don't see how we can friend them due to their streaming-nature without going to lower-level (the level of the individual check).
Overview
From
goodtabes@3(now early alpha) it will have a function likesystem.create_check('baseline/integrity/etc')returning a check object withcheck.validate_headers/row/tableavailable. The amount of checks have been drastically reduces (only 2 core checks and 6 advanced).So we will be able to integrate it into the
validateprocessor like (maybe adding core checks baseline/integrity by default):As an output, I think we just can add the
errorslist to a resource descriptor with a list of found errors.Initially
dataflows-goodtablesintegration is a @cschloer's idea but, for now, I don't see how we can friend them due to their streaming-nature without going to lower-level (the level of the individual check).