Author: atediarjo@gmail.com, joe.brew@gmail.com
R package for testing metadata and interact with AWS
To fully run this package you will require access to DataBrew AWS Accounts via SSO (please contact atediarjo@gmail.com for access). To configure AWS environment in your RStudio, run this command:
tryCatch({
logger::log_info('Attempt AWS login')
# login to AWS - this will be bypassed if executed in CI/CD environment
cloudbrewr::aws_login(
role_name = 'cloudbrewr-aws-role',
profile_name = 'cloudbrewr-aws-role')
}, error = function(e){
logger::log_error('AWS Login Failed')
stop(e$message)
})Installation can be done through Github installation:
devtools::install_github('databrew/dataqualitybrewr')
- Check zip file submission
check_results <- check(input = 'PATH/TO/HECON/ZIPFILE', func = check_healthecon)- After running checks, you will get an output of a
check_resultobject mapping. The object mapping will contain the list of all available errors.
To check errors from unit test:
check_results$err_dfTo check how your files will be stored in AWS (recursively):
check_results$output_map- Once all test have passed and resolved, parse
check_resultobject mapping to promote function
promote(check_results, store_historical = TRUE)- After promoting the
check_resultobject, data will be stored in AWS for tracking / validation