Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10000 commits
Select commit Hold shift + click to select a range
f843bd4
[Cleanup] Move normalization managers to ingest pipeline directory (R…
ageiduschek Sep 17, 2024
7513c6f
[raw data import] Small fixes (Recidiviz/recidiviz-data#33330)
emilyemilyemilyemilyemilyemily Sep 17, 2024
bdf61d9
[Docker] `chown` workdir in `asset-generation` build (Recidiviz/recid…
ohaibbq Sep 17, 2024
7699a2e
[US_AR] Modify county backup eligibility using updated location metad…
Sep 17, 2024
e57733e
[Ingest] Fully launch normalization in ingest in prod (Recidiviz/reci…
ageiduschek Sep 17, 2024
ccd7cce
[Cleanup] Delete should_run_normalization_in_ingest() feature gate (R…
ageiduschek Sep 17, 2024
06cabe3
Bump zod from 3.22.4 to 3.23.8 in /frontends/admin-panel (Recidiviz/r…
dependabot[bot] Sep 18, 2024
e499376
Bump @recidiviz/design-system from 4.2.0 to 5.7.0 in /frontends/proto…
dependabot[bot] Sep 18, 2024
45ac610
Bump vite from 4.5.3 to 4.5.5 in /nodejs/asset-generation (Recidiviz/…
dependabot[bot] Sep 18, 2024
74646c0
[Cleanup] Delete is_combined_ingest_and_normalization_launched_in_env…
ageiduschek Sep 18, 2024
81e029c
Bump @recidiviz/design-system from 2.7.0 to 5.7.0 in /frontends/admin…
dependabot[bot] Sep 18, 2024
44300dc
[Cleanup] Delete the legacy normalization pipeline (Recidiviz/recidiv…
ageiduschek Sep 18, 2024
c299bd7
Bump http-status from 1.7.3 to 1.7.4 in /frontends/admin-panel (Recid…
dependabot[bot] Sep 18, 2024
fc8be27
[Cleanup] Stop creating source tables for legacy normalization pipeli…
ageiduschek Sep 18, 2024
032b56b
[Cleanup] Delete more legacy normalization pipeline utils (Recidiviz/…
ageiduschek Sep 18, 2024
8c8e211
[Cleanup] Move remaining code out of recidiviz/pipelines/normalizatio…
ageiduschek Sep 18, 2024
55466c0
[Dataflow] Read from us_xx_normalized_state_new datasets in metric/su…
ageiduschek Sep 18, 2024
e5bb4f3
[US_TN] Adding infraction type to incident metadata (Recidiviz/recidi…
lydiamasri2 Sep 18, 2024
6b0115c
[Workflows] Pull employment status into the client_record.current_emp…
alexabatino Sep 18, 2024
1aab3f0
[admin panel] update dag flashing checklist conditional logic (Recidi…
ethan-oro Sep 18, 2024
2e61446
[raw data import dag] Enable raw data import dag in IA secondary for …
emilyemilyemilyemilyemilyemily Sep 18, 2024
0662386
[US_IX] Adding `RECIDIVIZ_REFERENCE_community_opportunities` in `stab…
santymendoza Sep 18, 2024
d73162e
[Cleanup] Rename tasks in calc DAG now that there are no normalizatio…
ageiduschek Sep 18, 2024
7f20a72
[admin panel] add cannot flash decision component (Recidiviz/recidivi…
ethan-oro Sep 18, 2024
d1ea610
[BigQuery] Adapt UnionAllBigQueryViewBuilder to work for source table…
ageiduschek Sep 18, 2024
6342f1e
[Admin Panel] Serve traffic to latest revision (Recidiviz/recidiviz-d…
ohaibbq Sep 18, 2024
0db69e8
[Prototypes] Add a README to the prototypes project. (Recidiviz/recid…
brandon-hills Sep 18, 2024
9ca9369
[admin panel] add new cancel reimport checklist (Recidiviz/recidiviz-…
ethan-oro Sep 18, 2024
738643b
Fix for NULL dates (Recidiviz/recidiviz-data#33464)
hugosr-r Sep 18, 2024
43a34f9
[admin panel] update legacy flash decision component (Recidiviz/recid…
ethan-oro Sep 18, 2024
60edf05
[Justice Counts] Migrate Row Errors to WorkbookStandardizer (Recidivi…
nichelle-hall Sep 18, 2024
be4b5a1
[raw data import] More small fixes (Recidiviz/recidiviz-data#33437)
emilyemilyemilyemilyemilyemily Sep 18, 2024
a0bac17
[US_ND] Program assignment work release sessions (Recidiviz/recidiviz…
justkunz Sep 18, 2024
409ad8a
[cloud functions] Remove old state specific cf service accounts (Reci…
emilyemilyemilyemilyemilyemily Sep 18, 2024
6f2a16d
Bump semiotic from 1.20.6 to 2.0.3 in /frontends/admin-panel (Recidiv…
dependabot[bot] Sep 18, 2024
f663249
[Workflows] MAU, WAU, Task Eligible aggregated metrics by Region (Rec…
morden35 Sep 18, 2024
833b9ad
[US_ME] Create overdue_from_discharge_request form (Recidiviz/recidiv…
hugosr-r Sep 18, 2024
59f765e
Bump @ant-design/icons from 4.8.1 to 5.5.0 in /frontends/admin-panel …
dependabot[bot] Sep 18, 2024
aa4d1ad
use only react-router-dom instead of react-router in the admin panel …
danawillow Sep 18, 2024
431acc5
[US_IA] Yamls for new tables (Recidiviz/recidiviz-data#33465)
lydiamasri2 Sep 19, 2024
51351d4
Revert "[Case Notes] Metric export case notes view" (Recidiviz/recidi…
roshcagra Sep 19, 2024
f312340
[Workflows] Apps Script Usage and Impact Text [10/n] (Recidiviz/recid…
morden35 Sep 19, 2024
5279cb5
[US_AZ] Bump validation thresholds (Recidiviz/recidiviz-data#33435)
elisegonzal Sep 19, 2024
f7d1c05
[Airflow] Remove metric pipelines dependency on update_normalized_sta…
ageiduschek Sep 19, 2024
e7219af
[Cleanup] Delete some old unused Justice Counts view code (Recidiviz/…
ageiduschek Sep 19, 2024
66591c4
[US_AZ] Validate against supervising officer staff_id instead of exte…
elisegonzal Sep 19, 2024
4f9b934
[Workflows] Apps Script Percent MAU (Recidiviz/recidiviz-data#33421)
morden35 Sep 19, 2024
b9be103
[BigQuery] Remove last reference to the state dataset in views (Recid…
ageiduschek Sep 19, 2024
b8e2cd7
[Airflow] Move update_state_dataset logic into BQ view graph (Recidiv…
ageiduschek Sep 19, 2024
022aa23
[raw data import dag] Update egt_vacations and egt_person yaml for DA…
emilyemilyemilyemilyemilyemily Sep 19, 2024
6e81508
[Prototypes] Add to the README a screenshot for staging/prod tokens a…
brandon-hills Sep 19, 2024
896d10e
[Airflow] Single dataflow pipelines group per state (Recidiviz/recidi…
ageiduschek Sep 19, 2024
8fde61c
[Case Note Search] Add relevance ordering field to case note results.…
brandon-hills Sep 19, 2024
2d82c24
[US_AZ] Ingest `state_sentence_status_snapshot` (Recidiviz/recidiviz-…
elisegonzal Sep 19, 2024
3259065
[Terraform] Upgrade Cloud Function runtimes from Python 3.8 to 3.11 (…
ageiduschek Sep 19, 2024
04c17c2
[Workflows] Apps Script Percent WAU (Recidiviz/recidiviz-data#33473)
morden35 Sep 19, 2024
586bd28
[raw data import] Fix raw file import table name is stable historical…
emilyemilyemilyemilyemilyemily Sep 20, 2024
af65c0e
[US_UT] Add base directory structure and raw data config for Utah (Re…
jkgerig Sep 20, 2024
36484fb
[Workflows][US_AR] Add Arkansas to workflows_enabled_states.yaml (Rec…
danielsmc Sep 20, 2024
7badd61
[US_PA] Quick fix for new state staff roster format (Recidiviz/recidi…
brendali121 Sep 20, 2024
6f6460c
[raw data dag] remove accidental ungating of `US_CO` (Recidiviz/recid…
ethan-oro Sep 20, 2024
41381e2
[Justice Counts] Move Defense `Open Cases` metric to Capacity and Cos…
nichelle-hall Sep 20, 2024
a7a0c44
[US_AZ] Refactor staff IDs (Recidiviz/recidiviz-data#33452)
elisegonzal Sep 20, 2024
a14f566
[Case Note Search] Fix exact search deduplication (Recidiviz/recidivi…
bpacker Sep 20, 2024
66e86ea
remove MI district transformation from current_staff query (Recidiviz…
danawillow Sep 20, 2024
49e2925
remove deprecated metrics from outliers configs (Recidiviz/recidiviz-…
danawillow Sep 20, 2024
20b1865
[US_IX] Adding fix for supervision period (Recidiviz/recidiviz-data#…
santymendoza Sep 20, 2024
65a82a2
[Bugfix] Add better log for non-ExternalIdEntity merging errors (Reci…
ageiduschek Sep 20, 2024
7b414c0
[raw data import] Fix update import status enum migration (Recidiviz/…
emilyemilyemilyemilyemilyemily Sep 20, 2024
e22e31a
Switch time periods to count backwards from end date instead of count…
JennaBellassai Sep 20, 2024
31d68c6
[US_ME] Change the supervision violation criteria (Recidiviz/recidivi…
hugosr-r Sep 20, 2024
5eb84bc
[pylint] update `too-many-positional-arguments` to 9, fix confusing a…
ethan-oro Sep 20, 2024
2071a1c
[SentencesV2] Add new sentence charges view (Recidiviz/recidiviz-data…
justkunz Sep 20, 2024
a83bca2
[Auth] Give BigQueryClient permissions to read from Google Sheets-bac…
ageiduschek Sep 20, 2024
537479a
[admin panel] add new proceed with flash checklist (Recidiviz/recidiv…
ethan-oro Sep 20, 2024
640e020
[raw data import dag testing] Add ignore_quotes: True to egt_person c…
emilyemilyemilyemilyemilyemily Sep 20, 2024
fc12c8c
[raw data import] Don't parse filenames that have timestamps using un…
emilyemilyemilyemilyemilyemily Sep 20, 2024
c19dde0
[admin panel] add resource lock page (Recidiviz/recidiviz-data#33521)
ethan-oro Sep 20, 2024
2c96978
Small changes to line_labels and group_into_other functionality (Reci…
seb2244 Sep 20, 2024
047c87d
Bump @ant-design/icons from 5.5.0 to 5.5.1 in /frontends/admin-panel …
dependabot[bot] Sep 23, 2024
3eb10a5
Bump csv-parse from 5.0.4 to 5.5.6 in /frontends/prototypes/admin (Re…
dependabot[bot] Sep 23, 2024
228a09c
[US_ND] Update datetime_parsers for columns in raw_data configs (Reci…
suhaasv-recidiviz Sep 23, 2024
d109340
[Justice Counts] Don't write to file to standardize Excel Workbook (…
nichelle-hall Sep 23, 2024
0a91611
[US_AZ] Refactor state_sentence_status_snapshot (Pre-migration only) …
elisegonzal Sep 23, 2024
616f7db
[Workflows] Apps Script Filter out Regions and 0 Values from Charts […
morden35 Sep 23, 2024
b46b4bc
[raw data import dag] Add header verification errors to coalesce_resu…
emilyemilyemilyemilyemilyemily Sep 23, 2024
d183be7
raise_header_verification_errors failure should not cause any downstr…
emilyemilyemilyemilyemilyemily Sep 23, 2024
fe30258
[US_OR] adding new enum value for admission and release reason IPs (R…
lydiamasri2 Sep 23, 2024
1f14e62
[US_TN] Editing caseload type mappings (Recidiviz/recidiviz-data#33548)
lydiamasri2 Sep 23, 2024
8f91df0
[Cost] Optimize tes_utils.py functions to query from task-specific vi…
ageiduschek Sep 23, 2024
dd47c31
[US_AZ] Add AZ to resident record view builder (Recidiviz/recidiviz-d…
elisegonzal Sep 23, 2024
fe0ed9e
[Airflow] Move update_normalized_state_dataset logic into BQ view gra…
ageiduschek Sep 23, 2024
288919d
remove unused dependencies from package.json files (Recidiviz/recidiv…
danawillow Sep 23, 2024
c7ec4a6
[Cleanup] Rename us_xx_normalized_state_new datasets back to us_xx_no…
ageiduschek Sep 23, 2024
a7cdd19
Bump styled-components from 6.0.0-beta.15 to 6.1.13 in /nodejs/asset-…
dependabot[bot] Sep 23, 2024
cac5b38
[US_TN] adding edge case to account for when someone has a custody ch…
gheidkamp11 Sep 23, 2024
c5606cf
[admin panel] resource locking action buttons (Recidiviz/recidiviz-da…
ethan-oro Sep 23, 2024
d2aa00e
[US_PA] Bump threshold for normalized_state_charge_missing_uniform_of…
brendali121 Sep 23, 2024
3422641
[US_IX] Adding new enum value for `StateIncarcerationPeriodReleaseRea…
santymendoza Sep 23, 2024
37477b1
[US_AZ] Add AZ staff information to resident_record and incarceration…
elisegonzal Sep 23, 2024
b9de47f
[US_AZ] Only store most recent home plan status (Recidiviz/recidiviz-…
elisegonzal Sep 23, 2024
c7bbf63
[US_ND] Add parole board review notes as side panel notes for ATP (Re…
hugosr-r Sep 23, 2024
6db95e2
Update asset-generation test after dependency bump (Recidiviz/recidiv…
Gina-Valderrama Sep 23, 2024
30d3323
[BigQuery] Clarify directionality of view address_overrides (input vs…
ageiduschek Sep 23, 2024
2500e81
[Insights] Add table helper text to OutliersMetricConfig (Recidiviz/r…
etiry Sep 24, 2024
ca5e3da
[US_AZ] Allow placeholder charges (Recidiviz/recidiviz-data#33614)
elisegonzal Sep 24, 2024
8cebdd1
[raw data import] add test for ignore_quotes and pre import normaliza…
ethan-oro Sep 24, 2024
b06345b
[US_TN] Prioritize InfractionType over IncidentType (Recidiviz/recidi…
DSharm Sep 24, 2024
e4258ee
[US_PA][Workflows] Remove probationers from candidate population (Rec…
seb2244 Sep 24, 2024
3c79b38
Add custody level info to TES session span (Recidiviz/recidiviz-data#…
DSharm Sep 24, 2024
604606d
[Justice Counts] Add consolidated race/ethnicity dictionary to API re…
nichelle-hall Sep 24, 2024
04f2f4e
[raw data import] Include microseconds in update_datetime for raw dat…
emilyemilyemilyemilyemilyemily Sep 24, 2024
dabbc20
[BigQuery] Delete unnecessary address_overrides_for_deployed_view_dat…
ageiduschek Sep 24, 2024
1276153
[US_PA] Revise processing for absconsion periods (Recidiviz/recidiviz…
brendali121 Sep 24, 2024
5f95ec5
[raw data import] Update temp file naming in normalization (Recidiviz…
emilyemilyemilyemilyemilyemily Sep 24, 2024
a3c8bab
[Justice Counts] Remove Extra `user_account` Assignment in `BulkUploa…
nichelle-hall Sep 24, 2024
a346bcc
[Justice Counts] Update CSG data pull script to use Metric Settings d…
brandon-hills Sep 24, 2024
0586b09
[Deploy] Fix `deploy_production.sh` unbound variable (Recidiviz/recid…
ohaibbq Sep 24, 2024
ed67f01
[US_PA][Workflows] Create criteria for not serving on sex offense pro…
seb2244 Sep 24, 2024
164a3a0
[US_AZ] Created criteria stubs & task queries for AZ TPR & DTP and mo…
shalindb Sep 24, 2024
86436d2
[Justice Counts] Audit Categories for JC Tier 1 Metrics (Recidiviz/re…
nichelle-hall Sep 24, 2024
ffe1e83
remove pathways export + DBs for NC, CO, MI, and ME (Recidiviz/recidi…
danawillow Sep 24, 2024
bb2d89a
[Insights][Bug] add earliest_person_assignment_date to officers respo…
jovergaag Sep 24, 2024
6a6f95e
Bump styled-components from 5.3.3 to 6.1.13 in /frontends/prototypes/…
dependabot[bot] Sep 24, 2024
316c524
Bump rollup from 3.29.4 to 3.29.5 in /nodejs/asset-generation (Recidi…
dependabot[bot] Sep 24, 2024
c06d3ba
[Justice Counts] Add an endpoint that gets data for just one agency. …
brandon-hills Sep 25, 2024
9dbcb92
[US_ND] Fix to registration requirements dates (Recidiviz/recidiviz-d…
hugosr-r Sep 25, 2024
6dc5ec4
[US_ND] Remove datetime parsers in downstream views (Recidiviz/recidi…
elisegonzal Sep 25, 2024
d182596
[Justice Counts] Turn off double-writes and delete agency-datapoint c…
brandon-hills Sep 25, 2024
bafe2e4
[US_ND] Hydrate incident severity (Recidiviz/recidiviz-data#33649)
elisegonzal Sep 25, 2024
ead1d4a
[Views] Move STATE and NORMALIZED_STATE dataset constants (Recidiviz/…
recidinick Sep 25, 2024
9652c9d
[Autogenerated] Update Pipfile.lock (Recidiviz/recidiviz-data#33569)
helperbot-recidiviz Sep 25, 2024
69cea18
[raw data import] Update stable historical counts validation to use u…
emilyemilyemilyemilyemilyemily Sep 25, 2024
26fbff4
[US_TX] Adding yaml files for `SupervisionSentence` , `SupervisionPer…
santymendoza Sep 25, 2024
179e8d9
[admin panel] enable cancel reimport for mixed raw data import dag en…
ethan-oro Sep 25, 2024
a8e3912
[Insights] use office name instead of unit name for CA supervision_un…
jovergaag Sep 25, 2024
872d064
[Testing] Add sqlglot helpers for enforcing selected columns and no s…
ageiduschek Sep 25, 2024
e867b27
[US_ND] Add criteria to exclude folks who have facility restrictions.…
hugosr-r Sep 25, 2024
b308d3f
[Workflows] Adjust Marked Ineligible Aggregated Metric to be a subset…
morden35 Sep 25, 2024
7d658aa
[Justice Counts] fix metric key typos in v2 agency dashboard API (Rec…
nichelle-hall Sep 25, 2024
32b02aa
[Justice Counts] add reporting_agency fields to dashboard response (R…
nichelle-hall Sep 26, 2024
19544e5
one more ND date parsing fix (Recidiviz/recidiviz-data#33703)
danawillow Sep 26, 2024
fb93aca
[raw data dag] only build branches for enabled pairs (Recidiviz/recid…
ethan-oro Sep 26, 2024
e2a6a14
[raw data import dag] make pre-import normalization failures blocking…
ethan-oro Sep 26, 2024
5b76ae3
[airflow] remove `branch_start` to override branch root trigger rules…
ethan-oro Sep 26, 2024
3101260
[Sentence Sessions] Aggregate sentence projected dates (Recidiviz/rec…
recidinick Sep 26, 2024
d7dcdf4
[Workflows] Adjust Aggregated Metrics min_date and rolling_period_len…
morden35 Sep 26, 2024
1cc390b
[US_ND] Choose highest incident severity level (Recidiviz/recidiviz-d…
elisegonzal Sep 26, 2024
3bee0a8
[US_AZ] Criteria: is US citizen or legal permanent resident without a…
hugosr-r Sep 26, 2024
682dafa
[US_MI][Workflows] Add `TEMPORARY_CUSTODY` to list of non active supe…
samanthahuff Sep 26, 2024
6e2568e
Migrate experiments dataset to experiments_metadata (Recidiviz/recidi…
hugosr-r Sep 26, 2024
3c50265
[Workflows] Fix Admin Panel Race Condition (Recidiviz/recidiviz-data#…
danielsmc Sep 26, 2024
5318e72
[Justice Counts] Add backend endpoints to only fetch user-agency asso…
brandon-hills Sep 26, 2024
dc4e395
[US_ND] Added not enrolled in programming criteria for MIN (Recidiviz…
hugosr-r Sep 26, 2024
2d47000
[raw data import] Add update_history field to raw_file_config (Recidi…
emilyemilyemilyemilyemilyemily Sep 26, 2024
0462cc3
Bump @google-cloud/firestore from 6.2.0 to 7.10.0 in /frontends/proto…
dependabot[bot] Sep 27, 2024
8e535c5
Bump rollup from 2.79.1 to 2.79.2 in /frontends/admin-panel (Recidivi…
dependabot[bot] Sep 27, 2024
24ec287
[Sentence Sessions] Aggregate state provided sentence group projected…
recidinick Sep 27, 2024
dcf0054
[Workflows] Adjust Impact Report Funnel Aggregated Metrics (Recidiviz…
morden35 Sep 27, 2024
ebdf984
[Workflows] Apps Script Usage and Impact Column Chart [12/n] (Recidiv…
morden35 Sep 27, 2024
ee83158
[US_AZ] Exclude person_caseload_location_sessions validations for no…
elisegonzal Sep 27, 2024
6afd2ae
[US_TX] Adding deleted flag column to staff and client data (Recidivi…
santymendoza Sep 27, 2024
6d25ee7
[US_AZ] Update raw data configs for sentencing related files (Recidiv…
recidinick Sep 27, 2024
5103231
[US_MI] Add value to ignore for ethnicity enum (Recidiviz/recidiviz-d…
brendali121 Sep 27, 2024
31f4b26
[US_ND][Workflows] Fix time zone bug (Recidiviz/recidiviz-data#33758)
samanthahuff Sep 27, 2024
18a9553
[US_PA] Ungate all SP related revisions to prod (Recidiviz/recidiviz-…
brendali121 Sep 27, 2024
bc17685
[raw data import] ungate oz primary (Recidiviz/recidiviz-data#33759)
ethan-oro Sep 27, 2024
8fda17c
[US_AZ] Hydrate state_task_deadline with DTP dates (Recidiviz/recidiv…
elisegonzal Sep 27, 2024
bdfd1e6
[raw data import dag] make chunking errors blocking (Recidiviz/recidi…
ethan-oro Sep 27, 2024
519aac4
[BigQuery] Add ability to merge two BigQueryAddressOverrides (Recidiv…
ageiduschek Sep 27, 2024
7d5d082
[BigQuery] Infra for splitting person_events, person_spans, etc into …
ageiduschek Sep 27, 2024
148f0fc
[BigQuery] Prep for allowing arbitrary source table overrides in sand…
ageiduschek Sep 27, 2024
bb59e22
[US_ND] Fix social security date parsing (Recidiviz/recidiviz-data#33…
ohaibbq Sep 27, 2024
d760aff
[US_ND] Review feedback on contact time date parsing (Recidiviz/recid…
ohaibbq Sep 27, 2024
1483780
Bump express from 4.20.0 to 4.21.0 in /frontends/prototypes/functions…
dependabot[bot] Sep 30, 2024
72cebaa
Bump styled-reset from 4.3.4 to 4.5.2 in /frontends/prototypes/protot…
dependabot[bot] Sep 30, 2024
1b99dc5
Bump rollup from 3.29.4 to 3.29.5 in /frontends/prototypes/prototype-…
dependabot[bot] Sep 30, 2024
68c7c57
Bump express from 4.20.0 to 4.21.0 in /nodejs/asset-generation (Recid…
dependabot[bot] Sep 30, 2024
b5bfcbc
[Justice Counts] Delete unused publisher endpoints. (Recidiviz/recidi…
brandon-hills Sep 30, 2024
3907639
[raw data import dag] make header validations blocking (Recidiviz/rec…
ethan-oro Sep 30, 2024
93379be
[Justice Counts] Convert Column Headers to String before Dropping Unn…
nichelle-hall Sep 30, 2024
3c6b331
[BigQuery] Fix determinism issue in aggregated_metrics queries (Recid…
ageiduschek Sep 30, 2024
5bee1a5
[BigQuery] client_record and TES query determinism fixes (Recidiviz/r…
ageiduschek Sep 30, 2024
11be726
[US_IX] Adding state sentence group (Recidiviz/recidiviz-data#30551)
santymendoza Sep 30, 2024
c2f25a4
set up caseload disaggregation for TN (Recidiviz/recidiviz-data#33553)
danawillow Sep 30, 2024
a1f812e
[raw data import] Add optional src to dst column name mapping to inse…
emilyemilyemilyemilyemilyemily Sep 30, 2024
027bf82
[Workflows] Adjust Impact Report Aggregated Metrics to always start o…
morden35 Oct 1, 2024
ef6298e
[Case Note Search] Make exact match search asynchronous. (Recidiviz/r…
brandon-hills Oct 1, 2024
bb3f240
Allow for small variations in timing in test_dag_perf_config_in_gcp_n…
emilyemilyemilyemilyemilyemily Oct 1, 2024
5a4b23f
[US_IA] adding new table yamls (Recidiviz/recidiviz-data#33824)
lydiamasri2 Oct 1, 2024
5fa8d0c
[BigQuery] Refactor aggregated_metrics assignment queries logic to it…
ageiduschek Oct 1, 2024
c679397
[Admin Panel] Bump hydrate cache job memory (Recidiviz/recidiviz-data…
ohaibbq Oct 1, 2024
635e424
[admin panel] update spin to only load while we fetch state codes (Re…
ethan-oro Oct 1, 2024
80dd253
[US_AR] Hydrate state_charge attempted and offense_type fields (Recid…
n-damiani Oct 1, 2024
872e48c
[admin panel] gray out legacy raw data status if dag is enabled (Reci…
ethan-oro Oct 1, 2024
14c5f78
[Autogenerated] Update Pipfile.lock (Recidiviz/recidiviz-data#33788)
helperbot-recidiviz Oct 1, 2024
8d19e6c
[admin panel] add dag trigger to ingest raw data page header (Recidiv…
ethan-oro Oct 1, 2024
0448a79
[US_IA][raw data] Add nonnull column validation exemption to PenaltyM…
emilyemilyemilyemilyemilyemily Oct 1, 2024
831cd84
[US_TN] Increasing violation upper bound window from 10 days to 1 mon…
lydiamasri2 Oct 1, 2024
7868c87
[US_AZ][Workflows] Criteria: Functional Literacy Enrollment, DTP Part…
shalindb Oct 1, 2024
69f5d62
[US_IX] Account for address changes in Tasks (Recidiviz/recidiviz-dat…
brendali121 Oct 2, 2024
5e544ed
[raw data import dag] wrap `TRIM` with `NULLIF` (Recidiviz/recidiviz-…
ethan-oro Oct 2, 2024
cceba74
[TES] Add state_code to task_to_completion_event (Recidiviz/recidiviz…
justkunz Oct 2, 2024
0531b5a
[US_AZ] Take some dates out of consideration (Recidiviz/recidiviz-dat…
elisegonzal Oct 2, 2024
b428af9
[US_AR] Exclude 0-day 90-day revocation periods (Recidiviz/recidiviz-…
Oct 2, 2024
623e5ed
[BigQuery] Expand _get_all_addresses_between_start_and_end_collection…
ageiduschek Oct 2, 2024
30ab25b
[Observations] Translate all EventQueryBuilder and SpanQueryBuilder t…
ageiduschek Oct 2, 2024
c27528a
[BigQuery] Eliminate all legacy EventQueryBuilder and SpanQueryBuilde…
ageiduschek Oct 2, 2024
80b33d0
[Observations] Query observation-specific views in SpanSelector and E…
ageiduschek Oct 2, 2024
7d6db45
[Observations] Remove all downstream view references to legacy person…
ageiduschek Oct 2, 2024
cbeb9bf
[US_TX] Adding `state_person` and `state_staff` (Recidiviz/recidiviz-…
santymendoza Oct 2, 2024
2399d6e
[raw data import] Only run import blocking validations on relevant co…
emilyemilyemilyemilyemilyemily Oct 2, 2024
4155db8
[raw data import] fix ungrouped file invalidation bug, missing invali…
ethan-oro Oct 2, 2024
49ef86a
[raw data import] Add check for config column missing from raw data f…
emilyemilyemilyemilyemilyemily Oct 2, 2024
8134c88
[raw data import dag] add `RawDataResourceLock` type (Recidiviz/recid…
ethan-oro Oct 2, 2024
8c9fa7b
[raw data import dag] adds explicit skip if we have no files to impor…
ethan-oro Oct 2, 2024
9b30022
Use state code for normalization dataset (Recidiviz/recidiviz-data#33…
not-a-doctor-stromberg Oct 2, 2024
a50e2cc
[US_AZ][Workflows] Additional Criteria and AZ Query Fragments file (R…
shalindb Oct 2, 2024
4bcb025
[Justice Counts] [Agency Dashboards] Save reporting_agency_id in `met…
nichelle-hall Oct 3, 2024
c4a4a85
[Workflows] Apps Script Chart Follow Ups (Recidiviz/recidiviz-data#33…
morden35 Oct 3, 2024
d7b2380
[Workflows] Apps Script Region Most Marked Ineligible [13/n] (Recidiv…
morden35 Oct 3, 2024
eabde02
[Workflows] Apps Script MAU, WAU by Workflow [14/n] (Recidiviz/recidi…
morden35 Oct 3, 2024
418fa81
[US_OR] Add enum mapping for StateSupervisionPeriodAdmissionReason PV…
n-damiani Oct 3, 2024
06b138c
[raw data import] Add support for ascii control characters (Recidiviz…
emilyemilyemilyemilyemilyemily Oct 3, 2024
787f898
[Infra] Adding plotting functions (Recidiviz/recidiviz-data#33117)
hugosr-r Oct 3, 2024
da248a8
[US_AZ] Refactor state task deadline (Recidiviz/recidiviz-data#33892)
elisegonzal Oct 3, 2024
04d98c3
[Justice Counts] User permission script should output # agencies, not…
lilidworkin Oct 4, 2024
9999fba
[Justice Counts] Increase timeout in `SpreadsheetInterface.save_inges…
nichelle-hall Oct 4, 2024
461e07b
[Justice Counts] Add Missing Metric Column Error (Recidiviz/recidiviz…
nichelle-hall Oct 4, 2024
25df435
[Justice Counts] Move agency_settings from Agency to Source in DB sch…
nichelle-hall Oct 4, 2024
260fa94
[US_MI] Add columns to ADH_SHPERSON raw data config (Recidiviz/recidi…
elisegonzal Oct 4, 2024
f4b0bdb
[US_AZ] Change opportunity names to match TES structure proposal (Rec…
hugosr-r Oct 4, 2024
83f1943
[US_AZ] Overdue for DTP and TPR (Recidiviz/recidiviz-data#33878)
hugosr-r Oct 4, 2024
628a143
[raw data import dag] Ungate MO + PA secondary for raw data dag (Reci…
emilyemilyemilyemilyemilyemily Oct 4, 2024
aaca59e
[raw data import] Fix lotr_demographics known_value (Recidiviz/recidi…
emilyemilyemilyemilyemilyemily Oct 4, 2024
962b2de
[US_IX/ND] Change all references to transfer_to_reentry_enter to tran…
hugosr-r Oct 4, 2024
fd82c63
[raw data import dag] change resource locking failure case, update da…
ethan-oro Oct 4, 2024
4666ff0
[raw data import dag] ensure we are not re-importing processed files,…
ethan-oro Oct 4, 2024
d3f79e9
[Monitoring] Report age of Airflow environments for alerting on unuse…
ohaibbq Oct 4, 2024
7b33d93
[BigQuery] Require nonnull project_id for all source table collection…
ageiduschek Oct 4, 2024
141a4c8
[Workflows] Preserve key order in response objects (Recidiviz/recidiv…
danielsmc Oct 4, 2024
969d541
[US_AZ] Edit conviction-related criteria (Recidiviz/recidiviz-data#33…
hugosr-r Oct 4, 2024
20cca5b
[BigQuery] Add support for arbitrary source table overrides in load_v…
ageiduschek Oct 4, 2024
cb30bfa
Use docker v2 command for pipenv run scripts (Recidiviz/recidiviz-dat…
mxosman Oct 4, 2024
8860284
[Viz Infra] Make secondary text in our plots a bit lighter than black…
hugosr-r Oct 4, 2024
2af1795
[Sentence Sessions] Create view of inferred group projected dates (Re…
recidinick Oct 4, 2024
55b96b6
[Airflow] Fix `entrypoint_executor` `ImportError` (Recidiviz/recidivi…
ohaibbq Oct 4, 2024
357a9c2
Bump moment-timezone from 0.5.45 to 0.5.46 in /frontends/admin-panel …
dependabot[bot] Oct 7, 2024
a3169c6
Bump antd from 4.24.15 to 4.24.16 in /frontends/admin-panel (Recidivi…
dependabot[bot] Oct 7, 2024
d269e30
Bump date-fns from 3.6.0 to 4.1.0 in /frontends/admin-panel
dependabot[bot] Oct 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
3 changes: 3 additions & 0 deletions .bandit
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[bandit]
targets: recidiviz
skips: B101,B311,B404,B410,B608
2 changes: 2 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[run]
omit = lib/*,recidiviz/tests/*,recidiviz/utils/data_migration.py
38 changes: 38 additions & 0 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Dev Container

This directory contains configurations for creating a [Dev Container](https://code.visualstudio.com/docs/devcontainers/containers) for pulse-data. Dev containers are an optional way to set up a VSCode environment, where development occurs inside a container (still on the user's machine) that is configured in a standardized way.

## Pre-reqs
1. If you do not already have gcloud configured on your computer, log in to gcloud by running the following two commands:

```bash
gcloud auth login # Gets credentials to interact with services via the CLI
gcloud auth application-default login # Gets credentials which will be automatically read by our client libraries
```
1. Verify that the directory `~/.config/gcloud` exists. It should have been created when you authenticated to Gcloud

## How to use

Dependencies:

1. [VSCode](https://code.visualstudio.com/download)
1. Docker as per [VSCode requirements](https://code.visualstudio.com/docs/devcontainers/containers#_system-requirements)
1. Install the Docker VSCode extension (`ms-azuretools.vscode-docker`)
1. Install the Microsoft Dev Containers VSCode extension (`ms-vscode-remote.remote-containers`)

To use:

1. Open the pulse-data repository inside of VSCode
1. You may see a pop-up that says you can open this workspace inside a dev container. If you do, click on "Reopen In Container". If not, open the command palette (cmd-shift-P on a mac) and select "Dev Containers: Reopen In Container".
1. Be patient; it takes some time to create the container and install all dependencies. You can click on "Starting Dev Container (show log)" (or "Dev Containers: Show Container Log" from the command palette) to see progress. When it's done, you'll see the message "Done. Press any key to close the terminal." in the logs.
1. [Optional] Open a new terminal window by choosing "Terminal: Create New Terminal" from the command palette.

After the first time you build the container, you'll also need to do the following:

1. Select the PipEnv python interpreter by choosing "Python: Select Interpreter" from the command palette and selecting the one that says PipEnv next to it

Limitations:

1. Running services via docker (e.g. admin panel, case triage backend, etc.) does not work (yet)
1. The container might not build on windows due to the way we share gcloud credentials between your machine and the container.
1. As of June 2023, this is new functionality for Recidiviz there are likely to be some rough edges! If you use this and encounter any weirdness, please post in Slack in #eng or file a ticket!
79 changes: 79 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/python
{
"name": "Python 3",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/python:0-3.11",
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/java:1": {},
"ghcr.io/devcontainers-contrib/features/pipenv:2": {},
"ghcr.io/devcontainers-contrib/features/postgres-asdf:1": {},
"ghcr.io/dhoeric/features/google-cloud-cli:1": {},
"ghcr.io/devcontainers/features/common-utils:2": {
"configureZshAsDefaultShell": true
},
// gdal and cmake are needed to install certain pip dependencies.
// micromamba lets us install these easily from conda with a small footprint.
"ghcr.io/mamba-org/devcontainer-features/micromamba:1": {
"channels": "conda-forge",
"packages": "gdal cmake"
},
"docker-in-docker": {
"version": "latest",
"moby": true,
"dockerDashComposeVersion": "v1",
"_comment": "VS Code runs as a Docker container. So if we want to build/run Docker, we need to support running Docker within Docker"
},
"terraform": {
// Note: this verison number should be kept in sync with the ones in Dockerfile,
// .github/workflows/ci.yml, recidiviz/tools/deploy/terraform/terraform.tf, and
// recidiviz/tools/deploy/deploy_helpers.sh
"version": "1.7.0",
"tflint": "latest",
"terragrunt": "latest",
"_comment": "Terraform is needed if we want to run deployments from within VS Code Containers"
}
},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "sh .devcontainer/postCreate.sh",
// The git directory isn't ready until the container has been started, so put `git config`
// commands into a separate script.
"postStartCommand": "sh .devcontainer/postStart.sh ${containerWorkspaceFolder}",

"customizations": {
"vscode": {
"extensions": [
"DavidAnson.vscode-markdownlint",
"dbaeumer.vscode-eslint",
"eamodio.gitlens",
"esbenp.prettier-vscode",
"hashicorp.terraform",
"mechatroner.rainbow-csv",
"ms-python.black-formatter",
"ms-python.python",
"samuelcolvin.jinjahtml",
"stkb.rewrap"
]
}
},

// Mount local .config/gcloud directory to get access to existing gcloud credentials. This way,
// the user doesn't have to reauthenticate with gcloud every time the container is rebuilt.
// This won't work with GH codespaces, but based on https://github.com/Recidiviz/pulse-data/pull/21325#issuecomment-1574052908
// it sounds like that's a non-goal.
// TODO(#21470): Figure out how to make this work for Windows (configs are in %APPDATA%\gcloud)
"mounts": [
"source=${localEnv:HOME}/.config/gcloud,target=/home/vscode/.config/gcloud,type=bind,consistency=cached"
]

// Configure tool-specific properties.
// "customizations": {},

// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
14 changes: 14 additions & 0 deletions .devcontainer/postCreate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env bash

# Add hooks
cp -R githooks/ .git/hooks/
chmod +x .git/hooks/*

pipenv sync --dev

# Install pre-commit hooks
pipenv run pre-commit install --overwrite

# Clear out cache which could have been populated by running tests under a different system
# architecture (such as outside the container). See https://nicolasbouliane.com/blog/importmismatcherror-python-fix
find . \( -name '__pycache__' -or -name '*.pyc' \) -delete
7 changes: 7 additions & 0 deletions .devcontainer/postStart.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

# See https://www.kenmuse.com/blog/avoiding-dubious-ownership-in-dev-containers/
git config --global --add safe.directory "$1"

# Set git blame to ignore noisy commits
git config --replace-all blame.ignoreRevsFile .git-blame-ignore-revs
224 changes: 224 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
# Ignoring Dockle Alerts

# use latest version: Ignoring because the container, that was Dockle-tested, was built with
# the latest image, and has nothing to do with the Dockerfile

DKL-DI-0006

# storing credentials in env var/files
# Ignoring because in prior conversations with Aurora, there are no actual secrets being exposed in the
# Dockerfile
CIS-DI-0010

# use non-root user
# Ignoring because there is no actual security risk here and no need to change the user
CIS-DI-0001

## -- Keep everything below in sync with .gcloudignore -- ##

# git
.git/

# Gitbook
docs/

## -- Keep everything below in sync with .gitignore -- ##

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Emacs files
*~
\#*\#
/.emacs.desktop
/.emacs.desktop.lock
*.elc
auto-save-list
tramp
.\#*

# Vim backup files
*.sw[op]

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib64/
parts/
.pytest_cache/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
lib/

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
coverage/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/

# Translations
*.mo
*.pot

# Django stuff:
*.log
.static_storage/
.media/
local_settings.py

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# R
.Rhistory

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# PyCharm files
.idea/

# VSCode files
.vscode/

# Environments
.env
.env.*
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
.firebase/
.runtimeconfig.json
service_account.json

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/

# Local development
local.yaml # keep in case any developers still have them
secrets.yaml # keep in case any developers still have them
/queue.yaml

#pytest
.pytest_cache/

# Mac
.DS_Store

# Terraform
**/.terraform/*
*.tfstate
*.tfstate.*
*.tfvars
# Used to store temporary files
**/.tfout/*

# Typescript
**/node_modules
.yalc
yalc.lock
*.tsbuildinfo

# Stackdriver Traces
traces

# Local emulator files
recidiviz/local/**/*

# Local Case Triage files
recidiviz/case_triage/local/**/*

# Local Justice Counts files
recidiviz/justice_counts/control_panel/local/**/*

# Local Asset Generation files
nodejs/asset-generation/local

# ESLint Cache
.eslintcache

# Deployment artifacts
recidiviz/tools/deploy/dist/
recidiviz/tools/deploy/log/

# Binaries
recidiviz/tools/postgres/log/
recidiviz/tools/postgres/vendor/

# Test fixtures
recidiviz/tests/big_query/fixtures/

# Tool logs
logs/
get-pip.py
*.code-workspace

# Local Justice Counts files
recidiviz/justice_counts/control_panel/local/**/*

# Terraform cache
recidiviz/tools/deploy/terraform/.terraform

Loading