Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10000 commits
Select commit Hold shift + click to select a range
49ef86a
[raw data import] Add check for config column missing from raw data f…
emilyemilyemilyemilyemilyemily Oct 2, 2024
8134c88
[raw data import dag] add `RawDataResourceLock` type (Recidiviz/recid…
ethan-oro Oct 2, 2024
8c9fa7b
[raw data import dag] adds explicit skip if we have no files to impor…
ethan-oro Oct 2, 2024
9b30022
Use state code for normalization dataset (Recidiviz/recidiviz-data#33…
not-a-doctor-stromberg Oct 2, 2024
a50e2cc
[US_AZ][Workflows] Additional Criteria and AZ Query Fragments file (R…
shalindb Oct 2, 2024
4bcb025
[Justice Counts] [Agency Dashboards] Save reporting_agency_id in `met…
nichelle-hall Oct 3, 2024
c4a4a85
[Workflows] Apps Script Chart Follow Ups (Recidiviz/recidiviz-data#33…
morden35 Oct 3, 2024
d7b2380
[Workflows] Apps Script Region Most Marked Ineligible [13/n] (Recidiv…
morden35 Oct 3, 2024
eabde02
[Workflows] Apps Script MAU, WAU by Workflow [14/n] (Recidiviz/recidi…
morden35 Oct 3, 2024
418fa81
[US_OR] Add enum mapping for StateSupervisionPeriodAdmissionReason PV…
n-damiani Oct 3, 2024
06b138c
[raw data import] Add support for ascii control characters (Recidiviz…
emilyemilyemilyemilyemilyemily Oct 3, 2024
787f898
[Infra] Adding plotting functions (Recidiviz/recidiviz-data#33117)
hugosr-r Oct 3, 2024
da248a8
[US_AZ] Refactor state task deadline (Recidiviz/recidiviz-data#33892)
elisegonzal Oct 3, 2024
04d98c3
[Justice Counts] User permission script should output # agencies, not…
lilidworkin Oct 4, 2024
9999fba
[Justice Counts] Increase timeout in `SpreadsheetInterface.save_inges…
nichelle-hall Oct 4, 2024
461e07b
[Justice Counts] Add Missing Metric Column Error (Recidiviz/recidiviz…
nichelle-hall Oct 4, 2024
25df435
[Justice Counts] Move agency_settings from Agency to Source in DB sch…
nichelle-hall Oct 4, 2024
260fa94
[US_MI] Add columns to ADH_SHPERSON raw data config (Recidiviz/recidi…
elisegonzal Oct 4, 2024
f4b0bdb
[US_AZ] Change opportunity names to match TES structure proposal (Rec…
hugosr-r Oct 4, 2024
83f1943
[US_AZ] Overdue for DTP and TPR (Recidiviz/recidiviz-data#33878)
hugosr-r Oct 4, 2024
628a143
[raw data import dag] Ungate MO + PA secondary for raw data dag (Reci…
emilyemilyemilyemilyemilyemily Oct 4, 2024
aaca59e
[raw data import] Fix lotr_demographics known_value (Recidiviz/recidi…
emilyemilyemilyemilyemilyemily Oct 4, 2024
962b2de
[US_IX/ND] Change all references to transfer_to_reentry_enter to tran…
hugosr-r Oct 4, 2024
fd82c63
[raw data import dag] change resource locking failure case, update da…
ethan-oro Oct 4, 2024
4666ff0
[raw data import dag] ensure we are not re-importing processed files,…
ethan-oro Oct 4, 2024
d3f79e9
[Monitoring] Report age of Airflow environments for alerting on unuse…
ohaibbq Oct 4, 2024
7b33d93
[BigQuery] Require nonnull project_id for all source table collection…
ageiduschek Oct 4, 2024
141a4c8
[Workflows] Preserve key order in response objects (Recidiviz/recidiv…
danielsmc Oct 4, 2024
969d541
[US_AZ] Edit conviction-related criteria (Recidiviz/recidiviz-data#33…
hugosr-r Oct 4, 2024
20cca5b
[BigQuery] Add support for arbitrary source table overrides in load_v…
ageiduschek Oct 4, 2024
cb30bfa
Use docker v2 command for pipenv run scripts (Recidiviz/recidiviz-dat…
mxosman Oct 4, 2024
8860284
[Viz Infra] Make secondary text in our plots a bit lighter than black…
hugosr-r Oct 4, 2024
2af1795
[Sentence Sessions] Create view of inferred group projected dates (Re…
recidinick Oct 4, 2024
55b96b6
[Airflow] Fix `entrypoint_executor` `ImportError` (Recidiviz/recidivi…
ohaibbq Oct 4, 2024
357a9c2
Bump moment-timezone from 0.5.45 to 0.5.46 in /frontends/admin-panel …
dependabot[bot] Oct 7, 2024
a3169c6
Bump antd from 4.24.15 to 4.24.16 in /frontends/admin-panel (Recidivi…
dependabot[bot] Oct 7, 2024
63b77e0
[US_NE] Adding new potential parole discharge date table (Recidiviz/r…
lydiamasri2 Oct 7, 2024
2cd5d17
[US_AZ] Update StateSentence entities with consecutive sentences, sen…
recidinick Oct 7, 2024
14b96bb
[Workflows] Impact Report Aggregated Metrics Adjust Rolling Period (R…
morden35 Oct 7, 2024
1ed43e1
[BigQuery] Add state_code column to all validation views (Recidiviz/r…
ageiduschek Oct 7, 2024
2bea26e
[Autogenerated] Update Pipfile.lock (Recidiviz/recidiviz-data#33998)
helperbot-recidiviz Oct 7, 2024
5c3cdb0
[US_AZ] Add details to TPR/DTP opportunity records (Recidiviz/recidiv…
hugosr-r Oct 7, 2024
bce3af9
[US_ND] Add no_escape_in_current_incarceration criteria (Recidiviz/re…
hugosr-r Oct 7, 2024
42b0058
[admin panel] fix dag links (Recidiviz/recidiviz-data#33977)
ethan-oro Oct 7, 2024
cff7609
[raw data import] add max update_datetime validation (Recidiviz/recid…
ethan-oro Oct 7, 2024
55bc3ab
[calc dag] update max `update_datetime` generator behind gates (Recid…
ethan-oro Oct 7, 2024
34b2362
[raw data import] Update transformation query to use temp table col n…
emilyemilyemilyemilyemilyemily Oct 7, 2024
0fb2153
Deletes the old google_bigquery_data_transfer_config for `experiment_…
hugosr-r Oct 7, 2024
f121eb9
[raw data import] add `raise_registration_errors` task (Recidiviz/rec…
ethan-oro Oct 7, 2024
b719d2e
[calc dag] update is flash in progress checkj (Recidiviz/recidiviz-da…
ethan-oro Oct 7, 2024
79a069c
[raw data import] Fix header error filtering (Recidiviz/recidiviz-dat…
emilyemilyemilyemilyemilyemily Oct 7, 2024
8a1ae62
Adds less granular LOCATION unit of analysis (Recidiviz/recidiviz-dat…
mayukas Oct 7, 2024
7f1d12c
[US_OR] Adding new enum value for Supervision termination reason (Rec…
lydiamasri2 Oct 7, 2024
cd37c1f
[Justice Counts] Reporting Agency Id Backfill Script (Recidiviz/recid…
nichelle-hall Oct 7, 2024
695aba0
[raw data import] Strip header row from first normalized chunk if pre…
emilyemilyemilyemilyemilyemily Oct 7, 2024
63d2f0c
[US_AZ] Ingest `state_sentence_length` (Recidiviz/recidiviz-data#33309)
elisegonzal Oct 7, 2024
b0e31c7
[Workflows][US_AZ] Add AZ to workflows exports/ETL (Recidiviz/recidiv…
ryanrecidiviz Oct 8, 2024
3c5a469
[US_ME] Prio 401 for sentence release dates (Recidiviz/recidiviz-data…
not-a-doctor-stromberg Oct 8, 2024
c0af8a0
[US_AZ] Filter out malformed eligibility dates in ingest (Recidiviz/r…
elisegonzal Oct 8, 2024
8576358
[Insights] Add action strategy copy fields to db (Recidiviz/recidiviz…
etiry Oct 8, 2024
3ba3ad6
[TES][US_OR] Support sentences almost eligible for earned discharge (…
justkunz Oct 8, 2024
e375c1e
[BigQuery] Add state_code column to new sentences views (Recidiviz/re…
ageiduschek Oct 8, 2024
a29f8f5
[US_AR] Ingest incarceration incident severity (Recidiviz/recidiviz-d…
n-damiani Oct 8, 2024
3cbcb2f
[US_MO][raw data import] Add update_history for FOCTEST_ORAS_ASSESSME…
emilyemilyemilyemilyemilyemily Oct 8, 2024
de5f68b
[BigQuery] Helper for getting StateCode from BigQueryAddress (Recidiv…
ageiduschek Oct 8, 2024
87670f5
update vitals queries to include people who are no longer on supervis…
danawillow Oct 8, 2024
b6f2f53
[BigQuery] Assert all views have a state_code column (Recidiviz/recid…
ageiduschek Oct 8, 2024
d73af99
[Workflows] Impact Report Aggregated Metrics Adjust Shift Daily and W…
morden35 Oct 8, 2024
e739df0
[US_AR] Specify AR external ID in all joins (Recidiviz/recidiviz-data…
Oct 8, 2024
23de251
[US_AZ] Creating AZ Projected Dates Analyst Data View (Recidiviz/reci…
shalindb Oct 8, 2024
70b7980
[Insights][Zero grants] Add task completion metrics to insights metri…
Gina-Valderrama Oct 8, 2024
917a807
[Insights][Zero Grants] Add prop_period_critical_caseload metric to i…
Gina-Valderrama Oct 8, 2024
43d539f
[Sentences V2] Add state_sentence_length.parole_eligibility_date_exte…
elisegonzal Oct 8, 2024
a8985df
Adds workflows primary user login event (Recidiviz/recidiviz-data#34012)
mayukas Oct 9, 2024
bdc5dd0
[raw data import] Update header error filtering logic (Recidiviz/reci…
emilyemilyemilyemilyemilyemily Oct 9, 2024
4d23c54
[BigQuery] Disallow fully qualified column references in BQ views (Re…
ageiduschek Oct 9, 2024
a690a8b
[Utils] Add attr_validators for set fields (Recidiviz/recidiviz-data#…
ageiduschek Oct 9, 2024
8514717
[US_AZ] Added new criteria to exclude people who have already been de…
hugosr-r Oct 9, 2024
f8f0459
[raw data import dag] Don't clean up temp resources for failed files …
emilyemilyemilyemilyemilyemily Oct 9, 2024
3c5d872
[raw data import] Add default expiration of 7 days to files in temp f…
emilyemilyemilyemilyemilyemily Oct 9, 2024
dcfe5d3
[BigQuery] Add BigQueryAddressFormatter and BigQueryAddressFormatterP…
ageiduschek Oct 9, 2024
d1514e6
[US_IX] [Sentencing] Fixing bug where city name was displayed as coun…
santymendoza Oct 9, 2024
74cb571
[BigQuery] Add state_code_filter to load_views_to_sandbox (Recidiviz/…
ageiduschek Oct 9, 2024
ff830d4
Adds sessionized views of client/record archive for surfaceable clien…
mayukas Oct 9, 2024
3332707
[Aggregated Metrics] Remove one last reference to legacy person_event…
ageiduschek Oct 9, 2024
fe3ec73
[US_ND] Create criteria query to exclude clients who are a month away…
hugosr-r Oct 9, 2024
2f63358
return internal server errors as JSON in the case triage BE (Recidivi…
danawillow Oct 9, 2024
5c45f5d
[BigQuery] Remove buggy load_views_to_sandbox UNION ALL view filterin…
ageiduschek Oct 9, 2024
9511f69
[BigQuery] Delete legacy person_events, person_spans, etc views (Reci…
ageiduschek Oct 9, 2024
2e021d0
[US_MO][raw data import] Add deleted columns in raw config (Recidiviz…
emilyemilyemilyemilyemilyemily Oct 9, 2024
d15a6a0
Change location_id to location_name in time_dependent_person_attribut…
JennaBellassai Oct 10, 2024
09ecfd2
Add Recommended X-Content-Type-Options Header (Recidiviz/recidiviz-da…
mxosman Oct 10, 2024
f7808de
[US_MO][raw data import] Add additional values to MO enum fields (Rec…
emilyemilyemilyemilyemilyemily Oct 10, 2024
e4bdf2e
[Workflows] Add prod preview apps to case_triage CORS allowlist (Reci…
danielsmc Oct 10, 2024
6e3fe6c
[US_ND] Bug bash fixes to opp record (Recidiviz/recidiviz-data#34109)
hugosr-r Oct 10, 2024
f352df5
[Sentencing] Add violent and sex offense info to offenses (Recidiviz/…
roshcagra Oct 10, 2024
b10d3a2
[US_AZ] Hydrate `state_sentence_group_length` with ruling sentence d…
elisegonzal Oct 10, 2024
f9806e1
[US_AZ] Fix date filtering bug (Recidiviz/recidiviz-data#34102)
elisegonzal Oct 10, 2024
7365701
[raw data import dag] Increase concurrent task count (Recidiviz/recid…
emilyemilyemilyemilyemilyemily Oct 10, 2024
b9320ec
[BigQuery] Assorted person_event deprecation TODOs (Recidiviz/recidiv…
ageiduschek Oct 11, 2024
1be1118
Create insights primary user registration sessions (Recidiviz/recidiv…
mayukas Oct 11, 2024
9b32ad0
replace COALESCE with an ARRAY_LENGTH check for the roles column in p…
danawillow Oct 11, 2024
52767e5
[US_TX] Adding functionality for duplicate external id types (Recidiv…
santymendoza Oct 11, 2024
7723b0e
[US_AZ] Documentation (Recidiviz/recidiviz-data#34124)
elisegonzal Oct 11, 2024
da9418a
[US_ME] Validation Bump (Recidiviz/recidiviz-data#34064)
not-a-doctor-stromberg Oct 11, 2024
dfa569b
[Performance] Require clustering for all UnionAllBigQueryViewBuilder …
ageiduschek Oct 11, 2024
58a79c9
[Justice Counts] Add geographic metadata about an agency to v2 dashbo…
nichelle-hall Oct 11, 2024
01abb06
[Admin Panel][Insights] Add action strategy copy to `AddConfigForm` (…
etiry Oct 11, 2024
876786e
[US_NE] Supervision tables (Recidiviz/recidiviz-data#34123)
lydiamasri2 Oct 11, 2024
42dae79
[Justice Counts] Add README and comment for duplicated TIG elements (…
mxosman Oct 11, 2024
4789642
[US_TX] Adding col to ClientData and adding raw data config for Asses…
santymendoza Oct 11, 2024
0454b6a
[Bugfix] Fix crash in export_csg_files script (Recidiviz/recidiviz-da…
ageiduschek Oct 11, 2024
a89e4ca
[Cleanup] Simply CloudSQL -> BQ refresh query builder (Recidiviz/reci…
ageiduschek Oct 11, 2024
89fb01c
Bump @recidiviz/design-system from 5.7.0 to 5.8.0 in /frontends/admin…
dependabot[bot] Oct 14, 2024
9d46a3d
Bump @recidiviz/design-system from 5.7.0 to 5.8.0 in /frontends/proto…
dependabot[bot] Oct 14, 2024
c8390aa
Bump @recidiviz/design-system from 5.7.0 to 5.8.0 in /nodejs/asset-ge…
dependabot[bot] Oct 14, 2024
0be99b9
Remove duplicates from session_location_names (Recidiviz/recidiviz-da…
justkunz Oct 15, 2024
43c60dc
Bump jwks-rsa from 2.0.5 to 2.1.5 in /frontends/prototypes/functions …
dependabot[bot] Oct 15, 2024
e3c1e4a
Bump express-rate-limit from 6.7.0 to 6.11.2 in /nodejs/asset-generat…
dependabot[bot] Oct 15, 2024
e6c9497
Bump firebase-admin from 9.12.0 to 10.3.0 in /frontends/prototypes/fu…
dependabot[bot] Oct 15, 2024
d165784
Bump react-error-boundary from 3.1.4 to 4.0.13 in /frontends/prototyp…
dependabot[bot] Oct 15, 2024
c27e85b
Bump express-rate-limit from 6.4.0 to 6.11.2 in /frontends/prototypes…
dependabot[bot] Oct 15, 2024
0578fb3
Bump firebase from 9.6.7 to 10.14.1 in /frontends/prototypes/prototyp…
dependabot[bot] Oct 15, 2024
287921a
Bump express-jwt and @types/express-jwt in /frontends/prototypes/func…
dependabot[bot] Oct 15, 2024
85b3ea4
[raw data import] Add test case for leading new lines in raw files (R…
emilyemilyemilyemilyemilyemily Oct 15, 2024
79b7c20
[US_MO][raw data import] Update raw file configs to pass pre-import v…
emilyemilyemilyemilyemilyemily Oct 15, 2024
c492ddf
[Workflows] Add support for opp id to ETL delegate (Recidiviz/recidiv…
alexabatino Oct 15, 2024
3a2b4ea
[US_NE] adding column descriptions for ones used in Ingest views (Rec…
lydiamasri2 Oct 15, 2024
e1faf7a
[Tools] Move sandbox pipeline Docker build to a separate function (Re…
ageiduschek Oct 15, 2024
5a1c8a4
[US_IX] Adding RECIDIVIZ_REFERENCE_release_to_crc_facility_mappings t…
santymendoza Oct 15, 2024
142788b
[Autogenerated] Update Pipfile.lock (Recidiviz/recidiviz-data#34164)
helperbot-recidiviz Oct 15, 2024
6b95da8
[Cleanup] Clean up some unused schema utils (Recidiviz/recidiviz-data…
ageiduschek Oct 15, 2024
2ab3188
[US_PA][Workflows] Create additional criteria for limited supervision…
seb2244 Oct 16, 2024
989d4da
[US_PA][Workflows] Update offense code references in administrative s…
seb2244 Oct 16, 2024
6b84b03
Fixes non-determinism in workflows_primary_user_product_roster_archiv…
mayukas Oct 16, 2024
a84211b
[US_IX] Adding new raw data config files for the new CLD tool (Recidi…
santymendoza Oct 16, 2024
c84a75a
[Justice Counts] V0 Agency Dashboard Homepage Response (Recidiviz/rec…
nichelle-hall Oct 16, 2024
f73e019
Change `GENERAL_INCARCERATION_POPULATION_FACILITY_FILTER` to `INCARCE…
DSharm Oct 16, 2024
d34a821
[raw data import] Modify NONNULL_VALUES to only run on primary key co…
emilyemilyemilyemilyemilyemily Oct 16, 2024
1a65e1c
[US_TN] Create TES stubs & TODOs for SDS (Recidiviz/recidiviz-data#34…
mhilderbran Oct 16, 2024
5b0fd6a
[Workflows][US_AZ] Prefix export filenames with state code (Recidiviz…
Catacola Oct 16, 2024
a380fc9
[Insights][Admin Panel][Bug] Add WAF exception rule for outliers conf…
jovergaag Oct 16, 2024
8ad05dd
Support large file exports with Metric Export (Recidiviz/recidiviz-da…
roshcagra Oct 16, 2024
aacaecf
[Sentencing] Add frequency to charge template (Recidiviz/recidiviz-da…
roshcagra Oct 16, 2024
9360ac7
[US_AZ] Self-Improvement program removals (Recidiviz/recidiviz-data#3…
hugosr-r Oct 16, 2024
2c03a75
[Insights][Admin Panel][Bug] update priority on WAF rule (Recidiviz/r…
jovergaag Oct 16, 2024
0fe3f9f
Adds new unit of observation, spans and population selectors for WORK…
mayukas Oct 16, 2024
21f6fb3
[Tools] Refactor load_views_to_sandbox prompting / auto mode output (…
ageiduschek Oct 16, 2024
61e1fc7
[Workflows] Apps Script MAU, WAU by Workflow and Location Chart [15/n…
morden35 Oct 16, 2024
43231d4
[US_AZ] Use same external ID type between resident record and eligibi…
elisegonzal Oct 16, 2024
9a14e5d
[Insights] Add `action_strategy_copy` to `configurations` endpoint re…
jovergaag Oct 16, 2024
545bc34
Adds WARRANT_STATUS to JUSTICE_INVOLVED MetricPopulationType (Recidiv…
mayukas Oct 17, 2024
17ac7aa
[sftp] move retries back to tasks (Recidiviz/recidiviz-data#34173)
ethan-oro Oct 17, 2024
147767b
[US_ID][JII] Update Text Copy for D1/D5/D7 Launch (Recidiviz/recidivi…
morden35 Oct 17, 2024
e8c7b79
[Tools] Add script for running an end-to-end data pipeline sandbox (R…
ageiduschek Oct 17, 2024
5eba54a
[Tools] Delete legacy full_sandbox_pipeline.sh script (Recidiviz/reci…
ageiduschek Oct 17, 2024
579a7d0
[Export] Fix CSV / Headerless CSV output formats (Recidiviz/recidiviz…
ohaibbq Oct 17, 2024
ca6e348
[US_NE] Sentences and Charges (Recidiviz/recidiviz-data#33873)
lydiamasri2 Oct 18, 2024
4283b9e
[boundary finder] add single, unescaped quote finding logic (Recidivi…
ethan-oro Oct 18, 2024
eac809b
[airflow] Add class for handling KubernetesPodOperator task output (R…
emilyemilyemilyemilyemilyemily Oct 18, 2024
640b23f
[Workflows][US_OR] Add opportunity config for sentence-level EDIS (Re…
danielsmc Oct 18, 2024
4f8aa1d
[US_IX] Fixed missing columns in yaml (Recidiviz/recidiviz-data#34297)
santymendoza Oct 18, 2024
ae3a8f7
[US_AR] Update YAML for statute reference table to include columns fo…
n-damiani Oct 18, 2024
178dd97
[US_TN] SORV and CSLR violations (Recidiviz/recidiviz-data#34156)
lydiamasri2 Oct 18, 2024
4cef256
return a more useful error message when the roster upload does not co…
danawillow Oct 18, 2024
59ec7e1
[US_TX] Hydrating Supervision contacts (Recidiviz/recidiviz-data#33810)
santymendoza Oct 18, 2024
08c0e25
[US_AZ] Update source of sentence expiration date (Recidiviz/recidivi…
elisegonzal Oct 18, 2024
7648ed0
[Insights][Auth] Have (`workflows` | `outliers`)_authorization.py acc…
nojibe Oct 18, 2024
5dad113
[BigQuery] Stop deploying SECONDARY raw data latest views (Recidiviz/…
ageiduschek Oct 18, 2024
3a963e8
[boundary finder] add support for newlines in single quotes (Recidivi…
ethan-oro Oct 18, 2024
5e36f09
Bump express from 4.21.0 to 4.21.1 in /frontends/prototypes/functions…
dependabot[bot] Oct 21, 2024
3f289d2
Bump react-error-boundary from 4.0.13 to 4.1.2 in /frontends/prototyp…
dependabot[bot] Oct 21, 2024
394cd62
Bump express from 4.21.0 to 4.21.1 in /nodejs/asset-generation (Recid…
dependabot[bot] Oct 21, 2024
61d7103
Bump mobx from 6.7.0 to 6.13.5 in /frontends/prototypes/prototype-app…
dependabot[bot] Oct 21, 2024
5b209b2
Bump d3-array and @types/d3-array in /nodejs/asset-generation (Recidi…
dependabot[bot] Oct 21, 2024
bdd5749
Bump @recidiviz/auth from 1.0.1 to 1.1.0 in /frontends/prototypes/pro…
dependabot[bot] Oct 21, 2024
195603b
Adds workflows user logins metrics to Looker (Recidiviz/recidiviz-dat…
mayukas Oct 21, 2024
6a459e9
[Justice Counts] Surface `No agency exists with the id` error to FE f…
nichelle-hall Oct 21, 2024
3184ebd
[Justice Counts] Check if agency has county before getting county cod…
nichelle-hall Oct 21, 2024
68c6839
[US_ID][JII] Add D5 additional contact info (Recidiviz/recidiviz-data…
morden35 Oct 21, 2024
02345d5
[US_IX] Fixing `person` view to accomadate for multiple email address…
santymendoza Oct 21, 2024
35f80ad
[US_TX] Hydrating assessments entity (Recidiviz/recidiviz-data#34233)
santymendoza Oct 21, 2024
8f1043d
[TES][US_AZ] Update completion events for TPR and DTP (Recidiviz/reci…
justkunz Oct 21, 2024
1b7be9a
[US_ND] Silly mistake (Recidiviz/recidiviz-data#34352)
hugosr-r Oct 21, 2024
3be8907
[Autogenerated] Update Pipfile.lock (Recidiviz/recidiviz-data#34361)
helperbot-recidiviz Oct 21, 2024
9c0db4e
[US_AZ] Ingest ACCAT data into state_assessment (Recidiviz/recidiviz-…
elisegonzal Oct 21, 2024
86726a6
[US_AZ] Improve state task deadline ingest logic (Recidiviz/recidiviz…
elisegonzal Oct 21, 2024
f4a6947
[US_ND][US_AZ] Bump validation thresholds (Recidiviz/recidiviz-data#3…
elisegonzal Oct 21, 2024
eff052f
[US_ME] Add edge case for validation (Recidiviz/recidiviz-data#34364)
hugosr-r Oct 21, 2024
003d410
Creates boilerplate Insights impact lookml generator (Recidiviz/recid…
mayukas Oct 21, 2024
45f47a5
Creates common tables for auth0 login and signup events (Recidiviz/re…
mayukas Oct 21, 2024
4e2aa63
[Insights] Add `supervision_clients` for "US_IX" (Recidiviz/recidiviz…
jovergaag Oct 21, 2024
6227903
[BQ] Bump aggregated metric view runtime thresholds, again (Recidiviz…
ryanrecidiviz Oct 22, 2024
d1f3337
Adds new span and events for INSIGHTS_USER unit of observation (Recid…
mayukas Oct 22, 2024
3863941
[boundary finder] add support for multiple quotes (Recidiviz/recidivi…
ethan-oro Oct 22, 2024
d3566b0
[Insights][Actions] Add markdown preview on `ActionStrategyFormItem` …
etiry Oct 22, 2024
4bdb493
[Deploy] Fix bug getting changes to release candidate merged since ta…
ageiduschek Oct 22, 2024
9c85181
[raw data import] move `CsvChunkBoundary` into `raw_data_types` (Reci…
ethan-oro Oct 22, 2024
1b8dfa0
[sftp] `state_code` as `StateCode` (Recidiviz/recidiviz-data#34349)
ethan-oro Oct 22, 2024
3b907e6
Uses product roster archive for reidentifying users (Recidiviz/recidi…
mayukas Oct 22, 2024
e4c149c
[US_PA][Workflows] Map admin supervision to transfer to limited type …
seb2244 Oct 22, 2024
ff6207c
[Justice Counts] Computationally Fetch `available_metrics` and `avail…
nichelle-hall Oct 22, 2024
a1b854b
[raw data import dag] Read/write KPO task output from/to GCS (Recidiv…
emilyemilyemilyemilyemilyemily Oct 22, 2024
5acf2b8
Add ACCAT to assessment_score_sessions (Recidiviz/recidiviz-data#34397)
hugosr-r Oct 22, 2024
eccbfd9
[Case Notes Search] Add metric export of case notes with the `include…
roshcagra Oct 23, 2024
2d97c00
[US_AZ] Refactor Home Plan preprocessed view (Recidiviz/recidiviz-dat…
elisegonzal Oct 23, 2024
289a12f
[Case Notes Search] Incude page_token as an optional field in the req…
roshcagra Oct 23, 2024
cd9d0c4
[Workflows] Add is_surfaceable to person impact funnel (Recidiviz/rec…
justkunz Oct 23, 2024
2cada7d
[TES][US_AZ] Create early release overdue/not overdue events (Recidiv…
justkunz Oct 23, 2024
0295746
[US_TN] Create completion event for SDS (Recidiviz/recidiviz-data#34290)
mhilderbran Oct 23, 2024
dd3c5de
[US_ND] Add raw data config for PSI data (Recidiviz/recidiviz-data#34…
elisegonzal Oct 23, 2024
4994c6b
Adds is_fully_launched flag to launch metadata yaml (Recidiviz/recidi…
mayukas Oct 23, 2024
4cebcaf
[US_AR] Limit state_staff ingest to relevant roles (Recidiviz/recidiv…
n-damiani Oct 23, 2024
f19c1fa
[US_IX][PSI] Adding functionality for NULL names in PSI case record (…
santymendoza Oct 23, 2024
3785890
[US_AZ] Create content for at_least_24_months_since_last_csed criteri…
hugosr-r Oct 23, 2024
5b53ce1
[Workflows] Refactor Impact Report Aggregated Metrics (Recidiviz/reci…
morden35 Oct 23, 2024
35b5e84
[raw data import] Supply encoding and delimiter to BQ load job (Recid…
emilyemilyemilyemilyemilyemily Oct 23, 2024
75fd5ad
[sftp] state code str fix (Recidiviz/recidiviz-data#34444)
ethan-oro Oct 23, 2024
6e598ca
[admin panel] always let flash checklist user click Mark Done (Recidi…
ethan-oro Oct 23, 2024
247209d
[airflow] add `branch_by_bool` task (Recidiviz/recidiviz-data#34357)
ethan-oro Oct 23, 2024
1e3ab30
[US_AZ] CSED criteria fix (Recidiviz/recidiviz-data#34448)
hugosr-r Oct 23, 2024
5fc92c2
[US_MI][Workflows] Update start dates for add and warden review count…
samanthahuff Oct 24, 2024
be20e5b
[US_AZ] Add raw data config for AZ_DOC_SC_EXCEPTION (Recidiviz/recidi…
elisegonzal Oct 24, 2024
e4ced44
[Sentence Sessions] Validation for Sentence Inferred Group Projected …
recidinick Oct 24, 2024
e79362d
[raw data import] Convert non standard chars to space character in tr…
emilyemilyemilyemilyemilyemily Oct 24, 2024
b788c66
[US_AZ] Pull flat sentence status into state sentence metadata (Recid…
elisegonzal Oct 24, 2024
b6da344
[Airflow] increase raw data file chunking pod resources (Recidiviz/re…
emilyemilyemilyemilyemilyemily Oct 24, 2024
f8883df
[US_AR] Task eligibility criteria - 309 & Work Release (Recidiviz/rec…
Oct 24, 2024
12f7be6
[sftp] another state_code bug fix (Recidiviz/recidiviz-data#34459)
ethan-oro Oct 24, 2024
4e4a9f2
[raw data] refine operations db resource description (Recidiviz/recid…
ethan-oro Oct 24, 2024
e0db634
[US_AZ] Update StateSentenceStatusSnapshot Recidiviz/recidiviz-data#3…
elisegonzal Oct 24, 2024
2b906c3
[Insights][Zero Grants] Add zero grants opportunity list to insights …
Gina-Valderrama Oct 24, 2024
7b33831
[US_PA] Add column update_history for dbo_Senrec (Recidiviz/recidiviz…
emilyemilyemilyemilyemilyemily Oct 24, 2024
07012ea
[TES][US_OR] Update sentence reasons fields (Recidiviz/recidiviz-data…
justkunz Oct 24, 2024
28404e9
[US_MI][Workflows] Remove residents in detention from scc (Recidiviz/…
samanthahuff Oct 24, 2024
8958f50
[US_CA] [US_ME] Unignore some validations (Recidiviz/recidiviz-data#3…
not-a-doctor-stromberg Oct 24, 2024
c96ebed
[Terraform] Add dataset to house outputs from compare_raw_data.py (Re…
emilyemilyemilyemilyemilyemily Oct 24, 2024
c2e7ad6
split up the insights staff queries + don't apply exclusions for supe…
danawillow Oct 24, 2024
729b5aa
[US_AZ] Fix bug in state sentence status snapshot & state sentence vi…
elisegonzal Oct 25, 2024
e0db0c9
[US_AZ] Changes to Projected Dates View & AE work (Recidiviz/recidivi…
shalindb Oct 25, 2024
f389f5a
[Case Note Search] Update Case Note search views (Recidiviz/recidiviz…
roshcagra Oct 25, 2024
fed74c1
Fix escape violations overlapping spans (Recidiviz/recidiviz-data#34524)
Oct 25, 2024
4f8a8a9
[US_AR] Fix Class I validation (Recidiviz/recidiviz-data#34526)
Oct 25, 2024
c2142eb
[Airflow] Fix KPO output bucket name (Recidiviz/recidiviz-data#34494)
emilyemilyemilyemilyemilyemily Oct 25, 2024
b237191
[Case Notes Search] Vertex Search Engine in Terraform (Recidiviz/reci…
roshcagra Oct 25, 2024
62c0d7c
[US_AR] Ingest staff missing valid roles but present in other tables …
n-damiani Oct 25, 2024
343b396
Bump firebase-functions from 3.24.1 to 4.9.0 in /frontends/prototypes…
dependabot[bot] Oct 25, 2024
692b704
Revert "[Case Notes Search] Vertex Search Engine in Terraform (Recidi…
ryanrecidiviz Oct 25, 2024
77fe9d1
Undo destroying Discovery Engine API Terraform resource (Recidiviz/re…
ryanrecidiviz Oct 26, 2024
e40fbdc
Bump react-router-dom in /frontends/prototypes/prototype-app
dependabot[bot] Nov 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
3 changes: 3 additions & 0 deletions .bandit
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[bandit]
targets: recidiviz
skips: B101,B311,B404,B410,B608
2 changes: 2 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[run]
omit = lib/*,recidiviz/tests/*,recidiviz/utils/data_migration.py
38 changes: 38 additions & 0 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Dev Container

This directory contains configurations for creating a [Dev Container](https://code.visualstudio.com/docs/devcontainers/containers) for pulse-data. Dev containers are an optional way to set up a VSCode environment, where development occurs inside a container (still on the user's machine) that is configured in a standardized way.

## Pre-reqs
1. If you do not already have gcloud configured on your computer, log in to gcloud by running the following two commands:

```bash
gcloud auth login # Gets credentials to interact with services via the CLI
gcloud auth application-default login # Gets credentials which will be automatically read by our client libraries
```
1. Verify that the directory `~/.config/gcloud` exists. It should have been created when you authenticated to Gcloud

## How to use

Dependencies:

1. [VSCode](https://code.visualstudio.com/download)
1. Docker as per [VSCode requirements](https://code.visualstudio.com/docs/devcontainers/containers#_system-requirements)
1. Install the Docker VSCode extension (`ms-azuretools.vscode-docker`)
1. Install the Microsoft Dev Containers VSCode extension (`ms-vscode-remote.remote-containers`)

To use:

1. Open the pulse-data repository inside of VSCode
1. You may see a pop-up that says you can open this workspace inside a dev container. If you do, click on "Reopen In Container". If not, open the command palette (cmd-shift-P on a mac) and select "Dev Containers: Reopen In Container".
1. Be patient; it takes some time to create the container and install all dependencies. You can click on "Starting Dev Container (show log)" (or "Dev Containers: Show Container Log" from the command palette) to see progress. When it's done, you'll see the message "Done. Press any key to close the terminal." in the logs.
1. [Optional] Open a new terminal window by choosing "Terminal: Create New Terminal" from the command palette.

After the first time you build the container, you'll also need to do the following:

1. Select the PipEnv python interpreter by choosing "Python: Select Interpreter" from the command palette and selecting the one that says PipEnv next to it

Limitations:

1. Running services via docker (e.g. admin panel, case triage backend, etc.) does not work (yet)
1. The container might not build on windows due to the way we share gcloud credentials between your machine and the container.
1. As of June 2023, this is new functionality for Recidiviz there are likely to be some rough edges! If you use this and encounter any weirdness, please post in Slack in #eng or file a ticket!
79 changes: 79 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/python
{
"name": "Python 3",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/python:0-3.11",
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/java:1": {},
"ghcr.io/devcontainers-contrib/features/pipenv:2": {},
"ghcr.io/devcontainers-contrib/features/postgres-asdf:1": {},
"ghcr.io/dhoeric/features/google-cloud-cli:1": {},
"ghcr.io/devcontainers/features/common-utils:2": {
"configureZshAsDefaultShell": true
},
// gdal and cmake are needed to install certain pip dependencies.
// micromamba lets us install these easily from conda with a small footprint.
"ghcr.io/mamba-org/devcontainer-features/micromamba:1": {
"channels": "conda-forge",
"packages": "gdal cmake"
},
"docker-in-docker": {
"version": "latest",
"moby": true,
"dockerDashComposeVersion": "v1",
"_comment": "VS Code runs as a Docker container. So if we want to build/run Docker, we need to support running Docker within Docker"
},
"terraform": {
// Note: this verison number should be kept in sync with the ones in Dockerfile,
// .github/workflows/ci.yml, recidiviz/tools/deploy/terraform/terraform.tf, and
// recidiviz/tools/deploy/deploy_helpers.sh
"version": "1.7.0",
"tflint": "latest",
"terragrunt": "latest",
"_comment": "Terraform is needed if we want to run deployments from within VS Code Containers"
}
},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "sh .devcontainer/postCreate.sh",
// The git directory isn't ready until the container has been started, so put `git config`
// commands into a separate script.
"postStartCommand": "sh .devcontainer/postStart.sh ${containerWorkspaceFolder}",

"customizations": {
"vscode": {
"extensions": [
"DavidAnson.vscode-markdownlint",
"dbaeumer.vscode-eslint",
"eamodio.gitlens",
"esbenp.prettier-vscode",
"hashicorp.terraform",
"mechatroner.rainbow-csv",
"ms-python.black-formatter",
"ms-python.python",
"samuelcolvin.jinjahtml",
"stkb.rewrap"
]
}
},

// Mount local .config/gcloud directory to get access to existing gcloud credentials. This way,
// the user doesn't have to reauthenticate with gcloud every time the container is rebuilt.
// This won't work with GH codespaces, but based on https://github.com/Recidiviz/pulse-data/pull/21325#issuecomment-1574052908
// it sounds like that's a non-goal.
// TODO(#21470): Figure out how to make this work for Windows (configs are in %APPDATA%\gcloud)
"mounts": [
"source=${localEnv:HOME}/.config/gcloud,target=/home/vscode/.config/gcloud,type=bind,consistency=cached"
]

// Configure tool-specific properties.
// "customizations": {},

// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
14 changes: 14 additions & 0 deletions .devcontainer/postCreate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env bash

# Add hooks
cp -R githooks/ .git/hooks/
chmod +x .git/hooks/*

pipenv sync --dev

# Install pre-commit hooks
pipenv run pre-commit install --overwrite

# Clear out cache which could have been populated by running tests under a different system
# architecture (such as outside the container). See https://nicolasbouliane.com/blog/importmismatcherror-python-fix
find . \( -name '__pycache__' -or -name '*.pyc' \) -delete
7 changes: 7 additions & 0 deletions .devcontainer/postStart.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

# See https://www.kenmuse.com/blog/avoiding-dubious-ownership-in-dev-containers/
git config --global --add safe.directory "$1"

# Set git blame to ignore noisy commits
git config --replace-all blame.ignoreRevsFile .git-blame-ignore-revs
224 changes: 224 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
# Ignoring Dockle Alerts

# use latest version: Ignoring because the container, that was Dockle-tested, was built with
# the latest image, and has nothing to do with the Dockerfile

DKL-DI-0006

# storing credentials in env var/files
# Ignoring because in prior conversations with Aurora, there are no actual secrets being exposed in the
# Dockerfile
CIS-DI-0010

# use non-root user
# Ignoring because there is no actual security risk here and no need to change the user
CIS-DI-0001

## -- Keep everything below in sync with .gcloudignore -- ##

# git
.git/

# Gitbook
docs/

## -- Keep everything below in sync with .gitignore -- ##

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Emacs files
*~
\#*\#
/.emacs.desktop
/.emacs.desktop.lock
*.elc
auto-save-list
tramp
.\#*

# Vim backup files
*.sw[op]

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib64/
parts/
.pytest_cache/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
lib/

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
coverage/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/

# Translations
*.mo
*.pot

# Django stuff:
*.log
.static_storage/
.media/
local_settings.py

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# R
.Rhistory

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# PyCharm files
.idea/

# VSCode files
.vscode/

# Environments
.env
.env.*
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
.firebase/
.runtimeconfig.json
service_account.json

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/

# Local development
local.yaml # keep in case any developers still have them
secrets.yaml # keep in case any developers still have them
/queue.yaml

#pytest
.pytest_cache/

# Mac
.DS_Store

# Terraform
**/.terraform/*
*.tfstate
*.tfstate.*
*.tfvars
# Used to store temporary files
**/.tfout/*

# Typescript
**/node_modules
.yalc
yalc.lock
*.tsbuildinfo

# Stackdriver Traces
traces

# Local emulator files
recidiviz/local/**/*

# Local Case Triage files
recidiviz/case_triage/local/**/*

# Local Justice Counts files
recidiviz/justice_counts/control_panel/local/**/*

# Local Asset Generation files
nodejs/asset-generation/local

# ESLint Cache
.eslintcache

# Deployment artifacts
recidiviz/tools/deploy/dist/
recidiviz/tools/deploy/log/

# Binaries
recidiviz/tools/postgres/log/
recidiviz/tools/postgres/vendor/

# Test fixtures
recidiviz/tests/big_query/fixtures/

# Tool logs
logs/
get-pip.py
*.code-workspace

# Local Justice Counts files
recidiviz/justice_counts/control_panel/local/**/*

# Terraform cache
recidiviz/tools/deploy/terraform/.terraform

Loading