userGuide.Rmd

---
title: "Colorado EnviroScreen Data Processing"
author: "dan carver"
date: '2022-06-10'
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

# Colorado EnviroScreen

Colorado EnviroScreen is a data processing code base with a public facing shiny application that highlights the spatial variability of specific Environmental Health and Environmental Justice challenges throughout Colorado. This document is intended to guide users in the generation of the Colorado EnviroScreen dataset that supports functions as the input to the shiny applications.

This content was developed by the Geospatial Centroid at Colorado State University under the direction and support of the Rojos Pulbic Health Lab at CSU, Institute for the Built Environment at CSU, and the Colorado Department of Public Health and Environment.

Questions and comments can be directed to Dan Carver at carverd@colostate.edu


## Vision

The code base for generating the Colorado EnviroScreen score dataset utilizes a hierarchical workflow that matches general scoring presented on the application
- EnviroScreen Score
  - Group Component Scores
    - Component Scores
      - individual Indicator Scores

This allowed for a module approach to the development. The primary motivator for this process was to allow for the quick regeneration of all elements if a single input feature was changed. This was accomplished by writing out intermediates at ever step in the process. All data processing functions have an version and overwrite parameter with give the user the specificity needed to control how much data is regenerated or compiled from the existing intermediate file. In short, the code base is going to included what's already there unless told otherwise. This keeps it fast and flexible.

As with any codebase, this will require some maintenance and adjustment going forward. We hope that what is provided is a viable starting place for a long-lived and successful project.

## About this Document

This document has x major sections

- Setting up the local environment
- running the processing code
- details on specific functions
- code standards


## Setting up the local environment

The data processing code is stored in the Geospatial Centroid's GitHub, and needs to be downloaded or cloned to render in the local environment.
https://github.com/GeospatialCentroid/COEnviroScreen_dataProcessing

We've utilized a .rproj file to organize this work, and we recommend that you do the same.

With the repository on your local device, open the R Project file and then the `0_main.R` file. Barring special use cases, all data connected to the EnviroScreen score and shiny applications can be generated through edits within this script alone.

### Input datasets

The input data sets for the enviroscreen project are store in the Geospatial Centroid's GitHub. Please download this content. The following section will describe how to bring it into the R project while maintaining all relative links.

Current the following indicators are store as 7zip files within the folder. They will need to be extracted before they can be processed.

- "data/input/EJScreen/EJScreen.7z"
- "data/input/floodPlains/floodPlains.7z"
- "data/input/noise/CONUS_L50dBA_sumDay_exi.7z"
- "data/input/spatialLayers/justice40.7z"




## Running the processing code : 0_main

### loading packages/functions

We utilize the `pacman` library to load all the required packages. The helper function `loadFunctions()` can be called to load the 40 plus functions associated with the codebase.

### Project file folder structure.

**R** : The folder for all functions connected to the data processing code base. All functions at this level are associated with the production of a score value.

**R/Utilities**: These helper functions are generally called from other functions and are unlikely to require any adjustment of input parameters. These functions also do not produce any indicator score values.

### Create file folder structure

As the codebase is writing out numerous intermediate files, we've opted to generate a standardized file folder structure that accounts for inputs, outputs, and shiny specific features. This folder structure also enables the development of version specific outputs. 	

Run the `createFolderStructure()` function. This will create a data folder in your R project directory. Within the data folder are subfolders for input, output, and shinyContent.

### Populate the input folder

Copy the extracted data from the COEnviroScreen_dataInputs repository into the `data/input` folder. There can not be any intermediate folders between the input folder and the specific indicator folders. For example you relative path the the asthma dataset should look like:

`"~data/input/asthma/Asthma_Hospitalization_Rate_(Census_Tracts).csv"`


### Gather Geometry Layers

The geometry features for Colorado Counties, Census Tracts, and Census Block Groups are used in the EnviroScreen data calculation. To ensure the consistency of these features the layers are pulled directly from the `tigris` R library and stored as geojsons within the `output/spatialLayers` folder.    

Run the `getGeometryLayers()` function to gather these layers.

### Census API key

Data pertaining to multiple indicators is gather directly from the 2019 American Community Survey. This content is accessed via the `tidycensus` library which requires a unique census API key. The census API key can be obtained [here](https://api.census.gov/data/key_signup.html). There is currently no costs associated with utilizing the Census API key.

Run the `tidycensus::census_api_key(key = "your key")` function to ensure you are able to pull dataset using the `tidycensus` package later in the data processing.


### Generate the score values

The `processData` function is used to generate all score values connected with the calculation of the EnviroScreen score. The function itself is the start of the hierarchical score calcualtions. This is important because the input parameters used for this function (processingLevel, version, overwrite) are passed to all other processing functions, unless specific altered within a separate function call.

Diagram for the three level of functions within the `processData` function.

*add image from powerpoint*

 As the bulk of the data processing is performed as part of this `processData` function this process can take a while to run. We recommend running the process with the "county" processingLevel first.

 This can be done with the following code
 ```r
 processData(processingLevel="county",
             version = version,
             overwrite = FALSE)
 ```
You can run all geographies utilizing a for loop

```r
geoms <- c("county","censusTract","censusBlockGroup")

for(i in geoms){
  print(i)
  processData(processingLevel=i,
              version = version,
              overwrite = FALSE)
}
```

### Compiling the data for shiny

The shiny applications require six rds files to run. This includes 5 map elements and a spatial object that combines all the calculated values from each of the three geographies. Note this function will not run unless scores have been created for all three geographies at the current version.

Diagram below shows the two levels of functions within the `shinyData` function.

*add image from powerpoint.

All outputs from this process will saved to the shinyContent folder



## Individual Function Details

The are three general classes of functions within the code base; processing, compiling, and helper.

- **Processing functions** are performing some type of analysis and returning new dataset.
  - ex: "asthma.R"
  - users are expected to adjust the input parameters which forcing specific rewrites or adjusting the input file path
- **compiling functions** calling processing functions or pulling data from existing source and combining datasets.
  - ex: "environmentalExposures.R"
  - users are expected to adjust input parameters when forcing specific rewrites or changing version of data generated.
- **helper functions** are chunks of processing code that are used by multiple processing functions. They are stored in "r/utils" and generally have set input parameters.
  - ex: bufferObjects
  - users are not expected to directly edit input parameters.


A short description of each function will be provided below.

Index for function descriptions

**processing functions**
- acs.r
- asthma.r
- drinkingWater.r
- drought.r
- ej_screen.r
- floodplain.r
- getCoal.r
- getDI.r
- getJustice40.r
- getOilGas.r
- getRural.r
- haps.r
- heartDisease.r
- heatDays.r
- houseBurden.r
- lifeExpectency.r
- lowBirthWeight.r
- mining.r
- noise.r
- otherHaps.r
- ozone.r
- placesData.r
- pm25.r
- proxyOilGas.r
- surfaceWater.r
- wildfire.r


**compiling functions**
- climate.r
- environmentalExposures.r
- environmentalEffects.r
- finalComponentScore.r
- getShinyData.r
- joinDataFrames.r
- processData.r
- sensitivePopulations.r
- socioEconomic.r



**helper functions**
- createFolderStructure.r
- getGeometryLayers.r
- gm_mean.r
- loadFunctions.r
- normalizeVector.r
- patternLayer.r
- setSpatialData.r


**Template**
**Name**: file name

**Function Name**: how to call the function in R

**Description**: What does the function do

**Inputs**: list of input parameters

**Outputs**: description of output values

**helper function**: list of helper functions used

**Updating Data**: What to consider when updating the input dataset

**Other Considerations**: Other notes and thoughts about the process



### Processing Functions


#### standard inputs
Not all functions have all these inputs but most do.

**filePath**: Location of the file within the input folder. For functions in which there are more then one input features, the filepaths are defined directly within the function.

**geometry**: Spatial object representing the geography of interest. Generally has some role in the processing of the dataset.

**processingLevel**: character vector describing the geography of interest ("county", "censusTract", "censusBlockGroup"). Generally only used to constructing file paths.

**version**: character value used to represent the version of the enviroscreen dataset. A new version will force all data processings scripts to regenerate content.

**overwrite**: Binary TRUE/FALSE to define if a existing file should be overwriten. IF FALSE the existing file will be read in and returned as a dataframe.  

---

**Name**: acs.r

**Function Name**: `asc()`

**Description**: Pulls 2019 ACS data utilizing the `tidycensus` library and calculated specific measures.

**Inputs**:
- processingLevel
- version
- overwrite

**Outputs**:
- dataframe with GEOID, eight indicator values, and a total population value used in generation of data for shiny applications.
-  saves dataframe as csv to

**helper function**: NA

**Updating Data**: 2019 ACS data is not expected to change. An update to 2020 census data is dependent on availability of health metrics.

**Other Considerations**: requires a census api key.

---

**Name**: asthma.r

**Function Name**: `getAsthma()`

**Description**: Reads in census tract level asthma hospitalization rate data and generalizes to specific geography.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite

**Outputs**:
- dataframe with GEOID and adjusted asthama hospitalization rate
-  saves dataframe as csv to

**helper function**: NA

**Updating Data**: ensure column headers remain consistent between new and old version of the dataset.

**Other Considerations**: NA

---

**Name**: drinkingWater.r

**Function Name**: `getDrinkingWater()`

**Description**:Multiple step evaluation relating number and longevity of drinking water violations to effected populations at the county level.    

**Inputs**:
- geometry
- processingLevel
- version
- overwrite

**Outputs**:
- dataframe with GEOID and adjusted asthama hospitalization rate
-  saves dataframe as csv to

**helper function**: NA

**Updating Data**: Three input datasets, some sensitivity to column names and `read.csv()` calls.

**Other Considerations**: Multiple input datasets are defined within the codebase. Fairly complex processing effort due to the number of steps.  

---

**Name**: drought.r

**Function Name**: `getDrought()`

**Description**: Summarize an area adjusted drought score for all counties and applies it to smaller geographies.   

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite

**Outputs**:
- dataframe with GEOID and drought measures
-  saves dataframe as csv to

**helper function**: NA

**Updating Data**: Easy to update do to standard data source. There are specific filter calls to look at the 2016:2020 years in the code. Those would need to be changes if input data was from a new time period.  

**Other Considerations**: NA   

---

**Name**: ej_screen.r

**Function Name**: `ej_screen()`

**Description**: Pulls specific data from the larger EJScreen dataset and generalizes to requested geography.    

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite

**Outputs**:
- dataframe with GEOID and drought measures
-  saves dataframe as csv to

**helper function**: NA

**Updating Data**: Easy to update do to standard data source.

**Other Considerations**: NA   

---

**Name**: floodplain.r

**Function Name**: `getFloodplain()`

**Description**: Calculated the total area of a geography within an active floodplain.   

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite

**Outputs**:
- dataframe with GEOID and drought measures
- saves dataframe as csv

**helper function**: NA

**Updating Data**: Unlikely to be able to use the same source for future updates. The processing code is not complicated but don't expect to be able utilize it as is when new floodplain feature is found.

**Other Considerations**: The floodplain input dataset is a complex polygon and therefore all the intersections require to generate results tend to be slow. This is not a script you want to rerun unless you have to.

---

- getCoal.r

**Name**: getCoal.r

**Function Name**: `getCoal()`

**Description**: Generates a RDS file for visualization and defining of coal communities in Colorado.  

**Inputs**: NA

**Outputs**:
- RDS file for visualization in shiny application
- RDS file used for compiling the dataset used in the shiny application

**helper function**: patternLayer

**Updating Data**: Coal counties are define by a list within the funciton. Editing this list will result in new content.

**Other Considerations**: The floodplain input dataset is a complex polygon and therefore all the intersections require to generate results tend to be slow. This is not a script you want to rerun unless you have to.


---

**Name**: getDI.r

**Function Name**: `getDI()`

**Description**: Utilizes `tidycensus` to grab acs data and define disproportionality impacted communities in Colorado.  

**Inputs**:
- removeNativeLands
- overwrite

**Outputs**:
- RDS file used for visualization and compiling the dataset used in the shiny application

**helper function**: NA

**Updating Data**: Since the DI communities are not actually a part of the enviroscreen score calculation, this could be one of the first map features to utilize the 2020 census changes in geographies. That said, it is tied to a house bill so there might be some stipulation in changing this.  

**Other Considerations**: removeNativeLands is a binary that excludes specific census tracts and census blocks that are part of the native american reservations in Colorado. This may need to be adjusted as work communication continues with those entities. overwrite is used in this case because there is a direct call on ACS data. It's recommened that overwrite=false is the standard to avoid numerous api calls.  

---

**Name**: getJustice40.r

**Function Name**: `getJustice40()`

**Description**: Summarizes an existing dataset for visualization within the shiny applications.   

**Inputs**:
- filepath
- removeNativeLands
- overwrite

**Outputs**:
- RDS file used for visualization and compiling the dataset used in the shiny application

**helper function**: NA

**Updating Data**: Standard input dataset. Should be easy to update.   

**Other Considerations**: removeNativeLands is a binary that excludes specific census tracts and census blocks that are part of the native american reservations in Colorado. This may need to be adjusted as work communication continues with those entities. overwrite is used in this case because this process requires reading in some large files and therefore takes some time.   


---

**Name**: getOilGas.r

**Function Name**: `getOilGas()`

**Description**: Generate a spatial object of all the counties in Colorado with Oil or Gas development.

**Inputs**: NA

**helper function**: patternLayer

**Outputs**:
- RDS file used for visualization in the shiny apps.
- RDS file used for data within the shiny apps.

**Updating Data**: Requires adjusting the County names in vector within the function.

**Other Considerations**: NA




---

**Name**: getRural.r

**Function Name**: `getRural()`

**Description**: Generate a spatial object of all the rural counties in Colorado.

**Inputs**: NA

**helper function**: patternLayer

**Outputs**:
- RDS file used for visualization in the shiny apps.
- RDS file used for data within the shiny apps.

**Updating Data**: Requires adjusting the County names in vector within the function.

**Other Considerations**: NA

---

**Name**: haps.r

**Function Name**: `getHAPS()`

**Description**: Applies an relative ranking and distance weight method to individual point source pollution measures.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: bufferObjects, normalizeVector

**Outputs**:
- dataframe with GEOID and singular HAPS score meaure
- saves dataframe as csv

**Updating Data**: Comes from APENS which is a standardized data source so updating should be easy. Temporal filtering is done in the generation of the input dataset.

**Other Considerations**: This is one of the more complicated processing methods. Three levels of evaluation. Normalization among specific indicators, distance weighted buffering method, and aggregation across all individual indicators.  

---

**Name**: heartDisease.r

**Function Name**: `getHeartDisease()`

**Description**: Aggregates census tract level heart disease data to census block groups. County and census tract values are provided in the datasource.   

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and heart disease score measure
- saves dataframe as csv

**Updating Data**: There is some specific text formating done to work with the column headers of the input datset, be sure to watch if alterations still apply to updated versions of the data.  

**Other Considerations**: NA


---

**Name**: heatDays.r

**Function Name**: `getHeatDays()`

**Description**: Calculates the average number of heat days at the census tract level for 2015-2019 and aggregates those values to the county and census block group levels.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and average number of extreme heat days over a 5 year period
- saves dataframe as csv

**Updating Data**: Temporal filtering is applied  to the input dataset, update the comments in the code once the year range changes.

**Other Considerations**: NA


---

**Name**: houseBurden.r

**Function Name**: `getHousingBurden()`

**Description**: Calculates the percentage of households that are housing burdened within a given area.

**Inputs**:
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and average number of extreme heat days over a 5 year period
- saves dataframe as csv

**Updating Data**: Input data is called directly from the 2019 ACS via tidy census. Can not update to 2020 until accounting for the redistricting of the decal census.

**Other Considerations**: NA


---

**Name**: lifeExpectency.r

**Function Name**: `getLifeExpectancy()`

**Description**: Aggregates life expectancy measured at the census tract level to county and census block groups.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and average life expectency
- saves dataframe as csv

**Updating Data**: The input dataset does not use the standard FIPS or GEOID to reference geography. The work around for this requires join to county data and some very specific text formating. Hopefully they transistion to standard unique ids in the future, but be careful regardless.

**Other Considerations**: While not present in this function. Life expectency is the only measure were a high value is better then a low value. We account for this in the `sensitivePopulations` function.


---

**Name**: lowBirthWeight.r

**Function Name**: `getLowBirthWeight()`

**Description**: Aggregates low birth weight measured at the census tract level to county and census block groups.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and low birth weight percentage
- saves dataframe as csv

**Updating Data**: The code base adds a leading zero to TRACT_FIPS column. This seems to be coming from the input dataset not the function used to read in the data. When updating the dataset ensure that the input values are behaving the same way. Saved a numeric rather then factor or character values.  

**Other Considerations**: NA

---

**Name**: mining.r

**Function Name**: `getmines()`

**Description**: Performs a distance weight measure of three input dataset to produce a single mine measure.

**Inputs**:
- geometry
- processingLevel
- version
- overwrite


**helper function**: bufferObjects

**Outputs**:
- dataframe with GEOID and distance weight measure for mining activity.
- saves dataframe as csv

**Updating Data**: Temporal filtering is done to the input dataset before reading in content.

**Other Considerations**: NA


---

**Name**: noise.r

**Function Name**: `getNoise()`

**Description**: Calcualtes the average noise value for each geography.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and distance weight measure for mining activity.
- saves dataframe as csv

**Updating Data**: If they keep it as a raster feature any updates should be very easy to impliment.  

**Other Considerations**: NA


---

**Name**: otherHaps.r

**Function Name**: `getOtherHAPS()`

**Description**: Normalized, distance weight measure of individual pollulations averaged across all point soure locations.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: normalizeVector, bufferObjects

**Outputs**:
- dataframe with GEOID and distance weight measure for mining activity.
- saves dataframe as csv

**Updating Data**: Standard datasource so updates should be easy.  

**Other Considerations**: Method is the same used in the `getHAPS` function but the pollutants are different. Temporal filtering is done n the input dataset. Iterative process is applied to the census block level analysis due to the long run times associated with the calculations.


---

**Name**: ozone.r

**Function Name**: `getOzone()`

**Description**: Values measured at the census tract level are aggregated to county and census block group levels.  

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and measure of yearly ozone concentrations
- saves dataframe as csv

**Updating Data**: Standard datasource so updates should be easy.  

**Other Considerations**: Input dataset is huge. We are storing it in the tar.gz file format it was delivered in. The `vroom` library can read this in. Could be packaged different in the future.


---

**Name**: placesData.r

**Function Name**: `getPlacesData()`

**Description**: Values measured at the census tract level are aggregated to county and census block group levels.  

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and cancer, mentalHealth, and diabetes measures.
- saves dataframe as csv

**Updating Data**: Standard data source so updates should be easy.  

**Other Considerations**: NA

---

**Name**: pm25.r

**Function Name**: `getPlacesData()`

**Description**: Values measured at the census tract level are aggregated to county and census block group levels.  

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and average yearly pm2.5 values.
- saves dataframe as csv

**Updating Data**: Standard data source so updates should be easy.  

**Other Considerations**: Input dataset is huge. We are storing it in the tar.gz file format it was delivered in. The `vroom` library can read this in. Could be packaged different in the future.



---

**Name**: proxyOilGas.r

**Function Name**: `getProxyOilGas()`

**Description**: Combines five input datasets and calculated a distance weight score value.   

**Inputs**:
- geometry
- processingLevel
- version
- overwrite


**helper function**: bufferObjects

**Outputs**:
- dataframe with GEOID and average yearly pm2.5 values.
- saves dataframe as csv

**Updating Data**: Standard data source so updates should be easy. Need to update file paths to new input data directly within the function.

**Other Considerations**: Iterative process is applied to generate the census block group meausres. Overall computationally intensive due to the number of oil and gas locations.

---

**Name**: surfaceWater.r

**Function Name**: `getSurfaceWater()`

**Description**: Combines impared surface water measures and the realtive number of evaluated streams to generate a single score value.  

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: bufferObjects

**Outputs**:
- dataframe with GEOID and impaired surface water measure.
- saves dataframe as csv

**Updating Data**: Standard data source so updates should be easy.

**Other Considerations**: Slightly complicated methodology please see technical document for more description on how score value came to be.

---

**Name**: wildfire.r

**Function Name**: `getWildfire()`

**Description**: Calculates average wildfire risk value for all geographies.

**Inputs**:
- filePath
- geometry
- processingLevel
- version
- overwrite


**helper function**: NA

**Outputs**:
- dataframe with GEOID and average wildfire risk.
- saves dataframe as csv

**Updating Data**: Standard data source so updates should be easy.

**Other Considerations**: NA



### compiling functions

#### standard inputs

Not all functions require each input.

**geometry**: Spatial object representing the geography of interest. Required by many individual data indicator processing functions.

**ejscreen**: A dataframe of all the EJScreen values. Generated as part of the `processData` function

**acsData**: A dataframe of all the ACS values. Generated as part of the `processData` function

**processingLevel**: character vector describing the geography of interest ("county", "censusTract", "censusBlockGroup"). Generally only used to constructing file paths.

**version**: character value used to represent the version of the enviroscreen dataset. A new version will force all data processings scripts to regenerate content.

**overwrite**: Binary TRUE/FALSE to define if a existing file should be overwriten. IF FALSE the existing file will be read in and returned as a dataframe.  


- climate.r

---

**Name**: climate.r

**Function Name**: `climate()`

**Description**: Render four indicator processing functions and compiles to produce climate score .

**Inputs**:
- geometry
- processingLevel
- version
- overwrite


**helper function**: gm_mean

**Outputs**:
- dataframe with GEOID, measures and percentile values for all input parameters, and climate score values.
- saves dataframe as csv

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: NA

---

**Name**: environmentalExposures.r

**Function Name**: `environmentalExposures()`

**Description**: Render seven indicator processing functions and compiles to produce environmental exposures score .

**Inputs**:
- geometry
- ejscreen
- processingLevel
- version
- overwrite


**helper function**:  gm_mean

**Outputs**:
- dataframe with GEOID, measures and percentile values for all input parameters, and environmental exposures score values.
- saves dataframe as csv

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: This is a time intensive function to run. Largerly due to the buffering methods that is part of the haps and other haps calculations. If you are troubleshooting something in the joinDataFrames function utilize with health and population compoment score input not this one.


---

**Name**: environmentalEffects.r

**Function Name**: `enviromentalEffects()`

**Description**: Render four indicator processing functions and compiles to produce environmental effects score .

**Inputs**:
- geometry
- ejscreen
- processingLevel
- version
- overwrite


**helper function**:  gm_mean

**Outputs**:
- dataframe with GEOID, measures and percentile values for all input parameters, and environmental effects score values.
- saves dataframe as csv

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: This is the time intensive function to run. Specifically at the census block group level. If you are troubleshooting something in the joinDataFrames function utilize with health and population component score input not this one.



---

**Name**: finalComponentScore.r

**Function Name**: `enviromentalEffects()`

**Description**: Take dataframes from all five component score and calculated group component score and final score values.

**Inputs**:
- dataframes


**helper function**: NA

**Outputs**:
- dataframe with GEOID, measures and percentile values for all measure used in the enviroscreen score.

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: NA

---

**Name**: getShinyData.r

**Function Name**: `getShinyData()`

**Description**: Compiles data for the primary input dataset for the shiny applications.

**Inputs**:
- removeNativeLands
- removeZeroPop
- version
- spanish

**helper function**: NA

**Outputs**:
- RDS file with all information needed to render tables and indicator map visualizations within the shiny app.

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: The output RDS file from this function defines a lot of the object names within the English version of the shiny application. We utilize a large `select` call to set column names. In the spanish version of the application we repeat this process to ensure no errors with special characters. The clause for spanish/english define what naming convention is used for the output data.  


---

**Name**: joinDataFrames.r

**Function Name**: `joinDataFrames()`

**Description**: Compiles multiple dataframes into one and calculates the percentile rank for all features.

**Inputs**:
- dataframes

**helper function**: NA

**Outputs**:
- returns a single dataframe.

**Updating Data**: NA  

**Other Considerations**: This functions does a lot and works on some big assumptions to do so. It's sensitive to the data type of input value and also utilizes some numeric column indexes. This is fine because there is a lot of structure organizing the data before it get to this point. That said, if changes are made to individual indicators or component score calculations I would expect errors to get to this function before showing up.   


---

**Name**: processData.r

**Function Name**: `processData()`

**Description**: Controls all the data processing efforts.  

**Inputs**:
- processingLevel
- version
- overwrite

**helper function**: setSpatialData

**Outputs**:
- returns a dataframe with all enviroscreen score values.
- writes out 8 csv from second level functions.

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: Top level data processing function. If everything works it should be the only you have engage with to regenerate score values.


---

**Name**: sensitivePopulations.r

**Function Name**: `sensitivePopulations()`

**Description**: Renders six indicator processing functions and compiles to produce sensitive populations score.  

**Inputs**:
- geometry
- processingLevel
- ejscreen
- version
- overwrite

**helper function**: gm_mean

**Outputs**:
- returns a dataframe with all sensitive population score values.

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: Includes a few life for inverting the life expectancy score so that high values are considered good. This function renders very quickly.

---


**Name**: socioEconomic.r

**Function Name**: `socioEconomic()`

**Description**: Renders three indicator processing functions and compiles to produce socioeconomic score.  

**Inputs**:
- geometry
- processingLevel
- acsData
- ejscreen
- version
- overwrite

**helper function**: gm_mean

**Outputs**:
- returns a dataframe with all sensitive population score values.

**Updating Data**: Set overwrite to true or change version to force overwrite of score value.

**Other Considerations**: This function renders very quickly.


### helper functions


**Name**: createFolderStructure.r

**Function Name**: `createFolderStructure()`

**Description**: Generates a folder structure require for effectively storing and indexing intemediate products from the data processing code.  

**Inputs**:
- fileLocation : by default this is set to null and will be assigned to the R project library.

**helper function**: NA

**Outputs**:
- numerous file folders and directories

**Updating Data**: NA  

**Other Considerations**: NA

---

**Name**: getGeometryLayers.r

**Function Name**: `getGeometryLayers()`

**Description**: Pulls geometry dataset from tigris or reads in existing features  

**Inputs**:
- overwrite : by default this is set to FALSE as input geometry features are not expected to change.  

**helper function**: NA

**Outputs**:
- geojson files for all require geographies

**Updating Data**: NA  

**Other Considerations**: NA

---

**Name**: gm_mean.r

**Function Name**: `gm_mean()`

**Description**: Calculates the geometric mean of the numeric vector.   

**Inputs**:
- x : a vector of numeric values

**helper function**: NA

**Outputs**:
- geometric mean as a numeric value.

**Updating Data**: NA  

**Other Considerations**: NA values and zero values can not be passed into the geometric mean calculation due to it's use of natural log.


---

**Name**: loadFunctions.r

**Function Name**: `loadFunctions()`

**Description**: Sources all functions into global environment.    

**Inputs**:
- NA

**helper function**: NA

**Outputs**:
- NA

**Updating Data**: NA  

**Other Considerations**: This is a helpful function to use when you are troubleshooting a specific processing function. Do you edits, save, then type `loadFucntions()` into the console. Save some time swtiching back to the 0_main.R script to source the edited content.


---

**Name**: normalizeVector.r

**Function Name**: `normalizeVector()`

**Description**: Normalizes a numeric vector.    

**Inputs**:
- x : numeric vector

**helper function**: NA

**Outputs**:
- a numeric vector

**Updating Data**: NA  

**Other Considerations**: It was very odd for us to not find a built in R function for this process. This version does account for NA within the vectors. We use it on the HAPS and otherHAPS dataprocessing.


---

**Name**: patternLayer.r

**Function Name**: `patternLayer()`

**Description**: Generates the line filled polygons used in the visualization of coal, rural, and oil and gas communities within the shiny applications.     

**Inputs**:
- NA

**helper function**: NA

**Outputs**:
- sf object

**Updating Data**: NA  

**Other Considerations**: This has been a worrisome point for use in the development of this enviroscreen tool. We did not write this functions and it's not something we have the ability to alter without significant effort. We've kept it because it's only providing a visualization, and people liked the visualization. Effectively the function takes a polygon object and creates a new polygon object that have lines running through it. It's not a fill. Very cleaver work overall, see code for reference to the creator.

---

**Name**: setSpatialData.r

**Function Name**: `setSpatialData()`

**Description**: Reads in specific geojson to use for the geometry object.
**Inputs**:
- processingLevel  

**helper function**: NA

**Outputs**:
- sf object

**Updating Data**: NA  

**Other Considerations**: NA









## Setting up the project workspace
- getting the code
- creating an rProject
- sourcing functions
- creating foldering structure
- assign tidy census key





## data access
- direct calls
  - acs data
    - geographies
    - census data
- transfer content
  - description on the file format of each input dataset
  - some datasets have been process to the format require for running within the tool. Often this is just moving from .xlsx to .csv but sometime more processing was applied. I've not included those processing steps in the codebase and it will be up to the user to manager those transformations.


## generalized workflow
- overview of classification structure
- calculation of component scores
- calculation of group component scores
- calcualtion of final scores

## details workflow
- helper functions
- individuals indicators
  - general structure of the function
    - inputs : geometry, processingLevel, overwrite
    - outputs : saves a csv, returns a dataframe with GEOID and values for indicator
- example of how versioning works
  - defining file path in processing functions
  - saving files for compiling functions


## Code Conventions

- camelCase is used
- functions file names match the function name
- processing functions generally include input parameters (filePath, geometry, processingLevel, version, overwrite)
- The GEOID is maintained in all intemediate datasets
- overwrite is defined by higher level function unless explicited defined by user.



mines and surface water do not require file inputs