April 10th 2017 release

Iterate blocks (#215)

* change get dataset to only return a df, and storing evaluations using model_id

* test using triage

* traige working with many test_dates and using metta to store matrices

* change the way we call the dates for generating the features

* use metrics for percentage and absolute values for evaluation

* added parallel execution of feature generation

* updated result schema with indexes

* change metadata to datetime

* change logistic regression parameter C_reg to C

* add test_time_ahead

* add raw incident type and levels

* change group incident to complaint type to add raw type

* delete grouped incident type or incident type levels from incidents table, just leaving the raw code

* create functions to create labels table from config file and reads the matrices for loading train and test matrices

* change example config file

* add dispatch type code for original and final response and add new categories for the dispatch lookup table to match for cmpd

* remove features using group incident type code

* added new field interview outcome type banned

* added new field interview types

* bracket was missing at dispatch config

* fixed bug with unclosed bracket and prepeard config

* updated config and featuers for new categories and limited selection based on 1600 colum limit

* changed config to be below 1600 columns

* merge with new configuration for labels and create function for generating all matrices

* delete events with NULL event_datetime and officer ids from labels table

* changes in config file for labels

* added locking function which is NFS save

* add make_hashable function for metadata

* removed debugging statements

* changed data-types to real in results to save space and added a timeout for locking

* fix hash function, DaysSinceLastCompletedAllegation and config file

* fix labels table removing null values

* make parallelize generate matrices

* use psycopg2 for reading features and naming cursor

* changed model running to train a single model first and then test across all test sets and then discard the model

* add labels_config to models_group

* added a delete statement before new evaluations for the same as_of_date and model_id are  written to the database

* add officer_id column and labels_config that passes into metta

* change query for grabbing labels to exclude as of date + prediction window interval

* add model_config to metadata for creating model_group_id

* added rounding for storing float to avoid underflow errors in postgres

* increased verbose output

* add function for generating as of dates for feature generation

* typo

* change officer roles table adding dates and department

* comment util functions and change train_metadata send to db_parameters to the config file in the db

* add ucr4 code grouped matching dispatch types, add lookuptables for ucr4 grouped and codes

* delete old code for feature generation, add docstrings

* remove unused config file

* be able to iterate through blocks leaving one block out and reading features tables directly from collate tables

* add arrest crime feature and compliments block

* add sworn flag

* fix stored procedure for reading feature blocks

* add individual importances table in production schema

* iterate through blocks and add post collate feature for dispatch movement

* change log info and add DispatchMovement to config

* add create patrol districts table and change config parameters

* stored procedure for getting complete feature description

* change evaluations as of date column to start and end time

* delete patrol distrincts

* added outside employment

* add train end time and experiment hash to results.models

* add experiment hash for triage and fix evaluation start time

* remove feature abstract unused classes, update config and add features descriptions