Skip to content

Update defaults and readme for deepdive 0.7.0-alpha #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
arichard4 wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Update defaults and readme for deepdive 0.7.0-alpha #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5271103
Update braindump default conf file from application.conf to deepdive.…
Jul 23, 2015
b5766be
Updated readme
arichard4 Jul 23, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 6 additions & 16 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,38 +56,28 @@ file has been configured for
Integrating with your DeepDive application
----

Suppose your app runs in `APP_HOME` and your outputs are saved in `DEEPDIVE_HOME/out`. Suppose you have a `run.sh` script that runs your application, and you want an automatic report each time after `run.sh` finishes:
Suppose your app runs in `APP_HOME`. Suppose you have a `run.sh` script that runs your application, and you want an automatic report each time after `run.sh` finishes:

Just add into the your `run.sh` a command to run `braindump`. e.g. If your `run.sh` looks like:

```
#! /bin/bash

. "$(dirname $0)/env.sh"

cd $DEEPDIVE_HOME

### Run with deepdive binary:
deepdive -c $APP_HOME/application.conf
deepdive run
```

Update it to look like this:

```
#! /bin/bash

. "$(dirname $0)/env.sh"

cd $DEEPDIVE_HOME

# Be sure to set this so that you are able to QUIT if deepdive fails.
set -e

# Run with deepdive binary:
deepdive -c $APP_HOME/application.conf
deepdive run

# Note that you should go back to your APP_HOME directory
cd $APP_HOME
braindump
```

Expand All @@ -103,8 +93,8 @@ APP_HOME/experiment-reports/v00001/:
├── calibration -- Calibration plots
│   ├── has_spouse.is_true.png
│   └── has_spouse.is_true.tsv
├── code -- Saved code for this run (default saves "application.conf" and "udf/")
│   ├── application.conf
├── code -- Saved code for this run (default saves "deepdive.conf" and "udf/")
│   ├── deepdive.conf
│   └── udf
├── dd-out -- A symbolic link to the corresponding deepdive output directory
├── dd-timestamp -- A timestamp of this deepdive run
Expand Down Expand Up @@ -185,7 +175,7 @@ Configuration Specification
- `VARIABLE_COLUMNS`: variable columns in the same order of VARIABLE_TABLES. Separated by space. e.g. "v1 v2"
- `VARIABLE_WORDS_COLUMNS`: if the variable is a mention, specify the words / description for the mention. This is used for a statistics with naive entity linking. If empty (""), do not count deduplicated mentions for that table. Separated by space. e.g. w1 ""
- `VARIABLE_DOCID_COLUMNS`: specify if there is a field in the variable table that indicates doc_id. This is used to count how many documents have extractions. If empty (""), do not count for that table. Separated by space. e.g. "" did2
- `CODE_CONFIG`: a config file that specifies what in $APP_HOME to save as codes, one file/folder per line. Default file is saving "application.conf" and "udf".
- `CODE_CONFIG`: a config file that specifies what in $APP_HOME to save as codes, one file/folder per line. Default file is saving "deepdive.conf" and "udf".
- `NUM_SAMPLED_FEATURES`: the number of sampled features for each feature table specified in "FEATURE_TABLES"
- `NUM_SAMPLED_SUPERVISION`: the number of sampled supervision examples for each variable specified in "FEATURE_COLUMNS"
- `NUM_SAMPLED_RESULT`: the number of sampled inference results for each variable
Expand Down
2 changes: 1 addition & 1 deletion util/code_default.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
application.conf
deepdive.conf
udf
6 changes: 3 additions & 3 deletions util/config-generator/generate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@
default='',
help='specify if there is a field in the variable table that indicates doc_id. This is used to count how many documents have extractions. If empty (""), do not count for that table. Separated by space. e.g. "" did2')
@click.option('--code_config',
prompt='Specify CODE_CONFIG, a config file that specifies what in $APP_HOME to save as codes, one file/folder per line. Default file is: \napplication.conf\nudf',
prompt='Specify CODE_CONFIG, a config file that specifies what in $APP_HOME to save as codes, one file/folder per line. Default file is: \ndeepdive.conf\nudf',
default='',
help='a config file that specifies what in $APP_HOME to save as codes, one file/folder per line. Default file is: \napplication.conf\nudf\n')
help='a config file that specifies what in $APP_HOME to save as codes, one file/folder per line. Default file is: \ndeepdive.conf\nudf\n')
@click.option('--num_sampled_docs',
prompt='Specify NUM_SAMPLED_DOCS',
default='100',
Expand Down Expand Up @@ -251,4 +251,4 @@ def generate(output_file,


if __name__ == '__main__':
generate()
generate()