R Documentation Needs some Work

[RishiSadhir]

I would absolutely love to be able to use this package - Unfortunately I have know idea how to call any of the seemingly useful R files in this repo. I would love to see some example where, in R, one would take a data set, train a rulefit, and convert it to SQL.

[gioseza]

Hi Rishi - the instructions for using REgo are under the heading "Commands." For instance, to build a model, you invoke the trainModel command from a terminal window (not from R):

$REGO_HOME/bin/trainModel.sh --d=DATA.conf --m=MODEL.conf

where:

• REGO_HOME is an environment variable to the folder where you downloaded REgo
• DATA.conf is a text file with specifications about the data (a sample is included)
• MODEL.conf is a text file with specifications about the model you want to build and the directory where you want the results stored (a sample is included)

Once you are happy with he resulting model, you can export it as a SQL expression invoking the exportModel command by:

$REGO_HOME/bin/exportModel.sh --m=MODEL.dir --c=EXPORT.conf

A sample of the EXPORT configuration file is also provided.

Happy to answer additional questions.

[RishiSadhir]

Thanks for the quick response gioseza. I suppose I was a bit confused. I assumed that with so much work done through R there would have been an interface through R with which one might be able to manage the whole process.

Is there interest in that kind of code being written? If so, I'd be happy to contribute.

[gioseza]

A key objective of REGo is to "enable rapid experimentation" by giving users the "ability to have multiple batch jobs running simultaneously." That is, instead of requiring an interactive R session REgo makes it easier to launch multiple learning jobs.

[jtenini]

To piggy-back on these comments: Is there any documentation describing the methods used to generate the the Model Summary? A few of my questions:

1. What does RE refer to? (root mean squared error?)
2. How is the test error being estimated?
3. Is there access to Friedman's rfxval function to get things like AUC and RMSE from a full cross-validation of the model?
4. How is the training error being computed?

Thank you so much for your work on this package, it is certainly an exciting piece of development!

[gioseza]

Hi Jtenini - answers to your questions below:

Q1. What does RE refer to?

RE stands for Rule Ensembles. As described at the top of the README, Rule Ensembles are "an interpretable type of ensemble model where the base-learners consists of conjunctive rules derived from decision trees." The original reference is "Predictive learning via rule ensembles" in The Annals of Applied Statistics, Volume 2, Number 3 (2008),

Q2 & 4. How is the train/test error being estimated?

The meaning of "error" depends on the loss function being used -- see "score.criterion" parameter in the MODEL.conf documentation. When the program prints "estimated," it refers to cross-validation based estimate -- see "crossvalidation" parameters in the MODEL.conf documentation.

Q3. Is there access to Friedman's rfxval function?

I have coded a stand-alone CV function in file rfCV_main.R under the src directory (I haven't written an entry in the README for it yet). But all original RUleFIt functions, like rfxval, can be manually called using a restore after REgo is finished.

[jtenini]

Thank you for the answers gioseza. Does your package require the data set to be centered and scaled before running or is that happening internally? (I'm thinking here of the regularization being computed appropriately.)

[gioseza]

No, it doesn't expect the data set to be centered and scaled. You can read about RE’s in this reference "Predictive learning via rule ensembles" in The Annals of Applied Statistics, Volume 2, Number 3 (2008).

[jtenini]

Thank you very much for your answers gioseza, I'm afraid I have a few more questions:

1. Friedman seems to indicate that his rf_go.exe will no longer function after tomorrow (12/29/2015), are you aware of the future of the project? Will rego will no longer function after this date?
2. I'm still not sure I understand how to read the Model summary.
   Suppose I run a regression model with the "score.criterion" set to "LS" and I receive the following:

Training Error:
Model Error
RE 912.9428
Median 2807.8187
ratio 0.33

Test Error:
Model Error +/-
RE 902.8 12

How should I interpret these numbers? 912.9428 is the average squared error (across all test data), 2807.8187 is the median squared error (across all test data), and ratio is . . . ?

902.8 is estimated average squared error from the cross validations? How can it be lower than the average squared error across all test data?

Thank you for your many excellent explanations!