Separate functions for loading data in Java and R

[hdbeukel]

When reading data from a file it is currently loaded in Java and in R. For very large datasets this duplication may cause memory issues. Therefore it would be nice to have an option to not load the data in R but in Java only.

If you use the database services and corehunter services, its all handled in Java. So I think this is best option. We can discuss.

[hdbeukel]

Well, the reason why I also load the data in R by default is because I figured this would be useful to for example explore the selected core in more detail. It could easily be omitted but I guess that's only necessary when running into trouble with memory right?

Yes, I only see the need to use the services and/or alternative data structures if running into memory issues. I suggest you continue to use the simple implementations. Once version 3.1 is released, I can focus on this issue.

[hdbeukel]

This issue is not really related to the data structures. The question asked is whether the data should be loaded in Java only to select the core or in R as well to allow further analysis of the core. It's more a use case question than a data structure question. What do you think about this?

I agree

If R can handle it then read it in R as well. It should be an option to load only in java in the case there is a memory issue.

[hdbeukel]

And what about loading only in R? Seems useful as well, for example when you have previously constructed cores and wrote the selected IDs to a file and now just want to analyze these cores in R.

Yes, that also should be an option.

[hdbeukel]

I guess it might be best to just split into two functions, one for loading in Java and one for R. This would avoid confusing the users.

This change requires quite some coding (especially reading in R without going through Java which currently handles the nasty details) so I suggest to postpone to version 3.2.

Agreed?