Documentation

[joelvh]

Hi Stephen,

Thanks for reaching out on twitter. I figured this might be a good place to ask if you could maybe provide some information on your use case and what you think needs to be improved? The code looks similar to what I'm working on, but it would be easier to understand the intent if you have some time. Thanks!

[stephenprater]

Hey Joel - the basic idea was that you created a 'transforms' file or put the appropriate classes in your Rails app somewhere, inherited from Datatron::Strategy and used the from, to, (most of which take blocks) and other methods to transform data from one Datatron::Formatter subclass to another. The Datatron::Formatter classes were pretty basic - just a thin wrapper around an external iterator and whatever was necessary to read and write the data formats, a kind of super-generalized facade object. The formatters were supposed to be resumable, although I don't know if I ever got that to work.

The basic use case was less 'import my data' and more 'keep my data in sync between exotic formats over weird and somewhat unreliable connections.' At one point I was writing a formatter class for UniVerse multivalue files to convert to a series of CSVs and JSON text files. The excess of abstraction was in service of the fact that the formats were sort of ever changing, and I wanted to separate all of those concerns.

The two big things I think need attention are:

• The Datatron::Translator is a particularly ugly class factory. I can't remember why I did it that way, and it seems really wrong in retrospect.
• Writing new formatters is WAY too complicated.

There's a quite a bit of weird 'worked for me' things in there too. I notice that I required my term_helper library, but don't seem to have but it the Gemfile.

That said, I actually have a need for this, and might pick it up again in a few months, so any improvements, patches, suggestions or ground-up rewrites you might complete would be most welcome.