Application for maintainership

[pudo]

Hey all. This repository seems to be semi-inactive, and it unclear to me what the path to merging a PR like #171 is (who would have to approve?). I use messytables in production code day to day, and this lack of clarity on process makes the library a liability. My understanding is that okfn's resources and interest is focussed on goodtables and the frictionlessdata toolchain.

I would therefore like to apply to become the maintainer for messytables, merge #171 & co., and generally make sure that changes in this thing are handled and bugs are actively tracked.

Thoughts, @pwalsh, @davidread, @rufuspollock? Please let me know.

[pwalsh]

Hey @pudo !

Absolutely. I'm away right now so can you contact @akariv to get all the details you need.

[rufuspollock]

@pudo 👍 👍

Assume this would supersede maintainership by scraperwiki folks (maintership implied here #163 (comment)). That same thread suggested @StevenMaude at ScraperWiki would be happy for others to take on maintainership:

Since our current work on databaker, which uses messytables, is soon coming to a close, we won't be actively working on messytables, but don't mind keeping half an eye on things to make sure that e.g. new releases get pushed out.

@pudo re

My understanding is that okfn's resources and interest is focussed on goodtables and the frictionlessdata toolchain

Note the direct replacement for messytables is not so much goodtables (which does data validation) but:

• https://github.com/frictionlessdata/tabulator-py - consistent interface for tabular data
• https://github.com/frictionlessdata/tableschema-py - this has the infer stuff (generates JSON table schema from tabular data etc)

(You may already know that but others may not who read this thread!)

[rufuspollock]

PS: @akariv @pwalsh if you guys are super busy I'm happy to do the necessary to get @pudo admin rights here (though I can't grant rights on pypi!)

[domoritz]

@pudo is already owner of messytables on PyPi.

[StevenMaude]

👍

Our (Sensible Code, was ScraperWiki) recent involvement with messytables has been maintenance to ensure Databaker runs, and the current phase of Databaker work is done.

Personally, I have been/am happy to review and merge in simple fixes, and release them. Anything more substantial or opinionated, such as #171, felt like changes that should be reviewed and decided by OKFN, or a maintainer who has a definite view on direction.

Not by an external contributor coming in and mashing anything that looks remotely like a merge button 😉

[davidread]

I guess one key aim of messytables is the "messy" bit. That's why its used in CKAN's DataPusher and tabular previewer.

When reading CSV do either of the new frictionless tools do:

• inferring character encoding
• filtering out BOMs
• cope with different CSV dialects - separator, quoting, EOL chars
• detect where the header row is

Or are the frictionlessdata tools more meant for pre-cleaned CSVs? Just trying to understand if there's still value in messytables or something like it.

[davidread]

I had a quick look at tableschema and it only infers the column types, and not dealing with the messiness that messytables does a decent job of coping with - character encodings, BOMs, dialects, multiple header rows. So I think there is no sign of messytable's functionality being replaced by frictionless data, there is definitely a place for messytables and I will continue to use it in ckanext-xloader.

And of course the python3 support in PR #171 is essential to get merged.

@rufuspollock: since you and @StevenMaude are happy with @pudo being maintainer, can you give him the permissions? And while you're at it, can I be added back too (I think I lost membership of okfn org in the past couple of years), so I can resolve PR conflicts etc.

[rufuspollock]

@davidread done - it def sounds like keeping this up to date is useful.

[davidread]

Thanks @rufuspollock and if you can add me on this repo too, that would also be useful.

[davidread]

Ah, I see I got an invite. Cheers!