Automatic charset detection

[Nyoroon]

No description provided.

[coveralls]

Coverage decreased (-85.0%) when pulling 3c9713a on Nyoroon:patch-1 into 3dae140 on mgedmin:master.

[mgedmin]

You need to add chardet to install_requires in setup.py.

[mgedmin]

I see chardet detects UTF-8, so why not use it always? I.e. lose the try/except.

[mgedmin]

I'd also be more comfortable if the input encoding could be specified explicitly, in case chardet guesses wrong. A command-line/config file option perhaps? Say, --input-charset={detect|hybrid|<charset>}, where detect would use chardet, hybrid would be http://xchat.org/encoding/ (i.e. the current behavior of irclog2html master), and <charset> would be s.decode(charset, 'replace').

The default should probably be chardet, unless I discover that it guesses wrong too often, in which case I'll revert the default to hybrid.

[coveralls]

Coverage remained the same when pulling 711e2e2 on Nyoroon:patch-1 into 3dae140 on mgedmin:master.

[mgedmin]

Chardet doesn't appear to be very-well maintained: the PyPI page points to https://github.com/erikrose/chardet, which doesn't allow anyone to file issues, and is a fork of https://github.com/dcramer/chardet, which has a bunch of 1-year-old unanswered issues.

One of those issues is Python 3 support: dcramer/chardet#7. irclog2html currently supports Python 3. Adding a dependency on chardet breaks that.

[Nyoroon]

Oh, thanks!
I'll think about this.

[Nyoroon]

https://github.com/sigmavirus24/charade seems better, but not ideal.

[coveralls]

Coverage decreased (-4.33%) when pulling a2f1fc4 on Nyoroon:patch-1 into 3dae140 on mgedmin:master.

[Nyoroon]

I'll test after every code change
I'll test after every code change
I'll test after every code change
I'll test after every code change
I'll test after every code change
I'll test after every code change

[coveralls]

Coverage decreased (-0.01%) when pulling 8399444 on Nyoroon:patch-1 into 3dae140 on mgedmin:master.

[mgedmin]

Do you still plan to work on this?

As you can see on Travis CI, the tests are broken for Python 2.x and 3.x.

[Nyoroon]

Yes, i work on it.