ValueError: unknown locale: UTF-8

[lcx]

I have this issue on every almir installation. I did solve it once, of course didn't write the solution down and I have no idea how to get almir starting again.

bin/supervisorctl restart all
HttpOk: stopped
Memmon: stopped
HttpOk: started
Memmon: started
almir: ERROR (abnormal termination)

Logfile:

Traceback (most recent call last):
  File "/opt/almir/bin/pserve", line 20, in <module>
    sys.exit(pyramid.scripts.pserve.main())
  File "/opt/almir/eggs/pyramid-1.3-py2.6.egg/pyramid/scripts/pserve.py", line 47, in main
    return command.run()
  File "/opt/almir/eggs/pyramid-1.3-py2.6.egg/pyramid/scripts/pserve.py", line 290, in run
    relative_to=base, global_conf=vars)
  File "/opt/almir/eggs/pyramid-1.3-py2.6.egg/pyramid/scripts/pserve.py", line 318, in loadapp
    return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/opt/almir/eggs/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 646, in find_egg_entry_point
    possible.append((entry.load(), protocol, entry.name))
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2108, in load
  File "/opt/almir/almir/__init__.py", line 13, in <module>
    from almir.meta import initialize_sql
  File "/opt/almir/almir/meta.py", line 20, in <module>
    from almir.lib.utils import timedelta_to_seconds, convert_timezone
  File "/opt/almir/almir/lib/utils.py", line 3, in <module>
    from docutils.core import publish_parts
  File "/opt/almir/eggs/docutils-0.9.1-py2.6.egg/docutils/core.py", line 20, in <module>
    from docutils import frontend, io, utils, readers, writers
  File "/opt/almir/eggs/docutils-0.9.1-py2.6.egg/docutils/frontend.py", line 39, in <module>
    import docutils.utils
  File "/opt/almir/eggs/docutils-0.9.1-py2.6.egg/docutils/utils/__init__.py", line 19, in <module>
    from docutils.io import FileOutput
  File "/opt/almir/eggs/docutils-0.9.1-py2.6.egg/docutils/io.py", line 18, in <module>
    from docutils.error_reporting import locale_encoding, ErrorString, ErrorOutput
  File "/opt/almir/eggs/docutils-0.9.1-py2.6.egg/docutils/error_reporting.py", line 47, in <module>
    locale_encoding = locale.getlocale()[1] or locale.getdefaultlocale()[1]
  File "/usr/lib/python2.6/locale.py", line 478, in getdefaultlocale
    return _parse_localename(localename)
  File "/usr/lib/python2.6/locale.py", line 410, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

my locale settings

LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

OS: Debian Squeeze (6.x)

[domenkozar]

Settings

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

should fix that, you sure almir has access to those variables?

[domenkozar]

Verify that works by doing:

$ python
>>> import locale
>>> locale.getdefaultlocale()

[lcx]

tried the export LC_ALL and LANG settings and any other pytho/django tip I found on google.
All don't work :(

python
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale;
>>> locale.getdefaultlocale()
('en_US', 'UTF8')

I really can't remember what I did last time to get it started. Something really strange.

Strike that, now I remember.
Starting it manually seems to work

/usr/bin/python2.6 -S /opt/almir/bin/pserve /opt/almir/production.ini

[domenkozar]

Ah, maybe supervisor is not letting those through. You are using the automatic deployment?

[domenkozar]

Could you try to run snippet at the bottom of blog post to figure out if supervisor has locales set? http://albertoconnor.ca/blog/2012/Jul/21/unicodeencodeerror-when-uploading-files-django-usi

[lcx]

Sorry it took me so long. This is what I have got

cat var/log/locale-stdout---supervisor-vqDkOn.log
ANSI_X3.4-1968
ANSI_X3.4-1968
ANSI_X3.4-1968
ANSI_X3.4-1968
ANSI_X3.4-1968
ANSI_X3.4-1968
ANSI_X3.4-1968
ANSI_X3.4-1968

This is on a fresh install from yesterday.

After a reboot however it looks like this:

UTF-8
UTF-8
UTF-8
UTF-8

This however gives me a new error:

Internal Server Error

The server encountered an unexpected internal server error

(generated by waitress)

and this is the logfile

2014-01-01 13:02:01,268 ERROR [exc_logger][Dummy-1] http://10.10.10.8:2500/
Traceback (most recent call last):
  File "/opt/almir/eggs/pyramid_exclog-0.5-py2.7.egg/pyramid_exclog/__init__.py", line 40, in exclog_tween
    return handler(request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/router.py", line 164, in handle_request
    response = view_callable(context, request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/config/views.py", line 333, in rendered_view
    result = view(context, request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/config/views.py", line 471, in _requestonly_view
    response = view(request)
  File "/opt/almir/almir/views.py", line 29, in dashboard
    upcoming_jobs = Job.get_upcoming()
  File "/opt/almir/almir/models.py", line 287, in get_upcoming
    return BConsole().get_upcoming_jobs()
  File "/opt/almir/almir/lib/bconsole.py", line 127, in get_upcoming_jobs
    p = self.start_process()
  File "/opt/almir/almir/lib/bconsole.py", line 69, in start_process
    return Popen(shlex.split(self.bconsole_command), stdout=PIPE, stdin=PIPE, stderr=PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
2014-01-01 13:02:01,271 ERROR [waitress][Dummy-1] Exception when serving /
Traceback (most recent call last):
  File "/opt/almir/eggs/waitress-0.8.1-py2.7.egg/waitress/channel.py", line 329, in service
    task.service()
  File "/opt/almir/eggs/waitress-0.8.1-py2.7.egg/waitress/task.py", line 173, in service
    self.execute()
  File "/opt/almir/eggs/waitress-0.8.1-py2.7.egg/waitress/task.py", line 380, in execute
    app_iter = self.channel.server.application(env, start_response)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/router.py", line 187, in __call__
    response = self.handle_request(request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/tweens.py", line 20, in excview_tween
    response = handler(request)
  File "/opt/almir/eggs/pyramid_tm-0.3-py2.7.egg/pyramid_tm/__init__.py", line 61, in tm_tween
    response = handler(request)
  File "/opt/almir/eggs/pyramid_exclog-0.5-py2.7.egg/pyramid_exclog/__init__.py", line 40, in exclog_tween
    return handler(request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/router.py", line 164, in handle_request
    response = view_callable(context, request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/config/views.py", line 333, in rendered_view
    result = view(context, request)
  File "/opt/almir/eggs/pyramid-1.3-py2.7.egg/pyramid/config/views.py", line 471, in _requestonly_view
    response = view(request)
  File "/opt/almir/almir/views.py", line 29, in dashboard
    upcoming_jobs = Job.get_upcoming()
  File "/opt/almir/almir/models.py", line 287, in get_upcoming
    return BConsole().get_upcoming_jobs()
  File "/opt/almir/almir/lib/bconsole.py", line 127, in get_upcoming_jobs
    p = self.start_process()
  File "/opt/almir/almirlib/bconsole.py", line 69, in start_process
    return Popen(shlex.split(self.bconsole_command), stdout=PIPE, stdin=PIPE, stderr=PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

If I kill supervisord and restart it manually I get the UTF8 error agains.
Killing it again and starting it with LC_ALL=C solves all the problems:

LC_ALL=C /usr/bin/python2.7 /opt/almir/bin/supervisord

The odd thing is, no matter what I do, after a reboot I get the "No such file or directory" I have to kill supervisord and restart it manually.

[domenkozar]

This means bconsole command is missing on your system

[lcx]

not quite, bconsole is installed. and if I kill supervisord and restart it manually, everything works. Without any "No such file or directory" error.

[domenkozar]

So the problem goes deeper, your shell environment is not passed in supervisor. What version are you using?