[Python3] `UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte`

[takano32]

Redis のオプションでデコードすると発生してしまう。

decode_responses=True

Traceback (most recent call last):
  File "/var/www/cgi-bin/sub/sun.cgi", line 1542, in <module>
    no.say(slack=no.is_slack, lingr=False)
  File "/var/www/cgi-bin/sub/sun.cgi", line 1522, in say
    resp = "\n".join(tuple(self.response())).rstrip("\n")
  File "/var/www/cgi-bin/sub/sun.cgi", line 1497, in response
    r = I.response()
  File "/var/www/cgi-bin/sub/sun.cgi", line 799, in response
    value = self.conn.get(key)
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/commands/core.py", line 1829, in get
    return self.execute_command("GET", name)
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/client.py", line 536, in execute_command
    return conn.retry.call_with_retry(
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/client.py", line 537, in <lambda>
    lambda: self._send_command_parse_response(
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/client.py", line 513, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/client.py", line 553, in parse_response
    response = connection.read_response()
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/connection.py", line 500, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/_parsers/resp2.py", line 15, in read_response
    result = self._read_response(disable_decoding=disable_decoding)
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/_parsers/resp2.py", line 67, in _read_response
    response = self.encoder.decode(response)
  File "/home/doge/workspace/sub.python3/lib/python3.10/site-packages/redis/_parsers/encoders.py", line 43, in decode
    value = value.decode(self.encoding, self.encoding_errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

解消

diff --git a/sun.py b/sun.py
index e16c371..4f499fe 100755
--- a/sun.py
+++ b/sun.py
@@ -52,7 +52,8 @@ class Subculture(object):
     @property
     def conn(self):
         if self._conn is None:
-            self._conn = redis.Redis(host='127.0.0.1', db=self.__redis_db, decode_responses=True)
+            # self._conn = redis.Redis(host='127.0.0.1', db=self.__redis_db, decode_responses=True)
+            self._conn = redis.Redis(host='127.0.0.1', db=self.__redis_db)
             try:
                 self._conn.ping()
             except redis.exceptions.ResponseError as e:

keys の文字列などが bytes で出てくるので困る。

文字列として扱うところをコツコツと特定して、コツコツと decode しつつ decode_responses=False に変更するのがよさそう。

参考

redis/redis-py#910

redis-py 3.0 drops support for the legacy “Redis” client class. “StrictRedis” has been renamed to “Redis”.

[takano32]

pickle で格納してるデータを JSON にするのがよさそう。

文字列じゃない箇所はたぶんそこだけ。