Skip to content

Revisar Crawler para tratar erros de conexão com o Opensearch #34

@josircg

Description

@josircg

Nos casos em que o Opensearch não responde (erro de rede ou no momento do backup), a gravação dos tweets é interrompida.

Erro ConnectionTimeout caused by - ReadTimeoutError(HTTPSConnectionPool(host=xxxx, port=443): Read timed out. (read timeout=60)) Traceback (most recent call last): File "/var/webapp/twitsearch/lib/python3.8/site-packages/urllib3/connectionpool.py", line 468, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/var/webapp/twitsearch/lib/python3.8/site-packages/urllib3/connectionpool.py", line 463, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 277, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")

  1. Não realizar a captura entre 0h e 1h da manhã
  2. Tratar o erro para interromper a carga de forma "elegante", interrompendo os processos seguintes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions