Skip to content

Get_test_output fails when config contains error_output #782

@kmeinerz

Description

@kmeinerz

Problem
Using the get_test_output function in combination with a config containing an error_output field returns FileNotFoundError: [Errno 2] No such file or directory

Minimal example

# pylint: disable=line-too-long
import tempfile
import time
from pathlib import Path

import pytest

from logprep.util.configuration import Configuration
from tests.acceptance.util import get_test_output

CHECK_INTERVAL = 0.1


def wait_for_interval(interval):
    time.sleep(2 * interval)


@pytest.fixture(name="config")
def get_config():
    config = {
        "version": "1",
        "logger": {"level": "DEBUG"},
        "process_count": 1,
        "timeout": 0.1,
        "profile_pipelines": False,
        "restart_count": -1,
        "pipeline": [
            {
                "pre_detector": {
                    "type": "pre_detector",
                    "outputs": [{"jsonl": "pre_detector_topic"}],
                    "rules": ["tests/testdata/acceptance/pre_detector/rules"],
                    "tree_config": "tests/testdata/acceptance/pre_detector/tree_config.json",
                }
            },
        ],
        "input": {
            "jsonl": {
                "type": "jsonl_input",
                "documents_path": "tests/testdata/input_logdata/selective_extractor_events.jsonl",
            }
        },
        "output": {
            "jsonl": {
                "type": "jsonl_output",
                "output_file": tempfile.mkstemp(suffix="output1.jsonl")[1],
                "output_file_custom": tempfile.mkstemp(suffix="custom1.jsonl")[1],
            },
        },
        "error_output": {
            "jsonl": {
                "type": "jsonl_output",
                "output_file": tempfile.mkstemp(suffix="error.jsonl")[1],
            }
        },
    }
    return Configuration(**config)


def test_bug(tmp_path: Path, config: Configuration):
    config_path = tmp_path / "generated_config.yml"
    config_path.write_text(config.as_yaml())
    get_test_output(str(config_path))

Cause
The function get_runner_outputs in tests\acceptance\util.py is supposed to delete all old output files, start the runner, create new output files, read the files and then delete them.
No new error_output file gets created, which causes the error, when trying to read the file.

Possible solutions

  1. Only try reading existing files
  2. Always create at least an empty error output file

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions