Reworked Configuration

[carsonoid]

Introduction

As discussed at the last PLUG I wanted to propose a more flexible config file structure. NOTE: I have no opinion on the exact format for the config (YAML, JSON, etc), My example below is simply in JSON for readability and syntax highlighting. It may not be a perfect format but adds a lot of functionality and flexibility while maintaining usability. Let me know what you think!

Sample Configuration

{
    "ARBITRARY_NAME_1" : {
        "files"   : ["LIST", "OF", "GLOB", "PATTERNS"],    // Required
        "exclude" : ["LIST", "OF", "GLOB", "PATTERNS"],
        "dir"     : "/path/to/working/dir",
        "steps"   : [                                      // At least one entry required. Run consecutively
                        {
                            "name"          : "STEP_NAME", // Required
                            "run"           : "COMMAND",   // Required
                            "is_daemon"     : false,
                            "enabled"       : true,
                            "on_success"    : "COMMAND",
                            "on_error"      : "COMMAND",
                            "error_status"  : [],
                            "ignore_status" : []
                        },
                        {
                            "name"          : "STEP_NAME",   // Required
                            "run"           : "COMMAND",     // Required
                            "is_daemon"     : true,
                            "enabled"       : true,
                            "on_success"    : "COMMAND",
                            "on_error"      : "COMMAND",
                            "ignore_status" : []
                        }
        ]
    }
}

Explanation

The configuration above would allow for as many filesets as you want to be defined in the config file. Each with a different set of steps to execute on file change. It also allows for a arbitrary number of steps to be defined per each fileset. So rather than hard-coding the step names we can leave that up to the config. Some filesets may have a full build->test->exec set of steps. Others may just need a build step alone. While more complicated projects can have build->test->deploy->test->exec or any imaginable set of steps when files change. Each step is only run if the run previous command succeeds or returns a code matched in the ignore_status directive.

fileset directives:

• files : A list of globs defining files to watch
• exclude : A list of globs defining files to exclude. NOTE: No include directive is needed and would be redundant
• dir : Path to the relative directory to match the files and exclude patterns against. (Defaults to CWD?)
• steps : A ordered list of steps to take when any matched files are changed.

step directives

• name : Arbitrary step name for display purposes. NOTE: This must be a property of the step as defined in JSON as if it were the key to a dictionary item it would not maintain it's order.
• run : Command to run for step
• is_daemon : whether the command should be treated as a daemon. i.e. killed and restarted in bg
• enabled : Whether command is enabled
• on_success : Command to run if run directive succeeds.
• on_error : Command to run if run directive has error status.
• ignore_status : list of status codes that don't count towards on\_error Defaults to anything non-zero

[TheTechmage]

Review

After reading through your proposed config file structure, I really like it and thing that I would be wise to follow it (or something very similar) for Renzoku. I started to implement the structure via structs (seen below) so that I could think through the config items a bit more carefully, and I did have a few questions regarding some of the config items.

struct CfgStep {
        char* name; // Required
        char* command; // Required
        bool daemon; // Optional, do we need this? (Not recommended?)

I'm assuming that we don't need the daemon value since we can just detect if it is still running before killing the process. Maybe a "kill signal" or an HTTP request for a soft kill would be good items to add?

        bool enabled; // Optional, default: true
        // on_success/on_error commands

I honestly don't understand the point of the on_success and on_error commands. Could you enlighten me with a possible use case?

        short* error_status; // Optional, default: NULL
        short* ignore_status; // Optional, default: NULL
        CfgStep* next; // Optional, Can be NULL

I figured that each step could point to the next one, and the names would be primarily used for serialization and/or logging.

};

struct CfgWatch {
        char* name; // Required
        char** filesFilter; // Required
        char** excludesFilter; // Optional, default: NULL
        char* workingDir; // Optional, default: cwd
        CfgStep* steps; // Require at least one
};

Proposed Config Format

I've generally been inspired by nginx, bftpd, and my job's config formats, and is what the below config is based off of.

Definition

All directives are case insensitive, and strings with spaces or symbols must be quoted, unless you want to escape all symbols and spaces. WaTcH Docker{ is the same as watch "Docker" {. Values that can be a list are comma separated and do not need anything else to signify that they are a list, this means that one item will look like a normal string.

Boolean values may true, false, yes, or no and must not be quoted.

Integers must not be quoted, so if they are the start of a string, the string must be quoted. You can specify a number range by separating 2 numbers with a dash.

The end of a value is signified by a newline or a semicolon, unless it is escaped.

Watch CSS {
    files = "*.css", "*.less"
    exclude = "/css/compiled_*"
    dir = "src/html"
    step Compile {
        command = "lessc --clean-css=\"--compatibility=ie8 --advanced\" \
        less/main.less css/compiled_style.css"
    }
}
Watch "C++ Code Base" {
    files = "*.cpp", "*.hpp", "test_*.rb", "Makefile"
    exclude = "main.*", "test_driver.*"
    step Compile {
        command = "make me a sandwich"
        enabled = true
    }
    step "Build Tests" {
        command = "make test"
        enabled = yes
    }
    step "Run Ruby Tests" {
        command = "ruby tests/ruby/test_driver.rb"; enabled = false;
    }
    step "Run Tests" {
        command = "tests/cpp/test_framework"
        error_status = 5
        ignore_status = 3-9, 27
    }
    step Run {
        command = "bin/my_program"
    }
}

I plan on writing the parser myself, since all of the config libraries that I've used thus far have changed their API in some way that makes it near impossible to package on older distros (or even newer ones like CentOS 7) without shipping the other libraries with Renzoku. I would like to know what you think before I get started 😄

[carsonoid]

The on_success and on_error directives are mostly a rip off from the AJAX request handling you see in things like jQuery. I would use them for things like desktop notifications when certain steps fail or succeed. Now that I think about it, on_success is probably not as useful. But if you are going to do one you might as well do the other.

[TheTechmage]

That makes sense. I'll see about adding it sometime in the future. :)