Automatically set the microgrid API host/port

[leandro-lucarella-frequenz]

What's needed?

We need the SDK to be able to connect to the microgrid API without SDK users passing them explicitly.

Proposed solution

Load the config from a file. The config file could come from the UI or not depending on if we want to allow this or not. If we don't allow it, we need to have a mechanism to make sure this config file is included when deploying apps.

The user might be able to override this host/port, but it is still undecided. @thomas-nicolai-frequenz mention we should consider the security implications of this before enabling it.

We could also use env variables for this.

Use cases

No response

Alternatives and workarounds

No response

Additional context

This could supersede #40.

[daniel-zullo-frequenz]

Probably we should align this issue with #191 which is still under discussion

[thomas-nicolai-frequenz]

We need the SDK to be able to connect to the microgrid API without SDK users passing them explicitly.

Whats on my mind here is that user are able to simple configure an app to run against different API endpoints. On a local test system the user might want to point to the cloud sandbox whereas after deployment it points to localhost on the edge controller. However, by default the host and port should be set in the SDK so that there is no config required. The config would only be used as an explicit override imho.

[leandro-lucarella-frequenz]

I think both play well together. The config at the end will come from a config file, which might be written by the cloud sync (production) or by the developer manually (testing).

I'm still not sure if (UI) users should be able to change the host/port of the microgrid API via the UI, if they don't, then we don't even have to go through the config system.

[thomas-nicolai-frequenz]

I'm still not sure if (UI) users should be able to change the host/port of the microgrid API via the UI

hell no. SECURITY ALERT! :-)

[leandro-lucarella-frequenz]

The more i look at this the more i think this should come from env variables. If there is no UI config there is no strong requirement for this to be in s config file, then env variables are the simplest option and if it becomes annoying we can add support to load them from an .env file.

Then when can inject the values for each microgrid when building or running the docker image and we are set.

[mathias-baumann-frequenz]

Sounds like a good first step at least

[thomas-nicolai-frequenz]

then env variables are the simplest option and if it becomes annoying we can add support to load them from an .env file.

Is that convenient for a user that wants to use the SDK locally on his computer without a container against the sandbox and/or if he uses the container locally? I'm not disagreeing just trying to understand how this could potentially work in a local environment / sandbox test environment. What about that if being given in the env variables it would override the default in the SDK or the other way around?

[leandro-lucarella-frequenz]

then env variables are the simplest option and if it becomes annoying we can add support to load them from an .env file.

Is that convenient for a user that wants to use the SDK locally on his computer without a container against the sandbox and/or if he uses the container locally?

Yes, very much. And pretty standard.There are plenty of tools to even set these automatically, from cli tools (like direnv, where you can configure a directory to automatically load certain env variables when you cd to it) to support for loading dotenv files directly in probably every possible language (js, py, ruby, c, rust, etc.).

I'm not disagreeing just trying to understand how this could potentially work in a local environment / sandbox test environment. What about that if being given in the env variables it would override the default in the SDK or the other way around?

The environment variable always wins. I'm not even sure if the SDK should provide any default. What would be a sensible default? localhost and some port we chose? Maybe, but I would be happy even if the SDK refuses to start if it's started without either the app passing the host, port in microgrid.initialize() or if the env variables are not present.

[leandro-lucarella-frequenz]

So, what I suggest is to keep the current initialize() but making the host and port optional. The user at the end wants to run some App, not the SDK, so the user could even decide to hardcode the host and port when calling initialize() for testing, but when this is deployed, we can still inject the correct host and port as defined by the controller (or microgrid configuration when we have more than one controller and the microgrid services runs only in one of these controllers) when deployed for real.

So the SDK initialize() should do something like:

def initialize(host = None, port = None, ...):
    if host is None:
        host = os.getenv("MICROGRID_HOST")
        if host is None:
            raise InitializationError("Missing MICROGRID_HOST env variable")
    if port is None:
        port = os.getenv("MICROGRID_PORT")
        if port is None:
            raise InitializationError("Missing MICROGRID_PORT env variable")
        port = int(port)  # Also raise if it's not an int
    ...

Alternatively, if we really want to have a SDK provided default:

DEFAULT_MICROGRID_HOST = 'localhost'
DEFAULT_MICROGRID_PORT = '9999'

def initialize(host = None, port = None, ...):
    if host is None:
        host = os.getenv("MICROGRID_HOST", DEFAULT_MICROGRID_HOST)
    if port is None:
        port = os.getenv("MICROGRID_PORT", DEFAULT_MICROGRID_PORT)
        port = int(port)
    ...

[thomas-nicolai-frequenz]

localhost and some port we chose

exactly, yes.

[thomas-nicolai-frequenz]

Maybe, but I would be happy even if the SDK refuses to start if it's started without either the app passing the host, port in microgrid.initialize() or if the env variables are not present.

yeah, also a way to do it.