Mocked request with response fails if host cannot be resolved.

[balazser]

Subject of the issue

The mocked request fails if the target host does not exist.

According to the doc
A request handler is triggered when the client makes a request. This occurs before anything is sent to the server

In practice it seems the host gets somehow resolved

Expected behaviour

In case of a match the request returns with value regardless the host.

Actual behaviour

Error message

502 Bad Gateway

ProtocolException('Server connection to (\'nodomain123.com\', 443) failed: Error connecting to "nodomain123.com": [Errno 111] Connection refused')

Steps to reproduce

Use the following config.

mitmproxy -s moxy.py --set mock=config/example.json

{ 
 "request": {
   "/part1/part2/part3": {
     "respond": {
       "content": {
         "foo": 15,
         "bar": "M5",
       }
     }
   }
 }
}

export https_proxy="http://127.0.0.1:8080"
curl https://nodomain123.com/part1/part2/part3 # fails
curl https://google.com/part1/part2/part3 # works

[arkku]

I suspect this might be a property of mitmproxy, but I'll look into it anyway

[balazser]

Thank you @arkku

[arkku]

@balazser Just as an update, I am able to reproduce this, and indeed it is a strange behaviour by mitmproxy that it acts as though an error response was received, but the request handler is not called. If you enable debug logging with --set console_eventlog_verbosity=debug --set termlog_verbosity=debug arguments to mitmproxy or mitmdump, you can see that the Moxy Response: log is output for the non-existent domain, but the Request: log is not.

My initial thought was to simply work around this by providing a "response" handler for the same path, but here the response doesn't even have a path associated with it, and trying to respond to the empty path crashes mitmproxy:

{
  "request": {
    "/part1/part2/part3": {
      "respond": {
        "content": "./data/foo.json"
      }
    }
  },
  "response": {
    "": {
      "status": 502,
      "replace": {
        "status": 200,
        "type": "application/json",
        "content": "./data/foo.json"
      }
    }
  }
}

So this seems unfixable from the script side (at least without deeper knowledge of what would be a valid way to override the error). I suggest posting an issue for mitmproxy itself. You can reproduce this with a simple script that just has something like:

def response(flow: http.HTTPFlow) -> None:
    ctx.log.debug("Response {}: {}".format(flow.request.path, flow.response))