turn on/off response model validation for specific routes

[seanpk]

there are several ways to solve this problem ...

here it is: after turning on response model validation for my project I've found that some responses are acceptably invalid and I'd like to keep having validation done for other API methods without filling up my logs with known and acceptable errors

some potential solutions (I'm sure there are more):

• provide a whitelist and/or blacklist config for excluding (or opting-in) API methods
• provide an error matcher config for exclusion (so that you could turn off only matching invalid responses, not everything

[seanpk]

@dbleakley what do you think?

now that I've had it in my subconsious for a few hours, I think some sort of exclusion matcher would be the most expressive

[dbleakley]

Hey @seanpk,

Yeah, I think the matcher option would be work well. A couple of considerations around implementation.

• For the set of routes that will be excluded, we should make sure we don't register a response model validation handler on the route, as opposed to registering a global handler and then by passing any action based on the blacklist.
• We should consider following the expressjs path expressions for the exclusion matcher.

Also what about allowing the configuration be established in the swagger specification via an x-* extension on the path registration.

• It is a customization feature or the route and since it affects the payload in some configurations, we may also consider surfacing the setting via a swagger UI of the API Documentation.

[ryshep111]

Why not allow both options for using an exclusion matcher on either the whole path or just some errors within that path?

[seanpk]

some thoughts ... looks like a solution starts to come into view towards the end
(numbered for reference)

1. putting the control on the swagger path definition sounds great
2. except this is really a development time feature and it seems "off" to include it in the API definition - like it's muddying the waters
3. regarding opting in certain routes (vs only having the global opt-in with a way to disable the behavior for certain responses):
   1. this isn't required for the scenario that I have right now (which lead me to open this issue)
   2. however, I can imagine cases where that support could be useful
   3. still, I'm concerned that we'd add a whole additional set of code paths that wouldn't provide anymore expressivity (only making it easier for some user with a situation that may never come to exist)
   4. so, I think we stick to a global on-off switch and some way of expressing malformed responses to ignore
4. how to ignore things? (conceptually)
   1. the simplest way is to exclude all responses from a given path (potentially with support for some sort of pattern matching)
   2. the next level of specificity is to exclude all responses from the give method(s) of a path (e.g. qualifying that it we should only ignore the response from the GET method, which would mean that we'd still validate those from the POST)
   3. the ultimate specificity is to ignore certain (parts of) responses from a method, some examples:
      1. an enumeration value that is not valid as input to the API (and isn't included in the model definition so that it can't be used in creating new resources), but could show up in the response model (likely because the model was constructed out of legacy data)
      2. some specific mock/test data that excludes data because its unnecessary for the tests
5. tbd: how to configure the "ignores" in the BOS config

@dbleakley, @ryshep111, what do you think?
@dannylevenson, you may also be interested