"not" assertions: Threat? Or menace?

(Kerrykimbrough) #1

This is a question about the problem of the “not” keyword. Or maybe, for you, it’s not a problem at all. That’s sort of the question.

I’ve posted here before about the problem of “type-ambiguous” schema definitions, i.e. schemas that do not define the “type” keyword. The “not” keyword can present the same kind of problem. No matter what the “not” schema is, it can validate almost any type of instance — almost any type of instance can NOT be that thing. Such type-ambiguity implies a much more complex API contract. Any implementation that is not prepared to handle all possible valid instances could no longer rely on schema validation to screen out errors.

But “not” can be used in a way that is unambiguous, if it is coupled with a schema that is type-specific. For example, consider this schema:

{ “type”: “integer”, “minimum”: 0, “not”: { “multipleOf”: 3 }}

This schema validates only integer instances, and the “not” schema acts to narrow the set of valid integers.

So here’s the question: Does anyone actually exploit “not” in a type-ambiguous way? Or do you always use it only in the “narrowing” form as shown above? I’m asking because I’m developing tools to automatically translate OpenAPI specs into test cases. Do these test cases need to verify all the possibilities allowed by an ambiguous type schema? Or should these tools flag ambiguous schema types with an error or a warning?

(Taylor Barnett) #2

@kerrykimbrough I like the title! :laughing: I haven’t put a ton of thought into the not keyword in this situation. But I am talking to some of my teammates about it soon. Hopefully someone will more of an opinion can jump in here.

(Me) #3

@kerrykimbrough hey there! I’m not sure I understand what the trouble is.

A not keyword is just like a ! in a conditional for most programming languages, it takes whatever validation rules are inside it and inverts them.

So using your example:

{ "type": "integer", "minimum": 0, "not": { "multipleOf": 3 }}

0 fails (needs to be above 0), 1 ok, 2 ok, 3 fails because the integer must not be a multiple of 3.

Change it to this:

{ "type": "integer", "minimum": 3, "not": { "multipleOf": 3 }}

Now 1 and 2 fail because they have not hit the minimum. 4 and 5 are ok, but 6 triggers the not validation error.

If you try and validate not an integer, then the type will fail.

should be integer.
type at "#/type"`

So that all seems great. Where does the threat or menace bit come in? How is this ambiguous, and what sort of exploitation are you imagining?

If you’d like to write a Spectral rule that throws warnings telling people not to use not then you could certainly do that. :slight_smile:

(Kerrykimbrough) #4

@philsturgeon Agreed, you are referring to my example of how to use “not” in a way that avoids the problem of type-ambiguity. Because it acts to “narrow” a parent schema that is type-specific.

But consider this example:

{ "not": { "uniqueItems": true}}

This schema is type-ambiguous. It will validate virtually any instance of any type. If you use this kind of schema in your API spec, you have to be prepared to handle the resulting contract complexity. My question is this: Does anyone actually rely on “not” schemas like this?

(Me) #5

I’m not equipped to speak on behalf of everyone, but speaking anecdotally this is not something I see people doing much of in the wild.

That said, tooling should support:

{ "uniqueItems": true}

and

{ "not": { "uniqueItems": true}}

Neither is any more complex or ambiguous. Neither schema care what type it is, and not has no impact on that. It’s just a branch which gets inverted after evaluation is done.

type is one of many validation properties which may or may not be there. No need to worry about it too much.