Contract testing not working


(Thomas De Groof) #1

Hi there,

I’ve created a very simple API and specification.
When writing a test scenario linked to the specification file, the endpoint is correctly found.

But, as seen in the screenshot below, when changing a property in the spec (“completed” to “compleeeeeeted”), the test does not fail because of incorrect contract test.

Am I missing any extra steps to make this work?
Any idea what might be wrong?

Thanks in advance,
Thomas


(Nicolas Tisserand) #2

(Almost) Same behaviour on my side.
I tried to reproduce a similar test. The document is still valid even after modifying the model.
Smells like a bug ?


(Taylor Barnett) #3

Interesting… I’m looking into it.


(Taylor Barnett) #4

@thomas.degroof I’m having a bit of trouble replicating this.

I took the example todos API and which had a passing Scenario. I changed one of the property names, ran it again, and it failed like it should.

I’m trying to think of other reasons why it would pass without failure. Could you send me some info on your project in a message or support@stoplight.io?


(Thomas De Groof) #5

Private message sent!


(Taylor Barnett) #6

I’ve followed up with both of y’all after a bit more digging.


(Taylor Barnett) #7

Hey everyone - I got this resolved with @ntiss and have a private message going with @thomas.degroof, so I am going to go ahead and close this.

For anyone else seeing this, make sure to watch if you have not required properties and are allowing additionalProperties. This is cause situations like this. The solution really depends on desired API behavior. Happy to chat about it more in another thread!


(Taylor Barnett) #8