Spectral Validation of Examples

We’re leveraging OAS3 examples pretty heavily for Prism, and love that we’re getting back response validation within Prism. But, we’d really like to validate our examples with our Spectral implementation. I seen in Spectral that example are validated, but not examples. We took a crack at trying to use the schemaPath function in a similar fashion, but I suspect that we’re not able to pass in an array of examples arrays to validate against the general spec. Here’s an example of what we’ve been trying:

  examples-responses-are-valid:
    description: is valid
    recommended: true
    severity: error
    given: "$.components.responses..[?(@property !== 'properties' && @.examples)]"
    then:
      function: schemaPath
      functionOptions:
        field: "examples"
        schemaPath: "$.*.schema"

Is this possible with Spectral schemaPath, or will we need to write our own function to handle this scenario? I suspect it’s the later, since it’s not implemented as a default Spectral rule, but wanted to reach out and see if someone has been able to make this work.

Thanks for taking a look!

Hey Andrew!
My apologies for a bit delayed answer.
If I understand the problem correctly, it seems like you may in fact need to create your own custom function, as schemaPath function does not support arrays well, therefore you cannot validate each example separately. Moreover, it doesn’t allow you to specify a JSON path expression in field, which might be seen as a bug, as all other functions let you do so.
I’ll add a support for both inside of Spectral.
Will post the link to the issue here, so that you can fork the function and put in your ruleset until we release the change.

1 Like

Just wanted to let you know the PR is up https://github.com/stoplightio/spectral/pull/917
Once it’s merged and released, the rule you posted could be changed to:

examples-responses-are-valid:
  message: "{{error}}"
  severity: error
  given: "$.components.responses..[?(@property !== 'properties' && @.examples)]"
  then:
    function: schemaPath
    functionOptions:
      field: "$.examples.*"
      schemaPath: "$.schema"
1 Like

I pulled develop and gave it a whirl. I’m seeing some validation happening now. I’ll keep investigating to see if there are any issues, but these changes help a ton, thank you so much!

I used the field parameter as mentioned https://github.com/stoplightio/spectral/issues/916#issuecomment-574652305

  examples-responses-are-valid:
    message: "{{error}}"
    severity: error
    given: "$.components.responses..[?(@property !== 'properties' && @.examples)]"
    then:
      function: schemaPath
      functionOptions:
        field: "$.examples.*.value"
        schemaPath: "$.schema"