Stoplight integration with Jenkins


(Nicolas Tisserand) #1

Hello,

I’m pleased to share with you my Jenkins plugin for Stoplight.
This plugin is published on the Jenkins Github : https://github.com/jenkinsci/stoplightio-report-plugin

Summary :

As you know, it is possible to launch a scenario or collection either in commandline or by requesting the prism url. As result, you’ll get the console log (or a JSON file).
My plugin anlyzes these outputs to extract the useful data and show them in the Jenkins UI by providing graphs and dashboard-like directly available from the Jenkins job.

How to install it
The plugin is referenced in the Jenkins Update Centre, so :

  • Log in Jenkins as administrator

  • In the left menu, choose “Manage Jenkins” and then “Manage Plugin”, tab “Avalaible” and search for “Stoplight Report” :

  • Check the box and complete the installation (don’t forget to restart Jenkins)

How to use it

  • Go to your Jenkins job configuration

  • In “Build” section, add a “Execute shell” or “Execute Windows batch command” depending on your OS.

  • Run the scenario by using the commandline or the url provided by Stoplight. For example :


    Of course, if you choose the commandline, please ensure that prism or prism.exe binary is available on your Jenkins server with the execution rights. (You can even install it with curl or wget in another “shell” or “windows batch”).

  • Just below, add now a post-build action. Search for “Publish Stoplight Report” in the dropdown list and configure it :
    image

  • Build your job and enjoy the reports from the Jenkins menu entries :

Trend graph on the job main page :

Builds History :


(The history depth depends on the “Discard old builds” option of your job)

Details for a specific build :

To go further
This plugin is also compliant with the new Jenkins pipeline feature :
You can use the Snipper Generator or this fragment :

...
...
post {
    always {
        // Choose one of these options

        //publishStoplight consoleOrFile: 'console'
        publishStoplight consoleOrFile: 'file', resultFile: "${env.WORKSPACE}/prism.log"   // Use double-quote to have env variables replacement
    }
}        

Please feel free to use, share, and give feedback here !

Regards.
Nicolas.


Fail scenario step if connected spec operation/response not found
(Kmeister) #2

Nice integration. One feature that would be nice is if the parser/html reporter is moving to its own repo, so other build systems would take advantage.

It looks like you read the text log file, did you look at the prisim --reporters flag? I see it mentioned in the prism docs.


(Nicolas Tisserand) #3

Hi @kmeister ,

What do you mean exactly by moving to its own repo ? Do you want that the parser have its own Github repo ?
If yes, it’s not planned to extract the parser for the moment. At the beginning, I wanted only to support Jenkins for my company.
If you want the parser, it’s developed in Java. Just pick it up from here (and take the package model. If you look in the plugin sources, you’ll see that Jenkins code is very rare (only 5 classes).
Let me know if I’ve not correctly understood what you mean.

Regarding to the prism --reporters, it’s relative to the test option and it’s not supported anymore by recent prism release (>= 2.x.x). Moreover, I don’t really understand the usage of this old command.
Hey @bear and @robbins, it’s perhaps time to update this page : https://help.stoplight.io/prism/getting-started/commands (and also other docs about prism) :wink:


(Amal Jayaraj) #4

Hi @ntiss,
Thanks for the plugin. The reports look really nice and readable.
Can you please explain how you are able to generate a prism log so that it can be passed to the plugin? Currently, I tried with the publishStoplight consoleOrFile: 'console' option and it gives mixed results. Even though I have 41 tests running the plugin shows random data. Running the same collection again provides a different data.
I am not aware of any command to generate a prism log which I can pass to the plugin to publish results.
I am attaching a screenshot of the random data shown by the plugin here.


(Nicolas Tisserand) #5

Hi @amal_jayaraj

Indeed, your report looks weird.
By choosing “console”, the plugin parses the Jenkins console with regexps. This parsing can fail if there are lines before or after prism execution that are looking like prism result : regexp catches all.

As a workaround, you can redirect the prism output into a file, like this :

prism conduct "https://exporter.stoplight.io/XXXXXX/scenarios.yml" --env "host=http://....." 2>&1 | tee ${WORKSPACE}/prism.log ; test ${PIPESTATUS[0]} -eq 0
(the test ${PIPESTATUS[0]} -eq 0 is here to return the exit code of prism and not the one of tee)

Then you can run the plugin against the prism.log file :

publishStoplight consoleOrFile: 'file', resultFile: "${env.WORKSPACE}/prism.log"

Let me know if it works for you.
If not, I propose you to send me your Jenkins console output in PM on this forum (and if it doesn’t contains any sensitive data). Can you also precise which version of prism you are running.

Regards.


(Amal Jayaraj) #6

Hi @ntiss,

Thanks for that! Your command worked. The published result has become consistent and is displaying the correct values.
My Prism version: v2.0.10

Cheers!
Amal


(Nicolas Tisserand) #7

Fine, thanks for the feedback.