newmanExecute¶

Installs newman and executes specified newman collections.

Deprecation notice

Details of changes after the step migration to a golang can be found below.

Description¶

This script executes Postman tests from a collection via the Newman command line tool.

Usage¶

We recommend to define values of step parameters via .pipeline/config.yml file.
In this case, calling the step is essentially reduced to defining the step name.
Calling the step can be done either in an orchestrator specific way (e.g. via a Jenkins library step) or on the command line.

JenkinsCommand Line

library('piper-lib-os')

newmanExecute script: this

piper newmanExecute

Outputs¶

Output type	Details
influx	measurement `step_data` newman

Prerequisites¶

prepared Postman with a test collection

Parameters¶

Overview - Step¶

Name	Mandatory	Additional information
script	(yes)	reference to Jenkins main pipeline script
cfAppsWithSecrets	no
failOnError	no
newmanCollection	no
newmanEnvironment	no
newmanGlobals	no
newmanInstallCommand	no
newmanRunCommand	no
runOptions	no
verbose	no	activates debug output

Overview - Execution Environment¶

Orchestrator-specific only

These parameters are relevant for orchestrator usage and not considered when using the command line option.

Name	Mandatory	Additional information
containerCommand	no
containerShell	no
dockerEnvVars	no
dockerImage	no
dockerName	no
dockerOptions	no
dockerPullImage	no
dockerVolumeBind	no
dockerWorkspace	no
stashContent	no

Details¶

cfAppsWithSecrets¶

Define name array of cloud foundry apps deployed for which secrets (clientid and clientsecret) will be appended to the newman command that overrides the environment json entries (--env-var =${clientid} & --env-var =${clientsecret})

back to overview

Scope	Details
Aliases	-
Type	`[]string`
Mandatory	no
Default	`$PIPER_cfAppsWithSecrets` (if set)
Secret	no
Configuration scope	☒ parameter ☐ general ☒ steps ☒ stages
Resource references	none

containerCommand¶

Jenkins-specific: Used for proper environment setup.

Kubernetes only: Allows to specify start command for container created with dockerImage parameter to overwrite Piper default (/usr/bin/tail -f /dev/null).

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

containerShell¶

Jenkins-specific: Used for proper environment setup.

Allows to specify the shell to be executed for container with containerName.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerEnvVars¶

Jenkins-specific: Used for proper environment setup.

Environment variables to set in the container, e.g. [http_proxy: "proxy:8080"].

back to overview

Scope	Details
Aliases	-
Type	`map[string]string`
Mandatory	no
Default
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerImage¶

Jenkins-specific: Used for proper environment setup.

Name of the docker image that should be used. If empty, Docker is not used and the command is executed directly on the Jenkins system.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default	`node:lts-buster`
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerName¶

Jenkins-specific: Used for proper environment setup.

Kubernetes only: Name of the container launching dockerImage. SideCar only: Name of the container in local network.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default	`newman`
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerOptions¶

Jenkins-specific: Used for proper environment setup.

Docker options to be set when starting the container.

back to overview

Scope	Details
Aliases	-
Type	`[]string`
Mandatory	no
Default
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerPullImage¶

Jenkins-specific: Used for proper environment setup.

Set this to 'false' to bypass a docker image pull. Useful during development process. Allows testing of images which are available in the local registry only.

back to overview

Scope	Details
Aliases	-
Type	`bool`
Mandatory	no
Default	`true`
Possible values	- `true` - `false`
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerVolumeBind¶

Jenkins-specific: Used for proper environment setup.

Volumes that should be mounted into the docker container.

back to overview

Scope	Details
Aliases	-
Type	`map[string]string`
Mandatory	no
Default
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

dockerWorkspace¶

Jenkins-specific: Used for proper environment setup.

Kubernetes only: Specifies a dedicated user home directory for the container which will be passed as value for environment variable HOME.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default	`/home/node`
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

failOnError¶

Defines the behavior, in case tests fail.

back to overview

Scope	Details
Aliases	-
Type	`bool`
Mandatory	no
Default	`true`
Possible values	- `true` - `false`
Secret	no
Configuration scope	☒ parameter ☐ general ☒ steps ☒ stages
Resource references	none

newmanCollection¶

The test collection that should be executed. This could also be a file pattern.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default	`*/.postman_collection.json`
Secret	no
Configuration scope	☒ parameter ☐ general ☒ steps ☒ stages
Resource references	none

newmanEnvironment¶

newmanGlobals¶

newmanInstallCommand¶

The shell command that will be executed inside the docker container to install Newman.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default	`npm install newman newman-reporter-html --global --quiet`
Secret	no
Configuration scope	☒ parameter ☐ general ☒ steps ☒ stages
Resource references	none

newmanRunCommand¶

+++ Deprecated +++ Please use list parameter runOptions instead.

back to overview

Scope	Details
Aliases	-
Type	`string`
Mandatory	no
Default	`$PIPER_newmanRunCommand` (if set)
Secret	no
Configuration scope	☒ parameter ☐ general ☒ steps ☒ stages
Resource references	none

runOptions¶

The newman command that will be executed inside the docker container. Env vars can be passed via template as in "{{getenv MY_ENV_VAR}}".

back to overview

Scope	Details
Aliases	-
Type	`[]string`
Mandatory	no
Default	- `run` - `{{.NewmanCollection}}` - `--reporters` - `cli,junit,html` - `--reporter-junit-export` - `target/newman/TEST-{{.CollectionDisplayName}}.xml` - `--reporter-html-export` - `target/newman/TEST-{{.CollectionDisplayName}}.html`
Secret	no
Configuration scope	☒ parameter ☐ general ☒ steps ☒ stages
Resource references	none

script¶

Jenkins-specific: Used for proper environment setup.

The common script environment of the Jenkinsfile running. Typically the reference to the script calling the pipeline step is provided with the this parameter, as in script: this. This allows the function to access the commonPipelineEnvironment for retrieving, e.g. configuration parameters.

back to overview

Scope	Details
Aliases	-
Type	`Jenkins Script`
Mandatory	yes
Default
Secret	no
Configuration scope	☐ parameter ☐ general ☐ steps ☐ stages
Resource references	none

stashContent¶

Jenkins-specific: Used for proper environment setup.

Specific stashes that should be considered for the step execution.

back to overview

Scope	Details
Aliases	-
Type	`[]string`
Mandatory	no
Default	- `tests`
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

verbose¶

verbose output

back to overview

Scope	Details
Aliases	-
Type	`bool`
Mandatory	no
Default	`false`
Possible values	- `true` - `false`
Secret	no
Configuration scope	☒ parameter ☒ general ☒ steps ☒ stages
Resource references	none

Side effects¶

Step uses dockerExecute inside.

Exceptions¶

The step has been migrated into a golang-based step. The following release notes belong to the new implementation:

newmanRunCommand:

The parameter newmanRunCommand is deprecated by now and is replaced by list parameter runOptions. For backward compatibility, the newmanRunCommand parameter will still be used if configured. Nevertheless, using this parameter can break the step in some cases, e.g. when spaces are used in single quoted strings like spaces in file names. Also Groovy Templating is deprecated and now replaced by Go Templating. The example show the required changes:

# deprecated groovy default
newmanRunCommand: "run '${config.newmanCollection}' --environment '${config.newmanEnvironment}' --globals '${config.newmanGlobals}' --reporters junit,html --reporter-junit-export 'target/newman/TEST-${collectionDisplayName}.xml' --reporter-html-export 'target/newman/TEST-${collectionDisplayName}.html'"

# new run options using golang templating
runOptions: ["run", "{{.NewmanCollection}}", "--environment", "{{.Config.NewmanEnvironment}}", "--globals", "{{.Config.NewmanGlobals}}", "--reporters", "junit,html", "--reporter-junit-export", "target/newman/TEST-{{.CollectionDisplayName}}.xml", "--reporter-html-export", "target/newman/TEST-{{.CollectionDisplayName}}.html"]

If the following error occurs during the pipeline run, the newmanRunCommand is probably still configured with the deprecated groovy template syntax:

info newmanExecute - error: collection could not be loaded info newmanExecute - unable to read data from file "${config.newmanCollection}" info newmanExecute - ENOENT: no such file or directory, open '${config.newmanCollection}'

newmanEnvironment and newmanGlobals:

Referencing newmanEnvironment and newmanGlobals in the runOptions is redundant now. Both parameters are added to runCommand using newmanEnvironment and newmanGlobals from config when configured and not referenced by go templating using "--environment", "{{.Config.NewmanEnvironment}}" and "--globals", "{{.Config.NewmanGlobals}}" as shown above.

Passing Credentials¶

If you need to pass additional credentials you can do so via environment variables. This is done via templating in the runOptions, as per this example:

runOptions: [
    "run", "{{.NewmanCollection}}",
    "--environment", "{{.Config.NewmanEnvironment}}",
    "--env-var", "username={{getenv \"PIPER_TESTCREDENTIAL_USERNAME\"}}",
    "--env-var", "password={{getenv \"PIPER_TESTCREDENTIAL_PASSWORD\"}}"
]

Example¶

Pipeline step:

newmanExecute script: this

This step should be used in combination with testsPublishResults:

newmanExecute script: this, failOnError: false
testsPublishResults script: this, junit: [pattern: '**/newman/TEST-*.xml']