This is an example to show how use the timeout and the input stepts to create a conditional step with determinated timeout. First, the timeout sets the time to wait until the second step complete their tasks, but the second step waits the user input to continue.
The next script waits two days until the user input is received. If no input is received the pipeline is aborted.
pipeline {
agent any
stages {
stage('Hello') {
steps {
timeout(time: 2, unit: 'DAYS') { //SECONDS | MINUTES | HOURS | DAYS
script {
def userInput =
input( id: 'input'
, message: 'Select options :'
, submitter: 'asanchez' // only this user can submit the form, comment to skip
, submitterParameter: 'submitterBy' // return the name of the user submitter
, parameters: [
//Params
booleanParam(name: 'Check1', defaultValue: false)
, [$class: 'TextParameterDefinition', defaultValue: 'Dummy Text', description: 'A simple text param', name: 'textInput']
, text(defaultValue: 'LOREM', description: '', name: 'textField')
, string(defaultValue: 'dummy text', description: '', name: 'inputText', trim: false)
, credentials(credentialType: 'com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl', defaultValue: '', description: '', name: 'credentialsParam', required: true)
, choice(choices: ['Foo', 'Bar', 'Sed'], description: '', name: 'Options')
, password(defaultValue: 'None', description: '', name: 'PasswordField')
])
echo("Env: " + userInput)
}
}
}
}
}
}
A common task in software development is the versioning of the code, in some maven projects I’ve used the maven-scm-plugin to automate the versioning and publish the tags, in my previous post I’ve explained how to configure the pom.xml in a Java project.
In this post I’m going to show how to use the same plugin with a Jenkins Pipeline, the script has a tree steps:
checkout, to get the code from a Git repository
delete tag, to delete tag in the repository
tagging, to publish a new version of code and create a new tag
In this post I’m going to show how to execute a bash script into the remote server using a Jenkins Pipeline Step using the sshPublisher plugin, in my previous post I’d showed the other way to configure without plugin.
Setup server autentication
1. Create a new key pair, login with service user for jenkins and create new key pair without passhrase (leave it in blank or you can set if you want secure the private key):
# using the service user
sudo su -s /bin/bash jenkins
# create key pair
ssh-keygen -f ~/.ssh/id_pipessh -t rsa
out:
$ ssh-keygen -f ~/.ssh/id_pipessh -t rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /c/Users/003084781/.ssh/id_pipessh
Your public key has been saved in /c/Users/003084781/.ssh/id_pipessh.pub
The key fingerprint is:
SHA256:rPIPQ8a74wwumZ7TrtXOXfKcWbcSBgUhgqRnSY9Evf4 GMX+003084781@IBM-PF398HP4
The key's randomart image is:
+---[RSA 3072]----+
| o=o. . oo |
| +.+.. . . |
| . = .. . |
| o ... . |
| .+ S . |
| +.o o |
| ++ *.. ..... |
| =+.B.=E= +.. . |
| .+=o.Boo = .. |
+----[SHA256]-----+
The private key (‘id_pipessh‘) and the public key (‘id_pipessh.pub‘) are created into ~/.ssh
I’ve upgraded my Sonarqube server from the 6.4 to 7.8 version, the last than support MySQL database, according to documentation, the Nodejs > 6.0 is required to analyse Javascript code.
But I didn’t know, until my Jenkins jobs failed. The error is clear, I need to install Nodejs:
org.sonarsource.nodejs.NodeCommandException: Error when running: 'node -v'. Is Node.js available during analysis?
I’ve installed Sonarqube over Red Hat Enterprise Linux 7 (RHEL), the process to install Nodejs is very simple. If the repository is enabled only have to execute this commands in terminal and check the version.
Recently I had to upgrade my Sonarqube server from 6.4 to 7.8 and I noticed a problem in few projects when the jacoco report is loaded by Jenkins with the jacoco-maven-plugin to Sonarqube using the plugin sonar-jacoco (1.0.1.143) , the error show this message:
[INFO] Sensor JaCoCo XML Report Importer [jacoco]
[ERROR] Coverage report '/var/lib/jenkins/workspace/test/dummy/target/site/jacoco/jacoco.xml' could not be read/imported. Error: {}
java.lang.IllegalStateException: Failed to parse JaCoCo XML report: /var/lib/jenkins/workspace/test/sge-webapp/target/site/jacoco/jacoco.xml
at org.sonar.plugins.jacoco.XmlReportParser.parse(XmlReportParser.java:96)
at org.sonar.plugins.jacoco.JacocoSensor.importReport(JacocoSensor.java:73)
at org.sonar.plugins.jacoco.JacocoSensor.importReports(JacocoSensor.java:64)
at org.sonar.plugins.jacoco.JacocoSensor.execute(JacocoSensor.java:48)
at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)
at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:400)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:395)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:392)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:73)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:67)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy43.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute(ScannerBootstrapper.java:65)
at org.sonarsource.scanner.maven.SonarQubeMojo.execute(SonarQubeMojo.java:104)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
Caused by: com.ctc.wstx.exc.WstxIOException: Input length = 1
at com.ctc.wstx.sr.StreamScanner.constructFromIOE(StreamScanner.java:633)
at com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1063)
at com.ctc.wstx.sr.StreamScanner.parseFullName2(StreamScanner.java:1994)
at com.ctc.wstx.sr.StreamScanner.parseFullName(StreamScanner.java:1952)
at com.ctc.wstx.sr.BasicStreamReader.handleNonNsAttrs(BasicStreamReader.java:3117)
at com.ctc.wstx.sr.BasicStreamReader.handleStartElem(BasicStreamReader.java:2968)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2839)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1073)
at org.sonar.plugins.jacoco.XmlReportParser.parse(XmlReportParser.java:53)
... 55 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.read1(BufferedReader.java:212)
at java.io.BufferedReader.read(BufferedReader.java:289)
at com.ctc.wstx.io.MergedReader.read(MergedReader.java:105)
at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:86)
at com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:56)
at com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1060)
... 62 common frames omitted
Even the job finished with success status the Sonarqube server don’t show the coverage report over the source code.
After some research I found the problem is because my jacoco.xml report has some characters in ISO-8859-1 that can’t be parsed by the sonar-jacoco plugin, this one load the xml file with charset in UTF-8 by default, you can check the source code in github.
I think the jacoco-maven-plugin is building the report in the right way because my project needs the charset encoding in ISO-8859-1 but the sonar-jacoco plugin only support UTF-8.
Then to resolve it there are two choices:
1. Change the output encoding for jacoco report
I don’t like this option but is the most rapid way, during the runtime the jacoco.xml report is built with the same charset encoding than sonar (sonar.sourceEncoding) but the output of jacoco can be modified with this property:
Then the inspection use correctly the encoding ISO-8859-1 but the output of the jacoco report is in UTF-8. The error in sonar-jacoco plugin is skipped but the impact of this change only affect the presentation in Sonarqube.
I downloaded the version release 1.0.2.475 and modified the source code to support other charset encoding, I’m not sure if is the best implementation but I’m gonna will send the pull request to the community. This is the class:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Because the jacoco use the same source encoding in maven, only if I set the parameter «sonar.sourceEncoding» the plugin will change the charset when the xml is loaded.
Even the plugin is not supported in my Sonarqube server, is working with my projects now.
In the previous post I’ve explained how to configure the login over ssh to execute remote commands with Jenkins with a Build Step, now it’s the same but using Pipeline Step.
Avoid login prompt
To avoid logging prompt on each execution with the pipeline script is necessary copy the ssh key to remote server.
Create a ssh key pair, login in the Jenkins server with the user that are executing the Jenkins as a service and execute the next commands:
sudo su -s /bin/bash jenkins
ssh-keygen -t -rsa
Leave the options by default, a file $HOME/.ssh/id_rsa is generated, take care if you have a previous file, you will not be able to authenticate using the previous key anymore if you choose overwrite
Copy the public key to remote server use only one of this commands:
# Using pub key by default ~/.ssh/id_rsa.pub
ssh-copy-id user@host
# Using another key
ssh-copy-id -i ~/.ssh/otherKey.pub user@host
# if ssh-copy-id is not available, do it manually
cat ~/.ssh/id_rsa.pub | ssh user@host"mkdir -p ~/.ssh && touch ~/.ssh/authorized_keys && chmod -R go= ~/.ssh && cat >> ~/.ssh/authorized_keys"
Test the connection
# Test the login with the private key
ssh -i ~/.ssh/id_rsa user@host
# Then connect without prompt password
ssh user@host
If the above steps were followed and ssh’ing to the appropriate user is still prompting for passwords, inspect the permissions on both the local and remote user’s files, per the following command:
[user@ssh-server ~]$ ls -ld ~/{,.ssh,.ssh/authorized_keys*}
drwx------. 25 user user 4096 Aug 21 11:01 /home/user/
drwx------. 2 user user 4096 Aug 17 13:13 /home/user/.ssh
-rw-------. 1 user user 420 Aug 17 13:13 /home/user/.ssh/authorized_keys
Don’t forget it, the PubkeyAuthentication yes must be enabled in the server into /etc/ssh/sshd_config
Configure the Pipeline
Create a new Pipeline project and set the next script, notice the end of the pseudo script is intentionally aligned at left, if I indent the text does not work.
pipeline {
agent any
stages {
stage('execute') {
steps {
script {
echo 'Using remote command over ssh'
sh 'echo "Today is:" date'
echo '*** Executing remote commands ***'
sh '''#!/bin/bash
date
ssh devops@hostname << ENDSSH
java -version
date
cd /tmp
pwd
ENDSSH
'''
sh '''
date
ssh devops@hostname << ENDSSH
javac -version
date
cd /tmp
pwd
ENDSSH
'''
}
}
}
}
}
Then save it and execute the job.
Checking the job log
The script have different ways to call a sh command, let’s check each one.
sh 'echo "Today is:" date'
The first is a simple sh command to get date, the interesting part is how Jenkins execute this, if see the job log each command is executed by separate, the echo first and date after.
[Pipeline] echo
Using remote command over ssh
[Pipeline] sh
[PL_REMOTE_COMMAND] Running shell script
+ echo Today is:
Today is:
+ date
mié dic 18 05:05:06 CST 2019
The second command is executed line by line, one per command:
echo '*** Executing local commands ***'
sh '''
ls -l
date
cd /tmp
pwd
'''
output:
[Pipeline] echo
*** Executing local commands ***
[Pipeline] sh
[PL_REMOTE_COMMAND] Running shell script
+ ls -l
total 0
+ date
mié dic 18 05:05:07 CST 2019
+ cd /tmp
+ pwd
/tmp
The next command use a pseudo script style to hide the commands and connect to remote server, note my servers have a different date time, that means the commands in the script are hidden in the log and only shows the result of each one:
echo '*** Executing remote commands ***'
sh '''#!/bin/bash
date
ssh devops@hostname << ENDSSH
java -version
date
cd /tmp
pwd
ENDSSH
'''
output:
*** Executing remote commands ***
[Pipeline] sh
[PL_REMOTE_COMMAND] Running shell script
mié dic 18 05:31:05 CST 2019
Pseudo-terminal will not be allocated because stdin is not a terminal.
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) Server VM (build 20.45-b01, mixed mode)
vie dic 20 18:20:22 CST 2019
/tmp
This is a good option if you don’t want to show sensible commands in other way, remove the script header to show them.
echo '*** Executing remote commands ***'
sh '''
date
ssh devops@hostname << ENDSSH
java -version
date
cd /tmp
pwd
ENDSSH
'''
output:
[Pipeline] echo
[Pipeline] sh
[PL_REMOTE_COMMAND] Running shell script
+ date
mié dic 18 05:31:05 CST 2019
+ ssh devops@hostname
Pseudo-terminal will not be allocated because stdin is not a terminal.
javac 1.6.0_45
vie dic 20 18:20:22 CST 2019
/tmp
Generic Webhook Trigger is a useful plugin for Jenkins to trigger jobs retrieving some parameters from webhook request, we can configure to do some actions like automate tasks and change their behavior.
In this example I’m gonna show how use it, if you want to install it please check the manage plugin section in your Jenkins instance.
Let’s start from the scratch: Hello Pipeline Message
Ok, first create a new Pipeline project Main page > new Item > Pipeline, and let’s start printing a simple a Hello Pipeline message. Seguir leyendo →
![2020-04-04 15_51_26-test Config [Jenkins] - Brave](https://jaehoo.wordpress.com/wp-content/uploads/2020/04/2020-04-04-15_51_26-test-config-jenkins-brave.png)
Jaehoo Weblog 