Docker, image layering/caching in action

This is a quick entry to show how the image layers works, according to docker documentation the «The order of Dockerfile instructions matter» and it is because each line in the configuration file is converted into an image layer (cached layer).

Docker image layers, taken from https://docs.docker.com/build/guide/layers/

This feature allows that every layer can be reused when the image is built or executed, each layer is read-only. For example, create the next python app and the next Dockerfile:

Seguir leyendo

Podman, basic steps for a Java project

Podman is an open source utility that can be used to create and mantain containers, and provides an good alternative to Docker.

In this entry I’m going to show how to execute some basic commands using a Java project. All source code is published in my GitHub repository.

Virtual machine management commands

Let’s take a look to the commands with the help parameter, I think they are very descriptive:

$ podman machine --help 
Manage a virtual machine

Description:
  Manage a virtual machine. Virtual machines are used to run Podman.

Usage:
  podman.exe machine [command]

Available Commands:
  info        Display machine host info
  init        Initialize a virtual machine
  inspect     Inspect an existing machine
  list        List machines
  rm          Remove an existing machine
  set         Sets a virtual machine setting
  ssh         SSH into an existing machine
  start       Start an existing machine
  stop        Stop an existing machine
Seguir leyendo

Docker, using bind mounts in a Java project

The bind mounts in a short is way to set a mount point from the host to the container, it means the files are accesible from the file system where the container is launched, for example it could be useful to mount the source code for a web site into the container and do some code changes in the UI and view them inmediatly without build again the Docker image.

Cool right?

Well for Java projects is not so cool, because when the source code is changed it must be compiled and packaged again to be executed or deployed, but the bind mounts could be useful too when there are some external files like configuration properties, input and output files to read, etc. So, if you are working on development environment or in your local machine, perhaps you would have to consider use the bind mounts to get access easily to the files when you are coding or testing new features.

In this entry I’m going to show how to apply in a Maven Java project. All code can be downloaded from my GitHub repository.

Seguir leyendo

Jenkins, Pipeline to versioning maven project with a Git Repository

A common task in software development is the versioning of the code, in some maven projects I’ve used the maven-scm-plugin to automate the versioning and publish the tags, in my previous post I’ve explained how to configure the pom.xml in a Java project.

In this post I’m going to show how to use the same plugin with a Jenkins Pipeline, the script has a tree steps:

  • checkout, to get the code from a Git repository
  • delete tag, to delete tag in the repository
  • tagging, to publish a new version of code and create a new tag

This is the code:

#!/usr/bin/env groovy

pipeline {
    agent any
    parameters {
        string(name: 'mvn_jdk', defaultValue: 'oraclejdk-1.8')
        string(name: 'mvn_version', defaultValue: 'maven-3.6.3')
        string(name: 'mvn_settings', defaultValue: 'custom-mvn-settings')
        string(name: 'branchName', defaultValue: 'feature/newBranch1', description: 'branch or tag name')
        string(name: 'repositoryUrl', defaultValue: '', description: 'git repository url')
        booleanParam(name: 'simulation', defaultValue: true, description: 'simulation mode?')
        booleanParam(name: 'removeTag', defaultValue: false, description: 'remove tag?')
    }
    stages {
        stage('checkout') {
            steps {
                
                script {
                    
                    echo repositoryUrl
                    
                    // clean ws before to start
                   cleanWs cleanWhenAborted: false, deleteDirs: true, notFailBuild: true,
                            patterns: [[pattern: '*', type: 'INCLUDE']]

                    // git checkout
                    checkoutSource()
                }
            }
        }
        stage('delete tag') {
            when{
                expression { return removeTag ==~ true }
            }
            steps {
                script{

                    tagName=input(id:'delTagField'
                        , message:'tag name to remove?'
                        , submitter: "asanchez"
                        , parameters:[
                                [$class: 'TextParameterDefinition'
                            , defaultValue: ''
                            , description: 'name of the tag to delete'
                            , name: 'tagName']
                            ]
                        )
                   
                    echo 'git tag -d '+tagName

                    withCredentials([usernamePassword(credentialsId: env.REPO_CREDENTIALS
                    , passwordVariable: 'GIT_PASSWORD'
                    , usernameVariable: 'GIT_USERNAME')]) {
                            
                            inMaven 'mvn org.apache.maven.plugins:maven-scm-plugin:1.12.2:untag \
                                     -DpushChanges=true \
                                     -Dtag='+tagName+'\
                                    "-Dmessage=Jenkins delete tag" \
                                    -Dusername=${GIT_USERNAME} -Dpassword=${GIT_PASSWORD} \
                                    -DdeveloperConnectionUrl=scm:git:[push=]'+repositoryUrl, mvn_settings
                        } 
                }
              
            }
        }
        stage('tagging') {
            steps {
                script{

                    sh 'git branch -l'
                    
                                     
                   withCredentials([usernamePassword(credentialsId: env.REPO_CREDENTIALS
                    , passwordVariable: 'GIT_PASSWORD'
                    , usernameVariable: 'GIT_USERNAME')]) {
                            inMaven 'mvn release:prepare -P scm-release -DskipTests -DdryRun=${simulation} -Darguments=-DskipTests \
                                        -DpreparationGoals=validate \
                                        -Dusername=${GIT_USERNAME} -Dpassword=${GIT_PASSWORD} \
                                        -DdeveloperConnectionUrl=scm:git:'+repositoryUrl, mvn_settings
                                        
                            inMaven 'mvn org.apache.maven.plugins:maven-scm-plugin:1.12.2:checkin -DpushChanges=true \
                                        "-Dmessage=Jenkins publish tag" \
                                        -Dusername=${GIT_USERNAME} -Dpassword=${GIT_PASSWORD} \
                                        -DdeveloperConnectionUrl=scm:git:[push=]'+repositoryUrl, mvn_settings
                                        
                        }   
                }
            }
        }
    }
    post{
        success{
            echo 'FinisheD!!'
            sh 'git log --pretty=oneline -5'
            sh 'git tag -l'
            git changelog: false, credentialsId: env.REPO_CREDENTIALS, poll: false, url: repositoryUrl
            
        }
    }
}

def checkoutSource(){

    echo 'branch to build:' + branchName;

    if(branchName == ''){
        currentBuild.result = 'FAILURE'
        error "Invalid branch name"
    }
    else{

        checkout scm: [$class: 'GitSCM'
                        , branches: [[name: branchName]]
                        , extensions: [
                                        [$class: 'LocalBranch' , localBranch: branchName] ,
                                        [$class: 'CloneOption', honorRefspec: false, noTags: false, shallow: false]
                                    ]
                        , userRemoteConfigs: [
                                                [url: repositoryUrl
                                                , credentialsId: env.REPO_CREDENTIALS
                                                , name: 'origin'
                                                , refspec: '+refs/heads/'+branchName
                                                ]]
                        ], poll: false

    }

}

def inMaven(command, settings) {

    withMaven( jdk: mvn_jdk,  maven: mvn_version, mavenSettingsConfig: mvn_settings,
            options: [jacocoPublisher(disabled: true),
                      junitPublisher(disabled: true),
                      openTasksPublisher(disabled:true)]){
        sh command
    }
}

Seguir leyendo

Compact VHD file in Windows 10 Home

I found a great tip in GitHub to compact a VHD file without Hyper-V tools and it works great. The Dynamic VHD don’t reduce their size even if the files were deleted from the hard drive, to fix it the «compact» operation is needed. This is included into the Hyper-V tools but this is only available in Windows 10 Pro.

The Compact operation is used to optimize the files. This operation reclaims unused blocks as well as rearranges the blocks to be more efficiently packed, which reduces the size of a virtual hard disk file.

Microsoft documentation

There are another way to do this in Windows 10 Home, using the next commands:

diskpart

# Open window Diskpart 

select vdisk file="C:\path\to\file.vhd"
attach vdisk readonly
compact vdisk
detach vdisk
exit

With this I’ve recovered arround 20 GB of space in my physical hard drive.

See ya.

References

Jenkins, pipeline execute script over ssh

In this post I’m going to show how to execute a bash script into the remote server using a Jenkins Pipeline Step using the sshPublisher plugin, in my previous post I’d showed the other way to configure without plugin.

Setup server autentication

1. Create a new key pair, login with service user for jenkins and create new key pair without passhrase (leave it in blank or you can set if you want secure the private key):

# using the service user
sudo su -s /bin/bash jenkins

# create key pair 
ssh-keygen -f ~/.ssh/id_pipessh -t rsa

out:

$ ssh-keygen -f ~/.ssh/id_pipessh -t rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /c/Users/003084781/.ssh/id_pipessh
Your public key has been saved in /c/Users/003084781/.ssh/id_pipessh.pub
The key fingerprint is:
SHA256:rPIPQ8a74wwumZ7TrtXOXfKcWbcSBgUhgqRnSY9Evf4 GMX+003084781@IBM-PF398HP4
The key's randomart image is:
+---[RSA 3072]----+
|   o=o. . oo     |
|   +.+.. .  .    |
|  . = ..   .     |
|   o ...  .      |
|     .+ S  .     |
|     +.o    o    |
|   ++ *.. .....  |
|  =+.B.=E= +.. . |
| .+=o.Boo =  ..  |
+----[SHA256]-----+

The private key (id_pipessh) and the public key (id_pipessh.pub) are created into ~/.ssh

Seguir leyendo

Bitbucket server + Jenkins

I’ve changed the way to build my projects from Bitbucket server with Jenkins using two plugins:

Each one is documented very well, I recommend both if you work whit this combination.

For me this are the best features:

  • Launch build from Bitbucket branches
  • Visual alert from repository
  • Multi job support with different behavior
  • Adding custom parameters in build
  • SCM integration in Jenkins

Here some screenshots:

Seguir leyendo

Jenkins – Sonarqube, Nodejs dependency to analyse CSS and Javascript

sonar logo-title

I’ve upgraded my Sonarqube server from the 6.4 to 7.8 version, the last than support MySQL database, according to documentation, the Nodejs > 6.0 is required to analyse Javascript code.

But I didn’t know, until my Jenkins jobs failed. The error is clear, I need to install Nodejs:

org.sonarsource.nodejs.NodeCommandException: Error when running: 'node -v'. Is Node.js available during analysis?

I’ve installed Sonarqube over Red Hat Enterprise Linux 7 (RHEL), the process to install Nodejs is very simple. If the repository is enabled only have to execute this commands in terminal and check the version.

$ su -
$ yum install rh-nodejs8
$ scl enable rh-nodejs8 bash
$ node --version
v8.6.0

Seguir leyendo