Welcome to Penso blog space

Oren Penso (Twitter: @openso)
Roie Ben-haim (Twitter: @roie9876)

 

Jenkins X is an open source platform for providing CI / CD pipeline on top of Kubernetes.

The project combines a set of open source core applications, so things will work out-of-the-box, providing a turn key experience.

The core applications are:

 

Jenkins X enable you to create a code project, commit the code, compile the docker image, save it on the private registry and deploy it to the relevant environment. The project allows you to promote the project from staging environment to prod environment in a seamlessly way from a graphical portal.

 

This blog will describe how to leverage the abilities of Jenkins X on top of Vmware PKS product with Harbor as private registry for docker images.

Jenkins X is a CI/CD solution for modern cloud applications on Kubernetes.

 

Let’s start by installing JX on top of PKS.

1. Enable privilege container:
   • During the JX application deployment we will create a docker image, this process require us to enable privilege container in the PKS (1.1) tile. (to be able to directly connect to the internet)
   • you need to go to the PKS tile and then go to Plan -> plan 1, 2 3 (depended on the plan you are working with) and add the following:

 

2. Create a new k8s cluster with the plan you configured in step 1

PKS create cluster {cluster name} -e {cluster name} -p {plan}

• Keep in mind that it’s better to spin up a medium cluster with sufficient storage for all JX components

3. Create a storage class with yaml file on the k8s cluster

kind: StorageClass

apiVersion: storage.k8s.io/v1

metadata:

name: thin-disk

provisioner: kubernetes.io/vsphere-volume

parameters:

diskformat: thin

• Apply the file on the k8s cluster

Kubectl create -f sc.yaml

4. Set the storage class as default for pods on the cluster with the command

kubectl patch storageclass thin-disk -p ‘{“metadata”: {“annotations“:{“storageclass.kubernetes.io/is-default-class“:”true“}}}’

 

5. Download the JX CLI

curl -L https://github.com/jenkins-x/jx/releases/download/v1.3.27/jx-linux-amd64.tar.gz | tar xzv sudo mv jx/usr/local/bin

 

6. Install JX
   • jx install

Chose PKS as provider

7. Create an ingress controller for the jx components

 

 

8. Configure DNS suffix jx on the domain you have and create wild card for that suffix

 

 

 

9. To be able to create a repository on GitHub we need an API TokenPlease click this URL https://github.com/settings/tokens/new?scopes=repo,read:user,user:email,write:repo_hook

 

 

• click on the link and provide username and password for your GitHub account
• generate API token from New Personal Access Token form

 

 

10. Copy the token and enter in into the form below

 

 

 

 

11. After the installation will finish you will need to enter a token of the jx portal, to do so you need to enter jx link https://jenkins.jx.sample.domain/me/configure and fetch the token. The password to log in is mention in the CLI

User: admin Password: Copy from the CLI

 

Show API Token, Copy and paste to the CLI

 

12. Integrate harbor with JX

 

we need to configure the harbor URL inside jx, to do so you need configure the Jenkins as followed:

go to Jenkins -> Configuration -> Manage Jenkins

 

 

Configure the name of the Harbor server in the Value filed and save.

 

13. JX and harbor authentication process

JX need to push the docker images to harbor, for that we need the user/password, this user and password and not stored as clear text.

We need to configure a JSON file to match the user and password of Harbor and then fetch the token with Base64 encoding to replace in the Secret.yaml

14. The file in the example uses user:admin and pass:VMware1!

 

cat /root/.docker/config.json

{

“auths“: {

“harbor.corp.local“: {

“auth“: “YWRtaW46Vk13YXJlMSE=”

}

},

“HttpHeaders“: {

“User-Agent”: “Docker-Client/18.01.0-ce (linux)”

}

}

 

15. You need to encode it in Base64

 

cat /root/.docker/config.json | base64

 

ewoJImF1dGhzIjogewoJCSJoYXJib3IuY29ycC5sb2NhbCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZWazEzWVhKbE1TRT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE4LjAxLjAtY2UgKGxpbnV4KSIKCX0KfQ==

 

 

16. in JX we have secret file to manage the authentication for different components. You need to replace the auth token with the one mentioned up. the secret runs in jx namespace as “jenkins-docker-cfg”.

Create a new secret.yaml file:

vi secret.yaml

 

apiVersion: v1

type: Opaque

kind: Secret

metadata:

name: jenkins-docker-cfg

data:

config.json: Authentication Token from above

ewoJImF1dGhzIjogewoJCSJoYXJib3IuY29ycC5sb2NhbCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZWazEzWVhKbE1TRT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE4LjAxLjAtY2UgKGxpbnV4KSIKCX0KfQ==

 

save the file and exit.

 

17. Replace the secret with the following commands:

 

kubectl delete secret jenkins-docker-cfg -n jx

 

kubectl apply -f secret.yaml -n jx

 

18. harbor works with the concept of projects, for example i have project name: roie9876

 

 

 

19. The project name here need to match the name of the GitHub account you planning to use. for example, GitHub account is: roie9876, as you can see the GitHub account match the project name created in harbor

 

 

20. When you create demos in JX, the name of the demo will create repository inside the GitHub main account and automatically created a repository in harbor. For example, Node app demo:

 

 

 

 

Flow of deploying a test app with JX command:

1. The flow of installing a test app will look like the next print screen

 

 

 

 

2. Let’s create a new NodeJS app as a test app, just run

JX create quickstart

 

In the demo we will create a GO Demo app

Chose the Git user that will accommodate the new repository

The process will create a new repository for the Go Demo app

The process creates a CI / CD pipeline, compiling the Image with maven, push the image to harbor registry and deploys it as pod on top of the k8s cluster.

 

When the process is finished, the application is running as pod in the JX-Staging Namespace as seen in the next screen shots

 

 

The process is successful in JenkinX

 

 

The application link is on the Jenkins X console

 

 

The application after opening it:

The pod is active in the Namespace

 

 

 

 
Integration between Kubeapps and JX

 

In a previous blog

http://www.routetocloud.com/2018/07/kubeapp-by-bitnami-over-pks-infrastructure/

we wrote on KubeApps project from Bitnami that allows you to deploy helm charts on PKS clusters from known repository’s.

When you create a pipeline with JX and commit a code, a part of the process is the creation of Helm chart that has your own application deployment chart.

You can integrate JX with KubeApps to publish your own app to the Kubeapp catalogue and throw that catalog to deploy the app and rollback the installation to previous versions if needed.

1. We need to add the JX repo to the JX helm repos:
   • List the ingress controller and find the chart museum project link, run the kubectl get ingress command as followed:

• Add the kubeapps to your own helm repo:

helm repo add myrepo  http://chartmuseum.jx.lab.local

• Do helm repo list, to check that the repo is added:

helm repo list

output:

stable          https://kubernetes-charts.storage.googleapis.com

local           http://127.0.0.1:8879/charts

monocular       https://kubernetes-helm.github.io/monocular

jenkins-x       https://chartmuseum.build.cd.jenkins-x.io

myrepo          http://chartmuseum.jx.pksdemo.ml

 

2. Now add the repo to the KubeApps repo URL’s
   • Open the KubeApps URL

 

Go to Configuration -> Add Repository

 

 

• Add the repo url:

http://chartmuseum.jx.pksdemo.ml

3. After you add the repo you can see at the Charts tab your own apps that was deployed using the JX pipeline:

 

 

4. When clicking the app you can change the version that is deployed, install the new version or rolling back if needed: