container-guide/DOCUMENTS/jenkins-pipline-usage.md

jenkins-pipline-usage.md
------------------------
```
@version    180808:1
@author     zhangxuhong <zhangxuhong@xitu.io>
```

Name
----

    jenkins-pipine-usage - jenkins流水线使用手册.

    
Table of Contents
-----------------

* [Name](#name)
* [Reference 参考文档](#reference)



Reference 参考文档
------------------

* [官方手册](https://github.com/jenkinsci/kubernetes-plugin)
https://zhangchenchen.github.io/2017/12/17/achieve-cicd-in-kubernetes-with-jenkins/
https://www.cnblogs.com/hahp/p/5812455.html
https://www.ibm.com/developerworks/cn/devops/d-based-ibm-cloud-private/index.html


Tips 
----

流水线构建过程将每个部分称作"stage", 语法是:  
```
stage('Stage Name') {
    // action
}
```
默认stage中的操作, 例如运行shell命令等, 如果没有指定container, 都是运行是在默认的jnlp容器上的.  
如果指定了container, 则会运行在指定的cloud字段中的kubernetes集群中.  
所以插件中配置kubernetes集群登录信息完全不是为了直接在上面部署业务, 而是提供了一个运行jenkins slave容器的环境而已.  
因此, 传统的 checkout, build, push, deploy 过程中最后的deploy过程就变成了:  
    - 有几个kubernetes集群就弄几个包含kubectl和认证文件的docker image.
    - 部署的时候用这个docker image执行kubectl来部署. 





下面是一段配置sample:  
```
def label = "worker-${UUID.randomUUID().toString()}"

podTemplate(label: label, containers: [
  containerTemplate(name: 'gradle', image: 'gradle:4.5.1-jdk9', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'docker', image: 'docker', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl:v1.8.8', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'helm', image: 'lachlanevenson/k8s-helm:latest', command: 'cat', ttyEnabled: true)
],
volumes: [
  hostPathVolume(mountPath: '/home/gradle/.gradle', hostPath: '/tmp/jenkins/.gradle'),
  hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')
]) {
  node(label) {
    def myRepo = checkout scm
    def gitCommit = myRepo.GIT_COMMIT
    def gitBranch = myRepo.GIT_BRANCH
    def shortGitCommit = "${gitCommit[0..10]}"
    def previousGitCommit = sh(script: "git rev-parse ${gitCommit}~", returnStdout: true)
 
    stage('Test') {
      try {
        container('gradle') {
          sh """
            pwd
            echo "GIT_BRANCH=${gitBranch}" >> /etc/environment
            echo "GIT_COMMIT=${gitCommit}" >> /etc/environment
            gradle test
            """
        }
      }
      catch (exc) {
        println "Failed to test - ${currentBuild.fullDisplayName}"
        throw(exc)
      }
    }
    stage('Build') {
      container('gradle') {
        sh "gradle build"
      }
    }
    stage('Create Docker images') {
      container('docker') {
        withCredentials([[$class: 'UsernamePasswordMultiBinding',
          credentialsId: 'dockerhub',
          usernameVariable: 'DOCKER_HUB_USER',
          passwordVariable: 'DOCKER_HUB_PASSWORD']]) {
          sh """
            docker login -u ${DOCKER_HUB_USER} -p ${DOCKER_HUB_PASSWORD}
            docker build -t namespace/my-image:${gitCommit} .
            docker push namespace/my-image:${gitCommit}
            """
        }
      }
    }
    stage('Run kubectl') {
      container('kubectl') {
        sh "kubectl get pods"
      }
    }
    stage('Run helm') {
      container('helm') {
        sh "helm list"
      }
    }
  }
}
```



Issues
------

- 多段构建问题
```
Step 1/8 : FROM harbor02.juejin.id/infrastructure/nginx-1.10.3-centos-with-lua-nginx-module:latest as template
Error parsing reference: "harbor02.juejin.id/infrastructure/nginx-1.10.3-centos-with-lua-nginx-module:latest as template" is not a valid repository/tag: invalid reference format
```

由于线上集群是docker 17.03.2-ce, 不支持多段构建. 所以 xxx as template 语法不能使用.

- 镜像push问题
```
+ docker push harbor02.juejin.id/test/suid-generator:1ea59b9
The push refers to a repository [harbor02.juejin.id/test/suid-generator]
1c91f1c79983: Preparing
f070d45fa624: Preparing
129f1de793ff: Preparing
0c195f651c3f: Preparing
289b7a478aed: Preparing
ae39cf183283: Preparing
ce860bbdcfdd: Preparing
129b697f70e9: Preparing
ae39cf183283: Waiting
ce860bbdcfdd: Waiting
129b697f70e9: Waiting
denied: requested access to the resource is denied
```

线上harbor集群push镜像需要docker登录, 因此需要将使用的构建镜像的docker增加登录信息. 登录信息在/root/.docker/config.json

- 镜像缓存问题
containerTemplate中特定tag的image 一旦被下载到本地一次就不会再从harbor下载了, 无论harbor是否更新了image.
所以要使用新的image一定要修改tag.

- kubeapi版本问题
```
+ kubectl --kubeconfig=/root/.kube/config apply -f ./Deployment.yaml
error: unable to recognize "./Deployment.yaml": no matches for kind "Deployment" in version "v1"
```
Deployment 支持的版本是 apps/v1, 具体可以用kubectl api-versions查看.