Gitlab Devops 配置

K8s 环境

在 Kubernetes 集群上安装 Gitlab-Runner

https://docs.gitlab.com/runner/install/kubernetes.html

1
2
3
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'

helm install --namespace gitlab --name gitlab-runner -f values.yaml gitlab/gitlab-runner
  • values.yaml 文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
## GitLab Runner Image
##
## By default it's using gitlab/gitlab-runner:alpine-v{VERSION}
## where {VERSION} is taken from Chart.yaml from appVersion field
##
## ref: https://hub.docker.com/r/gitlab/gitlab-runner/tags/
##
image: gitlab/gitlab-runner:alpine-v12.0.0-rc1

## Specify a imagePullPolicy
## 'Always' if imageTag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
imagePullPolicy: IfNotPresent

## The GitLab Server URL (with protocol) that want to register the runner against
## ref: https://docs.gitlab.com/runner/commands/README.html#gitlab-runner-register
##
gitlabUrl: https://gitlab.com/

## The Registration Token for adding new Runners to the GitLab Server. This must
## be retrieved from your GitLab Instance.
## ref: https://docs.gitlab.com/ce/ci/runners/README.html
##
runnerRegistrationToken: "NkeYywTkYwWf14JZxtM1"

## The Runner Token for adding new Runners to the GitLab Server. This must
## be retrieved from your GitLab Instance. It is token of already registered runner.
## ref: (we don't yet have docs for that, but we want to use existing token)
##
# runnerToken: ""
#
## Unregister all runners before termination
##
## Updating the runner's chart version or configuration will cause the runner container
## to be terminated and created again. This may cause your Gitlab instance to reference
## non-existant runners. Un-registering the runner before termination mitigates this issue.
## ref: https://docs.gitlab.com/runner/commands/README.html#gitlab-runner-unregister
##
unregisterRunners: true

## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use
## Provide resource name for a Kubernetes Secret Object in the same namespace,
## this is used to populate the /etc/gitlab-runner/certs directory
## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates
##
# certsSecretName:

## Configure the maximum number of concurrent jobs
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
concurrent: 10

## Defines in seconds how often to check GitLab for a new builds
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
checkInterval: 30


## Configure GitLab Runner's logging level. Available values are: debug, info, warn, error, fatal, panic
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
# logLevel:

## For RBAC support:
rbac:
create: true

## Run the gitlab-bastion container with the ability to deploy/manage containers of jobs
## cluster-wide or only within namespace
clusterWideAccess: false

## Use the following Kubernetes Service Account name if RBAC is disabled in this Helm chart (see rbac.create)
##
# serviceAccountName: default

## Configure integrated Prometheus metrics exporter
## ref: https://docs.gitlab.com/runner/monitoring/#configuration-of-the-metrics-http-server
metrics:
enabled: true

## Configuration for the Pods that that the runner launches for each new job
##
runners:
## Default container image to use for builds when none is specified
##
image: ubuntu:16.04

## Specify one or more imagePullSecrets
##
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# imagePullSecrets: []

## Specify the image pull policy: never, if-not-present, always. The cluster default will be used if not set.
##
# imagePullPolicy: ""

## Defines number of concurrent requests for new job from GitLab
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section
##
# requestConcurrency: 1

## Specify whether the runner should be locked to a specific project: true, false. Defaults to true.
##
# locked: true

## Specify the tags associated with the runner. Comma-separated list of tags.
##
## ref: https://docs.gitlab.com/ce/ci/runners/#using-tags
##
tags: "k8s-runner"

## Run all containers with the privileged flag enabled
## This will allow the docker:dind image to run if you need to run Docker
## commands. Please read the docs before turning this on:
## ref: https://docs.gitlab.com/runner/executors/kubernetes.html#using-docker-dind
##
privileged: true

## The name of the secret containing runner-token and runner-registration-token
# secret: gitlab-runner

## Namespace to run Kubernetes jobs in (defaults to the same namespace of this release)
##
namespace: gitlab

## Distributed runners caching
## ref: https://gitlab.com/gitlab-org/gitlab-runner/blob/master/docs/configuration/autoscale.md#distributed-runners-caching
##
## If you want to use s3 based distributing caching:
## First of all you need to uncomment General settings and S3 settings sections.
##
## Create a secret 's3access' containing 'accesskey' & 'secretkey'
## ref: https://aws.amazon.com/blogs/security/wheres-my-secret-access-key/
##
## $ kubectl create secret generic s3access \
## --from-literal=accesskey="YourAccessKey" \
## --from-literal=secretkey="YourSecretKey"
## ref: https://kubernetes.io/docs/concepts/configuration/secret/
##
## If you want to use gcs based distributing caching:
## First of all you need to uncomment General settings and GCS settings sections.
##
## Access using credentials file:
## Create a secret 'google-application-credentials' containing your application credentials file.
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-gcs-section
## You could configure
## $ kubectl create secret generic google-application-credentials \
## --from-file=gcs-applicaton-credentials-file=./path-to-your-google-application-credentials-file.json
## ref: https://kubernetes.io/docs/concepts/configuration/secret/
##
## Access using access-id and private-key:
## Create a secret 'gcsaccess' containing 'gcs-access-id' & 'gcs-private-key'.
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-gcs-section
## You could configure
## $ kubectl create secret generic gcsaccess \
## --from-literal=gcs-access-id="YourAccessID" \
## --from-literal=gcs-private-key="YourPrivateKey"
## ref: https://kubernetes.io/docs/concepts/configuration/secret/
cachePath: "/opt/gitlab_runner/cache"
cache: {}
## General settings
# cacheType: s3
# cachePath: "/opt/gitlab_runner/cache"
# cacheShared: true

## S3 settings
# s3ServerAddress: s3.amazonaws.com
# s3BucketName:
# s3BucketLocation:
# s3CacheInsecure: false
# secretName: s3access

## GCS settings
# gcsBucketName:
## Use this line for access using access-id and private-key
# secretName: gcsaccess
## Use this line for access using google-application-credentials file
# secretName: google-application-credentials

## Build Container specific configuration
##
builds: {}
# cpuLimit: 200m
# memoryLimit: 256Mi
# cpuRequests: 100m
# memoryRequests: 128Mi
# image: gitlab/gitlab-runner-helper:x86_64-latest

## Service Account to be used for runners
##
# serviceAccountName:

## If Gitlab is not reachable through $CI_SERVER_URL
##
# cloneUrl:

## Specify node labels for CI job pods assignment
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
# nodeSelector: {}

## Specify pod labels for CI job pods
##
# podLabels: {}

## Specify annotations for job pods, useful for annotations such as iam.amazonaws.com/role
# podAnnotations: {}

## Configure environment variables that will be injected to the pods that are created while
## the build is running. These variables are passed as parameters, i.e. `--env "NAME=VALUE"`,
## to `gitlab-runner register` command.
##
## Note that `envVars` (see below) are only present in the runner pod, not the pods that are
## created for each build.
##
## ref: https://docs.gitlab.com/runner/commands/#gitlab-runner-register
##
# env:
# NAME: VALUE


## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
# limits:
# memory: 256Mi
# cpu: 200m
# requests:
# memory: 128Mi
# cpu: 100m

## Affinity for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
##
affinity: {}

## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
# Example: The gitlab runner manager should not run on spot instances so you can assign
# them to the regular worker nodes only.
# node-role.kubernetes.io/worker: "true"

## List of node taints to tolerate (requires Kubernetes >= 1.6)
## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
tolerations: []
# Example: Regular worker nodes may have a taint, thus you need to tolerate the taint
# when you assign the gitlab runner manager with nodeSelector or affinity to the nodes.
# - key: "node-role.kubernetes.io/worker"
# operator: "Exists"

## Configure environment variables that will be present when the registration command runs
## This provides further control over the registration process and the config.toml file
## ref: `gitlab-runner register --help`
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html
##
# envVars:
# - name: RUNNER_EXECUTOR
# value: kubernetes

## list of hosts and IPs that will be injected into the pod's hosts file
hostAliases: []
# Example:
# - ip: "127.0.0.1"
# hostnames:
# - "foo.local"
# - "bar.local"
# - ip: "10.1.2.3"
# hostnames:
# - "foo.remote"
# - "bar.remote"

## Annotations to be added to manager pod
##
podAnnotations: {}
# Example:
# iam.amazonaws.com/role: <my_role_arn>
1
2
3
4
5
6
gcr.io/kubernetes-helm/tiller:v2.12.3

docker pull fishead/gcr.io.kubernetes-helm.tiller:v2.12.3
docker tag fishead/gcr.io.kubernetes-helm.tiller:v2.12.3 gcr.io/kubernetes-helm/tiller:v2.12.3

去 docker hub 找个镜像仓库

配置 Kubernetes cluster details

安装 Helm Tiller

  • 点击页面 Install 按钮

  • kubectl get pods -A 能看到 install-helm 的 pod

  • 根据 pod/install-helm 生成 yaml

1
kubectl get pod install-helm -n gitlab-managed-apps -o yaml > install-helm.yaml
  • 修改 yaml 中的 pod 启动命令, 加入阿里云仓库
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
apiVersion: v1
kind: Pod
metadata:

......

spec:
containers:
- args:
- -c
- $(COMMAND_SCRIPT)
command:
- /bin/sh
env:
- name: HELM_VERSION
value: 2.12.3
- name: TILLER_NAMESPACE
value: gitlab-managed-apps
- name: COMMAND_SCRIPT
value: |-
set -xeo pipefail
helm init --tiller-tls --tiller-tls-verify --tls-ca-cert /data/helm/helm/config/ca.pem --tiller-tls-cert /data/helm/helm/config/cert.pem --tiller-tls-key /data/helm/helm/config/key.pem --service-account tiller --upgrade -i registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.12.3 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
helm repo add stable https://burdenbear.github.io/kube-charts-mirror/
helm repo update

......
  • 根据修改后的 yaml 重新创建 pod
1
2
kubectl delete pod install-helm -n gitlab-managed-apps 
kubectl apply -f install-helm.yaml
  • 删除 namespace
1
kubectl delete namespaces gitlab-managed-apps
  • 重新点击页面 Install 按钮

非 K8s 环境

在 Docker 中部署 GitLab Runner

运行 GitLab Runner

  • 命令行

    1
    2
    3
    4
    $ docker run -d --name gitlab-runner --restart always \
    -v /path/to/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest
  • docke-compose

    1
    2
    3
    4
    5
    6
    7
    8
    9
    version: '3.7'
    services:

    gitlab-runner:
    image: gitlab/gitlab-runner:latest
    restart: always
    volumes:
    - ./gitlab-runner/config:/etc/gitlab-runner
    - /var/run/docker.sock:/var/run/docker.sock

注册 GitLab Runner

  • 执行命令注册

    1
    2
    3
    4
    5
    6
    7
    8
    9
    docker exec gitlab-runner gitlab-runner register -n \
    --url https://gitlab.com/ \
    --registration-token xxxxxxxxxx \
    --executor docker \
    --tag-list runInDk \
    --description "My Docker Runner" \
    --docker-image "docker:latest" \
    --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
    --docker-volumes /root/.m2:/root/.m2

优化工作

  • Runner 默认情况下每执行一个 Job 都会重新拉取一次所需镜像,我们可把策略改为:镜像不存在时才拉取,编辑 config.toml 文件,修改 [runners.docker] 栏中加入 pull_policy = "if-not-present"
  • 重启 Runner 容器,使之生效

编写项目的 gitlab-ci.yml 文件

  • 后端项目

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    stages: 
    - package
    - build
    - deploy
    my_package:
    image: maven:3.6.0-jdk-8-alpine
    stage: package
    script:
    - mvn clean package -DskipTests
    - cp Dockerfile target/Dockerfile
    cache:
    key: ${CI_PIPELINE_ID}
    paths:
    - target/
    only:
    - master
    tags:
    - lvTest
    my_build:
    stage: build
    cache:
    key: ${CI_PIPELINE_ID}
    paths:
    - target/
    script:
    - cd target
    - docker build -t 192.168.88.4:5000/${CI_PROJECT_NAME}:${CI_PIPELINE_ID} .
    - docker push 192.168.88.4:5000/${CI_PROJECT_NAME}:${CI_PIPELINE_ID}
    tags:
    - lvTest
    my_deploy:
    stage: deploy
    script:
    - docker stop lxwtest && docker rm lxwtest
    - docker run -d -p 8888:8080 --restart=always --name=lxwtest 192.168.88.4:5000/${CI_PROJECT_NAME}:${CI_PIPELINE_ID}
    tags:
    - lvTest
  • 前端项目