基于Gitea 和 Drone CI/CD的私有云研发环境

1. 基于 Drone CI 服务完成其自动化部署工作流的配置和演示

一个简陋的CI/CD的主要步骤包含： clone => test => build => image => deploy => notify

1.1 依赖组件

组件名称	组件功能	访问方式
DNS	域名管理	*.michaelapps.com
Caddy	域名访问转发和HTTPS证书管理	vim /etc/caddy/Caddyfile
gitea	私有代码仓库	https://gitea.michaelapp.com/ 用户名yypan
drone	CI/CD工具	https://drone.michaelapp.com/ 关联gitea账号，无需新帐号
harbor	私有docker镜像仓库	https://harbor.michaelapp.com/ 用户名admin
当前服务	webtest	https://webtest.michaelapp.com/ 负载均衡/访问IP获取

1.2 caddy 组件安装

1）安装

https://caddyserver.com/docs/install#fedora-redhat-centos

[root@master dockershell]#yum install yum-plugin-copr
[root@master dockershell]#yum copr enable @caddy/caddy
[root@master dockershell]#yum install caddy

2）查看安装后caddy的版本

[root@master dockershell]# caddy version
v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

3）域名代理配置，配置修改 vim /etc/caddy/Caddyfile

参考文档：https://caddyserver.com/docs/caddyfile/directives/reverse_proxy

webtest.michaelapp.com {
    reverse_proxy / 127.0.0.1:9090 127.0.0.1:9091 {           #反向代理
       #lb_policy round_robin
       lb_policy ip_hash                                  #负载均衡算法
       #lb_policy least_conn
       header_up Host {http.reverse_proxy.upstream.hostport}   #真实IP
       health_path /                                           #监控检查
       health_interval 60s
       health_timeout 15s
    }
    tls panyingyun@gmail.com
}

1.3 gitea是代码仓库

参考文档：https://docs.gitea.io/zh-cn/install-with-docker/

1) gitea.sh 并且执行即可

#!/bin/bash 
mkdir -p  /home/yypan/volumes/gitea
docker stop gitea
docker rm gitea
docker run -d --name=gitea -p 10022:22 -p 13000:3000 -v /home/yypan/volumes/gitea:/data gitea/gitea:1.13.1

2）域名代理配置，配置caddy

gitea.michaelapp.com {
    log {
        output file         /home/yypan/log/gitea.log
        format single_field common_log
    }
    reverse_proxy /* 42.192.207.198:13000
    tls panyingyun@gmail.com
}

3）访问主页进行配置

配置会自动保存配置文件为 /home/yypan/volumes/gitea/gitea/conf/app.ini，其内容如下

APP_NAME = Maxwell
RUN_MODE = prod
RUN_USER = git

[repository]
ROOT = /data/git/repositories

[repository.local]
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo

[repository.upload]
TEMP_PATH = /data/gitea/uploads

[server]
APP_DATA_PATH    = /data/gitea
DOMAIN           = gitea.michaelapp.com  (*)
SSH_DOMAIN       = gitea.michaelapp.com  (*)
HTTP_PORT        = 3000
ROOT_URL         = https://gitea.michaelapp.com/ (*)
DISABLE_SSH      = false
SSH_PORT         = 10022 (*)
SSH_LISTEN_PORT  = 22
LFS_START_SERVER = true
LFS_CONTENT_PATH = /data/git/lfs
LFS_JWT_SECRET   = Hbn83oCDPUbVp_gJJcUorsg2Ha9ZFcUpPgG0xnSAS5g
OFFLINE_MODE     = false

[database]
PATH     = /data/gitea/gitea.db
DB_TYPE  = sqlite3
HOST     = localhost:3306
NAME     = gitea
USER     = root
PASSWD   = 
LOG_SQL  = false
SCHEMA   = 
SSL_MODE = disable
CHARSET  = utf8

[indexer]
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve

[session]
PROVIDER_CONFIG = /data/gitea/sessions
PROVIDER        = file

[picture]
AVATAR_UPLOAD_PATH            = /data/gitea/avatars
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars
DISABLE_GRAVATAR              = false
ENABLE_FEDERATED_AVATAR       = true

[attachment]
PATH = /data/gitea/attachments

[log]
MODE                 = console
LEVEL                = info
REDIRECT_MACARON_LOG = true
MACARON              = console
ROUTER               = console
ROOT_PATH            = /data/gitea/log

[security]
INSTALL_LOCK   = true
SECRET_KEY     = TMPYQhBVvPbJ9Cbn7MkFv2QmZ9gYGyPCDS29NTIYDp8Lh0gnD9LpQQpywA6F5xj3
INTERNAL_TOKEN = eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYmYiOjE2MTAwMTQ1MTZ9.zpKvTzqCwIBp1b7uB-tLP2PhulZyK9Fri3ioOKI5GsM

[service]
DISABLE_REGISTRATION              = false
REQUIRE_SIGNIN_VIEW               = false
REGISTER_EMAIL_CONFIRM            = false
ENABLE_NOTIFY_MAIL                = false
ALLOW_ONLY_EXTERNAL_REGISTRATION  = false
ENABLE_CAPTCHA                    = false
DEFAULT_KEEP_EMAIL_PRIVATE        = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING       = true
NO_REPLY_ADDRESS                  = noreply.localhost

[oauth2]
JWT_SECRET = FLkRbfbtvCsb9qVvyGscKzQvikr00fQ5t3JwD0aWk3c

[mailer]
ENABLED = false

[openid]
ENABLE_OPENID_SIGNIN = true
ENABLE_OPENID_SIGNUP = true

1.4 镜像仓库harbor

1）下载harbor-online-installer-v2.1.2.tgz

tar zxvf harbor-online-installer-v2.1.2.tgz
cd harbor

2）配置harbor.yml

通过模板生成初始的harbor.yml,并且编辑harbor.yml

cp harbor.yml.tmpl harbor.yml  
vim harbor.yml

修改其中以下字段
hostname: 42.192.207.198  # 主机IP
http:
  port: 1080              # 访问端口
harbor_admin_password: xxxxxx # admin登录密码
database:                             
  password: xxxxxx            # 数据库root密码
  max_idle_conns: 50 
  max_open_conns: 1000

# The default data volume
data_volume: /home/yypan/volumes/harbor  #挂盘目录

3）运行

sh install.sh

4）域名代理配置，配置caddy

harbor.michaelapp.com {
    log {
        output file         /home/yypan/log/harbor.log
        format single_field common_log
    }
    reverse_proxy /* 127.0.0.1:1080 {
        header_up Host {http.reverse_proxy.upstream.hostport}
        header_up X-Real-IP {http.request.remote}
        header_up X-Forwarded-Port {http.request.port}
    }
    tls panyingyun@gmail.com
}

5) 重启项目服务

	docker-compose down 
	docker-compose up -d

6) 推镜像

docker login -u admin -p [xxxxxxx] 42.192.207.198:1080
docker build -t 42.192.207.198:1080/webtest/webtest:$version .
docker push 42.192.207.198:1080/webtest/webtest:$version

1.5 Drone 安装

1) Drone Server和Runner docker-compose.yml

使用docker-compose进行安装,创建 docker-compose.yml 文件内容如下：

version: '3'
services:
  # 容器名称
  yypan-drone-server:
    # 构建所使用的镜像
    image: drone/drone:1
    # 映射容器内80端口到宿主机的7079端口
    ports:
      - 7079:80
    # 映射容器内/data目录到宿主机的/data/drone目录
    volumes:
      - /home/yypan/volumes/drone:/data
    # 容器随docker自动启动
    restart: always
    environment:
      # Gitea 服务器地址
      - DRONE_GITEA_SERVER=https://gitea.michaelapp.com
      # Gitea OAuth2客户端ID
      - DRONE_GITEA_CLIENT_ID=4b51890e-6342-445f-8d47-aa78b11b1d31
      # Gitea OAuth2客户端密钥
      - DRONE_GITEA_CLIENT_SECRET=Lccyb22FOMnaCTqJusROQAJZMHnknhYbp9cq9FKiDmM=
      # drone的共享密钥
      - DRONE_RPC_SECRET=3db96e7d4978443106374e86ce388589
      # drone的主机名
      - DRONE_SERVER_HOST=drone.michaelapp.com
      # 外部协议方案
      - DRONE_SERVER_PROTO=https
      # 创建管理员账户，这里对应为gitea的用户名
      - DRONE_USER_CREATE=username:yypan,machine:false,admin:true,token:3db96e7d4978443106374e86ce388589


  yypan-docker-runner:
    image: drone/drone-runner-docker:1
    ports:
      - 7080:3000
    restart: always
    depends_on:
      - yypan-drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      # 用于连接到Drone服务器的协议。该值必须是http或https。
      - DRONE_RPC_PROTO=https
      # 用于连接到Drone服务器的主机名
      - DRONE_RPC_HOST=drone.michaelapp.com
      # Drone服务器进行身份验证的共享密钥，和上面设置一样
      - DRONE_RPC_SECRET=3db96e7d4978443106374e86ce388589
      # 限制运行程序可以执行的并发管道数。运行程序默认情况下执行2个并发管道。
      - DRONE_RUNNER_CAPACITY=2
      # docker runner 名称
      - DRONE_RUNNER_NAME=yypan-runner
      - DOCKER_API_VERSION=1.38

2) Drone Run

	docker-compose down
	docker-compose up -d

3) Drone 域名代理配置，配置caddy

drone.michaelapp.com {
    log {
        output file         /home/yypan/log/drone.log
        format single_field common_log
    }
    reverse_proxy /* 127.0.0.1:7079
    tls panyingyun@gmail.com
}

2 Golang 项目(Go-Web-App)为例

2.1 手动编译和运行

$ CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build
$ ./webtest --port 9090

2.2 创建Drone CI/CD文件 .drone.yml

.drone.yml

kind: pipeline
name: default

steps:
  - name: Test
    image: golang
    volumes:
      - name: cache
        path: /go
    commands:
      - export GOPROXY=https://goproxy.cn,direct
      - export GO111MODULE=on
      - go test
  - name: Build and Push 
    image: docker:dind
    volumes:
      - name: dockersock
        path: /var/run/docker.sock
    commands:
      - export version="v0.23"
      - docker login -u admin -p [xxxxx] 42.192.207.198:1080
      - docker build -t 42.192.207.198:1080/webtest/webtest:$version .
      - docker push 42.192.207.198:1080/webtest/webtest:$version
  - name: Deploy
    image: docker:dind
    volumes:
      - name: dockersock
        path: /var/run/docker.sock
    commands:
      - export version="v0.23"
      - docker stop webtest9090 || true
      - docker rm webtest9090 || true
      - docker stop webtest9091 || true
      - docker rm webtest9091 || true
      - docker run -p 9090:9090 --name webtest9090 -v /home/yypan/volumes/webtest1:/data -itd harbor.michaelapp.com/webtest/webtest:$version
      - docker run -p 9091:9090 --name webtest9091 -v /home/yypan/volumes/webtest2:/data -itd harbor.michaelapp.com/webtest/webtest:$version
volumes:
  - name: cache
    host:
      path: /var/lib/cache
  - name: dockersock
    host:
      path: /var/run/docker.sock
trigger:
  branch:
    - master

2.3 webtest 域名代理配置

参考文档：https://caddyserver.com/docs/caddyfile/directives/reverse_proxy

配置修改 vim /etc/caddy/Caddyfile

webtest.michaelapp.com {
    reverse_proxy / 127.0.0.1:9090 127.0.0.1:9091 {           #反向代理
       #lb_policy round_robin
       lb_policy ip_hash                                  #负载均衡算法
       #lb_policy least_conn
       header_up Host {http.reverse_proxy.upstream.hostport}   #真实IP
       health_path /                                           #监控检查
       health_interval 60s
       health_timeout 15s
    }
    tls panyingyun@gmail.com
}

```bash 
比如 我们修改一个版本号，并提交
git commit -m "deloy v2.3"
git push 
去drone管理后台查看具体的编译过程
https://drone.michaelapp.com/ 查看编译过程

2.4 Web访问并查看日志

docker logs -f webtest9090 | grep 183.157.56.31
docker logs -f webtest9091 | grep 183.157.56.31

观察负载均衡的过程，以及不同负载均衡算法的作用

2.5 webtest代码

//main.go 
package main

import (
	"fmt"
	"net"
	"os"
	"strings"

	"github.com/urfave/cli"
	macaron "gopkg.in/macaron.v1"
)

func Appserver(ctx *macaron.Context) string {
	return "Welcome " + ctx.RemoteAddr() + " Access Server " + getIPAddress()
}

func getIPAddress() string {
	ar := ""
	if addrs, err := net.InterfaceAddrs(); err == nil {
		for _, addr := range addrs {
			if strings.HasSuffix(addr.String(), "/24") {
				fmt.Println("network = ", addr.Network(), "addr = ", addr.String())
				ar = ar + " , " + addr.String()
			}

		}
	}
	return ar
}

func run(c *cli.Context) error {
	m := macaron.Classic()
	m.Get("/", Appserver)
	m.Run(c.String("host"), c.Int("port"))
	return nil
}

func main() {
	app := cli.NewApp()
	app.Name = "webtest"
	app.Usage = "test web and auto start server within centos7.2"
	app.Copyright = "yypan@michaelapp.com"
	app.Version = "0.0.1"
	app.Action = run
	app.Flags = []cli.Flag{
		cli.StringFlag{
			Name:  "host",
			Value: "0.0.0.0",
		},
		cli.IntFlag{
			Name:  "port",
			Value: 9090,
		},
	}
	app.Run(os.Args)
}

2.6 更加复杂的案例

https://github.com/go-gitea/gitea/blob/main/.drone.yml

2.7 caddy操作

启动
caddy start --config /etc/caddy/Caddyfile
重新加载
caddy reload --config /etc/caddy/Caddyfile
停止
caddy stop

参考：
https://caddy.community/t/caddy-v2-how-to-proxy-websoket-v2ray-websocket-tls/7040
https://caddyserver.com/docs/caddyfile/matchers#named-matchers
https://caddyserver.com/docs/caddyfile/directives/log
https://caddyserver.com/docs/caddyfile/concepts

基于Gitea 和 Drone CI/CD的私有云研发环境

1. 基于 Drone CI 服务完成其自动化部署工作流的配置和演示

1.1 依赖组件

1.2 caddy 组件安装

1）安装

2）查看安装后caddy的版本

3）域名代理配置，配置修改 vim /etc/caddy/Caddyfile

1.3 gitea是代码仓库

1) gitea.sh 并且执行即可

2）域名代理配置，配置caddy

3）访问主页进行配置

1.4 镜像仓库harbor

1）下载harbor-online-installer-v2.1.2.tgz

2）配置harbor.yml

3）运行

4）域名代理配置，配置caddy

5) 重启项目服务

6) 推镜像

1.5 Drone 安装

1) Drone Server和Runner docker-compose.yml

2) Drone Run

3) Drone 域名代理配置，配置caddy

2 Golang 项目(Go-Web-App)为例

2.1 手动编译和运行

2.2 创建Drone CI/CD文件 .drone.yml

2.3 webtest 域名代理配置

2.4 Web访问并查看日志

2.5 webtest代码

2.6 更加复杂的案例

2.7 caddy操作

See Also

最近文章

分类

标签

其它

1. 基于 Drone CI 服务完成其自动化部署工作流的配置和演示

1.1 依赖组件

1.2 caddy 组件安装

1） 安装

2） 查看安装后caddy的版本

3） 域名代理配置，配置修改 vim /etc/caddy/Caddyfile

1.3 gitea是代码仓库

1) gitea.sh 并且执行即可

2）域名代理配置，配置caddy

3） 访问主页进行配置

1.4 镜像仓库harbor

1） 下载harbor-online-installer-v2.1.2.tgz

2） 配置harbor.yml

3） 运行

4）域名代理配置，配置caddy

5) 重启项目服务

6) 推镜像

1.5 Drone 安装

1) Drone Server和Runner docker-compose.yml

2) Drone Run

3) Drone 域名代理配置，配置caddy

2 Golang 项目(Go-Web-App)为例

2.1 手动编译和运行

2.2 创建Drone CI/CD文件 .drone.yml

2.3 webtest 域名代理配置

2.4 Web访问并查看日志

2.5 webtest代码

2.6 更加复杂的案例

2.7 caddy操作

See Also

最近文章

分类

标签

其它

1）安装

2）查看安装后caddy的版本

3）域名代理配置，配置修改 vim /etc/caddy/Caddyfile

3）访问主页进行配置

1）下载harbor-online-installer-v2.1.2.tgz

2）配置harbor.yml

3）运行