构建与部署管线(中文译文)
原始 DeepWiki 页面:https://deepwiki.com/mayan-edms/Mayan-EDMS/6.2-build-and-deployment-pipeline
翻译时间:2026-05-27T08:44:30.092Z
翻译模型:deepseek-chat
原文字符数:13910
项目:Mayan EDMS (mayan-edms)
---
构建与部署管线
相关源文件
以下文件被用作生成此 Wiki 页面的上下文:
.gitlab-ci.ymlMakefileconfig.envdocker/Makefiledocs/parts/troubleshooting/static_files.txtmayan/apps/appearance/management/commands/appearance_prepare_static.pymayan/apps/platform/templates/platform/docker/dockerfile.tmplmayan/apps/platform/templates/platform/docker/entrypoint.tmplmayan/apps/platform/templates/platform/docker/supervisord.tmplmayan/apps/platform/templates/platform/gitlab-ci.tmplmayan/settings/literals.py
本文档说明了 Mayan EDMS 的自动化构建与部署管线,包括 GitLab CI/CD 集成、Docker 镜像构建、Python 包分发以及部署流程。该管线自动化了测试、构建和发布流程,可将 Python 包发布到 PyPI,将 Docker 镜像发布到 Docker Hub。
有关搭建本地开发环境的信息,请参阅开发环境搭建。有关版本管理和发布流程的详细信息,请参阅发布管理。
管线架构
Mayan EDMS 的构建管线使用 GitLab CI/CD 实现,包含多个阶段,这些阶段根据分支命名规范来执行。管线处理三种主要的制品类型:Python 包、Docker 镜像和文档。
graph TD
subgraph "源代码管理"
DEV["开发者"] --> CODE["代码变更"]
CODE --> GIT["Git 仓库"]
GIT --> GITLAB["GitLab 仓库"]
end
subgraph "GitLab CI 管线阶段"
GITLAB --> TRIGGER["管线触发"]
TRIGGER --> TEST_STAGE["stage_test"]
TEST_STAGE --> BUILD_PYTHON["stage_build_python"]
TEST_STAGE --> BUILD_DOCKER["stage_build_docker"]
TEST_STAGE --> BUILD_DOCS["stage_build_documentation"]
BUILD_PYTHON --> PUSH_PYTHON["stage_push_python"]
BUILD_DOCKER --> PUSH_DOCKER["stage_push_docker"]
BUILD_DOCS --> PUSH_DOCS["stage_push_documentation"]
PUSH_PYTHON --> DEPLOY["stage_deploy"]
PUSH_DOCKER --> DEPLOY
PUSH_DOCS --> DEPLOY
end
subgraph "制品与注册表"
PUSH_PYTHON --> PYPI["PyPI 仓库"]
PUSH_DOCKER --> DOCKERHUB["Docker Hub"]
PUSH_DOCS --> DOCS_SITE["文档站点"]
DEPLOY --> DEMO["演示环境"]
DEPLOY --> STAGING["预发布环境"]
end
来源: .gitlab-ci.yml:1-10, mayan/apps/platform/templates/platform/gitlab-ci.tmpl:1-10
GitLab CI/CD 配置
管线配置由模板系统生成,并为每个阶段定义了具体的作业。主配置使用基于分支的触发器来控制不同类型构建的执行时机。
阶段定义
| 阶段 | 用途 | 作业 |
|---|---|---|
stage_test | 运行测试套件和升级测试 | job_postgres_test、job_sqlite_test、job_postgres_upgrade_test、job_sqlite_upgrade_test |
stage_build_python | 构建 Python wheel 包 | job_python_build |
stage_build_docker | 构建 Docker 镜像 | job_docker_build、job_docker_nightly、job_docker_testing |
stage_build_documentation | 构建 Sphinx 文档 | job_documentation_build |
stage_push_python | 发布到 PyPI | job_python_push_pypi |
stage_push_docker | 发布到 Docker Hub | job_docker_push_major、job_docker_push_minor |
stage_push_documentation | 部署文档 | job_documentation_push |
stage_deploy | 部署到环境 | job_demo_deploy、job_staging_deploy |
来源: .gitlab-ci.yml:1-10, mayan/apps/platform/templates/platform/gitlab-ci.tmpl:1-10
基于分支的触发器
管线使用分支命名规范来决定执行哪些作业:
graph LR
subgraph "发布分支"
REL_ALL_MAJOR["releases/all_major"] --> ALL_JOBS["所有作业"]
REL_ALL_MINOR["releases/all_minor"] --> ALL_JOBS
REL_DOCKER_MAJOR["releases/docker_major"] --> DOCKER_JOBS["仅 Docker 作业"]
REL_PYTHON_MAJOR["releases/python_major"] --> PYTHON_JOBS["仅 Python 作业"]
REL_TESTING["releases/testing"] --> TEST_JOBS["测试作业"]
REL_NIGHTLY["releases/nightly"] --> NIGHTLY_JOBS["每日构建"]
end
subgraph "测试分支"
TEST_ALL["tests/all"] --> FULL_TEST["完整测试套件"]
TEST_BASE["tests/base"] --> BASE_TEST["基础测试"]
TEST_UPGRADE["tests/upgrade"] --> UPGRADE_TEST["升级测试"]
end
subgraph "作业类型"
ALL_JOBS --> job_docker_build
ALL_JOBS --> job_python_build
ALL_JOBS --> job_documentation_build
DOCKER_JOBS --> job_docker_build
PYTHON_JOBS --> job_python_build
end
来源: .gitlab-ci.yml:29-35, .gitlab-ci.yml:362-373
构建作业实现
Docker 镜像构建流程
Docker 构建流程采用多阶段方法,具备依赖缓存功能,并支持在测试中使用多种数据库后端。
graph TD
subgraph "job_docker_build"
SETUP_ENV["加载 config.env"] --> DOCKER_LOGIN["docker login CI_REGISTRY"]
DOCKER_LOGIN --> INSTALL_DEPS["安装构建依赖"]
INSTALL_DEPS --> GET_VERSION["DOCKER_VERSION=cat docker/rootfs/version"]
GET_VERSION --> MAKE_BUILD["make docker-build"]
MAKE_BUILD --> TAG_IMAGE["docker tag mayanedms/mayanedms:VERSION"]
TAG_IMAGE --> RUN_TESTS["使用 PostgreSQL 运行 docker 测试"]
RUN_TESTS --> PUSH_IMAGE["docker push CI_REGISTRY_IMAGE"]
end
subgraph "依赖"
CONFIG_ENV["config.env"] --> SETUP_ENV
DOCKERFILE["docker/Dockerfile"] --> MAKE_BUILD
VERSION_FILE["docker/rootfs/version"] --> GET_VERSION
POSTGRES_SERVICE["postgres:12.11-alpine"] --> RUN_TESTS
end
来源: .gitlab-ci.yml:13-58, docker/Makefile:15-17
Python 包构建流程
Python 包使用标准的 setuptools 工作流进行构建,并生成 wheel 文件:
graph TD
subgraph "job_python_build"
LOAD_CONFIG["set -a && . ./config.env"] --> SETUP_VENV["python3 -m venv venv"]
SETUP_VENV --> INSTALL_PIP["pip install pip==PYTHON_PIP_VERSION"]
INSTALL_PIP --> INSTALL_BUILD_DEPS["pip install requirements/build.txt"]
INSTALL_BUILD_DEPS --> MAKE_WHEEL["make python-wheel"]
MAKE_WHEEL --> ARTIFACTS["制品: dist/"]
end
subgraph "job_python_push_pypi"
LOAD_CONFIG2["加载 config.env"] --> SETUP_VENV2["设置虚拟环境"]
SETUP_VENV2 --> INSTALL_TWINE["安装构建工具"]
INSTALL_TWINE --> SETUP_PYPIRC["echo PYPIRC > ~/.pypirc"]
SETUP_PYPIRC --> TWINE_UPLOAD["twine upload dist/* -r pypi"]
end
ARTIFACTS --> SETUP_VENV2
来源: .gitlab-ci.yml:233-257, .gitlab-ci.yml:259-280, Makefile:201-204
本地开发集成
管线通过 Makefile 目标与本地开发工作流集成,这些目标镜像了 CI/CD 流程:
本地构建目标
| 目标 | 用途 | 对应的 CI 作业 |
|---|---|---|
make test-all | 运行完整测试套件 | job_postgres_test |
make python-wheel | 构建 Python 包 | job_python_build |
make docker-build | 构建 Docker 镜像 | job_docker_build |
make docs-html | 构建文档 | job_documentation_build |
来源: Makefile:58-59, Makefile:201-204, docker/Makefile:15-17
GitLab 集成命令
Makefile 提供了触发 GitLab CI 管线的命令:
graph LR
subgraph "发布命令"
MAKE_ALL_MAJOR["make gitlab-release-all-major"] --> PUSH_ALL_MAJOR["git push origin HEAD:releases/all_major"]
MAKE_DOCKER_MAJOR["make gitlab-release-docker-major"] --> PUSH_DOCKER_MAJOR["git push origin HEAD:releases/docker_major"]
MAKE_PYTHON_MAJOR["make gitlab-release-python-major"] --> PUSH_PYTHON_MAJOR["git push origin HEAD:releases/python_major"]
end
subgraph "测试命令"
MAKE_TEST_ALL["make gitlab-tests-internal-all"] --> PUSH_TEST_ALL["git push internal HEAD:tests/all"]
MAKE_TEST_UPGRADE["make gitlab-tests-internal-upgrade"] --> PUSH_TEST_UPGRADE["git push internal HEAD:tests/upgrade"]
end
PUSH_ALL_MAJOR --> GITLAB_PIPELINE["GitLab CI 管线"]
PUSH_DOCKER_MAJOR --> GITLAB_PIPELINE
PUSH_PYTHON_MAJOR --> GITLAB_PIPELINE
PUSH_TEST_ALL --> GITLAB_PIPELINE
PUSH_TEST_UPGRADE --> GITLAB_PIPELINE
来源: Makefile:322-327, Makefile:354-359, Makefile:368-373
配置管理
构建管线使用基于环境变量和模板的集中式配置系统:
配置来源
graph TD
subgraph "配置文件"
CONFIG_ENV["config.env"] --> LITERALS["mayan/settings/literals.py"]
TEMPLATE_CI["mayan/apps/platform/templates/platform/gitlab-ci.tmpl"] --> GITLAB_CI[".gitlab-ci.yml"]
TEMPLATE_DOCKER["mayan/apps/platform/templates/platform/docker/dockerfile.tmpl"] --> DOCKERFILE["docker/Dockerfile"]
end
subgraph "关键变量"
PYTHON_PIP_VERSION["PYTHON_PIP_VERSION=22.2"]
DOCKER_POSTGRES_VERSION["DOCKER_POSTGRES_IMAGE_VERSION=postgres:12.11-alpine"]
DEFAULT_DATABASE_NAME["DEFAULT_DATABASE_NAME=mayan"]
GUNICORN_WORKERS["GUNICORN_WORKERS=3"]
end
CONFIG_ENV --> PYTHON_PIP_VERSION
CONFIG_ENV --> DOCKER_POSTGRES_VERSION
CONFIG_ENV --> DEFAULT_DATABASE_NAME
CONFIG_ENV --> GUNICORN_WORKERS
subgraph "模板生成"
MANAGE_PY["./manage.py platformtemplate"] --> GITLAB_CI
MANAGE_PY --> DOCKERFILE
end
来源: config.env:1-74, mayan/settings/literals.py:1-63, Makefile:115-117
环境变量使用
管线广泛使用环境变量进行配置:
| 类别 | 变量 | 用途 |
|---|---|---|
| Docker 注册表 | CI_REGISTRY_USER、CI_REGISTRY_PASSWORD、CI_REGISTRY_IMAGE | GitLab 容器注册表认证 |
| Docker Hub | DOCKER_HUB_USERNAME、DOCKER_HUB_PASSWORD | Docker Hub 发布 |
| 数据库 | POSTGRES_DB、POSTGRES_USER、POSTGRES_PASSWORD | 测试数据库配置 |
| PyPI | PYPIRC | Python 包发布凭证 |
| 部署 | DEMO_SSH_KNOWN_HOST、STAGING_PRIVATE_KEY | 环境部署 |
来源: .gitlab-ci.yml:16, .gitlab-ci.yml:80, .gitlab-ci.yml:278
测试集成
管线包含跨多个数据库后端和升级场景的全面测试:
测试矩阵
graph TD
subgraph "数据库测试"
POSTGRES_TEST["job_postgres_test"] --> POSTGRES_DB["PostgreSQL 12.11"]
SQLITE_TEST["job_sqlite_test"] --> SQLITE_DB["SQLite"]
end
subgraph "升级测试"
POSTGRES_UPGRADE["job_postgres_upgrade_test"] --> INSTALL_PREVIOUS["pip install mayan-edms<VERSION"]
SQLITE_UPGRADE["job_sqlite_upgrade_test"] --> INSTALL_PREVIOUS
INSTALL_PREVIOUS --> INITIAL_SETUP["mayan-edms.py initialsetup"]
INITIAL_SETUP --> CREATE_DOC["创建测试文档"]
CREATE_DOC --> BUILD_NEW["make python-wheel"]
BUILD_NEW --> INSTALL_NEW["pip install dist/mayan_edms-*.whl"]
INSTALL_NEW --> PERFORM_UPGRADE["mayan-edms.py common_perform_upgrade"]
end
subgraph "测试执行"
TEST_COMMAND["make test-all SETTINGS=mayan.settings.testing.gitlab-ci"]
MIGRATION_TEST["make test-all-migrations SETTINGS=mayan.settings.testing.gitlab-ci"]
end
POSTGRES_TEST --> TEST_COMMAND
SQLITE_TEST --> TEST_COMMAND
POSTGRES_TEST --> MIGRATION_TEST
SQLITE_TEST --> MIGRATION_TEST
来源: .gitlab-ci.yml:313-343, .gitlab-ci.yml:375-398, Makefile:16
部署管线
部署阶段处理到演示环境和预发布环境的自动化部署:
部署作业
graph LR
subgraph "job_demo_deploy"
DEMO_SSH_SETUP["设置 SSH 密钥"] --> DEMO_SCP["scp docker.sh 到演示服务器"]
DEMO_SCP --> DEMO_EXEC["ssh DELETE_VOLUMES=true ./docker.sh"]
end
subgraph "job_staging_deploy"
STAGING_SSH_SETUP["设置 SSH 密钥"] --> STAGING_SCP["scp docker.sh 到预发布服务器"]
STAGING_SCP --> STAGING_EXEC["ssh DOCKER_MAYAN_IMAGE=registry.gitlab.com/mayan-edms/mayan-edms:staging ./docker.sh"]
end
subgraph "环境变量"
DEMO_IP_ADDRESS["DEMO_IP_ADDRESS"]
STAGING_IP_ADDRESS["STAGING_IP_ADDRESS"]
DEMO_PRIVATE_KEY["DEMO_PRIVATE_KEY"]
STAGING_PRIVATE_KEY["STAGING_PRIVATE_KEY"]
end
DEMO_IP_ADDRESS --> DEMO_EXEC
STAGING_IP_ADDRESS --> STAGING_EXEC
DEMO_PRIVATE_KEY --> DEMO_SSH_SETUP
STAGING_PRIVATE_KEY --> STAGING_SSH_SETUP
来源: .gitlab-ci.yml:402-430, contrib/scripts/install/docker.sh
两个部署作业都配置了 when: manual,这意味着在执行之前需要显式批准。这为生产部署提供了安全机制,同时保持了自动化能力。
来源: .gitlab-ci.yml:415, .gitlab-ci.yml:430