# Woodpecker CI - S3 对象存储上传指南

本指南介绍如何在 Woodpecker CI 中使用 S3 插件将文件和文件夹上传到对象存储（兼容 S3 API 的存储服务，如 AWS S3、MinIO、RustFS 等）。

## 📋 目录

- [前置准备](#前置准备)
- [配置密钥](#配置密钥)
- [基础用法](#基础用法)
  - [上传单个文件](#上传单个文件)
  - [上传多个文件](#上传多个文件)
  - [上传整个文件夹](#上传整个文件夹)
- [高级用法](#高级用法)
- [完整示例](#完整示例)
- [故障排查](#故障排查)

---

## 前置准备

### 1. 安装 plugin-s3

确保您的 Woodpecker Agent 环境中已安装 `plugin-s3`：

```bash
# 使用 Homebrew (macOS)
brew install plugin-s3

# 或使用 Go 安装
go install github.com/woodpecker-ci/plugin-s3@latest

# 验证安装
which plugin-s3
```

### 2. 准备 S3 凭证

您需要以下信息：
- **Access Key ID**: S3 访问密钥 ID
- **Secret Access Key**: S3 密钥
- **Bucket 名称**: 目标存储桶
- **Endpoint**: S3 服务端点（如使用 AWS S3 可省略）
- **Region**: 区域（如 `us-east-1`）

---

## 配置密钥

### 通过 Woodpecker UI 配置

1. 进入您的仓库页面
2. 点击 **Settings** → **Secrets**
3. 添加以下密钥：

| 密钥名称 | 说明 | 示例值 |
|---------|------|--------|
| `AWS_ACCESS_KEY_ID` | S3 访问密钥 ID | `AKIAIOSFODNN7EXAMPLE` |
| `AWS_SECRET_ACCESS_KEY` | S3 密钥 | `wJalrXUtnFEMI/K7MDENG/...` |
| `S3_BUCKET` | 存储桶名称 | `my-bucket` |
| `S3_ENDPOINT` | S3 端点（自建服务） | `https://s3.example.com:9000` |
| `AWS_DEFAULT_REGION` | AWS 区域 | `us-east-1` |

4. 确保勾选适当的事件类型（如 `push`、`manual`、`pull_request`）

### 通过 CLI 配置

```bash
# 添加 Access Key
woodpecker-cli repo secret add \
  --repository your-org/your-repo \
  --name AWS_ACCESS_KEY_ID \
  --value "your-access-key-id"

# 添加 Secret Key
woodpecker-cli repo secret add \
  --repository your-org/your-repo \
  --name AWS_SECRET_ACCESS_KEY \
  --value "your-secret-access-key"

# 添加 Bucket
woodpecker-cli repo secret add \
  --repository your-org/your-repo \
  --name S3_BUCKET \
  --value "my-bucket"

# 添加 Endpoint (可选，用于自建 S3 服务)
woodpecker-cli repo secret add \
  --repository your-org/your-repo \
  --name S3_ENDPOINT \
  --value "https://s3.example.com:9000"

# 添加 Region
woodpecker-cli repo secret add \
  --repository your-org/your-repo \
  --name AWS_DEFAULT_REGION \
  --value "us-east-1"
```

---

## 基础用法

### 上传单个文件

上传单个文件到 S3 存储桶：

```yaml
steps:
  - name: upload-single-file
    image: /bin/zsh
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        # 创建测试文件
        echo "Hello from Woodpecker CI" > hello.txt
        
        # 上传到 S3
        export PLUGIN_SOURCE="hello.txt"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="uploads/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        plugin-s3
```

**结果**: `hello.txt` 将被上传到 `s3://your-bucket/uploads/hello.txt`

---

### 上传多个文件

使用通配符上传多个文件：

```yaml
steps:
  - name: upload-multiple-files
    image: /bin/zsh
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        # 创建多个测试文件
        echo "File 1" > file1.txt
        echo "File 2" > file2.txt
        echo "File 3" > file3.log
        
        # 上传所有 .txt 文件
        export PLUGIN_SOURCE="*.txt"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="logs/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        plugin-s3
```

**结果**: 所有 `.txt` 文件将被上传到 `s3://your-bucket/logs/`

---

### 上传整个文件夹

递归上传整个目录及其子目录：

```yaml
steps:
  - name: upload-folder
    image: /bin/zsh
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        # 创建目录结构
        mkdir -p dist/css dist/js dist/images
        echo "body { color: red; }" > dist/css/style.css
        echo "console.log('hello');" > dist/js/app.js
        echo "placeholder" > dist/images/logo.png
        echo "<html>Hello</html>" > dist/index.html
        
        # 上传整个 dist 文件夹
        export PLUGIN_SOURCE="dist/**/*"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="website/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        export PLUGIN_STRIP_PREFIX="dist/"
        plugin-s3
```

**结果**: 
```
s3://your-bucket/website/css/style.css
s3://your-bucket/website/js/app.js
s3://your-bucket/website/images/logo.png
s3://your-bucket/website/index.html
```

---

## 高级用法

### 设置文件访问权限

```yaml
export PLUGIN_ACL="public-read"  # 公开可读
# 可选值: private, public-read, public-read-write, authenticated-read
```

### 设置缓存控制

```yaml
export PLUGIN_CACHE_CONTROL="max-age=3600"  # 缓存 1 小时
```

### 设置内容类型

```yaml
export PLUGIN_CONTENT_TYPE="text/html"
export PLUGIN_CONTENT_ENCODING="gzip"
```

### 删除目标文件夹中的旧文件

```yaml
export PLUGIN_DELETE=true  # 上传前删除目标路径的现有文件
```

### 使用服务器端加密

```yaml
export PLUGIN_ENCRYPTION="AES256"  # 或 "aws:kms"
```

### 完整高级示例

```yaml
steps:
  - name: deploy-website
    image: /bin/zsh
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        export PLUGIN_SOURCE="public/**/*"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="production/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        export PLUGIN_STRIP_PREFIX="public/"
        export PLUGIN_ACL="public-read"
        export PLUGIN_CACHE_CONTROL="max-age=31536000"
        export PLUGIN_DELETE=true
        plugin-s3
```

---

## 完整示例

### 最小化示例 - `.woodpecker.yml`

这是一个完整的 Woodpecker 配置文件，演示如何上传文件和文件夹：

```yaml
# .woodpecker.yml
labels:
  platform: linux/amd64

when:
  event: [push, manual]

steps:
  # 第一步：构建项目（示例）
  - name: build
    image: node:18-alpine
    commands:
      - echo "Building project..."
      - mkdir -p dist/assets
      - echo '<!DOCTYPE html><html><body><h1>Hello World</h1></body></html>' > dist/index.html
      - echo 'body { font-family: Arial; }' > dist/assets/style.css
      - echo 'console.log("App loaded");' > dist/assets/app.js
      - echo "Build complete!"
      - ls -R dist/

  # 第二步：上传单个日志文件
  - name: upload-build-log
    image: alpine:latest
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        # 创建构建日志
        date > build.log
        echo "Build completed successfully" >> build.log
        
        # 上传日志文件
        export PLUGIN_SOURCE="build.log"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="logs/build-${CI_COMMIT_SHA:0:8}.log"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        plugin-s3
        
        echo "✅ Build log uploaded to s3://$S3_BUCKET/logs/"

  # 第三步：上传整个 dist 文件夹
  - name: upload-dist-folder
    image: alpine:latest
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        # 上传整个 dist 目录
        export PLUGIN_SOURCE="dist/**/*"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="website/${CI_COMMIT_BRANCH}/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        export PLUGIN_STRIP_PREFIX="dist/"
        export PLUGIN_ACL="public-read"
        plugin-s3
        
        echo "✅ Website deployed to s3://$S3_BUCKET/website/${CI_COMMIT_BRANCH}/"

  # 第四步：上传构建产物（压缩包）
  - name: upload-artifacts
    image: alpine:latest
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        # 创建压缩包
        apk add --no-cache zip
        zip -r dist-${CI_COMMIT_SHA:0:8}.zip dist/
        
        # 上传压缩包
        export PLUGIN_SOURCE="dist-*.zip"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="releases/${CI_COMMIT_BRANCH}/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        plugin-s3
        
        echo "✅ Artifacts uploaded to s3://$S3_BUCKET/releases/${CI_COMMIT_BRANCH}/"
```

---

### macOS 本地 Agent 示例

如果您使用本地 macOS agent（如您的 Mac mini），配置略有不同：

```yaml
# .woodpecker.yml (macOS Local Agent)
labels:
  host: Mac-mini.local
  platform: darwin/arm64

when:
  event: [push, manual]

steps:
  - name: build-app
    image: /bin/zsh
    commands:
      - echo "Building on macOS..."
      - mkdir -p build/output
      - echo "Binary placeholder" > build/output/app
      - echo "Config file" > build/output/config.json
      - ls -R build/

  - name: upload-single-file
    image: /bin/zsh
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        echo "📦 Uploading config file..."
        export PLUGIN_SOURCE="build/output/config.json"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="configs/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        plugin-s3
        echo "✅ Config uploaded"

  - name: upload-build-folder
    image: /bin/zsh
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY:
        from_secret: AWS_SECRET_ACCESS_KEY
      S3_BUCKET:
        from_secret: S3_BUCKET
      S3_ENDPOINT:
        from_secret: S3_ENDPOINT
    commands:
      - |
        echo "📦 Uploading entire build folder..."
        export PLUGIN_SOURCE="build/output/**/*"
        export PLUGIN_BUCKET="$S3_BUCKET"
        export PLUGIN_TARGET="builds/macos-$(date +%Y%m%d-%H%M%S)/"
        export PLUGIN_ENDPOINT="$S3_ENDPOINT"
        export PLUGIN_PATH_STYLE=true
        export PLUGIN_STRIP_PREFIX="build/output/"
        plugin-s3
        echo "✅ Build folder uploaded"
```

---

## 故障排查

### 问题 1: 密钥未传递

**错误信息**: `No S3 credentials found`

**解决方案**:
- ✅ 确保使用 `from_secret:` 语法而不是 `${SECRET_NAME}`
- ✅ 检查 Woodpecker UI 中密钥名称是否完全匹配
- ✅ 确认密钥事件类型包含当前触发事件（如 `manual`）

```yaml
# ❌ 错误
environment:
  AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}

# ✅ 正确
environment:
  AWS_ACCESS_KEY_ID:
    from_secret: AWS_ACCESS_KEY_ID
```

### 问题 2: plugin-s3 未找到

**错误信息**: `plugin-s3: command not found`

**解决方案**:
```bash
# 安装 plugin-s3
brew install plugin-s3

# 或添加到 PATH
export PATH=$PATH:/usr/local/bin
```

### 问题 3: 端点连接失败

**错误信息**: `connection refused` 或 `timeout`

**解决方案**:
- ✅ 检查 `S3_ENDPOINT` 格式（需包含协议，如 `https://`）
- ✅ 确认防火墙规则允许访问
- ✅ 验证 SSL 证书（自签名证书可能需要额外配置）

### 问题 4: 权限被拒绝

**错误信息**: `Access Denied` 或 `403 Forbidden`

**解决方案**:
- ✅ 验证 Access Key 和 Secret Key 是否正确
- ✅ 检查 IAM 策略是否允许 `s3:PutObject` 权限
- ✅ 确认 bucket 存在且有写入权限

### 调试技巧

添加调试输出以检查环境变量：

```yaml
commands:
  - |
    echo "=== 调试信息 ==="
    echo "Bucket: ${S3_BUCKET}"
    echo "Endpoint: ${S3_ENDPOINT}"
    echo "Access Key (前10位): ${AWS_ACCESS_KEY_ID:0:10}..."
    echo "================="
    
    # 继续执行上传...
```

---

## 环境变量参考

| 变量名 | 说明 | 必需 | 示例 |
|--------|------|------|------|
| `PLUGIN_SOURCE` | 源文件路径（支持通配符） | ✅ | `dist/**/*` |
| `PLUGIN_BUCKET` | 目标 bucket | ✅ | `my-bucket` |
| `PLUGIN_TARGET` | 目标路径前缀 | ❌ | `uploads/` |
| `PLUGIN_ENDPOINT` | S3 端点 URL | ❌ | `https://s3.example.com` |
| `PLUGIN_PATH_STYLE` | 使用路径风格 URL | ❌ | `true` |
| `PLUGIN_STRIP_PREFIX` | 移除源路径前缀 | ❌ | `dist/` |
| `PLUGIN_ACL` | 访问控制列表 | ❌ | `public-read` |
| `PLUGIN_CACHE_CONTROL` | 缓存控制头 | ❌ | `max-age=3600` |
| `PLUGIN_DELETE` | 上传前删除目标文件 | ❌ | `true` |
| `PLUGIN_ENCRYPTION` | 服务器端加密 | ❌ | `AES256` |

---

## 总结

本指南涵盖了在 Woodpecker CI 中使用 S3 上传的所有常见场景：

✅ **单文件上传**: 适用于日志、配置文件等  
✅ **多文件上传**: 使用通配符批量上传  
✅ **文件夹上传**: 递归上传整个目录结构  
✅ **高级配置**: ACL、缓存、加密等  

如有问题，请参考 [Woodpecker 官方文档](https://woodpecker-ci.org/docs/) 或提交 Issue。

---

**License**: MIT  
**维护者**: Your Team  
**最后更新**: 2025-10-12