lingyuzeng/hydra-ml-config-template
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first add

Browse Source
This commit is contained in:
lingyuzeng
2025-10-12 19:39:12 +08:00
commit 592d4a824e
22 changed files with 469 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
.gitignore vendored Normal file
View File

@@ -0,0 +1,30 @@
# Python
__pycache__/
*.py[cod]
*.so
.Python
*.egg-info/
# Virtual Environment
.venv/
venv/
# Hydra
outputs/
multirun/
.hydra/
# IDEs
.vscode/
.idea/
.DS_Store
# Data and Models
data/
checkpoints/
*.pth
*.pt
# Logs
logs/
*.log

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

138
README.md Normal file
View File

@@ -0,0 +1,138 @@
# 🎯 Hydra ML Config Template
<div align="center">
**优雅的Python配置管理解决方案 | 告别杂乱的硬编码**
[快速开始](#快速开始) • [使用场景](#使用场景) • [最佳实践](#最佳实践) • [常见问题](#常见问题)
</div>
---
## 📖 项目简介
这是一个基于 [Hydra](https://hydra.cc/) 的Python配置管理模板仓库,专为机器学习和数据科学项目设计。通过分层配置、动态组合、命令行覆盖等特性,让你的项目配置管理变得简单优雅。
### ✨ 核心特性
- 🔧 **分层配置** - 模块化管理,配置文件清晰有序
- 🎨 **动态组合** - 灵活切换不同配置组合
- 🚀 **命令行覆盖** - 无需修改代码即可调整参数
- 📊 **自动实验管理** - 每次运行自动创建独立输出目录
- 🔄 **多任务运行** - 一次性运行多组超参数实验
- 💾 **配置版本化** - 完整保存每次实验的配置
---
## 🚀 快速开始
### 环境安装
本项目使用 [uv](https://github.com/astral-sh/uv) 进行依赖管理,这是一个极速的Python包管理工具。
#### 1. 安装 uv
```bash
# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
# 或使用 pip
pip install uv
```
#### 2. 克隆仓库
```bash
git clone <your-gitea-url>/hydra-ml-config-template.git
cd hydra-ml-config-template
```
#### 3. 创建虚拟环境并安装依赖
```bash
# 使用 uv 创建虚拟环境
uv venv
# 激活虚拟环境
# Linux/macOS:
source .venv/bin/activate
# Windows:
.venv\Scripts\activate
# 安装依赖
uv pip install -e .
```
#### 4. 运行示例
```bash
# 基础示例
python examples/01_basic/train.py
# 使用不同配置
python examples/02_ml_training/train.py model=transformer optimizer=sgd
# 多任务运行
python examples/02_ml_training/train.py -m learning_rate=0.001,0.01,0.1
```
---
## 🎯 使用场景
### 场景1: 基础配置管理
```bash
cd examples/01_basic
python train.py learning_rate=0.01
```
### 场景2: 机器学习训练配置
```bash
cd examples/02_ml_training
python train.py model=transformer optimizer=sgd
python train.py learning_rate=0.01 batch_size=64
```
### 场景3: 多环境配置
```bash
cd examples/04_multi_env
python app.py env=dev # 开发环境
python app.py env=prod # 生产环境
```
---
## 📚 学习资源
- [Hydra官方文档](https://hydra.cc/)
- [项目结构说明](PROJECT_STRUCTURE.md)
- [最佳实践](docs/best_practices.md)
---
## 🤝 贡献
欢迎提交Issue和Pull Request!
---
## 📄 许可证
MIT License
---
<div align="center">
**如果这个模板对你有帮助,请给个 ⭐️ Star!**
Made with ❤️ for ML Engineers
</div>

41
docs/best_practices.md Normal file
View File

@@ -0,0 +1,41 @@
# 🎯 Hydra配置管理最佳实践
## 1. 配置文件组织
按功能模块组织配置文件：
```
conf/
├── config.yaml
├── model/
├── optimizer/
└── dataset/
```
## 2. 变量插值
使用变量插值避免重复：
```yaml
data_dir: /data/project
train_dir: ${data_dir}/train
val_dir: ${data_dir}/val
```
## 3. 命令行覆盖
灵活调整参数：
```bash
python train.py learning_rate=0.01 batch_size=64
```
## 4. 多任务运行
批量实验：
```bash
python train.py -m learning_rate=0.001,0.01,0.1
```
更多信息请访问 [Hydra官方文档](https://hydra.cc/)

6
examples/01_basic/conf/config.yaml Normal file
View File

@@ -0,0 +1,6 @@
# 基础配置示例
learning_rate: 0.001
batch_size: 32
epochs: 100
model_name: "ResNet50"
device: "cuda"

27
examples/01_basic/train.py Normal file
View File

@@ -0,0 +1,27 @@
"""基础Hydra配置管理示例"""
import hydra
from omegaconf import DictConfig
from rich.console import Console
from rich.table import Table
console = Console()
@hydra.main(version_base=None, config_path="conf", config_name="config")
def train(cfg: DictConfig) -> None:
console.print("\n[bold green]🚀 基础Hydra配置示例[/bold green]\n")
table = Table(title="当前配置")
table.add_column("参数", style="cyan")
table.add_column("值", style="yellow")
table.add_row("学习率", str(cfg.learning_rate))
table.add_row("批次大小", str(cfg.batch_size))
table.add_row("训练轮数", str(cfg.epochs))
table.add_row("模型名称", cfg.model_name)
console.print(table)
console.print("\n[bold green]✅ 训练完成！[/bold green]\n")
if __name__ == "__main__":
train()

10
examples/02_ml_training/conf/config.yaml Normal file
View File

@@ -0,0 +1,10 @@
defaults:
- model: resnet
- optimizer: adam
- dataset: cifar10
- _self_
learning_rate: 0.001
batch_size: 32
epochs: 100
device: cuda

5
examples/02_ml_training/conf/dataset/cifar10.yaml Normal file
View File

@@ -0,0 +1,5 @@
name: CIFAR-10
num_classes: 10
image_size: 32
train_samples: 50000
val_samples: 10000

5
examples/02_ml_training/conf/dataset/imagenet.yaml Normal file
View File

@@ -0,0 +1,5 @@
name: ImageNet
num_classes: 1000
image_size: 224
train_samples: 1281167
val_samples: 50000

5
examples/02_ml_training/conf/model/resnet.yaml Normal file
View File

@@ -0,0 +1,5 @@
name: ResNet50
layers: 50
pretrained: true
num_classes: 1000
dropout: 0.5

7
examples/02_ml_training/conf/model/transformer.yaml Normal file
View File

@@ -0,0 +1,7 @@
name: ViT-B/16
layers: 12
pretrained: true
num_classes: 1000
hidden_dim: 768
num_heads: 12
patch_size: 16

4
examples/02_ml_training/conf/optimizer/adam.yaml Normal file
View File

@@ -0,0 +1,4 @@
name: Adam
lr: ${learning_rate}
betas: [0.9, 0.999]
weight_decay: 0.0001

5
examples/02_ml_training/conf/optimizer/sgd.yaml Normal file
View File

@@ -0,0 +1,5 @@
name: SGD
lr: ${learning_rate}
momentum: 0.9
weight_decay: 0.0005
nesterov: true

32
examples/02_ml_training/train.py Normal file
View File

@@ -0,0 +1,32 @@
"""机器学习训练配置示例"""
import hydra
from omegaconf import DictConfig
from rich.console import Console
from rich.tree import Tree
console = Console()
@hydra.main(version_base=None, config_path="conf", config_name="config")
def train(cfg: DictConfig) -> None:
console.print("\n[bold green]🤖 机器学习训练配置示例[/bold green]\n")
tree = Tree("🎯 Training Configuration")
model_tree = tree.add("[yellow]Model")
model_tree.add(f"Name: {cfg.model.name}")
model_tree.add(f"Layers: {cfg.model.layers}")
optimizer_tree = tree.add("[cyan]Optimizer")
optimizer_tree.add(f"Name: {cfg.optimizer.name}")
optimizer_tree.add(f"Learning Rate: {cfg.optimizer.lr}")
dataset_tree = tree.add("[magenta]Dataset")
dataset_tree.add(f"Name: {cfg.dataset.name}")
dataset_tree.add(f"Classes: {cfg.dataset.num_classes}")
console.print(tree)
console.print("\n[bold green]✅ 配置加载完成！[/bold green]\n")
if __name__ == "__main__":
train()

30
examples/04_multi_env/app.py Normal file
View File

@@ -0,0 +1,30 @@
"""多环境配置管理示例"""
import hydra
from omegaconf import DictConfig
from rich.console import Console
from rich.table import Table
console = Console()
@hydra.main(version_base=None, config_path="conf", config_name="config")
def run_app(cfg: DictConfig) -> None:
env_colors = {"dev": "yellow", "test": "cyan", "prod": "red"}
color = env_colors.get(cfg.env.name, "white")
console.print(f"\n[bold {color}]🌍 {cfg.env.name.upper()} 环境配置[/bold {color}]\n")
table = Table()
table.add_column("配置项", style="cyan")
table.add_column("值", style="yellow")
table.add_row("环境名称", cfg.env.name)
table.add_row("调试模式", "✅" if cfg.env.debug else "❌")
table.add_row("数据库主机", cfg.database.host)
table.add_row("服务器端口", str(cfg.server.port))
console.print(table)
console.print()
if __name__ == "__main__":
run_app()

11
examples/04_multi_env/conf/config.yaml Normal file
View File

@@ -0,0 +1,11 @@
defaults:
- env: dev
- _self_
database:
host: localhost
port: 5432
server:
host: 0.0.0.0
port: 8000

9
examples/04_multi_env/conf/env/dev.yaml vendored Normal file
View File

@@ -0,0 +1,9 @@
name: dev
debug: true
database:
host: localhost
port: 5432
server:
port: 8000

9
examples/04_multi_env/conf/env/prod.yaml vendored Normal file
View File

@@ -0,0 +1,9 @@
name: prod
debug: false
database:
host: prod-db.example.com
port: 5432
server:
port: 8000

9
examples/04_multi_env/conf/env/test.yaml vendored Normal file
View File

@@ -0,0 +1,9 @@
name: test
debug: false
database:
host: test-db.example.com
port: 5432
server:
port: 8080

28
pyproject.toml Normal file
View File

@@ -0,0 +1,28 @@
[project]
name = "hydra-ml-config-template"
version = "1.0.0"
description = "Python Hydra配置管理最佳实践模板，专为机器学习项目设计"
authors = [
{name = "Your Name", email = "your.email@example.com"}
]
readme = "README.md"
requires-python = ">=3.8"
license = {text = "MIT"}
dependencies = [
"hydra-core>=1.3.0",
"omegaconf>=2.3.0",
"pyyaml>=6.0",
"rich>=13.0.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0.0",
"black>=23.0.0",
"ruff>=0.1.0",
]
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

33
quickstart.sh Executable file
View File

@@ -0,0 +1,33 @@
#!/bin/bash
set -e
echo "🚀 Hydra ML Config Template - 快速开始"
echo "========================================"
# 安装 uv（如果需要）
if ! command -v uv &> /dev/null; then
echo "📦 安装 uv..."
curl -LsSf https://astral.sh/uv/install.sh | sh
fi
# 创建虚拟环境
echo "🔧 创建虚拟环境..."
uv venv
# 激活虚拟环境并安装依赖
echo "📥 安装依赖..."
source .venv/bin/activate
uv pip install -e .
# 运行示例
echo "🎯 运行示例..."
echo ""
echo "1️⃣ 基础示例"
cd examples/01_basic && python train.py && cd ../..
echo ""
echo "2️⃣ 机器学习训练示例"
cd examples/02_ml_training && python train.py && cd ../..
echo ""
echo "✨ 完成！查看 README.md 了解更多"

4
requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
hydra-core>=1.3.0
omegaconf>=2.3.0
pyyaml>=6.0
rich>=13.0.0