docker-jupyterhub

Getting started

main reference

构建自己的docker镜像

修改自己想要的python环境在文件Dockerfile.jupyterhub

基础镜像是：jupyter/scipy-notebook 里面含有conda

docker buildx build -t hotwa/jupyterhub:latest . -f Dockerfile.jupyterhub --load

修改docker-compose.yml文件环境变量：DOCKER_NOTEBOOK_IMAGE

DOCKER_NOTEBOOK_IMAGE: hotwa/jupyterhub:latest

也可以通过 docker-compose 构建镜像

docker compose build

启动docker-compose.yml

docker compose up -d

停止

docker compose down

基本镜像

https://github.com/jupyter/docker-stacks/tree/main/images

jupyter-gpu

构建自己的基础镜像：

git clone https://github.com/jupyter/docker-stacks
cd docker-stacks/images/base-notebook
# 修改这个目录下面的 Dockerfile文件, 在后面加就行，里面已经有了mamba、conda
docker buildx build -t hotwa/notebook:latest . -f Dockerfile

Alternatives

• rocker/binder - From the R focused rocker-project, lets you run both RStudio and Jupyter either standalone or in a JupyterHub
• jupyter/repo2docker - Turn git repositories into Jupyter-enabled Docker Images
• openshift/source-to-image - A tool for building artifacts from source code and injecting them into docker images
• jupyter-on-openshift/jupyter-notebooks - OpenShift compatible S2I builder for basic notebook images

nvidia-container-toolkit add

参考了 llama 容器部署

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
    | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
    | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
    | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

Configure Docker to use Nvidia driver

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

配置完成之后：

docker-compose up -d

在jupyterhub_config.py中添加：这样jupyterhub的容器启动镜像就可以使用宿主机的显卡了

# GPU 和网络配置
c.DockerSpawner.extra_host_config = {
    #'network_mode': 'host',
    'runtime': 'nvidia'
}
c.DockerSpawner.environment = {
    'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility',
    'NVIDIA_VISIBLE_DEVICES': 'all'
}

清除卷

docker volume ls
docker-compose down
docker rm <container_name_or_id>
docker volume rm basic-example_jupyterhub-data
docker volume rm docker-jupyterhub_jupyterhub-data
docker volume rm jupyterhub-user-admin

构建

构建基础notebook镜像

docker buildx build -t hotwa/notebook:latest -f Dockerfile.base-notebook . --load

构建jupyterhub镜像

docker compose build

推送镜像至阿里云私有仓库

docker login --username=ze.ga@qq.com registry.cn-hangzhou.aliyuncs.com
# 重命名
docker tag 2ad3860183ce registry.cn-hangzhou.aliyuncs.com/hotwa/jupyterhub:latest
docker tag ddf815cbaa9b registry.cn-hangzhou.aliyuncs.com/hotwa/notebook:latest
# 推送
docker login --username=ze.ga@qq.com registry.cn-hangzhou.aliyuncs.com
docker push registry.cn-hangzhou.aliyuncs.com/hotwa/jupyterhub:latest
docker push registry.cn-hangzhou.aliyuncs.com/hotwa/notebook:latest
# 拉取
docker pull registry.cn-hangzhou.aliyuncs.com/hotwa/jupyterhub:latest
docker pull registry.cn-hangzhou.aliyuncs.com/hotwa/notebook:latest