Jupyter Journey

IllustrisTNG - Data Access - FAQ

Using Docker Compose

Prepare folders

mkdir -p ~/jupyter && cd ~/jupyter
touch docker-compose.yml
mkdir -p data
mkdir -p configurations

Edit Docker Compose File

version: '3'

services:
  jupyter:
    image: jupyter/datascience-notebook:latest
    container_name: jupyter
    volumes:
        - ./data:/home/azat/
        - ./configurations:/home/azat/.jupyter
    ports:
      - 8888:8888
    restart: always
    labels:
        - "traefik.enable=true"
        - "traefik.docker.network=proxy"
        - "traefik.http.routers.jupyter-secure.entrypoints=websecure"
        - "traefik.http.routers.jupyter-secure.rule=Host(`lab.azat.host`)"
        - "traefik.http.routers.jupyter-secure.service=jupyter-service"
        - "traefik.http.services.jupyter-service.loadbalancer.server.port=8888"
    networks:
        - proxy
    user: root
    command: "start-notebook.sh"
    environment:
          NB_USER: azat
          NB_UID: 1001
          NG_GID: 1001
          CHOWN_HOME: 'yes'
          CHOWN_HOME_OPTS: -R
          GRANT_SUDO: 'yes'
          RESTARTABLE: 'yes'

networks:
  proxy:
    external: true

Run the docker compose

docker compose up -d

Setup the Jupyter password

docker compose run --rm jupyter bash

jupyter server password

CleanShot 2022-10-18 at 00.03.02

mv /home/jovyan/.jupyter/jupyter_server_config.json /home/azat/.jupyter/jupyter_server_config.json && exit

sudo chown -R azat:azat ./configurations

and restart the jupyter docker compose.

docker compose restart

CleanShot 2022-10-18 at 00.06.43

Notice: this should take some time.

Installing other kernels

from the jupyter online terminal:

mamba install xeus-cling notebook -c QuantStack -c conda-forge

mamba install -c conda-forge ipydrawio

Bare Metal

Prepare installation

sudo apt-get update --yes && \
    sudo apt-get install --yes --no-install-recommends \
    fonts-dejavu \
    gfortran \
    gcc && \
    sudo apt-get clean && sudo rm -rf /var/lib/apt/lists/*

Install Miniforge

Miniforge

wget "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh

Install Mamba

conda install mamba -n base -c conda-forge	

Install Nodejs and Jupyter Lab

mamba install -c conda-forge nodejs

mamba install -c conda-forge jupyterlab

Generate Jupyter Server Config

jupyter server --generate-config

CleanShot 2022-10-13 at 18.29.06

jupyter server password

CleanShot 2022-10-13 at 18.30.06

Setup the Jupyter Server

mkdir -p /home/azat/jupyter

code ~/.jupyter

Edit jupyter_server_config.py

# Configuration file for jupyter-server.

c = get_config()  # noqa


# Azat
# c.ServerApp.ip = '*'
c.ServerApp.allow_origin = '*'
c.ServerApp.port = 8888
c.ServerApp.open_browser = False
c.ServerApp.allow_remote_access = True
c.ServerApp.root_dir = '/home/azat/jupyter'
c.ServerApp.quit_button = False
# Azat 

Run the Jupyter Server

CleanShot 2022-10-17 at 00.10.26

Install Certbot and Generate SSL Keys

sudo snap install core; sudo snap refresh core

sudo snap install --classic certbot

sudo ln -s /snap/bin/certbot /usr/bin/certbot

Generate SSL certificate for domain name

sudo certbot --nginx -d <DOMAIN>

CleanShot 2022-10-17 at 00.17.02

Setup Nginx

sudo apt install nginx -y

create file /home/azat/.jupyter/jupyter.conf

# top-level http config for websocket headers
# If Upgrade is defined, Connection = upgrade
# If Upgrade is empty, Connection = close
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

# HTTP server to redirect all 80 traffic to SSL/HTTPS
server {
    listen 80;
    server_name <domain>;

    # Tell all requests to port 80 to be 302 redirected to HTTPS
    return 302 https://$host$request_uri;
}

# HTTPS server to handle JupyterHub
server {
    listen 443 ssl;
    server_name <domain>;

    ssl_certificate /etc/letsencrypt/live/lab.azat.host/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/lab.azat.host/privkey.pem;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-        SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security max-age=15768000;

    # Managing literal requests to the JupyterHub front end
    location / {
        proxy_pass http://127.0.0.1:8888;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # websocket headers
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header X-Scheme $scheme;
        proxy_buffering off;
    }

    # Managing requests to verify letsencrypt host
    location ~ /.well-known {
        allow all;
    }
}

Setup the `systemd`

edit /home/azat/.jupyter/jupyter.service

[Unit]
Description=Jupyter Lab
After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
Restart=always
RestartSec=1
User=azat
WorkingDirectory=/home/azat/Jupyter
ExecStart=/home/azat/mambaforge/bin/jupyter lab
[Install]
WantedBy=multi-user.target

sudo cp ./jupyter.service /etc/systemd/system

sudo systemctl daemon-reload

sudo systemctl enable jupyter

sudo systemctl restart jupyter

sudo journalctl -u jupyter

Bug Fix

Jupyter Resources 404 .

CleanShot 2022-10-17 at 00.32.43

CleanShot 2022-10-17 at 00.33.34

Solution: Turn off Rocket Loader from Cloud Flare.

PREVIOUSA Swift Tour

NEXT图解 HTTP （书）