Jupyter Journey

IllustrisTNG - Data Access - FAQ

Using Docker Compose

Prepare folders

mkdir -p ~/jupyter && cd ~/jupyter
touch docker-compose.yml
mkdir -p data
mkdir -p configurations

Edit Docker Compose File

version: '3'

    image: jupyter/datascience-notebook:latest
    container_name: jupyter
        - ./data:/home/azat/
        - ./configurations:/home/azat/.jupyter
      - 8888:8888
    restart: always
        - "traefik.enable=true"
        - "traefik.docker.network=proxy"
        - "traefik.http.routers.jupyter-secure.entrypoints=websecure"
        - "traefik.http.routers.jupyter-secure.rule=Host(`lab.azat.host`)"
        - "traefik.http.routers.jupyter-secure.service=jupyter-service"
        - "traefik.http.services.jupyter-service.loadbalancer.server.port=8888"
        - proxy
    user: root
    command: "start-notebook.sh"
          NB_USER: azat
          NB_UID: 1001
          NG_GID: 1001
          CHOWN_HOME: 'yes'
          CHOWN_HOME_OPTS: -R
          GRANT_SUDO: 'yes'
          RESTARTABLE: 'yes'

    external: true

Run the docker compose

docker compose up -d

Setup the Jupyter password

docker compose run --rm jupyter bash
jupyter server password

CleanShot 2022-10-18 at 00.03.02

mv /home/jovyan/.jupyter/jupyter_server_config.json /home/azat/.jupyter/jupyter_server_config.json && exit
sudo chown -R azat:azat ./configurations

and restart the jupyter docker compose.

docker compose restart

CleanShot 2022-10-18 at 00.06.43

Notice: this should take some time.

Installing other kernels

from the jupyter online terminal:

mamba install xeus-cling notebook -c QuantStack -c conda-forge

mamba install -c conda-forge ipydrawio

Bare Metal

Prepare installation

sudo apt-get update --yes && \
    sudo apt-get install --yes --no-install-recommends \
    fonts-dejavu \
    gfortran \
    gcc && \
    sudo apt-get clean && sudo rm -rf /var/lib/apt/lists/*

Install Miniforge


wget "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh

Install Mamba

conda install mamba -n base -c conda-forge	

Install Nodejs and Jupyter Lab

mamba install -c conda-forge nodejs
mamba install -c conda-forge jupyterlab

Generate Jupyter Server Config

jupyter server --generate-config

CleanShot 2022-10-13 at 18.29.06

jupyter server password

CleanShot 2022-10-13 at 18.30.06

Setup the Jupyter Server

mkdir -p /home/azat/jupyter
code ~/.jupyter

Edit jupyter_server_config.py

# Configuration file for jupyter-server.

c = get_config()  # noqa

# Azat
# c.ServerApp.ip = '*'
c.ServerApp.allow_origin = '*'
c.ServerApp.port = 8888
c.ServerApp.open_browser = False
c.ServerApp.allow_remote_access = True
c.ServerApp.root_dir = '/home/azat/jupyter'
c.ServerApp.quit_button = False
# Azat 

Run the Jupyter Server

CleanShot 2022-10-17 at 00.10.26

Install Certbot and Generate SSL Keys

sudo snap install core; sudo snap refresh core

sudo snap install --classic certbot

sudo ln -s /snap/bin/certbot /usr/bin/certbot

Generate SSL certificate for domain name

sudo certbot --nginx -d <DOMAIN>

CleanShot 2022-10-17 at 00.17.02

Setup Nginx

sudo apt install nginx -y

create file /home/azat/.jupyter/jupyter.conf

# top-level http config for websocket headers
# If Upgrade is defined, Connection = upgrade
# If Upgrade is empty, Connection = close
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;

# HTTP server to redirect all 80 traffic to SSL/HTTPS
server {
    listen 80;
    server_name <domain>;

    # Tell all requests to port 80 to be 302 redirected to HTTPS
    return 302 https://$host$request_uri;

# HTTPS server to handle JupyterHub
server {
    listen 443 ssl;
    server_name <domain>;

    ssl_certificate /etc/letsencrypt/live/lab.azat.host/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/lab.azat.host/privkey.pem;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security max-age=15768000;

    # Managing literal requests to the JupyterHub front end
    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # websocket headers
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header X-Scheme $scheme;
        proxy_buffering off;

    # Managing requests to verify letsencrypt host
    location ~ /.well-known {
        allow all;

Setup the systemd

edit /home/azat/.jupyter/jupyter.service

Description=Jupyter Lab
ExecStart=/home/azat/mambaforge/bin/jupyter lab
sudo cp ./jupyter.service /etc/systemd/system

sudo systemctl daemon-reload

sudo systemctl enable jupyter

sudo systemctl restart jupyter

sudo journalctl -u jupyter

Bug Fix

Jupyter Resources 404 .

CleanShot 2022-10-17 at 00.32.43

CleanShot 2022-10-17 at 00.33.34

Solution: Turn off Rocket Loader from Cloud Flare.