Skip to content

Home

  1. IDOML server This repository contains the configuration files to deploy the IDOML server. The IDOML server is a docker-compose based deployment of the following services

  2. IDOML jupyterhub After deploying the IDOML server, the subsequent step involves configuring the JupyterHub server. JupyterHub is a multi-user server that grants users access to Jupyter notebooks. This server comes pre-configured with extensions and libraries to streamline machine learning tasks and pipeline deployment into the Airflow server.

    To set up the JupyterHub server, refer to the instructions provided in the idoml jupyterhub repository.

  3. IDOML worker The Airflow worker node functions as a server dedicated to executing tasks outlined in the Airflow Directed Acyclic Graphs (DAGs). This node's primary responsibility involves executing machine learning tasks and deploying pipelines.

In the IDOML server, there is already a default worker provided. However, it's recommended to scale up the number of workers based on the number of tasks to be executed. This ensures efficient task execution and improves overall system performance.

To configure the Airflow worker node, please consult the instructions available in the idoml worker node repository.