Home

Main components of the Disco core

As of Disco 0.3, migration to the architecture below is still work in progress. Especially master and event server still need refactoring.

• Master disco_server.erl:
  • controls available resources (nodes)
  • maintains the global state of running workers
  • handles fault-tolerance

• Scheduler fair_scheduler*.erl
  • controls priority of all jobs and tasks
  • controls data locality and load balancing

• Event (log) server event_server.erl
  • controls global events and statistics
  • controls logs for archived jobs

• Job coordinator job_coordinator.erl
  • controls that all tasks belonging to a job are executed
  • handles events related to a job

• Worker supervisor disco_worker.erl
  • controls a single running task

• Web API web_server.erl, disco_web.erl, mochiweb/*.erl
  • Maps requests from the Web API to internal requests

• DDFS ddfs/*.erl
  • Storage subsystem