Skip to content

Mesos

September 29, 2023
March 21, 2016

Apache Mesos GitHub
Apache Mesos - Wikiwand

Apache Mesos is inspired by the Google's paper of their Borg system. It abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively. Twitter have been using it since 2007.
Mesos provides the low level API. It usually requires a "framework" (scheduler) to tell it what to do and on which node to execute the task.

It uses Zookeeper underneath for leader election. Non-responsive node will be removed, replace by new node; if the old node response later on the response will be disposed and it will be killed to prevent duplication.

Mesos is best run on bare metal or on VM.

A tale of two clusters: Mesos and YARN - O'Reilly Media

Mesos is low-level infrastructure; it needs frameworks like Aurora to give it something to do. As David Messina, VP of marketing for Docker, put it in an email, "Aurora is intended for humans to interact with and perform high-level tasks like 'run 100 instances of my service', while Mesos is a toolkit that programmers interact with to build other systems."

Video

Building Distributed Frameworks on Mesos - YouTube
How Airbnb Simplifies with Mesos - YouTube
Running Mesos at Atlassian - YouTube
Simplifying with Mesos and Marathon - YouTube
Operating Apache Aurora and Mesos at Twitter - YouTube
Run Any App on Mesos on Any Infrastructure Using Docker - YouTube
Running YARN Alongside Mesos - YouTube

Mini-Mesos

minimesos.org - Testing infrastructure for Mesos frameworks
Mini-Mesos: What’s a Nice XPer Doing in a Company Like This? - The New Stack

Mesosphere

Introducing The Mesosphere Datacenter Operating System GitHub
Blog - Mesosphere

Mesosphere OS manages entire data centers with one click | InfoWorld
Taking Mesos to the next level with DC/OS

Marathon

Mesosphere Becomes D2IQ, Moves Into Kubernetes, Big Data | Data Center Knowledge

Marathon: A cluster-wide init and control system for services in cgroups or Docker containers
mesosphere/marathon: Deploy and manage containers (including Docker) on top of Apache Mesos at scale.
Marathon is used for long running services.

Mesosphere Marathon for production container environments

Chronos

Chronos: Fault tolerant job scheduler for Mesos
mesos/chronos: Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules
Chronos is used to execute services at particular time, like cron, for batch processing.

Aurora

Apache Aurora GitHub
Aurora is a Mesos framework for long-running services and batch jobs. It is build by Twitter and the creator Bill Farner joined Docker Inc.

Introduction to Apache Aurora - YouTube
mesos - Marathon vs Aurora and their purposes - Stack Overflow