Skip to content

Message Queue

October 19, 2023
August 3, 2015

Message broker - Wikiwand
Choosing Your Messaging Protocol: AMQP, MQTT, or STOMP - VMware vFabric Blog - VMware Blogs
An overview of realtime protocols
消息中间件选型时,要关注哪些问题?
漫谈消息队列:以 Kafka 和 RocketMQ 为例
Kafka vs Pulsar vs RabbitMQ - Performance, Features, and Architecture Compared
Comparing Apache Kafka and Apache Pulsar
Comparing Apache Kafka, ActiveMQ, and RabbitMQ
Pulsar vs Kafka - Comparison and Myths Explored - Kai Waehner
Messaging & Multiplexing - 250bpm
Messaging System — Hands On! – Hacker Noon
How Message Queues Work in Distributed Systems #microservices
消息中间件部署及比较:rabbitMQ、activeMQ、zeroMQ、rocketMQ、Kafka、redis - 掘金

Queues Job queues, message queues and other queues. Almost all of them in one place.
Grow Up. Use Queues.
Bus or Queue | Blog
Publish-Subscribe Pattern vs Message Queues vs Request Response (Detailed Discussions with Examples) - YouTube
System Design Interview - Distributed Message Queue - YouTube
What is a Message Queue and When should you use Messaging Queue Systems Like RabbitMQ and Kafka - YouTube

goldsborough/ipc-bench: Benchmarks for Inter-Process-Communication Techniques

Microservices and Message Queues, Part 1: Understanding Message Queues - CloudAMQP
Microservices and Message Queues, Part 2: Message Queues use cases - CloudAMQP
Microservices and Message Queues, Part 3: RabbitMQ and CloudAMQP - CloudAMQP
Microservices and Message Queues, Part 4: Introducing the demo project - CloudAMQP
Microservices and Message Queues, Part 5: Building the first service - CloudAMQP
Microservices and Message Queues, Part 6: Building the second service - CloudAMQP
Microservices and Message Queues, Part 7: Caveats for real-world use cases - CloudAMQP

Event-driven Architecture

design-patterns#Event-Driven Architectures

AMQP

AMQP is the Internet Protocol for Business Messaging | AMQP
Advanced Message Queuing Protocol - Wikiwand
Messaging, AMQP and RabbitMQ - A Speed Guide - Compose Articles
Tech Spotlight: Advanced Message Queuing Protoc... | element14 | Tech Connection disable JavaScript

AMQP Protocol Downloads | AMQP
RabbitMQ - Protocol Documentation

Speaking Rabbit: A look into AMQP’s frame structure
Working with AMQP | Packt Hub

vs JMS

Understanding AMQP
java - JMS and AMQP - RabbitMQ - Stack Overflow
Using JMS Transport as the Connection Protocol
Understanding the Differences between AMQP & JMS - Mark Richards PDF

RabbitMQ

Gateways for AMQP, HTTP, STOMP, and MQTT protocols

RabbitMQ - Messaging that just works
RabbitMQ - Wikiwand
RabbitMQ Tutorials — RabbitMQ

rabbitmq/rabbitmq-server: Open source multi-protocol messaging broker
rabbitmq/internals: High level architecture overview

RabbitMQ - Getting started with RabbitMQ
RabbitMQ Is Boring, and I Love It - The New Stack

RabbitMQ in 5 Minutes - YouTube
What is RabbitMQ? - YouTube
RabbitMQ Crash Course - YouTube

RabbitMQ in Microservices - DZone Microservices
2 Hours RabbitMQ Course with NodeJS, Pros & Cons, Cloud RMQ, RMQ vs Kafka, RMQ in Wireshark & MORE! - YouTube

Configuring RabbitMQ Exchanges, Queues and Bindings: Part 1 - Compose Articles
Configuring RabbitMQ Exchanges, Queues and Bindings: Part 2 - Compose Articles

Blog | Part 1: RabbitMQ for beginners - What is RabbitMQ? - CloudAMQP, RabbitMQ as a Service
Blog | Part 2.2: RabbitMQ for beginners - Getting started with Node.js - CloudAMQP, RabbitMQ as a Service
Blog | Part 3: RabbitMQ for beginners - The management interface - CloudAMQP, RabbitMQ as a Service
Blog | Part 4: RabbitMQ for beginners - Exchanges, routing keys and bindings - CloudAMQP, RabbitMQ as a Service
Blog | RabbitMQ for developers by Derick Bailey - CloudAMQP, RabbitMQ as a Service

The Changelog #258: 10 Years of RabbitMQ with Karl Nilsson & Michael Klishin | Changelog

Ecosystem

Clients Libraries and Developer Tools — RabbitMQ

Protocol Extensions — RabbitMQ

Plugin Development Basics — RabbitMQ
Plugins — RabbitMQ

Clients

rabbitmq/amqp091-go: An AMQP 0-9-1 Go client maintained by the RabbitMQ team.
pika/pika: Pure Python RabbitMQ/AMQP 0-9-1 client library

LavinMQ

The Next-Generation Message Broker - LavinMQ
cloudamqp/lavinmq: Lightweight and fast AMQP (0-9-1) server

Guide: LavinMQ for beginners. What is LavinMQ? - CloudAMQP

ActiveMQ

Apache ActiveMQ ™
Apache ActiveMQ - Wikiwand
ActiveMQ in Action Introduction

Supports AMQP, REST, STOMP, MQTT, OpenWire, XMPP and WebSockets.

ActiveMQ Command Line Tools
ActiveMQ Unix Shell Script

Scaling SpringBoot with Message Queues and Kubernetes ♦︎ Learnk8s

ZeroMQ

not actually a message queue, but the building blocks of one

ZeroMQ
ZeroMQ - Wikiwand
The ZeroMQ project GitHub Org
Pieter Hintjens talks about ZeroMQ and open source at CERN, 27 June 2013 - YouTube history and design
Why should I have written ZeroMQ in C, not C++ part I part II

Sockets are created from context. The context internally acts as a storage mechanism for global state and, to the user, as a pool of I/O threads.
Its sockets are not thread-safe!!

ZeroMQ channel (or it's just the client?) out of the box cannot handle sending concurrent messages
A client-side queue is needed to serialize the send calls

Node.js: "Socket is busy writing; only one send operation may be in progress at any time"
Python: "ZMQError('Operation cannot be accomplished in current state')>"

ZeroMQ | Get started
ØMQ - The Guide - ØMQ - The Guide
23/ZMTP | ZeroMQ RFC
28/REQREP | ZeroMQ RFC
29/PUBSUB | ZeroMQ RFC
30/PIPELINE | ZeroMQ RFC
39/ZWS | ZeroMQ RFC
37/ZMTP | ZeroMQ RFC
7/MDP | ZeroMQ RFC Majordomo Protocol

Building Distributed Systems with Node.js and ØMQ - YouTube 2014
ZeroMQ is the answer - YouTube 2011
ZeroMQ@PDX: 07 - Pieter Hintjens - ZeroMQ Basics (and AMQP ranting) on Vimeo 2012
ZeroMQ@PDX: 02 - Pieter Hintjens - How to make money from ZeroMQ on Vimeo 2012

ZeroMQ (ØMQ) Crash Course - YouTube
pnumerics
ZeroMQ Ultimate Crash Course with C++ Examples | ØMQ | ZMQ - YouTube
ZeroMQ Crash Course - YouTube

ZeroMQ: Modern & Fast Networking Stack - igvita.com
Design of PUB/SUB subsystem in ØMQ - 250bpm
0MQExperiences

Distributed Messaging with ZeroMQ – Brave New Geek
Share data between C and Python with this messaging library | Opensource.com

ZeroMQ | C#
zeromq/netmq: A 100% native C# implementation of ZeroMQ for .NET
Home - NetMQ

ZeroMQ | NodeJS
zeromq.js
zeromq/zeromq.js: Node.js bindings to the ØMQ library
Error: Socket is busy writing · Issue #381 · zeromq/zeromq.js use p-queue to serialize sending of messages

ZeroMQ | Python
PyZMQ Documentation
zeromq/pyzmq: PyZMQ: Python bindings for zeromq
Learning ØMQ with pyzmq — Learning 0MQ with examples

rjoydip/node-py-rpc: Node.js and python communication using Remote Procedure Call (RPC)

zerorpc
zerorpc
0rpc/zerorpc-node: zerorpc for node.js
0rpc/zerorpc-python

Multipart messages

Multipart is used to send binary, and message with topic (PUB/SUB)

2. Sockets and Patterns | ØMQ - The Guide
python - How can I use send_json with pyzmq PUB SUB - Stack Overflow

zmq_send (worker, address, address_size, ZMQ_SNDMORE);
zmq_send (worker, empty, 0, ZMQ_SNDMORE);
zmq_send (worker, "OK", 2, 0);

Nanomsg

An IPC/RPC API that uses the Scalability Protocols. nanomsg and NNG should inter-operate (modulo a few caveats).
Successor of ZeroMQ.

About Nanomsg

nanomsg aims to provide a POSIX (BSD socket) API. It uses state-machines to archive thread-safeness.
nanomsg/nanomsg: nanomsg library
Getting Started with 'nanomsg'

Differences between nanomsg and ZeroMQ from Martin Sustrik, the author of both libraries
Optimising Subscriptions in nanomsg - 250bpm
Getting Rid of ZeroMQ-style Contexts - 250bpm
The Callback Hell - 250bpm
nanomsg: Towards Full-Blown Configuration Management - 250bpm

nickdesaulniers/node-nanomsg: Node.js binding for nanomsg

FLOSS Weekly 469 nanomsg

Scalability Protocols

About Nanomsg
Scalability Protocols
A Look at Nanomsg and Scalability Protocols (Why ZeroMQ Shouldn’t Be Your First Choice) – Brave New Geek

Bindings

nanocat(1)
https://nng.nanomsg.org/man/v1.3.2/nngcat.1.html

nanomsg-NG/nng

NNG - nanomsg-NG next generation of nanomsg
ditched POSIX API, allows callback rather than forcing one to poll file descriptor
NNG Reference Manual, 2nd… by Garrett D'Amore [PDF/iPad/Kindle]

Rational: Or why am I bothering to rewrite nanomsg?
Getting Started with 'nng'
https://nng.nanomsg.org/man/v1.3.2/index.html

jeikabu/nng.NETCore

This is Pynng’s Documentation
codypiersall/pynng: Python bindings for Nanomsg Next Generation.

nanomsg/mangos: mangos is a pure Golang implementation of nanomsg's "Scalablilty Protocols"
mangos package - go.nanomsg.org/mangos/v3 - pkg.go.dev
Message Queues, or how you can make processes talk to each other · Applied Go

Postgres as queue

Choose Postgres queue technology :: Adriano Caloiaro's personal blog

NATS

NATS - Open Source Messaging System | Secure, Native Cloud Application Development
NATS Documentation

Liftbridge

LIFTBRIDGE · Lightweight, fault-tolerant message streams
Introducing Liftbridge: Lightweight, Fault-Tolerant Message Streams – Brave New Geek

AMQP

Home | AMQP
Advanced Message Queuing Protocol - Wikiwand

Designed for Performance & Reliability - OpenAMQ obsolete

COAP

CoAP — Constrained Application Protocol | Overview
Constrained Application Protocol - Wikiwand
HTTP vs CoAP · Issue #10 · mcollina/node-coap
Tech Spotlight: The CoAP Protocol | element14 | Tech Connection disable JavaScript

mcollina/node-coap binary http over udp

MQTT

MQTT is a pub/sub broker rather than a message queue

MQTT
MQTT - Wikiwand

MQTT Specification
Differences between 3.1.1 and 5.0 · mqtt/mqtt.org Wiki

Serving Millions of Clients with Native MQTT | RabbitMQ - Blog in 3.12, 2023-03

What is MQTT and How It Works | Random Nerd Tutorials

MQTT with a Raspberry Pi and an Arduino - YouTube

MQTT 5

Getting started with Node.js and MQTT - LogRocket Blog
MQTT.js
mqttjs/MQTT.js: The MQTT client for Node.js and the browser

Brokers

MQTT Dashboard

Eclipse Mosquitto
Install Mosquitto Broker Raspberry Pi | Random Nerd Tutorials
mcollina/aedes: Barebone MQTT broker that can run on any stream server, the node way

HiveMQ - Enterprise ready MQTT to move your IoT data

XMPP

XMPP - Wikiwand
XMPP | XMPP Main
Tech Spotlight: The XMPP Protocol | element14 | Tech Connection disable JavaScript

STOMP

STOMP
Implementations
jasonrbriggs/stomp.py: “stomp.py” is a Python client library for accessing messaging servers (such as Apollo or RabbitMQ) using the STOMP protocol (versions 1.0, 1.1 and 1.2). It can also be run as a standalone, command-line client for testing.

Kafka

runs on JVM, requires ZooKeeper

Apache Kafka in 6 minutes - YouTube
Apache Kafka Explained (Comprehensive Overview) - YouTube
Apache Kafka Crash Course - YouTube
System Design: Why is Kafka fast? - YouTube

Introduction to Event-Driven Architectures with Apache Kafka on Heroku - Salesforce Live
HiveMQ Goes Open Source, Brings MQTT and Kafka Together

Redpanda

BSL, JVM-free, ZooKeeper-free, Kafka compatible, single binary

Redpanda | The streaming data platform for developers

Redpanda: Kafka Alternative with Alexander Gallego - Software Engineering Daily

Gravitee

event-native API Management solution that supports synchronous REST APIs, asynchronous APIs, and event brokers like Kafka and MQTT

Gravitee.io - API Management Platform

PubSub

Best pub/sub messaging brokers - LogRocket Blog

Faye: Simple pub-sub messaging for the web
faye Org

Building A Pub/Sub Service In-House Using Node.js And Redis — Smashing Magazine

Welcome to hpfeeds — hpfeeds documentation
hpfeeds/hpfeeds: Honeynet Project generic authenticated datafeed protocol

Outbox pattern

Microservices & Data - Implementing the Outbox Pattern with Hibernate

Others

Ponte - Bringing Things to REST developers Ponte allows you to publish and receive the data using any protocol: HTTP, MQTT and CoAP.
mcollina/ascoltatori: The pub/sub library for node backed by Redis, MongoDB, AMQP (RabbitMQ), ZeroMQ, MQTT (Mosquitto) or just plain node!

tj/axon: message-oriented socket library for node.js heavily inspired by zeromq
tj/punt: Elegant UDP messaging for nodejs

OpenHFT/Chronicle-Queue: Micro second messaging that stores everything to disk