Introduction

The Internet of Things (IoT) has revolutionized data transfer in industries through the MQTT protocol. MQTT is a lightweight messaging protocol designed for efficient communication among IoT devices. It operates on a publish-subscribe model where data producers (publishers) send messages to specific topics. Interested devices (subscribers) subscribe to these topics through an MQTT broker, facilitating communication. MQTT offers Quality of Service (QoS) levels, ensuring reliable message delivery, making it ideal for IoT with intermittent connectivity. It is valued for low overhead, scalability, and real-time communication. Traditional MQTT brokers are single-node setups, straightforward for small to moderate IoT applications. In contrast, distributed MQTT brokers are interconnected, offering scalability, redundancy, load balancing, and fault tolerance. They suit large-scale IoT deployments where reliability and performance are vital. The proposed Load-Balancer Redis-based architecture for distributed MQTT brokers overcomes the shortcomings of traditional brokers. The Load-Balancer transfers all incoming TCP packets to the brokers for processing, which ensures equal utilization of each broker node. Redis acts as a synchronization layer for the brokers where the state information is stored.

Proposed Architectural Pattern for Distributed MQTT Broker

System Architecture

Key Components

  1. Load Balancer:
    • Distributes incoming packets/messages to various broker nodes using a round-robin algorithm.
    • Ensures no single broker becomes a point of failure by evenly distributing the load.
  2. Redis:
    • Acts as a global shared memory to store client state information and subscription tables.
    • Updated only when a client connects, disconnects, or subscribes to a particular topic.
    • Stores inflight messages with a Time-To-Live (TTL) ensuring message delivery to intended clients via any broker.
  3. Gossip-based Protocol:
    • Utilized for discovering new nodes in the system, ensuring dynamic membership changes and fault tolerance.
    • Nodes share information at regular intervals to maintain system consistency over time.
    • Implemented using Hashicorp Memberlist Library for cluster management and failure detection.

Architecture Overview

  • Clients can connect to any broker in the cluster and consume subscribed topics.
  • Brokers fetch updated client state and subscription information from Redis for message processing.
  • Each publish message is broadcast to all other brokers through Redis streams, ensuring message delivery to subscribers and avoiding loops.

Benefits

  • Scalability: Enhanced client connectivity and load balancing improve system scalability.
  • Fault Tolerance: Distributed components and load balancing ensure system robustness and fault tolerance.
  • Reliability: Consistent client state and shared memory mechanisms contribute to a more reliable system.

Implementation Details

Client Distribution

  • Clients can connect to any broker, maintaining persistent client state across the network.

Subscription Distribution

  • A global store of subscription state information (centralized or distributed) ensures efficient data delivery to subscribers.

Message Distribution

  • Load balancer distributes messages evenly across brokers, preventing any single point of failure.