Real-time systems are critical for modern applications such as live dashboards, chat apps, IoT systems, and monitoring platforms. Traditional HTTP-based systems rely on request-response cycles, which are inefficient for continuous updates. This is where MQTT (Message Queuing Telemetry Transport) combined with Express.js provides a lightweight and scalable solution for building real-time update systems. Let us delve into understanding how to build real-time update systems with Mosquitto and Express.js
1. Introduction to MQTT
1.1 What is MQTT?
MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe messaging protocol designed for constrained devices and unreliable networks. It runs over TCP/IP and is optimized for scenarios where bandwidth, battery, and network reliability are limited. Originally created for oil pipeline monitoring, MQTT has become a standard protocol for IoT systems, real-time dashboards, event-driven architectures, and messaging between distributed services.
1.2 Why Use MQTT?
- Lightweight protocol: Minimal packet overhead compared to HTTP
- Efficient bandwidth usage: Ideal for mobile, IoT, and edge devices
- Real-time communication: Low-latency message delivery over persistent TCP connections
- Decoupled architecture: Producers and consumers do not need to know each other
- Bi-directional communication: Supports both upstream and downstream data flow
- Scalable: Handles thousands to millions of clients with proper broker setup
- Reliable delivery: Supports multiple Quality of Service (QoS) levels
1.2.1 MQTT Quality of Service (QoS)
MQTT provides three levels of delivery guarantees:
- QoS 0 (At most once): Message is delivered once with no guarantee
- QoS 1 (At least once): Message is guaranteed but may be duplicated
- QoS 2 (Exactly once): Message is delivered exactly once with higher overhead
1.2.2 MQTT Retained Messages & Last Will
- Retained Messages: Broker stores the last message on a topic and sends it to new subscribers immediately
- Last Will and Testament (LWT): Broker publishes a predefined message if a client disconnects unexpectedly
1.3 MQTT Architecture
MQTT follows a Publish-Subscribe Model where communication is mediated by a broker instead of direct client-to-client interaction.
- Broker: Central server responsible for receiving, filtering, and distributing messages
- Publisher: Sends messages to a topic without knowing who will receive them
- Subscriber: Subscribes to one or more topics to receive messages
- Topic: Hierarchical string (e.g.,
flights/delhi/arrivals) used to organize messages
1.3.1 MQTT vs HTTP vs WebSockets
- HTTP: Request-response model, not suitable for continuous updates
- WebSockets: Full-duplex but requires more complex connection handling
- MQTT: Lightweight, event-driven, and optimized for pub-sub communication
MQTT is particularly effective when systems require decoupled communication and efficient message distribution across multiple consumers. Think of MQTT as an event bus where producers emit events and consumers react to them asynchronously.
1.4 Setting Up an MQTT Broker
We can use Mosquitto, a popular open-source MQTT broker, for local development and testing.
# Install Mosquitto (Ubuntu/Debian) sudo apt update sudo apt install mosquitto mosquitto-clients # Start the broker sudo systemctl start mosquitto # Enable auto-start on boot sudo systemctl enable mosquitto # Check broker logs mosquitto -v # Open Terminal 1 - Subscribe mosquitto_sub -h localhost -t test/topic # Open Terminal 2 - Publish mosquitto_pub -h localhost -t test/topic -m "Hello MQTT"
The above commands demonstrate how to set up and test a local MQTT broker using Mosquitto on a Debian/Ubuntu system. First, the system packages are updated and Mosquitto along with its client utilities are installed. The broker is then started as a background service and enabled to automatically launch on system boot, ensuring availability without manual intervention. The mosquitto -v command runs the broker in verbose mode, allowing you to monitor logs and debug message flow. To validate the setup, one terminal acts as a subscriber using mosquitto_sub, listening to messages on a specific topic (test/topic), while another terminal acts as a publisher using mosquitto_pub, sending a test message (“Hello MQTT”) to the same topic. This verifies the core publish-subscribe mechanism, where the broker successfully routes messages from publishers to all active subscribers.
2. Complete Real-Time System with MQTT + Express.js + SSE
2.1 Server (Express + MQTT + SSE)
The following server implementation acts as the core of the system, connecting MQTT with Express and enabling real-time streaming using SSE. It handles message publishing, subscription, and broadcasting to connected clients.
// server.js
const express = require('express');
const mqtt = require('mqtt');
const bodyParser = require('body-parser');
const cors = require('cors');
const app = express();
app.use(cors());
app.use(bodyParser.json());
/* ================= MQTT SETUP ================= */
const client = mqtt.connect('mqtt://localhost:1883');
client.on('connect', () => {
console.log('Connected to MQTT Broker');
client.subscribe('matches/#'); // subscribe all match updates
});
/* ================= SSE CLIENTS ================= */
let clients = [];
app.get('/events', (req, res) => {
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
const clientId = Date.now();
const newClient = { id: clientId, res };
clients.push(newClient);
req.on('close', () => {
clients = clients.filter(c => c.id !== clientId);
});
});
/* Broadcast to all SSE clients */
function broadcast(data) {
clients.forEach(c => {
c.res.write(`data: ${JSON.stringify(data)}\n\n`);
});
}
/* MQTT → SSE bridge */
client.on('message', (topic, message) => {
const payload = message.toString();
console.log('MQTT:', topic, payload);
broadcast({ topic, payload });
});
/* ================= MATCH ROUTES ================= */
// Create/Update match
app.post('/match', (req, res) => {
const { matchId, score } = req.body;
const topic = `matches/${matchId}`;
client.publish(topic, JSON.stringify(score));
res.json({ status: 'Match updated' });
});
// Get match (mock)
app.get('/match/:id', (req, res) => {
res.json({ matchId: req.params.id, score: "Live data available via SSE" });
});
/* ================= STATIC FILES ================= */
app.use(express.static('public'));
app.listen(3000, () => console.log('Server running on 3000'));
2.1.1 Code Explanation
This Node.js server uses Express to create a real-time system by bridging MQTT messaging with browser clients via Server-Sent Events (SSE). It begins by importing required modules such as Express for HTTP handling, MQTT for broker communication, body-parser for parsing JSON requests, and CORS to allow cross-origin access. The MQTT client connects to a local broker and subscribes to all topics under matches/#, enabling it to receive updates for any match. For real-time streaming, the /events endpoint establishes an SSE connection by setting appropriate headers and storing each connected client’s response object in an in-memory array; when a client disconnects, it is removed to prevent memory leaks. A broadcast function iterates over all connected clients and pushes updates using the SSE protocol format. Whenever a message is received from MQTT, the server converts it to a string, logs it, and forwards it to all connected browsers via this broadcast mechanism—effectively acting as a bridge between MQTT and frontend clients. The /match POST route allows updating or creating a match by publishing the score to a topic like matches/{matchId}, which in turn triggers real-time updates to all subscribers, while a GET route returns a placeholder response, as real-time data is streamed via SSE. Finally, static files are served from a public directory, and the server listens on port 3000, making it a complete end-to-end system for publishing, consuming, and streaming live match data.
2.2 Admin Upload Interface
The admin interface allows users to push match updates into the system. It sends data to the backend API, which then publishes it to MQTT for real-time distribution.
<!-- public/admin.html -->
<!DOCTYPE html>
<html>
<body>
<h2>Admin Panel</h2>
<input id="matchId" placeholder="Match ID"/>
<input id="score" placeholder="Score"/>
<button onclick="send()">Update</button>
<script>
async function send() {
const matchId = document.getElementById('matchId').value;
const score = document.getElementById('score').value;
await fetch('/match', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ matchId, score })
});
alert('Updated');
}
</script>
</body>
</html>
2.2.1 Code Explanation
This HTML file represents a simple admin interface that allows users to send live match updates to the backend server. It consists of two input fields for entering the match ID and the score, along with a button that triggers the send() JavaScript function when clicked. Inside this function, the values entered by the user are retrieved using document.getElementById, and an asynchronous HTTP POST request is made to the /match endpoint using the Fetch API. The request includes a JSON payload containing the match ID and score, with the appropriate Content-Type header set to application/json. Once the request is successfully sent, an alert notifies the user that the update has been processed. This action ultimately publishes the data to the MQTT broker via the backend, triggering real-time updates to all connected clients through the SSE stream.
2.3 Live Viewer Interface (SSE)
The viewer interface listens for real-time updates from the server using Server-Sent Events. It continuously receives and renders match updates without requiring page refreshes.
<!-- public/viewer.html -->
<!DOCTYPE html>
<html>
<body>
<h2>Live Match Updates</h2>
<div id="output"></div>
<script>
const eventSource = new EventSource('/events');
eventSource.onmessage = function(event) {
const data = JSON.parse(event.data);
const div = document.getElementById('output');
div.innerHTML += "<p>" + data.topic + " : " + data.payload + "</p>";
};
</script>
</body>
</html>
2.3.1 Code Explanation
This HTML file acts as a live viewer interface that listens for real-time match updates from the server using Server-Sent Events (SSE). It establishes a persistent connection to the backend by creating a new EventSource instance pointing to the /events endpoint, which continuously streams updates without requiring repeated requests. Whenever the server pushes a new message, the onmessage handler is triggered, where the incoming event data is parsed from JSON format to extract the topic and payload sent via MQTT. The script then dynamically updates the DOM by selecting the output div and appending a new paragraph element containing the topic (e.g., match identifier) and its corresponding payload (e.g., score). This enables users to see live match updates instantly in the browser as they are published, without needing to refresh the page, effectively demonstrating a seamless real-time data flow from MQTT through the server to the frontend.
2.4 Home Page
The home page acts as a simple navigation layer, allowing users to access both the admin panel and the live viewer interface from a single entry point.
<!-- public/index.html --> <!DOCTYPE html> <html> <body> <h1>Real-Time Match System</h1> <ul> <li><a href="/admin.html">Admin Panel</a></li> <li><a href="/viewer.html">Live Viewer</a></li> </ul> </body> </html>
2.4.1 Code Explanation
This HTML file serves as the entry point or homepage of the real-time match system, providing simple navigation to the core functionalities of the application. It displays a title indicating the purpose of the system and includes an unordered list with two links: one directing users to the admin panel (/admin.html) where match data can be created or updated, and the other leading to the live viewer (/viewer.html) where real-time updates can be observed. By leveraging Express static file serving on the backend, these links directly load the corresponding HTML files from the public directory. Although minimal in structure, this page acts as a central hub that connects the user to both the data publishing interface and the real-time consumption interface, completing the overall flow of the system.
2.5 Code Output
When the complete system is run, it demonstrates a full real-time data flow from the admin interface to the viewer interface through MQTT and SSE. After starting the Node.js server and MQTT broker, opening the homepage (index.html) provides navigation to both the Admin Panel and Live Viewer. In the Admin Panel, entering a match ID and score and clicking “Update” sends a POST request to the backend, which publishes the data to the corresponding MQTT topic. The server, already subscribed to matches/#, receives this message and immediately broadcasts it to all connected SSE clients. On the Live Viewer page, the browser maintains an open connection to the server via EventSource, and as soon as the message is broadcast, it is received and rendered dynamically in the UI without any page refresh. This results in an instant update such as matches/123 : {"score":"120/3"} appearing on the screen. The system effectively showcases a seamless real-time pipeline where data flows from input to display with minimal latency, validating the integration of MQTT for messaging and SSE for frontend streaming.
3. Conclusion
Building real-time systems with MQTT and Express.js provides a powerful yet lightweight solution for modern applications. MQTT’s publish-subscribe model decouples systems and ensures efficient message delivery, while Express.js acts as a bridge between traditional HTTP APIs and real-time messaging. This architecture is especially useful in IoT, monitoring dashboards, notifications, and collaborative applications where instant updates are critical. By combining MQTT’s efficiency with Node.js flexibility, you can build scalable, real-time systems with minimal overhead.
