Every backend team building a distributed system eventually hits the same fork in the road: does this workload need a queue, or does it need a log? Amazon SQS and Apache Kafka both move messages between services, but they were built to solve different problems, and picking the wrong one shows up months later as a rewrite. SQS turns 20 years old in 2026 as AWS’s oldest managed service still in active development, while Kafka just shipped version 4.3.1 and finished its multi-year divorce from ZooKeeper. This comparison walks through the pricing, the throughput numbers, the code, and the migration path so the decision takes an afternoon instead of a quarter.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
The Quick Answer: SQS vs Kafka at a Glance
Amazon SQS is a fully managed message queue built for decoupling application components with zero infrastructure to run. You create a queue, send messages, and AWS handles the rest, including scaling, durability, and availability across zones. Apache Kafka is a distributed event streaming platform built for high-throughput, replayable logs that many consumers can read independently. It demands more operational investment, whether self-hosted or run through Amazon MSK, but it does things SQS fundamentally cannot, like replaying a week of events to rebuild a downstream service’s state.
In short: reach for SQS when you need simple, reliable task decoupling and don’t want to manage a cluster. Reach for Kafka when you need multiple independent consumers reading the same event stream, long-term replay, or throughput north of a few thousand messages per second sustained. The rest of this guide breaks down exactly where that line sits, with pricing, benchmark data, and code for both.
| Category | Amazon SQS | Apache Kafka |
|---|---|---|
| Model | Managed point-to-point / fan-out queue | Distributed, partitioned commit log |
| Ops burden | Zero — fully managed by AWS | Self-managed, or managed via Amazon MSK / Confluent Cloud |
| Best for | Task decoupling, background jobs, simple fan-out | Event streaming, replay, multi-consumer pipelines |
| Learning curve | Low — a queue is a queue | Steep — partitions, consumer groups, offsets |
| Starting cost | $0, pay only per request | $0.63/hour minimum for a 3-broker MSK cluster (~$460/month before storage) |
What Is Amazon SQS?
Amazon Simple Queue Service is AWS’s managed message queuing service, and one of the oldest pieces of AWS still standing largely unchanged in its core model. It offers two queue types. Standard queues give nearly unlimited throughput with at-least-once delivery and best-effort ordering, meaning messages can arrive out of sequence or, occasionally, twice. FIFO queues guarantee strict ordering and exactly-once processing within a message group, at a throughput ceiling that’s lower unless you enable high throughput mode.
There’s no cluster to size, no broker to patch, and no partition count to plan around. You create a queue through the console, the CLI, or infrastructure-as-code, and it’s ready. Consumers poll the queue with short or long polling, process a message, and explicitly delete it once done. If a consumer crashes mid-processing, the message becomes visible again after a configurable visibility timeout so another consumer can pick it up. That single mechanism is most of what makes SQS resilient, and it’s also why it isn’t well suited to workloads that need multiple independent readers of the same event.
SQS’s biggest architectural change in years landed quietly: AWS raised the maximum message payload from 256 KiB to 1 MiB in 2025, a four-fold jump that removes one of the most common reasons teams reached for S3-backed message references. AWS also shipped Fair Queues in July 2025, a mode that adds a small per-request surcharge in exchange for preventing one noisy message group from starving others sharing a standard queue — a problem that previously only FIFO queues addressed directly.
What Is Apache Kafka?
Apache Kafka is an open-source distributed event streaming platform originally built at LinkedIn and now maintained by the Apache Software Foundation, with commercial distributions from Confluent, AWS (via Amazon MSK), and others. Where SQS is a queue, Kafka is a log. Producers append records to topics, which are split into partitions for parallelism, and those records stick around for a configurable retention window rather than disappearing the moment they’re read. Multiple consumer groups can each read the same topic independently, at their own pace, without affecting one another — the core capability that a queue-based system doesn’t give you.
Kafka’s biggest structural change in its history finished in 2025. Apache Kafka 4.0, released March 18, 2025, removed the ZooKeeper dependency entirely and made KRaft (Kafka Raft) the only supported metadata mode, ending a multi-year migration that started with KIP-500. There’s no fallback to ZooKeeper in the 4.x line, which simplifies deployment considerably and, per Kafka’s own benchmarking notes, speeds up metadata operations by 30 to 40 percent. As of this writing, Kafka 4.3.1 is the current stable release, published June 25, 2026.
Running Kafka yourself means owning broker sizing, partition planning, replication factors, and JVM tuning. Most teams on AWS sidestep that with Amazon MSK, which runs Kafka as a managed service, or with Confluent Cloud, which layers additional tooling like schema registry and stream processing on top. Either way, Kafka asks for more upfront design work than SQS. What it gives back is replay, fan-out to many consumers, and throughput ceilings queues don’t approach.
Amazon SQS vs Apache Kafka: Full Specification Comparison
The table below lines up the two systems across the specs that actually change an architecture decision: message limits, ordering guarantees, retention, and operational model. Kafka figures reflect a self-managed or MSK-provisioned cluster running the current 4.3 line.
| Spec | Amazon SQS | Apache Kafka 4.3 |
|---|---|---|
| Max message size | 1 MiB (raised from 256 KiB in 2025) | ~1 MB by default (message.max.bytes ≈ 1,000,012 bytes), raisable per broker |
| Message retention | 4 days default, 14 days maximum | 7 days default (log.retention.hours = 168), configurable to indefinite |
| Ordering guarantee | Best-effort (Standard) or strict within a group (FIFO) | Strict within a partition |
| Delivery semantics | At-least-once (Standard), exactly-once (FIFO) | At-least-once by default; exactly-once with idempotent producers/transactions |
| Multiple independent consumers | Not natively — one message, one successful consumer | Native via consumer groups, each tracking its own offset |
| Replay past messages | Not supported once deleted or expired | Native — reset any consumer group’s offset |
| Infrastructure to manage | None | Brokers, partitions, replication (or MSK/Confluent Cloud managed) |
| Metadata layer | N/A (AWS-managed) | KRaft (ZooKeeper fully removed as of Kafka 4.0) |
| Protocol | AWS API (HTTPS/JSON) | Kafka binary protocol (TCP) |
| Typical p50 latency | Low tens of milliseconds per API call | 5-15 ms with batched, durable writes |
| Minimum monthly cost | $0 | ~$460 for a minimal 3-broker MSK Provisioned cluster before storage |
Throughput and Latency: What the Benchmarks Actually Show
Throughput is where the two systems diverge hardest, and it’s worth separating SQS’s two queue types before comparing either to Kafka. AWS’s own documentation on FIFO throughput puts default FIFO capacity at 3,000 messages per second with client-side batching, or 300 messages per second without it, per API action. Enabling high throughput mode raises that ceiling substantially, up to 70,000 transactions per second per API action in supported regions, which can translate to roughly 700,000 messages per second when batched aggressively. Standard queues don’t publish a hard ceiling at all; AWS describes them as scaling automatically to handle “nearly unlimited” throughput, because there’s no per-message ordering constraint holding the system back.
Kafka’s numbers come from a different place: raw log-append performance rather than API request limits. Intel’s Kafka optimization and benchmarking guide, along with independent 2025 benchmarks run on standardized 4 vCPU / 8 GB NVMe hardware, put a well-tuned Kafka broker at roughly 500,000 to over 1 million messages per second with batching and compression enabled. That figure scales close to linearly by adding brokers and partitions, which is the whole point of Kafka’s partitioned design. Google Cloud’s own guidance on benchmarking managed Kafka services echoes the same pattern: throughput is a function of partition count and broker sizing more than any single hard limit.
Standard vs FIFO Queue Throughput
The practical takeaway for SQS users: if you need ordering, budget for FIFO’s lower default ceiling or plan to request high throughput mode ahead of a launch, since it isn’t automatic. If you don’t need strict ordering, Standard queues remove the ceiling entirely and the tradeoff becomes deduplication logic in your consumer rather than a queue-level limit.
Kafka Partition Scaling
Kafka’s throughput scales with partition count, but more partitions also means more open file handles, longer leader elections during failover, and slower consumer rebalances. Teams chasing the million-messages-per-second figure typically land on double-digit partition counts per topic with compression (usually lz4 or zstd) enabled on the producer side, not a single overtuned broker.
Message Ordering, Delivery Guarantees, and Consumer Groups
This is the section that decides more architectures than pricing does. SQS Standard queues make no ordering promise: messages can arrive out of sequence, and under normal operation a message might be delivered more than once, so consumers need to be idempotent regardless of queue type. SQS FIFO queues fix this within a message group, delivering messages in the exact order they were sent and guaranteeing exactly-once processing as long as producers don’t send duplicates within a 5-minute deduplication window.
Kafka’s guarantee is different in shape. Ordering is strict within a partition, not across a topic, so two records with the same key always land in the same partition and stay in order relative to each other. Records with different keys can land anywhere, and there’s no ordering guarantee across partitions. Exactly-once semantics exist in Kafka too, but they’re opt-in: idempotent producers avoid duplicate writes on retry, and transactions extend that guarantee across a read-process-write cycle spanning multiple topics.
The feature that has no real SQS equivalent is the consumer group. In Kafka, any number of independent applications can subscribe to the same topic, and each tracks its own offset into the log completely independently of the others. A fraud-detection service and an analytics pipeline can both read the same order-events topic at their own pace without either one affecting the other or “consuming” a message the other still needs. In SQS, once a message is deleted by the consumer that processed it, it’s gone; the standard way to fan a single event out to multiple downstream services is to pair SQS with SNS topics, which adds a second AWS service to the design rather than solving it natively.
Retention, Message Size, and Durability
SQS retention is short and hard-capped by design: four days by default, configurable up to a 14-day maximum, after which unprocessed messages are silently deleted. That’s a deliberate constraint that keeps a queue from becoming an accidental database. Kafka retention defaults to 168 hours (7 days) on the broker side but is fully configurable per topic, up to and including indefinite retention if disk budget allows. Plenty of production Kafka clusters keep 30, 90, or even 365 days of history on topics used for reprocessing or compliance, something that’s structurally impossible in SQS regardless of configuration.
Message size tells a similar story about design intent. SQS’s 1 MiB ceiling (up from 256 KiB before the 2025 change) is a hard wall enforced by the API — anything larger has to be chunked or stored in S3 with a reference passed through the queue, a pattern AWS documents as the “extended client library.” Kafka’s default message.max.bytes sits close to the same 1 MB mark, at roughly 1,000,012 bytes, but it’s a broker-level config, not an architectural ceiling, and plenty of teams raise it for use cases like shipping full document payloads or image thumbnails through a topic.
Durability is where both systems are closer than their reputations suggest. SQS replicates messages across multiple availability zones automatically, with no configuration required. Kafka durability depends on how replication factor and acks are configured. A topic with replication factor 3 and acks=all offers comparable durability guarantees to SQS, but that configuration is a choice an operator has to make correctly, not a default baked into the platform.
Pricing Breakdown: SQS, MSK, and Self-Hosted Kafka
SQS pricing is pure pay-as-you-go with no floor. AWS’s published SQS pricing charges Standard queues $0.40 per million requests for the first 100 billion requests a month, stepping down to $0.30 per million and then $0.24 per million at higher volume tiers. FIFO queues run $0.50 per million requests at the entry tier. The first million requests every month are free, permanently, not just for a 12-month new-account window. Billing is metered in 64 KB chunks per request, so a message near the new 1 MiB ceiling can count as up to 16 requests rather than one. The Fair Queues feature added in July 2025 tacks on a further $0.10 per million requests when messages use a MessageGroupId on a standard queue.
Kafka has no request-based pricing because there’s no single vendor metering it; costs depend entirely on how you run it. Amazon MSK Provisioned bills by broker-hour: a kafka.m5.large broker runs $0.21 an hour, and since MSK requires a minimum of three brokers for a usable cluster, that floor works out to roughly $0.63 an hour, or about $460 a month, before adding the $0.10 per GB-month storage charge. MSK Serverless flips the model to consumption-based pricing: $0.75 per cluster-hour plus $0.0015 per partition-hour, $0.10 per GB of data written, and $0.05 per GB read. Self-hosting Kafka on raw EC2 removes the MSK management fee but shifts the cost to instance-hours, EBS storage, and the engineering time to patch and monitor brokers yourself.
| Option | Pricing model | Approximate floor |
|---|---|---|
| SQS Standard | $0.40/M requests (first 100B/mo), free tier 1M/mo | $0/month |
| SQS FIFO | $0.50/M requests, free tier 1M/mo | $0/month |
| SQS FIFO with Fair Queues | Standard/FIFO rate + $0.10/M surcharge | $0/month |
| Amazon MSK Provisioned | $0.21/broker-hour (kafka.m5.large) + $0.10/GB-month storage | ~$460/month (3 brokers, before storage) |
| Amazon MSK Express brokers | $0.408/hour (express.m7g.large) + $0.01/GB-month ingest | ~$894/month (3 brokers, before storage) |
| Amazon MSK Serverless | $0.75/cluster-hour + $0.0015/partition-hour + $0.10/GB in + $0.05/GB out | ~$550/month at modest partition/data volume |
| Self-hosted Kafka (EC2) | EC2 instance-hours + EBS storage + engineering time | Varies; smallest viable cluster often $150-$300/month in raw infra |
Real-World Cost at Scale: Three Traffic Tiers
Pricing tables only tell part of the story until they’re run against actual traffic. Here’s how the two systems compare across three tiers a growing product might realistically pass through in a year.
Low volume — 100 messages/second sustained. That’s roughly 260 million requests a month. On SQS Standard, after the first million free requests, that lands around $103 in request charges (259M × $0.40/M), plus negligible data transfer inside a region. Running MSK for this workload is almost pure overhead: the minimum 3-broker cluster still costs its ~$460/month floor whether it’s handling 100 messages a second or 100,000. At this tier, SQS is the cheaper choice by a wide margin, and it isn’t close.
Medium volume — 1,000 messages/second sustained. That’s about 2.6 billion requests monthly on SQS, which crosses into the second pricing tier and lands around $884 a month in request costs alone. A right-sized 3-broker MSK Provisioned cluster on kafka.m5.large instances, handling this volume comfortably with room to spare, still runs close to its $460-$600/month floor including storage. This is roughly the crossover zone: MSK starts to look competitive with SQS on pure infrastructure cost, though SQS still requires zero operational time while MSK needs at least light monitoring and partition planning.
High volume — 10,000 messages/second sustained. That’s roughly 26 billion requests a month on SQS, deep into the lower-priced tiers, landing somewhere around $6,700-$7,800 a month depending on exact tier mix — the five-figure-adjacent bill that shows up in almost every “SQS got expensive” postmortem online. A Kafka cluster sized for this load, likely 6-9 brokers depending on replication and partition count, runs in the $1,200-$2,500/month range on MSK Provisioned. At this tier, Kafka’s fixed-capacity pricing model wins decisively over SQS’s linear per-request billing, which is exactly why high-throughput event pipelines rarely run on SQS in production.
Code in Practice: Producer and Consumer Patterns
The API shapes reflect the architectural difference directly. SQS’s SDK calls are simple send/receive/delete operations against a queue URL. Here’s a minimal producer and consumer using boto3:
import boto3
sqs = boto3.client("sqs", region_name="us-east-1")
queue_url = "https://sqs.us-east-1.amazonaws.com/123456789012/orders-queue"
# Producer
sqs.send_message(
QueueUrl=queue_url,
MessageBody='{"order_id": "A1029", "status": "created"}'
)
# Consumer
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20 # long polling
)
for message in response.get("Messages", []):
print(message["Body"])
sqs.delete_message(
QueueUrl=queue_url,
ReceiptHandle=message["ReceiptHandle"]
)Kafka’s client model looks similar on the surface but carries more configuration, because a producer and consumer are talking to a partitioned log rather than a single queue endpoint:
from kafka import KafkaProducer, KafkaConsumer
import json
# Producer
producer = KafkaProducer(
bootstrap_servers="broker1:9092,broker2:9092",
value_serializer=lambda v: json.dumps(v).encode("utf-8"),
acks="all"
)
producer.send("orders", key=b"A1029", value={"order_id": "A1029", "status": "created"})
producer.flush()
# Consumer (independent consumer group)
consumer = KafkaConsumer(
"orders",
bootstrap_servers="broker1:9092,broker2:9092",
group_id="fraud-detection-service",
auto_offset_reset="earliest",
enable_auto_commit=True
)
for record in consumer:
print(record.key, record.value)The consumer snippet is the tell. A second, totally separate service could subscribe to the same “orders” topic with group_id=”analytics-pipeline” and read every event from the beginning of retention, without touching or being touched by the fraud-detection consumer. That’s not an SQS pattern at all; it’s the reason teams reach for Kafka once more than one downstream system needs the same event.
Integration With AWS Lambda, ECS, and EKS
SQS integrates with AWS compute about as natively as it gets. Lambda supports SQS as a direct event source: attach a trigger, and Lambda polls the queue, batches messages, and scales concurrent executions up or down automatically based on queue depth, with failed batches routed to a dead-letter queue with no extra plumbing. On ECS and EKS, SQS consumers are typically just application code running in a task or pod, polling the queue directly through the SDK, since there’s no cluster-level integration needed beyond IAM permissions.
Kafka’s Lambda integration works too — Lambda supports Kafka (both self-managed and MSK) as an event source in the same way it does SQS, polling topics and invoking functions in batches. Where Kafka’s AWS integration gets more interesting is on EKS, where KEDA (Kubernetes Event-Driven Autoscaling) can scale consumer pods directly based on Kafka consumer group lag, a tighter feedback loop than queue-depth-based autoscaling because it accounts for partition-level backlog rather than a single aggregate queue length. Teams running Kafka on EKS commonly pair MSK or a self-managed Strimzi cluster with KEDA specifically for this reason.
Neither system requires VPC peering headaches if everything stays inside AWS, but Kafka’s binary protocol over TCP means self-hosted or MSK clusters need more deliberate security group and network ACL planning than SQS’s HTTPS API, which rides on the same public or private AWS endpoints as every other managed service.
Security Model: IAM Policies vs SASL and mTLS
Security is one of the more underrated differences between these two systems, mostly because SQS makes it nearly invisible. Access control runs entirely through standard AWS IAM policies: a Lambda function, an ECS task role, or an EC2 instance profile either has sqs:SendMessage and sqs:ReceiveMessage permissions on a given queue ARN or it doesn’t. Encryption at rest is a checkbox using AWS KMS, encryption in transit is the default over HTTPS, and there’s no certificate management, no separate authentication server, and no protocol-level configuration to get wrong. For a team already living inside AWS’s IAM model, SQS security is close to a non-issue.
Kafka’s security model demands more deliberate setup because the protocol itself isn’t AWS-native. Self-managed clusters and MSK both typically rely on SASL/SCRAM or mutual TLS (mTLS) for client authentication, with access control layered on top through Kafka’s own ACL system, or IAM-based authentication specifically on MSK, which lets AWS-native teams reuse existing IAM roles instead of managing a separate credential store. Encryption in transit needs TLS configured explicitly on both brokers and clients, and encryption at rest depends on how the underlying storage (EBS on MSK, or whatever disk self-hosted brokers use) is configured. None of this is difficult, but it’s several more decisions than SQS asks for, and misconfigured Kafka ACLs are a recurring source of production incidents severe enough that most platform teams treat Kafka security review as a standing checklist rather than a one-time setup step.
Monitoring and Operational Visibility
Because SQS has no servers to watch, monitoring collapses down to a handful of CloudWatch metrics: ApproximateNumberOfMessagesVisible for queue depth, ApproximateAgeOfOldestMessage for backlog staleness, and NumberOfMessagesSent/Received for throughput. Alerting on those three is usually enough to catch a stuck consumer or a traffic spike before it becomes an incident, and it requires no agent installation or cluster access, just CloudWatch, which is already collecting the data by default.
Kafka observability is a bigger surface because there’s more happening underneath: per-broker CPU and disk I/O, per-partition consumer lag, under-replicated partition counts, and controller election frequency all matter, and any one of them can silently degrade a cluster well before it becomes visible from the application side. Amazon MSK exposes these through CloudWatch and open-monitoring endpoints compatible with Prometheus, but teams running Kafka at any real scale typically add a dedicated tool, commonly Grafana dashboards fed by JMX exporters or Confluent’s own Control Center, since default metrics alone rarely surface a slow-building partition skew problem before it affects consumer throughput. Consumer lag specifically deserves its own alert, since it’s the single number that most directly answers “is this pipeline falling behind,” and it has no real SQS equivalent beyond queue depth, which conflates lag across every producer and consumer rather than isolating it per downstream service.
Vendor Lock-In and Portability
This is the argument that rarely shows up in a pricing spreadsheet but eventually matters to any team planning more than a year ahead. SQS is proprietary AWS infrastructure with no open-source equivalent and no supported way to run it outside AWS. Moving off SQS later means a full rewrite of the messaging layer, not a configuration change, because there’s no drop-in replacement that speaks its exact API and semantics anywhere else.
Kafka is the opposite case by design. It’s an Apache Software Foundation project that runs identically whether it’s self-hosted on bare metal, deployed on Kubernetes through an operator like Strimzi, rented from Amazon MSK, or bought as Confluent Cloud. A team that starts on MSK and later wants to move to self-hosted Kafka on EKS, or to a different cloud entirely, is largely doing infrastructure work, not application rewrite work, because the client protocol and topic model stay the same. That portability has a cost of its own, since it’s also why Kafka never gets as tightly integrated with any single cloud’s other services as SQS is with the rest of AWS, but for teams that weigh multi-cloud optionality heavily, it’s a real point in Kafka’s favor that no pricing comparison captures on its own.
5 Real-World Use Cases: Who Should Use What
Specs and pricing only matter in context. Here are five concrete scenarios that come up repeatedly in production systems, and which side of this comparison actually fits each one.
1. Background job processing for a web app. Resizing uploaded images, sending confirmation emails, generating PDF invoices — classic decoupling work where one producer hands off to one consumer type. SQS is the obvious fit: no cluster to run, Lambda integration is a checkbox, and the at-least-once/idempotent-consumer pattern is easy to implement for jobs that are safe to retry.
2. E-commerce order events feeding multiple downstream systems. An order-placed event needs to reach inventory, billing, fraud detection, and analytics simultaneously, each at its own pace, and analytics wants to reprocess the last 30 days after a bug fix. This is the textbook Kafka case: multiple consumer groups, replay, and a single source of truth for the event rather than four separate SQS queues fed by SNS fan-out.
3. IoT telemetry ingestion from thousands of devices. High-volume, continuous, and often needing both real-time alerting and long-term storage for later analysis. Kafka’s partition model handles the sustained throughput, and its retention lets a time-series pipeline and a real-time alerting service both read the same stream independently. SQS would work for alerting alone, but not for the combined real-time-plus-historical requirement without bolting on a second system.
4. Microservices task queue for a mid-size SaaS product. A handful of services need to hand off discrete units of work, like “send this webhook” or “process this payment,” without needing replay or multiple consumers per task. This is squarely SQS territory, and it’s also the profile where a small team without dedicated infrastructure staff benefits most from paying AWS to own the operational burden entirely.
5. Clickstream and user activity analytics. Capturing every page view, click, and session event for both real-time personalization and batch analytics pipelines. This is close to the use case Kafka was originally built for at LinkedIn, and it remains one of the clearest wins for Kafka’s model: one topic, ingested once, read by a real-time recommendation service and a nightly batch job without either interfering with the other’s read position.
Migration Guide: Moving Between SQS and Kafka
Migrations in this space usually run one direction: teams that started with SQS because it was simple, and later need Kafka’s fan-out or replay. The reverse (Kafka to SQS) is rarer and usually means a team over-provisioned Kafka for a workload that never needed it. Here’s the practical path for an SQS-to-Kafka migration.
Step 1: Stand up the cluster without cutting over traffic. Provision Amazon MSK (or a self-managed cluster) in parallel with the existing SQS setup. Don’t touch production traffic yet — this step is purely about having a target environment to test against.
Step 2: Map queues to topics and design partition keys. Each SQS queue typically becomes one Kafka topic, but the partition key decision (usually an entity ID like order_id or user_id) determines ordering guarantees going forward, so this step needs real design thought, not a mechanical rename.
Step 3: Dual-write from producers. Update producers to write to both SQS and the new Kafka topic simultaneously. This is the safest way to validate the new pipeline against real production data without any risk to the existing system, and it should run for at least a full business cycle (a week minimum, a full month if traffic is seasonal) before moving on.
Step 4: Build and validate Kafka consumers in shadow mode. Stand up consumer groups that read from Kafka but don’t yet take any production action, just logging or comparing outputs against what the SQS-driven path produces. This is where ordering and duplicate-handling bugs surface before they matter.
Step 5: Cut consumers over service by service. Don’t flip every downstream consumer at once. Move the least critical consumer first, watch it in production against real traffic, then work through the rest in order of increasing business impact.
Step 6: Stop dual-writing and decommission the SQS queue. Once every consumer is reading from Kafka and has run cleanly for a full cycle, remove the SQS write path and let the queue drain naturally before deleting it. Keep the queue definition around (even if idle) for a few weeks as a rollback option, since recreating a deleted SQS queue is trivial but reconnecting all producers to it under time pressure is not.
Pros and Cons
Amazon SQS — Pros: zero infrastructure to manage, pay-as-you-go with no floor, native Lambda integration, simple mental model, strong default durability across availability zones, and a 2025 payload increase to 1 MiB that closed one of its longest-standing gaps with Kafka.
Amazon SQS — Cons: no native replay once a message is deleted or expires, no multi-consumer fan-out without adding SNS, a hard 14-day retention ceiling, and per-request pricing that scales linearly and gets expensive at sustained high volume.
Apache Kafka — Pros: native multi-consumer fan-out through consumer groups, configurable retention up to indefinite, strong ordering within a partition, throughput that scales close to linearly with brokers and partitions, and a simplified, ZooKeeper-free operational model since version 4.0.
Apache Kafka — Cons: real operational complexity even when managed through MSK, a fixed cost floor that doesn’t shrink for low-traffic workloads, a steeper learning curve around partitions and consumer groups, and meaningfully more code and configuration to get a production-grade producer/consumer pair running correctly.
The Verdict: SQS vs Kafka in 2026
Neither system is replacing the other in 2026, and the data explains why. SQS’s request-based pricing wins decisively below roughly 500-1,000 messages per second sustained, and its zero-ops model makes it the correct default for task decoupling, background jobs, and any workload with a single consumer type. Past that throughput threshold, or the moment a second independent consumer needs the same event stream, the math and the architecture both point at Kafka: MSK Provisioned’s fixed-capacity pricing stops scaling with request volume, and consumer groups solve fan-out natively instead of bolting on SNS.
The honest framing isn’t “which is better” but “which one matches the shape of the problem.” A single-consumer, bursty, low-to-medium-volume workload is over-engineered on Kafka. A multi-consumer, high-throughput, replay-dependent pipeline is under-engineered on SQS, no matter how cleverly SNS fan-out is layered on top. Most teams that reach real scale end up running both, side by side, for exactly the workloads each one was built for.
Frequently Asked Questions
Is Amazon SQS built on Kafka?
No. SQS predates Kafka’s public release and uses a completely different architecture — a managed distributed queue rather than a partitioned log. They share the goal of moving messages between services but nothing in the underlying implementation.
Can SQS replay messages like Kafka does?
No, not natively. Once a message is deleted by a consumer or ages past its retention period (maximum 14 days), it’s gone permanently. Kafka retains messages for a configurable period, including indefinitely, and any consumer group can reset its offset to reread history.
Is Amazon MSK the same thing as Kafka?
Amazon MSK runs actual, unmodified Apache Kafka as a managed service, handling broker provisioning, patching, and infrastructure. It’s not a Kafka-compatible clone; it’s Kafka itself, with AWS operating the undifferentiated heavy lifting.
Which is cheaper, SQS or Kafka?
It depends entirely on volume. SQS is cheaper, often dramatically so, at low and medium sustained throughput because there’s no fixed cost floor. Kafka (via MSK) becomes cheaper at high sustained throughput because its cost is capacity-based rather than per-request, so it stops scaling linearly with message volume the way SQS billing does.
Do I need Kafka if I only have one consumer?
Usually not. Consumer groups and replay are Kafka’s core value propositions, and neither matters much with a single consumer. SQS, or even SNS-to-SQS fan-out for a small number of static subscribers, is typically simpler and cheaper for that shape of workload.
What happened to ZooKeeper in Kafka?
It’s gone. Apache Kafka 4.0, released in March 2025, removed ZooKeeper entirely and made KRaft the only supported metadata management mode. Clusters still running on ZooKeeper need to migrate to KRaft before upgrading past the 3.x line.
Can I use SQS and Kafka together in the same architecture?
Yes, and plenty of production systems do. A common pattern uses Kafka as the durable, replayable backbone for core business events and SQS for simpler, single-consumer background tasks that spin out of those events, like sending a notification after an order-placed event is processed.
Does SQS support exactly-once delivery?
FIFO queues do, within a 5-minute deduplication window, as long as producers avoid sending true duplicates outside that window. Standard queues explicitly do not guarantee exactly-once and can deliver a message more than once, so consumers need to be written to handle duplicates safely regardless of queue type.
Can I run Kafka without Amazon MSK?
Yes. Kafka runs anywhere the JVM does, including self-managed EC2 clusters, on-prem hardware, or Kubernetes via an operator like Strimzi. MSK simply removes the operational burden of patching, scaling, and monitoring brokers yourself, at the cost of the MSK management fee layered on top of the underlying compute and storage.
Related Coverage
- Kafka vs Kinesis 2026: $0.015/Shard-Hour and 5x Latency Gap
- Kafka vs RabbitMQ 2026: The Definitive Message Broker Comparison
- Apache Kafka Tutorial: Build a KRaft Cluster in 13 Steps
- ECS vs EKS vs Fargate: $0 vs $73/mo Control Plane
- Karpenter vs Cluster Autoscaler vs KEDA: 3x Faster
- FinOps in 2026: How CFOs Are Finally Taming Runaway Cloud Costs
- More Cloud Computing Coverage
