Introduction

Amazon Web Services (AWS) offers two powerful messaging services, Simple Queue Service (SQS) and Simple Notification Service (SNS), that are frequently used for dispatching work to multiple listeners. Each service has distinct characteristics, making them suitable for specific use cases. This article explores the differences between SQS and SNS, explains when to use each, and examines scenarios where combining them is the best solution.

SQS: A Quick Overview

Amazon SQS is a fully managed message queuing service. It decouples components of a distributed system, ensuring reliable message delivery between producers and consumers. Key features include:

• Message Persistence: Messages are stored until they are processed or expire.
• At-Least-Once Delivery: Ensures every message is delivered at least once.
• Visibility Timeout: Prevents multiple consumers from processing the same message simultaneously.

When to Use SQS

• Decoupling Producers and Consumers: When producers and consumers operate at different speeds.
• Work Queues: For distributing tasks to multiple workers.
• Message Persistence: When it is critical to ensure messages are not lost.

Example: When SQS is the Best Choice

Producers --> [ SQS Queue ] --> Consumers

Producers: Upload endpoints
SQS Queue: Holds video processing tasks
Consumers: Video processors (e.g., workers)

Imagine a video processing service where users upload videos, and the system processes them in the background. Producers (upload endpoints) may generate tasks faster than workers (video processors) can handle. Using SQS ensures reliable queuing of tasks without overwhelming the workers.

Example: When SQS Should Be Avoided

In a real-time chat application, SQS might seem suitable for queuing messages between users. However, the latency introduced by polling the queue makes it a poor choice. Instead, using a real-time messaging service like WebSocket or SNS for event-driven updates is more appropriate.

SNS: A Quick Overview

Amazon SNS is a fully managed publish-subscribe messaging service. It delivers messages to multiple subscribers via multiple protocols (e.g., HTTP/S, email, SMS, SQS). Key features include:

• Fan-Out Architecture: Messages are sent to all subscribers simultaneously.
• Multi-Protocol Delivery: Supports delivery via HTTP/S, Lambda, SMS, email, and SQS.
• Event-Driven Systems: Ideal for triggering multiple downstream processes.

When to Use SNS

• Broadcasting Messages: When a message needs to reach multiple subscribers.
• Real-Time Event Notifications: For alerting systems or event-driven architectures.
• Multi-Protocol Communication: When you need to notify services or people across diverse channels.

Example: When SNS is the Best Choice

Producers --> [ SNS Topic ] --> [ Mobile Apps ]
                                --> [ Analytics Services ]
                                --> [ Auditing Systems ]

Producers: Stock price update generators
SNS Topic: Broadcasts price updates
Subscribers: Mobile apps, analytics services, auditing systems

Consider a stock trading platform where price updates must be sent to multiple clients and services (e.g., mobile apps, analytics services, and auditing systems). SNS can broadcast these updates in real-time, ensuring all subscribers receive the information simultaneously.

Example: When SNS Should Be Avoided

For scenarios requiring guaranteed delivery to each subscriber, such as processing orders in an e-commerce system, SNS alone is insufficient. A queue-based system like SQS or combining SNS with SQS for persistence is a better choice.

Combining SQS and SNS

While SQS and SNS can be used independently, combining them is often the best choice for complex systems with multiple consumers. Here are the two common patterns:

SNS in Front of SQS

• Use Case: Broadcasting work to multiple, independent queues.
• How It Works:
  1. An SNS topic receives messages from producers.
  2. Multiple SQS queues subscribe to the SNS topic.
  3. Each queue processes messages independently.
• Why Use It:
  • Fan-out messages to multiple processing systems.
  • Decouple publishers from downstream systems.
  • Enable independent scaling of consumers.

Example: When SNS in Front of SQS is Best

Producers --> [ SNS Topic ] --> [ SQS Queue: Payments ] --> Payment Service
                                --> [ SQS Queue: Inventory ] --> Inventory Service
                                --> [ SQS Queue: Notifications ] --> Notification Service

Producers: Booking system events
SNS Topic: Distributes events
SQS Queues: Independent processing queues for each service

A ticket booking system where different services handle payments, inventory, and notifications can benefit from this approach. SNS distributes the booking events to SQS queues, ensuring each service processes events independently and reliably.

Example: When SNS in Front of SQS Should Be Avoided

For a single consumer that requires strict ordering of messages, this pattern is unnecessary and might complicate the architecture. Using a single SQS queue suffices.

SQS in Front of SNS

• Use Case: Ensuring reliable delivery before broadcasting messages.
• How It Works:
  1. A producer sends messages to an SQS queue.
  2. A consumer reads from the queue and forwards messages to an SNS topic.
  3. The SNS topic distributes messages to its subscribers.
• Why Use It:
  • Add persistence and durability to message handling.
  • Prevent message loss due to transient SNS subscriber issues.
  • Introduce throttling or buffering between systems.

Example: When SQS in Front of SNS is Best

Producers --> [ SQS Queue ] --> [ SNS Topic ] --> [ Analytics Systems ]
                                         --> [ Alerting Services ]

Producers: IoT sensors
SQS Queue: Buffers sensor data
SNS Topic: Broadcasts data to subscribers
Subscribers: Analytics and alerting systems

An IoT system collecting sensor data can use SQS to queue incoming data and then forward it to SNS for broadcasting to analytics systems and alerting services. This setup ensures no data is lost if SNS or subscribers experience downtime.

Conclusion

SQS and SNS are indispensable tools for building scalable and resilient distributed systems. By evaluating your system requirements, you can architect solutions that efficiently dispatch work to multiple listeners while ensuring reliability and scalability.