Simple Queue Service (SQS)

Queuing of "messages" between components as a service to ensure architectures without component-order reliance.

Decoupled Architecture

Tightly Coupled System

• A system architecture of components that are not just linked together but are also dependent upon each other.

• If one component fails, all components fail

Loosely Coupled/Decoupled Systems

• Multiple components that can process information without being connected

• Components are not connected - if one fails the rest of the system can continue processing (fault tolerant/highly available)

• AWS Services that are used for distributed/decoupled system architectures:

  • SWF (Simple Work Flow Service)

  • SQS (Simple Queue Service)

Essentials

• Provides the ability to have hosted/highly available queues that can be used for messages being sent between servers.

• Creation of distributed/decoupled application components

• Used to create decoupled application environments

• Messages between servers are retrieved through polling

Polling

• Long Polling (1-20 seconds):

  • Allows the SQS service to wait until a message is available in a queue before sending a response, and will return all messages from all SQS services.

  • Long polling reduces API requests (over using short polling)

• Short Polling

  • SQS samples a subset of servers and returns messages from just those servers

  • Will not return all possible messages in a poll

  • Increases API requests (over long polling), will increase costs

Facts

• Each message can have up to 256KB of text (any format). For larger messages, use S3.

• Amazon SQS offers two different types of queues:

  • Standard Queue: Guarantees delivery of each message at least once but does not guarantee the order (best effort) in which they are delivered to the queue

    • Also has At-least-once delivery: will guarantee delivery at least once, but possibly actually more time (duplicates)

  • First-in-first-out (FIFO) Queue: Designed for applications where the order of operations and events is critical, or where duplicates can't be tolerated. Limited to 300 transactions per second.

• Visibility Timeout - how long messages are invisible after it has been polled

• Stores messages up to 14 days, then moved to "dead letter queue" if it was never processed

• Queues allow components of your applications to work independently of each other (Decoupled!)

SQS Workflow

• Generally a "worker" instance will "poll" a queue to retrieve waiting messages for processing.

• Auto Scaling can be applied based off of queue size so that if a component of your application has an increase in demand, the number of work instances can increase.