Simple Queue Service (SQS)

Queuing of "messages" between components as a service to ensure architectures without component-order reliance.

Decoupled Architecture

Tightly Coupled System

  • A system architecture of components that are not just linked together but are also dependent upon each other.

  • If one component fails, all components fail

Loosely Coupled/Decoupled Systems

  • Multiple components that can process information without being connected

  • Components are not connected - if one fails the rest of the system can continue processing (fault tolerant/highly available)

  • AWS Services that are used for distributed/decoupled system architectures:

    • SWF (Simple Work Flow Service)

    • SQS (Simple Queue Service)

Essentials

  • Provides the ability to have hosted/highly available queues that can be used for messages being sent between servers.

  • Creation of distributed/decoupled application components

  • Used to create decoupled application environments

  • Messages between servers are retrieved through polling

Polling

  • Long Polling (1-20 seconds):

    • Allows the SQS service to wait until a message is available in a queue before sending a response, and will return all messages from all SQS services.

    • Long polling reduces API requests (over using short polling)

  • Short Polling

    • SQS samples a subset of servers and returns messages from just those servers

    • Will not return all possible messages in a poll

    • Increases API requests (over long polling), will increase costs

Facts

  • Each message can have up to 256KB of text (any format). For larger messages, use S3.

  • Amazon SQS offers two different types of queues:

    • Standard Queue: Guarantees delivery of each message at least once but does not guarantee the order (best effort) in which they are delivered to the queue

      • Also has At-least-once delivery: will guarantee delivery at least once, but possibly actually more time (duplicates)

    • First-in-first-out (FIFO) Queue: Designed for applications where the order of operations and events is critical, or where duplicates can't be tolerated. Limited to 300 transactions per second.

  • Visibility Timeout - how long messages are invisible after it has been polled

  • Stores messages up to 14 days, then moved to "dead letter queue" if it was never processed

  • Queues allow components of your applications to work independently of each other (Decoupled!)

SQS Workflow

  • Generally a "worker" instance will "poll" a queue to retrieve waiting messages for processing.

  • Auto Scaling can be applied based off of queue size so that if a component of your application has an increase in demand, the number of work instances can increase.

Last updated

Was this helpful?