We’ve
been working with, building, and evangelising message queues for the last year,
and it’s no secret that we think they’re awesome. We believe message queues are
a vital component to any architecture or application, and here are ten reasons
why:
1. Decoupling
It’s extremely difficult to predict, at the start of a project, what the future needs
of the project will be. By introducing a layer in between processes, message
queues create an implicit, data-based interface that both processes implement.
This allows you to extend and modify these processes independently, by simply
ensuring they adhere to the same interface
requirements.
2. Redundancy
Sometimes processes fail when processing data. Unless that data is persisted, it’s lost
forever. Queues mitigate this by persisting data until it has been fully
processed. The put-get-delete paradigm, which many message queues use, requires
a process to explicitly indicate that it has finished processing a message
before the message is removed from the queue, ensuring your data is kept safe
until you’re done with it.
3. Scalability Because message queues decouple your processes, it’s easy to scale up the rate with
which messages are added to the queue or processed; simply add another process.
No code needs to be changed, no configurations need to be tweaked. Scaling is as
simple as adding more power.
4. Elasticity & Spikability
When your application hits the front page of Hacker News, you’re going to see unusual
levels of traffic. Your application needs to be able to keep functioning with
this increased load, but the traffic is anomaly, not the standard; it’s wasteful
to have enough resources on standby to handle these spikes. Message queues will
allow beleaguered components to struggle through the increased load, instead of
getting overloaded with requests and failing completely. Check out our Spikability
blog post
for more information about this.
5. Resiliency
When part of your architecture fails, it doesn’t need to take the entire system down
with it. Message queues decouple processes, so if a process that is processing
messages from the queue fails, messages can still be added to the queue to be
processed when the system recovers. This ability to accept requests that will be
retried or processed at a later date is often the difference between an
inconvenienced customer and a frustrated customer.
6. Delivery Guarantees
The redundancy provided by message queues guarantees that a
message will be processed eventually, so long as a process is reading the queue.
On top of that, IronMQ provides an only-delivered-once guarantee. No matter how
many processes are pulling data from the queue, each message will only be
processed a single time. This is made possible because retrieving a message
"reserves" that message, temporarily removing it from the queue. Unless the
client specifically states that it's finished with that message, the message
will be placed back on the queue to be processed after a configurable amount of
time.
7. Ordering Guarantees
In a lot of situations, the order with which data is processed is important. Message queues are inherently ordered, and capable of providing guarantees that data will be processed in a specific order. IronMQ guarantees that messages will be processed using FIFO (first in, first out), so
the order in which messages are placed on a queue is the order in which they'll
be retrieved from it.
8. Buffering In any non-trivial system, there are going to be components that require different
processing times. For example, it takes less time to upload an image than it
does to apply a filter to it. Message queues help these tasks operate at peak
efficiency by offering a buffer layer--the process writing to the queue can
write as fast as it’s able to, instead of being constrained by the readiness of
the process reading from the queue. This buffer helps control and optimise the
speed with which data flows through your system.
9. Understanding Data Flow
In a distributed system, getting an overall sense of how long user actions take to complete and why is a huge problem. Message queues, through the rate with which they are processed, help to easily identify under-performing processes or areas where the data flow is not optimal.
10. Asynchronous Communication
A lot of times, you don’t want to or need to process a message immediately. Message queues enable asynchronous processing, which allows you to put a message on the queue without processing it immediately. Queue up as many messages as you like, then process them at your
leisure.
Disclaimer: This is not an original post. It is taken from http://blog.iron.io/2012/12/top-10-uses-for-message-queue.html
been working with, building, and evangelising message queues for the last year,
and it’s no secret that we think they’re awesome. We believe message queues are
a vital component to any architecture or application, and here are ten reasons
why:
1. Decoupling
It’s extremely difficult to predict, at the start of a project, what the future needs
of the project will be. By introducing a layer in between processes, message
queues create an implicit, data-based interface that both processes implement.
This allows you to extend and modify these processes independently, by simply
ensuring they adhere to the same interface
requirements.
2. Redundancy
Sometimes processes fail when processing data. Unless that data is persisted, it’s lost
forever. Queues mitigate this by persisting data until it has been fully
processed. The put-get-delete paradigm, which many message queues use, requires
a process to explicitly indicate that it has finished processing a message
before the message is removed from the queue, ensuring your data is kept safe
until you’re done with it.
3. Scalability Because message queues decouple your processes, it’s easy to scale up the rate with
which messages are added to the queue or processed; simply add another process.
No code needs to be changed, no configurations need to be tweaked. Scaling is as
simple as adding more power.
4. Elasticity & Spikability
When your application hits the front page of Hacker News, you’re going to see unusual
levels of traffic. Your application needs to be able to keep functioning with
this increased load, but the traffic is anomaly, not the standard; it’s wasteful
to have enough resources on standby to handle these spikes. Message queues will
allow beleaguered components to struggle through the increased load, instead of
getting overloaded with requests and failing completely. Check out our Spikability
blog post
for more information about this.
5. Resiliency
When part of your architecture fails, it doesn’t need to take the entire system down
with it. Message queues decouple processes, so if a process that is processing
messages from the queue fails, messages can still be added to the queue to be
processed when the system recovers. This ability to accept requests that will be
retried or processed at a later date is often the difference between an
inconvenienced customer and a frustrated customer.
6. Delivery Guarantees
The redundancy provided by message queues guarantees that a
message will be processed eventually, so long as a process is reading the queue.
On top of that, IronMQ provides an only-delivered-once guarantee. No matter how
many processes are pulling data from the queue, each message will only be
processed a single time. This is made possible because retrieving a message
"reserves" that message, temporarily removing it from the queue. Unless the
client specifically states that it's finished with that message, the message
will be placed back on the queue to be processed after a configurable amount of
time.
7. Ordering Guarantees
In a lot of situations, the order with which data is processed is important. Message queues are inherently ordered, and capable of providing guarantees that data will be processed in a specific order. IronMQ guarantees that messages will be processed using FIFO (first in, first out), so
the order in which messages are placed on a queue is the order in which they'll
be retrieved from it.
8. Buffering In any non-trivial system, there are going to be components that require different
processing times. For example, it takes less time to upload an image than it
does to apply a filter to it. Message queues help these tasks operate at peak
efficiency by offering a buffer layer--the process writing to the queue can
write as fast as it’s able to, instead of being constrained by the readiness of
the process reading from the queue. This buffer helps control and optimise the
speed with which data flows through your system.
9. Understanding Data Flow
In a distributed system, getting an overall sense of how long user actions take to complete and why is a huge problem. Message queues, through the rate with which they are processed, help to easily identify under-performing processes or areas where the data flow is not optimal.
10. Asynchronous Communication
A lot of times, you don’t want to or need to process a message immediately. Message queues enable asynchronous processing, which allows you to put a message on the queue without processing it immediately. Queue up as many messages as you like, then process them at your
leisure.
We believe these ten reasons make queues the best form of communication between
processes or applications. We’ve spent a year building and learning from IronMQ,
and our customers are doing amazing things with message queues. Queues are the
key to the powerful, distributed applications that can leverage all the power
that the cloud has to offer.
processes or applications. We’ve spent a year building and learning from IronMQ,
and our customers are doing amazing things with message queues. Queues are the
key to the powerful, distributed applications that can leverage all the power
that the cloud has to offer.
Disclaimer: This is not an original post. It is taken from http://blog.iron.io/2012/12/top-10-uses-for-message-queue.html
No comments:
Post a Comment