AWS SDK / API aws throttling rate-limit
Sofia Reyes Cloud Infrastructure Engineer

AWS ThrottlingException: Rate exceeded

Encountering AWS ThrottlingException: Rate exceeded means your API calls are exceeding AWS service limits; this guide explains how to fix it.

What This Error Means

When you encounter an AWS ThrottlingException: Rate exceeded, it signifies that your application or script is making too many API requests to a particular AWS service within a specific time window. AWS services are designed with various quotas (often referred to as limits) to ensure fair usage, prevent abuse, and maintain service stability for all customers. When your request rate exceeds these predefined limits, AWS temporarily rejects your calls with this exception. It's AWS's way of saying, "Hold on, you're asking for too much too quickly."

In my experience, this error is a clear indicator that while your application is attempting to interact with an AWS service, it's not adhering to the operational boundaries set by the platform. It's not a persistent failure but rather a transient one, indicating that if you were to wait a moment and try again, your request might succeed. However, without proper handling, your application will simply keep failing, leading to degraded performance or complete service disruption.

Why It Happens

AWS service quotas are fundamental to how the platform operates. Each AWS service, and often individual API actions within those services, has default limits on the number of requests you can make per second, per minute, or sometimes even per hour. These limits are typically soft limits, meaning they can often be increased, but they exist by default for a reason.

The underlying mechanism often involves algorithms like "token bucket" or "leaky bucket," where requests consume "tokens," and tokens are refilled at a fixed rate. If you try to consume tokens faster than they're refilled, your requests are throttled. Different services have different limit characteristics:

  • Request Rate Limits: The most common type, measured in requests per second (RPS) for specific API actions (e.g., s3:GetObject, ec2:DescribeInstances).
  • Concurrent Resource Limits: The maximum number of concurrent operations (e.g., Lambda function invocations, EC2 instance launches).
  • Throughput Limits: Data transfer rates or specific operations per second for resources like DynamoDB tables or S3 buckets.

When your application operates without awareness of these limits, or when traffic patterns change unexpectedly, throttling becomes inevitable. I've seen this in production when a new feature dramatically increased API call volume, or when an existing batch job scaled unexpectedly.

Common Causes

Identifying the root cause of throttling is the first step towards a sustainable fix. Based on what I've encountered, here are the most common scenarios leading to Rate exceeded errors:

  • Missing or Inadequate Retry Logic: This is arguably the most frequent cause. Applications that don't implement exponential backoff with jitter will simply hammer the API when a transient error like throttling occurs, exacerbating the problem and ensuring continuous failures.
  • Burst Traffic: A sudden surge in user activity, a new marketing campaign, or a spike in background job processing can push a previously stable application beyond its limits.
  • Inefficient Polling: Frequently querying an AWS service for status updates (e.g., checking if an EC2 instance has started, or if an S3 object exists) without sufficient delays can quickly exhaust request quotas.
  • Bulk Operations Without Delay: Scripts designed to perform mass updates, deletions, or creations (e.g., updating tags on thousands of EC2 instances, or deleting many S3 objects) often fail if they don't introduce deliberate pauses between API calls.
  • Shared Account or Resources: In organizations where multiple teams or applications use the same AWS account or even the same AWS IAM role, their combined API calls can hit shared service quotas unexpectedly.
  • Incorrectly Configured Concurrency: If you're running containerized applications or serverless functions (like AWS Lambda) that scale rapidly, each instance might be making API calls. If the sum of these calls exceeds a service limit, throttling will occur.
  • New Deployments or Feature Releases: Sometimes, a new piece of functionality or an application update makes significantly more API calls than anticipated during testing, leading to throttling in production.
  • Monitoring and Alerting Systems: Ironically, over-aggressive monitoring scripts that frequently poll AWS APIs for metrics or resource states can sometimes contribute to throttling if not designed carefully.

Step-by-Step Fix

Addressing a ThrottlingException typically involves a combination of immediate mitigation and long-term architectural adjustments.

  1. Identify the Throttled Service and API Action:

    • Application Logs: Your application logs are the first place to look. The ThrottlingException message usually indicates which AWS service and sometimes even the specific API action (e.g., s3:GetObject, ec2:RunInstances) was throttled.
    • AWS CloudTrail: CloudTrail logs all API calls made to your AWS account. Filter CloudTrail events for errorMessage = *ThrottlingException* or errorCode = ThrottlingException. This will tell you exactly which user, role, service, and API action experienced the throttling.
    • AWS CloudWatch Metrics: Many AWS services provide CloudWatch metrics for ThrottledRequests or CallCount. Look at these metrics for the relevant service to identify spikes correlating with the throttling events in your logs.

  2. Implement or Enhance Exponential Backoff with Jitter:
    This is the single most important and effective step for handling transient AWS API errors. Most AWS SDKs provide this functionality built-in or make it easy to implement.

    • Exponential Backoff: When an API call fails due to throttling, don't retry immediately. Wait for a short period, then retry. If it fails again, wait for an exponentially longer period, and so on. This gives the AWS service time to recover and allows your application to "back off" gracefully.
    • Jitter: To prevent a "thundering herd" problem where many retries from multiple instances align their retry attempts and hit the API at the same exact moment, introduce a small, random delay (jitter) within your backoff algorithm. This spreads out the retry attempts, reducing contention.

    ```python
    import boto3
    import time
    import random

    Example: S3 client with basic retry logic

    s3_client = boto3.client('s3')

    def get_s3_object_with_retries(bucket_name, key, max_retries=5):
    for i in range(max_retries):
    try:
    response = s3_client.get_object(Bucket=bucket_name, Key=key)
    print(f"Successfully retrieved {key}")
    return response['Body'].read().decode('utf-8')
    except s3_client.exceptions.ClientError as e:
    if e.response['Error']['Code'] == 'ThrottlingException' or e.response['Error']['Code'] == 'TooManyRequests':
    wait_time = (2 ** i) + random.uniform(0, 1) # Exponential backoff + jitter
    print(f"Throttled. Retrying {key} in {wait_time:.2f} seconds...")
    time.sleep(wait_time)
    else:
    raise # Re-raise other exceptions
    except Exception as e:
    print(f"An unexpected error occurred: {e}")
    raise

    print(f"Failed to retrieve {key} after {max_retries} attempts due to throttling.")
return None

    Usage example

    content = get_s3_object_with_retries('my-bucket', 'my-file.txt')

    if content:

    print(content)

    ```

  3. Optimize API Call Patterns:

    • Batching: Many AWS services (e.g., SQS, DynamoDB, Lambda Invoke for asynchronous calls) support batch operations. Instead of making 100 individual API calls, make one call with 100 items. This drastically reduces the request rate.
    • Reduce Polling Frequency: If you're polling for resource status, evaluate if you can poll less frequently. Can you switch to an event-driven architecture (e.g., S3 events, CloudWatch events, DynamoDB Streams) instead of polling?
    • Caching: Cache API responses for data that doesn't change frequently. This reduces the need to hit AWS APIs for every request.
    • Pre-provisioning/Pre-warming: For bursty workloads, consider pre-provisioning resources or pre-warming caches instead of relying on on-demand scaling to catch up, which can cause initial API call spikes.

  4. Request a Service Limit Increase:
    If your application genuinely requires a higher request rate than the default limits, you can request a service quota increase.

    • Navigate to the AWS Management Console -> Service Quotas.
    • Search for the specific service (e.g., S3, EC2).
    • Find the relevant quota (e.g., "GET requests per second for S3 buckets").
    • Click "Request quota increase."
    • Provide a detailed business justification: Explain why you need the increase, your expected peak RPS, and how you're implementing retry logic. AWS support needs to understand your use case to approve the request. This process can take a few business days, so plan accordingly.

  5. Distribute Load and Scale Out:

    • Across Regions/Accounts: If a single region's limits are insufficient, consider distributing your workload across multiple AWS regions or even separate AWS accounts (with proper cross-account access). This allows you to leverage separate sets of service quotas.
    • Application Scaling: Ensure your application's scaling strategy aligns with AWS service limits. If you launch many instances, each hitting the same AWS API, you're likely to get throttled. Consider using a centralized queue (e.g., SQS) to manage API requests, ensuring a controlled rate.

Code Examples

Here are concise, copy-paste ready examples demonstrating retry logic.

Python (Boto3) with Exponential Backoff and Jitter

This example uses the botocore.exceptions.ClientError to catch throttling specific errors and applies the retry logic.

import boto3
import time
import random
from botocore.exceptions import ClientError

def call_aws_api_with_retries(service_name, api_method_name, **kwargs):
    client = boto3.client(service_name)
    max_retries = 8
    base_delay = 0.1 # seconds

    for i in range(max_retries):
        try:
            method = getattr(client, api_method_name)
            response = method(**kwargs)
            print(f"Successfully called {service_name}:{api_method_name}.")
            return response
        except ClientError as e:
            error_code = e.response.get("Error", {}).get("Code")
            if error_code in ["ThrottlingException", "TooManyRequestsException", "ProvisionedThroughputExceededException"]:
                delay = (base_delay * (2 ** i)) + random.uniform(0, base_delay) # Exponential backoff with jitter
                print(f"ThrottlingException received ({error_code}). Retrying {service_name}:{api_method_name} in {delay:.2f} seconds. (Attempt {i+1}/{max_retries})")
                time.sleep(delay)
            else:
                print(f"Non-throttling error: {error_code} - {e}")
                raise
        except Exception as e:
            print(f"An unexpected error occurred: {e}")
            raise

    print(f"Failed to call {service_name}:{api_method_name} after {max_retries} attempts.")
    raise Exception(f"Failed after {max_retries} retries due to throttling.")

# Example usage: Describe EC2 instances
try:
    ec2_instances = call_aws_api_with_retries('ec2', 'describe_instances')
    # Process ec2_instances
    # print(ec2_instances)
except Exception as e:
    print(f"Caught final exception: {e}")

# Example usage: Put an object to S3 (careful with object content and bucket)
# try:
#     s3_response = call_aws_api_with_retries(
#         's3', 'put_object',
#         Bucket='your-test-bucket',
#         Key='test-file.txt',
#         Body='Hello from Sofia Reyes!'
#     )
#     print(s3_response)
# except Exception as e:
#      print(f"Caught final exception for S3: {e}")

AWS CLI with Retry Logic (using jq and sleep for simple scripting)

While AWS CLI generally has built-in retry mechanisms, you might need custom logic for specific batch operations in shell scripts.

#!/bin/bash

BUCKET_NAME="my-awesome-bucket"
PREFIX="data/"
MAX_RETRIES=5
BASE_DELAY=0.5 # seconds

for i in $(seq 0 $((MAX_RETRIES - 1))); do
    echo "Attempt $((i+1)) to list S3 objects..."
    response=$(aws s3api list-objects-v2 --bucket "$BUCKET_NAME" --prefix "$PREFIX" 2>&1)

    if echo "$response" | grep -q "ThrottlingException"; then
        delay=$(awk "BEGIN {print ($BASE_DELAY * (2^$i)) + (rand() * $BASE_DELAY)}") # Exponential backoff + jitter
        echo "ThrottlingException detected. Retrying in $delay seconds..."
        sleep "$delay"
    elif echo "$response" | grep -q "Error"; then
        echo "An unexpected AWS CLI error occurred:"
        echo "$response"
        exit 1
    else
        echo "Successfully listed S3 objects."
        echo "$response" | jq '.Contents[].Key' # Example: pipe to jq for processing
        exit 0
    fi
done

echo "Failed to list S3 objects after $MAX_RETRIES attempts due to throttling."
exit 1

Environment-Specific Notes

The approach to handling ThrottlingException can vary slightly depending on your environment.

  • Cloud (Production):

    • Proactive Monitoring: This is critical. Use AWS CloudWatch Alarms to monitor ThrottledRequests metrics for services like Lambda, DynamoDB, SQS, Kinesis, etc. Set up alerts (e.g., SNS topics, PagerDuty) to notify your team when throttling events occur.
    • Service Quotas Dashboard: Regularly review your AWS Service Quotas dashboard for services you heavily rely on. It provides a centralized view of your current limits and usage, helping you anticipate potential bottlenecks before they become production issues.
    • Centralized Logging: Ensure all application logs are shipped to a centralized logging solution (CloudWatch Logs, ELK stack, Datadog, etc.) to quickly correlate throttling events with specific application behavior or deployments.

  • Docker/Containerized Environments:

    • Shared Credentials: If multiple containers share the same IAM role or access keys, they will collectively hit the same API limits. Ensure your application architecture distributes API calls effectively or relies on queues to regulate the rate.
    • Rapid Scaling: While containers enable rapid scaling, launching many containers simultaneously can lead to a "thundering herd" problem where all new containers immediately attempt to make initial API calls (e.g., fetching configuration, registering with a service), causing a burst of throttled requests. Implement controlled rollout strategies and robust retry logic.
    • Sidecar Proxies: Consider using a proxy or service mesh (like Envoy with AWS App Mesh) that can centrally manage and enforce retry policies and rate limiting for all outbound AWS API calls from your containers.

  • Local Development:

    • Avoid Hitting Real AWS Limits: During local development and testing, especially for integration tests, it's easy to accidentally hammer AWS APIs and get throttled, impacting your development speed or even affecting shared dev accounts.
    • Mock AWS Services: Use tools like moto (Python) or LocalStack to simulate AWS services locally. This allows you to test your application's AWS interactions without making actual API calls, thus avoiding throttling.
    • Smaller Datasets: When testing functionality that involves bulk operations, use significantly smaller datasets locally to reduce the API call volume.
    • Rate Limit Simulation: If you need to test your retry logic, you might consider introducing artificial delays or error injection into your local mocks to simulate throttling.

Frequently Asked Questions

Q: How do I know exactly which AWS service is throttling me?
A: Check your application logs for the ThrottlingException message; it usually includes the service. For definitive proof, consult AWS CloudTrail logs by filtering for errorMessage = "ThrottlingException". Also, CloudWatch metrics like ThrottledRequests under specific services can help pinpoint the culprit.

Q: What's the difference between hard and soft limits in AWS?
A: Soft limits are default quotas that AWS sets for most services, which can typically be increased upon request via the AWS Service Quotas console with a valid business justification. Hard limits are absolute maximums that cannot be increased, often due to architectural constraints of the service. Most throttling exceptions arise from hitting soft limits.

Q: Does increasing my service limits cost money?
A: Generally, increasing API request rate limits does not directly incur additional costs. However, higher limits might allow your application to consume more underlying AWS resources (e.g., more EC2 instances, higher DynamoDB throughput, more Lambda invocations), which will increase your billing. Always consider the cost implications of scaling up.

Q: Can I proactively monitor for potential throttling issues before they happen?
A: Yes. The AWS Service Quotas dashboard provides utilization metrics for many quotas, showing how close you are to hitting a limit. You can set up CloudWatch alarms on these utilization metrics to get alerted when you exceed a certain threshold (e.g., 80% of a quota) before actual throttling occurs.

Q: What is "jitter" in the context of exponential backoff?
A: Jitter refers to adding a small, random delay to the calculated backoff time. The purpose is to prevent a "thundering herd" problem, where multiple retrying clients or instances might synchronize their retry attempts and all hit the API at the same exact moment, causing another burst of throttling. Jitter spreads out these retries, making the overall load smoother.