A Python decorator that automatically rotates through multiple AWS accounts when making Amazon Bedrock API calls, providing automatic retry and load balancing across accounts.
- Automatic account rotation: Cycles through multiple AWS accounts to distribute load
- Throttling resilience: Automatically retries with the next account when hitting rate limits
- Simple decorator pattern: Easy to integrate with existing Bedrock code
- Zero configuration: Just provide your account credentials and start using
AWS Bedrock's default quotas are severely restrictive for any real-world application:
- Low default limits: Default quotas are often as low as 20 requests per minute and can be set to zero without warning
- Unpredictable quota cuts: AWS has been known to reduce quotas by up to 95% even for active accounts, sometimes with minimal notice
- Account-level throttling: Rate limits are enforced per AWS account across three dimensions: RPM (requests per minute), TPM (tokens per minute), and TPD (tokens per day)
AWS's official alternative, Provisioned Throughput, has major drawbacks:
- Wildly expensive: Costs can reach $30k per month for a single model unit, making it prohibitive for most use cases
- Unclear capacity: AWS doesn't provide specific numbers for what throughput you actually get per model unit - you must "contact your AWS account manager" for details
- Long-term commitments: Requires 1-month or 6-month commitments that you cannot cancel
Requesting quota increases through official channels is frustrating:
- Long approval times: Increases "might take a couple of days" to take effect, with some users reporting weeks of delays
- Requires justification: AWS reviews requests carefully and may deny them, requiring detailed business justification
- No guarantees: Approval is not guaranteed and may take significant time to process
- Support ticket dependency: Many quota adjustments require opening support cases, adding delays
AWS itself recommends using multiple accounts to distribute quotas and handle rate limits:
- Official best practice: AWS documentation explicitly states that multiple accounts help "distribute AWS Service Quotas and API request-rate limits"
- Isolation benefits: Multiple accounts provide "natural security, access, and billing boundaries" and help "limit scope of impact from adverse events"
- Scalability: Industry experts recommend "partition requests across multiple AWS accounts (each with its own quotas)" for high-volume workloads
Instead of fighting AWS's quota system, work with it by:
✅ Leveraging multiple accounts to multiply your effective rate limits
✅ Automatic failover when one account hits throttling
✅ Zero cost compared to expensive Provisioned Throughput
✅ Immediate deployment - no waiting for quota approvals
✅ Simple integration with existing code using a decorator pattern
This approach is not just a workaround—it's following AWS's own recommended architecture patterns for scalable, resilient applications.
# Clone the repository
git clone https://github.com/your-username/bedrock-bounce.git
cd bedrock-bounce
# Install in development mode
pip install -e .
# Or install directly from the source
pip install .from bedrock_bounce import bedrock_bounce
# Define your AWS accounts with credentials and preferred model
accounts = [
("AKIA...111", "secret123", "us-east-1", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
("AKIA...222", "secret456", "us-west-2", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
("AKIA...333", "secret789", "eu-west-1", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
]
# Decorate your Bedrock function
@bedrock_bounce(accounts)
def invoke_model(client, model_id, body):
return client.invoke_model(
modelId=model_id,
body=body,
contentType="application/json"
)
# Use it like a normal function
response = invoke_model('{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}')The bedrock_bounce decorator:
- Rotates accounts: Uses a round-robin approach to distribute requests across your AWS accounts
- Handles throttling: When one account hits rate limits, automatically tries the next account
- Injects dependencies: Automatically provides the configured
boto3client andmodel_idto your function - Fails gracefully: Raises an exception only when all accounts are throttled
Each account tuple should contain:
access_key: AWS Access Key IDsecret_key: AWS Secret Access Keyregion: AWS region where Bedrock is availablemodel_id: The Bedrock model ID to use for this account
accounts = [
(access_key, secret_key, region, model_id),
# ... more accounts
]The decorator catches ThrottlingException errors and automatically retries with the next account. All other ClientError exceptions are re-raised immediately.
If all accounts are throttled, the decorator raises:
Exception("All accounts throttled")- Python 3.6+
- boto3
- botocore
This project is licensed under the BSD-3-Clause License - see the LICENSE file for details.
Contributions are welcome! Please feel free to submit a Pull Request.