Skip to content

About

A Python decorator that automatically rotates through multiple AWS accounts when making Amazon Bedrock API calls, providing automatic retry and load balancing across accounts.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

8 Commits

Folders and files

Repository files navigation

bedrock-bounce

A Python decorator that automatically rotates through multiple AWS accounts when making Amazon Bedrock API calls, providing automatic retry and load balancing across accounts.

Features

  • Automatic account rotation: Cycles through multiple AWS accounts to distribute load
  • Throttling resilience: Automatically retries with the next account when hitting rate limits
  • Simple decorator pattern: Easy to integrate with existing Bedrock code
  • Zero configuration: Just provide your account credentials and start using

Why Use bedrock-bounce?

The Problem with AWS Bedrock Rate Limits

AWS Bedrock's default quotas are severely restrictive for any real-world application:

  • Low default limits: Default quotas are often as low as 20 requests per minute and can be set to zero without warning
  • Unpredictable quota cuts: AWS has been known to reduce quotas by up to 95% even for active accounts, sometimes with minimal notice
  • Account-level throttling: Rate limits are enforced per AWS account across three dimensions: RPM (requests per minute), TPM (tokens per minute), and TPD (tokens per day)

Provisioned Throughput Isn't a Solution

AWS's official alternative, Provisioned Throughput, has major drawbacks:

  • Wildly expensive: Costs can reach $30k per month for a single model unit, making it prohibitive for most use cases
  • Unclear capacity: AWS doesn't provide specific numbers for what throughput you actually get per model unit - you must "contact your AWS account manager" for details
  • Long-term commitments: Requires 1-month or 6-month commitments that you cannot cancel

Quota Increases Are Problematic

Requesting quota increases through official channels is frustrating:

  • Long approval times: Increases "might take a couple of days" to take effect, with some users reporting weeks of delays
  • Requires justification: AWS reviews requests carefully and may deny them, requiring detailed business justification
  • No guarantees: Approval is not guaranteed and may take significant time to process
  • Support ticket dependency: Many quota adjustments require opening support cases, adding delays

Multi-Account Strategy is AWS-Recommended

AWS itself recommends using multiple accounts to distribute quotas and handle rate limits:

  • Official best practice: AWS documentation explicitly states that multiple accounts help "distribute AWS Service Quotas and API request-rate limits"
  • Isolation benefits: Multiple accounts provide "natural security, access, and billing boundaries" and help "limit scope of impact from adverse events"
  • Scalability: Industry experts recommend "partition requests across multiple AWS accounts (each with its own quotas)" for high-volume workloads

The Solution: bedrock-bounce

Instead of fighting AWS's quota system, work with it by:

✅ Leveraging multiple accounts to multiply your effective rate limits
✅ Automatic failover when one account hits throttling
✅ Zero cost compared to expensive Provisioned Throughput
✅ Immediate deployment - no waiting for quota approvals
✅ Simple integration with existing code using a decorator pattern

This approach is not just a workaround—it's following AWS's own recommended architecture patterns for scalable, resilient applications.

Installation

# Clone the repository
git clone https://github.com/your-username/bedrock-bounce.git
cd bedrock-bounce

# Install in development mode
pip install -e .

# Or install directly from the source
pip install .

Quick Start

from bedrock_bounce import bedrock_bounce

# Define your AWS accounts with credentials and preferred model
accounts = [
    ("AKIA...111", "secret123", "us-east-1", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
    ("AKIA...222", "secret456", "us-west-2", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
    ("AKIA...333", "secret789", "eu-west-1", "anthropic.claude-3-5-sonnet-20241022-v2:0"),
]

# Decorate your Bedrock function
@bedrock_bounce(accounts)
def invoke_model(client, model_id, body):
    return client.invoke_model(
        modelId=model_id, 
        body=body, 
        contentType="application/json"
    )

# Use it like a normal function
response = invoke_model('{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}')

How It Works

The bedrock_bounce decorator:

  1. Rotates accounts: Uses a round-robin approach to distribute requests across your AWS accounts
  2. Handles throttling: When one account hits rate limits, automatically tries the next account
  3. Injects dependencies: Automatically provides the configured boto3 client and model_id to your function
  4. Fails gracefully: Raises an exception only when all accounts are throttled

Account Configuration

Each account tuple should contain:

  • access_key: AWS Access Key ID
  • secret_key: AWS Secret Access Key
  • region: AWS region where Bedrock is available
  • model_id: The Bedrock model ID to use for this account
accounts = [
    (access_key, secret_key, region, model_id),
    # ... more accounts
]

Error Handling

The decorator catches ThrottlingException errors and automatically retries with the next account. All other ClientError exceptions are re-raised immediately.

If all accounts are throttled, the decorator raises:

Exception("All accounts throttled")

Requirements

  • Python 3.6+
  • boto3
  • botocore

License

This project is licensed under the BSD-3-Clause License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Security Note

⚠️ Never commit AWS credentials to version control. Consider using environment variables, AWS credential files, or AWS IAM roles instead of hardcoding credentials.

About

A Python decorator that automatically rotates through multiple AWS accounts when making Amazon Bedrock API calls, providing automatic retry and load balancing across accounts.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages