Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions container-images/custom-dockerfiles.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8192

Dockerfiles for Cerebrium have three requirements:

1. Expose a port with the `EXPOSE` command. This port is referenced in `cerebrium.toml`
2. Include a `CMD` command to specify the container's startup process (typically the server)
3. Set the working directory with `WORKDIR` to ensure correct file paths (defaults to root if not specified)
1. Expose a port with the `EXPOSE` command. This port is referenced in `cerebrium.toml`.
2. Include a `CMD` command to specify the container's startup process (typically the server).
3. Set the working directory with `WORKDIR` to ensure correct file paths (defaults to root if not specified).

Update cerebrium.toml to include a custom runtime section with the `dockerfile_path` parameter:

Expand Down
4 changes: 2 additions & 2 deletions container-images/custom-web-servers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ fastapi = "latest"

The configuration requires four key parameters:

- `entrypoint`: The command that starts your server
- `port`: The port your server listens on
- `entrypoint`: The command that starts your server.
- `port`: The port your server listens on.
- `healthcheck_endpoint`: The endpoint used to confirm instance health. If unspecified, defaults to a TCP ping on the configured port. If the health check registers a non-200 response, it will be considered _unhealthy_, and be restarted should it not recover timely.
- `readycheck_endpoint`: The endpoint used to confirm if the instance is ready to receive. If unspecified, defaults to a TCP ping on the configured port. If the ready check registers a non-200 response, it will not be a viable target for request routing.

Expand Down
4 changes: 2 additions & 2 deletions networking/custom-domains.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Once configured, API calls use the custom domain while keeping the same path str

## How Custom Domains Work

- **Domains are region-specific** meaning each will always resolve to the selected region
- **Domains are app-agnostic** enabling connection to any number of deployed apps within a project via custom domain
- **Domains are region-specific**, meaning each will always resolve to the selected region
- **Domains are app-agnostic**, enabling connection to any number of deployed apps within a project via custom domain
- **Multiple domains** can be configured on the same project (useful for apps within the project deployed in different regions)

## Getting Started
Expand Down
6 changes: 3 additions & 3 deletions partner-services/deepgram.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Deepgram
description: Run self hosted Deepgram speech to text on Cerebrium with model file uploads, engine and api TOML setup, GPU scaling, and low latency voice agents.
description: Run self-hosted Deepgram speech-to-text on Cerebrium with model file uploads, engine and api TOML setup, GPU scaling, and low latency voice agents.
---

Cerebrium's partnership with [Deepgram](https://www.deepgram.com/) enables simple deployment of speech-to-text (STT) services with simplified configuration and independent scaling.
Expand All @@ -15,7 +15,7 @@ Consult the Deepgram representative on how to achieve parity with the Deepgram A
</Note>

<Note>
Deepgram Partner Service is available from CLI version 1.39.0 and greater
Deepgram Partner Service is available from CLI version 1.39.0 and greater.
</Note>

<Note>
Expand Down Expand Up @@ -316,7 +316,7 @@ replica_concurrency = 150

7. Run `cerebrium deploy`. After deployment, an endpoint for the Deepgram services appears in the terminal output. The URL is also available on the app's overview page in the dashboard.

8. Download an example audio file for use with the deepgram service:
8. Download an example audio file for use with the Deepgram service:

```bash
wget https://dpgr.am/bueller.wav
Expand Down
7 changes: 3 additions & 4 deletions v4/examples/comfyUI.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ description: Turn ComfyUI stable diffusion workflows into autoscaling API endpoi
ComfyUI is a popular no-code interface for building complex stable diffusion workflows. Its modular setup and intuitive flowchart interface have produced an extensive collection of community workflows. Several websites offer shared workflows:

- https://comfyworkflows.com/
- https://openart.ai/workflows/home‍
- https://openart.ai/workflows/home

Production-scale deployment guidance for ComfyUI is limited. This tutorial covers deploying ComfyUI pipelines on Cerebrium as autoscaling API endpoints with pay-as-you-go compute. Find the full example code [here](https://github.com/CerebriumAI/examples/tree/master/7-image-and-video/1-comfyui).

Expand All @@ -23,9 +23,8 @@ This tutorial uses Stable Diffusion XL and ControlNet to create custom QR codes.
3. Download the following models and install them in the appropriate folders within the ComfyUI folder:
- SDXL base in models/checkpoints.
- ControlNet in models/ControlNet.
4. Run ComfyUI locally from inside the cloned ComfyUI folder: python main.py --force-fp16 on MacOS.
5. A server should be loaded locally at http://127.0.0.1:8188/‍
4. Run ComfyUI locally from inside the cloned ComfyUI folder: python main.py --force-fp16 on macOS.
5. A server should be loaded locally at http://127.0.0.1:8188/

The default ComfyUI workflow interface appears in this view. Use this locally running instance to build the image generation pipeline.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This tutorial deploys a Vision Language Model (VLM) using SGLang on Cerebrium. A

The example builds an intelligent ad analysis system that evaluates advertisements across multiple dimensions, scoring how the advertisement relates to the business in question and how it performs on the given criteria.

SGLang (Structured Generation Language) differs from other inference frameworks such as vLLM and TensorRT by focusing on structured generation and complex multi-step LLM workflows. Teams at xAI and Deepseek use SGLang in production to power their core language model capabilities, making it a trusted choice.
SGLang (Structured Generation Language) differs from other inference frameworks such as vLLM and TensorRT by focusing on structured generation and complex multi-step LLM workflows. Teams at xAI and DeepSeek use SGLang in production to power their core language model capabilities, making it a trusted choice.

### SGLang Architecture

Expand Down
4 changes: 1 addition & 3 deletions v4/examples/langchain-langsmith.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Build an executive assistant agent with LangChain tool calling, mon

This tutorial builds Cal-vin, an executive assistant that manages calendar appointments (via Cal.com) with employees, customers, partners, and friends. It uses the LangChain SDK for agent creation and the LangSmith platform for monitoring scheduling activities and identifying failure points, deployed on Cerebrium for seamless scaling.

You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/4-integrations/2-tool-calling-langsmith)
You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/4-integrations/2-tool-calling-langsmith).

### Concepts

Expand Down Expand Up @@ -212,7 +212,6 @@ The code above:
1. Uses `@tool` decorator to identify functions as LangChain tools
2. Includes docstrings explaining functionality and required inputs
3. Uses `find_available_slots` helper function to format Cal.com API responses into readable time slots

The book_slot tool follows a similar pattern. It books a slot based on the selected time/day. Get the eventTypeId from the dashboard by selecting an event and grabbing the ID from the URL.

Expand Down Expand Up @@ -336,7 +335,6 @@ This code:

- Defines a Pydantic object specifying the expected API parameters: user prompt and session ID.
- The predict function (Cerebrium’s API entry point) passes the prompt and session ID to the agent and returns results.

Install pip dependencies locally: `pip install pydantic langchain pytz openai langchain_openai langchain-community`, then run `python main.py`. Replace secrets with actual values when running locally. Output looks similar to:

Expand Down
10 changes: 5 additions & 5 deletions v4/examples/livekit-outbound-agent.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Outbound Agent with LiveKit"
description: Build an outbound AI voice agent with LiveKit and Twilio SIP trunking on Cerebrium that makes calls and warm transfers callers to human agents
description: Build an outbound AI voice agent with LiveKit and Twilio SIP trunking on Cerebrium that makes calls and warm transfers callers to human agents.
---

Voice agents are transforming business operations by introducing efficiencies and personalization for each customer interaction. While most use cases focus on agents receiving calls, this tutorial covers outbound voice AI agents and the use cases they unlock.
Expand Down Expand Up @@ -64,14 +64,14 @@ Setting up an outbound calling agent requires a SIP trunk in Twilio. A SIP trunk
1. **Log in to your Twilio Console**:
Go to [Twilio Console](https://console.twilio.com/) and log in with your credentials. If you don’t already have an account, create one.
2. **Add Phone Numbers**:
Under the “Develop” tab is the "Phone numbers" section. Navigate to “Active numbers” and purchase a number if you don’t already have one. Toll free numbers won't work for our use case, so ensure that a "Local" number exists.
Under the “Develop” tab is the "Phone numbers" section. Navigate to “Active numbers” and purchase a number if you don’t already have one. Toll-free numbers won't work for our use case, so ensure that a "Local" number exists.
3. **Install the Twilio CLI and authenticate your CLI:**
```
brew tap twilio/brew && brew install twilio
twilio login
```
4. **Create a SIP trunk:**
The domain name for your SIP trunk must end in [pstn.twilio.com](http://pstn.twilio.com/). For example to create a trunk named My test trunk with the
The domain name for your SIP trunk must end in [pstn.twilio.com](http://pstn.twilio.com/). For example, to create a trunk named My test trunk with the
domain name [my-test-trunk.pstn.twilio.com](http://my-test-trunk.pstn.twilio.com/), run the following command:
```
twilio api trunking v1 trunks create \
Expand Down Expand Up @@ -180,7 +180,7 @@ Create a file called `main.py` for the agent code. The directory structure shoul
- requirements.txt
- outbound-trunk.json

Add the following code to your main.py
Add the following code to your main.py:

```
from fastapi import FastAPI
Expand Down Expand Up @@ -339,7 +339,7 @@ if __name__ == '__main__':

To test locally, run `python main.py` in one terminal. This keeps the agents running as an open process:

![LiveKit processes ](/images/examples/livekit-agent/livekit-processes.png)
![LiveKit processes](/images/examples/livekit-agent/livekit-processes.png)

Once the job processes initialize, open a separate terminal and run `python test.py`. This initiates a call to the provided phone number. Note: calls are limited to the region the number is purchased from.

Expand Down
6 changes: 3 additions & 3 deletions v4/examples/realtime-voice-agents.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Real-time Voice Agent"
sidebarTitle: "500ms Low-latency Voice Agent"
description: Build a low latency voice AI agent with PipeCat, Deepgram and a self hosted vLLM Llama endpoint on Cerebrium that responds in about 500ms
description: Build a low latency voice AI agent with PipeCat, Deepgram and a self-hosted vLLM Llama endpoint on Cerebrium that responds in about 500ms.
---

This tutorial creates a real-time voice agent that responds to queries via speech in ~500ms. The implementation supports swapping in any Large Language Model (LLM) or Text-to-Speech (TTS) model, making it ideal for voice-based use cases like customer support bots and receptionists.
Expand All @@ -18,7 +18,7 @@ Low latency is achieved because each service is hosted within Cerebrium. Communi

![Realtime Voice Agents](/images/examples/self-hosted-pipecat.png)

You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/6-voice/2-realtime-voice-agent)
You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/6-voice/2-realtime-voice-agent).

Create a Cerebrium account by signing up [here](https://dashboard.cerebrium.ai/register) and follow the [installation docs](https://docs.cerebrium.ai/getting-started/installation).

Expand Down Expand Up @@ -423,7 +423,7 @@ def create_token(room_name: str):

Summary of the code above:

- WebRTC functionality from Daily creates the room (swappable for Twilio/Telenyx). Two functions handle room creation and authentication: `create_room()` and `create_token()`.
- WebRTC functionality from Daily creates the room (swappable for Twilio/Telnyx). Two functions handle room creation and authentication: `create_room()` and `create_token()`.
- The Deepgram and LLM services use a local URL to connect within the Cerebrium cluster. Edit the project key in the URL as needed.
- TTS uses the Cartesia service to demonstrate Pipecat's versatility, but the Deepgram TTS service works as well.

Expand Down
Loading