Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion calculating-cost.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Deploying a model incurs two billable processes:
on cold start (e.g., loading a model into GPU RAM, importing packages). This
time is billed.
- <u>Function runtime</u>: Code inside the request function, executed on every
request
request.

**Example cost calculation**

Expand Down
2 changes: 1 addition & 1 deletion container-images/custom-dockerfiles.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Custom Dockerfiles"
description: "Run generic containerized applications on Cerebrium using your own custom Dockerfiles."
---

Cerebrium supports deploying existing containerized apps — from standard Python apps to compiled Rust binaries - using a custom Dockerfile. This allows portable, locally reproducible deployment environments.
Cerebrium supports deploying existing containerized apps — from standard Python apps to compiled Rust binaries using a custom Dockerfile. This allows portable, locally reproducible deployment environments.

## Building Dockerized Python Apps

Expand Down
2 changes: 1 addition & 1 deletion migrations/replicate.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ cerebrium init cog-migration-sdxl

Cerebrium and Replicate both use a setup file: **cog.yaml** and **cerebrium.toml** for Replicate and Cerebrium respectively.

Based on the cog.yaml, add/change the following in `cerebrium.toml`
Based on the cog.yaml, add/change the following in `cerebrium.toml`:

```python
[cerebrium.deployment]
Expand Down
2 changes: 1 addition & 1 deletion networking/custom-domains.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ After creating the domain, DNS configuration instructions will be displayed. Cre
1. After configuring DNS, return to the Cerebrium dashboard
2. Cerebrium will automatically attempt to validate DNS records every 30 minutes for up to 2 days
3. To trigger an immediate validation attempt, click "Validation Status" -> "Validate Domain" (this works even after the 2-day window has elapsed)
4. If validation fails, the dialog will show the last known error.
4. If validation fails, the dialog will show the last known error
5. Once validated, SSL certificates will be automatically provisioned

### Step 4: Start Using the Custom Domain
Expand Down
2 changes: 1 addition & 1 deletion partner-services/deepgram.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ cerebrium init deepgram

2. Create a self-hosted API key from the <b>Deepgram</b> dashboard. Navigate to the **Secrets** tab in the <b>Cerebrium</b> dashboard and add the API key with the name `DEEPGRAM_API_KEY`. This secret automatically becomes available as an environment variable in the deployment.

3. Download model files from Deepgram's self-hosted section in the <b>Deepgram</b> dashboard using the guide available, [here](https://developers.deepgram.com/docs/deploy-deepgram-services#pull-deepgram-container-images). Select the 'license proxy' deployment type. Upload downloaded model files using the links provided by your Account Representative (with `.dg` extension) to persistent-storage in the `/deepgram-models` folder. This folder automatically attaches to the engine container. Use this command to upload the files:
3. Download model files from Deepgram's self-hosted section in the <b>Deepgram</b> dashboard using the guide available [here](https://developers.deepgram.com/docs/deploy-deepgram-services#pull-deepgram-container-images). Select the 'license proxy' deployment type. Upload downloaded model files using the links provided by your Account Representative (with `.dg` extension) to persistent-storage in the `/deepgram-models` folder. This folder automatically attaches to the engine container. Use this command to upload the files:

```bash
cerebrium cp <model>.dg deepgram-models/<model>.dg
Expand Down
4 changes: 2 additions & 2 deletions v4/examples/comfyUI.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The default ComfyUI workflow interface appears in this view. Use this locally ru

### Export ComfyUI Workflow

The example GitHub repository contains a workflow.json file. Click the “Load” button on the right to load the workflow. It should appear populated
The example GitHub repository contains a workflow.json file. Click the “Load” button on the right to load the workflow. It should appear populated.

![ComfyUI Workflow](/images/examples/comfyui1.png)

Expand All @@ -41,7 +41,7 @@ To export the workflow in API format, click the gear icon (settings) in the top-

![Save ComfyUI API format](/images/examples/comfyui2.png)

A button appears in the right hover panel labeled “Save (API format)”. Use it to save the workflow as “workflow_api.json”
A button appears in the right hover panel labeled “Save (API format)”. Use it to save the workflow as “workflow_api.json”.

### ComfyUI Application

Expand Down
10 changes: 3 additions & 7 deletions v4/examples/deploy-a-vision-language-model-with-sglang.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Here are some frontend primitives for creating multi-step workflows:

![SGLang Architecture](/images/sglang-arch.png)

Here is a summary of key advantages over traditional inference engines
Here is a summary of key advantages over traditional inference engines:

| Feature | Traditional Engines (vLLM, TGI) | SGLang |
| ----------------------- | ------------------------------------------------ | -------------------------------------------------------------------------- |
Expand All @@ -40,7 +40,7 @@ Here is a summary of key advantages over traditional inference engines
| **Performance** | Standard inference optimization | PyTorch-native with `torch.compile()`, quantization, sparse inference |

For more details, see this [article](https://huggingface.co/blog/paresh2806/sglang-efficient-llm-workflows).
You can see the final code sample [here](https://github.com/CerebriumAI/examples/tree/master/5-large-language-models/7-vision-language-sglang)
You can see the final code sample [here](https://github.com/CerebriumAI/examples/tree/master/5-large-language-models/7-vision-language-sglang).

## Tutorial

Expand Down Expand Up @@ -239,7 +239,7 @@ Once deployed, test with a sample request:
curl -X POST "https://api.cerebrium.ai/v4/p-xxxxxxxx/7-vision-language-sglang/analyze" \
-H "Content-Type: application/json" \
-d '{
"company_description": "Nike is a global leader in athletic footwear, apparel, and sports equipment known for its innovative designs and the iconic “swoosh” logo. The brand embodies performance, style, and inspiration, empowering athletes worldwide to Just Do It."",
"company_description": "Nike is a global leader in athletic footwear, apparel, and sports equipment known for its innovative designs and the iconic “swoosh” logo. The brand embodies performance, style, and inspiration, empowering athletes worldwide to Just Do It.",
"image_base64": "<BASE64_ENCODED_IMAGE>",
"dimensions": ["Effectiveness","Clarity", "Appeal","Credibility"]
}'
Expand All @@ -263,7 +263,3 @@ curl -X POST "https://api.cerebrium.ai/v4/p-xxxxxxxx/7-vision-language-sglang/an
This example demonstrates how to leverage SGLang's structured generation capabilities to build an ad analysis system, using features like `fork()` for parallel processing and SGLang's built-in output control.

You can find the complete code for this tutorial in our [examples repository](https://github.com/CerebriumAI/examples/tree/master/5-large-language-models/7-vision-language-sglang).

```

```
3 changes: 1 addition & 2 deletions v4/examples/gpt-oss.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Serving GPT-OSS with vLLM"
description: "Deploy OpenAI's Latest Open Source Model"
---

GPT recently released GPT-OSS ([gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) and [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)) two state-of-the-art open-weight language models that deliver strong real-world performance at low cost. Available under the flexible Apache 2.0 license, these models outperform similarly sized open models on reasoning tasks, demonstrate strong tool use capabilities, and are optimized for efficient deployment on consumer hardware.
GPT recently released GPT-OSS ([gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) and [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)), two state-of-the-art open-weight language models that deliver strong real-world performance at low cost. Available under the flexible Apache 2.0 license, these models outperform similarly sized open models on reasoning tasks, demonstrate strong tool use capabilities, and are optimized for efficient deployment on consumer hardware.

## What Makes GPT-OSS Special?

Expand Down Expand Up @@ -53,7 +53,6 @@ max_replicas = 5
cooldown = 30
replica_concurrency = 32
scaling_metric = "concurrency_utilization"
scaling

[cerebrium.runtime.custom]
port = 8000
Expand Down
4 changes: 2 additions & 2 deletions v4/examples/langchain-langsmith.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@ Set up LangSmith monitoring:
2. Create a free LangSmith account [here](https://smith.langchain.com/)
3. Generate API key (click gear icon in bottom left)

Set the following environment variables at the top of main.py. Add the API key to Cerebrium secrets
Set the following environment variables at the top of main.py. Add the API key to Cerebrium secrets.

```python
import os
Expand All @@ -379,7 +379,7 @@ After a successful test run, data populates in LangSmith:

The Runs tab shows all runs (invocations/API requests).

In 1 above, the function name appears, with input set to the Cerebrium RunID (set to “test”. The input and total latency of the run are also visible.
In 1 above, the function name appears, with input set to the Cerebrium RunID (set to “test”). The input and total latency of the run are also visible.

LangSmith supports various data automations:

Expand Down
8 changes: 4 additions & 4 deletions v4/examples/livekit-outbound-agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -125,14 +125,14 @@ Each service offers a generous free tier:

- **Deepgram**

You can signup for a Deepgram account [here](https://www.notion.so/Livekit-Outbound-agent-13ebfd664fb180c5b02bf9c71f72b23d?pvs=21). Straight from the dashboard, you can create a API key.
Store this value for later
You can signup for a Deepgram account [here](https://www.notion.so/Livekit-Outbound-agent-13ebfd664fb180c5b02bf9c71f72b23d?pvs=21). Straight from the dashboard, you can create an API key.
Store this value for later.

![Deepgram Dashboard](/images/examples/livekit-agent/deepgram-dashboard.png)

- **OpenAI**

You can signup for a OpenAI account [here](https://platform.openai.com/signup). You can then click “Dashboard” top right and then “API keys” in the left sidebar.
You can signup for an OpenAI account [here](https://platform.openai.com/signup). You can then click “Dashboard” top right and then “API keys” in the left sidebar.
Create an API key and store this value to use later.

![OpenAI Dashboard](/images/examples/livekit-agent/openai-dashboard.png)
Expand Down Expand Up @@ -226,7 +226,7 @@ async def entrypoint(ctx: JobContext):
temperature=0.5,
),
tts=cartesia.TTS(),
# intial ChatContext with system prompt
# initial ChatContext with system prompt
chat_ctx=initial_ctx,
# whether the agent can be interrupted
allow_interruptions=True,
Expand Down
2 changes: 1 addition & 1 deletion v4/examples/realtime-voice-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -464,7 +464,7 @@ The endpoints are used in the frontend interface below.

A public fork of the PipeCat frontend demonstrates this application. Clone the repo [here](https://github.com/CerebriumAI/web-client-ui).

Follow the instructions in the README.md and then populate the following variables in your .env.development.local
Follow the instructions in the README.md and then populate the following variables in your .env.development.local.

```
VITE_SERVER_URL=https://api.cerebrium.ai/v4/p-xxxxxxxx/<APP_NAME> #This is the base url of your pipecat-agent. Do not include the function names
Expand Down
Loading