diff --git a/calculating-cost.mdx b/calculating-cost.mdx index c358a521..a05c59b9 100644 --- a/calculating-cost.mdx +++ b/calculating-cost.mdx @@ -21,7 +21,7 @@ Deploying a model incurs two billable processes: on cold start (e.g., loading a model into GPU RAM, importing packages). This time is billed. - Function runtime: Code inside the request function, executed on every - request + request. **Example cost calculation** diff --git a/container-images/custom-dockerfiles.mdx b/container-images/custom-dockerfiles.mdx index 766dc66c..fc80ef49 100644 --- a/container-images/custom-dockerfiles.mdx +++ b/container-images/custom-dockerfiles.mdx @@ -3,7 +3,7 @@ title: "Custom Dockerfiles" description: "Run generic containerized applications on Cerebrium using your own custom Dockerfiles." --- -Cerebrium supports deploying existing containerized apps — from standard Python apps to compiled Rust binaries - using a custom Dockerfile. This allows portable, locally reproducible deployment environments. +Cerebrium supports deploying existing containerized apps — from standard Python apps to compiled Rust binaries — using a custom Dockerfile. This allows portable, locally reproducible deployment environments. ## Building Dockerized Python Apps diff --git a/migrations/replicate.mdx b/migrations/replicate.mdx index 8bfcd014..ec5d8a37 100644 --- a/migrations/replicate.mdx +++ b/migrations/replicate.mdx @@ -19,7 +19,7 @@ cerebrium init cog-migration-sdxl Cerebrium and Replicate both use a setup file: **cog.yaml** and **cerebrium.toml** for Replicate and Cerebrium respectively. -Based on the cog.yaml, add/change the following in `cerebrium.toml` +Based on the cog.yaml, add/change the following in `cerebrium.toml`: ```python [cerebrium.deployment] diff --git a/networking/custom-domains.mdx b/networking/custom-domains.mdx index 648644fa..b88bbafb 100644 --- a/networking/custom-domains.mdx +++ b/networking/custom-domains.mdx @@ -77,7 +77,7 @@ After creating the domain, DNS configuration instructions will be displayed. Cre 1. After configuring DNS, return to the Cerebrium dashboard 2. Cerebrium will automatically attempt to validate DNS records every 30 minutes for up to 2 days 3. To trigger an immediate validation attempt, click "Validation Status" -> "Validate Domain" (this works even after the 2-day window has elapsed) -4. If validation fails, the dialog will show the last known error. +4. If validation fails, the dialog will show the last known error 5. Once validated, SSL certificates will be automatically provisioned ### Step 4: Start Using the Custom Domain diff --git a/partner-services/deepgram.mdx b/partner-services/deepgram.mdx index 1d90b5eb..d98ccd7f 100644 --- a/partner-services/deepgram.mdx +++ b/partner-services/deepgram.mdx @@ -42,7 +42,7 @@ cerebrium init deepgram 2. Create a self-hosted API key from the Deepgram dashboard. Navigate to the **Secrets** tab in the Cerebrium dashboard and add the API key with the name `DEEPGRAM_API_KEY`. This secret automatically becomes available as an environment variable in the deployment. -3. Download model files from Deepgram's self-hosted section in the Deepgram dashboard using the guide available, [here](https://developers.deepgram.com/docs/deploy-deepgram-services#pull-deepgram-container-images). Select the 'license proxy' deployment type. Upload downloaded model files using the links provided by your Account Representative (with `.dg` extension) to persistent-storage in the `/deepgram-models` folder. This folder automatically attaches to the engine container. Use this command to upload the files: +3. Download model files from Deepgram's self-hosted section in the Deepgram dashboard using the guide available [here](https://developers.deepgram.com/docs/deploy-deepgram-services#pull-deepgram-container-images). Select the 'license proxy' deployment type. Upload downloaded model files using the links provided by your Account Representative (with `.dg` extension) to persistent-storage in the `/deepgram-models` folder. This folder automatically attaches to the engine container. Use this command to upload the files: ```bash cerebrium cp .dg deepgram-models/.dg diff --git a/v4/examples/comfyUI.mdx b/v4/examples/comfyUI.mdx index 14e65596..1c1ad790 100644 --- a/v4/examples/comfyUI.mdx +++ b/v4/examples/comfyUI.mdx @@ -31,7 +31,7 @@ The default ComfyUI workflow interface appears in this view. Use this locally ru ### Export ComfyUI Workflow -The example GitHub repository contains a workflow.json file. Click the “Load” button on the right to load the workflow. It should appear populated +The example GitHub repository contains a workflow.json file. Click the “Load” button on the right to load the workflow. It should appear populated. ![ComfyUI Workflow](/images/examples/comfyui1.png) @@ -41,7 +41,7 @@ To export the workflow in API format, click the gear icon (settings) in the top- ![Save ComfyUI API format](/images/examples/comfyui2.png) -A button appears in the right hover panel labeled “Save (API format)”. Use it to save the workflow as “workflow_api.json” +A button appears in the right hover panel labeled “Save (API format)”. Use it to save the workflow as “workflow_api.json”. ### ComfyUI Application diff --git a/v4/examples/deploy-a-vision-language-model-with-sglang.mdx b/v4/examples/deploy-a-vision-language-model-with-sglang.mdx index 336ac68f..4e765713 100644 --- a/v4/examples/deploy-a-vision-language-model-with-sglang.mdx +++ b/v4/examples/deploy-a-vision-language-model-with-sglang.mdx @@ -29,7 +29,7 @@ Here are some frontend primitives for creating multi-step workflows: ![SGLang Architecture](/images/sglang-arch.png) -Here is a summary of key advantages over traditional inference engines +Here is a summary of key advantages over traditional inference engines: | Feature | Traditional Engines (vLLM, TGI) | SGLang | | ----------------------- | ------------------------------------------------ | -------------------------------------------------------------------------- | @@ -40,7 +40,7 @@ Here is a summary of key advantages over traditional inference engines | **Performance** | Standard inference optimization | PyTorch-native with `torch.compile()`, quantization, sparse inference | For more details, see this [article](https://huggingface.co/blog/paresh2806/sglang-efficient-llm-workflows). -You can see the final code sample [here](https://github.com/CerebriumAI/examples/tree/master/5-large-language-models/7-vision-language-sglang) +You can see the final code sample [here](https://github.com/CerebriumAI/examples/tree/master/5-large-language-models/7-vision-language-sglang). ## Tutorial @@ -239,7 +239,7 @@ Once deployed, test with a sample request: curl -X POST "https://api.cerebrium.ai/v4/p-xxxxxxxx/7-vision-language-sglang/analyze" \ -H "Content-Type: application/json" \ -d '{ - "company_description": "Nike is a global leader in athletic footwear, apparel, and sports equipment known for its innovative designs and the iconic “swoosh” logo. The brand embodies performance, style, and inspiration, empowering athletes worldwide to Just Do It."", + "company_description": "Nike is a global leader in athletic footwear, apparel, and sports equipment known for its innovative designs and the iconic “swoosh” logo. The brand embodies performance, style, and inspiration, empowering athletes worldwide to Just Do It.", "image_base64": "", "dimensions": ["Effectiveness","Clarity", "Appeal","Credibility"] }' @@ -263,7 +263,3 @@ curl -X POST "https://api.cerebrium.ai/v4/p-xxxxxxxx/7-vision-language-sglang/an This example demonstrates how to leverage SGLang's structured generation capabilities to build an ad analysis system, using features like `fork()` for parallel processing and SGLang's built-in output control. You can find the complete code for this tutorial in our [examples repository](https://github.com/CerebriumAI/examples/tree/master/5-large-language-models/7-vision-language-sglang). - -``` - -``` diff --git a/v4/examples/gpt-oss.mdx b/v4/examples/gpt-oss.mdx index 3d535b01..b2212f97 100644 --- a/v4/examples/gpt-oss.mdx +++ b/v4/examples/gpt-oss.mdx @@ -3,7 +3,7 @@ title: "Serving GPT-OSS with vLLM" description: "Deploy OpenAI's Latest Open Source Model" --- -GPT recently released GPT-OSS ([gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) and [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)) two state-of-the-art open-weight language models that deliver strong real-world performance at low cost. Available under the flexible Apache 2.0 license, these models outperform similarly sized open models on reasoning tasks, demonstrate strong tool use capabilities, and are optimized for efficient deployment on consumer hardware. +GPT recently released GPT-OSS ([gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) and [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)), two state-of-the-art open-weight language models that deliver strong real-world performance at low cost. Available under the flexible Apache 2.0 license, these models outperform similarly sized open models on reasoning tasks, demonstrate strong tool use capabilities, and are optimized for efficient deployment on consumer hardware. ## What Makes GPT-OSS Special? @@ -53,7 +53,6 @@ max_replicas = 5 cooldown = 30 replica_concurrency = 32 scaling_metric = "concurrency_utilization" -scaling [cerebrium.runtime.custom] port = 8000 diff --git a/v4/examples/langchain-langsmith.mdx b/v4/examples/langchain-langsmith.mdx index 6e8fdbe7..2a7ce1d1 100644 --- a/v4/examples/langchain-langsmith.mdx +++ b/v4/examples/langchain-langsmith.mdx @@ -354,7 +354,7 @@ Set up LangSmith monitoring: 2. Create a free LangSmith account [here](https://smith.langchain.com/) 3. Generate API key (click gear icon in bottom left) -Set the following environment variables at the top of main.py. Add the API key to Cerebrium secrets +Set the following environment variables at the top of main.py. Add the API key to Cerebrium secrets. ```python import os @@ -379,7 +379,7 @@ After a successful test run, data populates in LangSmith: The Runs tab shows all runs (invocations/API requests). -In 1 above, the function name appears, with input set to the Cerebrium RunID (set to “test”. The input and total latency of the run are also visible. +In 1 above, the function name appears, with input set to the Cerebrium RunID (set to “test”). The input and total latency of the run are also visible. LangSmith supports various data automations: diff --git a/v4/examples/livekit-outbound-agent.mdx b/v4/examples/livekit-outbound-agent.mdx index 3a6c5b76..6cf38b31 100644 --- a/v4/examples/livekit-outbound-agent.mdx +++ b/v4/examples/livekit-outbound-agent.mdx @@ -125,14 +125,14 @@ Each service offers a generous free tier: - **Deepgram** -You can signup for a Deepgram account [here](https://www.notion.so/Livekit-Outbound-agent-13ebfd664fb180c5b02bf9c71f72b23d?pvs=21). Straight from the dashboard, you can create a API key. -Store this value for later +You can signup for a Deepgram account [here](https://www.notion.so/Livekit-Outbound-agent-13ebfd664fb180c5b02bf9c71f72b23d?pvs=21). Straight from the dashboard, you can create an API key. +Store this value for later. ![Deepgram Dashboard](/images/examples/livekit-agent/deepgram-dashboard.png) - **OpenAI** -You can signup for a OpenAI account [here](https://platform.openai.com/signup). You can then click “Dashboard” top right and then “API keys” in the left sidebar. +You can signup for an OpenAI account [here](https://platform.openai.com/signup). You can then click “Dashboard” top right and then “API keys” in the left sidebar. Create an API key and store this value to use later. ![OpenAI Dashboard](/images/examples/livekit-agent/openai-dashboard.png) @@ -226,7 +226,7 @@ async def entrypoint(ctx: JobContext): temperature=0.5, ), tts=cartesia.TTS(), - # intial ChatContext with system prompt + # initial ChatContext with system prompt chat_ctx=initial_ctx, # whether the agent can be interrupted allow_interruptions=True, diff --git a/v4/examples/realtime-voice-agents.mdx b/v4/examples/realtime-voice-agents.mdx index 2e1a9380..a1026c1a 100644 --- a/v4/examples/realtime-voice-agents.mdx +++ b/v4/examples/realtime-voice-agents.mdx @@ -464,7 +464,7 @@ The endpoints are used in the frontend interface below. A public fork of the PipeCat frontend demonstrates this application. Clone the repo [here](https://github.com/CerebriumAI/web-client-ui). -Follow the instructions in the README.md and then populate the following variables in your .env.development.local +Follow the instructions in the README.md and then populate the following variables in your .env.development.local. ``` VITE_SERVER_URL=https://api.cerebrium.ai/v4/p-xxxxxxxx/ #This is the base url of your pipecat-agent. Do not include the function names