diff --git a/container-images/custom-dockerfiles.mdx b/container-images/custom-dockerfiles.mdx index ceb49d52..121c6316 100644 --- a/container-images/custom-dockerfiles.mdx +++ b/container-images/custom-dockerfiles.mdx @@ -48,9 +48,9 @@ CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8192 Dockerfiles for Cerebrium have three requirements: -1. Expose a port with the `EXPOSE` command. This port is referenced in `cerebrium.toml` -2. Include a `CMD` command to specify the container's startup process (typically the server) -3. Set the working directory with `WORKDIR` to ensure correct file paths (defaults to root if not specified) +1. Expose a port with the `EXPOSE` command. This port is referenced in `cerebrium.toml`. +2. Include a `CMD` command to specify the container's startup process (typically the server). +3. Set the working directory with `WORKDIR` to ensure correct file paths (defaults to root if not specified). Update cerebrium.toml to include a custom runtime section with the `dockerfile_path` parameter: diff --git a/container-images/custom-web-servers.mdx b/container-images/custom-web-servers.mdx index 83457301..f7631157 100644 --- a/container-images/custom-web-servers.mdx +++ b/container-images/custom-web-servers.mdx @@ -44,8 +44,8 @@ fastapi = "latest" The configuration requires four key parameters: -- `entrypoint`: The command that starts your server -- `port`: The port your server listens on +- `entrypoint`: The command that starts your server. +- `port`: The port your server listens on. - `healthcheck_endpoint`: The endpoint used to confirm instance health. If unspecified, defaults to a TCP ping on the configured port. If the health check registers a non-200 response, it will be considered _unhealthy_, and be restarted should it not recover timely. - `readycheck_endpoint`: The endpoint used to confirm if the instance is ready to receive. If unspecified, defaults to a TCP ping on the configured port. If the ready check registers a non-200 response, it will not be a viable target for request routing. diff --git a/networking/custom-domains.mdx b/networking/custom-domains.mdx index 42f1a43a..0f3679d4 100644 --- a/networking/custom-domains.mdx +++ b/networking/custom-domains.mdx @@ -16,8 +16,8 @@ Once configured, API calls use the custom domain while keeping the same path str ## How Custom Domains Work -- **Domains are region-specific** meaning each will always resolve to the selected region -- **Domains are app-agnostic** enabling connection to any number of deployed apps within a project via custom domain +- **Domains are region-specific**, meaning each will always resolve to the selected region +- **Domains are app-agnostic**, enabling connection to any number of deployed apps within a project via custom domain - **Multiple domains** can be configured on the same project (useful for apps within the project deployed in different regions) ## Getting Started diff --git a/partner-services/deepgram.mdx b/partner-services/deepgram.mdx index 9d3e7cd5..bf43940a 100644 --- a/partner-services/deepgram.mdx +++ b/partner-services/deepgram.mdx @@ -1,6 +1,6 @@ --- title: Deepgram -description: Run self hosted Deepgram speech to text on Cerebrium with model file uploads, engine and api TOML setup, GPU scaling, and low latency voice agents. +description: Run self-hosted Deepgram speech-to-text on Cerebrium with model file uploads, engine and api TOML setup, GPU scaling, and low latency voice agents. --- Cerebrium's partnership with [Deepgram](https://www.deepgram.com/) enables simple deployment of speech-to-text (STT) services with simplified configuration and independent scaling. @@ -15,7 +15,7 @@ Consult the Deepgram representative on how to achieve parity with the Deepgram A - Deepgram Partner Service is available from CLI version 1.39.0 and greater + Deepgram Partner Service is available from CLI version 1.39.0 and greater. @@ -316,7 +316,7 @@ replica_concurrency = 150 7. Run `cerebrium deploy`. After deployment, an endpoint for the Deepgram services appears in the terminal output. The URL is also available on the app's overview page in the dashboard. -8. Download an example audio file for use with the deepgram service: +8. Download an example audio file for use with the Deepgram service: ```bash wget https://dpgr.am/bueller.wav diff --git a/v4/examples/comfyUI.mdx b/v4/examples/comfyUI.mdx index 310b1f3a..de56c442 100644 --- a/v4/examples/comfyUI.mdx +++ b/v4/examples/comfyUI.mdx @@ -8,7 +8,7 @@ description: Turn ComfyUI stable diffusion workflows into autoscaling API endpoi ComfyUI is a popular no-code interface for building complex stable diffusion workflows. Its modular setup and intuitive flowchart interface have produced an extensive collection of community workflows. Several websites offer shared workflows: - https://comfyworkflows.com/ -- https://openart.ai/workflows/home‍ +- https://openart.ai/workflows/home Production-scale deployment guidance for ComfyUI is limited. This tutorial covers deploying ComfyUI pipelines on Cerebrium as autoscaling API endpoints with pay-as-you-go compute. Find the full example code [here](https://github.com/CerebriumAI/examples/tree/master/7-image-and-video/1-comfyui). @@ -23,9 +23,8 @@ This tutorial uses Stable Diffusion XL and ControlNet to create custom QR codes. 3. Download the following models and install them in the appropriate folders within the ComfyUI folder: - SDXL base in models/checkpoints. - ControlNet in models/ControlNet. -4. Run ComfyUI locally from inside the cloned ComfyUI folder: python main.py --force-fp16 on MacOS. -5. A server should be loaded locally at http://127.0.0.1:8188/‍ - ‍ +4. Run ComfyUI locally from inside the cloned ComfyUI folder: python main.py --force-fp16 on macOS. +5. A server should be loaded locally at http://127.0.0.1:8188/ The default ComfyUI workflow interface appears in this view. Use this locally running instance to build the image generation pipeline. diff --git a/v4/examples/deploy-a-vision-language-model-with-sglang.mdx b/v4/examples/deploy-a-vision-language-model-with-sglang.mdx index 732ea24b..80060988 100644 --- a/v4/examples/deploy-a-vision-language-model-with-sglang.mdx +++ b/v4/examples/deploy-a-vision-language-model-with-sglang.mdx @@ -7,7 +7,7 @@ This tutorial deploys a Vision Language Model (VLM) using SGLang on Cerebrium. A The example builds an intelligent ad analysis system that evaluates advertisements across multiple dimensions, scoring how the advertisement relates to the business in question and how it performs on the given criteria. -SGLang (Structured Generation Language) differs from other inference frameworks such as vLLM and TensorRT by focusing on structured generation and complex multi-step LLM workflows. Teams at xAI and Deepseek use SGLang in production to power their core language model capabilities, making it a trusted choice. +SGLang (Structured Generation Language) differs from other inference frameworks such as vLLM and TensorRT by focusing on structured generation and complex multi-step LLM workflows. Teams at xAI and DeepSeek use SGLang in production to power their core language model capabilities, making it a trusted choice. ### SGLang Architecture diff --git a/v4/examples/langchain-langsmith.mdx b/v4/examples/langchain-langsmith.mdx index 7b249606..0ef00953 100644 --- a/v4/examples/langchain-langsmith.mdx +++ b/v4/examples/langchain-langsmith.mdx @@ -5,7 +5,7 @@ description: Build an executive assistant agent with LangChain tool calling, mon This tutorial builds Cal-vin, an executive assistant that manages calendar appointments (via Cal.com) with employees, customers, partners, and friends. It uses the LangChain SDK for agent creation and the LangSmith platform for monitoring scheduling activities and identifying failure points, deployed on Cerebrium for seamless scaling. -You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/4-integrations/2-tool-calling-langsmith) +You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/4-integrations/2-tool-calling-langsmith). ### Concepts @@ -212,7 +212,6 @@ The code above: 1. Uses `@tool` decorator to identify functions as LangChain tools 2. Includes docstrings explaining functionality and required inputs 3. Uses `find_available_slots` helper function to format Cal.com API responses into readable time slots - ‍ The book_slot tool follows a similar pattern. It books a slot based on the selected time/day. Get the eventTypeId from the dashboard by selecting an event and grabbing the ID from the URL. @@ -336,7 +335,6 @@ This code: - Defines a Pydantic object specifying the expected API parameters: user prompt and session ID. - The predict function (Cerebrium’s API entry point) passes the prompt and session ID to the agent and returns results. - ‍ Install pip dependencies locally: `pip install pydantic langchain pytz openai langchain_openai langchain-community`, then run `python main.py`. Replace secrets with actual values when running locally. Output looks similar to: diff --git a/v4/examples/livekit-outbound-agent.mdx b/v4/examples/livekit-outbound-agent.mdx index d5c143f6..5d09b5bd 100644 --- a/v4/examples/livekit-outbound-agent.mdx +++ b/v4/examples/livekit-outbound-agent.mdx @@ -1,6 +1,6 @@ --- title: "Outbound Agent with LiveKit" -description: Build an outbound AI voice agent with LiveKit and Twilio SIP trunking on Cerebrium that makes calls and warm transfers callers to human agents +description: Build an outbound AI voice agent with LiveKit and Twilio SIP trunking on Cerebrium that makes calls and warm transfers callers to human agents. --- Voice agents are transforming business operations by introducing efficiencies and personalization for each customer interaction. While most use cases focus on agents receiving calls, this tutorial covers outbound voice AI agents and the use cases they unlock. @@ -64,14 +64,14 @@ Setting up an outbound calling agent requires a SIP trunk in Twilio. A SIP trunk 1. **Log in to your Twilio Console**: Go to [Twilio Console](https://console.twilio.com/) and log in with your credentials. If you don’t already have an account, create one. 2. **Add Phone Numbers**: - Under the “Develop” tab is the "Phone numbers" section. Navigate to “Active numbers” and purchase a number if you don’t already have one. Toll free numbers won't work for our use case, so ensure that a "Local" number exists. + Under the “Develop” tab is the "Phone numbers" section. Navigate to “Active numbers” and purchase a number if you don’t already have one. Toll-free numbers won't work for our use case, so ensure that a "Local" number exists. 3. **Install the Twilio CLI and authenticate your CLI:** ``` brew tap twilio/brew && brew install twilio twilio login ``` 4. **Create a SIP trunk:** - The domain name for your SIP trunk must end in [pstn.twilio.com](http://pstn.twilio.com/). For example to create a trunk named My test trunk with the + The domain name for your SIP trunk must end in [pstn.twilio.com](http://pstn.twilio.com/). For example, to create a trunk named My test trunk with the domain name [my-test-trunk.pstn.twilio.com](http://my-test-trunk.pstn.twilio.com/), run the following command: ``` twilio api trunking v1 trunks create \ @@ -180,7 +180,7 @@ Create a file called `main.py` for the agent code. The directory structure shoul - requirements.txt - outbound-trunk.json -Add the following code to your main.py +Add the following code to your main.py: ``` from fastapi import FastAPI @@ -339,7 +339,7 @@ if __name__ == '__main__': To test locally, run `python main.py` in one terminal. This keeps the agents running as an open process: -![LiveKit processes ](/images/examples/livekit-agent/livekit-processes.png) +![LiveKit processes](/images/examples/livekit-agent/livekit-processes.png) Once the job processes initialize, open a separate terminal and run `python test.py`. This initiates a call to the provided phone number. Note: calls are limited to the region the number is purchased from. diff --git a/v4/examples/realtime-voice-agents.mdx b/v4/examples/realtime-voice-agents.mdx index d86fcf11..c984711f 100644 --- a/v4/examples/realtime-voice-agents.mdx +++ b/v4/examples/realtime-voice-agents.mdx @@ -1,7 +1,7 @@ --- title: "Real-time Voice Agent" sidebarTitle: "500ms Low-latency Voice Agent" -description: Build a low latency voice AI agent with PipeCat, Deepgram and a self hosted vLLM Llama endpoint on Cerebrium that responds in about 500ms +description: Build a low latency voice AI agent with PipeCat, Deepgram and a self-hosted vLLM Llama endpoint on Cerebrium that responds in about 500ms. --- This tutorial creates a real-time voice agent that responds to queries via speech in ~500ms. The implementation supports swapping in any Large Language Model (LLM) or Text-to-Speech (TTS) model, making it ideal for voice-based use cases like customer support bots and receptionists. @@ -18,7 +18,7 @@ Low latency is achieved because each service is hosted within Cerebrium. Communi ![Realtime Voice Agents](/images/examples/self-hosted-pipecat.png) -You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/6-voice/2-realtime-voice-agent) +You can find the final version of the code [here](https://github.com/CerebriumAI/examples/tree/master/6-voice/2-realtime-voice-agent). Create a Cerebrium account by signing up [here](https://dashboard.cerebrium.ai/register) and follow the [installation docs](https://docs.cerebrium.ai/getting-started/installation). @@ -423,7 +423,7 @@ def create_token(room_name: str): Summary of the code above: -- WebRTC functionality from Daily creates the room (swappable for Twilio/Telenyx). Two functions handle room creation and authentication: `create_room()` and `create_token()`. +- WebRTC functionality from Daily creates the room (swappable for Twilio/Telnyx). Two functions handle room creation and authentication: `create_room()` and `create_token()`. - The Deepgram and LLM services use a local URL to connect within the Cerebrium cluster. Edit the project key in the URL as needed. - TTS uses the Cartesia service to demonstrate Pipecat's versatility, but the Deepgram TTS service works as well.