@@ -206,7 +206,7 @@ align: rm-lm
206206
207207:: title ::
208208
209- # Hands on
209+ # Hands- on
210210
211211# <mdi-arrow-right />
212212
@@ -217,87 +217,101 @@ Head over to app.portkey.ai and choose the "Login with SSO" option with your Net
217217
218218---
219219layout: top-title-two-cols
220- columns: is-3-9
220+ columns: is-6
221221color: violet-light
222222---
223223
224+
224225:: title ::
225226
226- # Interacting with LLM APIs using cURL
227+ # LLM API servers
227228
228229:: left ::
230+ - LLMs are programmatically accessible via an HTTP server, typically via the OpenAI API
231+ - To invoke the LLM, you send an HTTP request with
232+ - your ` API_KEY `
233+ - message body, i.e. prompt and any conversation history
234+ - parameters like maximum tokens, temperature, thinking/reasoning level, etc.
229235
230- ## API Request Flow
236+ :: right ::
231237
232- ``` mermaid {theme: 'dark', scale: 0.5 }
238+ ``` mermaid {scale: 0.75 }
233239sequenceDiagram
234- participant C as Client (cURL)
235- participant API as LLM API Server
236- participant M as Model
240+ actor U as You
241+ participant Server as LLM API Server
237242
238- C->>API: POST /v1/chat/completions
239- Note over C,API: Headers: Authorization, Content-Type
240- Note over C,API: Body: messages, model, parameters
243+ U->>Server: POST /v1/chat/completions <br/> Headers: Authorization (API_KEY)<br/>Body: messages, model, parameters
244+ activate Server
245+ Note over Server: LLM inference <br/> generates a response
246+ Server-->>U: JSON response contains:<br/>- choices []<br/>- usage stats<br/>- metadata
247+ deactivate Server
241248
242- API->>API: Validate request
243- API->>M: Process prompt
244- M->>M: Generate response
245- M->>API: Return completion
246- API->>C: JSON response
247-
248- Note over C: Response contains:<br/>- choices[]<br/>- usage stats<br/>- metadata
249249```
250250
251- :: right ::
252251
253- <v-switch >
252+ ---
253+ layout: top-title
254+ color: violet-light
255+ ---
254256
255- <template #1 >
257+ :: title ::
258+
259+ # Anatomy of an LLM API call
260+
261+ :: content ::
262+ <v-switch >
263+ <template #1 >
256264
257- Example cURL Command
265+ Example Query
258266
259- ``` bash {!children:text-xs}
260- curl -X POST " https://api.openai.com /v1/chat/completions" \
261- -H " Authorization: Bearer $OPENAI_API_KEY " \
267+ ``` bash
268+ curl -X POST " https://ai-gateway.apps.cloud.rt.nyu.edu /v1/chat/completions " \
269+ -H " x-portkey-api-key: $PORTKEY_API_KEY " \
262270 -H " Content-Type: application/json" \
263271 -d ' {
264- "model": "gpt-4",
272+ "model":"@vertexai/gemini-2.5-flash-lite",
265273 "messages": [
266- {
267- "role": "user",
268- "content": "Explain quantum computing in simple terms"
269- }
270- ],
271- "max_tokens": 150,
274+ {"role": "system", "content": "You are a helpful assistant." },
275+ { "role": "user", "content": "Explain quantum computing in simple terms"}],
276+ "max_tokens":"128"
272277 "temperature": 0.7
273278 }'
274279```
275- </template >
276-
277-
278- <template #2 >
279-
280+ </template >
281+ <template #2 >
280282Response Structure
281283
282- ``` json {!children:text-xs}
284+ ``` json
283285{
284- "choices" : [{
285- "message" : {
286- "role" : " assistant" ,
287- "content" : " Quantum computing uses quantum mechanics..."
288- }
289- }],
290- "usage" : {
291- "prompt_tokens" : 12 ,
292- "completion_tokens" : 150
293- }
286+ ...
287+ "object" :" chat.completion" ,
288+ "model" :" gemini-2.5-flash-lite" ,
289+ "provider" :" vertex-ai" ,
290+ "choices" :[
291+ {"message" :
292+ {"role" :" assistant" ,
293+ "content" :"Imagine a regular computer uses bits, which are like light switches that can be either ON (1) or OFF (0).
294+ This is how it stores and processes information.\n\n**Quantum computing is like a super-powered, mind-bending
295+ version of this.** Instead of just ON or OFF, a quantum computer uses **qubits**.\n\nHere's where it gets weird and wonderful:\n\n*
296+ **Superposition: The \"Both ON and OFF\" Trick**\n A qubit can be ON, OFF, or **both ON and OFF at the same time**
297+ . Think of it like a spinning coin. Until it lands, it" },
298+ "index" :0 ,
299+ "finish_reason" :" length" }],
300+ "usage" :
301+ {
302+ "prompt_tokens" :12 ,
303+ "completion_tokens" :128 ,
304+ "total_tokens" :140 ,
305+ "completion_tokens_details" :{"reasoning_tokens" :0 }
306+ }
294307}
295308```
296- </template >
297309
310+ </template >
298311</v-switch >
299312
300313
314+
301315---
302316layout: side-title
303317side: r
0 commit comments