Prompt Engineering and API Integration · Lesson 4
Advanced Handling of LLM Responses
Raw model output is text, but applications need reliable data. Ask for JSON, validate it against a schema (Pydantic in Python, Zod in TypeScript), and retry or repair when validation fails. Many APIs now offer structured-output or JSON modes that enforce a schema.
Streaming returns tokens as they are generated, so users see the answer forming instead of waiting. You handle a sequence of events and append text deltas to the screen.
Control randomness with temperature (lower = more deterministic) and top-p. Always check the stop reason: an answer cut off by the max-token limit is incomplete and should be handled.
Key points
- Validate structured output with a schema
- Stream for better perceived speed
- Low temperature for extraction, higher for creative work
- Check stop_reason for truncated answers