Deployment and Project Integration · Lesson 7
Monitoring & Securing GenAI Systems
Monitor latency, error rates, token usage, cost per request, and quality signals such as user feedback and sampled evaluations. Watch for drift as data and user behaviour change.
Security threats include prompt injection (instructions hidden in user input or retrieved documents), data leakage, jailbreaks, and insecure tool use. The OWASP Top 10 for LLM Applications lists the main risks.
Defences: least-privilege tools, input and output filtering, separating instructions from data, PII redaction, rate limiting, and audit logs.
Key points
- Track latency, errors, tokens, cost and quality
- Prompt injection is the top LLM risk
- Least privilege for tools and data
- Redact PII and keep audit logs