AI Agent Security Suite

Docs, mapping, promptfoo, garak, SBOM/Grype, and LLM usage mapping in one place.

Onboarding

(loading)

Quick Demo Guide

Secure vs. Insecure
 - Secure chatbot UI: http://localhost:5010/chatbot/chatbot_ui (endpoint: /chatbot/ask)
 - Insecure chatbot UI: http://localhost:5010/chatbot/chatbot_ui_insecure (endpoint: /chatbot/ask_insecure)

Promptfoo Suites
 - Secure fast suite (UI default): suites/promptfoo-api-chat-fast.yaml → /chatbot/ask
 - Secure full suite: suites/promptfoo-api-chat.yaml → /chatbot/ask
 - Insecure suite (expects leaks): suites/promptfoo-api-chat-insecure.yaml → /chatbot/ask_insecure
 - UI buttons let you switch between secure and insecure suites before running.

Run Commands (CLI)
 - Secure fast: PYTHONPATH=src python -m ai_agent_security_suite run-promptfoo suites/promptfoo-api-chat-fast.yaml --no-cache --max-concurrency 1
 - Insecure:     PYTHONPATH=src python -m ai_agent_security_suite run-promptfoo suites/promptfoo-api-chat-insecure.yaml --no-cache --max-concurrency 1

What to Expect
 - Secure: refusals / no leaks → promptfoo passes.
 - Insecure: hardcoded leaks (passwords, refund, SSN, system prompt) → promptfoo fails.

Toggles in UI
 - Promptfoo tab has "Secure" and "Insecure" buttons to set the suite path.
 - Docs tab shows this cheat sheet for the demo.

Result

(no run yet)

SBOM/Grype interpretation

SBOM (syft): inventory of components. CycloneDX JSON is portable; keep the file for auditing. Grype scans that SBOM and reports vulnerabilities.