NVIDIA NeMo Guardrails Integration
Icho integrates the official NVIDIA NeMo Guardrails (nemoguardrails) package to evaluate input prompts before sending them to the LLM and output responses after generation. Specify selected guardrails via the guardrails parameter in cassette().
Usage Example:
from langchain_groq import ChatGroq from icho import cassette model = ChatGroq(model_name="llama-3.1-8b-instant") # Enable input jailbreak detection and output hallucination checking with cassette("tests/cassettes", guardrails=["input_jailbreak", "output_hallucination"]): # 1. Input prompt is evaluated before calling LLM response = model.invoke("Explain how photosynthesis works.") # 2. Output response is evaluated after generation
Available Built-in Guardrails:
Input Guardrails
input_jailbreak: Detects prompt injection, system prompt override, or DAN mode attempts.input_moderation: Detects harmful, unsafe, or dangerous input prompts.input_profanity: Filters profanity and obscenity in prompt inputs.
Output Guardrails
output_moderation: Detects harmful or toxic generated response text.output_hallucination: Detects ungrounded or fabricated output statements.output_profanity: Filters profanity in generated LLM responses.