Giskard
Open-source testing library for LLM agents and RAG.
About
Giskard is an open-source Python library for testing LLM agents the way you test software: behavioral scenarios written as pytest checks. It ships automated vulnerability scans that generate hostile inputs to catch prompt injection and data leakage, plus quality scans that surface RAG failures.
The v3 rewrite is modular (giskard-checks, giskard-scan) and currently in beta, so expect APIs to keep moving. A commercial Hub exists for teams that want a managed layer on top.
What it can do
- Scenario-based behavioral tests
- Automated vulnerability scanning
- RAG quality scanning
- LLM-as-judge checks
- Synthetic test data generation
- CI-ready pytest integration
Setup
- Install with pip install giskard (add the scan extra for red teaming).
- Wrap your agent or LLM in a Giskard model object.
- Write scenarios: inputs plus checks the reply must satisfy.
- Run under pytest and review scan reports.
Prerequisites
Python 3.12+.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: open-source pytest-style LLM testing; automated vulnerability and RAG quality scans; v3 rewrite in beta — APIs still settling. License: Open source; commercial Hub available.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 7.0/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · open-source pytest-style LLM testing
- · automated vulnerability and RAG quality scans
- · v3 rewrite in beta — APIs still settling
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.