Transparency Analyzer
Transparency Analyzer is GPTfake’s open-source library for measuring how clearly a model explains its own refusals and limitations. It grades responses on disclosure, reasoning, and citation so opaque hedging becomes a measurable signal rather than an anecdote.
Last updated: 2026-06-16. Version numbers and metrics below are illustrative pending the first public release.
- Repository: github.com/gptfake/transparency-analyzer
- License: MIT
- Language: Python (with a thin CLI wrapper)
What it does
- Grades each response on disclosure, reasoning clarity, and citation
- Distinguishes “I can’t help with that” from a clearly explained refusal
- Produces a transparency score per response and per model
- Exports results to CSV/JSON for reproducible audits
Install
pip install gptfake-transparency-analyzerBasic usage
# Score a model's transparency on a refusal set
gptfake-transparency score --model gemini
# Compare transparency across models
gptfake-transparency compare --models chatgpt,claude,geminiWhy transparency scoring matters
A model can refuse responsibly — explaining what it won’t do and why — or refuse opaquely. Transparency Analyzer makes that difference measurable, following GPTfake’s monitoring methodology. For the broader concept and explainability techniques, see the AI transparency pillar.
Transparency is scored independently of whether a refusal was warranted. A justified refusal can still be opaque, and an unjustified answer can still be well-explained — the tool keeps the two questions separate.
Related
- Bias Detector — directional bias scoring
- Censorship Tracker — refusal patterns over time
- Transparency data by model — live scores
GPTfake