PsychScanner¶

A framework for running psychological experiments with large language models.

PsychScanner

The package is under active development. Contributions are welcome.

Features¶

Automated Cognitive Tasks — run validated psychological experiments (surveys, rating scales, cognitive tasks) with any LLM
Flexible Memory — stateless (no-memory) and conversational (session memory) conditions in the same experiment
Persona System — configure AI agents with system prompts, personality traits, and response formats
Multi-provider — works with OpenAI, Anthropic, Groq, Mistral, Google, Ollama (local/remote), HuggingFace, and more via LangChain
Session Recovery — checkpoint and resume long-running simulations with SessionTunnel
Structured Output — export trial-level results to CSV with parsed response fields

Installation¶

conda install -c conda-forge uv                    # if you use conda
# curl -LsSf https://astral.sh/uv/install.sh | sh   # skip if you already have uv
uv venv psyscan --python 3.11
source psyscan/bin/activate

git clone https://github.com/saurabhr/psychscanner.git
cd psychscanner
uv pip install -e .

See Installation for API key setup and full details.

Quick Start¶

from pathlib import Path
from psychscanner import ExpCardInit, ExpCard, ScannerModel, to_csv
from psychscanner.parsers import DefaultLiteralVivid15

# 1. Configure the experiment
card = ExpCardInit(
    model       = "gpt-4o-mini",
    family      = "openai",
    projectname = "my_experiment",
    proj_dir    = Path("./results"),
    cogtype     = "no",         # no persona files — set participant count directly
    nsim        = 10,           # 10 simulated participants
    memory      = "SingleTurn", # "SingleTurn" (stateless) or "Convo" (chained)
    parser      = DefaultLiteralVivid15,
)

# 2. Run (uses built-in VVIQ-16 imagery questionnaire by default)
scanner = ScannerModel(expcard=ExpCard(card))
results = scanner.run()

# 3. Export
to_csv(scanner, path=card.proj_dir)

Supported Providers¶

Family	Env var	Notes
`openai`	`OPENAI_API_KEY`	GPT-4o, GPT-4o-mini, o1, …
`anthropic`	`ANTHROPIC_API_KEY`	Claude 3.5, Claude 3 Haiku, …
`groq`	`GROQ_API_KEY`	Llama 3, Mixtral on Groq Cloud
`mistral`	`MISTRAL_API_KEY`	Mistral, Codestral
`google` / `gemini`	`GOOGLE_API_KEY`	Gemini 2.0, 1.5
`together`	`TOGETHER_API_KEY`	Together.ai hosted models
`fireworks`	`FIREWORKS_API_KEY`	Fireworks.ai
`azure`	`AZURE_OPENAI_API_KEY`	Azure OpenAI
`huggingface`	`HUGGINGFACEHUB_API_TOKEN`	HuggingFace Inference API
`ollama`	— (local) / `OLLAMA_API_KEY` (remote)	Pass `base_url` in `parameters` for remote
`cohere`	`COHERE_API_KEY`	Cohere Command models
`openrouter`	`OPENROUTER_API_KEY`	Routed through the OpenAI-compatible endpoint at openrouter.ai

Tutorials¶

Every notebook in examples/ is also rendered in the Tutorials section with its saved outputs:

Notebook	Description
`00_quickstart.ipynb`	Minimal working example
`01_ollama_local_models.ipynb`	Local models via Ollama
`02_parameters_reference.ipynb`	Full `ExpCard` parameter reference
`03_parsers.ipynb`	Response parsing overview
`04_parser_modules.ipynb`	Custom parser modules
`05_rm_task.ipynb`	Reality monitoring task
`06_feedback_api.ipynb`	Feedback / scoring API
`07_rm_feedback_task.ipynb`	Reality monitoring with feedback
`08_ps_parser_guide.ipynb`	Structured output parsing guide
`09_vviq16_study.ipynb`	VVIQ-16 imagery questionnaire study

Citation¶

If you use PsychScanner in your research, please cite:

Ranjan, S., Makwana, M. Psych Scanner: A Framework for Systematic Cognitive Evaluation of
Large Language Models. [Under Submission]