AIDRIN Skill

AIDRIN ships a Model Context Protocol (MCP) server and a Claude Code skill that together let Claude drive AIDRIN assessments on your behalf — running metrics, interpreting results, and writing a readiness report — all from a plain-language request.

No commands to remember, no argument ordering to look up. Claude handles it.

How It Works

Two components plug into Claude Code:

MCP server (aidrin-mcp)

Exposes all AIDRIN metrics as tools that Claude can call directly. Claude sends named parameters; the server runs the metric and returns structured JSON. Image side-effects are suppressed by default, so only the JSON result comes back.

Skill (.claude/skills/aidrin/)

Instructs Claude on the full assessment workflow: which metrics to run for which intent, what column roles to confirm before running privacy or fairness metrics, how to interpret scores, and how to format the report. The skill is read by Claude Code at session start when the AIDRIN directory is open.

Claude Code reads .mcp.json from the project root to start the MCP server automatically. Both files ship with the AIDRIN repository — no extra configuration is required.

Prerequisites

  • Claude Code installed (CLI, desktop app, or IDE extension). See the Claude Code documentation.

  • AIDRIN installed in a Python 3.10+ environment with its conda/virtual environment activated. If you haven’t done this yet, follow CLI Installation first, then return here.

Setup

Step 1 — Install AIDRIN with MCP support

Once your environment is active, add the [mcp] extra from the repository root:

pip install -e '.[mcp]'

Or, with uv:

uv sync --group mcp

Verify the command is on your PATH before continuing:

which aidrin-mcp   # should print a path inside your active environment

Custom criteria outlier rules can be supplied to run_custom_outlier_check or run_aidrin_metric as inline rules_json or as rules_file. A rules_file path is read on the MCP server host and must point to a UTF-8 JSON array in the same format as examples/custom_outlier_rules.json. Set "target_match": "regex" on a rule to apply it to every target whose complete column name or HDF5 dataset path matches target.

File manifests can be checked with the dedicated verify_file_references tool, or with run_aidrin_metric using metric file-reference-validation and the path_targets argument. Relative paths default to the manifest’s directory; pass base_dir when they use another root. max_results caps invalid and metadata details, while scan_limit optionally caps all inspected occurrences. Set target_match="regex" to interpret each path_targets value as a regular expression matched against the complete column or HDF5 dataset name. Both the manifest and its referenced files are read from the MCP server host, not the Claude client machine.

The local stdio MCP server intentionally does not apply the web interface’s configured root allowlist. It uses the filesystem permissions of the account that started aidrin-mcp and should be connected only to trusted clients.

Variable units can be audited with verify_variable_units or with run_aidrin_metric using variable-unit-validation. With no sidecar, the tools return an audit of existing metadata. Supply at most one of unit_metadata_json and unit_metadata_file to apply resolutions and revalidate the complete sidecar. File paths resolve on the execution host, including a selected remote endpoint. See Unit Metadata Audit and Repair for the canonical sidecar schema and deterministic validation rules.

Step 2 — Open the AIDRIN directory in Claude Code

The repository already contains .mcp.json at the root:

{
  "mcpServers": {
    "aidrin": {
      "type": "stdio",
      "command": "aidrin-mcp",
      "args": [],
      "env": {}
    }
  }
}

When you open this directory in Claude Code, the MCP server starts automatically and AIDRIN’s tools become available to Claude for that session.

Note

Using a different project directory? The skill ships inside the aidrin package, so from your project root run:

aidrin skill install

This installs into every skills folder already present in the project (.claude/skills, .agents/skills); pass --dir <folder> to pick one explicitly or to create it. Then copy .mcp.json alongside it; Claude Code picks both up on next launch. After pip install -U aidrin, re-run aidrin skill install to refresh the skill to the version that matches the package.

Step 3 — Verify the connection

Start a Claude Code session in the AIDRIN directory and ask:

List the available AIDRIN metrics.

Claude should call the list_metrics tool and return the full metric catalogue grouped by category. If it falls back to running aidrin list in the terminal instead, the MCP server did not connect — check that aidrin-mcp is on your PATH (which aidrin-mcp).

Alternative — Install via skills.sh

The skill is also published on skills.sh and can be installed into any project, for any agent that supports the skills format (Claude Code, Codex, Gemini CLI, GitHub Copilot, OpenCode, and others), without cloning the repository:

npx skills add idtlab/aidrin

This installs only the skill. It drives AIDRIN through the aidrin command-line interface, so AIDRIN must be installed and on your PATH (CLI Installation). To use the MCP server instead, complete Steps 1–2 above as well.

Installing this way also keeps the skill easy to update: the CLI records the source repository in a skills-lock.json next to the skill, and a single command pulls the latest version from main — no release or manual copy needed:

npx skills update aidrin

Available Tools

The MCP server exposes the following tools. You do not normally call these by name; Claude selects them from your request. They are listed here so you know what is reachable.

Tool

What it does

list_metrics

Lists all available metrics, grouped by category

summarize_dataset

Describes a dataset’s numerical and categorical features

run_data_quality_check

Runs the three core data quality metrics: completeness, duplicity, outliers

run_aidrin_metric

Runs a single built-in metric against a dataset

verify_variable_units

Audits unit metadata or revalidates a complete inline or host-local sidecar

verify_file_references

Validates file references stored in selected dataset targets

run_custom_outlier_check

Runs Custom Criteria Outliers against selected targets

run_batch

Runs multiple metrics declared in a YAML or JSON batch config

create_custom_metric

Generates a CustomDR template file with metric() and remedy() stubs

run_custom_metric

Runs the metric() method of a CustomDR class defined in a .py file

run_custom_remedy

Runs the remedy() method of a CustomDR class and applies it to the dataset

agentic_build_index

Builds the FAISS vector index from the domain-literature PDFs in the agentic config

agentic_run

Runs the full agentic evaluation pipeline

Warning

run_custom_metric, run_custom_remedy, and the agentic tools execute Python from the files you point them at. Only use them with code you trust.

Running an Assessment

Point Claude at a dataset and describe your intent:

Is my dataset at /path/to/data.csv ready for training a classifier?

Check fairness and privacy in my CSV - the target is the "approved" column,
and "age", "zipcode", and "gender" may be quasi-identifiers.

Run a full data quality check on /path/to/data.csv and write a report.

Verify the file paths in the path and image_path columns of
/path/to/manifest.csv, resolving relative paths from /data/project.

Claude follows a structured workflow:

  1. Confirms AIDRIN is available and lists the metrics.

  2. Inspects the dataset schema and sample statistics.

  3. Proposes a metric plan matched to your intent and asks you to confirm column roles.

  4. Runs the metrics via MCP tools.

  5. Writes an interpreted markdown report with scores, their directional meaning, and suggested next steps — without declaring a ready/not-ready verdict (that judgment is yours).

  6. Offers to evaluate custom metrics or apply remedies to the dataset.

Supported file formats: CSV, Excel (.xls / .xlsx / .xlsb / .xlsm), JSON, NumPy (.npz), HDF5 (.h5), Parquet.

Agentic Pipeline via MCP

The agentic pipeline is also available through MCP, letting Claude orchestrate the full literature-grounded evaluation without you running any commands.

What it does: Takes domain-specific questions you define in a YAML config, retrieves relevant passages from indexed PDFs (research papers, standards, regulations), generates Python analysis code and runs it against your dataset, scores complexity, and produces remediation recommendations — all grounded in your domain literature.

When to use it: When you have domain PDFs and want to evaluate the dataset against field-specific standards rather than (or in addition to) generic quality metrics.

Setup

Install agentic dependencies:

pip install -e ".[agentic]"

Set your API key:

export OPENAI_API_KEY="sk-..."   # or the key for your OpenAI-compatible endpoint

Create an agentic config YAML. All paths are resolved relative to the config file:

llm:
  base_url: "https://api.openai.com/v1"   # any OpenAI-compatible endpoint

paths:
  data_loader: "./loader.py:load_dataset"  # Python function returning a DataFrame
  # OR: data_csv: "./data/mydata.csv"      # for plain CSV files
  metadata_csv: "./data/metadata.txt"      # required: plain-text dataset description

vector_store:
  sources:
    - ./sources                            # directory containing domain PDFs
  embedding_model: text-embedding-ada-002
  vector_store_name: my_index
  chunk_size: 1000
  chunk_overlap: 200

retrieval:
  enabled: true                            # false = skip RAG, use LLM knowledge only
  answer_model: gpt-4o
  top_k: 3
  max_workers: 4                           # questions run in parallel
  question:
    - "Does the age feature satisfy the HIPAA Safe Harbor de-identification standard?"
    - "What resampling rate is recommended by IEC 62056 for smart meter data?"

executor:
  enabled: true
  max_attempts: 5
  model: gpt-4o
  temperature: 0.0

complexity_scorer:
  enabled: true
  model: gpt-4o

remediation:
  enabled: true
  model: gpt-4o

output:
  save_log: true

Note

paths.metadata_csv is required. It is a plain-text file describing your dataset (columns, units, provenance) — used to give the LLM structural context. Without it the pipeline will not start.

Running

Tell Claude to run it:

Run an agentic evaluation using my config at /path/to/config.yaml

Claude will:

  1. Call agentic_build_index to index your PDFs into a FAISS vector store (once; skipped automatically on subsequent runs if the index already exists).

  2. Call agentic_run to execute the full pipeline.

  3. Return a combined JSON result: profile, queries (one entry per question with retrieval passages, generated code, execution result, complexity score, and remediation recommendations), and token_usage.

To build the index separately first:

Build the agentic index using /path/to/config.yaml

Then run the pipeline, telling Claude to skip rebuilding the index:

Run the agentic pipeline with /path/to/config.yaml — skip the vector build.

For a full end-to-end example using a real dataset and literature, see the Agentic Evaluation section on the CLI Usage page.