New GitHub Tool Stops AI Agents from Hallucinating Database Schemas

Overview

A new open-source utility called schema-guard is aiming to solve one of the most persistent pain points in AI-assisted software development: database schema hallucination. Coding agents frequently generate SQL queries based on outdated README documentation, naming conventions, or inferred column structures rather than the actual state of the database. This disconnect often leads to silent failures in continuous integration pipelines, broken dashboards, or late-night alerts for engineering teams.

The tool addresses this by maintaining a local snapshot of real table and column definitions within the repository. It validates AI-generated SQL against this ground truth before the code is committed or executed. While similar issues have been documented by major cloud providers, including a recent discussion in Snowflake’s developer blog, schema-guard offers a practical, client-side solution that does not require agent access to the warehouse itself.

How It Works

The core mechanism of schema-guard is the creation of a schema snapshot, typically stored as a JSON file in a hidden .schema-guard directory. This snapshot captures table names, column names, and data types, but explicitly excludes sensitive data and credentials. Agents can read this file to understand the current state of the database, or a pre-commit hook can check the generated SQL automatically.

When an agent attempts to write a query that references non-existent columns, schema-guard intercepts the action. It provides a specific denial message suggesting the correct column names. For example, if an agent tries to query country in a customers table where the actual column is country_iso2, the tool will reject the query and offer the correct identifier.

This feedback loop allows the agent to self-correct before the code lands in the repository. In evaluation scenarios, agents were able to read these denials, fix their SQL, and proceed without human intervention. This prevents broken code from propagating through the development lifecycle, reducing the cognitive load on developers who would otherwise need to debug why a dashboard failed.

Measured Impact and Evaluation

To test the efficacy of the tool, a controlled experiment was conducted using an analytics repository with a deliberately stale README. The setup involved an agent that could read and write files but had no direct access to the live database. The goal was to see if the agent could write correct SQL models based on instructions that contradicted the actual schema.

The evaluation compared three approaches across different AI models, including Claude Haiku 4.5 and Sonnet 5. The baseline group, which relied solely on the repository content without schema-guard, produced zero working files out of twenty-four attempts. Both models trusted the outdated documentation and mixed stale names with correct ones found in existing models, resulting in consistent failures.

When the schema snapshot was introduced, the results changed dramatically. With the snapshot and a simple rule in the agent’s configuration file, every single SQL model ran successfully. The few remaining errors were logical rather than structural, such as incorrect week calculations, indicating that the tool effectively eliminated hallucination-based failures.

A key component of this success was the use of a pre-commit hook. The hook acted as a safety net for agents that might not proactively look for the schema file. In tests, the hook denied incorrect queries in nearly all instances, forcing the agent to correct them immediately. Notably, the cost impact was negligible; the hook arm cost approximately the same as the baseline, with minor variations attributed to additional API calls for validation.

Installation and Integration

Schema-guard is designed to integrate seamlessly into existing development workflows. It can be installed via pip directly from the GitHub repository and supports a variety of database systems. Users can generate snapshots from dbt targets, DuckDB, SQLite, BigQuery, Snowflake, Databricks, or standard database URLs.

For teams using Claude Code, the tool can be installed as a plugin, enabling automatic validation via hooks. Other environments, such as Cursor, VS Code, or Windsurf, can utilize the tool via MCP (Model Context Protocol), exposing functions like list_tables, describe_table, and check_sql.

Additionally, schema-guard supports traditional pre-commit hooks and CI/CD pipelines. In a CI context, the tool can exit with an error code if missing tables or columns are detected, or if the committed schema snapshot is out of date compared to the live database. This ensures that schema drift is caught early in the deployment process.

"Without a snapshot, neither model wrote one working file. With the snapshot, every file ran."

This integration flexibility allows organizations to adopt schema-guard incrementally. Whether used as a strict gate in CI or a helpful hint for local agents, the tool provides a layer of resilience against one of the most common failure modes in AI-augmented development. As AI agents become more deeply embedded in engineering workflows, tools that enforce ground-truth accuracy will likely become essential infrastructure for maintaining code quality and system reliability.