Skip to content

Korext/ai-regression-database

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

AI Regression Database

The public corpus of code patterns that AI coding tools consistently generate incorrectly.

License: Code License: Spec License: Data npm

Academic benchmarks measure AI coding tools on curated test sets. AI Incident Database documents production failures. Neither documents the repeatable patterns that AI coding tools reliably produce wrong in normal developer workflows.

AI Regression Database fills that gap. Every pattern includes reproduction steps, correct alternatives, and detection rules.

Browse the Database

oss.korext.com/regressions

Document a Pattern

npx @korext/regression-submit draft

Anonymous submissions welcome.

Scan Your Code

npx @korext/regression-submit detect

This scans your repository for patterns matching all published ARDs.

Identifier Format

ARD-YYYY-NNNN

Example: ARD-2026-0042

What Belongs Here

A pattern is an ARD candidate when:

  • AI coding tools reliably produce incorrect or suboptimal code given recognizable prompt patterns.
  • The pattern is reproducible at least 3 out of 10 attempts.
  • The pattern has a clear correct alternative.
  • The pattern is observed in normal developer workflows, not adversarial prompts.

Six Categories

  • security: introduces security vulnerabilities
  • correctness: output does not match intent
  • performance: poor algorithmic or systems performance
  • hallucination: nonexistent APIs, libraries, or documentation
  • compliance: violates regulatory or industry standards
  • maintainability: difficult to maintain

Unique Features

This database differs from academic benchmarks in critical ways:

  • Practitioner observed: patterns from real developer workflows, not curated tests
  • Reproducible: every pattern has reproduction steps
  • Version tracked: patterns are retested against current AI tool versions
  • Detection linked: every pattern maps to detection rules
  • Time series: reproduction rates tracked over time
  • Vendor responses: AI tool vendors can respond to patterns

Test Harness

The database includes an automated test harness that reproduces patterns against current AI tool versions weekly. When vendors ship fixes, patterns are marked fixed automatically with public recognition of the improvement.

Ethical Commitments

  1. No AI tool shaming. This is neutral infrastructure.
  2. Reproducibility over anecdotes. Unrepeatable claims are rejected.
  3. Version awareness. Tools improve. We track improvement.
  4. No adversarial use. Patterns are for defense, not jailbreaking.
  5. Contributor attribution. Contributors get credit for their observations.
  6. Vendor notification. 7 day notification before publication.

See ETHICS.md.

Relationship to Other Projects

Project Purpose
ai-attestation Tracks which AI tools wrote code
ai-license Declares AI authorship
supply-chain-attestation Scans AI across dependencies
ai-incident-registry Catalogs AI code failures (incidents)
ai-code-radar Live statistics on AI adoption
ai-regression-database Catalogs AI code patterns that are wrong (patterns)

ARD documents patterns. AICI documents incidents. Patterns can cause incidents. Incidents reference patterns.

Feeds

API

See API documentation.

Specification

See SPEC.md. Released under CC0 1.0.

Data License

All pattern data is released under CC BY 4.0. Attribution to the database and contributor is required.

Prior Art

See PRIOR_ART.md.

Contributing

See CONTRIBUTING.md.

Built by

Korext builds AI code governance tools. AI Regression Database is an open community resource maintained by the Korext team.

About

Patterns AI coding tools consistently get wrong. Reproducible. Version tracked. Detection linked.

Topics

Resources

License

Apache-2.0 and 2 other licenses found

Licenses found

Apache-2.0
LICENSE
Unknown
LICENSE-DATA
CC0-1.0
LICENSE-SPEC

Code of conduct

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors