Defining 'Good AI': Beyond the Hype

Overview

In the bustling discourse surrounding artificial intelligence, the term "good" is invoked with alarming frequency. It appears in vendor marketing, internal team debates, and technical reviews, yet it rarely carries a consistent definition. When developers or stakeholders claim an AI agent is "good," they may be referring to speed, safety, moral alignment, or raw reliability. These are distinct metrics, and conflating them risks significant miscalibration in adoption and evaluation.

The ambiguity of the term creates a critical communication gap. Two parties can agree that a specific system is "good" while implicitly judging it against entirely different standards. This semantic drift undermines rigorous analysis, leading organizations to deploy tools that are excellent for one context but dangerous or ineffective in another. Clarifying what constitutes "good" AI is therefore not merely a philosophical exercise but a practical necessity for software engineering and product development.

The Philosophical Roots of Ambiguity

The confusion surrounding "good" AI is rooted in the broader linguistic challenges of evaluating function and value. In his work on ethics, the philosopher Bernard Williams explored the nature of goodness, highlighting that it does not function like simple descriptive adjectives such as "red." If one states that a vehicle is red, the property of redness transfers seamlessly between the specific car and the general category of vehicles.

However, the logic breaks down when applied to functional excellence. Being a "good soccer player" does not automatically imply being a "good human being." The criteria for excellence in sport—agility, strategy, coordination—are entirely distinct from the moral or character traits that define ethical personhood. Williams argued that we cannot detach the concept of goodness from the kind of entity being judged. A good knife cuts efficiently, but applying that same standard to a good doctor or a good programmer is nonsensical.

This distinction is crucial for the AI landscape. Observers often commit a category error by assuming that proficiency in one domain translates to competence in another. An AI system might demonstrate exceptional ability in generating syntactically correct code, but that achievement belongs to the realm of pattern matching and language modeling. It does not inherently confer the virtues of a skilled engineer, such as ethical reasoning, strategic oversight, or professional accountability.

Beyond Code Generation

A common pitfall in AI evaluation is the conflation of task completion with role mastery. When a user observes that an AI agent can produce software snippets, essays, and legal memos, it is tempting to categorize it as a "good" programmer, writer, or lawyer. This inference is logically flawed. The agent may perform the isolated outputs associated with these roles well, but it does not embody the holistic responsibilities of the professions themselves.

Programming, for instance, involves far more than rapid code generation. It requires clarifying ambiguous requirements, navigating complex legacy systems, and coordinating across diverse organizational roles. Human programmers engage in nuanced trade-off analyses regarding maintainability, reliability, and security. They also handle interpersonal dynamics: training colleagues, persuading stakeholders, and disagreeing with flawed proposals. An AI tool that generates code quickly may succeed at the mechanical layer while failing completely at the systemic and social layers of software engineering.

Furthermore, the definition of a successful outcome is contingent on context. A simple, unverified script might be perfectly acceptable for a throwaway prototype at a fast-moving startup. The same lack of verification would be reckless and potentially catastrophic if applied to software relied upon by millions. Therefore, assessing whether an AI is "good" requires a deep understanding of the specific environment, the stakes involved, and the acceptable thresholds for failure.

A Framework for Evaluation

To move beyond vague enthusiasm, professionals must adopt a rigorous framework for evaluating AI tools. The first step is to demand specificity. When encountering claims of quality, the essential questions are: Good at what? For whom? Compared to what? And under which conditions?

Evaluation cannot rely solely on comparative rankings. Stating that a model is "better than most" is insufficient unless the baseline criteria for that domain are clearly defined. Instead, teams should establish explicit baselines, success metrics, and definitions of unacceptable failure modes before deployment. This involves understanding both the technological capabilities and the operational workflow the tool is intended to support.

Before calling a technology or tool good, we must ask: What outcomes are we measuring, and which failures are unacceptable?

Contextual mismatch is the primary source of disappointment in AI adoption. A vendor may correctly claim their product is excellent for generating quick prototypes, yet a team with strict long-term maintenance obligations may erroneously assume it is equally suited for that purpose. Shared history and context often allow individuals to communicate efficiently; a friend can call a restaurant "good" without elaboration because shared experiences supply the missing data. In professional settings, such shared assumptions rarely exist, making explicit specification vital.

Ultimately, judging whether something is a good X requires understanding what kind of thing X is, what it is designed to do, and the relevant factual constraints of the specific instance. By grounding evaluations in clear criteria rather than abstract praise, organizations can navigate the hype and make informed decisions about where AI adds genuine value.