Why LLMs Are Tools, Not Just Automation

Overview

A recent essay by Rich Hickey has reignited a critical debate within the software engineering community regarding the fundamental nature of large language models (LLMs). Hickey contends that LLMs are forms of automation rather than tools, arguing that they cannot be truly "wielded" because their outputs are probabilistic and opaque. This distinction, he suggests, has profound implications for how we approach AI literacy, work displacement, and the development of responsible software systems.

While Hickey’s concern about the risks of outsourcing cognitive labor is valid, a closer examination reveals a more nuanced reality. The binary division between "wielded tool" and "mindless automation" collapses under scrutiny. When evaluated against the practical criteria of instrument usage—input specification, result interpretation, and iterative correction—LLMs fit squarely within the definition of a tool. Their unpredictability is a limitation of control, not a disqualification from the category.

This analysis argues that the useful distinction is not between tools and automation, but rather between skillful wielding and uncritical delegation. Recognizing LLMs as complex instruments requires users to develop specific competencies, transforming the conversation from ontological debating into practical skill-building.

Rethinking the Definition of a Tool

The core of Hickey’s argument rests on a specific philosophical definition of what it means to wield an instrument. He implies that for something to be a tool, it must be transparent, predictable, and fully subordinate to human will at every step of its operation. By this strict standard, an LLM fails because it often generates surprising, incorrect, or "hallucinated" content. However, this standard sets an impossibly high bar that even traditional tools cannot meet.

The Myth of Total Control

Picking up a saw does not guarantee a straight cut. Holding a chisel does not automatically make one a sculptor. Tools amplify the judgment of their operators, whether that judgment is poor or profound. Their usefulness depends on the operator's ability to direct them, interpret feedback, recognize mistakes, and make corrections in real-time.

Unpredictability is a feature of many tools: A compiler performs substantial work without the programmer directing each individual logical operation. A spreadsheet recalculates thousands of values instantly, yet no one argues that a spreadsheet is an autonomous agent rather than a tool.
Feedback loops define expertise: The difference between a novice and a master craftsman is not the absence of surprise, but the ability to diagnose and correct it. Similarly, a skilled developer uses an LLM to propose code, inspects the source, tests the change, and refines the request based on the output.
Automaticity is not autonomy: The fact that text generation happens automatically does not strip the system of its status as an instrument. We specify inputs, examine results, and decide what to do next. If automatic calculation makes a spreadsheet a tool, automatic text generation should not disqualify an LLM.

"Automation can be a feature of a tool."

Hickey’s framework creates a false division. It assumes that if a system handles execution, the human has ceded agency. In reality, the human retains agency through the specification of intent and the evaluation of outcomes.

The Practical Reality of AI Collaboration

Critics often argue that using LLMs lacks the intention, feedback, direction, and expertise required for genuine tool use. This claim is difficult to defend when observing how proficient developers and researchers actually interact with these systems. The workflow is deeply interactive and iterative, mirroring the practices used with any complex software instrument.

Building Effective Feedback Loops

A programmer does not simply paste a query and walk away. A robust workflow involves:

Asking: The user prompts the LLM to explain unfamiliar code or generate a draft solution.
Challenging: The user critically evaluates the explanation, looking for logical gaps or security vulnerabilities.
Inspecting: The user examines the source code or referenced materials to verify claims.
Testing: The user runs the proposed code in a sandboxed environment.
Refining: Based on the test results, the user rejects faulty suggestions and refines the prompt for better accuracy.

This process is a feedback loop directed by a human pursuing a specific purpose. It requires knowing what to ask, what evidence matters, and when the answer is wrong. Doing it well requires expertise, just as configuring a build system or optimizing a database query does.

The Risk of Misuse vs. The Nature of the Instrument

The possibility of avoiding intellectual work does not establish that every use avoids it. A scholar might use an LLM to summarize a dense academic paper, potentially sacrificing the deep understanding that comes from reading the primary source. However, another scholar might read the paper first and then use the LLM to critique their own interpretation or explore counter-arguments.

The outcome depends on the user, not the instrument. Blaming the LLM for shallow engagement is akin to blaming a calculator for arithmetic errors made by a student who refuses to learn basic math. The tool provides capability; the user provides context and critical judgment.

Beyond Anthropomorphism: Engineering Problems, Not Conspiracy

One of the most troubling aspects of the current discourse is the reliance on anthropomorphic vocabulary perpetuated by some sectors of the AI industry. Terms like "going rogue," "escaping containment," or "conspiring" frame LLM failures as acts of malice or independent will. This language is not only inaccurate but also obscures the real engineering challenges we face.

Reframing AI Failures

Software can produce unexpected outputs, fail to follow instructions, or cause harm when given access to consequential actions. These are engineering problems, not moral failings.

No hidden motives: Calling a model’s error a "conspiracy" adds a suggestion of human-like intent where none exists. It risks leading developers and policymakers to prepare for sci-fi scenarios of rebellion rather than addressing the mundane but dangerous issues of bias, data leakage, and hallucination.
Real risks remain: Acknowledging LLMs as tools does not mean ignoring their dangers. It means treating those dangers as systemic reliability issues that require rigorous oversight, red-teaming, and defensive programming practices.
Avoiding the agency trap: Surrendering judgment to an LLM is a human error, not a system flaw in the definition of the tool. We must cultivate AI literacy—the ability to wield these systems with skill and skepticism—rather than debating their metaphysical status.

Conclusion: The Case for Skillful Wielding

The disappointment with Hickey’s conclusion is not that it is entirely wrong, but that it is unhelpfully restrictive. Declaring that LLMs are "not tools" because they are unpredictable and powerful risks paralyzing effective adoption. It suggests that because we cannot control them perfectly, we should not consider them instruments we wield.

The more practical distinction is between wielding an instrument with skill and delegating work without adequate understanding. This distinction applies to all technologies, from steam engines to spreadsheets, and now to large language models.

Moving Forward with Agency

To navigate this new landscape, we must focus on:

Developing expertise: Learning how to craft effective prompts, verify outputs, and integrate LLMs into existing workflows.
Maintaining oversight: Ensuring that humans remain in the loop for critical decisions, rather than blindly accepting automated suggestions.
Rejecting anthropomorphism: Speaking about AI in terms of probability, training data, and engineering constraints, not agency and intent.

LLMs are complex, sometimes frustrating, and undeniably powerful. But they are extensions of human capability, not replacements for it. By recognizing them as tools, we reclaim our role as the operators, retaining the agency to decide how, when, and why we use them.