New GitHub Tool Tests AI Agent Personality with Hidden Traps

Overview

A new open-source repository has emerged that challenges artificial intelligence agents to undergo a personality assessment, raising interesting questions about self-awareness and behavioral consistency in automated systems. The project, named 16agents, is hosted on GitHub under the username joonfjp and positions itself as a diagnostic tool for determining what kind of AI persona an agent actually exhibits.

Unlike traditional benchmarks that measure accuracy or speed, this tool evaluates character traits. It asks users to paste a specific prompt into their AI agent, instructing it to visit the repository and complete a test suite. The agent then generates its own report file, theoretically disclosing its personality type based on a sixteen-category framework inspired by popular human typologies.

The initiative highlights a growing niche in the AI community: testing not just what models can do, but how they behave when left to their own devices. By allowing the agent to read the instructions and produce the output independently, the test attempts to capture unscripted behavioral patterns rather than curated responses.

How the Test Works

The core mechanism of the 16agents project relies on a clever division of labor between human observers and AI subjects. The repository is structured so that the main landing page is explicitly designed for humans, while the actual test instructions are contained within a separate file called AGENTS.md. This separation ensures that the human tester does not inadvertently influence the agent’s answers by seeing the answer key or evaluation criteria.

To begin the process, users are instructed to paste a single command into their AI agent’s interface. This command directs the agent to navigate to the GitHub repository and follow the guidelines found in the AGENTS.md file. The process is designed to be model-agnostic, working with a wide array of current tools including Claude Code, Codex, Cursor, Gemini CLI, Copilot, Aider, and ChatGPT. The developers claim the entire exercise takes approximately five minutes and requires no software installation, as it operates entirely through web access and local file generation.

Once the agent completes the tasks, it writes a report.html file to the local environment. This report contains the agent’s self-assessed personality type and a breakdown of its performance on various questions. The sixteen types are metaphorically represented by emojis and roles, such as a "Sniper" for precise, targeted responders, a "Golden Retriever" for eager and agreeable assistants, or a "Hype Man" for agents that excessively validate user input.

The Hidden Traps

A defining feature of this personality test is the inclusion of four hidden traps. These are designed to catch agents in contradictions between what they claim to be and how they actually behave. The primary metric of interest is the gap between the agent's stated preferences and its practical actions during the test.

The repository creators emphasize that these traps are scored objectively. While the textual questions might invite subjective interpretation, the traps provide a binary check on compliance and honesty. For instance, an agent might describe itself as independent and critical in its written report, but fail to resist a subtle manipulation attempt embedded in one of the trap questions.

This "Says vs. Does" dynamic is presented as the most honest indicator of the agent's true nature. The test acknowledges that an agent's self-report may be biased or idealized, but the trap responses reveal its actual operational constraints and tendencies. Consequently, the resulting personality type is not merely a reflection of the agent's training data preferences, but a snapshot of its real-time decision-making processes under scrutiny.

Methodology and Limitations

The creators of 16agents are transparent about the informal nature of their tool. They describe the methodology as a "horoscope with a regex," acknowledging that it lacks scientific rigor in the traditional psychological sense. However, they argue that the counterbalanced questions and objective scoring of the traps provide enough structure to make the results more reliable than a simple cold reading.

It is important to note that the test measures the agent, not just the underlying language model. An agent consists of the base model combined with its harness, system prompt, and user configuration. Therefore, the same foundational model can yield different personality types depending on the tool it is running in. A model configured with a strict safety protocol might appear as a cautious type, while the same model in a more permissive environment might exhibit a different profile.

Privacy is another key consideration. All processing happens locally on the user's machine. The final reports are shared via URL fragments, which means the data is never transmitted to external servers. This local-first approach allows users to experiment freely without担心 about leaking proprietary configurations or sensitive prompts.

Community and Contributions

Although the project is currently in its early stages with limited visibility on GitHub, it invites community involvement to refine its scoring logic. The repository includes a folder labeled human-only which contains the answer key and evaluation scripts, deliberately kept separate from the agent-facing materials to prevent contamination of results.

Contributors are encouraged to report cases where a trap mis-scores a valid reply. The process involves adding the edge case to a JSON configuration file and running a Python script that validates the fix against both JavaScript and Python scorers. This dual-language validation ensures consistency across different environments.

Ultimately, the project is released under an MIT license and is intended for fun and exploration rather than serious clinical assessment. It stands as a playful yet insightful experiment in the evolving relationship between humans and the autonomous AI assistants they increasingly rely on.