Problems & fixes · AI coding tools
Security and quality problems with AI-generated code
Last updated 25 October 2026 · 1 source listed · Reading time about 3 minutes
Short answer
Generated code often looks right while hiding issues: insecure defaults, missing input validation, nonexistent or outdated dependencies, silent error swallowing, copied licensed snippets and tests that assert nothing. Catch these with the same safeguards used for human code: review, static analysis, dependency checks, meaningful tests and secret scanning, applied with extra suspicion toward confident-looking output.
Who this is for: A developer, reviewer or security engineer who wants to know what goes wrong with generated code and how to catch it before release.
What to evaluate
Insecure patterns
Models reproduce common but unsafe idioms such as string-built queries or weak cryptography choices.
How to check: Run static analysis and review against recognised secure coding guidance.
Hallucinated or risky dependencies
Suggested packages may not exist, and attackers can register names that assistants tend to invent.
How to check: Verify each new dependency's existence, maintainers and need before adding it.
Weak tests
Generated tests may only mirror the implementation or assert trivial conditions.
How to check: Mutate the code deliberately and see whether tests fail.
Error handling
Code may catch and ignore errors or log sensitive data.
How to check: Inspect failure paths and logging statements.
Licence and provenance
Large verbatim snippets can carry obligations.
How to check: Use licence scanning and similarity tools in your pipeline.
Secrets and data in prompts
Developers paste logs and configuration into chat.
How to check: Set guidance, scan prompts where possible and rotate anything exposed.
Comparison
| Problem | Why it happens | Check |
|---|---|---|
| Injection or unsafe input handling | Training data contains common insecure examples | Static analysis, parameterised interfaces, review |
| Invented library functions | Model fills gaps with plausible names | Compile and run; read the library documentation |
| Typosquatting-prone package names | Assistant suggests nonexistent packages | Dependency allow-lists and provenance checks |
| Tests that prove nothing | Tests generated from the same flawed logic | Mutation testing, independent test design |
| Subtle logic errors | Plausible but wrong reasoning | Code review with attention to edge cases |
Decision worksheet
Rate up to three options against this guide’s own criteria. It runs in your browser and saves only on this device. It is a thinking aid, not a recommendation.
1. Have you checked each criterion?
0 of 6 criteria checked.
2. Compare options (0 = fails or unknown, 3 = meets it well)
| Criterion | |||
|---|---|---|---|
| Insecure patterns | |||
| Hallucinated or risky dependencies | |||
| Weak tests | |||
| Error handling | |||
| Licence and provenance | |||
| Secrets and data in prompts | |||
| Total (max 18) | 0 | 0 | 0 |
Questions to ask before you commit
- Does our pipeline run static analysis and dependency checks on every change?
- Do we have an allow-list or review step for new dependencies?
- Who reviews generated code and what are they told to look for?
- Do we track defects traced to generated code?
- Are secrets scanned in repositories and in pasted content where feasible?
- Do our tests have independent coverage targets and mutation checks for critical areas?
- Is there guidance for marking generated sections in change descriptions?
- How do we handle licence scanning results?
Typical trade-offs
Strict automated gates versus developer speed
- Upside
- Gates catch repeated classes of mistakes.
- Downside
- False positives frustrate developers and encourage workarounds.
Labelling generated code in change requests versus not
- Upside
- Reviewers know where to focus.
- Downside
- Labels can be inconsistent and may stigmatise tool use.
Allow-listed dependencies versus open choice
- Upside
- Reduces supply chain risk.
- Downside
- Slows adoption of legitimate libraries.
Common mistakes
- Merging because tests pass without checking what the tests assert. Read them.
- Installing a suggested package without checking it exists and is maintained.
- Trusting security advice from the same assistant that wrote the code. Use independent tools and people.
- Logging full request bodies in generated handlers. Review what is logged.
Related questions
Is generated code less secure than human-written code?
It can be, depending on the task and tool, since models reproduce widespread patterns including flawed ones, and developers may review it less carefully because it looks polished. But human code has flaws too. The practical response is to apply consistent controls to all code: standards, review, automated analysis and testing, and to be alert to the tendency to over-trust fluent output.
What is package hallucination and why is it dangerous?
Assistants sometimes recommend libraries that do not exist. Attackers can publish malicious packages under those names so that developers who copy the suggestion install harmful code. Defences include verifying every dependency, using private registries or allow-lists, and pinning versions with integrity checks.
How do we get useful tests from an assistant?
Give it the requirements and edge cases rather than just the implementation, ask for tests of failure paths and boundaries, then review them yourself. Validate by introducing deliberate bugs to see whether tests fail. Tests derived only from implementation tend to confirm whatever the code already does.
Should we ban chat tools for security reasons?
A ban might be justified for the most sensitive projects, but blanket bans often push use to personal accounts. A better route is an approved tool with strong data terms, clear rules, scanning and training. Revisit the policy as tools and your evidence evolve.
Sources and further reading
These pages were reachable when we last checked. They informed definitions and rules referred to above; they do not endorse this guide. Where rules differ by country, check the authority for your location.
- Secure Software Development Framework — National Institute of Standards and Technology, retrieved 27 October 2026
Related guides
How to choose · AI coding tools
How to choose an AI coding assistant for an engineering team
Evaluate coding assistants on how they treat your source code, how well they understand your repository, where they run in the workflow, and how their output fits review and testing. Compare code retention and training terms, repository-aware context, editor and pipeline integration, administrative policy controls, licence-related filters, and measured effect on review burden rather than raw lines produced.
Comparison · AI coding tools
AI pair programmer versus autonomous coding agent
Assistive tools help inside the developer's flow with small, visible changes. Autonomous agents take a task description, edit many files, run commands and return a result for review. Agents suit well-bounded chores with strong tests; assistive tools suit exploratory and design-heavy work. The deciding factors are test coverage, review capacity, permission risk and how clearly the task can be specified.
Requirements · AI coding tools
What an engineering policy for AI coding tools should cover
A workable policy names approved tools and plans, forbids pasting secrets or customer data, sets repository eligibility, requires normal review and testing for all code regardless of origin, defines licence and provenance handling, covers agentic tool permissions, and assigns ownership for monitoring and updates. Keep it short enough that developers actually read it.
How to choose · Workflow automation
How to choose a no-code workflow automation platform
Choose by the apps you must connect, the complexity of logic you need, how failures are surfaced, and how pricing scales with volume. Connector coverage and error handling matter more than a polished builder. Also check credential security, team access controls and how easily workflows can be exported if you outgrow the platform.
Problems & fixes · AI writing and content tools
Common quality problems in AI-generated content and how to fix them
Typical problems are generic phrasing, invented facts, inconsistent voice, outdated claims, repeated structure across pages and tone that misreads the audience. Fixes sit in process: stronger briefs with source material, a mandatory fact-check step, editor ownership of voice, and a review rubric. Better prompts help, but human editorial standards do the real work.
Requirements · AI writing and content tools
Using AI image and video generators for commercial work
Before commercial use, check the tool's terms on ownership and permitted use, how training data was sourced, whether outputs can resemble existing works or real people, disclosure rules for synthetic media, and what provenance signals are embedded. Copyright treatment of generated material varies by country, so confirm with a qualified adviser for anything high stakes.
Requirements · Professional services
Conflicts of interest and confidentiality when hiring a lawyer
Lawyers owe duties of confidentiality and loyalty, which is why they run conflict checks before acting. A conflict arises when acting for you would clash with duties to another client or with the lawyer's own interests. Some conflicts can be managed with informed consent and safeguards; others bar the lawyer entirely. Rules differ by jurisdiction, so ask the firm to explain how they apply to your matter.