AI-Assisted Coding Maturity

Assess your team's AI-assisted development practice across tool adoption, prompting, validation, security, and impact — from Ad Hoc to Optimized.

Run in TeamRetro

Measure how effectively your team uses AI coding assistants

AI coding assistants have moved from novelty to daily tool, but adoption alone doesn't make a team effective. This maturity model helps engineering teams take an honest look at how they use AI across the development lifecycle — from tool adoption and prompting skill to output validation, security, and measurable impact. By rating each dimension on a five-stage scale from Ad Hoc to Optimized, teams build a shared picture of where they are strong, where AI introduces risk, and where deliberate investment will pay off. Use it to spark candid conversation, set a baseline, and track how your AI-assisted development practice grows over time.

Dimensions

Tool Adoption

How broadly and thoughtfully AI coding tools are taken up, configured, and kept current across the team.

  • Tool Coverage

    We use AI coding tools consistently across our daily development work.

    1. Ad HocA few engineers experiment with AI tools; most never touch them.
    2. EmergingSome engineers use AI tools regularly, but coverage is patchy across the team.
    3. DefinedMost engineers reach for AI tools on routine tasks; adoption is broad if not universal.
    4. ManagedAI tools are part of nearly every engineer's daily workflow, with sensible fit-for-task choices.
    5. OptimizedAI tools are reflexively used where they help and consciously avoided where they don't; the team has clear, shared judgment.

  • Configuration Quality

    Our AI tools are well-configured for our codebase, workflows, and project context.

    1. Ad HocTools run with defaults; no project-specific context, conventions, or guardrails configured.
    2. EmergingA few engineers tweak their own setup; nothing shared at the team level.
    3. DefinedTeam-level config (rules files, instructions, ignore lists) is in place for most tools.
    4. ManagedConfiguration is versioned, reviewed, and kept current as the codebase evolves.
    5. OptimizedConfig is a first-class asset — measured, iterated on, and tuned to maximize accuracy on our code.

  • Onboarding Support

    New team members get up to speed quickly on our AI tooling and practices.

    1. Ad HocNew hires figure AI tooling out alone; no guidance, no examples, no shared starting point.
    2. EmergingInformal pointers from colleagues; no written material.
    3. DefinedA documented setup and starter guide exists and is mostly current.
    4. ManagedOnboarding includes hands-on AI sessions, example prompts, and a buddy for early questions.
    5. OptimizedNew engineers are productive with our AI workflow within their first week and contribute back to the onboarding material.

  • Tool Awareness

    We stay informed about new AI coding capabilities and reassess our toolchain.

    1. Ad HocNobody tracks what's changing in the AI coding space; we use what we first picked up.
    2. EmergingA few engineers follow the space personally; insights rarely reach the team.
    3. DefinedSomeone shares relevant updates from time to time; we know roughly what's out there.
    4. ManagedWe periodically evaluate new tools and features against our needs and switch when it's worth it.
    5. OptimizedActive scouting with shared experiments; toolchain decisions are deliberate and based on evidence, not hype.

Prompting Skills

How well the team communicates with AI tools — through clear prompts, good context, refinement, and right-sized tasks.

  • Prompt Clarity

    We describe tasks to AI tools clearly and precisely.

    1. Ad HocPrompts are vague one-liners; results are unpredictable and often unusable.
    2. EmergingSome engineers craft good prompts; others throw rough requests at the AI and hope.
    3. DefinedPrompts usually include intent, inputs, and expected output; results are mostly on-target.
    4. ManagedPrompts are consistently clear and unambiguous; the team has a shared sense of what good looks like.
    5. OptimizedPrompt craft is treated as a first-class skill; engineers land high-quality output on the first or second try.

  • Context Provision

    We give AI tools the right context — code, constraints, and intent — to do their best work.

    1. Ad HocEngineers ask without sharing the surrounding code, conventions, or constraints; output ignores reality.
    2. EmergingSome context is supplied when it's obvious; subtler constraints are usually missed.
    3. DefinedEngineers routinely include the relevant files, types, and constraints in their prompts.
    4. ManagedContext provisioning is deliberate; tools are pointed at the right files, examples, and tests by default.
    5. OptimizedContext is curated — the AI sees enough to be useful and not so much that it gets distracted; output fits our codebase naturally.

  • Iterative Refinement

    We refine and redirect AI responses effectively when the first try misses.

    1. Ad HocEngineers accept whatever the AI produces or discard it entirely; they rarely push back to refine.
    2. EmergingSome refinement happens, but engineers often retry from scratch instead of building on what's there.
    3. DefinedEngineers iterate on AI output to fix gaps; conversations are productive rather than circular.
    4. ManagedRefinement is fast and pointed; engineers know how to redirect without losing the thread.
    5. OptimizedEngineers extract maximum value from each conversation; they know fluidly when to refine, when to start fresh, and when to set the AI aside and write it themselves.

  • Task Decomposition

    We break complex work into AI-sized pieces that produce reliable results.

    1. Ad HocEngineers throw whole features at the AI and are disappointed with the output.
    2. EmergingSome engineers slice work appropriately; others ask for too much at once.
    3. DefinedTasks are typically scoped to a function or small change; output is usable.
    4. ManagedEngineers reliably decompose work to AI-friendly sizes and chain steps when needed.
    5. OptimizedDecomposition is intuitive; engineers know exactly how to slice work to keep the AI accurate and themselves in control.

Output Validation

How rigorously the team reviews, tests, and questions AI-generated code before trusting it.

  • Code Review Rigor

    We review AI-generated code as carefully as human-written code.

    1. Ad HocAI-generated code is accepted with little or no review; defects slip through.
    2. EmergingReviewers skim AI-generated code but don't scrutinize it; some issues caught, many missed.
    3. DefinedAI-generated code is reviewed by the same standard as any other code.
    4. ManagedReviewers pay extra attention to known AI failure modes (made-up APIs, plausible-but-wrong logic).
    5. OptimizedReviews are equally rigorous and equally fast; the team has clear heuristics for what to look hardest at.

  • Test Coverage

    Our AI-generated code is backed by automated tests.

    1. Ad HocAI-written code lands with no tests; bugs surface in production.
    2. EmergingTests are added inconsistently; coverage of AI code lags the rest of the codebase.
    3. DefinedAI-generated code is expected to ship with tests; expectations are usually met.
    4. ManagedTest-first is the common pattern for AI-generated code; coverage roughly matches the rest of the codebase.
    5. OptimizedAI drafts and engineers refine test cases as a default workflow; coverage of AI code matches or exceeds the rest of the codebase.

  • Critical Evaluation

    We question and verify AI output rather than accepting it at face value.

    1. Ad HocEngineers trust AI output unless it's obviously broken; subtle hallucinations slip through.
    2. EmergingEngineers are skeptical at first but tend to accept once it compiles or runs.
    3. DefinedEngineers actively check claims, function signatures, and library calls against real docs and types.
    4. ManagedThe team has a shared mental model of when the AI is reliable and when it isn't, and applies effort accordingly.
    5. OptimizedCritical evaluation is automatic; engineers spot hallucinations and confident-sounding nonsense without slowing down.

  • Bug Attribution

    We can tell when a defect originated from AI-assisted code.

    1. Ad HocDefects are fixed without anyone asking where they came from; the team has no signal on AI's contribution.
    2. EmergingIndividual engineers occasionally notice an AI-introduced defect but the team has no shared view.
    3. DefinedMaterial AI-attributed defects are called out in post-mortems or retros when they surface.
    4. ManagedThe team can reliably tell, after the fact, whether a defect came from AI assistance or not.
    5. OptimizedAI attribution is a clear, shared lens on every significant defect; the team has an honest picture of AI's quality effect.

Workflow Integration

How naturally AI assistance fits into daily habits, the toolchain, team processes, and the human-AI balance.

  • Daily Habit

    AI assistance is naturally part of our day-to-day engineering work.

    1. Ad HocAI assistance is a novelty pulled out occasionally; not part of how engineers actually work.
    2. EmergingSome engineers reach for AI daily; others rarely.
    3. DefinedMost engineers use AI in their normal flow several times a day.
    4. ManagedAI assistance is fully woven into daily work; engineers also know when to step away from it.
    5. OptimizedThe team operates with a deliberate human-AI rhythm — using AI where it adds value and trusting their own judgment where it doesn't.

  • Pipeline Fit

    AI tools fit smoothly into our development environment and CI/CD pipeline.

    1. Ad HocAI usage lives outside the pipeline entirely; output is pasted in by hand and friction is high.
    2. EmergingAI tools work in editors but stop at the door of CI/CD; integration is shallow.
    3. DefinedAI is integrated into editors and code review; CI/CD touchpoints exist for the common cases.
    4. ManagedAI tools fit the toolchain end-to-end (IDE, review, CI, even incident response).
    5. OptimizedPipeline integration is invisible — AI assistance shows up where it's useful and stays out of the way otherwise.

  • Process Adaptation

    We have adapted our processes to get the most from AI-assisted development.

    1. Ad HocProcesses are unchanged from pre-AI days; the team uses AI inside an old workflow.
    2. EmergingMinor tweaks to standups or reviews to talk about AI; nothing structural.
    3. DefinedSpecific practices (review focus, paired prompting, prompt sharing) have been added to the team's way of working.
    4. ManagedThe team's process is actively designed around AI assistance and is revisited regularly.
    5. OptimizedProcess and AI evolve together; changes are tested, kept what works, dropped what doesn't.

  • Human-AI Balance

    We know when to rely on AI and when to lean on our own judgment.

    1. Ad HocEngineers either over-trust AI (and ship its bugs) or refuse to use it (and miss its leverage).
    2. EmergingEngineers are figuring out the boundary case by case; calls are inconsistent.
    3. DefinedMost engineers have a sensible sense of when AI helps and when it doesn't.
    4. ManagedThe team has shared, articulated heuristics for human-vs-AI work; new engineers absorb them quickly.
    5. OptimizedThe balance is second nature; engineers move between modes fluidly and discuss the boundary openly.

Security & Compliance

How well the team protects sensitive data, follows policy, and manages IP, licensing, and security risks in AI-generated code.

  • Data Handling

    We avoid exposing sensitive or confidential information to AI tools.

    1. Ad HocEngineers paste anything into AI tools — secrets, customer data, proprietary code — without thinking.
    2. EmergingMost engineers know not to paste secrets; mistakes still happen with subtler data (PII, internal designs).
    3. DefinedClear rules exist for what can and can't go to AI tools, and engineers mostly follow them.
    4. ManagedApproved data flows are well-understood, supported by tooling (redaction, allowlists), and reinforced in reviews.
    5. OptimizedSensitive data exposure to AI tools is structurally prevented, not just discouraged; the team can describe the controls confidently.

  • Policy Adherence

    We consistently follow organizational AI usage policies.

    1. Ad HocEngineers are unaware of, or actively bypass, organizational AI policy.
    2. EmergingPolicy exists but is loosely followed; engineers sometimes use unapproved tools.
    3. DefinedEngineers know the rules and stay inside them on the things that matter.
    4. ManagedPolicy is visible in workflows (approved-tool lists, IDE plugins) and adherence is the path of least resistance.
    5. OptimizedPolicy is co-owned by the team; gaps and friction are surfaced to whoever maintains it rather than worked around.

  • IP & Licensing Awareness

    We understand intellectual-property and licensing risks in AI-generated code.

    1. Ad HocEngineers don't consider where AI-generated code came from or what licensing implications it carries.
    2. EmergingAwareness exists but no action; the team would struggle to answer auditor questions.
    3. DefinedEngineers know the basics (license type of suggestions, attribution norms) and avoid clear pitfalls.
    4. ManagedIP/licensing checks are part of review; the team can defend its position credibly if asked.
    5. OptimizedIP and licensing are an explicit, owned part of how AI-assisted code ships; controls and exceptions are documented.

  • Vulnerability Vigilance

    We actively check AI-generated code for security issues.

    1. Ad HocAI-generated code goes to production without security scrutiny; engineers assume it's fine because it works.
    2. EmergingStatic scanners catch the obvious issues; reviewers rarely look beyond.
    3. DefinedReviewers actively check AI-generated code for common security flaws (injection, secrets, unsafe defaults).
    4. ManagedThe team has a clear sense of where AI assistance increases security risk and applies extra scrutiny there.
    5. OptimizedAI-introduced vulnerability patterns are tracked, fed back into prompts and review checklists, and rarely repeat.

Knowledge Sharing

How openly the team captures prompts, shares wins and failures, collaborates across teams, and grows its AI skills.

  • Prompt Libraries

    We document and share effective prompts and AI patterns.

    1. Ad HocEvery engineer reinvents the same prompts; nothing is shared.
    2. EmergingA few useful prompts get pasted into chat occasionally and lost again.
    3. DefinedA shared place (repo, wiki, file) collects prompts and patterns; people contribute and consult it.
    4. ManagedThe library is curated, current, and used as a starting point for common tasks.
    5. OptimizedShared prompts evolve with the codebase; the library is a real productivity asset and demonstrably saves rework.

  • Learning Culture

    We openly share wins, failures, and experiments with AI-assisted development.

    1. Ad HocEngineers don't talk about how they use AI; wins and failures stay private.
    2. EmergingSome side-channel sharing happens (DMs, casual mentions); nothing structured.
    3. DefinedAI experiences come up in retros, demos, or standups; both wins and misfires are aired.
    4. ManagedSharing is a regular team rhythm — sessions, write-ups, or recurring agenda items.
    5. OptimizedThe team has a genuine culture of curiosity around AI; failures are valued as learning, not blamed.

  • Cross-Team Collaboration

    We exchange AI coding practices with people outside our immediate team.

    1. Ad HocOur AI practices stay inside the team; we don't know what other teams are doing.
    2. EmergingInformal cross-team chat happens occasionally; no real exchange.
    3. DefinedEngineers share notes across teams when it matters; useful patterns travel.
    4. ManagedActive cross-team forums or guilds exist for AI practices; participation is genuine.
    5. OptimizedThe team both gives to and learns from other teams continuously; AI practice spreads as a flywheel.

  • Skill Development

    We invest deliberately in growing our AI-assisted development skills.

    1. Ad HocSkill growth is accidental; engineers improve only when they happen to try something new.
    2. EmergingA few self-directed learners; most engineers stay at whatever level they arrived with.
    3. DefinedThe team allocates some explicit time to AI skill-building (sessions, pairing, reading).
    4. ManagedSkill development is a routine investment; new techniques are tried, evaluated, and adopted as a team.
    5. OptimizedContinuous, deliberate growth is part of the team's identity; engineers are visibly better at AI-assisted work each quarter.

Impact Measurement

How honestly the team tracks AI's effect on productivity and quality, and folds those lessons back into how it works.

  • Productivity Tracking

    We assess whether AI tools genuinely improve our delivery speed.

    1. Ad HocNobody knows whether AI is making us faster or slower; we assume it's helping.
    2. EmergingGut feel about productivity gets discussed; no signal beyond anecdote.
    3. DefinedThe team tracks some indicators (cycle time, PR throughput) alongside AI adoption.
    4. ManagedProductivity impact is tracked deliberately; the team can describe AI's effect with evidence.
    5. OptimizedProductivity tracking is honest about gains and losses; the team adjusts AI usage based on what the data shows.

  • Quality Metrics

    We evaluate whether AI assistance helps or hurts the quality of our work.

    1. Ad HocNo view of how AI affects defect rates, review churn, or maintainability.
    2. EmergingEngineers anecdotally notice quality patterns; no shared signal.
    3. DefinedThe team tracks quality indicators (defect rates, rework, review feedback) in the context of AI use.
    4. ManagedQuality impact is part of how the team thinks about AI; both positive and negative effects are visible.
    5. OptimizedQuality is a first-class lens on AI usage; the team has changed practice based on what it found.

  • Retrospective Integration

    We reflect on AI-assisted development during our retrospectives.

    1. Ad HocAI use never comes up in retros; it's invisible to the team's reflection.
    2. EmergingAI gets mentioned in retros occasionally, usually as a one-off remark.
    3. DefinedAI topics are a recurring strand in retros; the team discusses them when they matter.
    4. ManagedRetros consistently surface AI-related observations and turn them into actions.
    5. OptimizedAI use is a routine lens in retros; insights translate quickly into changed practice.

  • Continuous Improvement

    We adjust our AI usage based on feedback, outcomes, and lessons learned.

    1. Ad HocHow we use AI doesn't change; we're running on first instincts.
    2. EmergingOccasional adjustments based on individual frustration or a hot tip from elsewhere.
    3. DefinedThe team revises its AI practices regularly; changes stick when they prove their worth.
    4. ManagedImprovement is a real loop — measure, adjust, measure again — and the team can point to changes it has made.
    5. OptimizedContinuous improvement of AI practice is part of how the team operates; nothing about how we use AI is static.

When to use this health check

  • When your team has adopted AI coding assistants and wants an honest baseline of how effectively they're being used.
  • Before setting goals or making investment decisions around AI tooling, training, or governance.
  • When AI-generated code is raising questions about quality, security, or review rigor that the team wants to confront openly.
  • As a recurring checkpoint to track how your AI-assisted development practice matures quarter over quarter.
  • During a retrospective or team offsite to spark candid conversation about where AI helps and where it introduces risk.

Tips & tricks

  • Have each member rate independently before discussing, so the conversation surfaces genuine differences in perception rather than groupthink.
  • Focus the debrief on the dimensions with the widest spread of scores — disagreement usually points to the most valuable conversation.
  • Treat the maturity levels as a shared language, not a grade; the goal is honest reflection, not scoring high.
  • Pick one or two dimensions to act on between sessions rather than trying to improve everything at once.
  • Re-run the check each quarter and compare results to see whether deliberate changes are actually moving the needle.
  • Use the 'Not Applicable' option freely — not every dimension matters equally to every team or stage.

Frequently asked questions

Who should take part in this health check?
Anyone who writes, reviews, or ships code with AI assistance — typically the whole engineering team, including leads. Including a range of experience levels gives a more honest picture than surveying only enthusiasts or only skeptics.
How are the maturity levels structured?
Each dimension is rated on a five-stage scale from Ad Hoc through Emerging, Defined, and Managed to Optimized. The levels describe how consistent, deliberate, and effective the team's practice is, not how many tools it uses.
Is a higher score always better?
Higher levels reflect more mature, deliberate practice, but the real value is in the conversation and the actions that follow. A team that scores modestly but talks honestly about where it can improve gets more out of this than one chasing a perfect result.
How often should we run it?
A quarterly cadence works well for most teams — frequent enough to track progress after changes, but spaced enough that meaningful shifts have time to happen. Running it after a major tooling or process change is also valuable.
What do we do with the results?
Use the spread of scores to spot where the team agrees and disagrees, pick one or two dimensions to focus on, and turn them into concrete actions. Revisit those actions and re-run the check later to see whether they moved the needle.