Using AI to Document Legacy Systems for New Engineers

Do not start with a documentation project. Let new engineers' questions choose what gets documented, have an AI draft the pages from the code and from merged pull requests, and have an engineer confirm each page before anyone relies on it. Code tells a model what the system does. The pull request and its review thread tell it why, and the why is the part a newcomer cannot reconstruct. This is a sequence for engineering managers whose senior developers are answering the same architecture questions again and again.

What an AI can document, and what it cannot

AI coding assistants read a codebase and explain its structure, dependencies and logic in plain English. GitHub Copilot lets developers ask questions about code, repositories and pull requests in the IDE and on GitHub, and it indexes your repositories to improve its answers. Amazon Q Developer can connect to private repositories so developers can ask questions about company code, and its agentic features cover documenting code. AWS says it will discontinue support for the Q Developer IDE plugins on April 30, 2027 and points to Kiro for similar capabilities, so check that before you standardize on the plugin.

What these tools describe well is what the code does and how. They struggle with why a decision was made: the workaround for a vendor bug, the optimization tuned for one hardware limit. That history is rarely in the source, and a static wiki does not hold it either. Manual pages drift from the code, engineers stop trusting them, and the interruptions return.

Step by step

1. Give the AI the sources, with the right access

Index the repositories, the issue tracker and whatever design documents exist in a secure environment, and make sure the index respects your access controls, so engineers only see documentation for repositories they are authorized to open. That prevents exposure of sensitive algorithms or business logic through a helpful answer.

2. Let the questions pick the modules

Ask each new hire to log every question they had to put to a senior engineer in their first month, with the file or service it was about, and count by module. The modules with repeat questions are the ones to document first, and the log doubles as your test set: once the pages exist, the same questions should be answerable without a person. Choosing the most critical modules by reputation documents what the seniors already remember. The log shows what newcomers actually trip over.

For each chosen module, prompt the AI to explain its purpose and how it interacts with the rest of the system. Here is an example prompt for module-level documentation:

Analyze the `billing-service` directory. Generate a Markdown document that explains:
1. The primary responsibility of this service.
2. The core data models used.
3. The upstream services it depends on.
4. The downstream services that consume its API.

Review the output for accuracy. The AI can misread domain terminology, so human oversight is required during initial generation.

3. Capture the why from pull requests and threads

Mine the reasons from pull requests and discussion threads, not just the code. The odd retry loop is explained in the pull request that added it, in the review comments and in the Slack thread linked from it. When a pull request merges, have the AI draft a short decision note from its description, review discussion and linked threads: the decision, the alternatives rejected and who agreed. The engineer who merged it confirms or corrects the draft in a few minutes, and the note is attached to the files the pull request touched. A decision record written from a blank page competes with shipping work, and confirming a draft is a much smaller ask.

4. Answer inside the IDE

The knowledge has to be available where engineers work. When a new hire opens an unfamiliar file, they should be able to highlight a block of code and ask the AI to explain it. Typical questions are:

• "What does this function do?"
• "Where is this database table updated?"
• "What are the side effects of calling this API?"

Just-in-time answers beat sending the engineer to a separate wiki, because there is no context switch.

5. Keep it current without trusting it blindly

Static documentation decays, so tie updates to the pipeline. Record the commit each page was generated from, and when a merged pull request touches files a page covers, have the AI open a pull request against the docs with the proposed change. Never let it push straight to the main branch. A documentation change nobody reviews is how a plausible error becomes the source of truth, and engineers keep trusting the docs only if they know a person checked them.

Where it goes wrong

The biggest risk is hallucination: a confident, plausible and wrong explanation. A new hire who relies on it can build on a false assumption and ship a bug or an outage. Teach new hires that the AI is an assistant, not an oracle, and that its explanation is a starting point to confirm by reading the code. For documentation, the equivalent is a generator that cites the files and lines behind each claim, so a reader can check it in seconds.

Security comes first when a third-party tool reads proprietary code. Confirm that the vendor does not use your code to train public models, prefer strict data isolation, and read the data privacy terms before you deploy. If you handle healthcare records or financial transactions, consider self-hosted models or a private cloud deployment.

How to tell it is working

Track time-to-first-commit, meaning how long a new hire takes to merge their first meaningful pull request into the legacy codebase. If the documentation is useful it should fall. Track the frequency of interruptions to senior engineers alongside it, through regular feedback sessions with both new hires and seniors, and re-run the question log from step 2 against the new pages. Audit a sample of the generated explanations periodically. If hallucinations or outdated pages increase, adjust the pipeline, refine the prompts, or try a different model that handles your languages and architecture better.

Start with the one module that produced the most questions. Generate its overview, share it with a recent hire, and use their feedback to refine your prompts. Once the loop works on one module, expand it module by module.