How to Set Data Retention for an AI Onboarding Company Brain

Keep the answer, the evidence and the conversation on three different clocks, and keep conversations out of the index that serves answers. A question about payroll, an accommodation or a manager can sit in logs long after it served its purpose, and a copy of it in a vector index is the hardest one to delete. We recommend a short, approved window for question text, the HR records schedule for small completion and acknowledgment records, and the knowledge-management lifecycle for source documents.

Why one retention rule cannot cover an AI onboarding assistant

An onboarding company brain is more than a chat window. One request crosses the employee's prompt, identity context, retrieved passages, a generated answer, citations, feedback, analytics and sometimes a human escalation. AWS describes retrieval-augmented generation as a process that retrieves external information and adds it to a model request before the model produces an answer, and each of those layers can write its own record.

Those records serve different purposes. A policy acknowledgment may need to stay available as operational evidence. A raw question about a medical accommodation needs only enough lifetime to reach the right person. A retrieval trace can help diagnose a wrong answer without ever joining a permanent employee profile. Encryption, role-based access and audit logging decide who can read a record while it exists. None of them supplies an expiry date, and retention is what decides why the record still exists at all.

The NIST Privacy Framework treats privacy as a risk-management problem tied to systems, data processing and organizational decisions. Apply it to the onboarding workflow: start from the processing purpose and the effect on the employee, then choose the smallest record and the shortest useful lifetime that still supports the task.

Inventory every record the system creates

Hold off on a blanket rule such as "keep chat for one year" until you know what is written. For each record, note its purpose, owner, storage locations, deletion trigger and any reason to pause deletion. Cover at least these types:

1. Raw employee questions and attachments.
2. Identity and role attributes sent with a request.
3. Retrieved passages, source identifiers and ranking scores.
4. Generated answers and citations shown to the employee.
5. Ratings, corrections and free-text feedback.
6. Human escalation messages and case notes.
7. Completion events and required policy acknowledgments.
8. Technical logs, security alerts and aggregated analytics.

The inventory should name actual systems. "AI platform" is too vague when the prompt also appears in an application log, an observability tool, a support ticket, a warehouse, a backup and a vendor console. Ask the engineering owner to trace one test question from submission to expiry; the exercise finds copies that the architecture diagram missed. Do not rely on a vendor's default history setting either. Defaults change, and they may not cover exports, observability data or integrations, so record the configured setting and verify the behavior through the API or admin surface your team has.

Keep source documents separate from conversation records. The handbook page about expense approval follows the knowledge-management lifecycle, and the employee's question about a disputed expense follows a conversation and case lifecycle. Deleting one should not silently delete the other.

Purpose before duration

Write one primary purpose for every record type. If the team cannot state it in a sentence, it cannot defend keeping the record or verify when it stopped being useful. Narrow purposes look like these:

• deliver the current answer;
• diagnose answer quality during a defined review window;
• complete a named onboarding task;
• preserve a required acknowledgment;
• investigate a security event;
• resolve a case handed to a human.

Labels such as "analytics", "AI improvement" or "future use" let every record survive every cleanup. If product analytics needs only counts, aggregate the event and drop the prompt. If quality review needs examples, sample a limited set, redact unnecessary identity fields and give the sample its own expiry.

The Microsoft Purview retention overview distinguishes retaining content from deleting it and explains how retention policies and labels can govern lifecycle behavior. Tooling differs by stack, but the design rule carries over: retention and deletion are explicit actions with precedence, scope and disposition behavior, not side effects of storage defaults.

The retention matrix

Turn the inventory into rows an engineer can execute. Each row answers these questions:

• What event starts the clock: record creation, task completion, onboarding completion, case closure or employment termination?
• What event deletes the record?
• Which service owns that deletion?
• Which downstream copies receive the deletion event?
• Does an approved hold pause deletion?
• What evidence proves the deletion ran?

A practical starting model keeps transient retrieval context only for request processing, keeps troubleshooting traces for a bounded quality-review window, and keeps task completion records under the organization's approved HR records schedule. These are design categories, not universal durations. The privacy or records owner approves the actual periods for your company and jurisdiction.

Do not copy the HR records schedule onto every AI artifact. A completion event and a raw prompt are different records. An acknowledgment needs an employee identifier, a policy identifier, the policy version, a timestamp and a result, and usually not the conversation that preceded it, so the small event can outlive the raw chat. Once a question enters an HR, IT or security case system, that system's owner and schedule apply, and the chat service should store the case reference instead of copying case notes back into the onboarding transcript.

Collect less in the first place

Deletion is easier when the data was never stored. Tell employees what the assistant is for and where sensitive requests go. Do not invite health, payroll, immigration, grievance or performance details into a general chat when a protected workflow exists, and put the human route beside the chat rather than after a disclosure. At request time, send only the attributes retrieval needs: a role and a region may select the right policy, while a home address, birth date or full HR profile usually does not. Keep authorization decisions server-side so the prompt does not carry a readable bundle of employee attributes.

In quality logs, redact or tokenize identifiers when diagnosis does not need identity, and store source identifiers instead of full retrieved passages where an authorized reviewer can reproduce the retrieval. Turn off body logging at gateways and error handlers unless a reviewed use case needs it, because error traces are a common place for prompts to survive outside the planned retention system.

The GitLab onboarding handbook shows onboarding organized around explicit tasks, owners, access requests and support roles. That helps retention, because records can be tied to a task or handoff instead of one indefinite conversation.

Keep conversations out of the answer index

We recommend that the retrieval index hold governed source documents only, never past questions and answers. A system that folds conversations back into retrieval turns every employee question into a permanent, searchable asset. Once a question has been chunked and embedded next to documents, it is harder to find and delete, and one hire's disclosure can surface in another hire's answer. If you want to learn from conversations, use the bounded, redacted and expiring review sample described above, and turn recurring questions into reviewed source pages that carry an owner.

Delete everywhere, then prove it

Deleting the primary chat row is not enough. The same content may sit in a search index, vector store, analytics export, cache, queue, ticket, evaluation set or backup catalog. Run deletion as a workflow keyed to a stable record identifier, with an acknowledgment from every participating store:

1. The system marks the record expired from its approved trigger and schedule.
2. The primary store blocks normal reads and emits a deletion event.
3. Each downstream service deletes or irreversibly de-identifies its copy.
4. Search and vector indexes remove associated chunks and metadata.
5. Caches expire and queued work referencing the record is cancelled.
6. The workflow records service acknowledgments without retaining the deleted content.
7. A verifier checks that the identifier no longer resolves in active systems.

Make the event idempotent, so a retry finishes missing work without recreating data or failing because one service already deleted its copy. Set a deadline for acknowledgments and alert an owner when any service misses it.

Selective removal from backups may not be possible in every design. In that case, limit backup access and enforce backup expiry under the approved schedule. We also recommend a deletion ledger: the identifiers of expired records and the dates they expired, with no content. Replay the ledger after every restore, before the assistant serves a single answer, so a restoration cannot bring expired data back into production. Test the restore path as well as the live deletion path.

Holds without permanent exceptions

A legal, security or employee-relations matter may require a temporary hold. Implement it as an explicit exception with an approver, reason code, scope, start date, review date and release action, so an informal message from a manager cannot suspend a whole deletion job. Scope the hold to the smallest relevant records: a case about one equipment request should not preserve every onboarding question from every employee, and the workflow can skip held identifiers while it keeps expiring unrelated data.

Review active holds on a schedule. When a hold ends, the record returns to its original lifecycle and is deleted promptly if its normal period has already elapsed. Keep evidence that a hold existed and was released, without copying the held content into the hold registry.

Test the lifecycle with synthetic records

A policy document and a scheduled job do not prove that copies disappear. Create synthetic onboarding records with known identifiers and run these cases:

• normal expiry after the configured trigger;
• early deletion through an approved request workflow;
• propagation to the vector index, logs, analytics and case integrations;
• a hold that pauses only the selected record;
• release of an expired held record;
• retries after one downstream service is unavailable;
• backup restoration without reviving expired active data;
• access attempts during the period between expiry and final deletion.

For each test, record the expected stores, the deadline and the result. A pass proves absence by identifier in active systems and shows deletion acknowledgments from the downstream services; it does not preserve the original prompt as proof. A job that exits cleanly after skipping a disconnected service has not passed, which is why per-store acknowledgments and a lookup after expiry are the measure, not job success.

Also test the employee experience. After a transcript expires, the company brain should still answer from current source documents and should not pretend to remember personal context. If an open task needs continuity, keep a minimal task state with its own purpose and expiry instead of retaining the whole conversation.

Owners, and the first test

Assign one accountable owner for the retention matrix and one technical owner for the deletion workflow. People Operations identifies task and employee-record purposes, privacy or compliance approves schedules and holds, engineering maps copies, implements triggers and maintains the tests, and knowledge managers govern source documents separately from conversation data.

The Kipwise employee onboarding workflow centers onboarding on assigned reading and searchable company knowledge. That separation is useful here: preserve maintained knowledge as the source of future answers, while expiring personal conversation data according to its narrower purpose.

Start with one end-to-end test question. Trace every copy, assign its purpose, approve its deletion trigger and run the expiry workflow in a non-production environment. Do not extend the assistant to more onboarding tasks until that test shows the prompt, answer, retrieval trace, analytics copy and escalation link all follow the intended lifecycle.