The First Thing You Get Is a List
A supervisory exam does not begin with a meeting. It begins with an information request: a list of documents, policies, data extracts, and loan files the examiner wants before anyone sits down. By the time you are in a room, the examiner has already read your policies and pulled your files, and the exam is a test of whether what you do matches what you said you do. An operation that runs AI agents in the origination or servicing path has to answer that request the same as any other, except that some of the questions in it now point at a machine, and the exam procedures were written before a machine was the thing being examined.
I work with chief compliance officers through this, and the failure I see is not a lender doing something wrong. It is a lender that did everything right and cannot prove it in the time the examiner gives. The agent made good decisions, and the record of why it made them is scattered, or reconstructable only by someone who understands the system, or not kept at all. An exam does not grade your intentions. It grades your evidence, and an AI-assisted operation lives or dies on whether the evidence was built as the agent worked or has to be assembled after the request arrives.
The Manual Is Public. Read It Before the Examiner Does.
The CFPB's expectations are not a mystery. The Supervision and Examination Manual is public, and so are the Mortgage Origination Examination Procedures, which lay out, module by module, exactly what an examiner reviews. The procedures are organized into modules covering the business model, advertising and marketing, loan disclosures and terms, underwriting and appraisals and originator compensation, closing, privacy, and fair lending, each with specific items the examiner works through. The procedures say the examination runs in conjunction with a review of the compliance management system, including how the institution handles consumer complaints. None of this is hidden, which means the request that arrives is largely predictable, and an operation that has read the procedures knows what it will be asked to produce.
The part that is new is that each module now has a version of the same question when a machine does the work. The underwriting module asks how a decision was made and whether it was consistent and compliant. When an agent made the decision, that becomes: how did the model decide, on what inputs, against what rule, and who reviewed it. The fair-lending module asks whether the process produced disparate outcomes. When an agent drove the process, that becomes a question about the agent's behavior across protected classes. The disclosures module asks whether the right disclosure went out on the right clock. When an agent managed the clock, that becomes a question about the agent's timing logic and its record of delivery. The modules did not change. The thing answering them did, and the operation has to be ready to answer in the module's own terms.
What an Examiner Actually Asks About the Machine
Across the modules, the questions an examiner raises about an automated decision come down to a few, and they are the questions to be able to answer before the request lands. What did the agent decide, and on what inputs. What rule did it apply, and could it cite the rule. Where did it stop and ask a human, and did the human actually review. How do you know when the model is wrong, and what do you do when it is. How is the vendor that supplies the agent governed. None of these is answerable with a confident sentence about the technology. Each is answerable only with a record.
So the preparation is not a narrative about how good the agent is. It is the evidence behind each question. For a decision, the file has to show the inputs the agent used, the rule it applied with a citation, and the output, for the specific loan the examiner pulled, not in aggregate. For the human-review question, the file has to show where the agent escalated and what the reviewer did, because an examiner testing a human-in-the-loop control will pull files and check that the human actually touched the ones the control says they touched. For the "how do you know when it is wrong" question, the answer is the monitoring and the sampling over the agent's cleared work, documented as a control with thresholds, which is also what the model-risk guidance in SR 11-7 expects of any model in the decision path. For the vendor question, the answer is the third-party risk file on the agent's provider, inside the institution's oversight program rather than beside it.
The File That Answers in a Sitting
The difference between an exam that goes well and one that does not is usually whether a single loan file can be made to explain itself. An examiner pulls a sample, picks a loan, and asks what happened on it. If the answer requires an engineer to query a log, a processor to recall a conversation, and a manager to explain a policy that was never written down, the exam has found its weakness, and it was not the agent's decision, it was the inability to show the decision.
The file that answers in a sitting is one where every action the agent took was recorded as it happened, with the inputs, the rule cited, the reasoning, the output, and, where a human was in the loop, what the human did. For a disclosure, the record shows the triggering event, the clock it started, and the delivery, tied together. For an income calculation, the record shows the documents, the arithmetic, the rule, and the underwriter who confirmed it. For an adverse action, the record shows the reason, in the terms ECOA requires, traceable to the facts in the file. The examiner's question "why did this happen" is answered by opening the file, not by convening the team. That is the standard to build to, and it is a higher one than most paper-and-queue operations ever had to meet, which is the quiet advantage of doing it with agents: the record can be complete because the machine leaves one by default, if the system was built to keep it.
It Is Not Only the CFPB
Mortgage supervision is not only federal. State regulators examine licensed lenders and servicers too, and the California DFPI, the New York DFS, and their counterparts run their own exams against their own requirements, sometimes tougher than the federal floor and increasingly focused on automated decision-making specifically. An operation preparing for supervision has to think in terms of every regulator that can walk in, and the good news is that the evidence that satisfies one largely satisfies the others, because they are all asking the same underlying question in different words: can you show me how this decision was made, and can you show me you would catch it if the system got it wrong. Build the record once, to the highest bar among your regulators, and most of the exam is already answered.
What We Do Before the Request Arrives
Exam readiness is not a project you start when the request lands. It is a property the operation either has or does not, decided long before, by whether the evidence was designed in. In the deployments we run, the record is built as the agent works: every decision cited to the rule and the document, every escalation and its human resolution logged, every disclosure clock and its delivery tied together, every monitoring threshold and breach recorded. We run a mock exam against the public procedures before go-live, pulling our own sample and answering the examiner's questions against real files, because the first time you find out your evidence does not hang together should not be when a regulator finds out with you.
The posture I recommend to every compliance officer is to treat the published exam procedures as the specification for what the operation has to be able to prove, and to build the evidence to that specification from the first live file. At Sei we design the agent so that the Evidence Trail is the file an examiner reads, cited and replayable, rather than something reconstructed under a deadline, because an exam is a document request before it is anything else, and the operation that can answer it in a sitting is the one that decided, before the agent ran, that every decision would keep its own proof. The exam is coming for everyone who runs regulated credit. The only variable you control is whether the answer is already written.
Pranay Shetty
CEO & Co-Founder