How to Organize Company Knowledge for AI: Every Document Needs a Job
How to Organize Company Knowledge for AI: Every Document Needs a Job
Say you ask an AI to draft a renewal quote for one of your customers. You point it at the shared pricing folder. In there are the current price sheet, last quarter's sheet, a draft of a price increase nobody has approved, a list of discounts sales has agreed to over the years, and a meeting note titled "pricing final final."
Every one of those files is relevant, and the AI can find all of them. Only one of them should set the price.
Most advice about organizing company knowledge for AI is about the finding part: better folders, cleaner documents, smarter search. That work matters. But the quote goes wrong for a different reason. Nobody wrote down which file decides the price, which ones are background and what should happen when two of them disagree. Someone who has worked in sales for a year carries that in their head. The AI doesn't.
So start with the documents that shape real decisions and give each one a job. Write down who owns it, what it's for and who it's meant for. Then add when it stops being safe to use, how much weight it carries, and what the work should do when it's missing or contradicted. We call that record a Source Job Map.
For most files the job is obvious and nobody needs to write it down. Do it for the few where a wrong guess reaches a customer.
Finding a document and trusting it are different questions
When a team says "the AI used the wrong document," it can mean the right file wasn't reachable, or search missed it. It can mean the file was found but was out of date for this use, or was only ever meant as background. Or the person asking wasn't allowed to see it at all.
Each of those has a different fix. Better search won't tell the AI that the draft price increase isn't approved, and a permissions system won't tell it that last quarter's sheet stopped applying on the first of the month.
Standard ways of describing documents keep these questions apart. The Dublin Core metadata terms have separate fields for who a document is meant for, who may access it, when it was changed, when it's valid and what replaces it. Last quarter's sheet shows why the two dates are separate: it may have been edited yesterday and still have stopped applying on the first of the month. None of those fields says which sheet is your company's final word on prices. Someone has to decide that.
The six fields
A Source Job Map is a short record for one document in one use. Here it is for the current price sheet, used for renewal quotes.
Owner
Who answers for this sheet being right, and who decides when it gets replaced? In our example that might be the head of sales or whoever runs finance. It isn't necessarily the person who typed it up or the person who manages the folder. When the sheet disagrees with the discount list, or with the price in a customer's signed contract, this is who the question goes to.
Use
What decision does this document shape? "Pricing" is too broad to act on. "Sets the list price and standard terms on renewal quotes" is specific enough. If the same sheet also feeds a board report, that's a second use, and it may need its own entry.
Who it's for
Which person, team or AI task should be reading it, and in what work? The renewal quote is a sales task, so the sheet is meant for whoever drafts and checks quotes. Writing that down doesn't let anyone open it. Your access settings still decide who can.
Valid until
What makes this sheet questionable for this use? A date, an approved price change, a new region, a scheduled review. "Last modified on Tuesday" isn't a validity rule. If the price increase is approved for next quarter, the current sheet stops governing new quotes on that date, even though nothing in the file changed.
Weight
For renewal quotes, is this document governing, evidence, reference or an example? The current price sheet governs. The discount list is evidence: it shows what sales has agreed to before, which helps you spot a customer who expects a concession, but it doesn't set the price. The meeting note is reference at best.
Write down where that rule comes from: a pricing policy, or a call by someone with authority to make it. The map can't make that call. Say sales agreed a discount two years ago and nobody has decided whether it holds at renewal. Leave that gap on the record and take it to the owner before the quote goes out.
When it breaks
Where does the sheet enter the work, and what should happen if it's missing, expired, locked or contradicted? For the quote, a sensible rule is: if the current sheet can't be found or has expired, stop and ask the owner, and don't fall back to last quarter's prices. Once it's written down, whoever reviews the quote can check the work against it.
| Field | Question to answer |
|---|---|
| Owner | Who answers for this document being right, and who settles a conflict? |
| Use | Which decision or output does it shape? |
| Who it's for | Which person, team or AI task is it meant for, and in what work? |
| Valid until | What date, event or replacement makes it questionable for this use? |
| Weight | Is it governing, evidence, reference or an example here, and what rule says so? |
| When it breaks | What should the work do if it's missing, expired, locked or contradicted? |
NIST's voluntary AI Risk Management Framework asks teams to write down what an AI system is for, who will use it, the limits of what it knows and how people will use and oversee its output. An AI system documented that carefully can still quote last quarter's price if nobody wrote down which sheet applies.
Weight, status and access are three separate things
It's tempting to fold these into one rule: the approved document governs, the draft doesn't count, and whoever can open a file can use it. Each part of that goes wrong somewhere.
Take the draft price increase. Its status is draft, because nobody has approved it. Its weight for the renewal quote is evidence. If the customer's renewal falls after the planned change, the person reviewing the quote needs to know the increase may be coming. The draft doesn't set the price, and it isn't worthless either. Whether the person or AI drafting the quote can open it is a third question, settled by your access settings rather than either label.
| The question | Starting labels | |
|---|---|---|
| Weight | How much may this document shape this specific use? | governing, evidence, reference, example |
| Status | Where is the document in its life? | draft, current, under review, superseded, historical |
| Access | Who or what may open it or act on it? | your authentication and permission rules |
The same document can have different jobs
Last quarter's price sheet shows why the job belongs to the use and not just the file. For a new quote it's superseded and shouldn't set anything. But suppose the customer replies, "We were told last year our price would hold." Now the old sheet answers a different question, what they were quoted when they signed, and it may be the most important document in the folder.
| Last quarter's sheet, used to... | Weight | Status |
|---|---|---|
| Price a new renewal quote | doesn't apply | superseded |
| Check what the customer was quoted when they signed | governing, for that historical question | historical |
| Show the layout a new sheet should follow | example | doesn't matter here |
Most files only ever do one thing. Record a second use only when getting it wrong would change the work, as the customer's reply did here.
What the map can't fix
Search can miss the right document. Anthropic's write-up on contextual retrieval explains one reason. Documents get split into chunks for search, and a chunk like "The company's revenue grew by 3% over the previous quarter" no longer says which company or which quarter. A row from the price sheet cut off from its header has the same problem. The same post notes that a knowledge base under about 200,000 tokens, roughly 500 pages, can go into the prompt whole. Your pricing folder might fit, which solves finding the sheet but not choosing which one to follow.
Putting the whole drive into the prompt so the price sheet is in there somewhere isn't a cure either. A 2024 study of 20 models found that retrieving more documents can improve performance, but only a handful of the newest models kept their accuracy consistent above 64,000 tokens of context.
A model can also misjudge what it has in front of it. In a paper at ICLR 2025, researchers separated questions where the retrieved material was enough to answer from questions where it wasn't. The larger models they tested did well when it was enough, but often gave a wrong answer instead of declining when it wasn't. Smaller models often got it wrong or declined even when the material held everything they needed. For the quote, the worrying case is the current sheet not coming through and the model producing a confident price anyway. That's why the "when it breaks" field says to stop and ask.
And documents can carry instructions. Greshake and colleagues showed that text planted in material an AI is likely to retrieve can steer a vulnerable AI application, and wrote that effective defenses were lacking. If the discount list includes text pasted in from customer emails, that text came from outside your company. Labeling the list "evidence" won't stop an instruction hidden in it. That's a job for your security controls.
So fix the layer that failed. A sheet that search can't find, or that the wrong person can open, needs work on search, on how it's written or on permissions. The map is for the other case: the right sheet was found and the quote still used the draft price.
When you don't need one
If your billing system already holds the price list with effective dates, an owner and an approval history, treat that as the record. Copying those rules onto a separate card only gives you something else to keep up to date. The map earns its place where the decision isn't written down anywhere, or is written down where nobody can find it.
Where to start
Don't map your whole drive. Pick one piece of recurring work where using the wrong document would reach a customer, a public claim or a policy decision. For a lot of companies, that's pricing.
Write the six fields for the document that should govern it. Then try the bad paths. What happens if the current sheet is missing? If it expired yesterday? If the discount list says one thing and the sheet says another? If the honest answer to any of those is "the AI picks one," you've found a decision nobody has made yet. Take it to the owner.
Then look at what the entry changed. Did it change which document the work used, who checked the result or when the work stopped to ask? If it did, keep it and move on to the next document that matters. If it didn't, you didn't need it.
Running the quote on Supanova
Say the renewal quote ran on Supanova atoms. You'd write the project brief the way you'd brief a new salesperson: here's the customer, here's the renewal date, and here's the job each file does. Attach the current price sheet and the draft increase to the project so the text in them can be part of what your atoms work from. In the brief, say that the current sheet sets the price, that the draft is there so the quote can mention the planned change, and that if the sheet looks expired they should ask you, not reach for last quarter's numbers.
Finished work comes back for your review. Before anything goes to the customer, you can check which prices the quote used and whether the draft increase stayed a note instead of becoming the price.
You don't have to clean out the pricing folder. Pick the file you'd least want an AI to guess about, write its job down, and put that in your next brief.
Start your first project at supanova.team.
Sources
- DCMI Metadata Terms, Dublin Core Metadata Initiative
- NIST AI Risk Management Framework Core
- Anthropic, Introducing Contextual Retrieval
- Leng et al., Long Context RAG Performance of Large Language Models
- Joren et al., Sufficient Context: A New Lens on Retrieval Augmented Generation Systems (ICLR 2025)
- Greshake et al., Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection