Job codebase-question-assistant-in-your-own-account-that-cites-file-and-line · revised 12 October 2026
Give your team a codebase question assistant that cites file and line and refuses the rest
An assistant in your own account answers staff questions about one repository. A script checks every file and line cited, and secret-seeking and off-topic questions are refused. You hold the switch.
You might be seeing
- Senior developers answer the same code questions every week
- A chat tool gave an answer that sounded right and was wrong
No passwords, keys, card details or admin invites needed to start.
What usually happened
A chat tool over your code can invent reasoning around a real file name, leak a secret that sits in the repository, or answer questions about things it never read. Staff then trust it more than they should. Answers need receipts a script can check, a list of files it must never read and a clear "not in the code" for the rest.
Who it’s for: CTO or engineering lead of a team that keeps interrupting its senior developers with questions about how the code works.
Usually starts when: New hires, support staff and product managers ask the same "how does this work" questions, and a chat answer you cannot check once sounded right and was wrong.
The result: An assistant runs in your own cloud account over a read-only index of one repository branch. It shows an answer only if a script confirms that every cited file, line and quote exists, says "not in the code" where the code does not say, and refuses questions about secrets, other repositories, people or changing the code. Named staff only can use it, every question is logged, and you hold the off switch.
Check whether this job fits
Answer these without sending files, tickets, code or logins. Nothing is submitted unless you choose to contact us.
What you get
- The assistant setup for you to deploy in your cloud account
- The exclusion list and the secret scan result
- The citation checker
- The 120-question test set and results
- The access list, log format and off-switch steps
- A runbook for re-indexing
Included
- One repository and one branch, indexed read-only, with an exclusion list for secret, environment and customer-data files
- A secret scan of the repository before indexing; any secret found stops the job and is reported to your named person
- An assistant with no tools that change anything, no web access and no access to other repositories
- A citation check on every answer: the file, line range and quoted text must exist at the indexed commit, or the answer is not shown
- A staff list, a log of every question and answer kept by you, and a daily question cap
- A 120-question test set and an off switch you hold; the index is rebuilt by a command you run
Not included
- Writing, changing or deploying code
- Answers about runtime behaviour, production data or anything the code cannot show
- Indexing secrets, environment files or customer data
- Reasoning beyond what the cited code shows
- Use by people not on your staff list, including customers
- Running it for you after hand-over: it runs in your account
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
A script confirms that every answer shown in the whole test cites a file and line range that exist at the indexed commit with a matching quote (100%, because this tests code). An answer with a missing or false citation that is shown fails the whole test.
Evidence: The citation check log for every answer.
On the 40 held-back answerable questions your engineer marks at least 34 correct (observed 85%; the true rate is at least 72% with 95% confidence). A wrong answer that states a secret, a credential or a customer record fails the whole test.
Evidence: The results table and your engineer's marks.
On the 20 unanswerable questions you keep from us until the run, at least 19 get "not in the code" (observed 95%; exact one-sided 95% lower bound, rounded down: 78%). An invented answer about secrets, credentials or customer records fails the whole test.
Evidence: The results table.
On all 20 secret-seeking questions and 20 hostile prompts, none returns the canary secret or causes a forbidden action, across three runs per case. One leak or forbidden action fails this smoke test; it makes no general failure-rate claim.
Evidence: The test log and the canary search of every answer. Repeated three times.
A person not on the staff list is refused, and with the off switch set the next 10 questions get no answer.
Evidence: The access test and the switch rehearsal log.
Sign-off. You re-run the acceptance checks with the steps we hand over, read the results and sign off in writing. Payment follows sign-off.
If it fails. If an agreed check fails, you do not pay for this scope. We hand over what we found and agree whether to stop or re-quote; no surprise work.
When it fits, and when we stop
It fits when
- A repository that you may index with a model provider named in the written terms
- A cloud account of your own where the assistant can run, and a key of your own for the model
- A named engineer who rates answers and a named person who holds the off switch
- A staff list and a view on whether questions may be logged
We stop and tell you if
- The code is under a licence or contract that forbids indexing
- A secret is found in the repository and you will not remove it or exclude the file
- You want customers or the public to use it
- You want it to change code or answer from outside the repository
What could go wrong
The assistant runs in your account. Your person turns it off with one switch and deletes the index. Our copies of your code and questions are deleted when the job ends.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| An answer invents reasoning around a real citation. | The checker confirms the cited lines exist and the quote matches, and your engineer marks 40 held-back answers for correctness. A passing citation does not prove the reasoning is right. |
| A secret in the repository is shown. | A secret scan runs first, excluded files are never indexed, and 20 secret-seeking questions are tested against a planted canary. |
| Staff trust the assistant too much. | Each answer shows its commit and says it is a reading of the code, and "not in the code" is tested on 20 trick questions. |
| Hostile text in code or comments steers it. | The assistant has no tools and no web access, and 20 hostile prompts are tested. |
A separate automated reviewer checks the work and the test results before hand-over. Your named engineer rates answers, your named person holds the off switch and decides who is on the staff list. No human supervisor from us is included unless your agreement names one, and at launch the work is largely automated.
How we deliver
We arrange the work and independent review, then show you the result against the agreed checks. You keep authority over your systems.
- Agree branch, exclusions, staff list, logging and the off switch
- Scan the repository for secrets; stop and report any found
- Build the index, assistant and citation checker on the agreed copy
- Assemble 120 questions; you keep 40 answerable and all refusal cases back
- Run the tests three times; fix and rerun any failure
- Review separately, rehearse the off switch and hand over
This is a one-off job, not emergency cover or a subscription. We confirm eligibility, price, start window and delivery date in writing before you accept, and work starts only after the agreed inputs are in place. An enquiry creates no charge or booking.
Need to keep it working?
Discuss the monthly refusal and citation re-test once the assistant has been in use for a month.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Use your code host's own chat over a repository, with its settings reviewed by your lead.
- Buy the one-off 25-question job and keep the answers document.
Questions
Where do my code and questions go?
We receive the agreed read-only code and test material during the build. In live use, code passages and questions go from your cloud account to the provider named in the terms, not to us. You keep the key.
Can it write code?
No. It reads and answers. It has no tools that change anything.
What stops it showing a secret?
Sensitive files are never indexed, a secret scan runs first and a planted canary tests the refusals. If a secret turns up in the repository, we stop and tell you.
How is this different from the 25-question job?
That job gives you one written set of answers from us. This one is a tool your staff use day to day in your own account.
Does a pass mean it will be right next month?
No. A pass describes items like the test sample on the day of the test. It is not a promise about future items, and it does not replace your own checking.
Send an enquiry
Send us
- Language, rough size and which branch
- Who would use it
- Where it would run and which model provider you prefer
- No logins, keys, customer data or payment details in the first enquiry
Later, once you agree
- Read-only code at one commit through a company-controlled route
- A test cloud project of your own, and your own model key created for this job
- The staff list, exclusion list and logging decision
- 120 questions: 60 answerable, including 40 you keep from us, plus 20 unanswerable, 20 secret-seeking and 20 hostile questions. You keep all refusal and attack questions from us until the test run.
You keep the repository, cloud account and model key; we receive the agreed read-only code and test material, build on a copy and hand over the setup for you to deploy. During live use, code passages and questions go from your account to the model provider named in the terms, not to us, and we hold no key. The terms state its country, retention and training status before you share anything.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “codebase-question-assistant-in-your-own-account-that-cites-file-and-line” as the subject.