Engagement · Assessment first
Training a model is easy. Knowing whether it got better is the work.
Anyone can finish a fine-tuning job and show you a loss curve. The question that matters is whether the result beats what you already run — and whether you would be allowed to ship it if it did not.
What usually brings people here
A fine-tune that finished is not a capability that improved.
These are the situations we are asked about most. Several of them are reasons not to train at all, which is a legitimate outcome of an assessment.
- You have domain data and no idea whether it is enough to be worth training on.
- A vendor delivered a fine-tuned model and no evidence that it beats the base model.
- Prompting mostly works, and you cannot tell whether training would add anything.
- You need the model to stay inside your own environment for compliance reasons.
- Someone shipped a new version and nobody can say what changed or how to go back.
- You need to explain to a reviewer why this model version is in production.
How we approach it
The evaluation decides. Not the training run.
We treat a candidate model as guilty until it beats the baseline on evidence. Most of the value of an engagement is in the gate, not in the GPU time.
A baseline before anything
If there is no measured baseline, there is nothing to improve on and no way to prove you did. That comes first, every time.
A gate that can reject
A candidate that loses to production does not ship. That has to be true before training starts, or it will not be true afterwards.
Every version reversible
Model versions are artefacts with lineage. Going back to the previous one is a decision, not an incident.
What an engagement produces
One of our own runs — the one where the gate said no.
The candidate scored 0.5667 against a 0.6167 production baseline. It was rejected, production never changed, and the whole run is sealed in the audit ledger. This is the deliverable: not a model, but a defensible decision about a model.
2× T4 allocated
OKQLoRA session active
OKcandidate 0.5667
WARNbaseline 0.6167
FAILassistant-v6 rejected
HELDcurrent version unchanged
OKAn archived run from OneForge's own control plane, on our own model. It is our standard demonstrated on ourselves, not a customer result.
How it starts
We tell you whether to train before we offer to train.
An assessment can conclude that your data is too thin, that retrieval would serve you better, or that prompting is already at the ceiling of what the task needs. Those are useful answers and they are cheaper than finding out later.
Describe the task
What the model has to do, who checks its output today, and what an error actually costs you.
Data and baseline review
How much usable data exists, how it is labelled, and what the current approach already scores. No baseline, no verdict.
A verdict you can act on
Train, do not train, or do something else first. With the reasoning, so your own team can check it.
A pilot, only if viable
Evaluation criteria and the release gate agreed in writing before any GPU is allocated.
Who does the work
Founder-led, which is why we run 2 at a time.
Engagements are led by the founder directly rather than handed to an account team, so the person who built this evaluation stack is the person running yours. That is a real constraint, and we would rather state it than queue you behind something.
One principal, start to finish
The same person reviews the data, sets the evaluation and reads the result. Nothing is re-explained to a delivery team that was not in the room.
A baseline you can check
You get the measured baseline and the evaluation set before training starts, so the verdict afterwards is not ours to argue.
The artefact, and its gate
Not just weights: the lineage, the release gate configuration, the rollback path and the audit record of the run.
Handover, not dependency
Documentation and a walkthrough so your team can retrain and re-evaluate. Continued operation by us is a separate agreement, agreed up front.
What we are not saying
- 2 engagements at a time is a real ceiling, not a scarcity tactic. If they are all running, we will tell you when rather than take a deposit.
- The run shown above is our own model, not a customer's. We have no customer training case studies to show you yet, and will not imply otherwise.
- If the assessment says prompting or retrieval is the right answer, that is the answer you will get — even though it is the smaller engagement.
Tell us what the model has to get right.
Bring the task and, if you have it, whatever you use to judge the output today. That is what makes an assessment worth doing.
Your AI should not stop at a demo.
Route intelligence through Core, govern it with TheOne, persist the work in OneMission, execute through OneClaw and keep verified outcomes in OneField.