Technology comes after understanding.
Before I choose a model, a framework or an automation platform, I want to understand the problem and the system it lives in. AI is one tool the system might use. It isn't the starting assumption.
The loop
Problem
What outcome are we trying to produce, and how does it happen today? Start with what we're trying to cause, not with what we want to build.
System
Map the people, process, technology, data, timing and ownership around the problem. Most failures live in the handoffs between them, not inside any one tool.
Hypothesis
Make the belief explicit: changing X should produce Y, because of Z. If we can't say it that plainly, we don't understand it yet.
Build
Build something real enough to test the riskiest assumption, inside the real operating environment rather than as an isolated demo.
Evidence
Look at what actually changed and compare it with what we expected. Decide in advance what result would prove us wrong.
Learn
Whether it worked or not, understand why. Then find the next constraint and go again.
↺ back to Problem
Questions I ask before building.
- What outcome are we trying to produce?
- What has to be true for it to work?
- Where does information move, and where does ownership change?
- What is actually constraining the outcome?
- Which assumptions haven't been tested?
- What should stay deterministic, and where can AI genuinely help?
- How will we know whether it worked?
Find the constraint.
Automating the greatest number of tasks doesn't necessarily produce the greatest improvement. The important question is what's currently constraining the outcome.
Try it: automate any step of a pipeline and watch the output →
Deterministic or probabilistic?
Not every step needs an LLM. Every Lab demo marks each step with one of these three.
Deterministic
If something can be handled reliably with rules, a query or an integration, that's often the better engineering choice.
AI / probabilistic
Where language, ambiguity, interpretation or synthesis create real value.
Human judgment
People stay in the loop where their judgment matters most.
AI when it earns its place.
I try not to use an uncertain tool on a problem that has a certain answer. Pick a job to see where it belongs, or sort them all.
Deterministic
Code, rules, queries, automation
- Validate whether a field is presentCode
- Look up an existing customerDatabase or API
- Apply a known eligibility ruleRules
- Move data after a known eventAutomation
- Transform a known data structureCode
AI / probabilistic
A language model
- Summarize an unstructured conversationLanguage model
- Interpret ambiguous intentLanguage model
- Compare messy language against a flexible standardLanguage model
Human judgment
A person who owns the outcome
- Make a consequential exception with incomplete evidenceHuman judgment
A language model can technically do several of the jobs in the first lane. That doesn't mean it should: every probabilistic step is another place the system can be wrong in ways that are hard to predict.
Deterministic where possible. Probabilistic where useful. Human where necessary.
Building it isn't enough.
An AI system that works in a demo still has to answer these before it goes near a real operation. Here's how my builds answer them.
- Can we tell when it's wrong?Every Diagnostic finding carries a confidence label, and a clean result over incomplete data is reported as "not enough evidence," never as clean. Business Diagnostic
- Can we reproduce the decision?The recruiting system keeps a decision trace for every candidate: what they said, which rule used it, and who decided. AI Recruiting System
- Do we know which evidence it used?The knowledge demo cites the passage behind every claim, and every sentence in a Diagnostic report is traced to its source. Knowledge Intelligence
- What happens below the threshold?An unclear answer goes to a recruiter instead of being guessed, and missing evidence produces a refusal, not an answer.
- Can a person override it?A recruiter can override any rule, and every Diagnostic threshold belongs to the business, versioned and printed in the report.
- Did it change the outcome?The Diagnostic follows the same group of leads across later exports and compares them, instead of comparing unlike populations.
From finding to running.
The loop above is how I think. This is how a change actually gets into an operation and stays there.
- DiscoverWalk the process with the people who run it, and get the real exports.
- SpecifyWrite down what will change, the data it touches, and how we'll know it worked, before any code.
- BuildThe smallest version that tests the riskiest assumption.
- IntegrateConnect to the systems already in place instead of replacing them.
- TestOn data where the right answer is known, including the ways it can fail.
- DeployInto infrastructure the business owns.
- TrainShort guides for the people who use it every day.
- MeasureCompare what changed with what we expected, then find the next constraint.