Skip to content
RebellionTechGet Audit
Research & methods

The questions we have not answered yet

We are an engineering firm, not a lab. What we call research is the set of unresolved problems we keep hitting in production — and being straight about the ones we have not solved.

6
Active questions
5
Model strategies
6
Stated limits
Open questions

What we are still working out

Research here means unresolved engineering questions we hit repeatedly in production, not papers. Each of these changes how we build, and each has an open end we have not closed.

01

Retrieval quality

Why does the right document not come back?

Chunking strategy, embedding choice and fusion weighting are still mostly folklore. We run them as measured experiments on each client corpus and keep the results, so the next project starts from evidence rather than a blog post.

Still open
Whether corpus-specific chunking can be learned rather than tuned by hand.
02

Calibrated refusal

How does a system know it does not know?

Confidence signals from language models are poorly calibrated. We work on external verification — checking claims against retrieved spans — because asking the model how sure it is mostly measures its fluency.

Still open
Cheap verification that does not double inference cost on every request.
03

Domain representation

How do you encode how a field actually reasons?

Every domain has structure that general models flatten: what counts as evidence, which distinctions matter, what a practitioner would never say. Capturing that as an explicit ontology consistently beats hoping it was in the pretraining data.

Still open
Semi-automated ontology extraction from expert interviews and corpora.
04

Evaluation under drift

How do you know it is still working?

A static benchmark stops being meaningful the moment inputs shift. We build evaluation sets that refresh from live traffic, with slice-level scoring so degradation shows up where it matters rather than averaging away.

Still open
Detecting quality decay before the business metric moves.
05

Bounded agency

How much autonomy is safe to grant?

Agents that can act are useful and dangerous in proportion. We work on budgets, dry runs, typed tools and verification gates — making the safety properties structural rather than instructions the model may ignore.

Still open
Formal guarantees on multi-step plans that touch irreversible actions.
06

Small-model viability

How little model can the job actually take?

Frontier models are the default answer and often the wrong one on cost and latency. Distillation and task-specific tuning frequently match them on narrow work at a fraction of the unit economics.

Still open
Where the quality cliff sits for each task family we deploy.
Model strategy

Choosing the approach on evidence

The last row is the one most firms will not show you, and it is the right answer more often than the industry admits.

Approach
Right when
Cost profile
Control
Prompted frontier model
Broad, low-volume tasks where quality matters more than unit cost
High per call
Low
Retrieval-augmented
Answers must come from your corpus and be citable
Medium
Medium
Fine-tuned / adapted
Narrow repeated task, consistent format, volume that justifies training
Low per call
High
Custom-trained
Novel problem, proprietary data, no adequate base model exists
High upfront
Total
Not a model at all
The problem is deterministic and a rules engine or query solves it
Lowest
Total
Limits

What this technology cannot do

Stated plainly, because you will find out eventually and it is better to hear it now.

It will still be wrong sometimes

Grounding, verification and refusal reduce error rates substantially. They do not reach zero. Any system design that assumes zero is a system design that will fail, so we design the handling of wrong answers as a first-class feature.

Retrieval cannot fix a bad corpus

If the knowledge is not written down anywhere, no amount of embedding cleverness will surface it. Sometimes the honest first deliverable is documentation, not a model.

Evaluation is harder than building

Getting a demo working takes days. Knowing whether it is actually good, on the cases you care about, takes considerably longer — and is where most of the value of an engagement sits.

Latency and quality trade off

Reranking, verification and multi-step reasoning all cost time. Where the interaction is real-time, some quality has to be traded away. We make that trade explicit rather than discovering it in production.

Drift is not optional

Your data shifts, your users adapt, providers update models underneath you. A system that is not monitored is degrading — the only question is whether anyone has noticed yet.

Some problems are not AI problems

Process failures, data-quality failures and organisational-clarity failures all present as AI opportunities. Building a model on top of one of those buries the problem rather than solving it.

Start here

Find out what you actually need
before you spend a rupee building it.

Send us the problem in whatever form you have it — a paragraph, a deck, a half-working prototype. You get a written architecture audit back in 72 hours.

A written read on what you are actually trying to build
The architecture we would use, and the two we rejected
Where this fails in production, and what that costs
A realistic budget range and timeline — or a reason not to start
Request the auditaurindesai@gmail.com

No discovery call required · No obligation · You keep the document either way