R.A.I.S.E.: Reasoning-Based AI for Startup Evaluation
Decision trees are interpretable but struggle with non-linear structure. Large language models reason well but are opaque. R.A.I.S.E. bridges the two: chain-of-thought prompting generates detailed reasoning logs, which are distilled into structured, human-readable logical rules that an investor can read, argue with, override and backtest.
Python · LLMs · Rare-event evaluation
The problem
Early-stage venture investing is a rare-event prediction task with terrible economics for the usual machine-learning playbook. Positives are scarce, labels arrive years late, and the cost of a false positive is an investment rather than a misclassification.
It is also a domain where an unexplainable model is close to useless. A partner who cannot interrogate why a company scored well cannot act on the score, and cannot override it when they know something the model does not. The objective was explicitly an explainable model that beats random selection while remaining fully transparent and editable by experts.
How it works
The framework is modular and iterative, and each component targets a specific failure mode. Chain-of-thought prompting generates natural-language reasoning logs. Those logs are distilled into explicit, verifiable IF-THEN rules and compiled into a policy. A two-step refinement pass improves the quality of the reasoning before distillation. Ensemble candidate sampling reduces the variance that makes single-shot LLM predictions unreliable. Simulated reinforcement-learning scoring refines output quality further. A persistent memory maintains context across interactions, so the system accumulates rather than resetting each time.
The interpretability is not bolted on afterwards. The rules are the model, which means an expert can review them, adjust them, backtest the change, and refine the policy over time. That turns a static prediction into a continuous feedback loop.
Results
Against a standalone OpenAI o3 model on curated startup datasets, the combined pipeline improved precision from 0.225 to 0.346 and accuracy from 0.46 to 0.70. It reaches more than twice the precision of a random classifier at the 16% base rate of that dataset. The implementation runs on o3-mini.
I designed the quantitative evaluation, which for a rare-event problem is most of the difficulty. With few positives and asymmetric costs, headline accuracy is close to meaningless, and the measurement has to be built as carefully as the model it judges.
- +54%
- precision, 0.225 to 0.346
- +50%
- accuracy, 0.46 to 0.70
- 2x
- the precision of a random classifier
Why it matters beyond venture
The framework generalises to any high-stakes domain where decisions must be defensible to someone other than the person who made them, and where the cost of a wrong positive dominates. Venture supplies the asymmetry and the scarcity; it is not the only place they occur.
Authors
With Joseph Ternasky, Fuat Alican and Yigit Ihlamur of Vela Research. Preprint on arXiv, April 2025.