Stop measuring how close you were. Start measuring if you would have won.
Drop any CSV with predictions & actual outcomes (6 columns maximum)
Works for stocks, sales, weather, sports-anything
Think of it like this: If you bet money on your predictions, would you make profit?
Being close isn't enough. Wrong direction can lose money.
Anyone who needs predictions that work, not just look good on paper.
This works for any prediction, not just finance
Scenario: You predict a stock at $101, it goes to $99.
Traditional: "Close! Only 2% error."
Quantsynth: "Wrong direction = you lost money."
Scenario: You predict 75°F, actual is 73°F.
Traditional: "Good job!"
Quantsynth: "Did people bring jackets? That's what matters."
Scenario: You predict a 10-point win, they win by 3.
Traditional: "Pretty close!"
Quantsynth: "You predicted a win and they won = profit."
Scenario: Predict $100K sales, actual is $80K.
Traditional: "20% error"
Quantsynth: "Did you overstaff? That costs real money."
Scenario: Model A: 95% accurate. Model B: 88% accurate.
Traditional: "Pick A"
Quantsynth: "Which one makes fewer expensive mistakes?"
Scenario: Predict price increase, it decreases 5%.
Traditional: "Small error"
Quantsynth: "Wrong direction = you bought at the peak."
Comparing what matters vs what gets measured
| WHAT MATTERS | TRADITIONAL APPROACH | QUANTSYNTH APPROACH |
|---|---|---|
| Getting Direction Right | ✕ Completely ignored (only cares about distance) | ✓ Primary focus (80% of success) |
| Actual Success Rate | ✕ No connection to outcomes | ✓ Directly measures if it works |
| When You're Wrong | ✕ Big mistakes kill your score | ✓ Understands some errors matter more |
| Confidence Levels | ✕ Treats all predictions equally | ✓ Rewards being confident when right |
| Real-World Use | ✕ Great metrics, bad decisions | ✓ Optimizes for actual utility |
| Industry Adoption | ✕ Used because it's familiar | ✓ Used by quant funds making real bets |
Not how close your predictions were — but how useful they actually are
FIS measures whether a prediction would have led to a good decision. Instead of rewarding forecasts for being close to the target, it rewards them for being right in the moments that matter.
CER answers a different question: How much confidence do you earn per unit of error? It balances how strong a prediction is with how efficiently it achieves that strength.
FIS tells you whether a prediction is worth acting on. CER tells you whether that confidence is efficiently earned. Together, they reveal why traditional accuracy metrics can be misleading.
Visualizing the difference in what gets rewarded
The "Precise Loser"
Close to the target price, but wrong on every direction. R² says "great!" FIS says "you'd lose money on every trade."
The "Imprecise Winner"
Wrong magnitude, but correct on every direction. R² says "terrible!" FIS says "you'd profit on every trade."
While R² rewards models for being near the target, FIS rewards models for being right on the trade. It's the difference between academic accuracy and profitable action.
Your model looks great in calm markets. What about chaos?
Copula-Based Tail Dependence
Most evaluation metrics treat all days equally. But in reality, extreme events are where fortunes are made or lost. A model that's "pretty good" 95% of the time but completely wrong during 5% black swan events is worthless.
We use regime-dependent copula analysis to separately evaluate your performance during calm vs volatile periods. Your score reflects how you perform when the market moves ±5% in a day, not just the easy days.
This is the same mathematics used by quant hedge funds to ensure their strategies don't blow up during market stress. Now you have it for your predictions.
Traditional metrics are easy to cheat. Ours aren't.
Predict the same value every time. You'll often get decent MSE and MAE scores because you're "consistently close to the average."
We catch it instantly. Flat forecasts = zero useful information.
Get lucky on 10 predictions? Traditional metrics will give you a great score even though it's statistically meaningless.
Dynamic λ adapts: we're skeptical of small samples, confident in large ones.
Exaggerate every move. You'll be wrong on magnitude but R² won't penalize you much if you're directionally inconsistent.
MASE-adjusted scoring makes magnitude errors impossible to hide.
"It's an honest mirror for your models. You can't hide poor performance behind statistical noise."
How traditional metrics get fooled by common prediction patterns
This model just predicts yesterday's price. It's incredibly common (moving averages, momentum strategies) and looks great on traditional metrics because it's "always close." But it's completely untradeable.
By the time you get the signal, the move already happened. You're always entering after the profitable moment. It's like reading yesterday's news.
✓ "Excellent fit!"
✓ "Very accurate!"
✕ "Worthless for trading"
Real Trading Result: You'd lose money on transaction costs alone. Every trade is a day late.
The transparency and privacy you need for sensitive data
Predictions processed in-memory. We evaluate your utility, we don't steal your alpha. No data storage, no tracking, no leaks.
Math is documented. Implementation stays private. You know what we do, just not how we optimize it.
Download reports include all the necessary data to complete your analysis. No black boxes.
50,000+ Monte Carlo simulations. Bootstrap confidence intervals. Peer-reviewed methodology. Not guesswork.