01 · WHAT
A benchmarking of open-source models built from real revenue-operations work.
We gave 117 models 40 revenue-operations tests built from real skills we run across our GTM team. Models are selected with recency in mind, and a distribution of providers to better represent the world of open-source models out there. We replenish and extend this exercise once a month.
02 · WHY
Available industry benchmarks show categories of work, but not specifically our work.
Our team builds with models every day, and early on we bet on open models for low cost and full control. Public charts never graded RevOps work: no rollforward, no hygiene audit, no forecast against messy extracts. So we built the test from our own jobs, to understand where models make sense vs. not.
03 · HOW
Anonymized GTM data, tests with planted traps to catch models, and fixed grading.
Each job hands the model the CSVs an operator would see, traps included: blank owner columns, renewal dates that disagree, call records tied to dead deal ids. Every answer faces the same three checks. The math must match to the dollar. Names must come from the data. Models aren't grading each other — there is a deterministic answer, and we're seeing how models find their way to it.
Read the exercise
04 · RESULTS
Top 15 of 103 full-coverage models
Mean score across tasks run. Full-coverage models only, ranked by mean. Partial runs sit outside the ranking.
05 · ARCHETYPES
Three groups worth knowing.
Behavioral clusters from the full run, not just scores. Example members verified against the cluster file. For what each model can read — text, images, files, media — see the modality archetypes.
Archetype
Complete Analysts
Elite scores across every category, with fast runs and no weak flank. These models read like a strong senior analyst: correct math, clean format, honest caveats.
Examples: meta/muse-spark-1.1, moonshotai/kimi-k3, meta/muse-spark-1.3
Archetype
Reliable Operators
Strong scores at budget prices, best on routine ops work. Steady and cheap, but thinner on the hardest multi-table joins.
Examples: stepfun/step-3.7-flash, tencent/hy4-preview, bytedance-seed/seed-2.0-mini
Archetype
Capable but Loose
Strong scores at budget prices, with invented names or accounts caught in this cluster. Correct on most work, but needs entity checks before production.
Examples: minimax/minimax-m2.1, deepseek/deepseek-v4-flash, xiaomi/mimo-v2.5
See all six archetypes
06 · WHAT WE LEARNED
A year of testing open models against real work.
Four takeaways from running this bench, building with the winners, and shipping two internal tools on top of it.
Takeaway 01
Modality matters.
The top-scoring model of the run could not read a screenshot because it is a text-only model. DeepSeek Pro leads the board on text and fails completely the moment someone shares a visual. Matching the model to your workflow changes the end result in a meaningful way.
See the modality archetypes →
Takeaway 02
Infrastructure compounds.
We spent months cleaning up our data, and now our cadence for monitoring our systems is more rigorous. Without that work, we would not be here today. Our first pass at 10x-ing the skill infrastructure was SignalForge, our business intelligence platform. That took several weeks to stand up from scratch. Because we did it well, launching our RevOps agent Vera took a weekend to reach the same level at meaningfully lower cost. The difference was not the model — it was that the skills, governance, and documentation were already sitting there, ready to transfer. Day to day, we run open models through OpenCode, Hermes, and Zed.
Takeaway 03
Your data matters more than the model.
Chasing the perfect model before understanding your use case is how teams burn months. The foundation — good data, real jobs-to-be-done, grading you can verify — is what makes any model swap cheap. Every number on this site traces back to a computed answer key, not another model's opinion.
Takeaway 04
Scale changes the answer.
At high volume, open models are genuinely more viable than at a fraction of the usage. Mid-market and enterprise buyers often need predictability more than the cheapest token, and a small consistent set of models beats a sprawling list. Neither answer is wrong; the math just changes with scale.
08 · METHOD
The report grows. It never re-runs.
We refresh this monthly to keep a rotating list of no more than 120 models. Each refresh tests only new and stale models against the same 40 jobs. Old scores stand. Past 120, the lowest-scoring open models leave. Frontier baselines and the reference model stay. Rebuild it on your own data with the open harness: revenuebench-starter.
Read the exercise Get the harness