Sansa Bench
Mathematics leaderboard for Sansa Bench, with charts, model comparison, and methodology. This dimension spans foundational through advanced math, including algebra, calculus, number theory, and abstract algebra.
Top models for Mathematics
- 1.
Gemini-3-Pro-Preview Reasoning High0.988 - 2.
Deepseek-V4-Pro Reasoning High0.905 - 3.
Mimo-V2-Pro Reasoning High0.904 - 4.
Claude-Sonnet-4.6 Reasoning High0.898 - 5.
Kimi-K2.5 Reasoning Low0.896
Methodology: Mathematics
What it measures
Tests mathematics problems and knowledge across multiple levels and domains. Covers foundational topics (algebraic manipulation, equations, inequalities, fundamental mathematical concepts) to advanced university-level mathematics including abstract algebra, group theory, ring theory, and field theory. Queries span computational problem-solving, mathematical reasoning, theoretical understanding of mathematical structures, and problem-solving across various mathematical domains including algebra, calculus, number theory, and other core mathematical areas.
Scoring & Criteria
For numeric questions: returns score 1.0 if the extracted numeric value matches the expected value within the specified tolerance (evaluation_criteria.tolerance), otherwise 0.0. Tolerance can be absolute (e.g., 0.01 for +/- 0.01) or relative (e.g., 0.01 for 1% when evaluation_criteria.relative=True). Defaults to 1e-9 if no tolerance is specified. Handles commas, dollar signs, and other formatting in extraction. For multiple choice questions: returns score 1.0 if the extracted answer letter exactly matches the expected answer (after normalization), otherwise 0.0. The system prompt requests answers in `<answer>X</answer>` format where X is a letter from the provided choices. The grader normalizes for models that include the full choice text instead of just the letter, or that violate the answer tag format.
Evaluation Type: Numeric / Multiple Choice
Uses both numeric matching for computational problems and exact match for multiple choice theory questions. Computational problems require extracting and matching numeric values, while theory questions use multiple choice format with exact string matching.