APE Framework¶

APE (Algebra Problems Evaluator) is a framework for building and running benchmarks for AI systems on algebra problems. It is designed for tasks where candidate answers are hard to generate, but easy to verify automatically.

APE helps you define a complete benchmark pipeline:

1. provide a dataset of problems,
2. describe how model output should be parsed,
3. check answers with a deterministic verifier,
4. run evaluations and collect results.

Start here¶

• Installation
• Quickstart
• Core Concepts
• Building a Custom Benchmark
• Developer Guide

What APE gives you?¶

• A shared structure for benchmark data, prompts, adapters, and checkers.
• A runner that coordinates solver execution and verification.
• Support for multiple tools and model backends.
• A clear path from a small toy benchmark to a full experiment setup.

How it fits together?¶

data -> prompt -> solver -> adapter -> checker -> results

The data layer defines what each problem instance looks like. The prompt and solver produce a candidate answer. The adapter turns the raw model output into a structured solution. The checker validates that solution and records the outcome.

Who this documentation is for?¶

• If you want to run an existing benchmark, start with Installation and Quickstart.
• If you want to create a new benchmark, read Core Concepts and Building a Custom Benchmark.
• If you want to understand or extend the framework itself, use the Developer Guide and API Reference.