CPP tests selective communication in three-model AI debates

Several AI models answering the same question can bring different information to it. Sharing every answer, however, also passes along mistakes and redundant reasoning. This study tests whether a learned communication policy can choose more useful exchanges. Its contribution is a mechanism for deciding what each model should receive as a discussion develops, alongside experiments on the accuracy and cost of those choices. 1, Sections 3–4.

What changed

An AI debate here means language models answer a question, read selected responses from the other models, and revise their answers over several rounds. A language model is a system that generates text; the “debate” is a computational workflow, not a human discussion. The paper uses a team of different models so their strengths and errors need not be identical.

Conditional Progressive Pruning (CPP) is a trained filter placed around that exchange. “Pruning” means removing a possible message from the information passed to a recipient. “Conditional” means the next selection depends on what has already been chosen. Rather than assign one fixed communication pattern to every question, CPP updates its decisions using the content generated during the debate. The underlying language models are not newly invented by this study. 1, Section 3.1.

How the method works

For each recipient, the filter considers the question, a candidate response, and a learned representation of the recipient's behavior. That representation is initialized from the model's responses to validation questions: examples used to develop the system rather than score its final test performance. It is intended to describe how the model behaves more concretely than a short written profile. 1, Section 3.1.1.

The filter first considers the recipient's own previous answer, then other models' answers in a fixed order. Accepted responses are added to the question context before the next selection. This matters because a message's usefulness can depend on what the recipient already has. A response that contributes a missing step may be valuable early, while another repeating that step may add little after it has been selected. This example explains the design logic; it is not a separately measured result. 1, Section 3.1.2.

The filter is trained with reinforcement learning: its choices receive rewards that encourage useful behavior. The reward distinguishes a wrong answer becoming correct from a correct answer becoming wrong, and considers consequences within and across discussion rounds. Training therefore has access to answer correctness, but the deployed filter is not handed the correct answer to a new question. It must learn from patterns in previous exchanges. 1, Section 3.2.

The team evaluates three models over three rounds. The main models are Grok-4.1-non-reasoning, GPT-4.1-nano, and Mistral-small-3.2. The tests sample 100 questions each from hard mathematics, graduate-level science, and broad subject knowledge. Training and validation use separate samples: 150/100 for mathematics and broad knowledge, and 98/50 for science. These are relatively small benchmark subsets, not the complete benchmark collections. 1, Section 4.1.

Two comparisons answer different questions. Full sharing asks whether selecting messages improves on letting every model read all responses. Independent answer sampling asks whether communication adds value beyond generating multiple answers separately and combining them. The latter matters because additional computation alone may improve an answer, even without a discussion. The older self-consistency approach supplies background for this second strategy. 1, Sections 4.2.1–4.2.2; 2.

What the experiments show

On the hard-math sample, CPP's reported average accuracy is 68.17%, versus 65.67% with full sharing. The calculated difference is 2.50 percentage points. Its reported cost is 98.61% of the full-sharing system's cost, which the table sets to 100%. In that comparison, selective communication improves the average score with a modest reduction in reported cost. 1, Table 2.

The broad-knowledge comparison with independent sampling is closer. CPP scores 66.33%, while the paper's Consistency-Fuse alternative scores 66.17%. Their reported costs differ more substantially: 99.43% and 78.94% of the full-debate cost, respectively. The result illustrates why an accuracy ranking and a value-for-money ranking can differ. The displayed figures do not establish superiority under an exactly matched spending budget. 1, Tables 2–3.

The researchers also run ablations, experiments that remove or change a component to see how the system responds. In the later setup, removing the behavior representation reduces accuracy from 65.17% to 57.83%, but also greatly reduces how many connections the filter keeps. This supports the component's importance in that configuration, while leaving two mechanisms intertwined: better recipient information and a different volume of communication. 1, Table 6.

Reward changes provide a further clue. Encouraging stronger pruning from the start lowers cost but also lowers accuracy; training first with more communication and then adjusting the reward can retain more accuracy. That pattern suggests learning which exchanges are useful before aggressively economizing on them. These experiments use Grok-4.3 in place of Grok-4.1, so their absolute scores should not be directly combined with the main table. 1, Sections 4.1 and 4.3.2.

The main accuracy table's “±” values do not define their meaning or repeated-run count. They therefore cannot be treated here as confidence intervals or proof that small gaps are statistically reliable. That reporting boundary matters most for close comparisons, rather than erasing the mechanism the experiments are designed to investigate. 1, Table 2.

Where the research can go next

The reward experiments suggest a natural next step: train communication policies for explicit budgets. Our proposed follow-up would compare selective debate and independent sampling across several matched cost levels, rather than ask for a single winner. It would reveal when an additional exchange is worth more than another independent attempt. 1, Tables 3 and 6.

A second direction follows from the recipient representations: test how a learned policy adapts when one model in the team changes. The manuscript already uses different teams in separate experiment groups, but that does not itself test transfer of a fixed communication policy. Such a study could distinguish reusable principles of information sharing from habits fitted to particular models.

Why it matters

CPP makes communication itself a learnable part of an AI system. Its most useful lesson is that the value of a response depends on the recipient, the question, and the exchange so far. That provides a concrete basis for studying when model collaboration earns its cost and how to allocate it more deliberately.