Instruction-Following Accuracy of Local Models on Structured Tasks

Local models’ instruction-following accuracy on structured tasks depends on more than their ability to produce valid JSON. Failures arise from how models handle logical dependencies, when they must commit to a format, and whether they can signal their own mistakes. These are distinct, diagnosable mechanisms.

Why structured tasks expose a different category of instruction-following failure

An instruction with several requirements is not always just a longer checklist. In ordinary instruction following, constraints tend to be independent, so satisfying each separately can produce a correct response. But in structured tasks, one requirement may decide whether another applies or whether a later step is valid. A failure early in that chain can cascade through an otherwise plausible answer.

LsrIF, from Fudan / Ant Group, describes these relationships as parallel (AND), sequential (First–Then–Finally), and conditional (If–Else) structures. Parallel requirements must all hold, sequential requirements must be met in the right order, and conditional requirements call for the right branch. LogicIF (COLM 2026) draws a related distinction between satisfying constraints and simulating every sub-step in order. That distinction applies to routine work including JSON extraction, tool calls, schema-bound outputs, and multi-step completions. A model can understand the words in these tasks while failing to execute the logic that connects them.

How logical constraint structure causes failures that flat reward training misses

LsrIF documents standard data construction that combines constraints without encoding their logical relationships. Most instruction-tuning frameworks then average the rewards for satisfying individual constraints. That makes an early sequential failure and a late one look equally costly, even though their effects on the task differ. This rewards the model for getting parts right without reliably teaching it how to get the whole response right.

LsrIF’s reward-aggregation diagram makes the alternative concrete through parallel averaging, sequential penalty propagation, and conditional branch selection. Averaging works for parallel constraints because each can be checked on its own. But in a sequential chain, a failure early on cuts the rewards for later steps. For a conditional instruction, only the active branch earns a reward.

A prompt can expose the order of operations or make a branch explicit, but it cannot by itself supply training that rewards correct execution of those relationships. LsrIF reports improvements in both in-domain and out-of-domain instruction following from structure-aware reward aggregation, with benefits for logical reasoning more broadly.

How premature serialization degrades structured output accuracy independent of logical complexity

A separate failure occurs when the model understands the dependencies but must commit to an output before finishing the reasoning. This is premature serialization, the demand to emit schema-compliant tokens while the answer is still being worked out. A format can thus make answers less accurate even when it adds no logical complexity. The act of packaging the answer interferes with producing it.

Answer-first field ordering is the clearest trigger. On arithmetic tasks, placing the answer before the reasoning can collapse small-model accuracy to near zero, even when reasoning-first ordering produces correct answers. Models with little capacity to spare do much worse, while those with spare representational capacity can meet format demands with little loss in accuracy. The same schema can therefore impose very different burdens at different model sizes. This timing constraint matters for a local model that is already near capacity on consumer hardware.

DICE, from SJTU / Shanghai AI Lab, describes the broader competition between reasoning and strict output requirements as a prioritization problem. Models may favor reasoning over instruction adherence because the two demands compete at the token level. Premature serialization exposes the other side of that tension, where committing to format can truncate or corrupt reasoning. Tool-call and function-calling scenarios are especially difficult because they require structured output from the first token. They leave no room for a reasoning-first remedy without changes to the architecture.

Why constrained decoding fixes schema validity but not semantic accuracy

Constrained decoding is a widely used response because malformed output is a real engineering problem. By enforcing token-level schema compliance, it eliminates structural invalidity within the enforced constraints. A downstream system can then parse the response. That is a meaningful gain. It does not, however, establish that the values inside the response are correct.

The diagnostic distinction is between schema validity and task accuracy. Grammar-constrained decoding can make format compliance much better while accuracy stays much the same or falls a little. A model that previously produced malformed JSON can instead produce perfectly parseable JSON containing the wrong values. The reasoning that supplied the response’s content remains unconstrained by the decoder. At the smallest model scales, that semantic gap persists even under full constrained decoding.

The strongest defense of constrained decoding is therefore also the limit of its promise. Parseable output is more useful downstream, but its usefulness makes semantic verification more important. Once structural errors are gone, the system that uses the output still has to tell a correct answer from a well-formed mistake. Constrained decoding should not be abandoned for failing to solve a different problem.

What uncertainty-blind deployment adds to the risk of structured output failures

A wrong answer becomes harder to manage when the model cannot reliably signal that it may be wrong. Local models can fail silently on structured tasks, giving output that passes schema validation without a useful signal that the model may be wrong. A malformed response gives the system a clear reason to reject it. A wrong-but-valid response can instead move into a downstream system and propagate its error.

ICLR 2025 research from Cambridge / Apple / NUS finds that current uncertainty estimation methods struggle with instruction-following tasks, especially when errors are hard to spot. Those methods were designed for factual tasks, where uncertainty is not the same problem as uncertainty about adherence to an instruction. Internal model states help somewhat more than surface-level confidence signals, but still fall short when tasks are complex. Even large models fall short of perfect adherence on simple, non-ambiguous benchmark instructions, while smaller models perform substantially worse.

A model may mishandle a conditional branch or commit to an answer too early without revealing either mistake through its apparent confidence. Schema validation cannot catch those failures just by checking the output’s shape. Nor is it safe to rely on the model to flag the cases that need a closer look. The missing capability is independent verification of whether the instruction was actually carried out.

How the failure mechanisms interact at scale — and where the performance ceiling sits today

As tasks get harder, these ways of failing make one another worse. A model may struggle to preserve logical dependencies while also having to serialize a tool call from the first token. If it lacks reliable self-signaling, the combined failure is then difficult to detect. More constraints do not simply mean more opportunities to forget an item. They can also mean more dependencies that break without the failure showing in valid output.

LogicIFEval shows that most of the LLMs tested, including many frontier cloud models without thinking enabled, fail to follow most logic-rich instructions. Local open-weight models without thinking enabled fall to the single digits on the hardest tasks. The large, consistent gap between thinking-enabled and NoThinking variants indicates that intermediate reasoning steps carry much of the work of logical compliance. The largest frontier local models come much closer to frontier cloud models. Mid-range models, the most common local deployment size, remain well below them on multi-constraint structured tasks.

Yet that picture does not mean models fail equally at every kind of constraint. IFScale 2026 replication shows reliable accuracy extending into the thousands of simultaneous named constraints for the strongest models. That result measures keyword inclusion, a narrow proxy for full semantic and logical adherence. Strong local models can perform well on simple constraint-satisfaction benchmarks while struggling with logic-structured tasks and tool calls. The performance ceiling therefore depends on model size, task type, and output demands.

Three engineering interventions that address the mechanisms, not just the symptoms

A smaller model on consumer hardware presents different limits from a larger model on a server, and the dominant failure mode determines which intervention fits. The first intervention addresses premature serialization through reasoning-first field ordering and sequenced output. Reasoning fields can precede answer fields, or a free-form reasoning pass can be separated from a structured submission pass. These approaches recover most of the accuracy lost to answer-first ordering in instruction-prompted and multi-turn scenarios, though they do not fully resolve first-token tool-call requirements.

That makes schema design part of accuracy engineering. Field ordering in JSON schemas should match reasoning order where the workflow permits it. The second intervention targets logical dependencies through structure-aware training and RL with verifiable rewards. LsrIF demonstrates the value of propagating sequential penalties and rewarding only active conditional branches. LogicIF shows that RL training on LogicIFTrain, comprising tens of thousands of verifiable instructions, significantly improves in-domain and out-of-domain performance. For teams unable to fine-tune, decomposing complex conditional instructions into sequential prompts with explicit branch-selection steps approximates that structure-aware effect at inference time.

The third intervention addresses the accuracy–adherence tradeoff through dynamic activation steering. Static steering can overcorrect, pushing a model to ignore either the instruction or the task. DIRECTER (ICLR 2026, Yonsei University) adjusts steering strength at each decoding step through a plausibility-guided loop, improving instruction-following accuracy by up to 6.5% over baselines while maintaining task correctness. It operates at inference time without an additional dataset, making it applicable where fine-tuning is not feasible, but requires additional inference-time compute, which matters in resource-constrained local deployments. None of these interventions removes the need to detect silent errors externally.

What monitoring and evaluation infrastructure makes structured instruction failures visible in production

Production monitoring must track schema validity, answer accuracy, and wrong-but-valid rate as three separate measures. Schema validity asks whether the output parses against the target schema. Answer accuracy asks whether its content is semantically correct given the instruction. Wrong-but-valid rate isolates outputs that pass schema validation but contain incorrect answers. Tracking only validity hides these silent failures until their effects become visible in a downstream system’s behavior.

For logically structured tasks, final-answer scoring is not enough either. Constraint-level evaluation must check each atomic requirement and its logical relationship to the others. A response that meets three of four sequential constraints may look good under aggregate scoring even though its structure is broken. Teams evaluating model swaps or quantization changes therefore need coverage of logic-structured and tool-call scenarios. Keyword inclusion and simple format compliance alone cannot expose regressions in the mechanisms that matter here.

An augmentation layer can help with these checks while leaving the model to make its judgments. A reasoning engine that understands the task context, tracks whether constraints are met across steps, and brings uncertainty signals to light can provide what model-level uncertainty estimation cannot reliably give. Its role is to make the model’s output assessable beyond its syntax and apparent confidence. Better prompts, training, and decoding can reduce failures, but they do not make external verification redundant. Monitoring structured tasks well means measuring what the model cannot measure about itself.