SymbolicGRPO improves exact recovery over SymbolicDPO in a budget-matched evaluation while requiring one model copy during refinement instead of two.
For GSoC 2026 in the ML4SCI SYMBA group, I developed SymbolicGRPO to improve Transformer refinement within an existing symbolic-regression pipeline. Working with this group has meant digging into a research-style problem end to end, from reading the precursor papers to building a protocol rigorous enough to trust the result, and I am grateful to my mentors for the guidance along the way.
In the published PIGP (Partially Initialized GP) design, a pretrained Transformer proposes expressions that seed a genetic-programming population; GP search then refines them. The published SymbolicDPO refinement step trains the Transformer on preferences from that population. My goal was to replace this step and evaluate the replacement under matched conditions.
SymbolicDPO pairs better-ranked and worse-ranked GP individuals and trains the Transformer with a preference loss using a frozen copy of itself as a reference. Refinement therefore requires two model copies in memory.
SymbolicGRPO samples 16 expressions directly from the Transformer per refinement step. Each receives a reward equal to R-squared clipped to be non-negative, evaluated on a shared sample of data rows. Its advantage is its reward minus the group mean, divided by the group’s own standard deviation. A PPO-style clipped policy-gradient update favors above-average samples. No frozen reference is needed, so refinement requires one model copy. Both methods sit in the identical slot of the loop (same beam banks, GP searches, initialization, and data rows per cycle), shown side by side below; only the refinement step differs.
Each equation’s 120,000 sampled rows occupy three non-overlapping ranges: search rows for GP fitting, training rows for refinement, and held-out reporting rows read exactly once at the end to compute the final score. This prevents formulas from receiving credit merely for fitting rows they have already seen.
Exact recovery is the primary, strict check: an isolated worker must prove symbolic identity with the true equation, without constant refitting. Functional recovery is secondary and easier: R-squared of at least 0.99 on reporting rows. A subtly incorrect formula can pass that threshold.
Across ten AI-Feynman equations and three seeds on clean data, SymbolicGRPO exactly recovered 20/30 cells versus SymbolicDPO’s 16/30. GRPO recovered every cell DPO recovered. Functional recovery was 24/30 versus 21/30.
With Gaussian noise injected into training targets at sigma = 0.01, 0.05, and 0.10, GRPO’s exact-recovery advantage was respectively 4, 4, and 1 cell out of 60. The margin narrowed with noise but never reversed. Refinement memory favors GRPO throughout: one model copy against DPO’s two.
Integrity manifests support the results, allowing independent reconstruction from raw per-cell records. An independent adversarial audit of the code, protocol, and claims reproduced every headline number and confirmed that the three row ranges never leak into one another.
The deliverable is a new method, SymbolicGRPO, and a from-scratch, budget-matched comparison protocol, released publicly with integrity manifests. It builds on the published PIGP seeding design and benchmarks directly against the published SymbolicDPO baseline.
Two findings shaped this project’s direction.
An early result showed the unrefined baseline slightly outscoring refined SymbolicDPO on clean data. Checking the code found no implementation error; the precursor paper’s own published results show the same pattern, with refinement’s advantage appearing mainly under noise.
Reading the original PIGP+DPO reference implementation surfaced a genuine issue: it computes R-squared using the same rows the GP search already fit the candidate to, with no held-out evaluation at all. This is the exact failure mode this project’s three-way row split was built to prevent.
Both findings point to the same discipline: check results against primary sources and structural guarantees, not assumptions.
Two things are next: testing on synthetic equations beyond the ten fixed AI-Feynman benchmarks, and writing up the method and results as a paper.