GPT-6.1 Sol scores 97.9% on OpenAI’s COBOL test

OpenAI said its newest models improved sharply at writing and migrating code in languages including COBOL, with GPT-6.1 Sol scoring 86.6% on its COBOL migration test.

The score, achieved at low reasoning effort, compared with 5.4% for GPT-5.6 Sol and came at 61% of that model’s API cost. At higher reasoning effort, GPT-6.1 Sol reached 97.9%, which OpenAI said brought the test close to saturation.

OpenAI designed and ran both evaluations announced on October 19, 2026. No outside group has verified the results.

The company said using these capabilities on production software “still requires expert review and testing against the system’s business rules and dependencies”.

The COBOL migration evaluation asks models to convert programs into C++, Java, Kotlin, Python or Rust without changing their behavior. Hidden tests require outputs to match the originals byte for byte, and a migration counts only if every test passes.

OpenAI said COBOL’s fixed-point arithmetic illustrates the difficulty: it drops extra decimal digits unless rounding is requested. A rewritten program that rounds instead can run without errors while producing different financial results.

The other evaluation, MultiLangBench, asks models to build a SQLite-compatible database engine from scratch in each of 20 programming languages. Results are checked against the reference SQLite implementation using tests adapted from SQLite’s own test suite.

At maximum reasoning effort, GPT-6 Astra averaged an 87.4% test pass rate on MultiLangBench, compared with 26.8% for GPT-5.6 Sol. Astra scored above 83% in 19 languages; COBOL was the hardest, with its score rising from 10.9% to 72.1%.

The cheaper GPT-6 Luna averaged 40.3%. Its estimated API cost was $0.77 per sample, compared with $3.48 for GPT-5.6 Sol, about 78% less.

OpenAI calls COBOL, Fortran and Visual Basic “low-resource” languages because less code and documentation are available for model training than for Python or JavaScript. Developers and enterprise customers told the company that coding models often struggle with these languages.

The company cited figures showing COBOL supports more than 40% of online banking systems and 95% of ATM transactions. It also said Fortran is used in weather forecasting and nuclear reactor safety analysis.

COBOL, created in 1959, climbed from 23rd place in October 2025 to 16th in the TIOBE popularity index, according to ITHardware. Companies continue to depend on COBOL systems while the pool of engineers able to maintain them is shrinking.

OpenAI is publishing the MultiLangBench setup, test suite and results for each language on GitHub.

Featured image credit