OpenAI’s COBOL Test: GPT-6.1 Sol Scores 86.6%, Up From 5.4%

OpenAI said on 19 October 2026 that its latest models are much better at building and migrating software in low-resource programming languages such as COBOL, Fortran and Visual Basic. On its own COBOL migration evaluation, it said, GPT-6.1 Sol achieves 86.6% accuracy at low reasoning effort, compared with 5.4% for GPT-5.6 Sol.

The company announced the results in a post titled “Advancing Coding in Low-Resource Programming Languages” and shared it on X at 22:30 UTC. All the scores below come from OpenAI’s own evaluations.

Business Pill · BENCHMARKS

A short explainer on why a benchmark score is not your own result. It teaches the general idea only and says nothing about any company in this story.

What OpenAI Published

OpenAI said these languages have less code and documentation for models to learn from than languages like Python and JavaScript, yet support critical infrastructure and essential services. Citing IBM, it wrote that COBOL supports more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions.

The company said the work is guided by feedback from developers and enterprise customers who told it that coding models often struggle with these languages. It built two evaluations: MultiLangBench, which tests whether a model can build a SQLite-compatible database engine from scratch across 20 programming languages, and a COBOL migration test.

What Was Posted on X

OpenAI’s own post at 22:30 UTC on 19 October says its latest models bring stronger capabilities for building and migrating software in low-resource languages such as COBOL, Fortran and Visual Basic, and asks teams which languages and workflows it should tackle next. The scores in this piece come from the post it accompanies.

The COBOL Migration Test

The migration evaluation asks models to migrate COBOL programs to C++, Java, Kotlin, Python or Rust while preserving what each program does, OpenAI said. It uses hidden tests that require byte-for-byte parity on observable outputs, and a migration counts as successful only if all tests pass.

GPT-6.1 Sol reached 86.6% at low reasoning effort, against 5.4% for GPT-5.6 Sol at the same setting, at 61% of the API cost, OpenAI wrote. At higher reasoning effort, it said, accuracy reaches 97.9%, close to saturating the evaluation.

OpenAI added that applying this to production software “still requires expert review and testing against the system’s business rules and dependencies.”

MultiLangBench, Language by Language

At maximum reasoning effort, GPT-6 Astra achieves an average test pass rate of 87.4% across the 20 languages, up from 26.8% for GPT-5.6 Sol, OpenAI said. It listed COBOL rising from 10.9% to 72.1%, Fortran from 21.4% to 83.5%, Visual Basic from 27.5% to 89.4% and OCaml from 25.6% to 85.8%.

Astra scores above 83% in 19 of the 20 languages, with COBOL remaining the hardest at 72%, according to the post. The lower-cost GPT-6 Luna reaches 40.3% at an estimated average API cost of $0.77 per sample, against $3.48 for GPT-5.6 Sol, which OpenAI put at approximately 78% lower.

OpenAI said it is releasing the MultiLangBench evaluation setup and test suite on GitHub, along with results for each language.

The Mainframe Tools Already Selling

Cloud and mainframe vendors already pitch AI for the same job. Amazon’s product page, read on 19 October 2026, says AWS Transform for mainframe is an agentic AI service that compresses multi-year modernization timelines into months.

IBM’s product page for watsonx Code Assistant for Z, read the same day, says it uses generative AI to accelerate COBOL to Java transformations in minutes, not months. Neither page gives a score on OpenAI’s evaluations, so the results are not comparable.

The Numbers, by Our Arithmetic

By our arithmetic, the COBOL migration score rose 81.2 percentage points, from 5.4% to 86.6%, and the MultiLangBench average rose 60.6 points, from 26.8% to 87.4%. Those are differences between OpenAI’s own reported scores, not independent measurements.

The Structural Read

As we read it, the evaluation design is the point: OpenAI says a migration counts only if hidden tests pass with byte-for-byte parity on observable outputs, and it gives rounding in financial results as the kind of behaviour a rewrite can silently change.

As we read it, OpenAI reports higher accuracy at lower cost: it puts GPT-6.1 Sol’s low-effort run at 61% of GPT-5.6 Sol’s API cost, and GPT-6 Luna at $0.77 per MultiLangBench sample against $3.48.

As we read it, these are OpenAI’s own tests of its own models, with no rival models scored, so they show OpenAI’s pitch for mainframe work rather than a ranking against AWS or IBM tools.

Three Implications

A COBOL SCORE OpenAI reports 86.6% on its COBOL migration evaluation at low reasoning effort and 97.9% at higher effort, against 5.4% for GPT-5.6 Sol.

A 20-LANGUAGE TEST GPT-6 Astra averages 87.4% on MultiLangBench across 20 languages, up from 26.8%, and OpenAI is releasing the test suite on GitHub.

A CAVEAT ON PRODUCTION OpenAI says production use still requires expert review and testing against each system’s business rules and dependencies.

The Business Engineer Lens

This story maps onto the Business Engineer framework Enterprise AI: From Software to Substrate.

The framework argues that enterprise switching costs do not come from data migration headaches: “They come from the fact that 47 other systems now assume your existence.”

As we read it, COBOL systems are that kind of substrate, and OpenAI’s migration test checks that a rewrite preserves what the existing program does. The framework was published in December 2025 and does not discuss OpenAI’s post.

What Is Not Established

These are OpenAI’s own evaluations of its own models, and the post does not report results for models from other companies. OpenAI did not name the enterprise customers whose feedback guided the work, or say how its models perform on any customer’s production code.

The Bottom Line

OpenAI said on 19 October 2026 that GPT-6.1 Sol scores 86.6% on its COBOL migration evaluation at low reasoning effort, against 5.4% for GPT-5.6 Sol, and that GPT-6 Astra averages 87.4% on its 20-language MultiLangBench, against 26.8%. The scores are OpenAI’s own, and it is releasing the MultiLangBench test suite on GitHub.