OpenAI says its latest models are far better at COBOL, Fortran and Visual Basic

OpenAI said on October 19 that its latest model advances bring stronger capabilities for building and migrating software in low-resource programming languages such as COBOL, Fortran and Visual Basic. These languages have less code and documentation for models to learn from than languages like Python and JavaScript.

OpenAI built two evaluations to measure progress.
MultiLangBench asks a model to build a SQLite-compatible database engine from scratch in each of 20 programming languages. A separate COBOL migration evaluation asks models to migrate COBOL programs to C++, Java, Kotlin, Python or Rust, using hidden tests that require byte-for-byte parity on observable outputs.

GPT-6 Astra averaged 87.4% on MultiLangBench.
That is up from 26.8% for GPT-5.6 Sol at maximum reasoning effort, with gains in every language tested: COBOL rose from 10.9% to 72.1%, OCaml from 25.6% to 85.8%, Fortran from 21.4% to 83.5%, and Visual Basic from 27.5% to 89.4%. GPT-6 Luna reached 40.3% at an estimated average API cost of $0.77 per sample, compared with $3.48 for GPT-5.6 Sol, about 78% lower.

GPT-6.1 Sol scored 86.6% on COBOL migration at low effort.
GPT-5.6 Sol scored 5.4% at the same setting, and GPT-6.1 Sol did it at 61% of the API cost. At higher reasoning effort, accuracy reached 97.9%, which OpenAI said brings the evaluation close to saturation.

OpenAI is releasing the MultiLangBench setup on GitHub.

The release includes the test suite and per-language results. OpenAI said applying migration capabilities to production software still requires expert review and testing against a system's business rules and dependencies.