AI for code: program synthesis, code completion and automated repair

Updated October 21, 2026

AI for code uses large language models to generate programs from natural language, suggest completions while typing, and repair defects. This article explains how these systems work, their evaluation benchmarks, and the benefits and challenges for development teams.

Program synthesis from natural language

AI code generation employs large language models to automatically produce computer code from natural language instructions or partial snippets, according to IBM's explanation. It can also modernize legacy code and translate between programming languages.

These tools integrate with an existing codebase and automate tasks such as bug fixing, vulnerability scanning, and code optimization, as described in AWS's overview of AI coding. They assist developers rather than replace them, allowing teams to focus on high-impact work.

The models are trained on vast datasets of open-source code, learning to predict the next token based on context, which lets them capture syntax and patterns of various programming languages. Programmers enter prompts, and the tool suggests relevant lines or full functions.

Code completion inside the editor

GitHub Copilot is an AI assistant that suggests code as you type, answers questions about a codebase, reviews changes, and works on assigned tasks, as stated in GitHub's Copilot documentation. It is built into GitHub, so it operates within the existing development workflow.

The tool's capabilities fall into assistive, agentic, customizations, and external tools. In the assistive mode, the developer reviews and applies each suggestion, while agentic mode lets Copilot work through multiple steps and prepare a pull request for review.

Developers remain responsible for reviewing and approving changes, and administrators can set policies for how Copilot is used within an organization.

Automated repair and evaluation

The SWE-bench benchmark evaluates a model's ability to resolve real GitHub issues from open-source Python repositories, as explained in Anthropic's SWE-bench description. The model receives a repository checkout and must modify, test, and submit a solution, which is graded against the unit tests from the original pull request.

Competition-level code generation is addressed by AlphaCode, a system described in the AlphaCode paper. It achieved an average ranking in the top 54.3% in competitions with more than 5,000 participants, using large-scale sampling and filtering based on program behavior.

These benchmarks show that AI can assist with repair and complex synthesis, but performance depends on the entire agent system, including the scaffolding and the interaction loop, as noted in the SWE-bench discussion.

Benefits, challenges, and evaluation results

The Codex paper introduced a model fine-tuned on public GitHub code and evaluated it on HumanEval, a set of programming problems, as documented in the Codex paper. The model solved 28.8% of problems, while GPT-3 solved none and GPT-J solved 11.4%. Repeated sampling with 100 samples per problem raised the success rate to 70.2%.

Benefits include accelerated workflows, improved productivity, and making coding more accessible, as described by IBM and AWS. Challenges include code quality concerns, potential erosion of developer skills, and security vulnerabilities, which require thorough review and testing.

Teams should treat AI as an aid, verify generated code with unit tests, and consider fine-tuning models on project-specific codebases to improve context awareness.

Frequently asked questions

What is program synthesis with AI?

Program synthesis uses large language models to generate computer code from natural language descriptions or partial snippets, as described by IBM and AWS.

How does code completion differ from program synthesis?

Code completion suggests lines or functions in real time as a developer types, while program synthesis can create entire functions or programs from a prompt. GitHub Copilot provides both types of assistance.

Can AI fix bugs automatically?

Automated repair is an emerging capability; benchmarks like SWE-bench test whether a model can resolve real GitHub issues by modifying code and passing existing unit tests, as explained by Anthropic.

What should teams check before adopting an AI coding tool?

Teams should evaluate code quality, security implications, and integration with their workflow, and they should review generated code and run unit tests, as outlined by IBM and AWS.