Google: AI-assisted code migrations

Google runs one of the largest codebases in the world, and changing something everywhere in it, such as widening an ID type or moving off an old test framework, is the kind of work that can take teams years. In January 2025, Google engineers described how they now do these migrations with an LLM in the loop: tools find what to change, the model makes the edit, builds and tests check it, and people review it. The headline numbers are large. The most useful line in the paper is about what slowed them down: people reviewing the changes.

1. The situation

The paper describes four migrations. In Google Ads, IDs stored as 32-bit integers risked overflowing, across a codebase of over 500 million lines. Tests written for JUnit 3 had to move to JUnit 4. Code using the Joda-Time library had to move to Java's built-in java.time. And old experiment flags had to be cleaned up. Each is simple in one file and enormous across thousands.

2. What they did, and what it achieved

Two lines from the paper matter as much as the numbers: "a combination of AST-based techniques, heuristics, and LLMs are needed to achieve success", and "The bottleneck in the process was the speed at which engineers could review the changes."

3. How the pipeline works

Each stage uses the cheapest reliable tool. The model only does the part that needs judgement.

4. The pipeline in code

A tiny version of the pipeline for an int32 to int64 ID migration. A finder (standing in for code search and AST filters) picks only variables named *_id, so retries and count stay untouched. A stand-in model rewrites the targets, and in one file also "helpfully" widens a counter. A validator rejects any change outside the selected lines (the paper uses AST diffing for this), and the results are split into shards for owners to review. The program was run in all three languages; each prints the output below it.

import re

# Invented source files before an int32 -> int64 ID migration.
FILES = {
    "ads/campaign.cc": ["int32 campaign_id = row.id();", "int32 retries = 3;", "Save(campaign_id);"],
    "ads/budget.cc": ["int32 budget_id = Lookup();", "int32 count = 0;"],
    "ads/report.cc": ["int32 report_id = Next();", "Print(report_id);"],
}
TARGET = re.compile(r"\bint32 (\w+_id)\b")     # the finder: a stand-in for code search + AST filters


def find(lines):
    return [i for i, line in enumerate(lines) if TARGET.search(line)]


def model_edit(name, lines, targets):
    """A stand-in for the LLM: rewrites the target lines, and in one file also 'helpfully' touches another line."""
    out = [TARGET.sub(r"int64 \1", line) if i in targets else line for i, line in enumerate(lines)]
    if name == "ads/budget.cc":
        out[1] = "int64 count = 0;  // widened for consistency"
    return out


def validate(before, after, targets):
    """Reject any change outside the lines the finder selected (an AST diff would do this in practice)."""
    stray = [i for i, (a, b) in enumerate(zip(before, after)) if a != b and i not in targets]
    return not stray, stray


changes, repaired = {}, []
for name, lines in FILES.items():
    targets = find(lines)
    after = model_edit(name, lines, targets)
    ok, stray = validate(lines, after, targets)
    if not ok:                                           # send back: keep only the target edits
        after = [after[i] if i in targets else line for i, line in enumerate(lines)]
        repaired.append(f"{name} (line {stray[0] + 1})")
    changes[name] = sum(a != b for a, b in zip(lines, after))
    print(f"{name}: {len(targets)} target line(s), {changes[name]} changed")
print("stray edits removed:", ", ".join(repaired) or "none")
SHARD = 2
shards = [list(FILES)[i:i + SHARD] for i in range(0, len(FILES), SHARD)]
print(f"{sum(changes.values())} lines changed in {len(FILES)} files; review in {len(shards)} shards: " + " | ".join(", ".join(s) for s in shards))

// Invented source files before an int32 -> int64 ID migration.
const FILES: Record<string, string[]> = {
  "ads/campaign.cc": ["int32 campaign_id = row.id();", "int32 retries = 3;", "Save(campaign_id);"],
  "ads/budget.cc": ["int32 budget_id = Lookup();", "int32 count = 0;"],
  "ads/report.cc": ["int32 report_id = Next();", "Print(report_id);"],
};
const TARGET = /\bint32 (\w+_id)\b/;   // the finder: a stand-in for code search + AST filters

const find = (lines: string[]) => lines.flatMap((line, i) => (TARGET.test(line) ? [i] : []));

function modelEdit(name: string, lines: string[], targets: number[]): string[] {
  // A stand-in for the LLM: rewrites the target lines, and in one file also 'helpfully' touches another line.
  const out = lines.map((line, i) => (targets.includes(i) ? line.replace(TARGET, "int64 $1") : line));
  if (name === "ads/budget.cc") out[1] = "int64 count = 0;  // widened for consistency";
  return out;
}

function validate(before: string[], after: string[], targets: number[]): [boolean, number[]] {
  // Reject any change outside the lines the finder selected (an AST diff would do this in practice).
  const stray = before.flatMap((b, i) => (b !== after[i] && !targets.includes(i) ? [i] : []));
  return [stray.length === 0, stray];
}

const changes: Record<string, number> = {}, repaired: string[] = [];
for (const [name, lines] of Object.entries(FILES)) {
  const targets = find(lines);
  let after = modelEdit(name, lines, targets);
  const [ok, stray] = validate(lines, after, targets);
  if (!ok) {                                            // send back: keep only the target edits
    after = lines.map((line, i) => (targets.includes(i) ? after[i] : line));
    repaired.push(`${name} (line ${stray[0] + 1})`);
  }
  changes[name] = lines.filter((l, i) => l !== after[i]).length;
  console.log(`${name}: ${targets.length} target line(s), ${changes[name]} changed`);
}
console.log("stray edits removed:", repaired.join(", ") || "none");
const SHARD = 2, names = Object.keys(FILES), shards: string[][] = [];
for (let i = 0; i < names.length; i += SHARD) shards.push(names.slice(i, i + SHARD));
const total = Object.values(changes).reduce((s, n) => s + n, 0);
console.log(`${total} lines changed in ${names.length} files; review in ${shards.length} shards: ` + shards.map((s) => s.join(", ")).join(" | "));

import java.util.*;
import java.util.regex.*;

public class Main {
    static final Pattern TARGET = Pattern.compile("\\bint32 (\\w+_id)\\b");   // the finder: a stand-in for code search + AST filters

    static List<Integer> find(List<String> lines) {
        List<Integer> out = new ArrayList<>();
        for (int i = 0; i < lines.size(); i++) if (TARGET.matcher(lines.get(i)).find()) out.add(i);
        return out;
    }

    static List<String> modelEdit(String name, List<String> lines, List<Integer> targets) {
        // A stand-in for the LLM: rewrites the target lines, and in one file also 'helpfully' touches another line.
        List<String> out = new ArrayList<>();
        for (int i = 0; i < lines.size(); i++) out.add(targets.contains(i) ? TARGET.matcher(lines.get(i)).replaceFirst("int64 $1") : lines.get(i));
        if (name.equals("ads/budget.cc")) out.set(1, "int64 count = 0;  // widened for consistency");
        return out;
    }

    static List<Integer> stray(List<String> before, List<String> after, List<Integer> targets) {
        // Reject any change outside the lines the finder selected (an AST diff would do this in practice).
        List<Integer> out = new ArrayList<>();
        for (int i = 0; i < before.size(); i++) if (!before.get(i).equals(after.get(i)) && !targets.contains(i)) out.add(i);
        return out;
    }

    public static void main(String[] args) {
        // Invented source files before an int32 -> int64 ID migration.
        Map<String, List<String>> files = new LinkedHashMap<>();
        files.put("ads/campaign.cc", List.of("int32 campaign_id = row.id();", "int32 retries = 3;", "Save(campaign_id);"));
        files.put("ads/budget.cc", List.of("int32 budget_id = Lookup();", "int32 count = 0;"));
        files.put("ads/report.cc", List.of("int32 report_id = Next();", "Print(report_id);"));
        int total = 0;
        List<String> repaired = new ArrayList<>();
        for (Map.Entry<String, List<String>> e : files.entrySet()) {
            List<String> lines = e.getValue();
            List<Integer> targets = find(lines);
            List<String> after = modelEdit(e.getKey(), lines, targets);
            List<Integer> bad = stray(lines, after, targets);
            if (!bad.isEmpty()) {                                    // send back: keep only the target edits
                List<String> fixed = new ArrayList<>();
                for (int i = 0; i < lines.size(); i++) fixed.add(targets.contains(i) ? after.get(i) : lines.get(i));
                after = fixed;
                repaired.add(e.getKey() + " (line " + (bad.get(0) + 1) + ")");
            }
            int changed = 0;
            for (int i = 0; i < lines.size(); i++) if (!lines.get(i).equals(after.get(i))) changed++;
            total += changed;
            System.out.println(e.getKey() + ": " + targets.size() + " target line(s), " + changed + " changed");
        }
        System.out.println("stray edits removed: " + (repaired.isEmpty() ? "none" : String.join(", ", repaired)));
        List<String> names = new ArrayList<>(files.keySet());
        List<String> shards = new ArrayList<>();
        for (int i = 0; i < names.size(); i += 2) shards.add(String.join(", ", names.subList(i, Math.min(i + 2, names.size()))));
        System.out.println(total + " lines changed in " + names.size() + " files; review in " + shards.size() + " shards: " + String.join(" | ", shards));
    }
}

ads/campaign.cc: 1 target line(s), 1 changed
ads/budget.cc: 1 target line(s), 1 changed
ads/report.cc: 1 target line(s), 1 changed
stray edits removed: ads/budget.cc (line 2)
3 lines changed in 3 files; review in 2 shards: ads/campaign.cc, ads/budget.cc | ads/report.cc

5. Try it: where's the bottleneck?

Plan a migration. Generating and testing changes is fast and parallel; reviewing them is limited by people. Change the numbers and see which stage sets the schedule. All rates are illustrative.

6. What the authors learned

• Simple prompting only works for the simplest migrations. Finding the right places to change needed code search, Kythe cross-references and AST filters.
• Agents planning their own work were usually unnecessary: "LLM planning capabilities are often not needed and add a layer of complexity that should be avoided when possible."
• The model sometimes changed too much, such as adding comments or touching unrelated methods. AST diffing caught it, and reviewers reverted some edits.
• "The most challenging part is cleaning up the right tests."
• Review was the limit, and there are hidden costs: training engineers and maintaining the custom tooling.

7. Lessons

• Find with tools, not the model. Code search and AST filters pick targets precisely and repeatably.
• Let the model edit, and let builds and tests judge. Failures go back to the model to repair (Module 13).
• Reject edits outside the target. A diff check keeps the model's "helpful" extras out of the change.
• Design for review. Small, consistent shards routed to the right owners are what make thousands of changes landable (Module 08).
• Same pattern, other companies. Amazon's Java upgrades follow the same shape (architecture case study C17).

8. Your turn

1. Pick a migration (10 min). An old API, library or type your codebase should move off. How many call sites?
2. Write the finder (10 min). What search or AST query finds exactly the targets, and what must it skip?
3. Plan the review (10 min). Who owns which files, how big should a shard be, and how many can a reviewer check a day?

9. Check yourself

10. Sources and further reading