Software Agents have Changed Science

“Dale, if I were in that situation where I did not have to work for money anymore, I’d spend 8 hours a day on AI.”

The coffee I had with the head of an R&D group had the following context: a former colleague-in-common hit it big with a nine-figure acquisition, and that this individual was one of the founding team members with an equity event in seven figures. We were talking about financial independence, and what we would do with ourselves if we didn’t need to work for a paycheck.

This senior scientist by his own admission was not a programmer. But programming was what he had in mind. He showed me an app he programmed for his son’s Little League baseball statistics. It was very impressive.

That moment stuck with me as I listened to Genentech / Roche’s Nayan Chaudhary give a talk at the Bio-IT World conference titled “Agentic coding for real-world data science - from research plans to reliable codebases”. Toward the middle of his talk he mentions casually, “this is the biggest change to my basic coding workflow in about 2 decades of programming”.

He then pulls up a screenshot of Andrej Karpathy, a founding member of OpenAI and a former Director of AI at Tesla, where he states on January 29, 2026 “This is easily the biggest change to my basic coding workflow in ~2 decades of programming and it happened over the course of a few weeks.” Karpathy indicates “Given the latest lift in LLM coding capability, like many others I rapidly went from about 80% manual+autocomplete coding and 20% agents in November to 80% agent coding and 20% edits+touchups in December.” (Link to his entire 1,000 word post.)

This inversion, from 20% automated to 80% agentic, is remarkable. How software engineers spend their time to produce software has radically changed. Has the world come to an understanding of this phase-shift in something as fundamental as the production of software?

At the Bio-IT World Conference (May 2026), Dr. Nayan Chaudhary (Genentech) said in his software engineering work he analyzes Real World Evidence (RWE) for treatment patterns in resections of non-small cell lung cancers (NSCLC). The software analyzing early-stage cancer treatments would normally take a week of coding with assistance from LLMs. Now he can generate a working codebase in a single day, using agents. He said that use of agents preserves code structure, making it easy for him to add additional functions such as sensitivity analysis without causing the “software to crash”.

This is software acceleration quantified, and the ratio is five days of work compressed to one, or a five-fold increase in productivity.

Think about that. Not 50% faster. Five times faster.

He then showed an example of one of his “structured research plans”, where RWE needs many items: hypothesis generation, data exploration, a composed research plan based on the prior two dependencies, the execution of the analysis and finally a review of results. A major dependency though is the format of the data, and its inherent heterogeneity. There are dozens of tables, hundreds of variables, thousands of definitions, across multiple Electronic Health Record (EHR) systems, patient registries, and insurance claim datasets.

The RWE data world is not the world of clinical trial data. There are no standards set. Each institution and organization (hospital, regional medical center, academic medical institution etc.) has their own non-standard terms and their own definitions of these terms, basic institution-specific knowledge codified by the people who use these systems.

In a corporate environment you could call this “tribal knowledge”. In these institutions it’s just another day at the research hospital.

His structured research plan was understandably obfuscated as far as the hypothesis they were testing, but the workflow was clear enough: 10 individual steps, with each step producing a given amount of code and number of files. The table was expansive: the first step produced 2,500 lines of code and 15 files; step 4 produced roughly 810 lines of code and 2 files; in total, agents added or modified roughly 4,640 lines of R code across 35 files, yielding a codebase of about 4,000 lines.

He contrasted the prior workflow to the agentic-enabled one. Before, it was an IDE-like interface within Microsoft CoPilot. Now it is a harness within Claude Code (Opus 4.7) or OpenAI Codex (5.5-Codex) optimized for multistep reasoning. He said it was a “paradigm shift from last year”. His slide said “Only code edits, No Handwritten Code (~4000 lines codebase)”.

A second contrast slide continued the theme: before, code was hard to control, had frequent errors and required frequent review. Now it offers finely-grained control, reliable data transformations, a consistent modular structure and a familiar consistent format that simplifies code review. He called it “end-to-end code generation”.

Adding to the agent are skills, API hooks to other resources, other agents, and MCP connections (Model Context Protocol, an interface enabling secure connection to external data sources and software tools). These skills call automated pipelines on top of external datasets. They also have developed internal knowledgebases (to help with variable definitions and disease ontology) and reference helper applications (code snippets to guide analysis toward specific statistical analyses for example).

As an aside, he pointed out an unusual side-effect, which is “agentic anxiety” as the agents await human attention where the resource is rate-limiting. Something else to ponder: the agent is waiting on human-in-the-loop input, and is anxious to move onto the next step. Human attention, as it is well known, is all you need.

Evaluating the quality of the output is paramount; at each major step, an artifact is produced that can be inspected, and the agent then steered in the appropriate direction. He emphasized the importance of well-defined definitions of quality output up-front.

This presentation, given in May 2026, was only five months after this huge phase-shift in software development has taken place.  A recognized leader in AI refers to an increase from 20% to 80% the fraction of code that is automatically generated. A software leader at a top oncology Pharma says work that used to take a week is now done in a day, a five-fold increase. This is an exponential shift in volume. And we are only at the beginning stages of this phase shift.

The Bottleneck Has Shifted

In early 2007 when I sold the first Solexa 1G’s (quickly renamed the Illumina Genome Analyzer) there was a struggle to handle and analyze the roughly 800 megabases of DNA sequence from each 3 day run. (At that time the readlengths started as single-end reads of 36 bases each, paired-end and longer readlengths would arrive soon enough). The first laboratories would store the output FASTQ files on external USB hard drives that would soon stack up on the laboratory shelf. It was a few years later the bottleneck shifted downstream to alignment, where Burrows-Wheeler Algorithm (BWA, which uses the Burrows-Wheeler Transform and an FM-index to make alignment dramatically faster and less memory-hungry than the hash-based aligners it replaced) and other techniques were developed to speed a time- and compute-intensive process, aligning primary sequence data to a reference genome, and calling variants from this aligned data. Hardware such as DRAGEN (FPGAs put into service for specialized alignment tasks) was developed to speed the process further. Then the bottleneck shifted downstream to tertiary analysis, or what the called variants mean in a disease research or a clinical context.

The timeline of this progress was over the past 19 years; now clinical applications are a majority of Illumina’s sequencing business (recently they claimed it was 65%), and new applications with substantial markets (such as Minimal Residual Disease testing for cancer) appear with regularity. But as the bottleneck shifted downstream, one important area was addressed all along the way: the accuracy of the final output, as technical advances kept shifting the bottlenecks.

I didn’t include the successive, iteration pressure of the volume of data being generated per run on a single NGS instrument. As I mentioned the Solexa 1G produced about 800 Mb of data per 3 day run in early 2007. Today in 2026 a NovaSeq X at its highest capacity can produce about 20 Terabases of data per 2 day run. That is 20 million Mb, or a factor of 25,000-fold over the course of 19 years.

There is a delicate interplay and balance, where sequencing throughput was steadily rising at an exponential rate (here is the famous NHGRI graph here for those unfamiliar) but a real concern for the accuracy of the alignment and variant calling, a critical function to ensure the integrity of the downstream conclusion.

One effort, initiated by the National Institute of Standards and Technology (NIST) was Genome in a Bottle, a “public-private-academic” initiative started in 2012, still ongoing 14 years later to develop reference materials, methods and datasets “dedicated to comprehensive characterization of benchmark human genomes”. (Link to the NIST GIAB project.) A few years later, the FDA would organize a “PrecisionFDA Truth Challenge” to evaluate the accuracy of variant-calling tools with the goal of assessment, comparison and improvement. These efforts to improve accuracy had measurable impact, and served as important third-party validation of a complex and difficult to parse output: millions of variants produced from billions of bases from a single human genome sequencing dataset.

Now with the production of software experiencing its own exponential growth phase (witness Dr. Chaudhary’s “a week to 1 day” comment) there needs to be a particular attention paid to the quality of the output. In the code itself, what kinds of internal checks are needed? What assurances that the data sources are valid? Are the conclusions traceable? If there is a human-in-the-loop, who signed off on what and when?

Where is GIAB for domain-specific, agent-generated code?

Dr. Chaudhary describes a rigorous quality assurance methodology for the artifacts produced at every step, where the criteria is known up-front. The challenge is that it only resides within the company he works for, Genentech. We are back in 2012, where genome variants were called and evaluated for accuracy by every individual laboratory. Everyone builds their own “truth set”, and none of it is compared or made available to others to use.

NIST did produce and maintain a software reference dataset called SARD. It contains 450,000 test cases that examine documented software weaknesses. It benchmarks whether a tool detects categories of defects, it has nothing to do with whether a software pipeline correctly interprets one hospital’s definition of the term “resection”.

Genome in a Bottle provided a common, public, neutral third-party “truth set”. For Real World Evidence and all those variable definitions, who will build a public “truth set”?

Reference:

Zook JM and Salit M et al. Nature Biotech (2014) “Integrating human sequence data sets provides a resource of benchmark SNP and indel genotype calls” doi:10.1038/nbt.2835