Software engineering for biotech
For a biotech team, software engineering usually means getting the bioinformatics pipeline out of one person's hands and making the analysis reproducible. Then the software around it needs to keep up as the company grows. At an early stage company, that work often has to happen before the next funding round.
Software in a growing biotech
A biotech team usually knows the science in depth, and the software often starts with the scientist who knows the assay best. That stops being enough once the data gets bigger, another site needs the same workflow, or someone has to reproduce a result from eighteen months ago.
Sequencing is often where the software starts to strain. A dataset can be hundreds of gigabytes for one sample and terabytes for a study, so a workflow that ran on a laptop ends up running on many samples, on a cluster or in the cloud. BWA, GATK and SAMtools may each do their job correctly, but the software around them has to keep the whole run moving without someone watching it overnight.
The software changes again when the work moves toward a clinical claim or a partner programme. Then the team must be able to reproduce the result and show how it was produced, and validation and traceability become part of the build.
Pipelines that depend on one person
The pipeline works because one person knows the exceptions, such as which step sometimes needs to be skipped and which file occasionally needs a manual fix. We write those rules into the workflow. We also pin the tool versions and define each stage, and reference samples get replayed after changes.
Reproducibility and evidence
A result has to stand up when a partner's scientists review it or a regulator asks how it was produced. We record the reference version, analysis parameters and raw inputs, and the provenance that traces those inputs to the final result. The tests need to be fine grained enough to catch different failure modes.
Literature and internal research
A small team cannot read everything that gets published. We build a search and retrieval layer over the literature and internal research, and the model runs against that material with the source kept next to each result. The retrieval system should stay manageable for whoever maintains it.
Validation
Once the work is regulated, the system itself has to show what the requirements were, where the evidence is, who approved it and whether the result can be reproduced. We put those records in place before study data enters the platform.
We work in Nextflow, WDL or Snakemake, depending on what the team already has. A new pipeline usually goes in Nextflow, where nf-core gives a tested starting point and the same pipeline can run on a laptop, an HPC scheduler or in the cloud.
Web applications, APIs, reference databases and job orchestration are part of the work when the company needs more than the pipeline itself.
We link each requirement to its tests and keep the results as evidence. When the software changes, a reference set is replayed.
On a Veeva Vault migration for a biotechnology company running clinical studies, more than 10,000 documents moved from AODocs with metadata mapped field by field. A trial migration ran first. Every file was hash verified and the audit trail was kept.
On an NGS mutation calling platform, a Cypress framework covered 98 percent of frontend workflows and every pipeline stage was validated programmatically.
Sometimes the pipeline already works and the problem is scale, so we add the reproducibility work, testing and documentation. In other projects the science is ready but the application, APIs and orchestration are not. For a new assay, we build those pieces together with the validation work.
Questions
Do we have to change workflow engine to work with you?
No. If the team runs WDL, we work in WDL, and the same goes for a team with years of Snakemake rules. For a new pipeline we suggest Nextflow.
What does reproducibility require?
Tool versions are pinned in containers and each stage has defined inputs and outputs. Reference samples with known results are replayed after changes, and raw inputs are kept so an output can be traced back to the file that produced it. A rerun should also be idempotent.
When should validation start?
Validation starts before the platform carries study data. Adding it later means going back through work that is already in place.
Revisiting an existing pipeline
Tell us what you run and where, and what has to be reproducible.
Start a conversation