Dev digest · 2026-10-25
1. CSIRO's sBeacon: Serverless Genomic Queries and Operational Limits on AWS
CSIRO's sBeacon uses Amazon S3, AWS Lambda, Amazon Athena and Amazon DynamoDB to query genomic data while leaving the genomic files in their original storage. In CSIRO's chromosome 1 benchmark with 2504 individuals, sBeacon onboarded a 1.1 GB compressed VCF dataset in 18 seconds. AWS reports that whole-genome range queries time out at API Gateway, and throttled Lambda calls can lose results on HTTP 429 unless the application handles them.
Read source ↗Source evidence
CSIRO's sBeacon uses Amazon S3, AWS Lambda, Amazon Athena and Amazon DynamoDB to query genomic data while leaving the genomic files in their original storage.
sBeacon is a production-ready implementation of this standard, built using AWS services: Amazon Simple Storage Service (Amazon S3) , AWS Lambda , Amazon DynamoDB , and Amazon Athena .
Evidence source ↗
Note that genomic data is not copied out of its original location but rather is referenced when needed.
Evidence source ↗
In CSIRO's chromosome 1 benchmark with 2504 individuals, sBeacon onboarded a 1.1 GB compressed VCF dataset in 18 seconds.
This multi-sample VCF is approximately 1.1 GB compressed, with data stored in Amazon S3.
Evidence source ↗
sBeacon can ingest chromosome 1 from the 2504 individuals in 18 seconds, for less than 1 cent (USD 0.00052).
Evidence source ↗
AWS reports that whole-genome range queries time out at API Gateway, and throttled Lambda calls can lose results on HTTP 429 unless the application handles them.
However, if you accidentally attempt to run a range query of the entire genome, the architecture times out at the Amazon API Gateway level.
Evidence source ↗
Similarly, because synchronous Lambda invoke does not automatically retry on throttle, a 429 response from a performQuery invocation means the result is silently lost unless the application handles it explicitly.
Evidence source ↗
Discussion