We want to hear from you about changes to NIH’s Sequence Read Archive data format and storage

NIH’s Sequence Read Archive (SRA) is the largest, most diverse collection of next generation sequencing data from human, non-human and microbial sources. Hosted by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), SRA data is also available on the Google Cloud Platform (GCP) and Amazon Web Services (AWS) as part of the NIH Science and Technology Research Infrastructure for Discovery, Experimentation, and Sustainability (STRIDES) Initiative.

SRA currently contains more than 36 petabytes (PB) of data and is projected to grow to 43 PB by 2023. Though the value of this resource grows with each new sample, the exponential growth experienced over the last decade (Figure 1) threatens SRA sustainability. The storage footprint is growing more costly to maintain and the data more difficult to use at scale. The situation has reached a tipping point. SRA must be refactored to support FAIR data principles into the future.

Figure 1. SRA data has grown exponentially over the last decade.

NIH remains committed to the SRA and hopes to establish a long-range plan for sustained resource growth. Considerations include a model wherein normalized working files without Base Quality Scores (BQS) are readily available through cloud platforms and NCBI FTP sites, and larger source files and normalized files with base quality scores will be distributed on cloud platforms based on prevalent use cases and usage demands. Further details regarding data formats are available here.

It is critical that as an SRA user, you participate in the review and testing of proposed data formats and infrastructure by commenting on how these developments impact your data usage. NIH has prepared a Request for Information (RFI) that details planned developments and would greatly appreciate feedback from the scientific community.

Continue reading “We want to hear from you about changes to NIH’s Sequence Read Archive data format and storage” →

Improved access to SARS-CoV-2 data

NCBI Datasets has a simple, new way to get Coronoviridae data, including from SARS-CoV-2 (Figure 1). The data package includes genomic, protein and CDS sequences, annotation and a comprehensive data report for all complete genomes. You can also target your search to major taxonomic ranks within Coronaviridae.

comm-1318_fig1 — Figure 1 – SARS-CoV-2 page within NCBI Datasets showing statistics as of June 16, 2020.

Interested in a specific protein? The SARS-CoV-2 protein page allows you to choose a protein and download the corresponding sequences, annotation and representative structures from all annotated genomes (Figure 2).

comm-1318_fig2 — Figure 2 – SARS-CoV-2 protein page within NCBI Datasets showing annotations on the SARS-CoV-2 reference genome.

Looking for programmatic access? NCBI Datasets offers the same Coronoviridae genomic data and SARS-CoV-2 protein data through a command-line tool and a RESTful API. These tools support additional filtering including the ability to download only those genomes released after a date you specify.

We appreciate your feedback. Try NCBI Datasets and let us know what you think!

New GenBank submission options for SARS-CoV-2 submitters

NCBI is pleased to announce ongoing enhancements to submission of SARS-CoV-2 assembled genomes to GenBank, including a streamlined workflow on the web and a new API option. Both new options mean that you can receive accessions for SARS-CoV-2 data submissions more quickly!

A streamlined workflow with improved interface and enhanced validation on both web and API saves you time and effort and, most importantly, makes it possible to get SARS-CoV-2 accession numbers and public release of data within hours. In addition, we automatically annotate all SARS-CoV-2 genomes to produce standardized, consistent annotation which saves you time and benefits researchers who find your data valuable. Continue reading “New GenBank submission options for SARS-CoV-2 submitters” →

New viral protein domain models for annotation of coronaviruses

NLM’s Conserved Domain Database (CDD) has expanded its scope to now include 153 new viral protein domain family models for the annotation of coronaviruses, including models such as for the S1 subunit of coronavirus Spike proteins (cd21527), the nucleocapsid (N) protein of coronavirus (cd21595), and the coronavirus RNA-dependent RNA polymerase (cd21530).

Each curated domain model consists of a multiple sequence alignment containing conserved sequence features that may have been confirmed experimentally, plus links to relevant publications. When available, the domain models include 3D structures with links to interactive 3D views and interacting partners.

Check out this tabular summary of SARS-CoV-2 gene products for links to matching conserved domain models and representative 3D protein structures.

Want to view these alignments in 3D space? We’ve updated iCn3D, a web-based 3D structure viewer, with new rendering, annotation, and alignment features. Read more about how you can use iCn3D to view and analyze SARS-CoV-2-related structures.

Don’t forget to review our SARS-CoV-2 resources page to keep up to date on other coronavirus data at NCBI!

The New and Improved PubMed® — We Are Listening

Today marks 5 weeks since the new PubMed was made the default version. Throughout this process, we promised to listen, and we heard from you!

This was a huge change

We know change isn’t always easy, especially with major changes to a familiar service or product. We are staunch believers in making incremental changes whenever possible: releasing small improvements, observing the effects, gathering user feedback, and then using that data to make further modifications. This time, an incremental approach to improving PubMed wasn’t feasible. We needed to make major changes under the hood (new databases, cloud delivery, new web architecture, etc.) for PubMed to be sustainable going forward.

User feedback is invaluable: it has played an enormous role in updates over the 24 years PubMed has been in existence, and it continues to do so. To prepare for new PubMed, we launched the beta version in 2017, then called PubMed Labs, as a way to set up the new framework and solicit feedback from our users. During development and since, we reached out to our stakeholders with presentations, webinars, handouts, FAQs, toolkits, and tutorials, including a series of four 90- minute online classes, How PubMed® Works, many of which continue to be available.

We understand that not everyone had a chance to put the new PubMed through its paces, and we’re grateful to those of you who provided feedback along the way, whether it was by sending questions or comments using the feedback button, by discussing with us how you accomplish your work with PubMed, or by filling out a survey.

For some, when the new version of PubMed became the default last month, it was a huge shift. The ways in which you were accustomed to working with the system changed. We heard from some of you that you were used to a particular feature being available on PubMed and now you don’t know where to find it.

Continue reading “The New and Improved PubMed® — We Are Listening” →

New BLAST default parameters and search limits coming in September

To provide a more efficient BLAST experience for everyone, we’re changing some parameters and limits on the web BLAST service on September 8, 2020. The new settings, listed below, will improve overall performance and make search times more consistent.

The Expect Value Threshold default setting will be reduced to 0.05.
The maximum number of target sequences (Max target sequences) limit will be no more than 5,000.
The maximum allowed query length for nucleotide queries (blastn, blastx, and tblastx) will be 1,000,000 and 100,000 for protein queries (blastp and tblastn).

These changes will help keep the BLAST service running smoothly as the already very large databases continue to grow rapidly. If you have any questions or concerns, please email us at blast-help@ncbi.nlm.nih.gov

dbSNP human build 154 release + ALFA data

dbSNP human build 154, now available, includes new ALFA (Allele Frequency Aggregator) variants and allele frequency. This build contains over two billion Submitted SNP (ss) records and 730 million Reference SNP (rs) records.

New features include:

dbSNP search now support HGVS and protein variant search
RefSNP page linked to Litvar information
Filter and facet to retrieve all SNPs with ALFA frequency

See the release notes for more information about what’s new in build 154.

New annotations in RefSeq: budgerigar, bony fish, fly and more

In May, the NCBI Eukaryotic Genome Annotation Pipeline released new annotations in RefSeq for the following organisms:

Acipenser ruthenus (sterlet)
Arvicanthis niloticus (African grass rat)
Cannabis sativa (eudicot)
Crassostrea gigas (Pacific oyster)
Cyclopterus lumpus (lumpfish)
Drosophila albomicans (fly)
Drosophila guanche (fly)
Drosophila innubila (fly)
Esox lucius (northern pike)
Gymnodraco acuticeps (bony fish)
Hippoglossus hippoglossus (Atlantic halibut)
Marmota flaviventris (yellow-bellied marmot)
Melopsittacus undulatus (budgerigar)
Osmia lignaria (orchard mason bee)
Pangasianodon hypophthalmus (striped catfish)
Pantherophis guttatus (snake)
Periophthalmus magnuspinnatus (bony fish)
Prunus dulcis (almond)
Pseudochaenichthys georgianus (South Georgia icefish)
Setaria viridis (monocot)
Thalassophryne amazonica (bony fish)
Thrips palmi (thrip)
Trematomus bernacchii (emerald rockcod)
Zea mays (maize)

See more details on the Eukaryotic RefSeq Genome Annotation Status page.

Enhanced prokaryote type strain report now with details on needed type strain data

The Prokaryote type strain report provides information on type-strains for over 18,000 species. We revised and expanded the report to make it easier to identify cases where sequencing or establishing type material would have the biggest impact on improving prokaryote taxonomy and accurate identification. These cases include species with designated type strains but without a sequenced type strain assembly and species without designated type material. We hope that the community will prioritize sequencing type strains for the former set of species (Table 1) and establishing a neotype or reftype, where applicable (as defined in Cuifo et al 2018) for the latter set (Table 2).

Other changes from the old format file are detailed in a recent genomes announce post.

Scientific Name	Type material/co-identical strains	Assemblies
Burkholderia ubonensis	CCUG:48852, CIP:1070, …	308
Escherichia albertii	Albert 19982, BCCM/LMG:20976, …	181
Xanthomonas perforans	AATCC:BAA-983, DSM:18975, …	153
Listeria innocua	ATCC:33090, BCCM/LMG:11387, …	106
Streptococcus iniae	ATCC:29178, BCCM/LMG:14520, …	94
Vibrio lentus	CECT:5110, CIP:107166, …	87
Vibrio cyclitrophicus	ATCC:700982, BCCM/LMG:21359, …	83
Pseudomonas coronafaciens	BCCM/LMG:5060, CFPB:2216, …	77
Aliivibrio fischeri	ATCC:7744, BCCM/LMG:4414, …	66
Xanthomonas fragariae	ATCC:33239, BCCM/LMG:708, …	61

Table 1. The top 10 candidate species for sequencing type-strains sorted by the number of assemblies. These have designated type strains but no type strain assembly. We generated the list by sorting by “number of assemblies from type materials per species”, then by decreasing “number of assemblies per taxon”, then filtering out “type materials and coidentical strains” = “na”.

Scientific name	Assemblies
Nostoc linckia	14
Blattabacterium punctulatus	10
Microcystis flos-aquae	10
Microcystis panniformis	9
Microcystis wesenbergii	8
Aphanizomenon flos-aquae	6
Arthrobacter bacterium	6
Denitromonas halophilus	6
Ensifer aridi	6
Methylacidiphilum fumariolicum	6

Table 2. The top 10 candidates for proposing a reftype assembly, or neotype where applicable sorted by the number of assemblies. These species have no designated type strain. We generated the list by selecting for “type materials and coidentical strains” = “na”, “number of assemblies from type materials per species” = 0, and sorting by decreasing “number of assemblies per taxon”, then filtering out Candidatus.

Please contact info@ncbi.nlm.nih.gov if you want to provide information about missing type-strains.

June 24 webinar: An insider’s guide to creating Federal grant BioSketches

Join us on June 24 to learn how to use My Bibliography and SciENcv, My NCBI applications that help you to create biographical sketches for grant applications for the National Institutes of Health (NIH), National Science Foundation (NSF) and the Institute of Education Sciences (IES). You will learn how create a profile, add citations and import information from 3rd party accounts like ORCiD

Date and time: Wed, June 24, 2020 12:00 PM – 12:45 PM EDT
Register

After registering, you will receive a confirmation email with information about attending the webinar. A few days after the live presentation, you can view the recording on the NCBI YouTube channel. You can learn about future webinars on the Webinars and Courses page.