We’ve launched the updated version of E-Utilities API for PubMed. Thank you to all who tested the updated API on the test server and provided feedback.
This updated version now aligns the functions of the E-utilities with the web version of PubMed released in 2020. For example, search results returned by the updated ESearch E-utility will now match those of web PubMed. To be clear, this update only affects E-utility calls with &db=pubmed. The behavior of all other Entrez databases will not change.
Why did NCBI do this?
NCBI released this new API version to provide both consistent behavior for both web and API PubMed interfaces, as well as more reliable performance. To accomplish this, we transferred all E-utility functions to the technology stack that supports web PubMed, so that all PubMed requests use the same stack. This means that previous version of the PubMed E-utilities is no longer available, but that the new version provides the benefits listed above
Have the URLs for PubMed E-utility calls changed?
Previous E-utility URLs for PubMed (&db=pubmed) will continue to function with this updated release, with one exception. To obtain more than 10,000 PubMed records, consider using EDirect, which now contains additional logic to batch PubMed search results automatically so that an arbitrary number can be retrieved. See our updated documentation for more details.
Has the output of PubMed E-utility calls changed?
Again, in almost all cases, no. Here are the exceptions:
- ESearch will now return the same PubMed IDs (PMIDs) that are currently returned by web PubMed
- EFetch will now return XML data by default (&retmode is not set) rather than ASN.1. In other words, the default value of &retmode will become “xml”.
What should I do if I have trouble using the new API?
Write to us if you have any questions or concerns.
Yay, congrats! Could you point me to documentation specifically for the new EDirect logic for batching PubMed retrieval for results >10K (with examples, if possible)? I couldn’t find it in the updated documentation linked above. Thank you!
Thanks for asking. As far as I know the logic isn’t in the documentation. I will check with the developer.
The current strategy repeats the query with a sliding window of Create Dates. The window is dynamically resized to get each combined query below the 10K per query truncation limit. But another date field always applies to an entire day, so if more than 10K were created, it cannot be used to avoid the limit. We’re working on an improvement to this strategy.
Hi guys, may I know if any updates or better strategies to extract more than 10K papers?
Thanks for the question. EDirect’s efetch can work around the 10K limit. It now uses an updated strategy that uses a binary search on the PMID values. This became possible when the UID field started to support range queries, and replaces the original sliding CRDT window approach. The new code is faster, simpler, and more reliable than the old version, and can be used in combination with date queries.
How about a new webinar to go with the new release. Some of us users are a bit rusty.
Thank you for your feedback!