Webinar Materials

On November 30, 2022, Stacy Lathrop from NLM's National Center for Biotechnology Information (NCBI) discussed the use of Bookshelf for accessing technical reports, systematic reviews, clinical guidelines, and reference materials, including the GeneReviews, LactMed, and LiverTox resources, as well as finding information in specialty areas, such as in toxicology and medical genetics. Following the presentation, Stacy was joined by Barbara Bushman and Tina Shrader from the Metadata and Cataloging Section to answer audience questions about Bookshelf selection policies and interface features, how to get alerts for new content, how to find practice guidelines from NLM sources, and more.

Links

• Bookshelf Home

• Learn about Bookshelf

• NLM Technical Bulletin

Transcript

>>Welcome everyone to NLM Office Hours. The goal behind NLM Office Hours is to give you a chance to learn more about NLM products and services, and to get your questions answered. We do record and post these sessions for broader and continuing access to the information provided. NLM Office Hours is co-hosted by the Network of the National Library of Medicine, the outreach and education arm of the NLM. We work to advance the health and well-being of everyone through improved health, information literacy, and information access. We work regionally, helping organizations within communities serve the public they know best and nationally, offering online educational programs for people who serve people. We provide funding to organizations to work within their own communities to improve health literacy and access to health information, and like today, we offer a wide range of learning opportunities for the novice to the health information expert. We hope today's experience inspires you to consider how to advance the health and well-being of your own community. Visit nnlm.gov to learn more.

Today for NLM Office Hours, we have Stacy Lathrop, the Bookshelf Product Manager at the National Center for Biotechnology Information at the National Library of Medicine. Stacy will be providing a quick overview of Bookshelf, and then she will be joined by Barbara Bushman and Tina Shrader, both from our Cataloging and Metadata Management Section to help answer your questions about accessing NLM collections within Bookshelf and beyond. Assisting me in managing today's session is Molly Knapp from the Network of the NLM Training Office. And there's Brittney Davis and Michael Tahmasian from the Office of Engagement and Training. So thank you all for being here. We do have three ways to communicate during today's session. You may use the chat feature, which I think some of you have already found. Please say hello in chat. Molly will be watching for your questions in chat and relaying them verbally to our guests. You may also raise your hand if you'd like to unmute and ask questions verbally. I will be looking for those hands. And you may also use the feedback icons to give nonverbal feedback. So let me pass this on to Stacy to tell us about Bookshelf. And while she's getting set up to share her screen, I just want to point out that Zoom allows you to adjust your screen to increase the size of the presentation. So look for the View Options menu in the upper right and then you can find the Zoom Ratio if you'd like to increase the screen. Take it away, Stacy.

>>Thank you, Kate. I joined Bookshelf in 2009 and have seen it grow significantly. This is going to be a general overview. I anticipated some questions that might come up so. So first, what is Bookshelf? We're a full text repository of over 10,000 - as of my last check yesterday, nearly 11,000 books and documents - in the life sciences and healthcare. And we provide free full text browse and search via our website. We add about 300 new books and documents a month from over 150 sources and content providers. And we have about 25 to 30 million visitors a month who are students, health professionals, researchers, educators and other members of the public. And here you see some of our largest providers, like WHO, HHS, the National Academies, and NHS in the UK.

Another question I often get is what will Bookshelf take? And that's not always so easy to provide an answer to, but it all starts that content must be in scope of core biomedical subjects defined in the NLM collection development guidelines. On the left of the slide are the publication types that we collect - and that's not even all of them, but there's quite a lot of-- about a dozen of publication types that leads to complexity, but we do a lot to provide access to this content. Most of our content include technical reports and gray literature, so about 40%, but we also have quite a few systematic reviews and clinical guidelines and then reference works which are both databases and large contributed books on a particular subject. And then what most people think we are solely - the monographs, but that's not actually most of what we collect. Lastly, where we started with textbooks. We have a few of those and those tend to be some of our most highly used content.

The next question we often get is who can apply to have their content made available through Bookshelf? And the way that this is organized is through three entry points. Before we even get to the entry points, all applicants must have peer review and editorial policies that meet best practices for the content they seek to add to Bookshelf. And we do have a questionnaire to make sure that they have a publicized and well thought out peer review. Which may be guided by an Advisory Board, if it's an edited resource such as a database. That is looked at by our collections and selectors staff. The first entry point is that we have these conversion projects. They tend to be a toxicology or medical genetic resources NIH or NLM funds. This is content converted by source Word documents because it's regularly updated, so that's about 10% of how content comes to us. And then the next is that we will enter arrangements with funders who have public access or Open Access policies. These are HHS agencies and even other federal US agencies, and some funders outside of the government, such as the WHO or Welcome Trust or The Gates Foundation. What I'm really going to focus on is our content provider program deposit, which is very similar to PMC. It’s a voluntary arrangement with usually commercial or scholarly or other publishers who will provide the content that they're going to add as XML. Again, like I say, content, because we do have a lot of publication types and then we normalize that XML to a data format that we can use for indexing and making the content available for interchange with other databases, both here at NLM, but elsewhere as well.

The next question I often get asked is how does Bookshelf decide what to take? And there are three stages. When we receive an application from a content provider, first we have our development and acquisitions section of selectors review the to make sure the content is in scope, that it meets the collection guidelines. If it does meet the guidelines, then it goes to scientific reviewers, who tend to - have PHDs, know the science, and they're part of the Technical Services division and they look for particular quality criteria regarding the content that I'll show in the next slide. If it does pass that stage, then my staff, the production and operations, we look at the technical quality and I'm not going to get too much into that during this talk. But what most people ask is what are the criteria for participating in Bookshelf and this is it. Our selector looks for objectivity, credibility and scientific quality of the content. A lot of that is by looking at the peer review policies and their best practices. Then they look that the content is supported by a range of cited references, so its evidence based and thus scientific literature. Then they look that the content is adequately copy edited and structured. Largely, that's because we want to make sure that the collection of this content in a production workflow is sustainable, and so that they do have the staff to do that, and then the next is that looking at the expertise of the authors or editorial panel in the subject area. So our selectors oftentimes look at CVs and citations in PubMed. That's kind of what they're doing there. I think what's really important though is that we also look at-- we have a lot of regularly updated content that are integrated resources or series or clinical guidelines and even part of our agreements are that the participant will send us all updates and communicate the status if some somethings change, if they withdraw a guideline, we also report that on Bookshelf but that needs to be communicated to us. So we do ask that applicants tell us what their review and update cycles are, and we are finding those are becoming much more rapid recently.

I wanted to put a few places you can find Bookshelf records - not just in Bookshelf. So I tried to, and I'm sure there's things missing here. And feel free to add anything else, but these are the main places at NLM you can find Bookshelf records, obviously Bookshelf and from Bookshelf you can retrieve all the collections and book and chapter records in the repository by a query. Both of the metadata and the full text. But most people actually start their search in PubMed and in PubMed you can retrieve the individually contributed and peer reviewed book and chapter records that have the full text in Bookshelf, and you can query their citations and abstracts in PubMed. You can also find Bookshelf content in the NLM catalog or LocatorPlus and there you can retrieve the records by either the series, such as a book series or a report series - and/or sometimes it's both - the book records that have the full text in Bookshelf, and it has the link to the full text in Bookshelf by a query of their bibliographic information. Then programmatically, we do have an Open Access subset and about 60% of our content is in the Open Access subset, and from there you can find the full text XML, images and supplementary files via an FTP service. Those are grouped by the book title on Bookshelf. All of this is made by agreement with our content providers as part of participation. And lastly, we do have MARC records and we make those available on our FTP. They get updated monthly.

And so from there I'm going to give two demos of what we're getting asked more and more. We have two main subject areas -- of collections based on subject that are rapidly growing. So first I'm going to show finding and navigating medical genetics information and then finding and navigating toxicology grey literature. I am going to move to my browser which I set up. So, the first-- And we know that this needs to be improved with better and clearer filters that you can find quickly. But right now, the best way is to go to the Advanced Search. And here under fields you can go to Resource Type. And then under Show Index List you can—here are all of kinds of the curated collections that we have. They tend to be by publication type or subject that we have focused a lot of our collection on. So right now, I'm going to go to medical genetics resources. And I'm going to search. And here I do have it sorted by Relevance. You see we have 118 books. Those books can be databases with hundreds and hundreds of documents that are regularly updated. Some of them are textbooks, like the Genetics for Surgeons, some are systematic reviews. So, the types of content can vary. You see here reference books.

But now what I want to do is go into GeneReviews because it is one of our most used resources on Bookshelf. So, I'm going to-- if I click on this cover it's going to take me to the landing page of GeneReviews. And just to kind of give you the lay of the land here - All the records in GeneReviews you can bulk download as part of our Open Access subset via FTP. Just to show you what that looks like from that link we take you directly to the package - in this compressed file you will find all the XML images, supplementary files and PDFs for each GeneReviews record and you can see when it was last modified, so if you do want to do data mining or reuse the content this is probably the best way. Some people try to crawl our site, and if you do, you're going to get blocked, so if you do want to do bulk downloads, you should use the FTP service. And then we do have interlinks with other resources, so you can, if you wanted to, you could get to the NLM catalog record, which you can see links back to the Bookshelf record.

But then if you want to search within GeneReviews we have an advanced search. Not all of our resources have an advanced search, but GeneReviews does, and I actually think it-- If you know what you're searching for, it provides the best results, so I'm going to click on that. And I am going to look up for the disorder: marfan. And you can see it shows me right away what they have for this disorder, because they tend to have just one record per disorder. I'll say Search, I get the result right away. And you have the main bibliographic details, the author, when it was first created and when it was last updated. It also shows you all of the details here with tables and sections, but I'm going to click on this: the GeneReviews summary record.

Now GeneReviews, every single one has the same structure in terms of sections, so this is very helpful if you're trying to do interchange with other databases which we here do at NCBI. What I want to show now is integration with other NCBI resources. If I were here, I'm reading about the disorder and the genetics and diagnosis, I might want to know what genetic tests are available and you can do that straight by this section. Here I can find all the tests related to this disorder and gene. And the genetic testing registry and other NCBI database. If I click here. There are 387 genetic tests. And I'm not going to go too much into the curation. That could be another Office Hour with the curators of this resource. So, then I'm going to go back. And the other thing I wanted to show, some people don't like to start from the Bookshelf homepage. Many people are avid GeneReviews users, and they have it bookmarked, and they come in, you know, once a week or a couple times a month or, you know quarterly. But if you start in PubMed, I just want to show how you can. Query "pmcbook" is a filter to get all of the records, the citations from Bookshelf and I can say "AND GeneReviews." So, there are 851 records for GeneReviews and the default search is by publication date and what that means for GeneReviews is you're going to see the latest added ones or the last updated ones at top in your results. And if you wanted to, you could create an alert or an RSS feed if you were just interested in GeneReviews.

So, from there I'm going to turn to a demo of toxicology content because we've really grown since 2018 with the collection of toxicology content. So, I'm going to go back here. And to my advanced search, it's very similar. I'm going to go to All Fields. Going to go back to Resource Type. Show Index List. And here I can find all of the toxicology resources on Bookshelf. I'm going to say Search. And here we have 541 books. You see the databases at top. Just quickly the main sources of toxicology content that we have right now beyond the databases would be the IARC monographs on the evaluation of carcinogen risk. We also have the EPA IRIS assessments. We have the ATSDR toxicology profiles, and we also have a lot of the National Academies reports. I think those are our main sources right now. And I'm going to go just like I did with GeneReviews to one of our most used toxicology databases.

I'm going to click on LiverTox here. Very same kind of lay of the land for navigation. You could get to the catalog. We also have the Open Access data. If somebody wants to mine the XML files for patterns they can get it through the FTP service. Here we do not have an advanced search, but I can search for aspirin. If I could spell it. Did I not spell it right? Ask for one. So here it is. The LiverTox, which is now owned by NIDDK, and I can see that by looking at the bibliographic details we have a link to the owner. Here they have a similar structure with an overview, case report, product information, and the chemical formula. And then they have an Annotated Bibliography. They do tell you the last time they updated the references, which I think they're trying to do every three to five years now. So, if I want to learn more about the substance, the structure, I can click on the registry number, the CAS registry number, or the structure it's going to give me just information about that structure. But if I want to have what PubChem has harvested about the compound, I can go to the Related Compounds here. And they have all their content here that they take from many sources, because we're talking about toxicology, I'm going to go to toxicity. And they have many sources. If I keep on scrolling down. They have About the liver induced toxicity here and they take what they think is the most prominent summary describing that and what we know from the LiverTox editorial team is many people want to know what the likelihood score is. And here you can see it is very likely that there is an effect. If I want to get back to Bookshelf, I click on the source LiverTox and it's going to give me the URL to Bookshelf, so I get back there. And for now I think I'm going to stop and wait for questions.

>>Hi, great. Our first question is from Ramooni at Northwestern University. And please share any questions you may have in the chat while I feed you this first one. And the question is how does one find out about new additions to Bookshelf?

>>OK, so I luckily-- I got this question before and thought about it, so I'm going to look at my cheat sheet. But just so you know, there is an RSS feed here for Bookshelf. There is a limit to this feed because it only reports the new books and collections, not any new chapters that have been added or updated within a book. Before I show you where I think the better place to do that is, you can also follow the Twitter feed. We do try to announce new content and when a large resource like LiverTox, LactMed or GeneReviews is updated we report it through our Twitter feed - but we don't do everything. So, if you really want every everything I think-- I mean I am going to PubMed but you can find all of the Bookshelf content through the query “pmcbook” and then you can create an alert here and you can-- Just like why is this? It's trying to do-- I'm not going to be able to demo this because my-- I'm not already logged in MyNCBI. I thought I had, but you can create alert just like you would with any query, and you can also create an RSS feed and here it's going to give you the newly added and updated chapters as well as books. So, it's a little bit more granular. That answers that question.

>>Great. Thanks. Our next question is about NCBI Bookshelf titles, are they available elsewhere? For example, what's the arrangement with the National Academies Press?

>>Right. So, most of the content on Bookshelf is available by the publisher or sponsor or content provider either in print or online. They're just a few, such as the conversion projects that I mentioned that are funded by NIH or NCBI, that are only available through Bookshelf. So, I just want to show you the National Academies. And I'll try to explain the arrangement here. So, I want to go to their collection. So, this is just the collection of the NIH funded ones. We do have reports that are not funded by NIH as well, but if I go here - We always show the publisher or owner of the content and from there you can oftentimes get to the content from their site. With some publishers, particularly those who have DOI, so we know that they're permanent links. We, as part of our agreements with them before adding the content to Bookshelf, I'm going to show you here we can make available a link directly to the hard copy version on the publisher site and not just to the main publisher who's the owner source. I can get straight to this document on the National Academies site. Similar by clicking on the cover. We really prefer to only make these direct links if we know that the content provider with whom we have an agreement has a stable link such as the DOI. I hope that answers that question. If not, let me know if there's any follow up needed.

>>Yes, please do. And thank you for your questions in the chat, I'm going to address a few of those right now. And the first one is from Sue, who wants to know are all of the systematic reviews in Bookshelf also in PubMed or PubMed Central?

>>So, all of the systematic reviews on Bookshelf are also indexed as their citations and abstracts in PubMed. There are, I know it gets very confusing, but there is a small subset of those systematic reviews that were accepted by LISTRIC as part of the MEDLINE collection, and so you can also find them in PubMed Central, but they will link to the actual systematic review in Bookshelf.

>>Wonderful. And then Jen and Margot-- Jennifer and Margot both asked similar questions, which is, what does the future look like for Bookshelf? Do you anticipate growing the textbook collection? What can we look out for?

>>That's a great question. I've been wanting to expand the textbook collection for years. Those who work with me-- that was one of the first things I wanted to do because I think there's a great need. So we had a fellow as part of the National Library of Medicine where we kind of did a landscape analysis to see where we could fit in, you know. One concern is we want to find quality textbooks and there's been a lot of changes. There's now the Open Education Resources community. But we are starting a pilot. I guess I could announce that there have been some nursing textbooks and also some biomedical and life sciences textbooks that were funded by the Department of Education. And because of large things that are happening with public access for non-journal content, all of that is voluntary, but we will be, because they are Open Access content, adding those to Bookshelf as a pilot. So we do, we hope that might be able to take off into even greater areas.

>>Oh that is super exciting news. Thank you for sharing.

>>I'm very excited. I can't tell you how long it's going to take us before we get them indexed, but we're working on that now. We've been talking to folks and we have a plan for it. So you should start seeing those hopefully in the new year.

>>Well, let's take another question. This was an emailed question from Beatrice at the University of Bern. I'm going to paste it in chat. It's a little long. "Do you agree with me that for a thorough search for practice guidelines, MEDLINE Ovid is not enough? It is important to search PubMed because it is only place that contains the Bookshelf?" Not all guidelines are published as articles, so we'd love to hear your thoughts about that.

>>So, I agree, but I think the answer is complicated and I'm going to show you two things and then turn you over to Barbara. So just to show you what you would find in a PubMed search, I'm going to go to Advanced Search. So, I want to go by Publication Type here too-- Where was it? Why am I not seeing it? Oh, publication type. I need to do the field and I'm going to go to "practice guidelines." And I will Search. So, this search is giving you all practice guidelines just so everybody knows what this is giving you. It's giving you practice guidelines that have been MeSH indexed with a practice guideline publication type. Or for books, it was hard coded for just a few collections, and we do know-- So I do want to say that there's a lot of improvements that we're trying to make, and all your questions are providing feedback and how important it and prioritization of making those changes. So, we do need to update that but if I want to then limit a these 30,000 plus practice guidelines - mostly their journals - to just books, those in Bookshelf, I can go to the Filters and look at Books and Documents. As I said, this is not a complete list so I would say if you wish to know what's really in Bookshelf by groups that see themselves as clinical or practice guideline developers you can go into Bookshelf and again go into Resource Type here Show Index List. And here they're called Clinical Guidelines. And here you have-- I'm on collection still. I knew we had more than seven. Those were collections of clinical guidelines, so you are going to get a larger list if you're doing a search by the content in Bookshelf but even then, it's probably a not enough to find all the guidelines at NLM. You want to do an NLM-- a search of the NLM catalog. And now I'm going to turn it over to Barbara to show you that.

>>Hi everyone. So I'll just quickly show you how you can find the practice guidelines, all of them that we have in our collection and I'm in the NLM catalog, I'm in the Advanced Search builder, and as you can see, I've kind of done this before.

>>Let me stop sharing before you can share. Sorry.

>>Go ahead. OK, now you should be able to see the NLM catalog. Great. So I've got the catalog open. I've already done some of these searches today just in preparation, but I will do this one again. So what this is, is I've gone here in All Fields and I've selected Publication Type because I know that for practice guidelines the MeSH publication type is practice guideline. So I've done that and when you do that then you will get the 16-- you get that 48197. And again, this is all practice guidelines that are in our collection. So they're in our catalogs. That includes print materials and electronic materials. So if you wanted the comprehensive list, this is really it. If you just wanted ones that were accessible on online, then you could filter by Internet here. And when you do that, then you get 16,139 Practice guidelines that are both they could be in the Bookshelf. They could also be in digital collections, so our institutional repository, they may be available just from the publisher website as well. So if they're in our catalog, then we've made them available. So that's a more comprehensive search. So I think that the PubMed will give you some and this will give you some. So if you really want a comprehensive, you kind of have to do it all and then kind of you know, did you filter your results however you want? Do that if you're really trying to get a comprehensive listing of all the practice guidelines available.

>>Wonderful. Thank you, Barbara. So my take away there, Bookshelf, PubMed, NLM catalog. Dump it all into a spreadsheet and deduplicate and boom, you got it. You got everything right and that makes me think, if I may, I have a question for our audience today and that's what sources do you turn to for practice guidelines beyond NLM? We'd love to hear in chat as well as if you have any more questions, we have about 5 more minutes with you today, so thank you so much. In the meantime, while we're-- OK. So Mark says he turns to the-- they turn to the Trip database. Emmy says EzriECRI. And yes, keep those suggestions coming for practice guidelines and we've given you three ways to do that today. Now one final question for you today and that is turning back to Bookshelf. And hopefully this is not too much that-- your question is, is there Bookshelf content in other languages, and if so, how can I find it?

>>Yes, we do have content and other languages. So when we enter an agreement based on funding, we will take whatever language that funded content is. So we have some French, we have some Italian and we have some Spanish. We also have a pilot for collecting Spanish language content and we've been working with that pilot with our folks we already have our relationships. Like WHO and their Spanish translated guidelines by PAHO.

So how to find that content? I will share my screen and just show you that there's not such a great way to do it in Bookshelf. If I, you know, know the Spanish word I could put it in here. If I know the Spanish title I could put it and it would retrieve it. But we don't have one place to find everything that we have in that language. So, I am again going to turn you to PubMed. And do an Advanced Search. So, if you wanted to find everything that we have say in Bookshelf that's in Spanish. You could go to the filter here for Language. And Show Index or just type in Spanish. And I'm going to do a search. And I already have the limit on for Books and Documents. So we only started this pilot a couple months ago. So we have 20 of them right now. And here's the count. So you can find the whole collection that we have in Spanish.

>>Great. It also makes me think I could create an alert for this search and then find out. Wonderful. OK, well, I'm going to turn it over to Kate Majewski. Thanks for your questions everyone.

>>Yeah, thank you all. I'm going to stop sharing.