[CANMEDLIB] Recent surgery articles not being indexed as surgery-related in Medline

Wed Sep 6 09:16:29 EDT 2023

Thank for both for highlighting these issues.

It isn’t a solution, but the few times I have contacted them when I’ve noticed key MeSH not being added recently, the record is updated quite quickly.

You can send a request in to their Help Desk<https://support.nlm.nih.gov/support/create-case/>

Fingers crossed submitting issues will help them document the issues.

Cheers,
Amy

Amy Faulkner, MISt, BA (she/her)
Librarian, Population Health Assessment, Surveillance and Evaluation (PHASE) Program
t: Currently WFH. Contact for phone number f: 705-721-1495
e: amy.faulkner at smdhu.org<mailto:amy.faulkner at smdhu.org>
e: hublibrary at smdhu.org<mailto:hublibrary at smdhu.org>

Simcoe Muskoka District Health Unit, 15 Sperling Dr, Barrie, ON  L4M 6K9
[cid:image001.png at 01D9E0A1.4A3CDF00]

From: CANMEDLIB <canmedlib-bounces at chlaabsc.koumbit.org> On Behalf Of Alexandre Amar-Zifkin via CANMEDLIB
Sent: Tuesday, September 5, 2023 7:26 AM
To: canmedlib at chlaabsc.koumbit.org; Niki Baumann <nbaumann at cpsbc.ca>
Subject: Re: [CANMEDLIB] Recent surgery articles not being indexed as surgery-related in Medline

CAUTION: This email originated from outside of SMDHU. Exercise caution when opening attachments or clicking links, especially from unknown senders.

this is a consequence of the shift to algorithmic indexing.

For a lot of details on how it works, there's this: https://lhncbc.nlm.nih.gov/ii/tools/MTI/Medical_Text_Indexer_Processing_Flow.pdf ; for two recent publications that are more germane to indexing and lit searching, there's Chen, Bullard & Giustini https://jmla.mlanet.org/ojs/jmla/article/view/1588 and (with apologies for blowing my own horn) Amar-Zifkin, Paquet, Ekmekjian & Landry https://papyrus.bib.umontreal.ca/xmlui/handle/1866/28262.

briefly: the indexing algorithm is basically automatic term mapping, influenced by indexing of previous records with similar uncommon words, and then preferring subheadings to headings when both exist (surgery, rehab and imaging are examples of places where this can happen. ) The algorithm can't read between the lines, so a phrase like "breast, colorectal, prostate, and cervical cancers" won't be understood as relevant to 'breast cancer'.

I suspect the algorithm property that suppresses headings in favor of subheadings is responsible for a lot of what you're describing - it's not finding a mesh-subheading pair it's confident enough in, and so it ends up dropping the concept rather than backtracking.

In our sample (dataset getting published soon, manuscript getting submitted soon) we also saw that (slightly less than half the time) the algorithm would sometimes mix up subheadings (good mesh, good subheading, unjustified mesh-subheading pair), sometimes 'pick wrong' when it chose one from between headings like Eye/, Vision Disorder/ and Eye Diseases/, and (rarely) completely overlook a concept in MeSH.

It further seems to me that alerts are likely to be disrupted by this -- our intuition is that systematic reviews are comprehensive enough with textwording to compensate for this kind of indexing variability.

When we shared our findings with the NLM, one big takeaway was that they got a lot more feedback about terms inaccurately applied (like, publications about 'red flags' getting indexed with 'Emblems and Insignia', which they've since fixed) than about missing concepts (papers about surgery not getting any mesh indicating it) so I'm glad you've already reached out to them.

Hoping this is good food for thought,

Alex

Alexandre Amar-Zifkin - alexandre.amar-zifkin at umontreal.ca<mailto:alexandre.amar-zifkin at umontreal.ca>
Bibliothécaire, Bibliothèque de la santé - Université de Montréal
(514) 343-6111 poste 3585
https://orcid.org/0000-0002-1196-5209

________________________________
De : CANMEDLIB <canmedlib-bounces at chlaabsc.koumbit.org<mailto:canmedlib-bounces at chlaabsc.koumbit.org>> de la part de Niki Baumann via CANMEDLIB <canmedlib at chlaabsc.koumbit.org<mailto:canmedlib at chlaabsc.koumbit.org>>
Envoyé : vendredi 1 septembre 2023 21:50
À : canmedlib at chlaabsc.koumbit.org<mailto:canmedlib at chlaabsc.koumbit.org> <canmedlib at chlaabsc.koumbit.org<mailto:canmedlib at chlaabsc.koumbit.org>>
Objet : [CANMEDLIB] Recent surgery articles not being indexed as surgery-related in Medline

Hi colleagues,

I’ve just come across a phenomenon in Medline – it seems that there is a problem with the indexing for surgery-related articles.

I have some automated searches set up for surgery-related topics. Over about the past year, I’ve noticed that they are finding fewer and fewer results. Up until today, I thought it was because of research funding diversion to COVID research. However, it seems it is instead due to them not being indexed with surgery MeSH and subheadings.

Original search strategy:

exp *Melanoma/su, tr or exp *Sarcoma/su, tr or exp Surgical Oncology/ or *Neoplasms/su, tr

Test search:

(((melanoma* or sarcoma*) and (surg* or operati*)) or surgical oncology).ti.

I combined these with some searches for Canadian authors and articles added to the database in the past 12 months. The original search strategy found zero results – the test search found 20, which were relevant, but had not been indexed with surgery-related MeSH nor subheadings.

I then switched to PubMed and tried the following:

((surger*[ti] OR surgical*[ti] OR operati*[ti] OR resect*[ti] OR intraoperati*[ti] OR perioperati*[ti]) AND medline[sb] AND english[lang] AND (2023[mhda] OR 2023[edat] OR 2023[crdt]))

There were 21000+ results. I then combined this with surgery[subheading], which found only 12000+ results. I looked through some samples from the first page of the remaining 8000+ results, and although some had been indexed with Postoperative Complication terms, the majority had no surgery-related MeSH nor subheadings, even though they were definitely about surgeries. It seems that about 38% of the surgery related articles from the past year have not been indexed with surgery subheading, and 3700+ (18%) of these also were not indexed with ("Surgical Procedures, Operative"[Mesh] OR "Postoperative Complications"[Mesh]) – I sorted by most recent, and 18 of the first 20 results were about surgery.

I have reported the phenomenon to NLM, but in the meantime, you might want to try some keyword searches for any surgery-related topics while they investigate.

Hope you have a happy long weekend,

Niki

Niki Baumann
Librarian
Library Services
nbaumann at cpsbc.ca<mailto:nbaumann at cpsbc.ca>
Pronouns: No preference

College of Physicians and Surgeons of BC
300-669 Howe Street
Vancouver, BC V6C 0B4
604-733-6671
www.cpsbc.ca<http://www.cpsbc.ca/>
Twitter<https://twitter.com/cpsbc_ca> | Facebook<https://www.facebook.com/cpsbc.ca> | LinkedIn<http://www.linkedin.com/company/2905395>

Serving the public by regulating physicians and surgeons

The College is located on the unceded territory of the Coast Salish peoples, including the territories of the xʷməθkwəy̓əm, Skwxwú7mesh, and Səl̓ílwətaʔ/Selilwitulh Nations.

The content of this email communication, including any attachments, is considered confidential, privileged or otherwise exempt from disclosure. It is intended only for the person(s) to whom it is addressed, and further distribution is strictly prohibited without the consent of the original sender. If you have received this message in error, please immediately notify the sender by telephone at 604-733-7758 or by return email, and delete this communication. Thank you.

The links in this email will take you to website(s) where you can access copies of the listed materials. Each website may require you to agree to additional terms and conditions. Your use of all copies of such materials is subject to those terms and conditions.
For any materials attached to this email message, Canadian Copyright laws require that the copies be used solely for private study, research, education, criticism, or review. Any such copies should be permanently deleted and/or destroyed after use. Resale, further copying, or distribution of such material, by electronic transmission or by any other means, is strictly prohibited. [Copyright Act, R.S.C., 1985, c.C-42, Part III, section 30.2(5). http://laws-lois.justice.gc.ca/eng/acts/C-42/].

*** CONFIDENTIALITY NOTICE ****
This electronic transmission and any accompanying attachments may contain privileged or confidential information intended only for the use of the individual or organization named above. Any distribution, copying or action taken in reliance on the contents of this communication by anyone other than the intended recipient(s) is STRICTLY PROHIBITED. If you have received this communication in error please notify the sender at the above email address and delete this email immediately.
Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/canmedlib-chlaabsc.koumbit.org/attachments/20230906/94b0cf27/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 3079 bytes
Desc: image001.png
URL: </pipermail/canmedlib-chlaabsc.koumbit.org/attachments/20230906/94b0cf27/attachment-0001.png>