What is wrong with full text searches and why do I need an index where full text searches are available?

Where electronic text is available, whether in websites, eBooks or PDFs, providing full text searching is often thought all that is necessary to make the information accessible, but they have distinct shortcomings. Exactly the same problems occur with a poor-quality back-of-book index which has been constructed by identifying every occurrence of specific words throughout the text.

Problems with full text searches are:

• Full text searches do not cope with homographs (words spelt the same but with different meanings), for example:

  • a book on boat-building might refer to the front of the boat (the bow), instruct that something is tied temporarily (with a bow) and that, as part of the construction, something should be bent into an arch (like a bow as in a bow and arrow).

  • an Internet search for information on the pop performers Madonna, Prince and the group Queen and you will get millions of unwanted references to religious art and royalty.

  • an Internet search for "lead" will find "Lead in paint...", "LEAD International: Leadership for Environment and Development..." and "12-lead ECG library ..."

• Full text searches do not cope with synonyms (words spelt differently but with the same meanings), for example:

  • bruise / contusion,

  • flammable / inflammable,

  • Çanakkale Bogazi / Dardanelles / Hellespont.

• Full text searches do not distinguish between significant and trivial references to a topic, for example:

  • it would not be helpful to the index user seeking information on 'children' to direct them to the texts

    • "teaching adults (as opposed to children) ..."
    • "the subject of children is dealt with later ...".

• Full text searches do not pick up inferences, where a concept is discussed but the actual search term is not used, for example:

  • a biography which says about the subject that he was "a strong advocate of religious tolerance" and, elsewhere in the text, "he was not a member of the Church" should show both these references in the index under 'religion'.

• Full text searches do not cater for graphics - they may pick up the caption to a picture, but cannot access or assess the content of a picture, for example:

  • The New Yorker Cartoon "On the Internet, Nobody Knows You're a Dog" (viewable here) is of interest to searches on security and identity, but probably not for searches about dogs.

After a full text search you have two unanswered questions:

• Are all these hits really relevant ?
• Have I missed any significant information ?

Furthermore, a back-of-book style index provides scope for value judgements to be made by the indexer and communicated, for example:

• In a back-of-book index, page numbers for the main discussions on a topic may be shown in bold typeface
• On a website index, the index heading may include explanatory text such as "unverified source"

Summary

Full text searches are quick and easy to supply, saving time and cost for the information provider. They are suited to rapidly changing material.

Back to FAQs