Cloud Indexing

Last week, Seth Earley blogged about the inefficacy of social tagging, but there's one scenario in which social tagging will breathe new life into an esoteric, 200-year industry: book indexing.

I've written hundreds of book indexes, presided over the American Society for Indexing, managed an international indexing partnership, taught courses, established standards, built tools, and consulted with a lot of influential folks, so trust me when I tell you that it pains me to see this happening. I believe with every fiber of my professional being that the human work of subject indexing is and will continue to be superior in quality to every alternative ever imagined. Oh well.

There is just too much information to index by hand, period. Books, periodicals, websites, blogs, messages, and documents are being produced or transformed too quickly for humans to keep pace, regardless of training and tools. Perhaps in response, the use of search algorithms becomes ever more popular, while overly optimistic expectations of retrieval quality grows increasingly preposterous. A more realistic response would be an increase in subject indexers' fees -- after all, demand is outpacing supply at an astounding rate -- but indexers haven't experienced a rate increase since the 1990s. The truth is that editorial indexing and all smart hands-on tagging is disappearing in favor of automatic approximations. And it is a reasonable argument that the substandard tagging of millions of pages and documents is better than leaving most of them without any subject metadata whatsoever.

As industries grow more and more interested in taxonomy and semantics and autoclassification, it is the book publishing industry that will experience the most pronounced change. Printed books will disappear, and electronic book readers will become networked. It is intriguing that books, with their inherent depth and complexity, can be completely deconstructed into chapters, pages, and sentences, and then subjected to the same social ingenuity applied to photographs and songs. Individuals will be able to "mark up" (tag) their copies of books, and those notes will feed into a cloud of content that can be shared among all readers. Following in the footsteps of social bookmarking comes social book-marking.

Readers of popular books (and in particular works of fiction) will enjoy having access to deep and faceted lists of keywords, for use in both search- and browse-based user environments. These "cloud indexes" can be combined arbitrarily to create global indexes for entire book collections, personal or otherwise, and these global indexes can be refined using standard book metadata (e.g., publisher), personal metadata (e.g., date of first read), cloud metadata (e.g., Lady Gaga's favorites), and thesauri or taxonomies.

Books that aren't popular are doomed in the cloud indexing paradigm. Publishers and authors must work hard to encourage readers to contribute, especially because traditional SEO techniques won't work well with book content. Book publishing will remain as competitive as always. And for the first time in the history of book publishing, it will be impossible to judge a book by its cover. Instead, we'll use the index.