Metasearch Bake-off
September 11th, 2006 by Roy TennantMetasearching is hard. Metasearching well is darn near impossible. Meanwhile, hapless libraries are left trying to sort out the good from the not so good vendor offerings. Wouldn’t it be nice if LITA put together a “bake-off” of metasearch applications? Like picking half-a-dozen or so databases that all must set up and then running some test queries and comparing results? Other issues such as the amount of work required to customize them would be useful as well, although potentially more time consuming to determine. So is it just me, or would such a thing be interesting to others as well?
September 11th, 2006 at 3:12 pm
Great idea!
September 11th, 2006 at 3:14 pm
How about extending it to federated searching as well?
September 11th, 2006 at 3:33 pm
I’d like to see an evaluation of some reasonable search pathways for researchers.
If I have a link resolver, can’t I just choose whatever has good coverage for my interests, be it Google Scholar / MS Live Academic / Scopus or an A&I database?
Do I actually need to send out search queries in real-time to my particular licensed holdings? Do the advantages of real-time federated/meta search actually outweigh the complexity and ranking challenges?
September 12th, 2006 at 6:05 pm
I think this would be a great idea. I support Bill Drew’s idea of extending it to federated searching.
September 13th, 2006 at 5:00 pm
Hi Roy,
I think this is an excellent idea. Aside from comparing user experiences it would also illuminate that Federated Search vendors (such as my company) access the metadata in several ways - Z39.50 servers, XML Gateways, and via HTTP – and that each type has advantages and disadvantages.
I would suggest that the bake-off include information about “how” the citations are retrieved . . which connection type . . . which could facilitate a very good discussion about the “best” way to retrieve metadata from disparate resources (as well as explain why in some metasearch instances results differ from the standard Web interface the publsher/aggregator provides).
And, I agree with you Metasearch is hard! Improving the paradigm will require coordinated efforts from both vendors and content providers. And, there is no better way to expedite these efforts than by educating the market (i.e. librarians). An educated consumer base will ask more informed questions and help vendors and content providers more accurately prioritize key development projects.
Let me know what I can do to facilitate such a bake-off. I will be attending the LITA conference in October. Look forward to seeing everyone.
September 18th, 2006 at 11:18 pm
This would be a god-send for all librarians who have been tasked with evaluating federated search systems. However, it will not be easy…all aspects of the test environment need to be well-documented and standards strictly enforced for all resources and systems.
October 10th, 2006 at 2:45 pm
OK, would people rather see this in a LITA Forum environment, or at ALA Annual? Or at a Midwinter?
I suspect we could get it into a LITA Forum considerably sooner than an ALA. But would we be missing most of our audience that way?
November 19th, 2006 at 1:23 am
How will that be possible? There would need to be a database that would hold all the information.