Interoperability Options: Lessons Learned from an IMLS National Leadership Grant

In 2005, the Texas State Library and Archives Commission received an IMLS National Leadership grant to develop a multi-component federated search tool on behalf of the Texas Heritage Digitization Initiative that can search across digital collections of cultural heritage materials in Texas libraries, archives, and museums. Successful digitization projects in other states have focused on creating one or more centralized repositories of electronic resources and associated metadata. In contrast, the THDI project provides a single interface to decentralized repositories across the state. This interface, which will be available June 1, 2007 at http://www.texasheritageonline.org, has three components: a federated or broadcast search application, which uses Z39.50 to interact with library systems in real time; an OAI harvester operated by the University of North Texas Libraries to harvest metadata from institutions that do not have a Z39.50 front end; and increasing amounts of support for other APIs such as those used by A9 and Yahoo! In the process of developing this application and connecting collections, the THDI development team has learned some useful lessons. In particular, this presentation will focus on OAI-PMH implementation issues and the need for sharable metadata. Many projects, including the National Science Digital Library, have reported on the difficulty of combining metadata from multiple institutions, even when commons standards and controlled vocabulary sources are required. The solutions we have developed, including automated segmentation of OAI harvests and development of custom XSL transformations to map harvested metadata into common formats, are relevant to institutional repositories as well as to participants from the cultural heritage sector. The THDI development team has also gained experience working with lightweight search protocols including SRU and RESTful APIs, such as those from Yahoo! and A9, which are remarkably simple to implement when contrasted with the Z39.50 protocol, still widely used in library catalogs. REST, or Representational State Transfer, is a stateless, cacheable client/server architecture allows collections to share data over HTTP. Because THDI has worked with a wide variety of institutions, we are confident that this approach is both scalable and transferable to other types of digital library architectures. In a state the size of Texas, digital libraries cannot be "one size fits all." Instead, they must be flexible, adaptable, and offer institutions local control. The lessons learned from the THDI IMLS National Leadership Grant can help institutions develop new models of collaboration and distributed interaction in digital library development.