Discovery layer decisions, configurations and strategies

. The problem that patrons frequently encounter with search ing via a discovery layer linked to a knowledgebase is the overwhelming number of hits. Few if any research or discussion compares how libraries resolve this difficulty. To begin a discussion, the author disseminated an open invitation for academic libraries to participate on three panels from October 2018 to June 2019 to discuss their implementation processes and decisions. Five academic libraries accepted the invitation. This paper reviews the approaches of the five American academic library systems with the intent to encourage further examination of the issues of design, implementation, and maintenance of discovery layers. The conclusions reached point to the evolving nature of discovery layer interfaces and the struggle for libraries to adapt their needs to this still changing product type. Within this group, the larger institutions with more financial and staff resources have designed a ‘’bento box’’ interface with open source software, while the other institutions with lesser means have chosen to cre ate different instances of their subscribed proprietary discovery layers to meet their specific institutional demands and the specific needs of their patrons. A statistically relevant examination with rigorous comparison of usability testing would have to be done to see how these and other factors play out in the wider field.


Introduction
Compared to the online library catalog (OPAC), the discovery layer 1 is relatively new in the world of library services. The first discovery layer services were introduced in the early 2000s (Antelman et al., 2006). Some examples of proprietary products are Primo, Summon, WorldCat Discovery and Ebsco Discovery Service (EDS), and open source software options are VuFind and Blacklight. Oya Y. Rieger (Rieger, 2019) states in her issue brief that "while there are disciplinary differences, most researchers prefer using search engines, academic networks, familiar databases, and peer networks to find and filter information, and the library catalog is no longer the primary point for discovering and accessing content" (Blankstein, Wolff-Eisenberg, 2019). Changes in the information environment characterized by information scarcity to one of information abundance has encouraged the devel- 1 The discovery layer has been also referred to as discovery tool, discovery service, discovery system, web-scale discovery, resource discovery service (RDS) to name a few. opment of these products. To meet this challenge, discovery layers are central indexes (also known as a knowledgebase) that are web-based, highly customizable and independent of the library management system, be it an integrated library system (ILS) or a library services platform (LSP). In addition to the library's catalog records, discovery layer indexes can include the metadata from various commercial database products (both full text and A&I services), open access collections, institutional repositories, and many more. This combined search of all available databases offers the advantage of leveraging all resources. Discovery layers provide an interface with search and retrieval capabilities, with relevancy ranking and various facets to narrow the results, and interoperability with a link resolver (Breeding, 2015). Many discovery layer products can also limit access to materials by the user's location, the content type, and/or license arrangement.
Given that the technology is still evolving and experience is limited, decisions abound when implementing a discovery layer. Does the library eliminate the OPAC or keep it alongside with the discovery Статья поступила в редакцию 19.11.2019 Получена после доработки 10.03.2020 Принята для публикации 15.04.2020 layer? Or does the library use one discovery layer to access all their materials? Do they select multiple discovery layer products, or create different instances of the same product? And if a library does have more than one discovery layer or several instances of the same product, which parts of the collection are targeted? Examples such as separating search options for the catalog, or various types of collections, be it music, the sciences, research datasets, or institutional collections. And how have they reached their decisions? What research was done to inform the process? Did their selection of a knowledgebase affect their decision? Then how were the discovery layers implemented? And most importantly, how did the libraries assessed their impact? Which usability testing techniques did they use? Did they also use surveys and focus groups? When did they access the impact and how did they implement changes?
At William Paterson University, we were unsatisfied with the functionality of our online library catalog Webvoyage -the OPAC module of Voyager. So in 2011, we implemented VuFind as our OPAC. Then in 2014, we implemented Summon as our discovery layer, and kept VuFind as the interface to our catalog. In the summer of 2015, we switched from Summon to EDS, still keeping VuFind as our OPAC.
Early last year, we signed a contract to subscribe to Ex Libris' Alma and Primo. This summer, we replaced our Voyager system and VuFind with Alma and Primo. Prior to and during our Primo implementation, I organized panels at three different conferences from October 2018 to June 2019 for various library to discuss their implementation processes and decisions 2 . Five academic libraries accepted my invitation to participate. This paper reviews the discovery layer planning, implementation, and current assessment and maintenance practices at these five institutions. They are the New School of New York City, Ball State University of Muncie, Indiana, Skidmore College of Saratoga Springs, New York, the University of Michigan of Ann Arbor, and Temple University of Philadelphia.

The Schools and their Configurations
The institutions range in size from small to large, urban to college town, and research to teaching institution. The New School has about 13,000 students of which 10,544 are enrolled in a degree program (7,436undergraduates, 3,108 graduates), 409 fulltime faculty and 1,837 part-time, and degree programs from certificate to doctorate. They are part of a consortium with New York University, Cooper Union, the Brooklyn Historical Society, the New York Historical Society, and the New York School of Interior Design. Ball State has about 22,000 students of which 16,160 are undergraduate and 5,724 are graduate. They have over 1,046 faculty and have degree programs from associate to doctorate. Skidmore College has about 2,600 studentsall are undergraduates. They have over 320 faculty and offer bachelors degrees. The University of Michigan has over 46,000 students of which 30,318 are undergraduate and 16,398 are graduate. They have over 8,000 faculty of which 6,706 are full-time and over degree programs from bachelors to doctorate. Temple University has near 40,000 students of which 29,484 are undergraduate and 10,256 are graduate. They have almost4,000 faculty, of which 2,296 are full-time. They both developed an interface deploying a 'bento box' design -referred to as such because of its resemblance to the shape of Japanese lunch boxes (Singley, 2016).

The New School
The New School defined their functional requirements as 1) they should have a single discovery product and a single search box, 2) keep the search within the discovery layer as much as possible, 3) the layer should display as much of the collection as possible, 4) the Library should be able to control and tune the search experience, 5) linking to interlibrary loan, holds and article requests should be directly from the discovery layer, and 6) it should support microservices interactively. Their decision in selecting which product was made within the constraints of their consortium.
The New School's single search box decision 3 is based on the changes in what they collect. Eighty percent of their physical collection is stored off-site.
In 2016, they implemented demand driven acquisitions (DDA). As much of their 'collection' is either electronic or physically stored in a variety of locations, their goal was to move from strictly owned to access as needed, display availability and wait times, integrate as many other services into the discovery layer as possible, enable to explore other parts of the collections and build relationships between those items. Their assessments show a marked increase in requesting of material and that labeling in the system is critical for patron use of the system.
As displayed in the figure below, the first tab is for searching the entire collection, when the other two tabs are subsets, visual resources and digital collections from the archives respectively.
Once a search has been performed, aside from the facets to refine the results, along the top of the screen the results page contains five options, 1) new search, 2) search for journal titles, 3) search specific databases for more specialized content via Libguides, 4) search for newspaper content, and 5) browsing via specific fields such as subject or author through Primo. Their default search is a blended search, of articles, consortial holdings, and their own local holdings. While their local title count is less than 300,000, their affiliation with NYU exposes millions of titles potentially requestable through their borrowing network, PAL-CI's EZBorrow resource sharing network (fig 1, 2).

Ball State University
Ball State's requirements were 1) they should have a single discovery product and a single search box, and 2) ease of maintenance. Ball State does not perform assessments of their patrons' satisfaction with search results, though they have an internal Alma Summon stakeholders group consisting of 15 members from all functional areas of the library.
With figure 3 below, we can see that they have taken the one search box approach 4 . Links to more specific tools are located lower on the page and under the links in the "Do Research" and "Use Services" boxes figure 4.
Their results page, as seen in figures 5 and 6, has some modifications depending on the search terms. For example, in figure 6, we see database recommendations, research guides, and the reference librarian's contact information included in the results, whereas in figure 5, they are not included.

Skidmore College
Skidmore's requirements were 1) a Google-like search experience, and 2) minimal customization. Originally the order of the tabs of the opening page were SearchMore, Subject Guides, Journals, Library Catalog, and Site Search. However, after working with the system, the librarians reported the results of the discovery layer were too overwhelming and that books and media materials were lost in the results. In addition, the lack of clarity in how the  search relevancy algorithms worked was a factor. Hence many of the librarians do not use the discovery layer in their instructions. In general the students are directed to use Subject Guides to choose a particular database to perform their searches. Despite this practice, the Library's usability studies and tracking via Google Analytics show that both systems are used at an equal rate. In the end, Skidmore emphasizes the physical resources over the electronic -thus placing the link to the catalog as the first tab and the discovery layer as the third tab 5 .
In figure 7, the first tab "Library Catalog" links to Webvoyage -the OPAC module of Voyager. While the Subject Guides tab links to Libguides, the SearchMore tab links to EDS, the Journals tab links to EDS Publication Finder, and Site Search links to a google search of Skidmore College Library's webpages (figure 8, 9).

University of Michigan
University of Michigan's requirements were 1) to improve the user experience for all userswith emphasis on 'all kinds of users', 2) reduce instr uction complexities, 3) mo dernize infrastructure -hopefully 'future-proofing' the architecture and 4) manage the interface -not the data. With a department of developers, they opted for an open source software solution 6 6 GitHub. University of Michigan Library. URL: https://github. com/mlibrary (accessed 27.03.2020). using a combination of Solr, Blacklight, Ruby on Rails, React.js, Citation Style Language, and many others 7 . This direction allowed them to design a 'bento box' display of the 'everything' search results. Though like the other libraries they do have tabbed options for searching, the 'bento box' display breaks down the results in discrete sections. They felt that some users slanted toward novice researchers and would have like to have a breadth of resources available.   While others have needs that focus more narrowly. The assessments they perform are rigorous and often, ensuring that areas worked on have led to improvements and also identify areas that need future work.
Below one can see that the discovery layer 'everything' search option is first, followed by the catalog, an articles only search, a database listing search, an online journal title search, and a search of the Library's website -all through the home grown interface 8 (fig. 10).
The search 'everything' breaks down the results within five boxes ( fig. 11, 12).

Temple University
Temple' s requirements were similar to the University of Michigan' s. Originally, they had two separate systems, Millennium for the catalog and Summon for article searching. Like the University of Michigan, they also have a department of developers. They have deployed Blacklight and designed a 'bento box' display of their search results. Their assessment studies and click tracking led to the one discovery system and the single search box.
Similar to the University of Michigan's interface, their bento box results display in five boxes -books and media, articles, databases, journals, and library website searches. Unlike the University of Michigan, only three of the five separate searches are available on the Library's opening page -books and media, articles, and databases as shown in figure 13. The bottom of the first column has a link to "Search our collections", that opens up an initial search page 9 which includes the five direct searches ( fig. 14, 16

Conclusions
Academic libraries have shifted from an environment of information scarcity to information abundance to the point where the catalog is no longer the primary link to the library's resources. And with that abundance, the common concern is the overwhelming number of results displayed in a discovery layer search. A concern for which each library has developed different solutions to accommodate the various types of searching. The decision to emphasize the catalog or a 'search all' box varies from institution to institution. Given local conditions, organizational culture, and past history with handling technical transitions with patron responses, many academic libraries have grappled with the transition in a myriad of ways.
All choices of the five libraries are mostly based on functional requirements and on the resources available to the institution -not only budget funds but also the ability to manage technical debt. The New School, Skidmore and Ball State made the decision to go with an industry product rather than a custom built one (Primo, EDS, and Summon respectively). The industry products 'out of the box' solutions do allow customization and integrations to external systems, of which both the New School and Ball State have taken advantage. The New School, for example, heavily customized their user interface because they had some development resources and chose to focus on integrations with external systems such as Atlas-System's Illiad, Ares and OCLC's D2D softwares (Rose, Jones, 2019). Skidmore has kept the functionality of the catalog, the discovery layer and other systems separate. This arrangement may have to do with Skidmore being solely an undergraduate liberal arts college and the library's organizational culture. The University of Michigan and Temple have large departments of developers and have chosen a custom built solution using open source software. The option to offer a bento box display of search results is only available through a customization of open source software -no commercial product offers it. In the future, it remains to be seen whether there will be enough of a perceived need for such an interface to motivate the commercial industry to offer it. Not all institutions have fully implemented a rigorous program of assessing their tools as the University of Michigan and Temple University. Where Ball State does not systematically assess their users' experiences, the New School does have targeted assessments that inform the design of their systems, and Skidmore's usability testing has not to a great extent informed the design of their systems.
Though this particular group of libraries is not statistically representative of the range of discovery layer implementations, it does lay out the pressures, constraints, and affordances that are part of each of the libraries' environments. Further examination with rigorous comparison of usability testing would have to be done to see how factors such as size of the institution, discipline areas, and undergraduate versus graduate versus faculty research are playing out in the wider field.