Creation of the field team

A field team of librarians at ARL institutions in the United States and Canada was recruited to interview faculty members on their campuses about the digital scholarly resources they find useful in their work. ARL invited librarians from all its member institutions and other institutions involved with the Institute on Scholarly Communication to participate, and assembled a field team of 301 volunteers from 46 colleges and universities in the United States and Canada.[4] Librarians from three pilot institutions—the University of Washington, Cornell University, and the University of British Columbia—signed on to help develop the interview protocols. In some cases, the entire library staff assigned to faculty liaison work was asked by their director to participate; at other institutions, interested librarians joined independently.

The participating librarians embraced this project not only because of their interest in new digital scholarly resources, but also because it provided an opportunity for structured interaction with faculty members at their institutions. In fact, many participating library directors voiced support for this project as a means to initiate or continue deep engagement between librarians and scholars on campus. The value of this field team in facilitating conversations about digital resources between librarians and faculty could extend well beyond its role in this particular study.

Ithaka and ARL conducted Web-based training sessions to provide the field team members with necessary background on the project’s goals and to present the questionnaire that would structure their conversations with faculty. A question-and-answer session followed the presentation and the webcast was archived on the ARL Web site so that new field team members joining after that date could still benefit from the training. As the field work started, an electronic discussion list allowed members of the field team to share questions and tips with the entire group.

Among the topics discussed in detail during the training was how to define the new models of digital scholarship we were looking for so librarians could ensure that their conversations focused on similar resources. When the library field team participants conducted their interviews, they asked faculty members to identify for them the “online works you rely on to keep up with current research,” and, specifically, those that could be described as containing original scholarly work. They were asked not to focus on search engines or sites that provided only collections of links.[5]

Identifying new digital scholarly resources

Even so, faculty responses included quite a range of sites. From among the hundreds of sites faculty mentioned, ARL’s primary interest was in conducting a close investigation of a particular type of website: those resources that are the most central to scholarly communication.  The study, thus, focused on sites designed specifically to serve scholarly audiences, those with content deemed to be both original and scholarly. These criteria were considered essential, and these resources more likely to closely resemble the type of content created by faculty coming to the library seeking advice on developing their digital projects. “Original” was defined as born-digital material (content that appeared first on the Web site in question). In cases where the resource consisted of digitized primary source content, such as medieval manuscripts, we considered it original if the site facilitated some sort of new scholarly manipulation of the content, or if it was published alongside a layer of original scholarly annotation. Though digitized versions of print publications and search tools pointing to content hosted elsewhere may be extremely useful to scholars, they were outside the scope of this study. “Scholarly” resources are those authored by and for the scholarly community. This definition encompassed a wide variety of formal and informal resources, including peer-reviewed publications like e-only journals, but also sites that scholars use to share casual information or thoughts-in-progress, including discussion forums or blogs. Considered outside the scope of the study, however, were popular-interest resources, such as YouTube and Wikipedia. Although it would be fascinating to study the way that sites like these are becoming both the subject of and a resource for scholarly work, because the focus of these resources is not scholarly communication they were considered outside the scope of this study.

The data gathering process

Between April 1 and June 17, 2008, field team members conducted interviews with faculty members to ask them about the online resources they use that contain original scholarly work. The faculty members were encouraged to offer as many examples as they felt were relevant. For each resource cited, the field team member asked a set of questions including how the scholars use the resource, their opinions on how the resource accomplishes something innovative, and how often they use it. In addition, field team members asked a series of questions relating to the resource, including its method of selecting content and business model. In some cases, faculty members knew this information. When they did not, field team members conducted follow-up research about the sites after the interviews to verify factual information and to fill in answers to questions that the faculty member could not answer. Finally, field team members entered this information into a Formspring database set up by ARL. The field study conversations yielded 358 responses. Three of these were from interviews with scholars who said they used no digital resources at all; the other 355 described digital resources.

The Ithaka team then reviewed each suggested resource to determine if it met the agreed-upon definition of a new digital scholarly resource. About two-thirds of all responses—240 resources—met the requirement of containing some original scholarly content.[6] This set included several resources named by more than one respondent. The number of unique resources that scholars reported to the library field team participants was 206.[7]

At this stage, Ithaka staff assessed all of the resources in this group, and assigned each to one of eight categories of resource types for purposes of analysis. Ithaka then selected eleven representative projects and conducted in-depth phone interviews with project leaders and key staff to gain a deeper understanding of the mission, goals, and organization of the different publishing models. These interviews helped Ithaka gain insight into how their leaders think about strategies for creating and developing site content over time, metrics for understanding the site’s users, experimentation with technical innovations, and different sustainability strategies.[8]

This report relies upon three sources of data:

• Data gathered by the field team through faculty interviews.

  The method employed by the field team, known as convenience sampling, is commonly used in exploratory research. It suited the time and resource constraints of the project as well as the goal of creating a qualitative assessment of a specific type of resource. The information gathered by the field team helped guide us to examples of innovative digital scholarly resources that faculty are using today. Faculty members’ answers to the librarians’ questions also helped to explain why professors use certain resources, what they find most innovative and valuable about them, how often they use them, and whether they contribute to the resource as authors or editors. These data points provided a glimpse into the way that these digital publications are being incorporated into the workflow of some scholars.

• Data about the resources themselves.

  This information, supplied originally by the faculty member, and corrected or verified by the field team member, helped to describe the types of content on the site, methods of selecting content for the site, whether or not there is a peer review process in place, if there appeared to be institutional support for the site, and more. In addition, the team at Ithaka inspected the sites to verify as much of this information as possible.

• In-depth interviews conducted with the project leaders and key personnel.

  The interviews Ithaka conducted with the leaders of eleven representative projects provided a picture of just how these projects have grown and developed over time, the strategies that have worked (or not) for some of them, and the ways in which project leaders think about fulfilling their mission. The interviews were conducted, when possible, by both an interviewer and a note-taker.

By integrating these three sources of information, this project offers a snapshot of what innovation in digital scholarly resources looks like today. The report begins by assessing the ways in which faculty report using the new digital resources they recommended. Then, we examine eight types of digital scholarly resources, describing their content, the ways faculty report using them, and their strategies for success. Examples drawn from specific digital projects provide a fuller picture of the kinds of resources in use, the innovations underway, and the challenges their leaders face in sustaining them.

How do scholars use these sites?

How do scholars use digital resources to help with their research needs? Providing access to the most current research was by far the most frequent reason suggested by the faculty with whom field team librarians spoke. Facilitating exchanges among scholars and supplying useful co-location of works were also mentioned often, field librarians reported. Many faculty indicated that the resource they suggested accomplished not one, but all three of these goals. Scholars in different disciplinary categories—humanities, social sciences, and science, technical, and medical (STM) fields—tended to note different reasons for finding a resource useful. Librarians noted that across all disciplines faculty emphasized their interest in access to current research as the most important benefit of the resources they identified. Field librarians speaking with scholars about humanities and social sciences resources, however, were much more likely to find that faculty appreciated those resources for facilitating exchanges between researchers than did faculty nominating resources in STM subjects. This does not mean that scientists are not interacting with their colleagues using the Internet, but that they less frequently cited those types of sources, such as blogs and discussion lists, where “exchanges among scholars” is a primary function. By far, STM resources—data sites and e-journals were most often named—seemed to be valued for providing access to new research; we heard this response more frequently for STM resources than for any others. Faculty members told librarians that other aspects they appreciated in these sites included news and alerts about publications, conferences, and other developments in the field; and access to different types of content including data, primary source material, reviews, and teaching materials. [9]

How often do scholars report using these resources?

About half of the faculty in our study told field team librarians that they used the site they named at least weekly. This was similar for interviewed scholars in all disciplines. Certain content types, however, seem to demonstrate specific patterns of use; faculty who used data sites, for example, told librarians that their usage was not constant, but rather varied depending on their current research project. In addition, it is interesting to note that many of the resources scholars mentioned using daily were those considered outside of the scope of this study, such as search sites like Google, reference sites like Wikipedia, and other finding tools like online catalogs and aggregations of links pointing to content held elsewhere.

In what ways do scholars engage with new models of digital publishing?

Scholars are both the producers of and the audience for digital scholarly communications. While many of the ways in which scholars contribute to print publications also exist in the digital world—writing and peer-reviewing articles, editing journals, reviewing books, etc.—electronic media also facilitate new forms of scholarly contributions. Examples of this include moderating a discussion list, contributing data to a shared repository, posting thoughts and annotations on a blog, or editing a wiki. Almost half of the interviewed faculty contributed to the resource they suggested to the librarians. Nearly a third of these contributions consisted of offering content; other forms of participation included contributing reader commentary, contributing editing services, managing a site, providing peer review to e-only journal articles or moderating a discussion list.[10] And not surprisingly, it appears that those scholars who nominated a site to which they contribute were more likely to tell librarians that they visit that site daily. Our findings suggest that scholars at all stages of their careers are experimenting with participation in new models of digital publishing. We found no evidence that the assistant professors with whom librarians spoke—presumably younger and more immersed in a digital world—contributed to the resources they suggested more frequently than their more senior colleagues. Perhaps there are other factors encouraging senior faculty to participate (the protections of having tenure, invitations to participate from other colleagues) and discouraging younger faculty (a sense of caution regarding where to publish work before receiving tenure). Further research would be needed to better understand the factors motivating or discouraging faculty to participate in different forms of digital communication.

Figure 1. Frequency of Use of Digital Scholarly Resources (n=240)

*“Infrequently” includes resources that scholars reported using a few times a year, annually, or less often.

E-only journals

E-only journals—the resource type suggested most frequently by the scholars interviewed for this study—are shaped by the same forces in online academic culture as their print forebears, including credibility, prestige, and tenure and promotion decisions. They strongly resemble traditional print journals in terms of editorial guidelines, peer review, and a well-defined scholarly mission, while also incorporating a variety of innovations made possible by the digital environment. The journals named by scholars in this study represented a nearly even cross-section of disciplines, including humanities (17 titles), social sciences (12 titles), and STM (22 titles).

Incrementally, some e-only journals are using digital technologies to improve the publication process in a variety of ways. Faculty frequently told librarians that speedy access to new work, the open access model, and the benefits of being part of a network or online community of scholars made the e-only journals they suggested innovative. Other innovations mentioned relate to novel features like associated discussion lists or public commenting. PLoS, a collection of online journals in the biological sciences, includes the option to comment on an article, as does Industrial and Organizational Psychology: Perspectives on Science and Practice, which encourages both peer commentary and responses by an article’s author. Other innovations in this category take advantage of the flexibility that the digital environment allows to accelerate the speed of publication and the peer review process, and to explore the possibilities of including new media formats.

Ecology and Society is an example of an e-only journal that conforms in many ways to traditional ideas about what a journal should be, while also exploring the advantages and opportunities of the new online publishing environment.[12] For example, Ecology and Society publishes new articles as soon as they are ready in an “Issue in Progress,” allowing readers to access new content much more quickly than they would be able to if they had to wait for several articles to be published in a print issue. Every six months, the issue is “closed” and given a volume and issue number, facilitating citation and enabling traditional citation analyses to be applied. This strategy nods to the imperative to catalog and cite scholarly content in familiar ways while still facilitating more rapid dissemination of content.

Digital publication can allow a range of types of written content to co-exist more easily than they would in print. The peer-reviewed journal The Asia-Pacific Journal: Japan Focus includes scholarly articles, English translations of articles originally written in Japanese, as well as shorter, more journalistic or thought pieces, and many articles that include extensive photographs and multimedia. The journal editor feels strongly that this addresses the needs of his online audience for material on contemporary issues or offering historical perspectives on contemporary issues. Compared to other Asian journals, he told us, Japan Focus is “a different kind of hybrid . . . Other journals may limit publication to research articles of 6,000 to 10,000 words, extensively footnoted . . .” Because the journal’s position is that much important writing in contemporary affairs is being done by journalists or by scholars adopting journalistic techniques, their editor believes online readers will benefit most from a range of work: “short, hard-hitting, direct articles to long, heavily footnoted research articles.” [13]

Concerns persist in the academy that publication in e-only journals will be perceived as less prestigious than publishing in print. Will the work be cited in the best journals? Will it be considered a legitimate publication by a tenure review board? Misperceptions about the level of peer review of open access publications have contributed to these concerns, although leaders of the open access movement have continued to argue that quality and cost are not synonymous, and that notions of access and prestige can be separated.[14] [15] Online journal publishers take strategic action in this area to re-create the measures of legitimacy and excellence that long-established scholarly publishers enjoy due to their longevity and reputation. The vast majority of e-only journals in our sample rely on a peer review process to regulate the quality of their publication. Furthermore, the publications that instead employ editorial selection tend to describe themselves as “magazines”—publications for which one would not expect formal peer review. Experimentation with peer review among the titles we saw has come mostly in digital management of the process. For example, Ecology and Society has a custom-built system to help automate the process of distributing manuscripts and reminding reviewers to respond, and Western Journal of Emergency Medicine developed an online training module for reviewers. The few journals in our sample that experiment with enabling open and public peer review alongside traditional peer review, like Atmospheric Chemistry and Physics (see inset below), see limited scholarly participation in the public process, emphasizing the strength of cultural norms about traditional blind peer review for the titles we examined.

Experimenting with Public Peer Review: Atmospheric Chemistry and Physics

Once the editors of Atmospheric Chemistry and Physics determine that a submitted article meets basic standards of quality, it is posted to the site as an open access discussion paper. For eight weeks, anyone may read and comment on the paper; these comments are posted and archived alongside the paper itself. After this period, the paper undergoes a more traditional peer review process by referees who have the option to remain anonymous.

This novel process has not hurt the credibility of the journal; Atmospheric Chemistry and Physics has the highest ISI ranking of any journal in its field. However, the site only receives about one comment from a member of the public for every four discussion papers posted, suggesting that their core audience has not yet embraced this modified peer review process.

Many of the e-only journals that emerged from the field study included some form of multimedia content. In many journals, the digital environment enables the publication of data visualizations, large data sets, or audio and video clips that serve to illustrate the text of scholarly articles. A few publications, like JoVE: Journal of Visualized Experiments (see inset below) are making multimedia a more central element of the work.[16] However, it is worth noting that the pressures of traditional scholarly publishing may affect the opportunities for an e-only journal to innovate in this way. The editor of Ecology and Society told us that though they tried to promote the innovative use of digital technologies in the articles they published, they found that many times the scholars submitting the best research were reluctant to incorporate multimedia elements, instead presenting their work in more traditional forms. Faced with a conflict between including more innovative multimedia content and including the best-quality scholarship, they choose to publish the more traditional works.

Expanding the Definition of a Journal: JoVE’s video articles

JoVE: Journal of Visualized Experiments has been explicit and successful in its effort to be considered a journal, referring to its contributions as “video-articles.” The founders of JoVE seek to speed up the knowledge transfer that currently takes place in laboratories, researcher to researcher. While the “methods and materials” section of a scientific article currently serves this function, the founders realized first-hand just how difficult it was to re-create experiments, a critical aspect of the scientific process. As they point out on the Web site, “written word and static picture-based traditional print journals are no longer sufficient to accurately transmit the intricacies of modern research.”

Most of the e-only journals that emerged through our study use an open access model. In fact, the few examples of subscription-based support were for e-only journals published by commercial publishers or scholarly societies; the independent titles tended to be open access. Even when editorial labor is donated, however, publications still need to generate revenue to support costs such as Web hosting and copy editing. Many of the e-only journals we found have in-kind support from their host institution, in the form of server space, technical support, or the contributed staff time of programmers. Other revenue-generating strategies we observed included soliciting donations from readers, advertising, and (particularly in STM fields) author fees.

Reviews

Reviews of scholarly works have emerged as an important form of digital scholarly publishing, meeting a real need in the scholarly community for rapid notification about and evaluation of new works. These articles, which analyze and summarize the arguments of recently published monographs or scientific studies, have long been important in many disciplines, where scholars appreciate this means of learning of new works and of having the guidance of a peer to place the work in context. And the process is valuable for the authors themselves, as a strong review in a well-regarded publication can be essential for scholars who hope that their work will be embraced by scholars and purchased by libraries. Unfortunately, the process of writing, editing, and publishing a review in a traditional print journal can take so long after the monograph’s publication that one of the major benefits of the review—to help scholars identify the best new scholarship—can be greatly diminished.

Ten different online sources of reviews were mentioned by the scholars who spoke with the field team librarians. Six of these resources review works in the humanities, reflecting the long-standing importance of the monograph in that scholarly community. One resource focused on social science, and three covered STM content. Several sites were mentioned by multiple scholars, including the Bryn Mawr Classical Review in the humanities, and UptoDate and Faculty of 1000 in medicine and biology. Regardless of discipline, scholars universally reported turning to these resources for the same reasons. Every scholar who reported using a review site said that the resources provide access to current research or updates on developments in fields of interest.

The digital review resources innovate in several ways. First, digital reviews are not subject to the space restrictions of their print cousins. H-France Review’s Editor-in-Chief was pleased not to have to restrict authors to a short word limit, allowing space to include a detailed review of the literature in the book review.[18] This context-setting is not always possible when space is in short supply but is an extremely valuable element in the review for the scholars and students reading it. Not only can digital reviews be longer, but a greater number of reviews can be written by a widely distributed group and published at low cost. In addition, digital publication enables reviews to be published as soon as they are prepared, without a wait for a new print cycle—a major benefit both for authors and readers. For example, the Bryn Mawr Classical Review strives to deliver “a review a day, every day,” to the nearly 10,000 subscribers to its e-mail list.[17] The greater volume of digital reviews (and the fact that faculty are often able to receive updates about new reviews via e-mail) may contribute to the fact that many of the scholars who use reviews reported relying on them daily or weekly.

Establishing Legitimacy for Digital Book Reviews: Bryn Mawr Classical Review and H-France Review

While all scholarly publications must establish credibility in their field, this is especially important for e-only publications. Reviews use a variety of techniques to demonstrate that their resource can be trusted to have high-quality content. Founders of the Bryn Mawr Classical Review, the second-oldest e-only humanities journal, addressed this both through the early involvement of prominent scholars and also by initially publishing a print supplement to the review to help reassure readers of the content’s quality.

In addition, establishing high standards for submissions can help reviews establish credibility. While book reviews rarely engage in the double-blind peer review process favored by many scholarly journals, they often use a non-blind peer review process that allows them to carefully manage the quality of the content accepted for inclusion in their publications. H-France Review requires that all of their editors have both a PhD and a published book, and only those with a PhD are permitted to submit reviews to the journal. In addition, the editor carefully manages the assignment of reviews to encourage participation of senior scholars in the field and insure that significant new works and works by new scholars are assigned for review with particular care.

The low cost of adding digital content allows scholars writing for a digital format to engage more substantively with the work they are reviewing, while not reducing the number of works that the publication reviews. Here, however, book review editors still face an “old media” challenge. While the book reviews may benefit from the economics of online space, they must still confront the high cost of mailing printed monographs to an international body of reviewers. The director of one highly successful review site told us that his single greatest expense each year is the approximately $10,000 needed to mail books to reviewers around the world.

Preprints and working papers

Preprint and working paper servers provide scholars with access to new research and permit them to share their own work without the delay a journal’s lengthy peer review and publication process can cause. Of course, there are many vehicles for scholars to exchange early versions of their work. Scholars may post preprints on personal Web pages or in institutional repositories, share them through society-sponsored channels, and e-mail draft copies to networks of trusted colleagues. In addition, dedicated preprint servers play an important role in certain disciplines. Our study results suggest that the landscape for these servers is dominated by the oldest, largest preprint servers like Social Science Research Network (SSRN) and arXiv, although we also found smaller working paper exchanges in some niche fields as well.[19]

We classified ten of the resources faculty suggested to field team librarians as preprint or working paper servers. Two of these resources were humanities-focused, six were in the social sciences, and two were in STM fields. The scholars who suggested preprint servers tend to use them very frequently. We suspect that this heavy use may be due to several factors. First, since the barriers for publication on these sites are low, often involving just a simple vetting for broad disciplinary relevance, the volume of new content appearing on preprint servers can be significant. arXiv, the preprint server hosting papers in physics, mathematics, computer science, quantitative biology, and statistics, receives thousands of new papers a month, with over 5,000 in July 2008 alone. This creates an incentive to check back frequently to learn about new research. In addition to using preprint servers to learn about developments in the discipline and the new work of their peers, many respondents reported using the servers to share their own work, as well.

Although our sample shows that niche working paper exchanges exist in some fields, these results suggest that it is still the older, more traditional servers that attract the most use. SSRN, a preprint server focused on economics, business, and law, while also expanding into the humanities, was mentioned by eleven different scholars, and arXiv was mentioned by seven; the other preprint servers were each mentioned once. These sites are large—SSRN contains over 190,000 working papers, and arXiv has over 490,000. Almost all the scholars who said they contributed to preprint servers contributed to SSRN or arXiv, and a large majority of the scholars who told us they use preprint servers daily or weekly use one of these sites.

Nearly all of the preprint resources we examined are open access. One exception was the National Bureau of Economic Research (NBER), which offers institutional subscriptions to the working papers NBER scholars produce. Even SSRN, a commercial site, makes any paper uploaded voluntarily by a researcher freely available, though it generates its revenue through institutional subscriptions to curated networks of content and through its Partners in Publishing program. The other preprint sites we looked at make their content available for free and had few apparent strategies to generate revenue outside of grants and support from host institutions.

In the Footsteps of arXiv: PhilSci Archive

PhilSci Archive, which serves scholars in the philosophy of science, further demonstrates how the preprint culture of one discipline may influence practices in adjacent fields. Philosophy of science is a discipline that overlaps significantly with theoretical physics. “Philosophers of science have a strong interest in all sciences including physics, and PhilSci Archive was inspired in large part by the success of arXiv. Since the three founders of PhilSci Archive (Clifton, Earman, Norton) all work or worked in philosophy of physics, we knew the model of [arXiv] very well and just thought that philosophy of science should have the same thing,” according to founder John Norton.

Our study results underline the importance of disciplinary culture in influencing the extent to which preprint sites are valuable. The field of economics, for example, has a particularly strong tradition of working paper exchange; NBER distributed printed and bound working papers for decades before use of the Internet was widespread. The abundance of economics-related preprint and working paper resources highlighted by our study may be explained by the fact that the distribution and use of this kind of literature has been a part of the culture and workflow of economists for a long time. The same may be said for arXiv, which has become an important site for those in physics, math, and computer science. Scholars in other STM fields, however, have been much less likely to post preprint work.[20] One attempt to change this is Nature Precedings, the preprint server for the biological sciences started by the Nature Publishing Group; since 2007, more than 1,100 papers have been uploaded to the site.

While large preprint resources are expanding into new disciplines—SSRN recently branched into the humanities, for example—others are content to deliver a service to a well-defined niche audience. PhilSci Archive, for example, focuses on the philosophy of science, taking care to limit the submissions to the archive to just those addressing topics in this field. While the project’s leaders would like to see greater participation from across the full range of philosophy of science, the Archive’s greatest strength today is within “the sub-niche of philosophy of physics.” Its leaders are willing to be patient, though, and are in “no hurry” to force the issue “since pressing people who are reluctant will surely backfire. The success of the archive itself will be its best advertisement.”[21] While some niche preprint servers like PhilSci Archive model themselves after existing sources, other sites experiment with newer models for exchanging work. The Online Feminist Philosophy Draft Exchange, for example, utilizes a Google Group to exchange working papers.

Encyclopedias, dictionaries, and annotated content

This category of new digital publication includes resources attempting to provide comprehensive, authoritative reference for a topic, as well as resources that layer primary source material with definitive scholarly commentary. Whether or not they explicitly call themselves “dictionaries,” “encyclopedias,” or “documentary editions,” they all in some way declare their mission to methodically gather valuable materials on a given topic, often subject to a rigorous review process. Our sample included thirteen humanities resources, three social science resources, and eight STM resources that fit in this category. Most of the resources we found through our field study are completely open access, although a few require some minimal level of registration for visitors.

Articles in the Stanford Encyclopedia of Philosophy (SEP) are individually authored, often by leaders in the field, and may contain the level of analysis and footnoting seen in journal articles. SEP describes itself as a unique “scholarly dynamic reference work,” priding itself on its ability to maintain high academic standards while benefiting from the expertise of over 1,000 professional philosophers who contribute articles. Encyclopedia of Life, which describes itself as an “online reference and database” of information about Earth’s 1.8 million known species, encourages contributions from the lay public but has a team of experts to authenticate and select the material that will ultimately appear in each entry. Partnerships with data-gathering projects including Fishbase and Tree of Life provide additional sources of authenticated content that the project will use.

Images Before Words: Visualizing Cultures and Image-Driven Scholarship

One of the most vibrant examples of experiments in multimedia scholarship, Visualizing Cultures provides a place for scholars “devoted to transcending the printed word and hard-bound text” to create original works on topics relating to Chinese and Japanese social and cultural history. Called “units,” these works consist of four elements: an essay, heavily illustrated by images and other media; a visual narrative (almost an image essay) annotated with captions to guide the reader through the sequence of images; an image database, including all images used in the essay as well as additional ones on the topic; and when available, video clips on the topic which may include author commentaries, interviews, animation, and source footage.

The site, first launched in 2002 at MIT on their OpenCourseWare platform, is original in the types of multimedia, image-driven scholarship it makes possible, and in the project’s many partnerships with the academic and cultural institutions that provide many of the images available on the site.

While encyclopedia projects like these attempt comprehensive coverage of a topic through articles summarizing current knowledge, other sites feature primary source content at their core. The digital environment enables scholars to publish commentary and annotations around this content, making them richer forms of publication than simple libraries of digital images. For example, Roman de la Rose Digital Library is a collection of digital surrogates of versions of that medieval illuminated text, whose originals are dispersed in special collections around the world.[22] The project allows side-by-side comparisons of digitized manuscripts that would be impossible otherwise. In addition, scholars contributed to the development of metadata for these digital surrogates, based on different critical interpretations of this work. The digitized texts are therefore searchable based on criteria reflecting scholarly output—and the research enabled by these searches will lead to new scholarly conclusions not possible in an analog world.

Several projects in this group are attempting to engage a wide community of scholars to contribute content, benefitting from the decentralization that new technologies allow. Examples of this include the Stanford Encyclopedia of Philosophy and, on a more modest scale, PlanetMath.[23] This model offers some benefits of scale: a decentralized format allows scholars to contribute far more and more quickly than a print publication of comparable size; updating entries—the bane of a reference editor’s existence—can be managed online, protecting against the obsolescence of the work minutes after printed copies leave the press; and the work of this updating can be entrusted to the person who wrote the article in the first place, in whose interest (in theory) it is to have only the best, most accurate information available.

This model is not without its challenges, though. While digital encyclopedias and dictionaries built on principles of crowd-sourced editing, like Wikipedia, can benefit a great deal from decentralizing content generation, scholarly communication still emphasizes the necessity of expert editorial vetting to determine (or at least approve) who will be creating and editing the entries. In the case of PlanetMath, getting mathematics professors and graduate students to contribute articles was only the first step. The founders quickly ran into trouble with “orphaned” entries, abandoned when the authors who “owned” the entries either moved on or simply stopped contributing, and resulting in entries that quickly became obsolete. By instituting a “use it or lose it” policy requiring that corrections to entries be made within a certain period of time, PlanetMath now manages the editorial process more closely, allowing articles to be transferred to new authors when necessary.

Data resources

The sciences were among the first fields to use technology to aggregate and share the results of research. For example, one of the projects investigated here, the Protein Data Bank, can trace its roots to the database its founders created in 1971.[24] Today, this and other data projects are now available through the Internet to scholars around the world who can contribute their own data to them and harvest the data of others for their own research aims.

The field team interviews yielded 41 examples of data projects, a number second only to e-only journals in our sample. In some cases, these sites host the data output of a particular scientific endeavor for others to use and analyze. In other cases, the database itself is a dynamic entity, allowing scientists to deposit the output of their individual work, making it a valuable tool and source of reference for others, who contribute their work and benefit from the scale of the communal data. For example, the RCSB Protein Data Bank, part of “the single worldwide depository of information about the three-dimensional structures of large biological molecules,” collects data on protein structures from scientists around the world, who are often required by funding agencies to submit this information. Once submitted, data is reviewed and validated by the project’s over 30 US-based, full-time staff before being made freely accessible via the Internet.

Still other projects have taken the notion of data contribution a step further, beyond the walls of the academy. Community data initiatives[25] such as these aggregate the collective observations of large groups of users to build a single source that can have valuable applications to research. For example, by collecting the recorded observations made by amateur bird-watchers, eBird has been able to develop a large set of data regarding bird sightings, which proves valuable both to the scientific research community and to other non-academic parties interested in avian migration patterns.[26]

Harnessing the Power of Users: eBird

eBird, a community data project, relies on devoted amateur bird-watchers to supply their own observations to a large central database, where the results can be mined by professional ornithologists and environmentalists. The founders were aware of the intense birding activity taking place around the country, but realized that simply asking birders to enter their data online for research’s sake was not enough. “People did not care about helping birds; engagement was an issue... initially we weren’t giving them enough candy; enough tools to get them excited and to participate in the project...” So, they invested some time, spoke to people in the bird-watching community, and determined that users would find the most value in “the ability to manage and maintain their lists online, to compare their observations with others’ observations.” Once these tools were in place, eBird’s founders noticed a steep increase in participation: from 40,000 observations a month, to 1 million observations per month today (only 4 years later).

When asked how the data resources contribute to their scholarship, most faculty cited the importance of “access to current research.” The elements that faculty and librarians found the most innovative regarding the online databases, however, were much more clearly expressed as the benefits of having a variety of data all in one place, and being able to search that data. While having current data is certainly important, the increased functionality that the databases themselves allow is what is truly new. Nearly half of the faculty responses indicated that data resources were used either daily or weekly. The “other” responses almost all offered variations on this response by one faculty member: “intensely when working on a project, rarely otherwise.”

Many of the data projects in our sample are supported by grants from foundations or government sources. For example, the Protein Data Bank has been able to sustain itself through a series of grants, in large part due to the prominence and importance of the resource to the scientific community. “Last time we counted, we had 16 different grants worldwide to fund this thing; 8–9 in the US from different agencies,” one of the founders told us. Because of the unpredictability of the revenue stream, and the labor involved in monitoring and applying for so many grants, project leadership feels this model is not ideal, and has begun discussions about other sustainability options to pursue.[27]

Many data projects also receive some kind of support from their home institutions and some, though not many, have tried advertising or corporate sponsorship. Chemspider offers ads on its home page, as well as “compound-based advertising,” which allows advertisers to display ads in proximity to materials relevant to the products being advertised. Similarly, eBird has a corporate sponsor in Zeiss, a manufacturer of the optic devices that birders use.

Blogs

Though blogs may receive more attention in the popular media as vehicles for political commentary, celebrity gossip, or personal musings, this form of digital content is being put to interesting use by scholars, as well. In some ways, blogs may be thought of as an “updated” version of the traditional listserv (described in more detail in the section on Discussion Forums, below). For example, the scholars who created PEA Soup, a blog focused on philosophy and ethics, were eager to re-create the “water cooler conversations” about their work—working through new ideas informally—that they otherwise lacked as members of relatively small departments.[28] “We were more interested in the electronic equivalent of walking down the hall to talk to your colleague, but with people all over the country and world,” according to one of its founders.

Accelerating the Speed of Scholarly Discussion: PEA Soup

PEA Soup, a blog for scholars of philosophy and ethics, serves as a tool for scholars to explore and develop nascent ideas with a community of interested individuals. PEA Soup’s founder commented that this method of communication was very important in allowing them “to work through their ideas in an online community,” and to “air ideas and get feedback and comment on current issues.”

The benefits of a tool like this may be great; we heard one scholar speculate that, with blogs, a scholar can get “feedback in five days that once would have taken three months to get back. People could come to a settled view (on a philosophical question) in a number of months that even a few years ago might have taken years.”

Blogs are being employed across the disciplines—the study turned up seven humanities blogs, three social science blogs, and five science blogs—and tend to be read daily or weekly by the scholars who reported using them. Many blogs like RealClimate, a resource focusing on advances in climate science, are used primarily to alert readers (both scholars and laypeople) to new and interesting research and events in their community and field. Blogs like these add an interesting layer of commentary to published literature. Scholars told us that one blog like this “gives frequent updates of researchers’ opinions rather than just facts,” and another “attracts well established, well known writers in the field,” with lead articles that are “substantial, although no one is sharing original research.” Blogs also add value to resources focused on other sources of content, like e-only journals or encyclopedias. In addition to the fifteen blogs described here, at least 29 other resources from our sample include blogs as a supplemental form of content.

Though blogs are clearly an informal method of scholarly communication, posting restrictions allow them to maintain some degree of quality control and content vetting. Although all the blogs in our sample were completely open access, requiring no registration for readers, and although in many cases any reader may post a comment to a blog, the right to author blog posts must be specifically granted. This is not to say that blogs are closed endeavors—PEA Soup, for example, has 46 contributors, and frequently invites new ones to join. However, unlike discussion lists where all readers of the list are also potential contributors to the list, blogs tend to be a more “controlled” form of informal scholarly communication, allowing a limited number of authors to post work to a much wider audience. The higher bar for contribution was reflected in conversations field team librarians had with faculty. Only about a third of those scholars who nominated a blog indicated that they contributed to it through authoring, editing, or adding reader commentary; for comparison, close to two-thirds of those who nominated discussion lists contributed content to them.

Early ambiguity about the place of blogs in scholarly workflow may have limited their uptake to some degree at first. While this has not been a problem for PEA Soup contributors, its founder mentioned hearing worrisome stories about young scholars being denied interviews or jobs when other scholars mistakenly assumed that informal, unpolished ideas that they had published on a blog were representative of their formal scholarly output. Rumors like this may have made some scholars reluctant to post on blogs, but we heard that the general scholarly community is increasingly coming to understand that while blogs may be an interesting (and citable) record of the development of scholarly thought, they represent interim stages, not a final product.

Although some larger resources, like the ScienceBlogs network of 74 science-related blogs, have begun to experiment with advertising, most of the blogs that emerged through our study operate without advertising or other forms of earned revenue. Many are built on free blogging software like Blogspot, LiveJournal, or WordPress. PEA Soup pays $60 a year for their blog to be hosted and its four founders split this minor cost.[29] While they have considered adding advertising, they felt that the potential for revenue generation would not be worth the added clutter to the blog; in addition, they worried about the potential for controversy if one of the blog readers were to have an ethical objection to an advertiser. For many blogs, extremely low costs mean this lack of revenue may not be a problem.

Discussion forums

Although discussion forums—message boards, listservs, or other sites to which scholars can post comments and respond to others’ thoughts—have been used by scholars nearly since the advent of the Internet, they are still important and heavily used in many disciplines today. Some examples of innovative approaches to facilitating discussion emerged in our study, but most discussion forums took the form of an archived e-mail list. Perhaps the “oldest” of these new models of digital scholarly publishing, the continued importance of discussion forums is likely due both to their long-established presence in certain communities, and to the fact that their relatively basic technology is well-suited to facilitating a simple form of communication appreciated by scholars. By facilitating informal exchanges and enabling the quick dissemination of announcements and updates, these resources still play a critical role in many academic communities in the humanities and social sciences.

Twenty-one resources suggested by scholars fit into the category of discussion forums; about three-quarters of these were traditional discussion lists, or listservs. Fourteen of the resources are humanities-focused, six are in the social sciences, and one includes all disciplines.[30] Our results included five different resources affiliated with H-Net, an organization of over 180 discussion networks in the humanities and social sciences. Comments from scholars about these twenty-one resources highlight the role discussion forums play in connecting scholars to each other and to new ideas. Scholars reported that discussion lists:

• Allow the scholar “to keep in touch with everyone, all the time,” and to “keep abreast of research in the field.”
• Bring “together a large quantity of timely, reliable information online.”
• Provide “daily updates in the field” and allow “scholars to post queries and get immediate (and delayed) responses that are archived and available to the entire community of scholars.”
• Act as “a forum for scholarly discussion prior to a peer review process; [cover] diverse topic areas; open to anyone; discussion of recently published works; the dialogue is more like a conference dialogue with a wider audience.”

Although these e-mail lists enable a wide range of interaction, some scholars report that the lists are of only variable utility. In Ithaka’s 2006 report on scholarly communications in the history discipline, we found that every historian with whom we spoke used H-Net in some form. While the lists sometimes facilitated discussion about topics of broad interest, they were used more often to post basic questions about, for example, obtaining access to a particular archive or finding recommendations for good teaching materials. We found that the lists were used relatively rarely to work through nascent ideas in detail, or to share preprints or working papers.[31]

Although scholars may not use discussion lists to engage in in-depth scholarly exchanges, they still use them heavily as sources of information and ways to communicate with others. Most of the scholars who nominated these discussion forums author posts or contribute commentary to them. Discussion forums also saw more daily use by the faculty who mentioned them than any other content type in our study. To some degree, this high level of use is a result of the format of the resources; subscribers receive listserv updates in their inboxes, so “use” is as easy as opening an e-mail. The informal nature of a discussion forum may contribute to the frequency of its use as well.

Though the H-France Review program generates many of the costs and faces many of the issues of the review sites described above, the discussion list itself is relatively inexpensive to run. H-France spends a few hundred dollars per year on Web site hosting, and is able to cover these costs from small grants and member donations. Many of the independent discussion lists in our sample likely have similar costs. Because listserv technology is inexpensive, many of these resources do not need independent sources of support. Five discussion lists in our sample use either free software like that provided by Google Groups, or were created with mailing list capability provided by a scholar’s institution, and others were supported with mailing list tools provided by a scholarly society or association. One notable exception to this among the discussion lists is H-Net; it combines university support, grant support, donations, and some revenue from click-through book sales to support the organizational structure that houses many lists.

The Continuing Relevance of the Listserv: H-France Forum

H-France is not a “new” resource — it was founded in 1991 and has had a long history of connecting scholars in its discipline. The list was originally started with a small group of scholars so “that the types of conversations that occurred around the coffee machine [at Parisian archives] would occur online,” according to H-France’s Editor-in-Chief. Today, it boasts a subscriber list of over 2,300 scholars of French culture and history.

While newer forms of online discussion are available today – chat and instant messaging, real-time forums and discussion boards – the listserv has remained a mainstay of scholarly communication, allowing slightly more formalized communication than other, newer methods. Restricted access, list moderation, and list archiving are some elements that lend a greater sense of credibility and enduring value to the exchanges of ideas and information via the lists.

Although most of the resources in the discussion category are more traditional discussion lists described above, we also found evidence that some organizations are starting to innovate with new technology that facilitates informal interaction between scholars. For example, the Emerging Scholars Interdisciplinary Network was established to provide a networking forum for young scholars of color doing research in health-related fields. The resource includes a “Scholars Only Lounge” where members can discuss issues, share information, and read news alerts. Incorporating aspects of Web 2.0 functionality, the resource also allows members to create personal profiles, and to develop their own mini-networks around topics of interest. While resources like this suggest that Web 2.0 technology will enable new forms of scholarly exchange and interaction in the future, our study indicates that there may still be a place for more traditional listservs and discussion forums for some time to come.

Professional and academic hubs

While the majority of resources faculty mentioned focused on delivering one type of content, such as journal articles or data, a group of resources stood out for combining a wide range of content types in a single site. These “hubs,” often the digital portal for a scholarly society or professional membership organization, may offer e-only journals, reviews, access to preprints and conference papers, grey literature, blogs or newsletters that disseminate timely content, and functionality for networking with other scholars. Although these sites combine content from many of the other genres we describe in this paper, we consider them separately here because their core innovation relates to their particular mix of multiple content types.

One-Stop Shopping: Alzheimer Research Forum

The Alzheimer Research Forum brings together a variety of content types related to the study of this disease. The resource in part serves as a convenient way to access content published elsewhere; its paper-search functionality supports targeted searching of Alzheimer-related papers in the PubMed Central database, including papers recommended by ARF as being of particularly high quality. The site also includes original content, publishing news updates with brief articles and reviews of important developments and upcoming conferences, grant announcements, and job postings, as well as reference material. In addition to more traditional message boards/discussion forums, the sites also include user-generated content such as a “Hypothesis Factory,” a forum in which people can post and comment on others’ ideas, and a Knowledge Base which allows users to publish comments and annotations on materials.

Many of the scholars who described these hub resources find them innovative or valuable because they are portals, or “one-stop shops” for information. A scholar who uses IBMS BoneKEy, the web portal for the International Bone and Mineral Society, for example, described the site as combining “original articles and commentaries with other tools and resources, providing a portal-type approach to specialized subject information, continuing education, news, directories, employment services and retail offerings.” Another scholar said that the resource Information for Practice “represents a synthesis of information gathered by and directed toward social work practice professionals.”

Thirty-four of the resources mentioned by scholars are best described as this kind of professional or academic hub. Seventeen of these were in the social sciences, and sixteen focused on STM subjects; several were in applied branches of these fields, like nursing. Humanities hubs were rare; the study sample includes just one hub focused on music that fit this category. While most of the scholars who mentioned these sites rely on them primarily as clearinghouses for relevant information about a topic of interest, many also use hubs as tools to locate and network with colleagues, and to keep up to date with news, grant announcements, and conference information.

Large sites such as these require many resources to build and update regularly. Because many of these resources are built as the portal or Web-presence for a scholarly society, that society’s membership fees help to finance the sites. Perhaps because these large sites likely attract large audiences, they frequently support themselves in part with advertising or corporate sponsorships, as well.