Barbara Opar and Barret Havens, column editors
Submitted by Rose Orcutt, Architecture and Planning Librarian, University at Buffalo Libraries
In Maya Gervits’s article “Citation Analysis and Tenure Metrics in Architecture and Design-Related Disciplines,” she articulately defines the issues surrounding the growing trends in citation metrics and analysis and how they apply to architecture faculty. Her article acknowledges that many existing systems of citation metrics do not provide as complete a view of the scholarly output of faculty in architecture and design-related disciplines as they do of the STEM (Science, Technology, Engineering, Mathematics) disciplines.
Though citation metrics tools each measure scholarly output in different ways, most tend to focus on the impact of publications in traditional formats. However, a new wave of “altmetrics” citation metrics tools, which include Plum Analytics and Scopus, go beyond traditional scholarly publications such as journal articles to track a wide variety of non-traditional works. The definition of altmetrics, according to altmetrics.org is “the creation and study of new metrics based on the social web for analyzing and informing scholarship.” Altmetrics bridges the gap in current citation analysis by capturing alternative publications and “real-time metrics of scholarly impact” (Sutton, 2014). This article will present several citation metrics tools widely used in academic settings today, including some of the more traditional variety along with a few that employ altmetrics.
In addition to assessing the impact of a single scholar’s publications and other works, a growing trend in higher education is the use of citation metrics systems by administrators to evaluate the impact of scholarly output at the institutional or departmental level (Hazelkorn). Two examples of subscription-based citation tools that provide measurable data for assessing institutional, departmental, and faculty performance are Academic Analytics and Plum Analytics. Both tools are prorated based on the institution’s full-time equivalent (FTE) but are expensive.
Academic Analytics is known as a “business intelligence” software tool or database that collects data on scholarly citations. It compares a university’s productivity discipline-by-discipline and tracks the institution’s overall performance. The database tallies faculty activities in these five areas of research: journal articles, citations, books, research grants, and awards. The citations are derived from Scopus. The database offers a number of different visualization tools to show data trends, patterns, and the strengths and weaknesses of the institution.
Plum Analytics (PlumX) is an EBSCO database and is similar to Academic Analytics as it also measures scholarly productivity discipline by discipline and the overall performance of the university by tracking traditional publications. However, it also tracks output using altmetrics, which take into account evidence of impact garnered from social media discussions, article views and downloads, news media mentions, conference proceedings, videos, blogs, tweets, grants, patents, presentations, clinical trials, book chapters, and more. The citations originate from PubMed Central, Scopus, and USPTO. Individual author records are used to populate the citation data. Graphs and comparison charts are easy to generate from the statistical output.
Scopus and Web of Science are both library subscription-based databases that analyze author output by tracking citations. As competing products, they cover all disciplines but have origins in the STEM disciplines. The cost of each is prorated based on the institution’s FTE.
Scopus is a relatively new database by Elsevier. It was introduced as a citation tool in 2004. The database tracks scholarly citations from peer-reviewed journals, books, patents, and conference proceedings from all disciplines and has a large international coverage. The metrics tracking covers author’s total citation and document counts, citations per year and h-index. Charts and tables can be manipulated easily. Altmetric, which will be discussed further along in this article, is a third party web application that has been integrated into Scopus. The application runs in a sidebar within the abstract page, providing additional altmetrics statistics. Scopus also announced the creation of their own Article Metrics app that will be made available at the end of this summer.
Web of Science (WOS) was created from the Science Citation Index which began in the 1960s by Eugene Garfield (Aghaei Chadegani et al., 2013). WOS includes the Arts & Humanities Citation Index, Science Citation Index, Science Citation Index Expanded, Social Science Index, Current Contents, conference proceedings, and two chemistry indexes. Citation reports are created on an author, displaying the number of times the author is cited, the average citations per item and the h-index in graph and chart form. The results of the reports are based on only those journals that are indexed in the Web of Science database collection. In March 2015, a press release on the WOS Thomson ISI website announced a collaboration with Google Scholar to enable citation count cross-checking.
Google Scholar generates statistics from the author’s own articles and then tracks their published work. The author creates and maintains his/her own profile which can be made public, graphs citations over time and generated reports on a publication’s h-index number. Google Scholar Metrics is free, and covers a large portion of scholarly articles published in the last five years but “only includes publications with at least a hundred articles” from that same time period.
Harzing Publish or Perish is a free, downloadable software program that retrieves raw academic citations from Google Scholar and Microsoft Academic Search. In addition to capturing journal citations in English, Harzing also includes LOTE (Language Other Than English) citations and book chapters. The reports include the total number of citations, number of citations per year and the h-index. Along with the Author Impact Analysis, another feature is the Journal Impact Analysis.
Altmetric, a fee-based tool, was founded in 2011 and according to its website, its mission “is to track and analyze the online activity around scholarly literature.” Data is collected from newspapers, government policy documents, and mentions of scholarly publications in online conversations on social media sites and elsewhere. A score is given based on volume of mentions, number of sources, and how often that author is discussed unless the article was published before 2011 then the score may be inaccurate. An embedded API donut badge displays a color-coded visualization of each social media type represented in the score and the number is placed in the middle of the donut.
ORCID is an open, community-driven platform where a scholar can create their own research profile and add publications and other works including “research objects such as datasets, equipment, articles, media stories, citations, experiments, patents, and notebooks.” An author creates a record of their scholarly work which is tracked by an ORCID iD, a unique digital identifier assigned to each author. This digital identifier distinguishes one author from another and is used by other platforms such as PlumX, Academic Analytics, and Scopus to enable cross-platform sharing of metrics data. ORCID maintains a public registry of the all the ORCID iD profiles which is searchable in order to facilitate communication and collaboration among researchers
Used together, altmetrics and traditional citation metrics systems provide a holistic profile of an author’s body of work. The use of citation metrics is an established trend with a tangible impact within the academic community in some disciplines and an emerging methodology among others. Statistical analysis of faculty and departmental performance is critically important to higher education administrators as they seek to demonstrate productivity and compete for funding. Architecture and design subject librarians can play a vital role in faculty tenure and promotion by advocating for the adoption, support and use of altmetrics as a way to produce a more well-rounded profile of the scholarly output of architecture and design faculty.
Aghaei Chadegani, A., Salehi, H., Yunus, M. M., Farhadi, H., Fooladi, M., Farhadi, M., & Ale Ebrahim, N. (2013). A comparison between two main academic literature collections: Web of Science and Scopus databases. Asian Social Science, 9(5), 18-26.
Hazelkorn, E. Rankings and the reshaping of higher education : The battle for world-class excellence (2nd ed.). New York: Palgrave Macmillan.
Sutton, S. W. (2014). Altmetrics: What good are they to academic libraries? Kansas Library Association College and University Libraries Section Proceedings, 4(2), 1-7.