In keeping with the research line the EC3 Research Group began several years ago aimed at unravelling the inner depths of Google Scholar and testing its capabilities as a tool for scientific evaluation, this time we have turned our efforts to finding new uses for Google Scholar Citations (GSC). Based on the information available on every GSC public profile, a procedure has been developed to collect data from the scientists working on a given field of study, and to aggregate that data in order to present metrics at various levels: authors, documents, journals, and book publishers. Thus, GSC data would presumably allow us to present a picture of the history and scientific communication patterns of a discipline. In order to explore the feasibility of this project, we decided to select the field of Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics as our test subject.
Once we’ve seen the picture of the discipline that can be observed through the data available in GSC, we also want to compare it to its counterparts in other academic web services, like ResearcherID, a researcher identification system launched by Thomson Reuters, mainly built upon data from Web of Science (which has been and still is the go-to source for many researchers in the field of research evaluation), and other profiling services which have arisen in the wake of the Web 2.0 movement: ResearchGate, an academic social network, and Mendeley, a social reference manager which also offers profiling features. These are the most widely known tools worldwide for academic profiling (1, 2). They offer researchers the chance of creating an academic profile, as well as the chance of uploading their publications, which are therefore available for other researchers to access, download, and comment upon. Researchers can also feed these databases with other kinds of data (tagging and following profiles, asking and answering specific questions) which might be useful for the rest of users in the platform. In addition, we also include the links to the authors' homepages (the first tool researchers used to showcase their scientific activities on the Web), and Twitter, the popular microblogging site, in order to learn how much presence bibliometricians have in this platform and the kind of communication activities in which they take part there.
In short, our aim is to present a multifaceted and integral perspective of the discipline, as well as to provide the opportunity for an easy and intuitive comparison of these products and the reflections of scientific activity each of them portrays.
This project can also be considered as an attempt to deconstruct traditional journal, author, and institutional (mainly university) rankings, which are usually built upon data from traditional citation databases (Web of Science, Scopus) and are based exclusively on journal impact indicators. In this product, we are using a bottom-up approach by analysing the documents that are either published by a group of authors associated with the discipline, those which are published in the main journals of the discipline, or those which use the most common and significant keywords in the discipline. This is done in keeping with the widespread notion that the impact of the various scientific units (documents, individuals, organizations, subject domains) should be evaluated directly, using appropiate indicators for each unit, and not by using proxys like the average impact of the journals where documents are published.
Google Scholar is the most comprehensive database for scientific information at the moment. It indexes a much larger document collection than any of the traditional citation databases, covering all kinds of document types (journal articles, books, theses, reports, conference proceedings…), and without any language or country restrictions. Mainly because of these reasons, Google Scholar is able to find a greater number of citations to the documents it covers than traditional sources with more restricted indexing policies.
In addition, we also open the door to new platforms that are offering scientific performance metrics, even if their meaning is still not yet completely understood due to their recent creation. We delve into these metrics to see what they can tell us about researchers’ communication practices. With this step, we enter the altmetrics conversation by presenting some data from the authors’ perspective, and not only from the perspective of the documents they publish.
This product has two main objectives:
- Displaying the diverse researcher bibliometric portraits that are generated by the various profiling platforms available at the moment (Google Scholar Citations, ResearcherID, ResearchGate, and Mendeley), as well as the communications patterns of these authors on Social Media (Twitter), in order to assess their coverage, the level to which they have been adopted, as well as to hypothesize on their meaning.
- Testing the completeness, reliability, and validity of the information provided by Google Scholar Citations for the purpose of generating a ranking that contains the core authors, documents, journals and book publishers in Bibliometrics.
We insist that our goal is to display the various “mirrors” in which the activity of researchers is being reflected nowadays, and to observe the state of the discipline as a whole by gathering the information about all the authors associated with it. The data speaks for itself, and its interpretation, as could not be otherwise, is left to the individual observer.
Juan Manuel Ayllón enjoys a four-year doctoral fellowship (BES-2012-054980) granted by the Ministerio de Economía y Competitividad (Spain). Alberto Martín-Martín enjoys a four-year doctoral fellowship (FPU2013/05863) granted by the Ministerio de Educación, Cultura, y Deportes (Spain).