A Metric for Academics: A Personal Suggestion

Every year in the US, and at various intervals in other countries, academics must pull together what they have done to provide administrators with the data required for their indicators of performance. Just as metrics provided baseball teams with a new tool for more systematically choosing players, based on their stats, as portrayed in the popular film Moneyball, so universities hope to improve their performance and rankings by relying more on metrics rather than the intuitions of faculty. Metrics are indeed revolutionizing the selection, promotion, and retention of academics, and units within universities. Arguably, they already have done so. The recruitment process increasingly looks at various scores and stats about any given candidate for any academic position.

Individual academics can’t do much about it. And increasingly, the metth-1rics will be collected without the academic even doing any data gathering, as data on citations, publications, and teaching ratings get generated in the course of being an academic. Academic metrics are becoming one more mountain of big data ready for computational analysis.

I am too senior (old) to be worried about my own metrics. They are not great, but they are as good as they will ever be. My concern is most often with administrators tending to count everything that can be counted, rather than trying to develop indicators that get to the heart of academic performance. Of course, this is extremely difficult since academics seldom agree on the rating of their colleagues. A scholar who is a superstar to one academic is conceptually dead from another academic’s perspective. So this controversy is one of many factors driving academia towards more indicators or hard evidence of performance. The judgments of scholars vary so dramatically. At least by counting what can be counted, there is some harder evidence that might be indicative of what we try to measure – quality.

So what can we count? It varies by university, but I’ve been in universities that count publications, of course, but every kind of publication, from refereed journal articles to blogs. And each of these might be rated, such as by the status of the journal in which an article appears, or the prestige of the publisher of a book. But that is only the beginning. We count citations, conference papers, talks, committees, awards, and more. Therefore, we perennially worry about whether we published enough in the right places, and did enough of anything that is counted.

In the UK, there has been an effort to measure the impact of an academic’s work. There have been entire conferences and publications devoted to what could be meant by impact and how it could be measured. Arguably, this is a well intentioned move toward measuring something more meaningful. Rather than simply counting the number of publications (output), why not try to gauge the impact (outcomes) of the work? It is just that it is difficult to reliably and validly measure impact, given that the lag between academic work and its impact can be years or decades. Take Daniel Bell’s work on the information society, which had a huge impact, which went well beyond what might have been expected in the immediate aftermath of his publication on The Coming of Post-Industrial Society. Nevertheless, indicators of impact will inevitably be added to all the other growing number of indicators, even thought universities will spend an unbelievable amount of time trying to document this metric. th

In this environment, because I am a senior in academia, I sometimes get asked how a colleague should think about these metrics. Where should they publish? How many articles should they publish? Which publisher should they submit their book for publication? It goes on and on.

I try to give my opinion, but my most general response, when I feel like it will be accepted as advice and not criticism, is to focus on contributing something new to your field. Rather than think about numbers, think about making a contribution to how people think about your field.

This must go beyond the topic of one’s research. It is okay to know what topics or areas an academic works in, but what has he or she brought to that field? Is it a new way for doing research on a topic, a new concept for the area, or a new way of thinking about the topic?

In sum, if an academic’s career was considered, by another academic familiar with their work, could they say that the person had made an original, non-trivial contribution to the study of their field? This is very subjective and difficult to answer, which may be why administrators move to hard indicators. Presumably, if someone has made an important new contribution, their work will be published and cited more than someone who has not. That’s the theory.

However, the focus on contributing new ideas can give academics a more constructive motivation and an aim to guide their work. Rather than feeling that your future is based on getting x number of journal articles published, you make publication a means to a more useful end in itself, furthering progress in your field of study. If you accomplish this, the numbers, reputation, and visibility of your work will take care of themselves. What would be a new contribution to your field? That is exactly the right question.

 

 

 

 

Web Science 2014: CALL FOR PARTICIPATION

The 6th ACM Web Science Conference will be held 23-26 June 2014 on the beautiful campus of Indiana University, Bloomington. Web Science continues to focus on the study of information networks, social communities, organizations, applications, and policies that shape and are shaped by the Web.

The WebSci14 program includes 29 paper presentations, 35 posters with lightening talks, a documentary, and keynotes by Dame Wendy Hall (U. of Southampton), JP Rangaswami (Salesforce.com), Laura DeNardis (American University) and Daniel Tunkelang (LinkedIn). Several workshops will be held in conjunction with the conference on topics such as Altmetrics, computational approaches to social modeling, the complex dynamics of the Web, the Web of scientific knowledge, interdisciplinary coups to calamities, Web Science education, Web observatories, and Cybercrime and Cyberwar. Conference attendees will have an opportunity to enjoy the exhibit Places & Spaces: Mapping Science, meant to inspire cross-disciplinary discussion on how to track and communicate human activity and scientific progress on a global scale. Finally, we will award prizes for the most innovative visualizations of Web data. For this data challenge, we are providing four large datasets that will remain publicly available to Web
scientists.

For more information on the program, registration, and a full schedule please visit http://WebSci14.org and follow us on Twitter (@WebSciConf) or like us on Facebook
(https://www.facebook.com/WebSci14).

Web Science Conference 23-26 June 2014 at Indiana University

I have agreed to co-chair the next Web Science Conference, Web Science 2014, which will be held in 2014 at Indiana University. The lead chairs are Fil Menczer and his group at Indiana University, and Jim Hendler at Rensselaer Polytechnic Institute, and one of the originators of the Semantic Web. The dates are 23-26 June 2014.

My mission is to help bring social scientists and humanities scholars to this conference to ensure that it is truly multi-disciplinary, and also to help encourage a more global set of participants, attracting academics from Europe but also worldwide. IU_H_P2_S1_T1

For those who are not quite sure of the scope and methods of Web Science, let me recommend a chapter in my handbook by Kieron O’Hara and Wendy Hall, entitled ‘Web Science’, pp. 48-68 in Dutton, W. H. (2013) (ed.), The Oxford Handbook of Internet Studies. Oxford: Oxford University Press.The core of the Web Science community sometimes view this as a field or discipline on its own, while I would define it as a topic or focus within a broader, multdisciplinary field of Internet Studies.

In any case, I will be adding to this blog over the coming months as the conference planning progresses, but please consider participating. Information about the conference is posted at: http://websci14.org/#

 

Scholarship in the Networked World, Professor Christine Borgman, 6 June 2013, 5pm at Balliol College

 Scholarship in the Networked World

Oliver Smithies Lecture

 6 June 2013, 5pm

 Lecture Room XXIII, Balliol College

Christine L. Borgman

 Professor & Presidential Chair in Information Studies

 University of California, Los Angeles

 and

 Oliver Smithies Visiting Fellow and Lecturer

 Balliol College, University of Oxford

Scholars are expected to publish the results of their work in journals, books, and other venues. Now they are being asked to publish their data as well, which marks a fundamental transition in scholarly communication. Data are not shiny objects that are easily exchanged. Rather, they are fuzzy and poorly bounded entities. The enthusiasm for “big data” is obscuring the complexity and diversity of data and of data practices across the disciplines. Data flows are uneven – abundant in some areas and sparse in others, easily or rarely shared. Open access and open data are contested concepts that are often conflated. Data are a lens to observe the rapidly changing landscape of scholarly practice. This talk is based on an Oxford-based book project to open up the black box of “data,” peering inside to explore behavior, technology, and policy issues.

Christine L. Borgman is Professor and Presidential Chair in Information Studies at UCLA. Currently (2012-13) she is the Oliver Smithies Visiting Fellow and Lecturer at Balliol College, University of Oxford, where she also is affiliated with the Oxford Internet Institute and the eResearch Centre. Prof. Borgman is the author of more than 200 publications in information studies, computer science, and communication. Her monographs, Scholarship in the Digital Age: Information, Infrastructure, and the Internet (MIT Press, 2007) and From Gutenberg to the Global Information Infrastructure: Access to Information in a Networked World (MIT Press, 2000), each won the Best Information Science Book of the Year award from the American Society for Information Science and Technology. She conducts data practices research with funding from the National Science Foundation, Sloan Foundation, and Microsoft Research. Current collaborations include Monitoring, Modeling, and Memory, The Transformation of Knowledge, Culture, and Practice in Data-Driven Science, and Empowering Long Tail Research.

 

The Library of Congress and The John W. Kluge Center

I spent two stimulating days at the Library of Congress (LC) last week. The first involved meeting with staff of the LC who are involved in thinking through short and long-range plans for the Library’s future, everything from space to holdings that will help the LC take a leading position in the US and globally through its many initiatives and collaborations. All that I could imagine the library doing seemed already on-track in one or more of their many new and existing programs and planned initiatives, leaving me with a positive sense about their direction of change. Staff members are engaging their colleagues in wide-ranging discussions about shaping the LC for the digital age in ways that nevertheless respects what the Librarian defines as the ‘culture of the book’. All the many themes emerging from our Oxford lecture series on ‘innovation and digital scholarship‘ are coming into play in their discussions.

Library of Congress
Library of Congress

 

The second day of meetings shifted to whether and how The John W. Kluge Center of the LC might develop a new program around Digital Studies or Internet Studies – the exact nature of their prospective program has yet to be decided. The Oxford Handbook of Internet Studies (OUP 2013) helped secure me a seat for a wonderful debate over what the Center might do that would be unique and have major global implications, but also help the Library shape its holdings and activities for 21st Century digital research and scholarship. With the support of the Librarian and the ability to bring in scholars on the forefront of the field, such as Professor Manuel Castells, who the was the LC’s Kluge Chair in Technology and Society, during the summer of 2012, and continues as a member of the Library’s Scholars Council. I am quite optimistic about their prospects. The Center is developing an innovative program that will help build the larger field of Internet Studies, as I would define it, as well as support the LC.

 

I left with the impression that the LC does not trumpet its own work as much as we might do in academia, but they are involved in major initiatives at a scale most universities could not match. Academics should be tracking the Kluge Center and the LC over the coming years as its initiatives around the digital age take shape.

 

 

 

Internet Studies: Perspectives on a Rapidly Developing Field

Internet Studies: Perspectives on a rapidly developing field

Charles Ess, William Dutton

doi: 10.1177/1461444812462845

New Media & Society, April 29, 2013

<http://nms.sagepub.com/content/early/2013/04/24/1461444812462845.full>

To quote from the introduction – which is available as a free download – We have organized the contributions to this issue such that they flow across four general areas. The first focuses on the field as a whole, and is filled by our lead article, by Tai-Quan Peng, Lun Zhang, Zhijin Zhong and Jonathan JH Zhu, ‘Mapping the Landscape of Internet Studies: Text mining of social science journal articles 2000–2009’. We then shift focus to specific Perspectives from Different Arenas, beginning with Jingyan (Elaine) Yuan’s ‘culturalist critique of “online community” in new media studies’, followed by Heidi Campbell’s ‘Religion and the Internet as a microcosm for studying trends and implications within Internet Studies’, then an article by Jessie Daniels, ‘Race and racism in Internet Studies’, and Michel van Eeten and Milton Mueller’s ‘Where is the governance in Internet governance?’.

The next set of articles focus more on Methodological Perspectives, beginning with Juliette De Maeyer’s ‘Towards a hyperlinked society: A critical review of link studies’, followed by Niels Brügger’s ‘Web historiography and Internet Studies: Challenges and perspectives’. The two final articles are both tied to Critical Perspectives on User Empowerment, a cross-cutting theme of Internet research across various research arenas. Anja Bechmann and Stine Lomborg’s article is entitled ‘Mapping actor roles in social media: Different perspectives on value creation in theories of user participation’, and this is followed by Christian Fuchs and Nick Dyer-Witheford’s challenge to Internet Studies, entitled ‘Karl Marx @ Internet Studies’.

We conclude with a more general account of what we have learned about this evolving field from this special issue in light of work on our respective handbooks.

Several of the articles are already published online; the print version of the complete issue will appear later this year.

 

We would also like to express our gratitude to numerous reviewers and to Editors, Steve Jones and Nickolas Jankowski, for their constant support and assistance in developing and bringing this special issue to fruition.

 

Charles Ess and Bill Dutton

 

 

Writing a Refereed Journal Article: A Personal Perspective on Strategies for Doctoral Students

Develop a Set of Realistic Expectations

  •  X (4?) articles accepted, in press, or published before completion of DPhil
  • 2 or more in peer reviewed journals or equivalent outlets
  • book chapter(s) are good, more valued with other professional journal articles

(Co-)Authorship issues vary across disciplines, but in Internet Studies:

  •  One or more single authored publications idea
  • Co-authored publications fine, but not only co-authored publication
  • Agree a strategy to manage co-authorship over two or more works (don’t agree to be the last co-author on all publications, unless that is fair
  • Co-authorship is growing more common with team-based research

Present Your Work

  • Present any piece being developed for publication
  • Discover flaws and missing links, ordering problems in the argument and its presentation, in addition to getting feedback
  • Often the source of suggestions of appropriate journals, even invitations to submit to a particular journal
  • Don’t present too many conference papers relative to your publications – suggesting a lack of focus on getting your work published

Be Your Own Toughest Critic on whether Your Idea or Analysis is Publishable

  • Is it an original contribution (empirically (new data set, new operational definitions, original observations or case studies), theoretically, otherwise)?
  • Is it sufficiently important? A relatively simple contribution might merit a blog, or a research note, but not justify the time required for a full journal article.

Prioritize your Time, but be Flexible

  • Focus your attention on the most important original contribution you can make, rather than saving it for future publications
  • Create files, stacks or folders for other ideas, papers, which might rise or diminish in significance over time.
  • Keep your priority, but if you can’t make progress, don’t stop writing. Move to another paper, where you feel able to make progress.

Follow a Simple, Clear Structure Reflecting Basic Research Processes

  • Problem, research question, literature, approach, methods, findings, limitations, discussion of implications and further research
  • Explain what you are going to do. Do it. Tell the reader what you’ve done.
  • Do not write a mystery novel.

Literature Review

  • Essential Element, but don’t Over Kill
  • Are you aware of relevant research?
  • Has related research been published in the journal you are considering?

Carefully Consider the Journal(s) in Which to Choose to Publish

  • Centrality to your work based on Track Record of Published Articles
  • Links to the Academic Community of the Editor, and Editorial Board (Have you read or heard of these scholars?)
  • Do you publish in refereed journals in your field of specialization?
  • Among the fitting journals, it is best to have your article accepted in one with a higher impact factor, and indexed by the right sources.

Write for the Chosen Journal

  • Follow the journal’s style guidelines
  • Keep to guidelines on length, word count
  • Do not submit to another journal while being considered by your chosen journal. This may cause you to think twice about submitting to some journals, such a one noted for slow turnaround of reviews.

Respond to Reviewers

  • Good luck on first review and chosen set of reviewers
  • Most journals will return your manuscript to the initial reviewers, so it is practical to focus on understanding and being responsive to review
  • Explain how you’ve responded to reviews, particularly when reviewers offer contradictory suggestions.
  • Don’t be discouraged by critical reviews, and don’t blame the reviewers, if your writing has not convinced them of the merits.
  • Be attentive to positive reviews: Why did the reviewer like your piece?
  • If unsuccessful, consider an alternative journal, in light of the reviews.

The Importance of Focused Time

  • Not Alcohol, Drugs, or Sleep Deprivation
  • Time on Task in Revision after Revision[1]
  • Consistent Discipline in Reading and Organizing Notes and Research
  • Record your ideas, notes, readings, systematically. Read: C. Wright Mills, ‘On Intellectual Craftsmanship’[2]
  • Focus on the Article, get feedback from colleagues who read or discuss your ideas, and revise, and revise again.

 

 


[1] Take a look at Galbraith’s wonderful essay on Writing Typing and Economics: http://www.theatlantic.com/magazine/archive/1978/03/writing-typing-and-economics/305165/