Measuring what matters: Towards consequential research
To move beyond ‘publish or perish’, universities worldwide must promote contributions that deliver tangible societal benefits
THESE were questions put to me during the recent Nathan Lecture on research impact: Are researchers in a university context interested in making a difference to society? And if they were, how would that impact be assessed and would they be recognised for it?
At Singapore Management University (SMU), we have been having these conversations for much of the last year, and indeed, we have recently broadened the dialogue. When you start talking to the SMU faculty, many of them have a sense of what they could do with their research that could potentially make a difference in society.
However, the global world of academia is structured in such a way that it does not have a measurement framework for this sort of impact. When it is not measured, people tend not to respond to it. If you want your credentials to be portable – say you want to work in the US, New Zealand or somewhere else – the academic world generally uses technical academic metrics.
There is not a widely accepted alternative framework yet, despite initiatives such as the San Francisco Declaration of Research Assessment, the Leiden Manifesto, the European Science Framework, the UK’s Research Excellence Framework (REF), Hong Kong’s Research Assessment Exercise, and Australia’s Engagement and Impact Framework.
These efforts represent initial attempts to acknowledge and valorise other kinds of impact and to figure out how to measure it. As with any measurement, it will take time as we learn the merits and drawbacks before settling on something. The recent Financial Times rankings of business schools is the first I know of that looks not only at academic metrics (where you have published, impact factors, etc.) but also at whether your work is taken up in government documents, patents and so forth. Those are more concrete metrics that people feel comfortable with.
The UK’s REF uses case studies, which requires someone to write up the impact of their work qualitatively. However, qualitative approaches are often dismissed as subjective. We all gravitate towards measurement that is deemed objective and fair. The answer will likely be somewhere in between – some combination of qualitative and quantitative metrics. We do not yet know which measurement framework we will land on, but these various efforts point the way.
BT in your inbox
Start and end each day with the latest news stories and analyses delivered straight to your inbox.
At SMU, we have for some time now been developing something that recognises what is happening in the world and adopts much of that thinking, but also has some elements tailored to the Singapore context.
To ensure our academics’ achievements are recognised globally, our standards cannot deviate too far from international norms. Otherwise, we create a situation where a professor’s local credentials are not valued elsewhere.
A global shift in assessment is needed, and it will likely be driven by funding agencies. The change will happen when these agencies signal new priorities – effectively saying: “This is what we are now looking for” – as researchers will inevitably adapt their work to secure grants.
The writer is the president of Singapore Management University and the Institute of Policy Studies’ 15th S R Nathan Fellow for the Study of Singapore. This is an edited excerpt from the new book Universities Reinvented: Shaping Legacy and Impact for a New World, which collects her three IPS-Nathan Lectures and highlights of dialogues with the audience.
Share with us your feedback on BT's products and services
