- Title
- Measuring research impact in Australia's medical research institutes: prioritising productivity within research impact assessment (to optimise the value from investment into health and medical research)
- Creator
- Deeming, Simon
- Relation
- University of Newcastle Research Higher Degree Thesis
- Resource Type
- thesis
- Date
- 2023
- Description
- Research Doctorate - Doctor of Philosphy (PhD)
- Description
- Across Australia and comparable international jurisdictions, suites of policies have been progressively introduced to increase the health, social and economic impacts realised from the investment in health and medical research (HMR). Assessment of the impact generated from this investment, commonly termed research impact assessment (RIA), represents one facet of these policies. The definitions, paradigms, frameworks & methods applied to RIA, have been extensively reviewed in the literature. However, the merits of alternative approaches to assess the impact from research institutes, programs, funders, projects, or individual researchers remains an area of on-going debate. The holistic aim of this thesis is to reassert that productivity represents the primary objective of the original RIA policies, and to examine whether the process of conducting an indicator-based approach to RIA can make a direct contribution to the realisation of this objective. The research methods utilised to address the aims included a scoping literature review and a document review to evaluate the capacity of alternative Research Impact Assessment Frameworks (RIAFs) to realise alternative objectives. To ground the study within real-world HMR, qualitative research was undertaken to examine stakeholders’ perspectives of RIA policy and practice within a research-focussed setting, Australia’s independent medical research institutes (iMRIs). The results from these studies were then synthesised to identify some key principles for an indicator-based RIA that directly contributes to the productivity focussed objectives of the original policy. Finally, these principles were applied to the inception and conduct of two health economic studies to provide preliminary evidence that the productive impacts from research projects could be optimised through this mechanism. The literature review identified eight specified objectives for RIA. A review of existing impact frameworks found that most satisfied the requirements to provide Accountability and Advocacy, providing evidence to justify research investment. In contrast, few frameworks were primarily designed to directly contribute to the productivity goals of the original policy. Indeed much of the current academic debate regarding the merits, issues and theoretical constructs behind alternative approaches to RIA, has lost reference to the goals of the original policy. The qualitative research with iMRIs and key stakeholders found understanding of the need for greater research translation and impact but disparity in their comprehension and implementation of RIAFs. The issues raised regarding RIA were less about methods and choice of indicators, and more about the research activities that the measurement of research translation and impact may or may not incentivise. Significantly, the study identified that RIAFs have the potential to orientate needs-driven research more explicitly towards the realisation of the policy goals. Indicator-based methods utilise logic models, theory of change and similar frameworks to identify metrics or indicators and thereby demonstrate generative causation from research activities to the generation of research outputs, outcomes, and impacts. Indicator-based approaches to RIA are widely implemented but increasingly criticised as theoretically problematic, unfair, and burdensome. The third synthesis study prioritised productivity as the primary purpose for RIA, and conjectured that the incentives generated through an indicator based approach to RIA can contribute to this objective, through application of the following principles: A focus upon researchers and the research process; A unit of analysis that provides control and supports collaboration and accountability; The capacity for prospective implementation of RIA enabling the prospective orientation of research; A line of sight to the ultimate anticipated beneficiaries and benefits; Inclusion of process metrics/indicators that provide for interim targets on the pathway to the final impacts; A logic model that embeds ‘next’ users and generates outcomes from outputs along the pathway; Alignment with a potential incentive mechanism within the existing research system to motivate researchers/research teams to optimise the impacts from their research; Alignment with existing peer-review processes and norms; System thinking; Incremental improvements in the probability of translation from individual research projects should translate to higher impact across the whole funding portfolio. Many of these principles are not novel but this research argues that their integration within an indicator-based RIA process, specifically at the inception of research proposals, will increase the probability that the insights from any individual research project will translate. Application of the principles to two economic studies, a public health study and a clinical evaluation, demonstrated that the process is both pragmatic and meaningful, acting to change the research question, the methods, the resulting insights and, in principle, the probability of translation for both. In conclusion, RIA has been predominantly introduced at a program or institutional level. Indicator-based approaches to this assessment have been widely implemented but are increasingly criticised. This research found that, if productivity is established as the primary objective, then an indicator-based RIA mechanism can assist researchers to optimise the translation potential from any given project, and accordingly, make a direct contribution to the original policy goals.
- Subject
- research on research; research on health and medical research; research policy; research translation; research impact; health and medical research; health economics; health research economics
- Identifier
- http://hdl.handle.net/1959.13/1493471
- Identifier
- uon:53558
- Rights
- Copyright 2023 Simon Deeming
- Language
- eng
- Full Text
- Hits: 198
- Visitors: 244
- Downloads: 65
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | ATTACHMENT01 | Thesis | 6 MB | Adobe Acrobat PDF | View Details Download | ||
| View Details Download | ATTACHMENT02 | Abstract | 552 KB | Adobe Acrobat PDF | View Details Download |