Office for Learning and Teaching Workshop - Melbourne - October 2014
-
Upload
independent -
Category
Documents
-
view
2 -
download
0
Transcript of Office for Learning and Teaching Workshop - Melbourne - October 2014
26/10/2014
1
Articulating Impact in Your OLT Grant Application
Hosted by VicTas PEN and Monash UniversityThursday 23 October 2014
Tilly Hinton, Abound Consulting
1
Why are OLT grants important?• Share with the person next to you one reason that OLT grants are important• You might like to think about importance to one of the following:
• The higher education sector• Students• Australia as a nation• Academics• General staff• The future (of….)• Communities• Employers\industry• Regions• Cities• The government• Universities• Disciplines• Graduates
Session overview
• What is this thing called impact?• Impact in 2015 grant applications• Climbing the impact ladder – with time for work on your application• The template questions – with time for work on your application• Dissemination• Key roles• Sources of support• Questions and reflections
3
WHAT IS THIS THING CALLED IMPACT?
4
26/10/2014
2
Impact = the difference that a project makes in its sphere of influence, both during and after
the funding period
5
Definitions*
• Deliverable/output = a product or resource• Outcome = a change or benefit that the project is designed to bring about, preferably described in a form that is measurable or for which evidence can be provided
• Also look at definitions for dissemination and transfer
* (see full definitions in Innovation and Development Grant Application Instructions p.46) 6
What’s the difference between impact and outcomes?• Changes and benefits during the project = outcome• All changes and benefits, during and after = impact
7
Impact – some recent history2002 DEST report concludes that ‘more could be done to ensure the investment in past grants
programmes produces benefits for the quality of teaching and learning across the sector’
2004 In a report for the AUTC, Hicks concludes ‘It is currently difficult to pass judgment about the outcomes of these projects with respect to the impact. How much dissemination is enough? Over what period? With what impact?’
2005 McKenzie and Southwell reports both concur that for the Carrick Institute/ALTC dissemination should be viewed as something greater than merely spreading the word about a project
2011 D‐Cubed project finds a range of ways that projects had disseminated their projects; finds gaps in terms of reportable changes and benefits
2011 Alison Johns emphasises the need for ‘enhancing an expectation of impact in all programs and ensuring value for money’
2012 Inaugural secondment focusses on research impact (the Influence Factor Project)
2013 Secondment concludes that expectations of impact need to be embedded into routine OLT work, made an explicit expectation for applicants and projects
2014 Application instructions extensively address impact expectations
8
26/10/2014
3
Impact for the OLT
“All Innovation and Development Grants are required to project and plan for positive and substantial impact (changes and benefits) for students, staff, institutions and the higher education sector, as appropriate to the scale and scope of the project. “ (Innovation and Development Grant Application Instructions, p. 12)
9
IMPACT IN 2015 INNOVATION ANDDEVELOPMENT GRANT APPLICATIONS
10
Quick references for new applications*• The grant is provided for the purpose of achieving the deliverables, outcomes and impact of the project for which it is approved (p. 42)
• Incorporate an Impact Plan and strong dissemination strategy (p. 12 and Appendix 6)
• Four assessment criteria – Project Impact, Project Approach, Project Budget and Project Evaluation – explicitly address impact (pp. 12 and 13)
• The aim expressed in the online form focusses on impact (p. 22)
• Evaluation needs to contribute to establishing the impact of the project, and its value to other higher education institutions (p. 13)
• The reference group needs to give constructive advice on conduct and direction of the project, and to ensure the project has maximum impact (p. 40)
• Letters of support need to reference impact (p. 35)
• Key element in progress and final reports (p. 38)
• Remember: Your plan is an indication, not a promise. (see Appendix 6)
* Page numbers refer to the Innovation and Development Grant Application Instructions for 2015 11
Application components (p. 28)EOI Full Proposal
Online form Yes YesApplication Maximum five pages Maximum 20 pages, and responding to all
criteria as per section 4Project Impact plan n/a, description of impact
insteadYes, within the 20‐page limit
Budget and budget justification n/a Yes, within the 20‐page limit, and using the budget template (refer Appendix 8)
Timeline n/a, indication of expected timeframe
Yes, within the 20‐page limit
Brief bios and experience of team members including involvement in OLT/ALTC projects
n/a Yes, maximum one page per team member as appendix to the application
Design specs, if relevant n/a If relevant, maximum one page, as appendix to the application
Letter of support from DVC (A) or equivalent at lead and partner institutions
n/a Maximum two pages per letter, as appendices included in application
26/10/2014
4
Appendix Six Questions1. What indicators exist that there is a climate of readiness for change in relation to your intended project? 2. Completion of the matrix3. What are your strategies for engaging with stakeholders throughout the project? 4. How will you enable transfer, that is ensuring that your project remains impactful after the funding period? 5. What barriers may exist to achieving change in your project? 6. How will you keep track of the project’s impact? What analytics may be useful? 7. How will you maintain relevant project materials for others to access after the project is completed? 13
Appendix Six MatrixAnticipated changes at:
NB: these are approximations & text is not expected in every cellProject completion Six months post‐completion Twelve months post‐
completionTwenty‐four months post‐
completion1234567
14
The Impact Plan• Impact Plan = matrix + responses to the questions in Appendix 6• Responses can be in any sequence, so long as there are clear headings• The plan is incorporated into your application• The matrix doesn’t need every box filled in, and remember what you write is likely to change throughout the project life
• Think about mechanisms to achieve the anticipated impacts – these should appear throughout your application
15
Your project approach• A strong application identifies the impacts and then proposes appropriate mechanisms to achieve them.
16
26/10/2014
5
For example: A transition program• Overall goal: increased rates of successful transition from school to university for Aboriginal and Torres Strait Islander students
• Anticipated impact: an evidence‐based transition program embedded at several schools and universities
• Mechanisms: high levels of engagement with education departments and with individual schools
• A set of brochures and aspirational videos is unlikely to be enough.
17
For example: A new kind of PhD• Overall goal: to address the concerns that Australian PhD programs may be too narrow
• Anticipated impact: a broader, ‘scholar‐citizen’ PhD program open for enrolment or a co‐curricular PhD supplement program
• Mechanisms: in‐principle support from national organisations (such as DDOGs) and from likely adopting institutions
• Undirected recommendations for reform, with no implementation approach is unlikely to be enough.
18
For example: Curriculum reform• Overall goal: prepare entry level health professionals to respond more confidently and competently to complex mental health needs
• Anticipated impact: a capstone assessment requirement for all students in accredited degree programs in the discipline
• Mechanisms: liaison/collaboration with industry peak bodies, accreditation agencies and/or discipline bodies
• A set of standards on a website is unlikely to be enough.
19
CLIMBING THE IMPACT LADDER
20
26/10/2014
6
Snapshot findings from a sample of completed ALTC/OLT projectsPrevalent:• Impacts for the project team members themselves (IMPEL Rung 1)• Distributing information about projects (IMPEL Rung 3)
Elusive:• Instances or evidence of changes in academic practice (IMPEL Rungs 4 – 7): ‘[the project] probably [has had] a large but invisible influence’
• Student impacts (IMPEL Rungs 2 and 4‐7): ‘…it's almost impossible to say. I'd like to say it […] had an effect but it's a drop in the pond of many things having an effect’
Timing:• Changes usually take considerable time to emerge
21
Scope of the IMPEL model• Model incorporates research findings, stakeholder perceptions and funding body culture/imperatives
• Understanding of impacts was built primarily from self‐reporting by project leaders, who didn’t anticipate this in advance
• Model concerned with funded learning and teaching development programs, specifically for the Australian Government
22
Your project…Identify a small/narrow impact your project could create.
Identify a large/broad impact your project could create.
23 24
26/10/2014
7
How the model works• Moves from narrow impact (ie. on one’s own practice or profile) to broad impact (ie. on all relevant staff and students)
• Not all projects will anticipate nor achieve impacts at every IMPEL rung• Anticipated impacts are mapped against rungs of the IMPEL model, and the timing point/s they are expected to occur (completion, 6, 12 and 24 months)
25
The changes the project brings about for members of the project team.
For example:• Recognition of project participants’ contributions to learning and teaching through promotion and awards
26
The changes for students that project team members make directly (in teaching/support/etc).
For example:• Project team members’ students report benefits of being more networked through an online tool
• Changes to curriculum by project team members equip students to feelmore work‐ready
27
The lasting contributions to knowledge that the project makes, which may lead to serendipitous adoption or adaptation beyond the project’s intended reach.
For example:• Special issues of journals that further explore themes of the grant
• Publications from a grant cited fifty‐eight times in other publications
• An updated online resource to support robust curriculum planning for teaching of Australian literature
• Materials used to promote teaching quality to international prospective students and partner universities
• Cascading influence through engagement with the early childhood sector and relevant government departments
• Receipt of national and international awards recognising outstanding practice along with sustained high rates of downloads, views and linking 28
26/10/2014
8
Changes for students that happen because opportunistic adopters within participating institutions adopt or adapt ideas from the project.
For example:• Education students supported to develop resilience, a factor in retaining staff in the teaching profession
29
Changes for students that happen because of systemic adoption or adaptation of project ideas at one or more participating institutions.
For example:• Online graduate attributes system deeply embedded across an entire school
• Almost three quarters of a million dollars of industry and philanthropic funds secured for the lead institution to further the work started by the grant
• University‐wide academic leadership program with ongoing funded based on the success of the grant project
30
Changes that happen because opportunistic adopters beyond participating institutions adopt or adapt ideas, leading to changes for their students.
For example:• Professional development delivered to an estimated 20% of science academics in Australia
• An estimated 2,500 students benefited from active learning materials in undergraduate science programs
31
Changes for students that happen because of systemic adoption or adaptation of project ideas at a broad level, beyond the participating institutions.
For example:• Report cited as key evidence for a wage increase across the profession
• Changes to professional accreditation requirements ensuring students are better equipped for study and employment
32
26/10/2014
9
Map your small and large impact on the ladder
• Which rungs?• Reflections for project design?• Surprises or confirmations?
Back to one of those earlier examples…• Transition program
• Rung 1: strengthened cross‐cultural awareness and agency amongst team members
• Rung 2: team members in direct teaching, liaison or support roles will build project findings into their work supporting student transition
• Rung 3: community visits, publications, papers at key conferences
• Rungs 4 and 6: Aspects of the project may inspire change at participating and other institutions, communities and schools
• Rung 5: Some or all participating institutions will have the transition program embedded into routine practice
• Rung 7: Not within the scope of this project34
35
• team’s academic knowledge
• educational practice
• legitimacy and profile
• career progression
• research and project management expertise
• curriculum• resources• support• classroom
activities• learning
environment
• publications• citations by
others• online
materials• repositories
• individual academics or teaching teams
• Individual professional staff or teams
• people shifting their thinking and therefore their practice
• peak body change
• policy change• management
change• budget, funding,
workload allocations
• incentive schemes
• accreditation requirements
• wide‐scale change
Prompts for the seven IMPEL rungs
Drafting your Impact Plan (A3 sheets)
• Focus on the matrix side…we’ll get to the questions next!
26/10/2014
10
THE TEMPLATE QUESTIONS
37
1. What indicators exist that there is a climate of readiness for change in relation to your intended project?
• People and systems need to be ready to change, or at least able to be convinced that change is worthwhile.
• Identify project beneficiaries (end‐users), those likely to adopt or adapt your work (targeted potential adopters), and change enablers
• Describe the shared interest/concern in the issues being addressed and gauge receptiveness
• Assess resources, momentum and willingness for change amongst your targeted potential adopters
• Project rationale (filling a gap + more)• If climate isn’t ready, what role can your project play?
3. What are your strategies for engaging with stakeholders throughout the project?
• Effective dissemination can’t be an afterthought, and it needs to occur throughout the project.
• Timing and nature of connections with targeted potential adopters• Dealing with unanticipated opportunities• Plans for cultivating a climate of readiness for change, and building
credibility, familiarity, empowerment and ownership• Identify most useful dissemination activities
4. How will you enable transfer, that is ensuring that your project remains impactful after the funding period?
• The impact of the project is expected to continue after the money runs out.
• Plans to make outputs findable and adaptable• Identify those who might step in with financial or other support to
keep the work going• Embedding opportunities that can start or finalise during the
project• Team member involvement post‐completion
26/10/2014
11
5. What barriers may exist to achieving change in your project?
• A strong application will show that potential obstacles have been identified and addressed.
• Barriers could include:• People• Operating environment• Expense• Complexity• Rules or norms• Resources• New information• Priority shifts
6. How will you keep track of the project’s impact? What analytics may be useful?
• Being able to trace changes and benefits provides you with rich information about the project impact, and can be useful for other purposes also.
• Indicators for each anticipated impact (think passive and active)• Ways of knowing that change has occurred• Call on librarians and learning analytics researchers• Identify audiences and channels for your impact stories
7. How will you maintain relevant project materials for others to access after the project is completed?
• Others are likely to want to know about your work in the future.• Identify what could be of use to other project teams or applicants ,
such as project management documentation, data sets, bibliographic materials or instruments
• Decide if your own project website is likely to have longevity, or if it would it be better nested within an established organisation?
• Think about data repositories, librarians, archivists, your research office, social media, your website and search engine optimisation
Drafting your Impact Plan (A3 sheets)
• Focus on the questions • Remember you can address these in any order in your application.
26/10/2014
12
DISSEMINATION
45
• Dissemination is the label given to the strategic things we do to make impact happen
• “the planned process of understanding potential adopters and engaging with them throughout the life of the project, to facilitate commitment to sustained change” (Hinton, Gannaway, Berry and Moore, 2011, p. 6)
• Sustained change – impact – is the end game. Dissemination is the means of getting there.
Dissemination Activities• Branding
• Conferences
• Email lists, discussion forums, and other social networking tools
• Funding sub‐projects at other institutions, mentoring, and participatory dissemination
• Guides and teaching materials
• Influencing policy
• Journal articles and book chapters
• Media releases
• Meetings, discussions, roundtables and invited presentations
• Networks and communities of Practice
• Newsletters
• Project conferences, workshops, showcases and forums
• Project final report
• Webpages, online repositories, audio‐visual material and other online content
(Hinton, Gannaway, Berry, & Moore, 2011, pp. 22‐29)
Plus:
• Embedding into existing structures
• Seeking industry funding
• Paying ongoing attention
• Gathering information about usage
• Cultivating peer acceptance and recognition
• Passing it on
• Lobbying and campaigning
• Harnessing organic uptake/readiness
• Media engagement
• Consulting
• Governance
• Strategic planning
• Incentives
• Awards
• Funding for adopters
• ... and more ….
26/10/2014
13
(A few of the many) Resources Available• http://olt.gov.au/dissemination
• Gravestock, P. (2002). Making an impact through dissemination. In C. Baume, P. Martin & M. Yorke (Eds.), Managing educational development projects: Effective management for maximum impact (pp. 109‐124). London: Routledge.
• http://www.melaniebarwick.com/training.php
• http://www.mathcentre.ac.uk/resources/uploaded/getting‐started‐in‐stem‐pedagogic‐research.pdf
KEY ROLES
50
Key roles in achieving impact
• Your project is likely to be of interest to a range of people. • To shift from interest to action – from idea to impact – it can be useful to think in depth about who your key players are.
• These understandings are powerful in informing project design.• Targeted potential adopters: People with whom the project seeks to engage and to whom project deliverables are transferred for the purpose of change.
• Change enablers: People who can increase the likelihood of embedding, upscaling and sustainability of project outcomes.
• End‐users: People for whom the intervention is designed, such as students.
Selling your grant
“…one‐line ‘grabs’ intentionally highlighted the benefit of simulation to the various segments of groups or individuals who were decision makers, in some respect, in relation to the uptake of simulation. Each ‘grab’ was developed for the distinct characteristics of the market segment, with their specific agendas and priorities in mind, taking into account whether their attitudinal stance was positive, neutral or negative (if known). The aim was to devise initial statements that were easily grasped, resonated and persuasive, in turn, influencing an ongoing process of dialogue.” ‐ OLT Fellow Prof Cobie Rudd, Fellowship Final Report, p. 11
26/10/2014
14
Extracted from Rudd, C. J. (2013). Enhancing the uptake of learning through simulation in health. Sydney, Australia: Office for Learning and Teaching, DIICCSRTE.
Back to your project…
• Identify 1 or 2 key players (“market segments”) in your project• Draft a “grab” for each one
SOURCES OF SUPPORT
55
• Web page ‐ www.olt.gov.au/impact• Application instructions ‐ http://www.olt.gov.au/grants‐and‐projects/innovation‐and‐development
• Your Institutional Contact Officer• The OLT• Within your project team• Your prospective external evaluator• Your prospective Reference Group members• D‐Cubed Materials ‐ http://www.olt.gov.au/dissemination• This workshop repeated as a webinar on Monday 27 October
56
26/10/2014
15
Your closing reflections
• Identify something you’ll incorporate into your application writing process about impact
and questions…
57
Tilly Hintonhttps://independent.academia.edu/HintonTilly
58