Tag: engagement

Categories
Universities

Why most university impact studies are flawed

A version of this article also appears in ACEEU Spotlight magazine 📖

I enjoyed speaking on a lively panel yesterday about regional development and innovation as part of the UIIN conference, relocated successfully from Budapest to Zoom. Together with Matthew Guest from GuildHE we discussed how to better understand the local role of small and specialist providers.

The work builds on experimental alternatives to traditional economic impact studies. I first explored the idea of institutional heatmaps on a post here in 2018, and then expanded on this at a workshop in South Africa later that year. Over the past 12 months I have been working with GuildHE to ‘map’ the impact of some of their members. In yesterday’s presentation I set out why I think the traditional ‘big number’ approach to measuring economic impact is out of step with what places need from their universities. Below I go further and list why I feel these studies are, mostly, flawed endeavours. (I should add that these are my personal views, not those of GuildHE!).

You don’t have to look far to see economic impact studies. My former employer had a flagship biennial report with a steadily-increasing figure for the impact of UK universities – £21.5 billion to UK gross domestic product at last count – which it has used successfully for lobbying and campaigning. As long as this figure keeps increasing, everybody is happy. Many institutions have their own studies – £650 million of impact here, £400 million impact there – and often with LEP-level or regional disaggregation. Of course, such studies are not limited to higher education. We’re informed that shooting contributes £2 billion to the UK economy and supports the equivalent of 74,000 full-time jobs. Ornamental horticulture and landscaping contributed £24.2 billion to national GDP in 2017.

Why we need change

There are helpful academic papers which deconstruct the methodologies for calculating economic impact, and the common pitfalls. Instead, I want to challenge the preoccupation we seem to have with ‘one big number’ impact studies and what we lose in the process.

There are two shifts taking place which render the traditional impact study less effective:

  1. A single large number fails to capture what is increasingly important. The shift towards universities being ‘for’ a place, rather than simply ‘in’ or ‘from’ a place, means this data needs to be far more nuanced. We need to know specifically who is benefitting, and how, and who is missed out. We need to know the businesses and the communities behind these numbers. As disillusionment grows with traditional methods of measuring economic success – GDP, GVA – and attention on ‘inclusive’ and social development begins to be translated into policy change, economic impact analysis needs to keep up.
    Traditional impact studies simply don’t do justice to the range of university activities. They measure spending, output and employment, but do not capture the full impact of engaging with communities in a marginalised neighbourhood, or working with small businesses to strengthen their supply chains, for example – activities that may have huge impact but make little difference to a £400 million impact figure. (Accounting for social value can help here).
  2. As we grapple with recovery from Covid-19, it is both tone-deaf and ineffective for universities to be shouting about how good they are, whilst also asking for assistance from government. Rather than communicating about the size of their value-added, university messaging needs to focus on solutions and partnerships. Policymakers need a more sophisticated understanding of impact which moves beyond broad figures to specific information on which communities, businesses and industries have benefited from the university, and who stands to benefit from future support.

What else is wrong with traditional impact studies?

I should note that economic impact studies are not all bad. It is helpful to see returns on investment, and to raise awareness that universities have economic clout and should be seen alongside other major industries. But they risk being a blunt instrument, obscuring what is often highly patchy and inconsistent local impact behind impressively large numbers. Economic impact studies need to be married to a rich understanding of local impact – perhaps through something like an institutional heat map combined with a survey of perceptions or social impact assessments.

Four further shortcomings that come to mind:

  • Uniformity. Despite huge variation in local contexts across the UK, and the individual histories and missions of universities, impact studies all end up looking pretty much the same. As with my engagement strategies test, if you line up five university impact studies and remove the university name, can you tell who (or where) they are talking about? The uniformity of approach, and measuring success against numerical benchmarks, means we lose out on what may be needed. By working towards what is measured and counted, impact ends up converging into a standardised set of headline numbers and we lose the local context.
  • Impact. Slightly tongue-in-cheek, I would like to see an impact study of impact studies. Do they lead to positive change? Or boost perceptions of universities? Quite possibly. But next time you are in a taxi to a university, ask the driver about the impact of the university. You’re unlikely to be quoted an economic impact figure of £450 million a year to the LEP’s economy. You’ll probably be told about the business that decided to open a new site near the university, or the impact of students volunteering with communities (and how the university is good business for the taxi company – at least before lockdown). You might argue that economic impact analysis is aimed instead at funders and policymakers. But should it not also reach residents and businesses?
  • Fatigue. Somewhat cynically, does anyone really care whether the economic impact is £600 or £900 million? Beyond a certain point, big number fatigue sets in. Figures between institutions are not always directly comparable, and the process of reaching the figures is not always transparent (or easily replicable).
  • Unintended consequences. We are not at this point, but I can imagine a league table of economic impact rankings. Universities should be well aware of the limitations of league tables, and the uncanny ability of rankings to shape and warp policies away from what is important – both for the institution and for the place.

Above all, my concern is that economic impact analysis can mask inequalities and ‘cold spots’ in university engagement. Of course, heatmapping as an experimental alternative brings its own set of issues. Consistency between institutions, subjective judgements over the importance and intensity of shading, and the complexity of trying to map such a wide range of activity are issues that need to be resolved. But they may also expose quite starkly where a university is not working, and not having an impact – things that are hidden in the ‘one big number’ approach.

(Image credits: original images from Unsplash here and here.)

Categories
Universities

What central banks and universities have in common

This post originally appeared on the Yorkshire Universities website. I am delighted to have recently joined Yorkshire Universities as an Associate.

Last month Andy Haldane, Chief Economist at the Bank of England, gave a speech at the University of Sheffield asking if all economics, like politics, is ultimately local. The speech attracted attention for its discussion of whether we can capture and model detailed data on the economy at a far more local level. But there are two other points in the speech worth exploring further.

The first is recognition that higher education, alongside financial services and the creative industries, are sectors that ‘exhibit the highest economic complexity and thus potentially generate the highest value-added’. Economic complexity means the amount of embedded knowledge. Translated into places, a high level of economic complexity means a diverse set of highly-specialised industries, and tends to result in a more prosperous place.

Secondly, Haldane discusses the Bank’s response to the ‘deficit in public understanding and the deficit in public trust’ that central banks are facing. The Bank has responded by rolling out citizens’ panels across the UK, with independent chairs from the local area. Some of this discussion mirrors the public discourse around the role of universities in society, with the flurry of institutions signing Civic University Agreements and reasserting their public missions a reaction to this. The Bank has had a network of 15 regional branches since 1825. Many universities have a long history of civic engagement. There has, however, been a clear need for both to demonstrate this more clearly to those who stand to gain the most in areas which have traditionally been served the least.

Taken together, these two points nicely capture two key roles of universities: generators of knowledge and the economic benefit that can result, and shapers of place and society. I make this point in a recent report for the British Council, noting that these two roles are significant because they challenge different ends of the traditional university mission: research and the so-called ‘third mission’ of economic and social engagement. The report looked at universities and the development of ‘smart’ cities across Europe, concluding that work between universities and city hall often draws on both of these missions, which prove to overlap and reinforce each other. The urgent calls for universities to ‘do more’ for their place, the challenges that local areas are facing, the strengths that universities have, and the work that they are doing, mean there has never been a better time for universities to build stronger links with their LEP, Combined Authority, and Metro Mayor partners.

When partnerships are missing

A couple of days after my report on universities and smart cities was launched, a smart city project in Toronto hit the headlines for all the wrong reasons. Plans for a Google-affiliated company to redevelop land near the waterfront met with opposition from citizen groups concerned about long-term motives and a lack of transparency. The Canadian Civil Liberties Association is suing three levels of government over its plans to build the smart neighbourhood.

So what went wrong? Clearly a lack of effective citizen consultation – a basic prerequisite for any smart city initiative – is part of the problem. I would also expect any successful project to have at least some involvement of universities. Given the complexity of any widespread urban development scheme, bringing in universities would have been a sensible move (but not a panacea), especially given the excellent work of Toronto’s universities in relation to the development of the city that I’ve explored elsewhere.

Proactive universities

If we take all of this together – the value of higher education to local economies, the need to build trust with people who live in these places, and ambitious regeneration projects that go wrong – we are reminded of the unique position of universities as powerful institutions that can connect communities, decision-makers and the private sector. In this connecting position, with an obligation to support all three but beholden to none, universities have difficult decisions to make. Given limited time and resources, these can require tricky trade-offs.

Often universities contribute to activity in local areas where they are not necessarily obliged to act, but in doing so can add great value. One such area is in place promotion and the attraction of foreign direct investment (FDI). Such activity can boost the prospects of communities and businesses (and increase the economic complexity of a place). It is also essential if we are to meet the government’s target to increase innovation and R&D investment to 2.4 percent of GDP. At Yorkshire Universities we are exploring the role that universities are playing, and the further contribution they can make, to increase trade and FDI in Yorkshire. In doing so, we are reminded again of the crucial dual role universities play, as generators of knowledge and shapers of place.

Photo via Unsplash

Categories
Universities

Entrepreneurship in UK education: the local dimension

The National Centre for Entrepreneurship in Education (NCEE) launched the Inspiring Entrepreneurship in Education report last week at the House of Lords, capturing the views of 62 Heads of Enterprise from UK universities (I provided the analysis).

The report was covered by Times Higher Education (paywall article), who led with the finding that university support for enterprise in local schools and communities has significantly decreased in the past six years. The full report here (PDF) covers a wide range of activity and indicators, perhaps best summed up by this visual heat map from the annex. The first column is 2018 activity and the second column 2012 data, and greener is better:

The article ends by summing up the report conclusions focusing on local activity:

The report also recommends that universities consider how existing activity and the work of students in particular could engage with schools and communities, and that staff promote the work of local entrepreneurs, in preference to tales of high-profile examples such as Mark Zuckerberg and Richard Branson, to make entrepreneurship seem more accessible to students.

The final recommendation in particular was also echoed by Professor Alison Wolf, Baroness of Dulwich, who also spoke at the event.

Categories
Universities

What universities can learn from excellent libraries about public space

Most people interested in the role of universities in society would, I think, agree on the following:

  1. Society as a whole benefits from universities and the contribution they make to research, education and local development.
  2. Universities benefit when they work closely with diverse groups of people: communities, businesses, international visitors.
  3. Therefore it makes sense for universities to be as ‘open’ as possible and to exist as spaces and places that make (1) and (2) possible.

Why, then, are many universities pretty poor at putting (3) into practice? There are some that do this well: in the UK, a university in the north east that chose to build a five-a-side football pitch on a piece of prime campus real estate, rather than financially lucrative labs or incubator space, to encourage local youth to play and become comfortable on campus. Or the university shaping its campus to provide a new short-cut from the train station to the city centre, and showing off the work of students and the facilities of the university for all those walking through. Internationally, I’ve written previously about Johns Hopkins in Baltimore, and Ryerson in Toronto.

But these are exceptions. Many university buildings – at least in the UK – are woefully under-utilised, and security concerns tend to trump openness and access. Tuition fees act as a wall, restricting access to university buildings to those who pay the fees. In contrast you can walk straight into and work within many of the excellent libraries in universities across Europe. Universities should be places for people to meet and ideas to flow, and they need to be open and welcoming environments. Fortunately there are models of excellence to learn from.

Helsinki’s Library Oodi


Last week I visited Helsinki’s new public library – a glistening 98m euro building near the Central Railway Station and spread over three floors. There were several thousand people within the building when I visited, but it didn’t feel crowded. The library’s website tells the story better than I could here, but there are some key points universities could learn from:

  • The library markets itself as a ‘living meeting place’ that is ‘open for all’, functioning as ‘a living room for residents’. It is truly open – no need to sign up, no turnstiles. As the Johns Hopkins example above showed, when you strip down the perimeter or facade you invite foot traffic and create a safer environment, but you also truly engage (signing up for a free membership may seem inclusive, but there are always parts of society who will be excluded by this).
  • As such, Oodi reimagines what a library should be for. There are still 100,000 books on the top floor, but the space has been designed from scratch to reflect today’s society. The second floor has an ‘urban workshop’ containing 3D printers, sewing machines, large format printers and computing equipment. There are also meeting and conference rooms, online gaming rooms – all available to the public. Anyone can pick a tablet up from the shelf. There’s a vast amount of comfortable seating – some in quiet zones, others resembling a co-working space. Freelancers, businesses, tourists, residents, students, families are all catered for. What would a university building that did this look like?
  • There’s a consultation area, where residents can look at models of city plans, view new developments with VR headsets, and provide feedback notes on a giant map of the city. And the building itself was designed with an emphasis on service design and user input.
  • The basics are all covered. There’s very fast internet (no login), long opening hours, a good restaurant and multiple coffee shops, and full accessibility. It’s big enough to never feel too busy.

 

One might argue that all this comes at a huge cost – and why give all this away for free (or why should universities provide such services, when state-run institutions such as libraries should do so). But there’s a real competitive advantage to be found from being the provider of such a space – the ideas that can be generated, connections made, and the long-term engagement with community that could help break down some of the barriers in society. It’s an investment in a place and people – a gift that pays interest in stronger relationships into the future.

There’s also a high degree of trust involved in completely opening doors to all (which I think the UK struggles with more than some of our European counterparts). What if crime skyrockets, or people decide they don’t want to pay for study when they can use facilities for free? Risks indeed, but my guess is the more people that flow through from all parts of society, the lower the crime. And the more people who see what such an institution can offer, the more they are willing to support it.

(Photos: Tuomas Uusheimo)

Categories
Universities

How to write an engagement strategy

This post first appeared on Wonkhe.

Writing an effective strategy for local engagement is difficult.

Read through the following statements, taken from a UK university’s strategic plan, and see if anything looks familiar:

As a vibrant knowledge hub, we have an important role to play both locally and globally. The university is a large employer and economic contributor in the region.

We aim to build a clear and distinctive reputation for excellence through strategic engagement and communication with our regional, national and international communities.

We will help the region address its challenges and opportunities while incorporating its many possibilities in education and research.

Could this be from your university’s strategic plan or external engagement strategy? I actually lied at the start: these statements aren’t from a single university. They’re a jumbled mix of statements from five universities from across the UK nations, and from various mission groups. But if they look familiar this could be a problem.

A Barnum effect?

Speaking at recent events, I’ve asked the audience – mostly of higher education professionals – whether these statements about engagement could be from their strategic plan. Nearly everyone raises their hand.

This reminds me, perhaps a little uncharitably, of the Barnum effect. Emerging from a series of psychological experiments in the 1940s, the effect involves showing participants – for example, a class of students – an individual personality statement based on a handwriting sample or a piece of written work. Statements include:

  • You have a great need for other people to like and admire you.
  • You have a tendency to be critical of yourself.
  • You have a great deal of unused capacity which you have not turned to your advantage.

Because the assessment statements are so vague, people interpret their own meaning, and the statements become personal to them. Nearly all participants believed their assessment to be uncannily accurate.

Engagement expectations

There is a growing expectation for universities to be regionally engaged, and this follows from a recognition of the role universities can play in their area and with local communities. The interest and enthusiasm around the Civic University Commission is evidence of this.

This local role isn’t new. Many universities have had engagement as a core part of their mission since their founding. But in recent years government has focused on the role of cities and regions in devolved policymaking, as well as the institutions, such as universities, that can help steer this.

As regional plans – from City Deals to Local Industrial Strategies – shape the policy environment university planners need to factor the issue of place more highly. Often, however, the result is strangely place-neutral: a series of engagement strategies and university planning documents that are largely indistinguishable from one another.

Place, positioning and partnerships

There are positives to be drawn from my (admittedly unscientific) scan of university engagement strategies. Many recognise how the local and international activities of universities reinforce each other. Universities are bridges for their towns, cities and regions to reach the world, and this offer to connect the local and the global is made loud and clear.

In my work for the British Council on universities and cities working together on internationalisation, I found that effective planning was built on a deep understanding of place, positioning and partnerships. It is long-term, deliberate and part of a wider vision of the future of the local area. Universities often have a clear sense of their positioning, but articulating this in the context of place and partnerships can be tricky.

University strategies are not the place to detail individual activities and actions. But a greater degree of specificity is often needed, and this is likely to mean dropping some of the broad statements to focus on a few areas of institutional strength, perhaps joining up with other local universities.

Rooting strategies in place, positioning and partnerships is a good start. The Civic University Commission’s progress report notes that measuring civic engagement is not widespread (but does highlight Cardiff University’s use of quantitative and qualitative analysis). Further work on this is also important.

A good incentive to write a long-term strategy (image credit)https://unsplash.com/photos/cKjxGyfNdQc
A good incentive to write a long-term strategy (image credit)

So too is taking a longer-term perspective. Many universities proudly promote their anchor presence in their area, but their plans are often on five or ten-year cycles. The city of Amsterdam has a 200-year strategy. Although this is prompted by climate change (as an official said, “we don’t want to just let the water flow and all have to move to Germany”), thinking about what a university strategy spanning one or two centuries may look like is perhaps a useful exercise.

A test when looking at your university engagement strategy: if you remove your institution’s name, and mix it up in a pot with other anonymised strategies, is it clear which strategy is yours?

Main image credit

Categories
Universities

The six stages of engagement

Over the past couple of months I’ve given several presentations on university engagement in cities (slides below from a workshop in South Africa). One of the things I like to test with the audience is a sliding scale I’ve developed1 of why universities choose to engage with local communities, businesses and other organisations:

1. Financial incentives
2. Government push through policy (e.g. for knowledge transfer)
3. Branding; demonstrating social relevance
4. Enlightened self interest
5. Public/local/civic duty; (rediscovering) a historic mission
6. A strategic ‘urban turn’

The theory is that, over time, many universities have moved from point one – choosing to engage locally because of perceived or actual financial benefits – to point five – a sense that engagement is part of the duty of the institution. The route is rarely a straightforward journey through all five, and in the messy reality of day-to-day engagement many stages will look remarkably similar.

Between financial incentives and civic duty we have universities responding to government policy pushes (point two), described well by the likes of Rhiannon Pugh and colleagues regarding initiatives such as Growth Hubs in the UK.

Point three – engagement as a means to improving an institution’s branding and demonstrating social relevance – is often a scrambled response to universities coming under fire (usually from the press) for being societally irrelevant or out of touch.

Enlightened self interest, point four, is the recognition that the fortunes of a university are often closely intertwined with the health of its locality.

Point five is having civic duty at the heart of the university mission. This is garnering a lot of attention in the UK through the likes of the UPP Civic University Commission but, for many universities, is aspirational at present.

Point six I’ve written about extensively on this blog. It combines points one to five, and proposes that the relationship between universities and cities is evolving. For some universities, the city has become a greater strategic concern and opportunity. There is evidence of universities slowly undertaking an ‘inward’ or local turn, from nation to city – for example university leaders prioritising city trade delegations over national ones – and institutions looking to take advantage of the globalisation of urbanisation and responding to the narrative over the ‘rise of cities’ and their interconnectedness. At the same time, universities themselves become vehicles for cities to achieve their goals – but I’ll save this for another post.

Locating the Urban University slides

Finding Dan | Dan Grinwis

  1. With many influences, who shouldn’t be held accountable for my findings. Much of this thinking was prompted by an article by van Schalkwyk & George de Lange (The engaged university and the specificity of place: The case of Nelson Mandela Metropolitan University in the journal Development Southern Africa) and their call for ‘the delegitimisation of one form of university-community engagement that values exchange with external communities for the financial benefit of the university (and is tenuously linked to the core functions of the university) and the institutionalisation of a form of university-community engagement that values place-specific development (while simultaneously strengthening teaching and research).’ A second nod to the work of Jean-Paul Addie at Georgia State University (and convener of the Cape Town workshop) whose research on rethinking the urban university has been particularly influential. ↩