FREE ELECTRONIC LIBRARY - Theses, dissertations, documentation

Pages:   || 2 |

«Abstract: ROI (Return on Investment) is an increasing consideration of government and other funders with respect to activities that they support, ...»

-- [ Page 1 ] --

A Holistic Model Of Outcome Evaluation For Arts Engagement

Kim Dunphy


PhD candidate, Deakin University, Melbourne, Australia.

Paper presented at the Australasian Evaluation Society International

Conference, Brisbane, Australia, 2 - 6 September 2013.

Abstract: ROI (Return on Investment) is an increasing consideration of government

and other funders with respect to activities that they support, including the arts. This

approach is used ostensibly to ensure most prudent use of limited public resources.

However, a challenge with this approach in arts contexts is that, as ROI is generally only considered in economic terms, much of the benefit of arts engagement can be missed. A similar under-assessment is made by evaluation processes that only examine social outcomes of arts engagement.

This paper proposes a model of evaluation for outcomes of arts engagement that takes a holistic approach. Categories for outcome evaluation are drawn from two models of community sustainability; Hawkes’ (2001) four pillars of sustainability and Ife’s (2002) six dimensions of community development. This comprehensive approach eliminates the need for the problematic intrinsic and instrumental classifications of outcomes of arts engagement. The model is illustrated by case examples from the author’s research into participatory arts initiatives in Timor-Leste.

A pictorial representation of outcomes of arts engagement offers a quantitative perspective on qualitative data. In this, the model addresses a further challenge for qualitative researchers who need to present easily digestible results from large amounts of data.

The challenges in evaluating impact of the arts Evaluation of participatory arts is a burgeoning practice, with evaluation taking place more frequently across the spectrum from academic research to project related assessments, driven by funder requirements for accountability and increasing interest by program hosts in effective practice. However, evaluation of arts participation typically has a number of limitations as a result of practical and conceptual challenges. The first is that outcomes are often reported from the perspective of limited sub-section of stakeholders. Organisers’ perspectives have traditionally provided a major source of data for all types of arts engagement (McQueen-Thomson, James & Ziguras, 2004). These are limited for the obvious reasons of bias (Etherton & Prenkti, 2007).

In initiatives that focus on participation, the next most frequently considered group in outcome evaluation are active participants. In arts initiatives that focus on receptive participation, such as theatre or gallery attendance, strategies for considering impact on audiences are growing (Radbourne, Johanson & Glow, 2010; Brown & Ratzkin, 2012). However, fewer evaluation approaches apply integrated methods, where perspectives of a range of stakeholders, which might include participants, audience members, the wider community, arts leaders, other staff, funders and policy makers, are considered.

In other fields, awareness of the different perspectives and priorities of stakeholders has led to growth of methodologies that accord value to different worldviews. The methodology Most Significant Change (MSC) is used to provide a framework for systematic consideration of input from different stakeholders, from program beneficiaries through to program managers and funders (Davies & Dart, 2005). MSC is also being used in evaluation of participatory arts programs (Johnson & Stanley, 2007; Laidlaw Foundation, 2010).

The next recurrent challenge for outcome evaluation of arts initiatives is the issue of what to measure and how to measure it (South, 2004). Evaluation strategies about arts participation have traditionally focussed only on outputs, that is, the amount of opportunity provided (for example, as measured by numbers and diversity of participants), rather than outcomes, in terms of the contribution to desired goals (Matarasso, 1996; Blomkamp, 2011). This challenge is being addressed in other fields as well, with growing concern to prioritise outcome over output evident in international development (Morra-Imas & Rist, 2009) and local government (West & Cox, 2009).

However, when outcomes are emphasised in arts evaluation, a further challenge is encountered; the complexities of measuring value and outcomes of the arts because of their perceived intangible nature. These difficulties are evidenced in the persistent tensions about the intrinsic versus instrumental valuing of the arts, where intrinsic value is considered to pertain to the quality of the arts experience itself and instrumental seen as the contribution of the arts to other goals, such as social or economic progress (McCarthy, Ondaatje, Zakaras & Brooks, 2004;

Throsby, 2001; Holden, 2006).

McCarthy et al (2004) argue that intrinsic effects of the arts preclude the need for any other justification. These include immediate benefits of pleasure and captivation;

growth in individual capacities such as enhanced empathy for other people and cultures; and benefits that accrue largely to the public, including the social bonds created and expression of community identity (2004, p. 56). Throsby proposes a different perspective of intrinsic value, identifying six categories of aesthetic, spiritual, social, historical, symbolic and authenticity value (2001, p. 28-29).

An opposing view of the benefits of arts participation is instrumentalist, in which value is measured by agendas considered outside the arts, such as economic or social goals. O’Brien’s (2010) comprehensive literature review concluded that economic value is the only useful frame for evaluation of the arts as it is the major decision point for governments. A different instrumental view recognises the value of the arts for its contribution to social outcomes, including health, wellbeing and social inclusion (McQueen-Thomson et al, 2004; Kelaher et al, 2007). Another instrumental agenda examined in more recent research is the contribution of arts participation to civic engagement goals (Stern & Seifert, 2009; Mulligan and Smith, 2010).

Cultural analyst Jon Hawkes responds to this dilemma by positing that intrinsic and instrumental categorisations are only necessary because the appropriate instrumental measures have not been considered (Hawkes, 2010). He proposes that all value of arts participation is instrumental and can be categorised within one of the four dimensions of social equity, economic viability, environmental responsibility and cultural vitality. While Hawkes does not offer an evaluation framework, he does argue that arts and all other areas of public policy and investment should be subject to evaluation against those four dimensions because they are equally important aspects of a meaningful and sustainable human existence (Hawkes, 2001).

While measurement of social and economic impacts of arts participation is not new, Hawkes does pose two categories that are conceptually more recent: environmental sustainability and cultural vitality. Assessments of environmental impact are being undertaken with increasing frequency in many areas of public endeavour, although this consideration is not yet commonplace within arts evaluation. Hawkes’ most significant contribution is the addition of a cultural vitality dimension, which he posits as an essential aspect of any holistic framework for planning or evaluation. At the time of his writing in 2001, this was an infrequently considered dimension, with culture mostly not considered or subsumed under other headings, such as social.

The cultural dimension is still infrequently included in measurement frameworks because of the perception that its intangible nature renders it immeasurable (McGillivray, 2009).

A related problem is the common use of ‘arts’ and ‘culture’ as interchangeable terms.

This makes measurement of the impact of arts on culture tautological, and therefore problematic. However, if the arts are seen as one dimension of a much broader concept of culture, then it is possible to consider the cultural impact of arts initiatives.

In so doing, the troubling intrinsic/instrumental dichotomy can be addressed.

Arguments for the intrinsic value of the arts (or any other concept) are particularly risky because of their circular nature. A proposition that a concept or activity is valuable by its very nature, that is, that arts participation is valuable because it is arts, must be seen as flawed. A stronger argument can only be made against a set of shared ideas about what is valuable. What do we value, and therefore what do we need to measure, about arts participation, or any other activity?

If Hawkes’ framework is applied, almost all of the concepts that Throsby considered as intrinsic contributions of the arts can be reframed as instrumental. His aesthetic, spiritual, historical, symbolic and authenticity values can be considered within the cultural dimension. Social value can be considered within the dimension of social equity. Aspects that McCarthy et al consider as intrinsic such as “enhanced empathy for other people and cultures”, and “social bonds” (2004, p. 56), fit those dimensions well.

McCarthy’s “pleasure and captivation”, “expression of common values and community identity” and “understanding of the world” are not so easily categorized within this four pillar model. However, a model related to Hawkes’, but offering two additional dimensions offers a solution. Ife’s model of integrated community development has six categories: the same four as Hawkes, of social, cultural, environmental and economic, but also two additional categories of civic engagement and personal-spiritual well-being (Ife, 2002). If these six categories are used, they allow for categorisation of “pleasure and captivation” within the personal-wellbeing dimension, and “expression of community identity” and “understanding of the world” within the civic engagement dimension. By offering a comprehensive framework for categorising outcomes of arts participation, these models eliminate the need for the problematic conception of intrinsic value. They also connect thinking about arts participation with emerging frameworks for human progress.

Frameworks for human wellbeing and development are increasingly holistic, considering all aspects of human experience and the natural world as interconnected. Along with Hawkes and Ife’s ideas, such approaches include Clammer’s “vision of integrated development in which social, economic, cultural, political, spiritual and environmental elements are holistically related” (Clammer, 2012, p. 53).

James’ Circles of Sustainability acknowledges culture, politics, economy and ecology as interlinked dimensions within a larger framework of social sustainability (Global Compact Cities Program, 2013).

Arts evaluations are not yet indicating significant influence from these broader conceptual approaches. One significant recent exception is Cooper, Bahn and Giles (2012) who considered dimensions of economic, social, artistic and personal wellbeing in their evaluation of an indigenous arts centre.

In addition to all of the aforementioned challenges of arts evaluation, further limitations are that evaluations employing qualitative methodologies tend to be limited in consideration of direction and degree of change. Evaluation frameworks like those proposed by Morra-Imas and Rist (2009) recommend consideration of the full range of directions: positive and negative, direct and indirect, intended or unintended change. However these different directions are infrequently considered in arts outcome evaluations with assumptions of positive change prevalent (Rosenberg, 2008; Galloway, 2009). The degree of change is also infrequently specified, which leads to difficulties in determining whether resources were used wisely.

Considerations about return on investment (ROI) are increasingly included in policy and decision-making to provide assessment of effective use of resources (Farris, Bendle, Pfeifer and Reibstein, 2010). While traditional uses of ROI focus on economic return, this is being challenged by wider conceptions of social return on investment (Goodspeed, Lawlor, Neitzert & Nicholls, 2009). Again, however, integrated approaches are as yet not applied. Few arts evaluations consider amount of change proportionate to investment across a range of dimensions.

One final challenge for arts evaluation is shared with other fields in which qualitative methodologies are most frequently used. The complex findings from qualitative research can be difficult to express and share in ways that are accessible to those who make decisions, who need concise and compelling information founded on robust data. Jupp, Ali and Barahona’s (2009) model for quantifying qualitative data offers an interesting response to this dilemma, as it was developed to measure empowerment, a concept like the intrinsic value of the arts is considered problematic because of its amorphous nature. Some of the strategies Jupp et al suggest such as development of project indicators by project participants and use of symbols to reflect direction and degree of change, may offer a way forward for arts research as well.

Presenting the model To address the challenges discussed above, a holistic evaluation model is presented.

This offers a means of considering different perspectives of change (who perceived and experienced the change); dimensions of change (what kind of change occurred) and degree of change (how much change occurred).

Taking this approach, data is first categorised according to the perspectives of different stakeholders. This strategy is informed by Most Significant Change methodology (Dart & Davies, 2003), which acknowledges that an initiative might also impact on these stakeholders in different ways and these different experiences need to be considered in an evaluation. Figure 1 below offers a pictorial representation of the different types of stakeholders and their proportionate significance. This indicates that a project might primarily be expected to impact participants, but also audience members, artistic leaders, the host organisation, funders and the wider community, in lesser degrees, and each of these groups might perceive outcomes differently. These circles can be adjusted in response to the particular circumstances of any initiative or evaluation focus.

Pages:   || 2 |

Similar works:

«BioMarine Business Convention Final Report POWER BREAKFAST 1: Understanding the successful model of 3 European SMEs Thursday 8 Sept. 2011 8:30 AM Moderator: Jean Michel Panelist: Olivier Panelist Bruno Berheide, Panelist Levent Piker, Pommet, Gate2Tech, Bourtourault, Aleor, Spanish Bank of Algae, OceanBasis & Coastal Director & Chairman, President, France Marine Biotechnology Research & Management France Center, Manager,Spain (CRM) Executive Director,Germany Discussing their approach to...»

«NORBERTO BOBBIO 1 LAS PROMESAS INCUMPLIDAS DE LA DEMOCRACIA 2 Traducción: Marc Granell ————— La democracia nació de una concepción individualista de la sociedad, es decir de aquella concepción para la que ―contrariamente a la orgánica, dominante en la antigüedad y en la Edad Media, según la cual el todo es antes que las partes― la sociedad, toda forma de sociedad, especialmente la política, es un producto artificial de la voluntad de los individuos. A la formación de la...»

«Discriminación arancelaria y decisiones. / José Méndez5-19 Estudios de Economía. Vol. 33 Nº 1, Junio 2006. Págs. Naya 5 DISCRIMINACIÓN ARANCELARIA Y DECISIONES DE LOCALIZACIÓN DE LAS EMPRESAS* JOSÉ MÉNDEZ NAYA Abstract The purpose of this paper is to analyse the effects of economic integration among countries on firms investment decisions when such decisions are taken strategically. It is shown that, in this context, the traditional “tariff jumping” argument is not always...»

«An Oracle White Paper September 2011 Oracle Fusion Applications The New Standard for Business Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s...»

«Outcome-based conditionality: Too good to be true?EURODAD REPORT February 2008 About EURODAD EURODAD (the European Network on Debt and Development) is a network of 51 non-governmental organisations from 16 European countries who work together on issues related to debt, development finance and poverty reduction. The Eurodad network offers a platform for exploring issues, collecting intelligence and ideas, and undertaking collective advocacy. Eurodad’s aims are to: Push for development policies...»

«0 FACULTAD DE CIENCIAS ECONOMICAS Y ADMINISTRACION El tribunal docente integrado por los abajo firmantes aprueba la Monografía: Responsabilidad Social: Energías Renovables y Balance Social de ALUR. Autores: Grimaldi Mesecke, Ana Karina. Perdomo Pérez, María Fernanda. Soarez Cardozo, Zulma Mary. Tutor: Cra. María Noel López Gagnone. Carrera: Contador Público. Cátedra: Teoría Contable Superior. Puntaje:. Tribunal: Profesor.. Profesor.. Profesor.. Fecha: ABSTRACT El presente trabajo de...»

«Business Services / Travel Travel/Business Expenses This information outlines the general guidelines regarding travel expense reimbursement and answers the most frequently asked questions. The Business Services’ Travel Clerk is available to answer your questions and provide more detailed information at 507-389-5006 or visit the travel website at: http://www.mnsu.edu/busoff/travel/index.html. Prior Approval Do I Need Prior Approval To Travel Out-Of-State? Yes, the President needs to sign and...»

«Peer-to-Peer Rental Markets in the Sharing Economy∗ By SAMUEL FRAIBERGER AND ARUN SUNDARARAJAN To investigate whether peer-to-peer rental markets for durable goods are welfare-improving, we develop a new dynamic model of such markets in which users with heterogeneous utilization rates may also trade in secondary markets. We calibrate our model with US automobile industry data and transaction-level data from Getaround, a large peer-to-peer car rental marketplace. Counterfactual analyses...»

« AGEING POSITIVELY FESTIVAL 2014 AGEING POSITIVELY FESTIVAL 5 October 19 October 2014 Looking After Yourself. Mayor’s Message The Ageing Positively Festival is a celebration of the contributions of our older residents. In 2014 the theme is looking after yourself and we have our largest ever selection of activities for you to choose from. From cooking and fitness, to health and technology there is something for everyone. We want our residents to be healthy, engaged and connected. In Casey our...»

«FRENCH EMBASSY SMALL GRANTS PROGRAMME IN THE HUMANITIES AND SOCIAL SCIENCES. ACCRA 2009 Institutions and Sanitation in Ghana: A Case of Kumasi Metropolitan Assembly (KMA) Alexander Orall AMOAH Political Science Department, University of Ghana, Legon January 2010 Summary Ghana is facing a scaling urban sanitation crisis that debilitates, causes diseases, drains national resources, and retards the development of cities. Most affected are the major business areas in most cities where large...»

«Euclidean Geometry And Convexity Slam on the universe, economic times of business and questions and be you out that with a research. Learning to many outsourcing buyers, this income why a protection invoice advance that a Philippines filed here added although all a business expanded with the wedding to be is in of an bad example of their store office consideration eyes on programmers and relationships for without the inflation, entirely for a Federal. Euclidean Geometry and Convexity Our...»

«Wayne College Community Advisory Committee Meeting Minutes of October 25, 2013 Breakfast Interim Dean Deckler welcomed all who came to the meeting. He would like to change how we interact. Wayne College should be listening more and the community members should talk to us more. We need advice from you. We would also like to take any questions you may have from the departmental updates previously sent and if there are any questions address them. Dr. Denise Uitto was introduced as the new Interim...»

<<  HOME   |    CONTACTS
2016 www.theses.xlibx.info - Theses, dissertations, documentation

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.