Interim Evaluation Report, April 2005

Introduction

PROJECT ñ engaging artists in the built environment provides financial assistance to support collaboration between artists and design, planning and construction professionals working proactively with public and private sector agencies on projects in the built environment. The intention is to facilitate the artist's role as a creative thinker, working within the team appointed to deliver the project.

Evaluation

Comedia was appointed by Public Art South West in August 2004 to carry out independent evaluation of projects funded during the two year pilot period of PROJECT. The research brief asks Comedia to look in detail at how the artist works within the design team and the effect of their collaboration on the design process and the design team dynamics. The three main research questions are:

Did the people involved in PROJECT funded schemes change their usual mindsets and/or working practices through collaboration with an artist?
Has the involvement of artists made any demonstrable difference to projects?
Has there been any impact on long term practice of the project players?

A supplementary question is: under what conditions does artist input have a positive effect and when does it not?

Process rather than product

The evaluation is concerned with the effects of the involvement of artists on the people engaged in the projects, rather than with the products which are ultimately created. We therefore developed a framework for the evaluation which seeks to examine these effects from several points of view to achieve an appraisal of what happens, taking into account both intended and unintended effects. We are looking at the expectations of participants in the first instance, at their experience of the process as it happens, and at their perception of what happened in retrospect. In selected awards we are taking a participant-observer position, watching at first hand what happens during key interactions of the participants, with an eye to individual and group behaviours.

Evaluation materials

A set of materials to assist this process has been created:

Questionnaire 1 ñ an initial questionnaire for participants in the projects supported by PROJECT awards, focusing on their motivation and expectation of how the project will be affected by the involvement of an artist.
Personal Journal ñ All the people involved directly in schemes supported by awards are asked to keep a personal journal of their experience of the project to assist the evaluation. The personal journal is designed to help capture information about how participants experience what happens and to establish what, if any, change has come about as a consequence of the scheme.
Questionnaire 2 ñ a final questionnaire for participants designed to elucidate their perception of what the effect of the involvement of an artist in the project has been.
Participant Observer Workbook ñ this workbook is designed as a guide for the researchers taking the role of participant observer in selected projects, and provides a checklist of group and individual behaviour characteristics to assist in making consistent records.
Structured interview proforma ñ Selected awards will be interviewed towards the end of their project, and this proforma assures consistency in carrying out the interviews.

Types of evaluation

Within the available time and resources, we expect to gather data from a proportion of the anticipated total of 30 awards over the two year period, as follows:

Initial and Completion Survey - 85-100% of awards (25 ñ 30) projects
Introspective Journals - 70 ñ 100% of awards (21 ñ 30) projects
Semi-structured interviews ñ 33% of awards or ten projects
Participant Observation ñ 17% of awards or five projects

All awards are asked to undertake the two questionnaires and the Personal Journals. The choice of projects for which to conduct participant observation and follow-up interviews has been informed by the need to produce a representative sample of awards. We take account of the following criteria in seeking to achieve a balanced selection:

Type of award and number of awards in category;
Nature of stakeholders involved, including the bodies which have initiated and commissioned PROJECT, artists, town planners, architects, relevant policy-makers, property developers and other business sector actors, and local citizens;
Relationship to key issues in urban and rural regeneration;
Relevance to different social groups, including young people, the elderly, the disabled, women, ethnic minorities and people with different occupational profiles;
Regional and national geographic distribution.

Participants

Since the evaluation focuses on the participants rather than the product, a key question is ëwho counts as a participant?' All projects are different, so it is a matter of judgement, but we give a general guideline. The object of PROJECT is to engage artists with planning developments in the built environment. This will involve a range of development professionals such as developers, planners, architects, local authorities and development agencies, and other partners including community groups. What we are seeking to evaluate is the effect of introducing an artist into this milieu, both on the various professionals and community groups involved and the artist. So in deciding who to involve, we suggest that projects choose people who have some extended involvement, rather than a passing contact, and as far as possible a range of professionals and involved people from relevant communities. The sponsors of PROJECT are particularly interested in its impact on the private sector, so we encourage projects to include developers, builders, architects etc where possible.

Progress

The nine awards in the first round of PROJECT received the evaluation materials and guidance in early January 2005. We have been consulting with individual awards on the detail of the process. The first batches of returns began to arrive in February, and are continuing to do so as projects get under way. So far, two awards have returned their initial batch of questionnaires, and six others expect to do so during April. Meanwhile the seven awards in the second round have had their evaluation packs and arrangements for their process are being set up.

Emerging issues

Though the response is limited so far, some themes and questions are emerging even at this early stage.

Expectations are high that the artists' involvement will contribute positively to the projects.
Some of the local authority professionals involved see the involvement of an artist as a stimulus to challenge their own thinking and working practice.
Others, while anticipating a positive contribution from the artist, do not expect the experience to change their established way of doing things.
From a developer's perspective, there is a question whether the involvement of an artist will have any effect on the final outcome of the building project itself, the impact perhaps being limited to the public realm.
An issue for artists is the risk of being seen as a substitute architect or designer, rather than as making a professional contribution in their own sphere.

The PROJECT scheme is a complex one, with six different classes of award, and it is difficult to create in advance a single evaluation process that can accommodate to the diversity of an unknown range of actual projects. Many of the awards are linked to capital development projects, giving plenty of scope for things to change and timetables to slip. The pressing demands and contingencies of capital projects tend to push the requirements of evaluation towards the back burner.

Nonetheless, the value of the evaluation process is recognised by most of the awards so far made, and we are encouraged by the quality and seriousness of the response from those whose projects have so far got under way.

Fred Brookes
Comedia
22 04 2005


Skip to content Back to Public Art Online main site Home The Scheme Awards Evaluation • Process • Reports News Contact Download Logo Using the Site Copyright Privacy	Interim Evaluation Report, April 2005 Introduction PROJECT ñ engaging artists in the built environment provides financial assistance to support collaboration between artists and design, planning and construction professionals working proactively with public and private sector agencies on projects in the built environment. The intention is to facilitate the artist's role as a creative thinker, working within the team appointed to deliver the project. Evaluation Comedia was appointed by Public Art South West in August 2004 to carry out independent evaluation of projects funded during the two year pilot period of PROJECT. The research brief asks Comedia to look in detail at how the artist works within the design team and the effect of their collaboration on the design process and the design team dynamics. The three main research questions are: Did the people involved in PROJECT funded schemes change their usual mindsets and/or working practices through collaboration with an artist? Has the involvement of artists made any demonstrable difference to projects? Has there been any impact on long term practice of the project players? A supplementary question is: under what conditions does artist input have a positive effect and when does it not? back to top Process rather than product The evaluation is concerned with the effects of the involvement of artists on the people engaged in the projects, rather than with the products which are ultimately created. We therefore developed a framework for the evaluation which seeks to examine these effects from several points of view to achieve an appraisal of what happens, taking into account both intended and unintended effects. We are looking at the expectations of participants in the first instance, at their experience of the process as it happens, and at their perception of what happened in retrospect. In selected awards we are taking a participant-observer position, watching at first hand what happens during key interactions of the participants, with an eye to individual and group behaviours. Evaluation materials A set of materials to assist this process has been created: Questionnaire 1 ñ an initial questionnaire for participants in the projects supported by PROJECT awards, focusing on their motivation and expectation of how the project will be affected by the involvement of an artist. Personal Journal ñ All the people involved directly in schemes supported by awards are asked to keep a personal journal of their experience of the project to assist the evaluation. The personal journal is designed to help capture information about how participants experience what happens and to establish what, if any, change has come about as a consequence of the scheme. Questionnaire 2 ñ a final questionnaire for participants designed to elucidate their perception of what the effect of the involvement of an artist in the project has been. Participant Observer Workbook ñ this workbook is designed as a guide for the researchers taking the role of participant observer in selected projects, and provides a checklist of group and individual behaviour characteristics to assist in making consistent records. Structured interview proforma ñ Selected awards will be interviewed towards the end of their project, and this proforma assures consistency in carrying out the interviews. back to top Types of evaluation Within the available time and resources, we expect to gather data from a proportion of the anticipated total of 30 awards over the two year period, as follows: Initial and Completion Survey - 85-100% of awards (25 ñ 30) projects Introspective Journals - 70 ñ 100% of awards (21 ñ 30) projects Semi-structured interviews ñ 33% of awards or ten projects Participant Observation ñ 17% of awards or five projects All awards are asked to undertake the two questionnaires and the Personal Journals. The choice of projects for which to conduct participant observation and follow-up interviews has been informed by the need to produce a representative sample of awards. We take account of the following criteria in seeking to achieve a balanced selection: Type of award and number of awards in category; Nature of stakeholders involved, including the bodies which have initiated and commissioned PROJECT, artists, town planners, architects, relevant policy-makers, property developers and other business sector actors, and local citizens; Relationship to key issues in urban and rural regeneration; Relevance to different social groups, including young people, the elderly, the disabled, women, ethnic minorities and people with different occupational profiles; Regional and national geographic distribution. back to top Participants Since the evaluation focuses on the participants rather than the product, a key question is ëwho counts as a participant?' All projects are different, so it is a matter of judgement, but we give a general guideline. The object of PROJECT is to engage artists with planning developments in the built environment. This will involve a range of development professionals such as developers, planners, architects, local authorities and development agencies, and other partners including community groups. What we are seeking to evaluate is the effect of introducing an artist into this milieu, both on the various professionals and community groups involved and the artist. So in deciding who to involve, we suggest that projects choose people who have some extended involvement, rather than a passing contact, and as far as possible a range of professionals and involved people from relevant communities. The sponsors of PROJECT are particularly interested in its impact on the private sector, so we encourage projects to include developers, builders, architects etc where possible. Progress The nine awards in the first round of PROJECT received the evaluation materials and guidance in early January 2005. We have been consulting with individual awards on the detail of the process. The first batches of returns began to arrive in February, and are continuing to do so as projects get under way. So far, two awards have returned their initial batch of questionnaires, and six others expect to do so during April. Meanwhile the seven awards in the second round have had their evaluation packs and arrangements for their process are being set up. back to top Emerging issues Though the response is limited so far, some themes and questions are emerging even at this early stage. Expectations are high that the artists' involvement will contribute positively to the projects. Some of the local authority professionals involved see the involvement of an artist as a stimulus to challenge their own thinking and working practice. Others, while anticipating a positive contribution from the artist, do not expect the experience to change their established way of doing things. From a developer's perspective, there is a question whether the involvement of an artist will have any effect on the final outcome of the building project itself, the impact perhaps being limited to the public realm. An issue for artists is the risk of being seen as a substitute architect or designer, rather than as making a professional contribution in their own sphere. The PROJECT scheme is a complex one, with six different classes of award, and it is difficult to create in advance a single evaluation process that can accommodate to the diversity of an unknown range of actual projects. Many of the awards are linked to capital development projects, giving plenty of scope for things to change and timetables to slip. The pressing demands and contingencies of capital projects tend to push the requirements of evaluation towards the back burner. Nonetheless, the value of the evaluation process is recognised by most of the awards so far made, and we are encouraged by the quality and seriousness of the response from those whose projects have so far got under way. Fred Brookes Comedia 22 04 2005 back to top	Index: Introduction Evaluation Process or product Evaluation materials Types of evaluation Participants Progress Emerging issues