Performance and Development Review Evaluation Project

Transcription

1 PDR Evaluatin Prject 1 Perfrmance and Develpment Review Evaluatin Prject The 2004 survey f staff Lucy O Sullivan Dr Anne O Brien Prfessr Alex Haslam Schl f Psychlgy Nte. The material in this dcument remains the cpyright f the authrs. N part f this publicatin can be reprduced, stred in a retrieval system, r transmitted in any frm r by any means (electrnic, mechanical, phtcpying, recrding r Staff grup undisclsedwise) withut their written permissin. Enquiries cncerning the intellectual and practical cntent f the reprt (including the survey instrument) shuld be directed t the reprt s first authr: Lucy O Sullivan, Schl f Psychlgy, University f Exeter, EX4 4QG (l.k. sullivan@ex.ac.uk).

2 Cntents PDR Evaluatin Prject 2 Page 1. Executive Summary 3 2. Backgrund 5 3. The PDR Evaluatin Prject 5 4. Survey cntext and prcess 6 A. Schematic representatin f the survey prcess 7 5. The questinnaire 8 6. Sample 9 A. Respnse rate B. Demgraphic infrmatin C. PDR Quantitative findings Qualitative findings Summary f findings References Appendices 25 A. The questinnaire B. Infrmatin fr staff C. crrespndence sent t University staff

3 1. Executive Summary PDR Evaluatin Prject 3 The Perfrmance and Develpment Review (PDR) was designed t facilitate emplyee develpment and t enable alignment between the gals f individuals, the gals f their Schl r Divisin, and the gals f the University. Nw appraching the third year f its implementatin, the PDR is a permanent feature f University prcedure: Individual staff members meet annually with a reviewer t discuss prgress and set wrk gals. Hwever, t ensure that the PDR is useful and effective as a means f setting gals, staff feedback n the PDR prcess is crucial. Fr this reasn, the University s Persnnel and Staff Develpment Divisin cmmissined a prject t btain an verview f staff views abut the PDR which culd infrm the future develpment f the PDR. At the invitatin f Tash Khan-Davis, Head f Training and Develpment, the Evaluatin Prject was cnducted by a grup f researchers at the Schl f Psychlgy. The prject was led by Lucy O Sullivan, a Masters student whse research fcuses n grup prcesses and rganisatinal gal-setting. The prject tk place ver tw phases. In Phase 1, extensive facet-face interviews were cnducted with seventeen senir members f staff frm the range f Schls and Divisins f the University. The aim f this phase was t gain a thrugh understanding f hw the implementatin f the PDR ccurred acrss the University, and t identify key issues n which t base the develpment f the staff questinnaire used in Phase 2 f the prject. The results f Phase 2, which cnstituted a survey f staff views abut the PDR, frm the basis f this reprt. All staff received a cpy f the questinnaire and were encuraged t cmplete it after their annual PDR meeting. Results f the staff survey indicate that abut half f respndents hld bradly psitive attitudes twards the PDR. In line with this, nly a small prprtin f respndents reprted having fund the PDR prcess stressful. Hwever, there was lack f cnsensus in respndents views abut the usefulness f the PDR fr achieving wrk gals, with nly 25% indicating that the PDR has been helpful in helping them t achieve their gals ver the past year. In terms f the brader impact f the PDR n relatinships in the wrkplace, the findings indicate that respndents d nt cnsider the PDR t be a means f prmting cperative practices at wrk. The findings als indicate that respndents perceive there t be a link between the PDR and decisins abut cntributin-based rewards (CBR). Furthermre, respnses t pen-ended questins suggest that this link is cnsidered t be prblematic by many respndents, bth academics and nn-academics alike. Nting the substantial time cmmitment that the PDR entails, respndents als expressed the view that the PDR has nt been wrthwhile given the amunt f effrt invested in it. The questinnaire als cntained a range f scales designed t identify the factrs that cntributed t (and predicted) psitive perceptins f the utility f the PDR fr individuals wrk. Of these, three were particularly imprtant: (a) cnsultatin abut the PDR befre ne s review meeting, (b) experiences f a review meeting which reflected cllabrative dialgue, and (c) identificatin with the University (which cntributed t feelings f trust in management). In additin, respnses t pen-ended questins ffered cnsiderable insight int staff views abut the benefits f the PDR (bth fr individuals and fr Schls and Divisins), the prblems assciated with the PDR, and suggested imprvements fr the develpment f the PDR in the future.

4 PDR Evaluatin Prject 4 While, n the whle, staff wh respnded t the survey reprted finding the PDR prcess (and the dialgues that it facilitates) useful fr their wrk, several emergent issues can be taken frward t infrm the lng-term use f the PDR. On the basis f the cllected findings, it is recmmended that in rder t enhance staff engagement with the PDR prcess (and thus t imprve its effectiveness), the University needs t address tw particular issues. First, further clarificatin is necessary with respect t the relatinship between the PDR and decisins abut cntributin-based rewards (CBR). Clarificatin n this issue may g sme way t reducing staff cynicism abut (a) the PDR itself and (b) the use f the infrmatin staff prvide n their PDR frms. Mre bradly, addressing these cncerns abut the PDR will als serve t cmmunicate t staff the precise functin f PDR within the brader plans f the University, and thus t build up trust in management. Secnd, a key finding t emerge frm the Evaluatin Prject is that there is cncern amngst staff regarding the University s cmmitment t acting n issues raised by staff in the curse f these (and similar) feedback exercises. Fr this reasn, there is a particular need fr the University t acknwledge the findings utlined in this reprt and t use the findings as the basis fr the future develpment f the PDR. Again, this will help t build n the three key predictrs f evaluatins f the utility f the PDR cnsultatin, cllabratin, and trust. In additin t the present reprt, which will be available t all staff via a secure link n the Persnnel and Staff Develpment Divisin website, a feedback summary detailing the main findings f the staff questinnaire will be sent t all members f staff. In additin, staff will be able t respnd t the findings via Heads f Schls and Divisins, and Lucy O Sullivan will be available via until Nvember 19th , shuld staff wish t ffer cmments r seek clarificatin n the findings. These strategies are intended t ensure that the Evaluatin Prject has prvided a valid, cmprehensive, and useful verview f staff s views abut the PDR and ne that accrded with emplyees wn experiences. 1 The address fr Lucy O Sullivan appears n Page 1 f this reprt.

5 PDR Evaluatin Prject 5 2. Backgrund The PDR was develped t facilitate alignment between the gals f individuals, the gals f their Schl r Divisin, and the gals f the University. With a particular emphasis n alignment f gals, the PDR represents a significant culture change in the University s apprach t perfrmance management. Mre specifically, this change is articulated in the University s Human Resurces strategy, which espuses a mve twards a high-perfrmance, high-reward culture. Given that the PDR affects emplyees acrss the University, it was very imprtant that the Persnnel and Staff Develpment Divisin shuld prvide a means thrugh which staff culd ffer their views abut the PDR prcess. On the basis f the data cllected in an evaluatin f the PDR, the University shuld be in a psitin t incrprate suggestins frm staff int future changes t the PDR prcess. In rder t address pssible cncerns amngst staff regarding annymity and cnfidentiality as well as t ensure that the evaluatin wuld prvide independent feedback abut the PDR the evaluatin was cnducted by a grup f researchers in the Schl f Psychlgy, and thus data cllectin and cllatin adhered t strict ethical standards. 3. The PDR Evaluatin Prject In Phase 1 f the prject, which tk place between September and Nvember 2003, interviews were cnducted with senir members f staff frm different Schls and Divisins. The aim f this phase was t identify key issues relating t the develpment and implementatin f the PDR. On the basis f the utcmes f this phase, the questinnaire fr Phase 2 f the prject was devised t address the range f issues likely t pertain t the implementatin f the PDR acrss the University. Drafts f the Phase 2 questinnaire (i.e., the staff questinnaire) were pilted with a small crss-sectin f staff t ensure that the instrument was cmprehensive and wuld allw staff t express their views abut the PDR. The staff questinnaire asked staff abut the fllwing three aspects f the PDR: a) Overall pinins abut the implementatin f the PDR in Schls and Divisins b) Views abut the effectiveness f the PDR fr setting wrk gals c) Views abut pssible imprvements t the PDR prcess in the future In additin t specific questins relating t these tpics, the questinnaire included a sectin in which staff culd raise thse issues that they felt were mst relevant t understanding the impact f the PDR. The questinnaire als infrmed staff that they wuld receive feedback n the findings f the Evaluatin Prject and wuld be able t cmment n these findings in advance f the next iteratin f the PDR.

6 4. Survey cntext and prcess PDR Evaluatin Prject 6 As utlined abve, the develpment f the Phase 2 staff questinnaire was infrmed by extensive earlier research bth qualitative and quantitative with a sample f senir staff members and with a crss-sectin f staff frm different staff grups. After pilting, the questinnaire was finalised by the research team (see Appendix A), and in rder t ensure that all staff received a cpy f the questinnaire, its distributin was c-rdinated by the Persnnel and Staff Develpment Divisin. The distributin f the questinnaire was timed t ccur in advance f the secnd iteratin f the PDR. Staff were invited t cmplete the questinnaire after their review meeting and then t return t it t the Schl f Psychlgy via internal mail. The questinnaire was accmpanied by an infrmatin sheet (see Appendix B). This utlined the backgrund t the Evaluatin Prject, what staff culd expect in participating, as well as infrmatin relating t the annymity and cnfidentiality f the respnses prvided. T encurage participatin, key members f staff in all Schls and Divisins were cntacted via by the Head f Training and Develpment and by the lead researcher (see Appendix C), wh bth emphasised the imprtance f the Evaluatin Prject in prviding staff with an pprtunity t express their views abut the PDR. Ethical apprval fr the prject was btained frm the Ethics Cmmittee in the Schl f Psychlgy, and the cntact details f an independent expert were prvided n the infrmatin sheet fr participants wh had queries regarding the use f the questinnaire data. The diagram verleaf prvides a schematic representatin f the PDR Evaluatin Prject, up t and including the Phase 2 survey f staff 2. 2 While fr purpses f brevity the utcmes f Phase 1 are nt presented in this reprt, the representatin n page 7 includes details f this phase t give an verview f the earlier develpment f the Evaluatin Prject; in the representatin PSDD dentes Persnnel and Staff Develpment Divisin.

7 A. Schematic Representatin f the Survey Prcess PDR Evaluatin Prject 7 July 2003 First meeting with PSDD Prepared drafts f interview schedule and questinnaire fr Phase 1 August 2003 September t December 2003 Cntacted members f SMG t set up interviews Obtained ethics apprval fr Phase 1 Pilted Phase 1 questinnaire Cnducted Phase 1 meetings with members f SMG: semi-structured interviews and Phase 1 questinnaire January 2004 Distributed summary f Phase 1 findings and cnducted feedback sessins fr members f SMG Analysed Phase 1 data Prepared draft f Phase 2 staff questinnaire February 2004 Obtained ethics apprval fr Phase 2 Pilted Phase 2 questinnaire with staff grups July 2004 September 2004 Octber 2004 PDR 2004 Finalised Phase 2 questinnaire and sent t PSDD fr distributin t all staff Prepared draft final reprt & present t PSDD Analysed Phase 2 data Staff cmpleted and returned survey Presented final reprt t SMG Octber 8th Disseminated feedback t staff

8 PDR Evaluatin Prject 8 5. The questinnaire As mentined abve, the survey instrument was develped n the basis f the Phase 1 interviews with senir members f staff acrss the University. The findings frm this phase identified key issues relating t the develpment and implementatin f the PDR. The utcmes f Phase 1 als highlighted several factrs likely t be relevant t develping an understanding f staff experiences f the PDR prcess. In additin, the research team als cnducted extensive pilt research by cnsulting with a small crss-sectin f University staff during the develpment f the Phase 2 questinnaire. The final questinnaire addressed issues including (a) general attitudes t the PDR, (b) hw the PDR was implemented in Schls and Divisin, and (c) staff experiences f their 2004 PDR meeting. In terms f the impact f the PDR, the questinnaire assessed views abut the effectiveness f the PDR fr (a) setting wrk gals, (b) achieving wrk gals, and (c) fr encuraging c-perative wrk practices. The questinnaire als included a range f scales designed t affrd additinal insight int the mechanics f staff attitudes twards the PDR. Specifically, there were measures f (a) perceptins f participatin in the PDR prcess (b), identificatin with the rganisatin, and (c) trust in the rganisatin and in management. Five pen-ended questins allwed respndents t prvide their views abut the main benefits and prblems assciated with the PDR, as well as their suggestins with respect t pssible imprvements t the prcess. The final sectin n the questinnaire requested demgraphic infrmatin that wuld prvide a general prfile f the PDR frm the perspective f different staff grups. The questinnaire is presented in full in Appendix A.

9 PDR Evaluatin Prject 9 6. Sample A. Respnse Rate 285 f the 1,883 emplyees wh received a questinnaire cmpleted the survey, a respnse rate f 15%. The respnse rate amng different staff grups is indicated in the table belw. The table als utlines the crrespnding prprtin f staff in these staff grups within the University. % f University staff N. f respndents in sample % f sample Clerical Technical Academic-related Academic Research Teaching Fellws Staff grup undisclsed Frm this table it is clear that the prprtin f respndents in different grups parallels the prprtin f these grups in the University. Therefre, while the verall respnse rate is nticeably lw, this pattern suggests that the prfile f the sample in terms f the different staff grups is bradly representative f staff in the rganisatin as a whle. It is nticeable, hwever, that the prprtin f academic-related staff in the sample was higher than the verall prprtin f this grup in the rganisatin. It is pssible that this pattern may reflect slight differences between hw staff are categrised fr fficial purpses and hw staff categrise themselves. Tw staff grups (research staff and teaching fellws) were underrepresented in the sample. The reasns fr this are unclear, thugh it is wrth nting that a significant minrity f respndents (5.6%) did nt indicate their staff grup. Quite a lw prprtin f staff cmpleted and returned a questinnaire (a respnse rate f 15%). A further 6% f staff returned uncmpleted questinnaires. This suggests that staff may nt have anticipated any utility in participating in the Evaluatin Prject, and wished t register their views by returning blank questinnaires. While several factrs may underlie the lw respnse rate, the pattern verall suggests that the majrity f staff were nt mtivated t engage with the Evaluatin Prject. Earlier discussins with staff representatives (frm the range f different grups) during pilting suggest that lw levels f participatin may reflect (a) the time invlved in cmpleting bth the PDR and the Evaluatin questinnaire and (b) cynicism abut the impact that participatin wuld have. Overall, it is als imprtant t mentin that the respnse rate may have been higher if the questinnaire had been shrter. In rder t ensure that staff culd prvide their feedback n the range f issues relating the PDR, the questinnaire was cmprehensive and lengthy. Fllwing what was fr many respndents a fairly time-cnsuming PDR prcess (bth in terms f preparatin and the meeting itself), it is pssible that the request t cmplete the Evaluatin questinnaire may have been an extra task that many staff were nt in a psitin t cmplete. In additin, as will becme clear, the emergent findings utlined in this reprt pint t the pssibility

10 PDR Evaluatin Prject 10 that the lw respnse rate may reflect a lack f understanding amngst staff as t the purpse f the PDR. It is imprtant t nte that the findings utlined in this reprt are based n cmprehensive cmpletin f the questinnaire by respndents. This is exemplified by (1) the very small amunt f missing data (i.e., mst respndents answered all questins) and (2) the fact that 80% f the respnding sample prvided substantial respnses t the pen-ended questins. This suggests that the respnding sample was quite an engaged grup with respect t the PDR: as will becme clear belw, the views expressed can be seen t demnstrate sme cmmitment t the PDR prcess. Therefre given that the sample appears bradly t have engaged with the PDR (and the Evaluatin Prject), any negative views cannt easily be dismissed as representing the perspective f a particularly disgruntled sub-sample. B. Demgraphic infrmatin The table belw prvides a general prfile f the sample. Demgraphic Variable Categries N. f respndents % f sample female Gender male undisclsed Streatham campus Lcatin St. Luke s campus ther undisclsed permanent fixed-term Nature f cntract secndment ther undisclsed A ntewrthy feature f the infrmatin presented abve is the small but significant prprtin f respndents wh did nt prvide demgraphic infrmatin. Althugh the questinnaire attempted t speak t pssible cncerns abut annymity and cnfidentiality, it seems likely that there were residual cncerns abut the use f the data prvided by respndents. C. PDR 2004 Fr a quarter f respndents, the 2004 PDR meeting tk place in March, while fr mre than 50% the meeting had ccurred between May and July. Apprximately 30% f respndents indicated that they spent less than ne hur preparing fr their PDR meeting, with a large prprtin f respndents (ver 50%) spending between ne and tw hurs in preparatin fr the meeting.

11 7. Quantitative Findings PDR Evaluatin Prject 11 This sectin details the key findings t emerge frm the questinnaire. Fr purpses f cnvenience, the number f respndents in the different staff grups are presented again belw. N Overall 285 Clerical 63 Technical 23 Academicrelated 79 Academic 79 Research 20 Teaching Fellw 5 Staff grup undisclsed 16 All items n the questinnaire were scred n a 7-pint scale (see als belw), where a scre f 1 indicates strng disagreement and a scre f 7 indicates strng agreement. The midpint f the scale is 4. A. General attitudes t the PDR T ascertain the general attitude f staff twards the PDR, we began the survey by asking tw questins directly addressing this issue. 1 = very negative, 4 = neutral, 7 = very psitive Current verall attitude t the PDR Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed I was happy t take part in the PDR prcess While the average scre n bth these questins suggest verall neutral attitudes t the PDR, ne third f respndents expressed negative attitudes, arund half expressed psitive attitudes, and ne fifth expressed a neutral attitude. This suggests there was sme plarisatin acrss the sample in terms f their attitudes t, and willingness t engage with, the PDR. 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed The PDR prcess was stressful fr me On average, respndents disagreed that the PDR was stressful. Indeed, less than a fifth f the sample indicated that they fund the prcess stressful.

12 PDR Evaluatin Prject 12 B. Satisfactin with the PDR i. Hw the PDR was implemented in Schls and Divisins 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed I feel I was infrmed abut the aims f the PDR I feel I was given the pprtunity t raise cncerns abut the PDR Overall, respndents agreed that they were infrmed abut the aims f the PDR as it ccurred within their Schl r Divisin, with 70% scring 5 r abve. Furthermre, abut half f respndents agreed that they were given pprtunities t raise cncerns abut the PDR at the level f their Schl r Divisin, while a quarter disagreed with this questin. ii. The PDR review meeting Fur questins assessed respndents satisfactin with the reviewer-reviewee match. 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed My PDR reviewer was smene wh was in an apprpriate psitin t review me My PDR reviewer was smene wh culd prvide meaningful feedback n my wrk/research activity I feel my reviewer was sufficiently prepared fr my PDR meeting Overall, I was satisfied with the way I was matched t my PDR reviewer Overall, the means suggest that respndents were satisfied with the reviewee-reviewer match. Indeed, fr each f the abve questins, arund 80% f respndents agreed that the reviewer was (a) apprpriate, (b) prvided meaningful feedback, (c) was sufficiently prepared, and this culminated in (d) high levels f satisfactin with the match.

13 PDR Evaluatin Prject 13 Three questins asked abut satisfactin with the cmpleted PDR frm. 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed My final frm was the result f tw-way dialgue between me and my reviewer Final frm was a reflectin f what happened in my review meeting Final frm was a reflectin f what I wanted t get ut f my review meeting These means illustrate that respndents were generally satisfied that their cmpleted PDR frm accurately reflected the cntent and the aims f the PDR meeting. Mre than 70% f respnses indicated agreement with each f these statements. C. Staff views abut the verall utility f the PDR in terms f gal setting i. Perceived usefulness f the PDR review meeting 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed In terms f setting gals fr my wrk, I feel that the guidelines fr the PDR meeting were sufficiently flexible I feel the PDR made it easy fr me t set useful gals fr my wrk Overall, respndents fund the PDR meeting and guidelines t be useful fr setting wrk gals, with mre than half f the sample agreeing with bth questins. Hwever, it is als wrth nting that a significant prprtin f respnses indicated either disagreement (up t 25%) r attitudes that were neither negative nr psitive (apprximately 25%).

14 PDR Evaluatin Prject 14 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed The PDR is an effective way f setting my gals and targets The PDR prcess has helped me t achieve my wrk gals ver the past year While the average scre suggests sme general level f agreement that the PDR was an effective way f setting gals and targets, less than half f the respndents indicated agreement with this statement and a third expressed disagreement. This pattern suggests that respndents were divided n this questin. A mre negative pattern emerged in relatin t the utility f the PDR prcess in helping t achieve gals ver the past year. Mre than half f respndents disagreed with this statement with less than a quarter agreeing. D. Staff views abut the general impact f the PDR n wrking life i. On relatinships at wrk 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed The PDR gives me a sense that my supervisr/line manager is supprtive f the wrk that I d The PDR serves t prmte a sense f cmmn identity within my Schl/Divisin The PDR encurages cperative wrk practices within my Schl/Divisin Half f the respndents agreed that the PDR prcess served t signal a supervisr s supprt fr their wrk, while a third f respndents disagreed with this statement. A reverse pattern emerged n items assessing brader Schl- r Divisin- based relatinships. Mre than half f respndents disagreed that the PDR (a) prmted a sense f cmmn identity r (b) encuraged c-perative wrk practices within their Schl r Divisin. Mrever, less than a fifth f respndents expressed agreement with these tw statements, while a quarter neither agreed nr disagreed.

15 PDR Evaluatin Prject 15 ii. On aligning Individual, Schl/Divisin, and University gals 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed The PDR helps me t see hw my wrk relates t what my Schl/Divisin aims t achieve The PDR helps me t see hw my wrk relates t what the University aims t achieve Over 40% f respndents disagreed that the PDR helped in aligning gals, bth at (a) the Schl/Divisin and (b) the University level. Furthermre, less than 20% f respndents agreed with these tw statements. iii. Overall assessment f the PDR 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed Staff received sufficient feedback abut the PDR frm the Schl/Divisin after the prcess was finished The PDR was wrth the amunt f effrt invested in it The PDR serves the interests f management mre than the interests f emplyees With respect t the first statement, the findings indicate that staff perceived a need fr imprved fllw-up n the PDR actin pints at a schl r divisinal level (e.g., in terms f training aims). Mre than half f respndents disagreed that staff received sufficient feedback abut the PDR after the prcess was finished, with a third neither agreeing nr disagreeing. Similarly, almst half f respndents disagreed that the PDR was wrth the amunt f effrt invested in it. Hwever, a third f respndents agreed with this statement, suggesting that respndents differed in terms f their views abut the verall value f the PDR. Over half f respndents agreed that the PDR serves the interests f management mre than the interests f emplyees, with less than a fifth disagreeing with this statement.

16 E. What ther factrs are related t staff attitudes abut the PDR? i. Cntributin-based rewards PDR Evaluatin Prject 16 The table belw presents mean respnses n three items that assessed respndents views abut the PDR with respect t perfrmance-related pay. 1 = strngly disagree, 4 = neither agree nr disagree, 7 = strngly agree Overall Mean Clerical Technical Academicrelated Academic Research Teaching Fellw Staff grup undisclsed I believe there is a link between the PDR prcess and perfrmance-related pay When cmpleting my PDR frm, I was influenced by the issue f perfrmance-related pay I am clear abut hw the PDR will be used t evaluate my future perfrmance The mean scre t the first statement abve suggests that n average respndents believed there t be a link between the PDR and cntributin-based rewards, with ver 60% f the sample agreeing with this statement and less than 20% agreeing. Nevertheless, while a quarter f respndents neither agreed nr disagreed with this statement, half f respndents indicated that they were nt influenced by the issue f cntributin-based rewards when filling ut their PDR frm. Apprximately half f respndents indicated that they were clear abut hw the PDR will be used t evaluate future perfrmance, while ne third f respndents reprted feeling unclear with respect t this issue. ii. Factrs that are assciated with mre psitive attitudes t the PDR Based n previus research in the field f rganisatinal gal-setting and perfrmance management (e.g., see Haslam et al., 2003; Fletcher, 2001), and based n statistical examinatin f respnses (using factr analysis), we cnstructed scales t measure three distinct variables which appeared t be implicated in utcmes relating t the PDR (i.e., the verall perceived utility f the PDR). These scales were measures f (a) cnsultatin prir t the implementatin f the PDR, (b) cllabratin during ne s PDR meeting, and (c) identificatin with the University. In additin, n the basis f previus research (e.g., Tyler & Kramer, 1996; Terry & Jimmiesn, 2003) regarding the mechanisms thrugh which the abve variables impact n attitudes twards rganisatinal interventins, we included measures f (d) appraised stress arising frm the implementatin f the PDR and (e) trust in management and rganisatinal prcedures.

17 PDR Evaluatin Prject 17 The table belw utlines a subset f the questinnaire items that related t these scales and f the verall scale reliability (as; N = 285). The cnventinal criterin fr satisfactry reliability is.70 (after Crnbach, 1990). Scale Example items a Prir cnsultatin Cllabrative meeting University identificatin Perceived utility f the PDR Appraised stress Trust in management Overall, I feel that I was infrmed abut the aims f the PDR I feel I was given an pprtunity t ask questins abut the PDR.85 I feel my PDR meeting was cllabrative My PDR reviewer was smene with whm I shared an understanding f what my PDR meeting shuld achieve I feel my PDR meeting allwed me a fair pprtunity t talk abut my achievements at wrk I feel strng ties with the University f Exeter The University f Exeter is imprtant t me.93 The PDR prcess is an effective way f setting my gals and targets The PDR prcess helps me t cpe with the demands f my jb The PDR prcess has helped me t achieve my wrk gals ver the past year In general, the PDR prcess was stressful fr me Overall, I am anxius abut hw the PDR will be used t evaluate my future perfrmance I trust my Head f Schl/Divisin t represent the interests f my Schl/Divisin at the University level I trust the University t act in the best interests f its emplyees Having created these scales we can examine the relatinship f the first three factrs abve t the verall utcme (i.e., attitudes regarding the utility f the PDR). The simplest way t cnduct this analysis is t establish the statistical crrelatin between the varius measures. Crrelatins can vary between 1.00 and A psitive crrelatin (between 0 and +1) indicates a psitive relatinship, such that a higher scre n ne variable is assciated with a higher scre n the ther, and a negative crrelatin (between 0 and 1) indicates a negative relatinship, such that a higher scre n ne variable is assciated with a lwer scre n the ther. The further a scre is frm 0, the strnger the relatinship is, and a value abve.30 is indicative f a mderately strng relatinship (fr mre detail see Haslam & McGarty, 2003). The crrelatin matrix fr the abve variables is presented belw. prir cnsultatin cllabrative meeting University identificatin appraised stress trust in management prir cnsultatin cllabrative meeting University identificatin appraised stress trust in management perceived utility f the PDR

18 PDR Evaluatin Prject 18 Lking at the shaded area f this table, it is clear that there are relatinships between the five factrs predicted t cntribute t respndents evaluatins f the PDR, such that higher scres n fur f these five factrs (and lwer scres n appraised stress) are assciated with mre psitive attitudes t the PDR. All f these relatinships are statistically significant. Extending this analysis, we can examine these relatinships further by cntrlling fr the pssible rle f varius backgrund variables (e.g., respndents staff grup, and their level f identificatin with their wrkgrup) t see if these relatinships hld. Mrever, if the prpsed relatinships d hld, we can attempt t psitin these variables (and the relatinships between them) within a brader cnceptual mdel. Cnducting this analysis, the fllwing mdel emerges. In this mdel, the sign (+ r ) indicates whether a relatinship is psitive r negative and the thickness f the line indicates the strength f the relatinship. prir cnsultatin appraised stress + cllabrative meeting + perceived utility f the PDR rganisatinal identificatin + + trust in management + In the abve mdel, we can see that psitive attitudes t the PDR are predicted by prir cnsultatin (b, a standardised measure f the unique relatinship between these variables, =.39), cllabrative meeting (b =.20) and identificatin with the University (b =.14). Beynd this, hwever, there is evidence that the impact f prir cnsultatin (and t a lesser extent, cllabratin) can be attributed t relatinship with lwer levels f appraised stress. In ther wrds, if respndents reprted having felt cnsulted abut the PDR, they were less likely t have fund it stressful, and this is turn was assciated with mre psitive attitudes twards the prcess. Significantly, the mdel als indicates that the psitive relatinship between rganisatinal identificatin and perceived utility f the PDR can be interpreted in terms f the assciatin between rganisatinal identificatin and trust in management. Essentially, the pattern suggests that if respndents identified strngly with the rganisatin, they were mre likely t trust the University management and its prcedures and therefre t reprt having fund the PDR prcess useful.

19 8. Qualitative Findings PDR Evaluatin Prject 19 In additin t the numerical infrmatin reprted in Sectin 7, staff were als asked pen-ended questins abut their experiences f the PDR. Specifically, the questins asked fr staff views abut the benefits f the PDR (1) fr individual respndents and (2) fr the schl r divisin f which they are a member, (3) the main prblems assciated with the PDR, and (4) suggested imprvements t the PDR. A final questin gave staff the pprtunity t raise issues that they felt had nt been cvered elsewhere. It is imprtant t nte that many respndents prvided extensive cmments t these fur questins, thus prviding very useful further insights int the issues raised by the quantitative findings. As mentined earlier abve, the substantial amunt f infrmatin prvided t these pen-ended questins suggests respndents wh were bradly speaking engaged with the PDR prcess and with the Evaluatin Prject. A: Main benefits f the PDR fr yu 75% f respndents ffered their views n this questin. The table belw utlines the main issues raised by bth academic and nn-academic staff. Respnses have been categrised thematically and indicate the percentage f respnses which reflected a particular theme. N benefits (25%) Nn-academics (N = 125) Academics (N = 80) Nne beynd usual annual gal-setting Cmpliance with management requirements N benefits (38%) N great benefit as it s nt a mtivatr Nne: a bureaucratic exercise Setting gals (34%) Setting relevant future targets Gives fcus t future plans Gain manager s supprt fr persnal gals Setting gals (30%) Clarifying and fcussing research agenda Priritising gals fr cming year Dialgue (18%) Time t have a cnstructive discussin Chance t let ff steam and seek supprt Recgnitin f achievements (15%) Reflecting and listing achievements Recgnitin frm manager fr achievements Feedback & reflectin (15%) Chance t seek feedback n wrk Discuss rle and clarify expectatins Training (10%) Identify training needs Gain manager s supprt fr training gals Financial benefits (2%) Pssibility f future financial reward Expectatin f merit award Dialgue (19%) Opprtunity fr a cnversatin A gd chat with a clleague Recgnitin f achievements (15%) Satisfactin in listing achievements Discvering that effrts are valued Feedback & reflectin (15%) Ask fr feedback n/supprt fr wrk agenda Chance t take stck and seek feedback Training (3%) Identify training needs Gain manager s supprt fr training gals Financial benefits (3%) Perhaps receive a merit award

20 PDR Evaluatin Prject 20 B: Main benefits f the PDR fr yur Schl/Divisin 68% f respndents ffered their views here. It is wrth nting that fewer academics than nnacademics respnded t this questin. Nn-academics (N = 115) Academics (N = 72) N benefits (17%) Nne that are apparent Schl can tick a bx t say PDR is cmpleted N benefits (14%) Des nt imprve n previus review system Nne, beynd fulfilling university requirement Unknwn (14%) Dn t knw N idea hw the PDR has affected Schl Unknwn (26%) Dn t knw, even thugh everyne cmpletes it annually It remains t be seen/as yet unclear hw PDR is useful fr the Schl Aligning individual and S/D gals (29%) Setting f individual gals in relatin t schl/divisin gals Encurages gal cngruence Management planning Ensures that staff are aware f their rle within Divisin Clarificatin f rles with respect t shared gals Overview f activities (11%) Department can see prgress since last review General perspective n wrking practices Prvides picture f everyne s training needs Identifies training gaps Staff participatin and vice (10%) Helps ensure that staff needs are heard Means f encuraging staff t air ideas Discussins give mutual supprt t staff Prvides feedback n individuals ideas Aligning individual and S/D gals (25%) Identifies hw individual gals serve Schl gals Specifies Schl aims and bjectives Helps t streamline staff rles within the Schl Overview f activities (11%) Mnitring staff fr the purpses f the RAE Reviewers may get insight int life f Schl Staff participatin and dialgue (13%) Gets all staff grups invlved Opprtunity t share supprt fr activities by cnstructive discussin Equal pprtunity t talk abut gals Recgnitin f achievements (4%) Prvides understanding f staff achievements Shws hw individual achievements can help team wrking Cllective visin (3%) Wrks as a tl fr encuraging teamwrk Prvides a general cllective fcus Cllective fcus (3%) Helps t cnvey cllective directin

21 PDR Evaluatin Prject 21 C: Main prblems assciated with the PDR 75% f respndents ffered their views t this questin. Nn-academics (N = 126) Academics (N = 79) Lack f clarity regarding aim f the PDR (21%) Cnfusin regarding nature f link with CBR Link with CBR will be divisive and will undermine teamwrk Inadequate cmmunicatin f aim f the PDR Uncertainty surrunding the underlying purpse f the PDR Perceptin that senir management d nt subscribe t the PDR and are fulfilling university requirement Lack f clarity regarding aim f the PDR (37%) The link with pay may lead t cnflict Link with CBR will be divisive and will undermine teamwrk Perceptin that the PDR is a tl f management because its purpse is unclear Cncerns regarding hw (and by whm) PDR infrmatin is used Uncertainty as t exactly what staff are signing up t Time invlved (19%) PDR is a big time cmmitment, especially fr reviewers Staff unclear as t why PDR is imprtant Prblems with setting gals (15%) PDR des nt have flexibility t allw all staff grups t set gals Nature f certain jbs means setting annual gals is difficult Diversity f tasks, staff grups, and gals makes cmparisn difficult Represents a gal-centric view f perfrmance and prductivity, and therefre perhaps nt a gd basis fr decisins abut prmtin Lack f fllw-thrugh (18%) Cmpleted PDR frms are filed withut training requests being delivered upn Lack f infrmal dialgue t ensure cntinuity Highlighted issues remain unreslved Lack f feedback frm Schl/Divisin abut actin taken based n the PDR Prcedural prblems: PDR frm (12%) Frm is lng and difficult t navigate Difficult t understand the purpse f varius sectins n the PDR frm Frm is nt very amenable t electrnic cmpletin Prcedural prblems: PDR reviewers (6%) Difficult t ensure cnsistency f apprach acrss reviewers Apprpriateness f designated reviewer Time invlved (24%) Cnsiderable administrative burden Sense that the PDR is a bureaucratic exercise The PDR places a big time cmmitment n reviewers Prblems with setting gals (24%) Difficulty f being required t set quantifiable gals acrss different rles Insufficient scpe fr different frms f perfrmance Insufficient recgnitin that gals change ver time Over-emphasis n individual gals and utputs ignres imprtance f teamwrk Reflects a view f perfrmance that nly rewards individuals and which prevents a general increase in resurces fr all Lack f fllw-thrugh (6%) Lack f feedback frm university and schl regarding actin taken after PDRs Perceptin that the PDR is just a ne-way prcess f frm-filling Prcedural prblems: PDR frm (4%) Frm seems repetitive and therefre cmpletin is an administrative burden Prcedural prblems: PDR reviewers (4%) Nn-cnstructive dialgue Ill-prepared reviewer

22 PDR Evaluatin Prject 22 D: Hw culd the PDR be imprved in the future 64% f the sample respnded t this questin. Nn-academics (N = 107) Academics (N = 68) Develp alternative gal-setting tl (13%) Ablish the PDR Return t previus system f appraisal Develp alternative gal-setting tl (18%) Ablish the PDR Intrduce different system fr prmtin Link with cntributin-based reward (13%) Remve link with CBR Clarify link with CBR Clarify purpse f the PDR (9%) Clarify exactly why the PDR has been implemented Explain t staff hw the PDR benefits their wrk Setting gals (15%) Allw prcess that acknwledges different types f gals fr different staff grups Allw scpe fr interim review f prgress in less frmal circumstances Develp the PDR s that staff can set wrk gals as well as individual gals PDR meetings (12%) Train reviewers t encurage and allw equal participatin in review meeting Allw chice in terms f reviewer and venue PDR frm (11%) Make PDR frm simpler, shrter and less repetitive Allw scpe fr nn target-related aims: general aims sectin Fllw-thrugh (10%) Increased fllw-thrugh n gals set ut in PDR, especially training gals Feedback frm Schl/Divisin and University n actin taken n matters arising N imprvements (3%) The PDR prcess requires n change Link with cntributin-based reward (20%) Remve link with CBR Clarify link with CBR Clarify purpse f the PDR (18%) Address cncerns abut mtive f PDR by clarifying hw the infrmatin is used Mre infrmatin n hw the prcess wrks n a larger scale Setting gals (9%) Allw prcess that acknwledges different types f gals fr different staff grups Allw scpe fr interim review f prgress in less frmal circumstances Encurage team awareness PDR meetings (9%) Prvide guidelines n the rle f reviewer (what srt f feedback reviewer shuld ffer) Encurage a mre infrmal PDR meeting Ensure useful match between reviewer and reviewee PDR frm (13%) Make PDR frm mre cncise and mre amenable t speedy cmpletin Prvide guidelines n hw sectins shuld be cmpleted, especially fr new staff Fllw-thrugh (9%) Encurage grup meeting t share experiences abut PDR benefits Prvide regular interim fllw-up N imprvements (2%) The PDR prcess requires n change

23 9. Summary f findings PDR Evaluatin Prject 23 The results presented abve indicate that, n the whle, staff wh respnded t the evaluatin questinnaire were reasnably psitive in their attitudes twards the PDR. In line with this, nly a small prprtin f respndents reprted having fund the experience stressful. In terms f the PDR prcess as it ccurred in Schl r Divisin, many respndents held psitive views in this respect. Specifically, the findings suggest that many respndents were satisfied with the way they were matched t their PDR reviewer and that the meeting was characterised by cllabrative dialgue with respect t wrk gals and related career develpment issues. In this vein, the qualitative findings (see Sectin 8A abve) suggest that a main benefit f the PDR prcess fr many respndents was the fact that it prvided an pprtunity fr pen discussin abut wrk and related issues with a supprtive clleague, as well as the chance t recrd achievements. Hwever, it is als imprtant t nte that a significant minrity f respndents felt that there were n substantive benefits f the PDR fr their wrk. In terms f the specific prblems assciated with the PDR, there was sme indicatin that staff did nt receive sufficient feedback frm their Schl r Divisin when the prcess was cmplete. This finding was als brne ut in respnses t the pen-ended questins (see Sectin 8C), where bth nn-academic and academics indicated that there was a lack f feedback arising frm the PDR, bth with respect t gals and issues arising (e.g., training, resurces). In terms f the brader impact f the PDR, there was less cnsensus in respndents attitudes regarding the verall utility f the PDR prcess as a tl fr setting gals. Many respndents indicated that they have nt fund the PDR t be effective in this respect. Mrever, many respndents indicated that they were unclear as t the purpse f the PDR. Taken tgether, these findings suggest that negative evaluatins f the utility f the PDR as a gal-setting tl may reflect (a) the cnsiderable time invlved in cnducting the PDR given that its purpse was unclear and (b) difficulties in setting gals (e.g., team gals) given jb and rle diversity. Significantly, respndents views regarding the utility f the PDR in raising awareness abut Schl, Divisinal and University gals suggest that the PDR was nt fund t aid alignment between gals at these different levels. Mrever, the findings strngly indicate that the PDR is perceived nt t prmte team-based wrking dynamics such as c-perative wrk practices and develping a sense f cmmn identity in Schls and Divisins. The findings als indicate that fr this sample f respndents at least the PDR is nt perceived t have been wrthwhile given the amunt f time invested in it. Significantly, it als appears that the PDR is currently perceived t serve management interests mre s than the interests f emplyees. Furthermre, the findings shw that many respndents believe there t be a link between the PDR and cntributin-based rewards. Interestingly, hwever, many respndents indicated that they were nt influenced by the issue f cntributin-based rewards when cmpleting their PDR. This pattern may imply that respndents psitive attitudes t the PDR reflect the fact that it prvided an pprtunity t spend valuable time discussing their wrk. As such, it is pssible that in cntext f the PDR meeting, the issue f cntributin-based rewards was f secndary imprtance given the nature f the dialgue between reviewer and reviewee.

24 PDR Evaluatin Prject 24 Overall, the main message t emerge frm this bdy f findings is that while the PDR was fund t be useful n several levels specifically the pprtunity fr cnsultatin and supprtive mentring that it prvides there remains the need fr clarificatin with respect t (a) the underlying purpse f the PDR and (b) hw infrmatin prvided n PDR frms is used by management and by the University, specifically in relatin t cntributin-based rewards. The findings als suggest that it may be useful t extend the psitive rle f the PDR in enabling cnstructive dialgue t address team issues at wrk (e.g., grup gal-setting, grup reviews), but withut increasing the administrative burden f the prcess as a whle. Finally, an imprvement in fllw-thrugh actins t issues arising frm PDR meetings wuld help t demnstrate the nging nature f the PDR cycle and may enhance staff satisfactin with the PDR prcess. 10. References Crnbach, L. J. (1990). Essentials f psychlgical testing (5 th Ed.). New Yrk: Harper & Rw. Fletcher, C. (2001). Perfrmance appraisal and management: The develping research agenda. Jurnal f Occupatinal and Organizatinal Psychlgy, 74, Haslam, S. A., Eggins, R. A., & Reynlds, K. J. (2003). The ASPIRe mdel: Actualizing scial and persnal identity resurces t enhance rganizatinal utcmes. Jurnal f Occupatinal and Organizatinal Psychlgy, 76, Haslam, S. A. & McGarty, C. (2003). Research methds and statistics in psychlgy. Lndn: Sage. Terry, D. J. & Jimmiesn, N. L. (2003). A stress and cping apprach t rganisatinal change: Evidence frm three field studies. Australian Psychlgist, 38, Tyler, T. R. & Kramer, R. M. (1996) (eds.). Trust in rganizatins: Frntiers f thery and research. Thusand Oaks, CA: Sage.

25 Appendix A. The questinnaire 11. Appendices PDR Evaluatin Prject 25 SCHOOL OF PSYCHOLOGY WHAT DO YOU THINK ABOUT THE PDR? This questinnaire relates t the University s new appraisal scheme, the Perfrmance and Develpment Review (PDR). The questinnaire will ask yu abut the fllwing three aspects f the PDR prcess: 1) Yur verall pinins abut the implementatin f the PDR in yur Schl/Divisin 2) Yur views abut the effectiveness f the PDR fr setting wrk gals 3) Yur views abut pssible imprvements t the PDR prcess in the future As yu wrk thrugh the sectins f the questinnaire, bear in mind that the fcus is n yur experiences f the PDR prcess as a reviewee - there are n right r wrng answers. All yur respnses are annymus and cnfidential. The questinnaire will take abut 20 minutes t cmplete. Please try t cmplete all sectins, even if there appears t be sme repetitin in the questins we are asking. At the end f the questinnaire, we ask yu t generate a self-identifying cde. This is because t track changes in pinins ver time, we need t be able t match this questinnaire with respnses t later questinnaires, while still ensuring that yur respnses are annymus. Yur participatin in this questinnaire is cmpletely vluntary. PLEASE NOW TURN TO THE NEXT PAGE TO BEGIN THE QUESTIONNAIRE