Bridging the gap in public sector evaluation: reconciling best practices and client recognition in a mandated review of a program

• Main Author: Barnes, Maria Paulette
• Other Authors: Higenbottam, John Allyn
• Format: Others
• Language: English; en
• Published: 2018
• Subjects: Public administration; Canada; Evaluation; Social sciences
• Online Access: https://dspace.library.uvic.ca//handle/1828/10186

With a history going back to the beginning of this century, issues of accountability and fiscal responsibility — often under the guise of program evaluation or review — have been at the forefront of decision-making in recent years for programs that rely on government funding. The dissertation concerns the utility of evaluation and review in shaping public policy, and consists of three distinct elements. Starting with an examination of what is required to carry out a review function in complex organizational contexts, the best practices available in the evaluation literature were identified with the purpose of creating a review framework for a program, AMPA, which is administered by a department within the federal government (Agriculture and Agri-Food Canada [AAFC]) to the agriculture and agri-food sector. Given that the framework was put in place to enable its direct clients - program and senior managers at AAFC - to obtain a higher calibre review of the program than if it had not been available, it served as an exemplary case study in discovering robust and unique solutions to the barriers facing review initiation and implementation. This strategy for reviewing AMPA included the development of a detailed implementation plan and the situation of the framework in its organizational context The second element in the dissertation was an empirical test of the strategy to prepare AAFC for the review of AMPA, and a methodology was devised to appraise the degree of success achieved in serving the program’s direct clientele. In short, a questioning of whether or not the review framework was an effective utilization-centered evaluation tool was carried out The extent to which the framework was implemented two years after it was created was probed, and it was found that these efforts had been moderately successful. However, in the dissertation’s third part it is revealed that the definition of success derived from best practices in the evaluation literature was inadequate; It should have included an understanding of what the ultimate clients in the review of AMPA had in mind in initiating the review. And it is only with an extraordinary - and in terms of everyday review practices, impractical - amount of investigation into Parliamentarians’ purpose that this motivation was detected. My final analysis began by examining the Agricultural Marketing Programs Act which governs the program; more precisely, its mandatory review clause which requires AAFC to review AMPA five years after the legislation was enacted. Apart from a few passing references, such clauses have not been examined by academic commentators or public servants in any systematic manner even though there are early indications that program reviews driven by, and supported under the law, may become more prevalent Five possible explanations to account for the appearance of this clause were proposed, and the available evidence supports the government’s concern over the potential trade-distorting implications of AMPA at the time the legislation was debated in, and subsequently passed through, the House of Commons. In conclusion, by identifying the government’s tactic in allaying international attention over the program’s impact on trade, one must confront the realization that review efforts cannot meet Parliamentarians’ needs, given that these could not have been known within AAFC as the review framework was being developed. Nor are Parliamentarians’ general expectations for performance information widely known). In retrospect, it appears that the formative review tools lauded in the evaluation literature will not meet the requirement of serving a broad public interest In terms of the vast reporting to Parliament literature, as assessed from a broad interdisciplinary perspective, it is possible to observe that the methods available to practitioners presently are unable to bridge a profound between carrying out summative evaluation and identifying effective public policy. This gulf between the promise and performance of evaluation is highlighted in the dissertation, as is the suggestion that doing something right in this domain is not the same as doing the right thing. === Graduate