Bridging the gap in public sector evaluation: reconciling best practices and client recognition in a mandated review of a program
With a history going back to the beginning of this century, issues of accountability and fiscal responsibility — often under the guise of program evaluation or review — have been at the forefront of decision-making in recent years for programs that rely on government funding. The dissertation con...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Others |
| Language: | English en |
| Published: |
2018
|
| Subjects: | |
| Online Access: | https://dspace.library.uvic.ca//handle/1828/10186 |
| Summary: | With a history going back to the beginning of this century, issues of accountability and fiscal
responsibility — often under the guise of program evaluation or review — have been at the
forefront of decision-making in recent years for programs that rely on government funding.
The dissertation concerns the utility of evaluation and review in shaping public policy, and
consists of three distinct elements. Starting with an examination of what is required to carry
out a review function in complex organizational contexts, the best practices available in the
evaluation literature were identified with the purpose of creating a review framework for a
program, AMPA, which is administered by a department within the federal government
(Agriculture and Agri-Food Canada [AAFC]) to the agriculture and agri-food sector. Given
that the framework was put in place to enable its direct clients - program and senior
managers at AAFC - to obtain a higher calibre review of the program than if it had not been
available, it served as an exemplary case study in discovering robust and unique solutions
to the barriers facing review initiation and implementation. This strategy for reviewing
AMPA included the development of a detailed implementation plan and the situation of the
framework in its organizational context
The second element in the dissertation was an empirical test of the strategy to prepare AAFC
for the review of AMPA, and a methodology was devised to appraise the degree of success
achieved in serving the program’s direct clientele. In short, a questioning of whether or not
the review framework was an effective utilization-centered evaluation tool was carried out
The extent to which the framework was implemented two years after it was created was
probed, and it was found that these efforts had been moderately successful.
However, in the dissertation’s third part it is revealed that the definition of success derived
from best practices in the evaluation literature was inadequate; It should have included an
understanding of what the ultimate clients in the review of AMPA had in mind in initiating
the review. And it is only with an extraordinary - and in terms of everyday review practices,
impractical - amount of investigation into Parliamentarians’ purpose that this motivation was
detected.
My final analysis began by examining the Agricultural Marketing Programs Act which
governs the program; more precisely, its mandatory review clause which requires AAFC to
review AMPA five years after the legislation was enacted. Apart from a few passing
references, such clauses have not been examined by academic commentators or public
servants in any systematic manner even though there are early indications that program
reviews driven by, and supported under the law, may become more prevalent Five possible
explanations to account for the appearance of this clause were proposed, and the available
evidence supports the government’s concern over the potential trade-distorting implications
of AMPA at the time the legislation was debated in, and subsequently passed through, the
House of Commons.
In conclusion, by identifying the government’s tactic in allaying international attention over
the program’s impact on trade, one must confront the realization that review efforts cannot
meet Parliamentarians’ needs, given that these could not have been known within AAFC as
the review framework was being developed. Nor are Parliamentarians’ general expectations
for performance information widely known). In retrospect, it appears that the formative
review tools lauded in the evaluation literature will not meet the requirement of serving a
broad public interest In terms of the vast reporting to Parliament literature, as assessed from
a broad interdisciplinary perspective, it is possible to observe that the methods available to
practitioners presently are unable to bridge a profound between carrying out summative
evaluation and identifying effective public policy. This gulf between the promise and
performance of evaluation is highlighted in the dissertation, as is the suggestion that doing
something right in this domain is not the same as doing the right thing. === Graduate |
|---|