A encrypted bulletin has just been given to all the departments of Quebec by the Treasury Board. At the back of the pack? Higher Education, with an overall score of 47%, and Education, which collects 58%, while the Ministry of Health ranks third with the worst scores (67%). The Ministries of Tourism, Economy and Innovation and the Executive Council, all above the 90% mark, are good students.
The “Public Administration Performance Dashboard” 2021-2022 has not been publicized and you have to search hard to find it on the internet. It is made up of dozens of indicators (compliance with deadlines, management of state resources, good practices, etc.) and targets (based, these, on the own goals set by the departments themselves)1.
The Ministry of Education suffers in particular from not having created enough kindergarten 4-year-old classes, from not having reached the target rates of high school graduation and from not having sufficiently closed the gap of success between boys and girls. Also not achieved: the target (set at 48%) of school buildings “whose condition is satisfactory”.
The Ministry of Higher Education has not achieved the desired rates of on-time completion of a college diploma, bachelor’s degree six years after enrollment, or master’s degree four years after enrollment. It also does not display the target number of enrollments in college and university programs.
The Ministry of Health is also among those who could do better, with its long series of unmet targets (average length of stay on a stretcher, percentage of Quebecers with a family doctor, wait times for surgery, etc.) .
Unsurprisingly, the Ministère de la Famille lost points because it did not create the 5,000 subsidized childcare spaces (in facilities) as it was aiming for, but only 3,201.
“It is not satisfactory”
The office of Sonia LeBel, president of the Treasury Board, indicated that it would not comment.
The office of Bernard Drainville, Minister of Education, indicates “to take note of the findings and use them to improve the performance of the ministry”. “It’s not satisfactory, but we will work to improve things. »
The Minister of Higher Education, Pascale Déry, says she is “obviously concerned about the results obtained by the ministry”.
“Achieving the targets of the strategic plan represented certain challenges,” she adds, referring to the impact of the pandemic and the labor shortage on enrollment and on the educational path of students.
At the top of the rankings, the Ministry of Tourism, which notably exceeded its target of Quebec respondents intending to travel to Quebec in the summer, believes that its good performance is explained by its significant efforts made “to promote the recovery tourist activities “after the worst moments of the pandemic, advances Virginie Rompré, from the communications department of the Ministry of Tourism.
An important exercise, a perfectible tool
Steve Jacob, a political science professor at Université Laval specializing in program evaluation, believes that this public administration performance scorecard “is a good start” and that this accountability exercise is necessary, although that the tool is imperfect and can raise eyebrows in several respects.
Mr. Jacob gives as an example the target of 33.7% of Olympic medals at the 2021 Olympic Games collected by Quebec athletes compared to all Canadian athletes. “Why 33.7%? Why not 31% or 38%? »
But above all, he notes that while the Ministère de l’Éducation, des Loisirs et du Sport has set a target for obtaining medals, there is none that relates to Aboriginal academic success, for example.
The big question is whether the targets set by each department “really reflect the priorities of Quebec society”.
But is everything measured? A lot of things, yes, but it is still necessary, replies Mr. Jacob, that the quantitative analysis be accompanied by a qualitative analysis and that we ask ourselves if what we are measuring “makes it possible to change things”.
And to understand what is being evaluated. Because, as Mr. Jacob points out, the methodology is more than 40 pages.
This type of management by results also has certain perverse effects, observes Mr. Jacob, who points to the numerous studies showing that it often leads people in office to cheat in different ways to look good.
For his part, M.e Patric Besner, vice-president of the Institute on Governance (IGOPP), also finds the tool “extremely interesting”, although perfectible.
He is surprised, for example, that the Ministry of Justice has not set a target for “the interminable delays for civil cases”.
“How to explain, also, that there are no clear objectives of this same ministry to solve the glaring problem of shortage of judges and clerks? asks M.e Besner.
That being said, he considers such a “dashboard” important which, he reminds us, must be a tool and not an end. Because if a budget is slightly exceeded to implement an important measure for the population, it would be regrettable if those in authority refrained from doing so for fear of losing points.
The SAAQ in decline
PHOTO ALAIN ROBERGE, LA PRESSE ARCHIVES
It is especially next year, in terms of customer service, that the SAAQ newsletter is likely to do the most harm.
Already in 2021-2022 – therefore before the IT rout –, the Société de l’assurance automobile du Québec (SAAQ) was showing worrying signs in the eyes of the Treasury Board. She then obtained a score of 69% and lost 16 percentage points compared to the previous year.
These data are taken from the “Public administration performance dashboard” in which the Secretariat of the Treasury Board distributes a numerical bulletin to each of the departments (see other text) and to five public bodies.
On paper, for the SAAQ, in 2021-2022, it was fine: its strategic plan received a score of 94%. It is in terms of targets reached or not reached (44%) that things went wrong. Even before the failed IT shift, customer satisfaction was not sufficiently met, although this SAAQ objective (obtaining a satisfaction rating of at least 8.6 out of 10) was still high and was narrowly missed.
The target of at most 3.9 accidental deaths per 100,000 inhabitants was narrowly missed (we are at 4). On the other hand, the rate of people injured “with bodily injury per [tranche de] 100,000 inhabitants” gave good points to the SAAQ (we are at 187.5 compared to an initial target of 260.1).
It is especially next year, in terms of customer service, that the SAAQ newsletter is likely to do the most harm.
Other organizations
In addition to the SAAQ, four other public bodies were assessed. The Régie de l’assurance maladie, for example, scores points with its good rate of citizen satisfaction with regard to the provision of services by the Régie in prescription drug insurance.
The Commission for Standards, Equity, Health and Safety at Work (CNESST), which won an overall score of 86%, however, appears to be poor in several respects, missing by many important targets. The organization had set itself the objective that in 70% of cases, an eligibility decision for occupational diseases be rendered within 60 days. It is far from the mark, since it was ultimately the case for only 41.7% of this type of file. For a work accident, the CNESST aimed for a decision within a maximum period of 15 days for 60% of cases; it only succeeded for 32.5% of them.
Revenu Québec and Retraite Québec have achieved all of their targets, obtaining, for example, high rates of client satisfaction and good rates of use of electronic services.
For the five organizations evaluated, the overall score awarded by the Conseil du trésor “is made up of the quality of the most recent publicly released strategic plan” (value of 50% of the score) as well as “the degree of achievement of the plan’s targets presented in the annual management report for the year in question” (value of 50% of the mark).
1. For the calculation of the ministries’ overall score, the “State effectiveness” component counts for 50%, “State resource management” for 35%, “results-based management and review programs” is worth 15% of the mark. Due to the new composition of the departmental index, for 2021-2022, no comparison is made with previous years.