A study1 on the effects of State Secularism Act (“Law 21”) piloted by three researchers from McGill and Concordia universities has circulated a lot in recent days. It was relayed by The duty, The Journal of MontrealCTV and CBC despite significant scientific shortcomings.
Posted yesterday at 4:00 p.m.
The study sought to gather information on students in education and law, two areas directly affected by Bill 21. Has the law changed the perception of these students about Quebec? Has it affected their experience of discrimination?
You have to read the study to understand the portrait painted by the authors, but the simple answer to these questions is as follows: many students say they have experienced more discrimination since the adoption of the law and are even considering leaving Quebec because of lack of job opportunities.
Methodology problem
Such a study seems interesting at first glance, but there are several problems, especially on the methodological level. The study acknowledges that the survey sample “is relatively small and not necessarily representative”. It adds that “there is a strong possibility of a selection bias in favor of those opposed to the law”.
This is because the researchers notably sent their questionnaire to student organizations, especially English-speaking ones, as well as to religious groups so that they could distribute them within their network. As the population grows as the questionnaire is distributed, this method is called in scientific jargon “snowball sampling”.
It must be said bluntly: such a study can have no claim to representativeness. We were looking for students who had a lot to say against the law and we found some.
The authors can always defend themselves by saying that the limits of their study are written in black and white and that they cannot be accused of lacking transparency. But we cannot trivialize such gaps, especially when it comes to such a sensitive subject.
How can three university researchers pilot such an incomplete study? It is certain that these professors knew full well that their study was going to be relayed by the major media.
There is in particular CTV which headlined ” More than half of Quebec law and education students in survey say Bill 21 is making them want to leave the province2 “. Considering that the survey is not representative of students in these fields, there is no doubt that this title misleads the reader.
It was better before ?
The study’s sampling method is highly dubious, but where the study takes an unforgivable step is when it claims to analyze the effects of Bill 21 on discrimination.
The problem is that in no way do the researchers show how the situation has changed, they are content to collect testimonies from students who say they have experienced more discrimination since the adoption of the law in 2019.
We can read the testimony of an Arab man who says he suffers the suspicious gaze of school secretaries. Elsewhere, a Jewish student is reported to have experienced anti-Semitic attacks and a female student was once yelled at to “go back to her country”.
It is certain that reading such comments offends our feelings and all will agree to condemn them. On the other hand, the honest reader finds it difficult to understand how these behaviors are attributable to Bill 21. Could it be that the respondents to the survey interpret their experience in the light of Bill 21, but that it has absolutely no no connection with the behaviors suffered?
We must also compare these data with those from Statistics Canada which show that hate crimes in Quebec for religious reasons have decreased by 36% since 2018.3. Such a statistic does not say everything about discrimination, but it nevertheless makes it possible to qualify the scope of the testimonies collected.
A conclusion in mind
In carrying out this study, did the authors have a conclusion in mind and sought evidence based on it? We can rightly ask the question. However, such a study and its dissemination in the media only serve to stir up divisions in the population without seeking any possible solutions.