How the Survey was Conducted

 

How the Survey was Conducted


Research approach. The study employed established survey research methods to capture and document the lived experience of individuals recruited from the Black population across the GTA, through in-depth, in-person interviews. This type of in-depth interviewing made it possible to cover a broad range of topics and issues, well beyond what is normally covered in public opinion or social surveys. The approach is based on a previous Environics Institute study conducted in 2008-09 with the urban Aboriginal population in the country’s largest 11 cities. (see Urban Aboriginal Peoples Study).

Survey questionnaire. The questionnaire was developed through an extensive process, guided by the Phase 1 input, an extensive literature review, and direction provided by the Research Advisory Group. The survey included both structured questions (e.g., with response scales) and open-ended questions (to capture qualitative information). Prior to being finalized, the survey was pilot-tested twice with small numbers of individuals from the Black community to evaluate the questions and the overall interview experience. The final version was vetted and approved by the research ethics process at Ryerson University, as well as by the Durham District School Board and Lakeridge Health (two of the locations at which survey participants were recruited).

Sample design and participant recruitment. The target population for this study was defined as individuals 16 years and older living in the Greater Toronto area who self-identify as Black or of African heritage. The study was designed to provide representation of this population to the extent possible across a number of relevant characteristics, including location of residence (region and local municipality), age, gender, household income and ethnic background.

The sampling plan was based on the development of a population profile which identified how the Black population is distributed across each of the characteristics described above (for instance, how many males aged 16 to 24 of African heritage live in Peel Region with household incomes between $60,000 and $80,000). This information was used to create a sample matrix to determine how many individuals with each unique combination of characteristics should be interviewed for the study in order to result in a sample that approximates the Black population.[1]

Individuals were recruited to participate in the survey through extensive outreach across the GTA to build awareness and interest in the study. This included advertising through conventional and social media, attendance at community events, tapping into existing social and community networks, and reaching out to Black community organizations and Collaborating Partners. Individuals expressing interest in participation were contacted by telephone to screen for eligibility based on the sample matrix for their Region. Those who qualified were then scheduled for an in-person interview at a time and place of their convenience. This important work was conducted by a dedicated team of individuals (mostly youth) from the Black community working on a full or part time basis over the period from February to December 2015.

Study sample. A total of 1,504 interviews were completed with individuals across the GTA who self-identified as Black or of African heritage. The following table presents how this sample is distributed across regions of the GTA, in comparison with the GTA Black population.

Table 2 – Distribution of Study Sample across GTA Regions
Region Black population* Sample Distribution
City of Toronto 218,160   (53%) 734    (49%)
Peel Region 116,265   (28%) 373    (25%)
Durham Region 41,890    (10%) 189    (13%)
York Region 25,870      (6%) 175    (12%)
Halton Region 10,970      (3%) 33      (2%)
Total 413,155 (100%) 1,504 (100%)

*Source:  Statistics Canada: 2011 National Household Survey

The sampling approach was successful in completing interviews with individuals in every combination of demographic characteristics included in the sample profile for each GTA region. In some cases, however, several groups are not represented in the same proportion as in the population; in particular the sample under represents those who have no more than a high school education, those 55 years of age or older, and males (despite considerable efforts to recruit more individuals with these characteristics). The following table presents a profile of the study sample across a number of demographic and personal characteristics.

Table 2 – Distribution of Study Sample across demographic groups
  GTA Black population % Study Sample
(before weighting)
%
Gender Female 56% 64%
Male 44% 36%
Age 16 – 24 11% 15%
25 – 34 16% 16%
35 – 44 24% 27%
45 – 54 19% 20%
55+ 30% 23%
Ethnicity Caribbean 55% 64%
African 31% 19%
Caribbean + African 4% 3%
 Other 9% 13%
Household income < 20K 11% 15%
20K – 40K 16% 16%
40K – 70K 24% 27%
70K – 100K 19% 20%
100K plus 30% 22%
Education level < HS 17% 4%
HS Grad 29% 18%
Trade / Community 33% 25%
University Grad 17% 36%
Post-Graduate 4% 15%
Sexual orientation Heterosexual NA 94%
LGBTQ/other NA 6%

The final data set used for the analysis and reporting were weighted to match the composition of the Black population across the demographic characteristics outlined above, so that each group is represented in the total sample in proportion to its actual size (including those under-represented in the sample).

Because the sampling approach was not based on probability sampling (i.e. each individual in the population having an equal chance of being selected) the final sample cannot be considered statistically representative of the GTA Black population, and caution should be used in generalizing the survey results to this population. However, the study included participation from across most socio-demographic segments of the GTA Black population, and is large to provide for valid insights for many of these groups within this population. Further details on the research methodology are available in a separate report.

Limitations of the research. This type of community-based research presents inherent challenges with respect to surveying a highly diverse population, and completing interviews with a sample that approximates this population across important characteristics such as gender, ethnic background, age, sexuality, income and education. Some groups within the population were especially difficult to recruit for the study, and special effort was required to ensure they were represented in the final sample. Added to this was the challenge of capturing participants’ lived experience through a lengthy interview involving both closed and open-ended questions that asked much of both participants and interviewers.

Central to the study’s success was the recruitment and training of a dedicated team of individuals from the community (mostly youth) to coordinate and handle all aspects of the process, and building this team from the ground up took time. For these reasons, completion of the survey fieldwork took considerably longer than anticipated and entailed higher costs, and resulted in the completion of fewer interviews than the original target of 2,000.

[1] The population profiles were developed using the 2011 National Household Survey conducted by Statistics Canada., which is the most current population statistics available (the relevant 2016 census data will not be available until later in 2017).  Because the 2011 NHS was a voluntary survey, the data are not as complete nor as reliable in comparison with other years in which a true census was conducted.

How the Survey was Conducted


Research approach. The study employed established survey research methods to capture and document the lived experience of individuals recruited from the Black population across the GTA, through in-depth, in-person interviews. This type of in-depth interviewing made it possible to cover a broad range of topics and issues, well beyond what is normally covered in public opinion or social surveys. The approach is based on a previous Environics Institute study conducted in 2008-09 with the urban Aboriginal population in the country’s largest 11 cities. (see Urban Aboriginal Peoples Study).

Survey questionnaire. The questionnaire was developed through an extensive process, guided by the Phase 1 input, an extensive literature review, and direction provided by the Research Advisory Group. The survey included both structured questions (e.g., with response scales) and open-ended questions (to capture qualitative information). Prior to being finalized, the survey was pilot-tested twice with small numbers of individuals from the Black community to evaluate the questions and the overall interview experience. The final version was vetted and approved by the research ethics process at Ryerson University, as well as by the Durham District School Board and Lakeridge Health (two of the locations at which survey participants were recruited).

Sample design and participant recruitment. The target population for this study was defined as individuals 16 years and older living in the Greater Toronto area who self-identify as Black or of African heritage. The study was designed to provide representation of this population to the extent possible across a number of relevant characteristics, including location of residence (region and local municipality), age, gender, household income and ethnic background.

The sampling plan was based on the development of a population profile which identified how the Black population is distributed across each of the characteristics described above (for instance, how many males aged 16 to 24 of African heritage live in Peel Region with household incomes between $60,000 and $80,000). This information was used to create a sample matrix to determine how many individuals with each unique combination of characteristics should be interviewed for the study in order to result in a sample that approximates the Black population.[1]

Individuals were recruited to participate in the survey through extensive outreach across the GTA to build awareness and interest in the study. This included advertising through conventional and social media, attendance at community events, tapping into existing social and community networks, and reaching out to Black community organizations and Collaborating Partners. Individuals expressing interest in participation were contacted by telephone to screen for eligibility based on the sample matrix for their Region. Those who qualified were then scheduled for an in-person interview at a time and place of their convenience. This important work was conducted by a dedicated team of individuals (mostly youth) from the Black community working on a full or part time basis over the period from February to December 2015.

Study sample. A total of 1,504 interviews were completed with individuals across the GTA who self-identified as Black or of African heritage. The following table presents how this sample is distributed across regions of the GTA, in comparison with the GTA Black population.

Table 2 – Distribution of Study Sample across GTA Regions
Region Black population* Sample Distribution
City of Toronto 218,160   (53%) 734    (49%)
Peel Region 116,265   (28%) 373    (25%)
Durham Region 41,890    (10%) 189    (13%)
York Region 25,870      (6%) 175    (12%)
Halton Region 10,970      (3%) 33      (2%)
Total 413,155 (100%) 1,504 (100%)

*Source:  Statistics Canada: 2011 National Household Survey

The sampling approach was successful in completing interviews with individuals in every combination of demographic characteristics included in the sample profile for each GTA region. In some cases, however, several groups are not represented in the same proportion as in the population; in particular the sample under represents those who have no more than a high school education, those 55 years of age or older, and males (despite considerable efforts to recruit more individuals with these characteristics). The following table presents a profile of the study sample across a number of demographic and personal characteristics.

Table 2 – Distribution of Study Sample across demographic groups
  GTA Black population % Study Sample
(before weighting)
%
Gender Female 56% 64%
Male 44% 36%
Age 16 – 24 11% 15%
25 – 34 16% 16%
35 – 44 24% 27%
45 – 54 19% 20%
55+ 30% 23%
Ethnicity Caribbean 55% 64%
African 31% 19%
Caribbean + African 4% 3%
 Other 9% 13%
Household income < 20K 11% 15%
20K – 40K 16% 16%
40K – 70K 24% 27%
70K – 100K 19% 20%
100K plus 30% 22%
Education level < HS 17% 4%
HS Grad 29% 18%
Trade / Community 33% 25%
University Grad 17% 36%
Post-Graduate 4% 15%
Sexual orientation Heterosexual NA 94%
LGBTQ/other NA 6%

The final data set used for the analysis and reporting were weighted to match the composition of the Black population across the demographic characteristics outlined above, so that each group is represented in the total sample in proportion to its actual size (including those under-represented in the sample).

Because the sampling approach was not based on probability sampling (i.e. each individual in the population having an equal chance of being selected) the final sample cannot be considered statistically representative of the GTA Black population, and caution should be used in generalizing the survey results to this population. However, the study included participation from across most socio-demographic segments of the GTA Black population, and is large to provide for valid insights for many of these groups within this population. Further details on the research methodology are available in a separate report.

Limitations of the research. This type of community-based research presents inherent challenges with respect to surveying a highly diverse population, and completing interviews with a sample that approximates this population across important characteristics such as gender, ethnic background, age, sexuality, income and education. Some groups within the population were especially difficult to recruit for the study, and special effort was required to ensure they were represented in the final sample. Added to this was the challenge of capturing participants’ lived experience through a lengthy interview involving both closed and open-ended questions that asked much of both participants and interviewers.

Central to the study’s success was the recruitment and training of a dedicated team of individuals from the community (mostly youth) to coordinate and handle all aspects of the process, and building this team from the ground up took time. For these reasons, completion of the survey fieldwork took considerably longer than anticipated and entailed higher costs, and resulted in the completion of fewer interviews than the original target of 2,000.

[1] The population profiles were developed using the 2011 National Household Survey conducted by Statistics Canada., which is the most current population statistics available (the relevant 2016 census data will not be available until later in 2017).  Because the 2011 NHS was a voluntary survey, the data are not as complete nor as reliable in comparison with other years in which a true census was conducted.