School-Based Evidence For The Validity Of Curriculum-Based Measurement Norms For French Immersion Students In School District #57 Sheralee Thibault B. G. S., Simon Fraser University, 1996 Project Submitted In Partial Fulfillment Of The Requirements For The Degree Of Master Of Education m Educational Counselling · University OfNorthern British Columbia August, 2006 © Sheralee Thibault UNI'VEWW !OR"rnE'RN BRITISH ",..., olll I Llu rlY Prine George, B.C. 111 ABSTRACT There is a vast amount of research that supports the use of Curriculum-Based Measurement as a valid assessment tool in English. In 1995-1996, School District #57 completed a norming project for CBM probes in English. In 2000 Fewster completed a validity study which looked at CBM scores to predict academic performance in Junior Secondary School. In 2002 St-Pierre developed CBM probes for French Immersion. This study utilizes the work of Fewster and St-Pierre. I( examines the validity of the French Immersion CBM probes. Correlations confirmed the reliability of CBM measures and letter grades. Correlation and regression analysis confirmed the predictive ability of grade 7 Words Read Correctly probes to letter grades earned in grade 8 and 9 language based courses of Language Arts French, Social Studies and English. Findings support that the existing body of research for English CBM is applicable to French Immersion CBM. IV TABLE OF CONTENTS Abstract 111 Table of Contents IV List ofTables v List of Figures Vll Acknowledgements Vlll Introduction 1 Method Subjects CBMNorming Letter Grades Materials CBM Measures School Letter Grades for Grades Four to Seven Procedure Data Collection French Immersion CBM Scores School Letter Grades Cleaning and Matching Ethics 6 6 6 7 7 7 9 Results Preliminary Analysis Reliability of CBM Probes Reliability of Letter Grades Validity of Spring CBM Scores to Letter Grades Correlations of CBM Scores to Letter Grades Regression Analysis 9 9 9 10 10 11 13 13 13 14 20 20 27 Discussion Overview Summary and Conclusions Limitations Implications 29 29 29 References 32 Appendices 35 31 31 v Appendix A: School District #57 Problem-Solving Model 35 Appendix B: Approval School District #57 37 Appendix C: Approval UNBC Ethics 39 Vl LIST OF TABLES Table 1 Reliability Test-retest from SD#57 November 1996 3 Table 2 Pearson Correlations between CBM Scores and Secondary School Marks 4 Table 3 Reliability Test-retest for French Immersion St-Pierre 2002 5 Table 4 Descriptive Statistics for CBM 11 Table 5 Correlation Matrix for CBM 14 Table 6 Descriptive Statistics for Letter Grades 15 Table 7 Correlation Matrix for Letter Grades 17 Table 8 Descriptive Statistics for Students Who Complete CBM Assessment in Grade 4 20 Table 9 Descriptive Statistics for Students Who Complete CBM Assessment in Grade 5 21 Table 10 Descriptive Statistics for Students Who Complete CBM Assessment in Grade 6 22 Table 11 Descriptive Statistics for Students Who Complete CBM Assessment in Grade 7 23 Table 12 Correlation Matrix for Grade 4 Spring CBM to Letter Grades 4-7 24 Table 13 Correlation Matrix for Grade 5 Spring CBM to Letter Grades 5-7 25 Table 14 Correlation Matrix for Grade 6 Spring CBM to Letter Grades 6-8 25 Table 15 Correlation Matrix for Grade 7 Spring CBM to Letter Grades 7-9 27 Table 16 Predictive Ability of Grade 8 & 9 Letter Grades with Spring CBM WRC 28 Vll LIST OF FIGURES Figure 1 . Grade 4 CBM WSC correlated to Grade 4 Language Arts French student letter grades 24 -Vlll ACKNOWLEDGEMENTS There are several people I need to acknowledge for their part in the production of this project: Bridget Moran and my mother Phyllis Parker for being my inspiration; Saima Fewster and Sylvie St-Pierre for their earlier work; Peter MacMillan for his insistence; Brian for his encouragement and my boys for their patience. 1 INTRODUCTION Curriculum Based Measurement (CBM) is one strand of Curriculum Based Assessment . Deno, Mirkin & Chiang (1982) originally developed CBM. It presents an alternative to standardized testing that utilizes text from the curriculum but, as with standardized testing is administered and scored based on local norming. The summarized results can be the basis for making educational decisions. CBM assessment has been determined to be reliable and valid (Deno, Mirko & Chiang, 1982; Elliot & Fuchs, 1997; Espin & Deno, 1993; Fuchs & Deno, 1992; Hintze & Shapiro, Conte & Basile, 1997; Marston, 1989; Shinn & Habedank, 1992). This form of assessment is simple and efficient to administer. The writing probe requires three minutes and is scored on total words written (TWW) and total words spelled correctly (WSC). The reading probe requires one minute to administer and is scored on total words read correctly (WRC). The process is simple to understand and is very inexpensive. The material is deemed to be relevant, the testing can be repeated, removes ethnic bias, can be charted over time and provides instructional feedback for the classroom teacher. Curriculum Based Measurement was initially designed to monitor individual student progress in a quick and simple framework. With local norming it can now be used to determine the discrepancy between an individual and their peers. This helps to identify students who require additional support beyond the regular classroom. Discrepancy between an individual and his or her peers is one component of the Problem Solving Model used in School District #57. The Problem Solving Model is the process that determines if additional support is required for a student. Step one is to identify the problem. Is the problem significant 2 enough to give the student additional support? Step two is to analyze the problem. During this step the length and the severity of the problem are investigated. Step three is to develop an intervention plan. Objectives must be established that are specific and observable. Step four implements the plan. Data is collected, graphed and recorded to determine if the plan is effective. The fifth step is to evaluate the plan to determine if the intervention was effective and determine if the student still requires support. CBM is one measurement tool used in the Problem Solving Mo-del. CBM assessment removes some of the pressure placed on School Psychologists because CBMs can be administered to an individual, at the school level and across the district. In 1995-1996, School District #57 proceeded with a norming project for CBM probes. A group of 639 students, approximately 20% of the district population, was used in the norming project. It was a stratified random sample with proportional representation from each of the elementary schools. Level one and two students were excluded from the project. These were students who qualified as English as Second Language, students with intellectual disabilities, students with hearing/visual disabilities, autistic students and students with multiple disabilities. The norming project was completed over three probes (fall, winter and spring) during the school year, fall, winter and spring. Table 1 shows the reliability scores from the norming project. Fewster (2000) completed a validity study which looked at using CBMs to predict academic performance in Junior Secondary Schools. In her study she was able to match 465 students from the original norming project total of 639. Fewster collected junior secondary marks from the students in the 1998-1999 school year and tracked them over three years. The goal of her work was to determine the strength of the relationship 3 between elementary CBM scores and junior secondary marks in English and Socials. Fewster chose English and Socials as relevant criteria because they demonstrate proficiency of reading and writing. A second goal of her work was to use CBM scores as one tool for screening into secondary school programs. The programs she investigated included remedial support, alternate education, general course work and honours programs. The goal was to confirm the ability of the CBM score to differentiate student's academic achievement and to confirm previous work on screening for special education. Table 1 Reliability Test-retest from SD#57 November 1996 Grade 2 3 4 5 6 7 Oct-Jan .55 .63 .67 .59 .64 .60 TWW Jan-A2r .68 .67 .69 .62 .68 .64 Oct-Apr .48 .53 .61 .57 .62 .61 Oct-Jan .85 .89 .81 .85 .88 .86 wsc Jan-Apr .84 .89 .80 .84 .85 .86 Oct-Apr .81 .86 .77 .83 .81 .86 The results of the study indicated that there was a significant correlation and medium effect size between Grade 6 and 7 CBM scores and academic achievement in English and Socials Studies grades 8 to 10. Correlations were stronger for reading fluency. The correlations and effect size of the predictions were stronger for early junior secondary (Grade 8 and 9) letter grades than those in later junior secondary (Grade 10). The study also argued that although the study confirmed the predictive ability to help screen into special education, CBM scores should only be one part of the assessment for screening and placement. The correlations are shown in Table 2. 4 Table 2 Pearson Correlations between CBM Scores and Secondary School Marks Secondary Course English 8 English 9 English 10 Social Studies 8 Social Studies 9 Social Studies 10 wsc .34** .29** .28** .24** .16* .21 * WRC .46** .38** .32** .39** .36** .30** St-Pierre (2002) developed CBM probes for written expression and reading fluency in French Immersion. St-Pierre developed the administration procedures and scoring rules. As part of the study she created local norms. The study also included verification of the reliability and stability of the probes. This study was a replication of the English CBM written expression and reading fluency and done by School District #57 in 1996. The results of the study are illustrated in Table 3. St-Pierre's study indicated that there was no significant difference in difficulty between the three reading probes. There was a significant difference with some writing probes but there was one group of students that was stronger in written expression. The study concluded that the probes for written expression and reading fluency were reliable. It demonstrated that there were similar reliability results to the previous study in English (1996a). French Immersion teachers pushed to have the CBM norms completed. The French Immersion program is a smaller program in the District that often feels left out of District supports and initiatives. This study extends the work ofFewster (2000) and StPierre (2002) by repeating the validity and predictive ability study ofFewster (2000) on the French Immersion norming project of St-Pierre (2002). 5 Table3 Reliability Test-retest for French Immersion St-Pierre 2002 Grade 2 .... .) 4 5 6 7 Oct-Jan .57 .73 .42 .32 .66 .49 wsc Jan-AQr .71 .75 .45 .52 .77 .84 Oct-AQr .48 .72 .30 .60 .65 .56 Oct-Jan .77 .79 .74 .87 .76 .68 WRC Jan-AJ2r .78 .80 .73 .87 .69 .81 Oct-AJ2r .77 .70 .80 .86 .79 .74 Research Questions What is the reliability of French Immersion written expression and reading fluency CBMs? How does the reliability of the French Immersion CBMs compare to English CBMs? What is the strength of the relationship between written expression and reading fluency French Immersion CBMs to student letter grades in core courses? How does the strength of the relationship for French Immersion CBMs to student letter grades for language based courses compare to English? Are French Immersion CBMs valid predictors of future academic achievement in language based courses? How does the validity of French Immersion CBMs as predictors of future academic achievement compare to English? 6 METHOD The current study examined the relationship between French Immersion CBM norms for written expression and oral reading fluency to letter grades received in current years and the predictive reliability of the CBM norms for future academic achievement. Collecting a relevant amount of information in a short period of time was critical for second language assessment. The predictive value of the instrument is key. There is a need for a "placement test to measure abilities which are common to these courses and, consequently, that show some correlation with the assessments which are currently done in the classroom" (Laurier, 1999). Subjects French Immersion is a program offered by School District #57 for students from Kindergarten to Grade 12. Most students in the program have English as the primary language at home. From the day they start school, students are immersed into the French language. English instruction begins in Grade 3 or 4. Three elementary schools, one middle school and one secondary school offer French Immersion. Two elementary schools offer French Immersion from kindergarten through to Grade 7, one elementary school offers a kindergarten through Grade 5 program; these students then continue on to the middle school for Grade 6 and 7 French Immersion. There is one secondary school for French Immersion students in Grades 8 to 12. CBM Norming Project All French Immersion students in Grades 1 to 7 were tested in the CBM norming project conducted in 2001-2002 (St-Pierre, 2002). Tests for written expression and reading fluency were administered three times during the school year. Grade one students 7 were not tested in the first two assessments. St-Pierre's study included between 240 and 321 students depending upon the testing period. Letter Grades The population used for the present study is the Grade 2 to 7 students from the 2001-2002 school year. Approximately 262 students were matched from the original norming study (St-Pierre, 2002). The difference in the number is attributed to three reasons : a) difficulty in matching school student numbers to Provincial Education Numbers (PEN), b) students leaving the French Immersion program and moving back into the English program and c) students leaving the district. French Immersion students in Grade 4 to 7 were used to examine the reliability and validity of CBMs and letter grades earned in the same school year. A second part this research examined the ability for CBM scores to predict future school achievement. For this part of the research all students from grade 2 to 7 were used. School letter grades were collected for two years following the CBM testing. Materials CBM Measures Written Expression norrning probes in St-Pierre (2002) used the translation of sentence starters from English CBM probes. The original CBM norming project completed in School District #57 (School District #57, 1996b) used the written expression criteria established by Tilly and Carlson (1992). The test protocols were established in consultation with a French Immersion Kindergarten to Grade 5 teachers and a secondary French Immersion Learning Assistance teacher. Rules were based on the CBA Institute, University of Oregon (Baker, Collins & Goodwin, 1992). For French 8 Immersion an additional rule was to receive a full score, students needed to use the correct accents. The criteria for administering the written expression probes followed a set of detailed instructions. Different probes were used in different schools with the starter probes being different in each of the three CBM testing periods throughout the year. In the norming study completed by St-Pierre (2002), reading fluency probes were developed using grade level texts. Three individual probes were created. The probes were expected to be equivalent in difficulty (Marsten & Deno, 1982). English norming done in School District #57 in 1995 used Fry's Readability Graph. Unfortunately this tool was not available for French Immersion. St-Pierre created tables of readability based on work done in the Abbotsford B.C., School District. St-Pierre also used a second method to determine readability. She piloted the tests with six to seven students per grade. Students who were judged to be in the average ability range were chosen. These students then attempted the probes. If students struggled with more than 20 words of the first 100, the passage was considered for a higher level. As recommended by Shinn (1989) and Tilly and Carlson (1992) poems and plays were not used. Administration criteria of the Reading probes were translated from the CBA Training Institute University of Oregon (Baker, Collins & Goodwin, 1992). St-Pierre met with the Learning Assistance teachers from each French Immersion school for training on the administration and scoring of the CBMs. Each school started with a specific reading probe. Tracking ensured that during the three norming periods, each student would be administered a different reading probe. The reading probes were recorded on tape so that 9 the Learning Assistance teachers could listen to the one minute reading as often as necessary. School Letter Grades for Grades Four to Seven Letter grades (marks) were collected for this study for the 2001-2002 school year. Letter grades from the 2002-2003 and 2003-2004 school years were also collected. Student achievement is reported in letter grades at the intermediate level. Secondary student achievement is sometimes reported as botli letter grades and percentages. Letter grades were always available and therefore chosen as the measure for secondary student achievement. Letter grades were converted to numerical values. The letter grades A = 5, B = 4, C+ = 3, C = 2 and C- = 1. P (pass) was assigned a C-. F (fail) and WD (withdrawn) were assigned 0. Procedure Data Collection French Immersion CBM Scores CBM scores were retrieved from tables created by St-Pierre for her study. Data was exported into an Excel spreadsheet. St-Pierre's study had collected CBM scores by school student number. During the creation of the excel spreadsheet, student numbers were matched to Provincial Education Numbers (PEN). A total of 41 students CBM scores were not matched to PEN numbers. Recall there are three reason for not matching: errors in matching school numbers to PEN numbers, students leaving the French Immersion program and students leaving the district. 10 School Letter Grades Administrators from each of the schools that had a French Immersion program were contacted and asked to forward letter grades for the reporting periods of the 2001-2002 through 2003-2004 school years for students in Grades 4 to 7. Letter grades are not issued until Grade 4. This data was exported from a computer program called Turbo School. In the case of one elementary school, data was lost from the 2001-2002 school year. Most of this data was recovered from Permanent Report Cards. The data were received for Language Arts (French), Social Studies, Science, Math and English. Provincial Education Numbers were used to match the student's CBM scores from the 2001-2002 school year to their letter grades. The students' letter grades were first recorded by letter grade and were later converted to numerical values. Cleaning and Matching In St-Pierre (2002) depending on the testing period, between 240 and 321 students were tested during the project. The current study matched student data from St-Pierre's project by matching school student numbers to PEN numbers. From St-Pierre's study 262 sets of student data were matched. Then letter grades were matched for between 208 and 212 students. The difference in the number of students matched is attributed to errors in matching student numbers and PEN numbers, students leaving the French Immersion program and students moving out of district. Descriptive statistics were calculated for the matched (CBM scores to letter grades) and not matched (CBM scores only) groups. Table 4 shows the number of students, the mean, standard deviation, t test and significance (p). The mean score for each of the CBM tests is lower for the not matched students than those students matched. 11 Levene's test was used to examine sample variances prior to the t test conducted to determine the equality of variances. All results were greater than +2.00 and -2.00. There was a significant difference between the matched and not matched students. Matched students outperformed the not matched students in every instance. Students that we were not match could be the more transient students, perhaps less prepared or less able. The impact of this will be discussed later. Table 4 DescriQtive Statistics for CBM CBM FTWW matched not matched FWSC matched not matched FWRC matched not matched WTWW matched not matched wwsc matched not matched WWRC matched not matched STWW matched not matched swsc p(2 tail) n Mean SD 209 36 32.82 23.78 17.45 17.66 -2.87 .01 209 35 28.52 20.54 16.74 16.93 -2.61 .01 212 37 68.25 42.97 26.39 28.44 -5.31 .00 211 36 38.59 30.31 18.84 18.24 -2.45 .02 211 36 33.86 25.03 18.35 17.01 -2.70 .01 205 36 75.07 51.97 26.50 24.37 -4.88 .00 208 32 39.84 30.94 17.71 19.18 -2.62 .01 matched 209 35.34 17.21 -2.66 .01 not matched 32 26.59 17.99 SWRC matched 209 78.68 25.01 -4.25 .00 not matched 34 58.74 27.67 Note. The abbreviation FTWW = fall total words written; WTWW = winter total words written; STWW = spring total words written. 12 Ethics An overview of the research proposal was presented to Bonnie Chappell, Director of School Services for School District #57. Approval was granted and letters were sent to all principals to introduce the project and welcome questions. The study is considered a research project for School District #57. The proposal was approved by the UNBC ethics committee. Dr. Peter MacMillan has a secure database that identifies students in the CBM project. A file was created with Grade 2 to 7 students who were in the French Immersion program during 2001-02. Letter grades for these students were collected. Once the school grades and CBM letter grades were matched, all identifying factors were removed. During the study there was no contact with students, only CBM scores and grades earned. Because students were not contacted, parental consent was not required. 13 RESULTS In the following chapter the results of analysis are presented. In the preliminary analysis the reliability of the French Immersion CBM probes and the reliability of the French Immersion letter grades were examined. In the main analysis correlation coefficients of Spring CBM scores and letter grades earned in the concurrent year and future years were examined. Regression analysis was completed using the Spring CBM score for reading fluency to predict letter grades in- Grade 8 and 9 language-based courses. Preliminary Analysis Reliability of CBM Probes Correlations coefficients were calculated for CBM tests conducted in the fall with winter and spring tests. Results are shown in Table 5. The table examines correlations with TWW for all three terms. The fall score is related to the winter with a correlation of .80 and to the spring score of .81. Applying Cohen's effect size, effect sizes were in the large range. These results support the reliability of the tests between terms. The test included correlations between CBM scores for TWW, WSC and WRC. In analysis of correlations between TWW and WSC, correlations were significant within terms at FTWW to FWSC .97, WTWW to WWSC .99 and STWW to SWSC .98. As a result, during future analysis WSC will be used and TWW will be dropped. This is supported by Fewster and MacMillan (2002) who dropped TWW because WSC is a little more complex and more accepted by teachers. Examining WSC across three terms FWSC correlations were significant at .80 to WWSC and .82 to SWSC. Looking at WRC across three terms FWRC had significant 14 correlations to WWRC at .84 and spring at .78. This establishes the reliability of the test overtime. The last correlations to be examined are the relationship of WSC to WRC within each term. Fall WSC has a significant correlation to fall WRC at .50. Winter WSC was significant at .49 and Spring WSC correlated at .36. Using Cohen's effect size, all correlations .35 to .97 are significant at medium to large level. These results support the validity of the CBM scores. Table 5 Correlation Matrix for CBM FTWW FWSC FWRC WTWW FWSC .97** FWRC WTWW .46** .80** .50** .80** .49** wwsc wwsc WWRC STWW swsc .80** .80** .52** .99** WWRC STWW .42** .81 ** .81 ** .44** .84** .45** .46** .87** .49** .87** .44** swsc .81 ** .82** .47** .87** .88** .45** .98** SWRC .35** .39** .78** .43** .45** .81 ** .34** .36** SWRC Note. All correlations marked with ** are significant at p < .0 1. Correlations marked with * are significant are significant at p < .05. All correlations were significant atp < .01. Reliability of Letter Grades The descriptive statistics for letter grades of students in the project are shown in Table 6. Subjects included French Language Arts, Social Studies, Science Math and English. All ofthe core subjects are taught in French with the exception of English. Language Arts (French) refers to the language arts program which includes many of the reading and writing activities. In the French Immersion program, the primary language used is French. For most students in the program, English is their first language. 15 Table 6 DescriQtive Statistics for Letter Grades Subject Gr4 LAFr ss Sc Ma En Gr 5 LA Fr ss Sc Ma En Gr 6 LA Fr ss Sc Ma En Gr7 LA Fr ss Sc Ma En Gr 8 LA Fr ss Sc Ma En Gr9 LA Fr ss Sc Ma En n 107 107 107 107 107 105 104 104 104 104 89 89 89 89 89 119 119 119 119 119 68 68 68 68 68 39 38 38 38 38 Mean 3.86 4.21 3.88 4.20 4.09 3.92 4.25 4.16 4.01 4.17 3.66 3.83 3.70 3.85 3.74 4.03 4.15 3.99 3.95 4.07 3.50 3.90 3.59 3.81 4.01 3.51 3.37 3.68 3.47 3.76 SE 0.23 0.23 0.23 0.23 0.23 0.24 0.24 0.24 0.24 0.24 0.26 0.26 0.26 0.26 0.26 0.22 0.22 0.22 0.22 0.22 0.29 0.29 0.29 0.29 0.29 0.38 0.38 0.38 0.38 0.38 SD 0.95 0.91 1.14 0.86 0.90 0.99 0.91 0.83 0.95 1.01 1.12 1.05 1.30 1.16 1.26 1.05 0.91 1.11 1.18 1.04 1.42 1.30 1.43 1.39 1.09 1.45 1.51 1.41 1.59 1.17 There isn't much variation from one subject to the next. In Grades 4 through 7 the same students are with the same teacher across all subjects. By Grade 8, different subjects are often taught by different teachers. The students who are in the program tend to receive a lot of parental support and are motivated to be successful. There is a natural attrition out of the program by those students who are not being successful. They often 16 leave the French program and return to the English track. The number of students in each subject ranges between 38 and 119. Standard errors of the means (SE), increases over time from .23 in Grade 4 to .38 in Grade 9. This could be a result of the attrition. Correlation coefficients between letter grades were calculated. These are shown in Table 7. Sc Ma L_§l1 ss Gr6LAFr Sc Ma En ss Gr5LAFr Sc Ma En ss GR4LAFr Sc .68** .53** Sc .77** .66** Gr5SS .69** Gr6SS .69** Ma .68** .68** .70** Ma .65** .42** .51** Sc Ma .62** .59** .52** .61 ** .44** Gr4SS .56** Correlation Matrix for Letter Grades Table 7 En .71 ** .68** .76** .68** Gr7LAFr .70** .52** .54** .62** .56** En Gr6LAFr .78** .60** .69** .55** .54** .57** .67** .53** .51** .67** .41 ** .48** .57** .51** ss .45** .39** .32* .43** .48** ss Sc .73** .60** .61 ** .72** .75** Sc .65** .57** .61 ** .69** .65** Sc En Gr5LAFr ss .61 ** .67** .62** .55** .46** .36** .49** .42** .57** .65** .63** .60** .46** .49** .56** .47** .53** .48** .52** Ma .58** .41 ** .51** .74** .41 ** Ma .66** .55** .56** .73** .65** En .72** .46** .55** .52** .50** En .72** .64** .55** .69** .69** Ma En .50** .67** .40** .50** .37** .54** .65** .60** .44** .58** Gr8LAFr .72** .39 .71 ** .73** .47 Gr7LAFr .55** .59** .47* .58** .51** Gr6LAFr .48** .47** .38* .42** .42** .93** .69** .85** .70** .67* ss .49** .49** .45* .40* .. 28 ss .39* .45** .31 .31 .52** ss Sc .92** .62* .87** .62* .66* Ma .76** .63* .77** .84** .64* -- ----- --- Sc Ma .79** .56** .69** .59** .78** .52** .79** .66** .59** .46** Sc Ma .43** .43** .40** .36* .48** .39* .46** .58** .60** .43** ' En .87** .63* .80** .76** I .73** En .58** .40* .54** .39* .29 En .49** .44** .56** .45** .58** I7 Ma En Sc ss GRlOLAFr Ma En Sc ss Gr9LAFr Ma En Sc ss Gr8LAFr Ma L_§n___ . Sc ss GR7LAFr Sc .68** .67** Gr8LAFr .65** .79** .65** .72** .63** .65** .62** .81 ** .81 ** En Sc .75** .54** .83** .79** .57** Ma .58** .76** .57** .68** En ss ss .65** .81 ** .73** .65** .76** .86** .62** .55** .65** .67** .70** · - - - - - - --- - - - - - - - - En Gr9LAFr .81 ** .80** .79** .77** .73** .82** .75** .85** .82** .70** .82** .83** Ma .75** .74** .68** Ma ---- -- - - - - - - - - - - - - - - - - - - · - - Sc .78** .86** GrlOSS .82** ss .72** Gr7SS .75** Sc .71 ** .83** .78** .76** .79** .85** .88** Sc .68** .65** .67** .76** .74** .74** .71 ** .79** .84** .80** .85** .78** .87** Ma .67** .71 ** .71 ** .78** .80** Ma .56** .75** .80** .59** . .73** . .81 ** • .78** .80** .71 ** En .74** .75** .66** .79** .82** En Gr9LAFr .69** .59** .71 ** .73** .78** ss .62** .47** .46** .65** .70** Sc .69** .58** .56** .76** .76** .61 ** .64** .60** .83** .78** Ma ' .63**! .48** .56** .61 ** .57** En 18 19 Correlation coefficients (r) were significant p < .05. Grade 4 Language Arts (French) has significant correlation coefficients with letter grades earned in all core subjects during the Grade 4 year. Correlations ranged from .56 to .62. The Grade 4 Language Arts (French) letter grade continues to have a high correlation with letter grades earned in Grade 5, .50 to .67 and Grade 6, .39 to .49. The correlation strength does decrease by Grade 6. Correlations within the same year tended to be higher and decreased over time. Over time many factors can effect student achievement. Some of these factors are increased teacher expectations, social pressures, puberty and commitments outside of school such as work or athletics. A higher correlation would be expected between letter grades and spring CBM scores in the same year. Correlation coefficients increase in strength with the current years from Grades 4 to 10. This may be attributed to the natural attrition from the program, parental/teacher support and students increase in academic preparation. Within each grade analysis, correlations dropped in the second and third years of tracking students' letter grades. Using the example of Grade 7, letter grades in Grade 7 core subjects of Language Arts (French), Social Studies, Science, Math and English correlated to themselves between .65- .75, Grade 8 core subjects correlated to Grade 7 at .62- .79 and Grade 9 core subjects correlated to Grade 7 at.61 - .69. In all cases Cohen's (1992) effect size were medium to large. The significance level of the correlations confirms the reliability of teacher-awarded letter grades. 20 Validity of Spring CBM Scores to Letter Grades In the preliminary analysis reliability results supported the CBM scores and letter grades. For the main analysis fall and winter CBM scores were dropped. Spring scores were used because the testing is completed close to the time letter grades are issued. In the main analysis the relationship between Spring CBM scores and letter grades earned in the concurrent year and following years are going to be examined. Correlations of CBM Scores to Letter Grades Table 8 shows the data from students who wrote the Grade 4 CBM had letter grades matched for their Grades 4 to 6 school years. Letter grades were matched for Language Arts (French), Social Studies, Science, Math and English. Students have the same teacher for most core subjects during each of the elementary years. Thirty-three were in Grade 4 during the norming year; this gradually decreased over the next two years to 28 students. Table 8 DescriQtive Statistics for Students Who ComQlete CBM Assessment in Grade 4 Subject Gr4 LA Fr ss Sc Ma En Gr 5 LA Fr ss Sc Ma En Gr 6 LA Fr ss Sc Ma En n 33 33 33 33 33 31 31 31 31 31 28 28 28 28 28 Mean 3.97 4.30 4.03 4.27 3.91 4.13 4.42 4.23 4.29 4.61 3.82 3.86 4.04 4.07 4.18 SE 0.15 0.17 0.22 0.13 0.20 0.14 0.11 0.18 0.12 0.12 0.20 0.23 0.16 0.19 0.15 SD 0.85 0.95 1.26 0.76 1.13 0.76 0.62 1.02 0.64 0.67 1.06 1.21 0.84 0.98 0.77 21 Students who wrote the Grade 5 CBM had letter grades matched for their Grade 5 to 7 school years. Table 9 shows the number of students (range = 24- 29) for whom letter grades were matched at grade 5 through 7 and the mean letter grades for core subjects. Most letter grades range between B- to B+. Standard errors of the means (SE) and standard deviations (SD) remained consistent across all three grades. This indicated consistency in letter grades awarded between courses and among grades. Table 9 Descriptive Statistics for Students Who Complete CBM Assessment in Grade 5 Subject Gr 5 LA Fr ss Sc Ma En Gr6 LAFr ss Sc Ma En Gr7 LAFr ss Sc Ma En n 24 24 24 24 24 29 29 29 29 29 26 26 26 26 26 Mean 3.46 4.42 4.08 3.46 3.67 3.66 3.76 3.59 3.69 3.62 3.73 3.96 3.77 3.50 3.92 SE 0.23 0.16 0.17 0.26 0.24 0.21 0.18 0.27 0.24 0.25 0.19 0.15 0.20 0.23 0.16 SD 1.41 0.78 0.83 1.29 1.12 1.11 0.95 1.43 1.31 1.38 0.96 0.77 1.03 1.18 0.80 Grade 6 students wrote the grade 6 CBM for both writing and reading. Letter grades were matched to their CBM scores in Grade 6 to 8. During Grade 6 and 7, core subjects tend to be taught by the same teacher. When looking at the grade 8 letter grades, students often have several different teachers. Table 10 shows that 19students had matched letter grades in Grade 6. This number increases to 35 in Grade 7 and drops to 28 in Grade 8. The lower number of students matched in the grade 6 year may be linked to 22 one school loosing their letter grades in the 2001-2002 school year. Some of the letter grades were recovered by looking at the permanent record cards at the secondary school. Most letter grades fell within the B- to B+ range. Table 10 Descrigtive Statistics for Students Who Comglete CBM Assessment in Grade 6 Subject Gr6 LAFr ss Sc Ma En Gr 7 LA Fr ss Sc Ma En Gr8 LAFr ss Sc Ma En n 19 19 19 19 19 35 35 35 35 35 28 28 28 28 28 Mean 3.26 3:95 3.37 3.84 3.37 4.31 4.20 4.14 4.20 4.34 3.79 3.82 3.86 3.89 4.18 SE 0.30 0.24 0.34 0.28 0.33 0.17 0.17 0.21 0.21 0.15 0.21 0.26 0.26 0.25 0.17 SD 1.28 1.03 1.50 1.21 1.42 0.99 1.02 1.22 1.23 0.91 1.13 1.39 1.35 1.32 0.91 Grade 7 CBM scores were matched to letter grades received in Grades 7 to 9. Table 11 shows that the number of students with matched letter grades dropping from 47 in grade 7 to 3 8 in grade 9. Mean scores for grade 7 ranged from 3.83 in Language Arts (French) to 4.09 in Social Studies. Grade 8 scores ranged from 3.30 in Language Arts French to 3.95 in Social Studies. The overall drop in marks in the grade 8 year may be attributed to different teachers assigning letter grades and the social issues related to the transition in secondary school. Grade 9 letter grades ranged from 3.47 in Math to 3.76 in English. 23 Table 11 DescriQtive Statistics for Students Who ComQlete CBM Assessment in Grade 7 Subject Gr7 LAFr ss Sc Ma En Gr 8 LA Fr ss Sc Ma En Gr 9 LA Fr ss Sc Ma En n 47 47 47 47 47 40 40 40 40 40 39 38 38 38 38 Mean 3.83 4.09 3.91 4.00 3.89 3.30 3.95 3.40 3.75 3.90 3.51 3.37 3.68 3.47 3.76 SE 0.16 0.14 0.16 0.16 0.16 0.25 0.20 0.23 0.23 0.19 0.23 0.25 0.23 0.26 0.19 SD 1.09 0.93 1.08 1.08 1.09 1.57 1.24 1.46 1.45 1.19 1.45 1.51 1.41 1.59 1.17 Students who wrote the Grade 4 CBM probes for writing and reading had their letter grades compared to the spring CBM probes. Letter grades from Grade 4 to 6 were used. Table 12 shows the Grade 4 CBM for WSC and WRC measured concurrent letter grades in Language Arts (French), Social Studies, Science, Math and English. The letter grades are tracked over 2 more years as the students move through Grade 5 and 6. The student group and core subjects remain consistent but the teachers change each year. 24 Table 12 Correlation Matrix for Grade 4 Spring CBM to Letter Grades 4 - 7 Gr4 LAFr ss Sc Ma En wsc -.58 -.16 -.92 -.11 .19 Gr5 wsc WRC .22 -.82 .23 .05 .17 .19 -.10 .10 -.19 -.01 WRC .41 * .29 .52** .08 .30* Gr6 wsc .22 .20 .34* .14 .34* WRC .39* .14 .43* .23 .19 Note. All correlations marked with** are significant atp < .01. Correlations marked with * are significant are significant at p < .05. A one tailed test was used for the correlations. Concurrent year data for grade 4 CBM to letter grades showed negative correlations for Language Arts (French), Social Studies, Science and Math. A scatter plot was created for Language Arts (French). It determined that the negative correlation existed because students who had not scored well on the CBM tests, still received marks in the B range. 20 40 swsc eo Figure 1. Grade 4 CBM WSC correlated to Grade 4 Language Arts French student letter grades. 25 As students moved from Grade 4 to Grade 5, positive correlations were found. Correlations for WRC were significant with letter grades earned in the Grade 5 subjects of Language Arts (French) (.41), Science (.52) and English (.30). At the grade 6 level WSC correlated with Science (.34) and English (.34). Grade 6 letter grades were correlated with WRC for Language Arts (French) (.39) and Science (.43). Correlations existed for both WSC (.65) and WRC (.68) for English. Cohen's effect size for correlations puts the correlations for WSC and wRC at medium to high. Grade 5 students who wrote the Grade 5 CBM tests were matched in the concurrent year and following two years. The letter grades were tracked over the same core subjects throughout this project. Table 13 Correlation Matrix for Grade 5 Spring CBM to Letter Grades 5 - 7 Gr5 Gr7 wsc WRC wsc WRC WRC .25 .52** .25 .37* .56** .54** ss .29 .36* .12 .30 .26 .19 .31 Sc .29 .33* .31 .27 .24 Ma .14 .44* .12 .32 .2 1 .21 En .21 .56** .17 .62** .68** .40* Note. All correlations marked with ** are significant at p < .0 1. Correlations marked with * are significant are significant at p < .05. LAFr wsc Gr6 At the Grade 5 level, correlations were stronger from the WRC probe. The Spring CBM for reading was a valid predictor of Language Arts (French) and English at the Grade 5 level. It also predicted the English 6, Language Arts (French) 7 and English 7. WSC had significant correlations with Grade 7 subjects of Language Arts (French) and English. The Grade 6 probes were the only time where WSC correlated more highly than WRC. Cohen's effect size puts correlations for WSC and WRC at medium to high. 26 Grade 6 CBM scores were compared to letter grades earned in grades 6 through 8. Table 14 shows which subjects were predicted by the CBM probes. Table 14 Correlation Matrix for Grade 6 Spring CBM to Letter Grades 6- 8 Gr6 Gr7 Gr8 wsc WRC wsc WRC WRC .50* .55** .40* .14 .37* .00 ss .43* .27 .34 .39* .25 .18 .39* Sc .25 .32 .50** .34* .22 Ma .47* .39 .45** .14 .42* .05 En .52* .49** .32* .32* .14 .24 Note. All correlations marked with** are significant at p < .0 1. Correlations marked with * are significant are significant at p < .05. LAFr wsc Grade 6 Spring WSC probes predicted letter grades earned in Grade 6 for Language Arts (French), Math and English. At the Grade 7 level WSC predicted Language (French), Social Studies, Science, Math and English. At the Grade 8 level WSC predicted Language Arts (French), Social Studies, Science and Math. Grade 6 Spring WRC predicted letter grades received in Grade 6 Language Arts (French) and English. At the Grade 7 level WRC predicted the Science and English letter grades. At Grade 8 there was no predictive ability ofWRC. Cohen's effect size for correlations puts correlations outlined above at medium to high. Grade 7 CBM tests for WSC and WRC were compared to letter grades received in Grade 7 through 9 in the core subjects. Table 15 illustrates those correlations. Grade 7 Spring CBM had a strong ability to predict the Grade 7 letter grades in all subjects. The grade 7 CBM Reading probe predicted letter grades in all core subjects for grades 8 and 9. Cohen's effect size for correlations puts correlations at medium to high. 27 Table 15 Correlation Matrix for Grade 7 Spring CBM to Letter Grades 7 - 9 Gr7 wsc Gr8 Gr9 wsc WRC wsc WRC WRC .61 ** .45** .52** .14 LAFr .32* .25 .04 .48** ss .32* .50** .03 .45** .34** .52** .55** Sc .35* .19 .06 .47** .28 .54** .25 Ma .29* .57** .58** .37* .06 .23 En .39** .62** Note. All correlations marked with** are signific~nt atp < .01. Correlations marked with * are significant are significant at p < .05. Regression Analysis Another part of the study was to look at the predictive ability of the Grade 7 CBM scores with letter grades in early secondary. During the analysis of Grade 7 spring CBM and letter grades, WRC correlated more highly with letter grades than with WSC. As a result regression analysis was conducted using the spring CBM WRC to predict letter grades earned in Grade 8 and 9. It was decided to compare WRC with language based courses of Language Arts (French), Social Studies and English. This comparison was also conducted with English CBMs by Fewster and MacMillan (2002). In the initial analysis, multiple regressions were run using all three subjects together. No single subject had a significant predictor likely because of the high correlations between the subjects. Therefore each of the subjects had separate regressions calculated. In Table 16 the subjects of Language Arts French, Social Studies and English were used as the predictor variables and spring CBM WRC was the criterion measure. 28 Table 16 Predictive Ability of Grade 8 & 9 Letter Grades with SQring CBM WRC Gr8 LAFr Gr9 En LAFr ss ss En R .52 .45 .58 .45 .48 .37 R adj. .27 .2 1 .32 .20 .23 .14 SEE 18.72 19.56 17.91 18.39 18.00 19.02 F 13.11 9.07 17.57 8.68 9.72 5.29 J2 .00 .01 .00 .01 .00 .03 Grade 8 and 9 letter grades made statistically significant contributions to the regression equation. There were higher correlations at Grade 8 then Grade 9. This would be expected because the length of time since the CBM test and academic and social pressures of high school. Standard error of estimate (SEE) provides the range of errors for the true score. The range was +-18. 00% to 19.56% 29 DISCUSSION Overview There is a vast amount of research that supports the use of CBM as a valid assessment tool in English (Deno, Mirko & Chiang, 1982; Elliot & Fuchs, 1997; Espin & Deno, 1993; Fuchs & Deno, 1992; Hintze & Shapiro, Conte & Basile, 1997; Marston, 1989; Shinn & Habedank, 1992). This study was conducted to look at the validity of locally developed French Immersion CBM probes by comparing CBM norms for both written expression and reading fluency to concurrent letter grades and determining the predictive ability of the CBM scores on subsequent letter grades. Evidence from this study confirms that there are strong correlations between CBM scores in both written expression and reading fluency with concurrent letter grades. It also confirms that there is a predictive ability of CBM scores. It supports that the existing body of research for English CBM is applicable to French Immersion CBM. Summary and Conclusions Grade by grade comparison of Pearson's correlations for CBM scores were made between English results for TWW and French Immersion results for WSC. The English re1iabilities for TWW ranged between .48 to .62 for six months (School District #57, 1996b). French Immersion reliabilities for WSC ranged from .30 to .72 for six months (St-Pierre, 2002). Using Cohen's (1992) effect size, all correlations for English and French Immersion are classified as medium to large effect. Grade by grade comparison of Pearson's correlations (r) for CBM WRC were made for English and French Immersion. English reliability results ranged from .77 to .86 for six months (School District #57, 1996b). French Immersion reliability results ranged 30 from .74 to .86 (St-Pierre, 2002). Using Cohen's (1992) effect size, all correlations for English and French Immersion are classified as large effect. In the English study (Fewster, 2000) correlations were run with Grade 6/7 CBM WSC and WRC and letter grades earned in English and Social Studies Grade 8 to 10. In the French Immersion study (St-Pierre, 2002) correlations were run with Grade 7 CBM WSC and WRC to letter grades earned in Language Arts (French), Social Studies and English Grade 8 and 9. In comparing English and French Immersion correlation results in all cases French Immersion WSC correlation scores were less than English. The results still indicated a significant relationship. When comparing WRC, French Immersion results correlated higher in all cases and indicated a significant relationship. Regression analyses were conducted for both English grade 6/7 and French Immersion grade 7 CBM scores for WRC compared to letter grades earned at the Grade 8 level. English CBM scores were compared to English and Social Studies (Fewster, 2000). In this study French Immersion CBM scores were compared to Language Arts (French), English and Social Studies. Regression analysis indicated stronger correlations for French Immersion than English in all cases. The CBM reliability for French Immersion WSC and WRC are similar in magnitude to English. CBM WSC to letter grades earned, predictive validity for French Immersion was less than English but still significant. CBM WRC to letter grades earned, predictive validity for French Immersion was greater than for English. The overall conclusion is that French Immersion CBM performs in a similar manner to English (native language); therefore French Immersion teachers may cautiously apply the 31 significant body of literature relating to reliability and validity of CBM as applicable to French Immersion setting. Limitations One of the possible limitations of this study, is the number of participants. The original norming study included between 240 and 321 French Immersion students. In this validity study, 262 were located and letter grades were matched to between 205 and 211 students. A second factor may be there are only folir French Immersion schools in one district. The entire population was used in the norming study as apposed to a sample. A third factor may be that there is a natural attrition rate in the French Immersion program. Students who remain in the program tend to do well academically. Most marks remain within the B range throughout all subjects. Implications This study supports that CBMs are a predictor of school letter grades. Other districts may be interested in this research but norming and additional research would be required at the district level. Second language research could replicate or expand upon this study. There are implications for CBM scores to be used as a screening tool to assist with interventions so students remain in the French Immersion program. Fewster (2000) confirmed using English CBM as an indicator for special education eligibility decisions. 32 REFERENCES Burns, M. K., MacQuarrie, L. L., Campbell, D. T. (1999). The difference between curriculum-based assessment and curriculum-based measurement: A focus on purpose and result. NASP Communique, 27 (6). Retrieved November 17, 2002 from http://www .nasponline.org/pub lications/cq 2 76cba.html Chapelle, C. (1999). From reading theory to testing practice. In M. Chalhaub-Deville (Ed.), Studies in Language Testing 10- Issues in computer-adaptive testing of reading proficiency (pp. 150-159). Cambridge: University Press. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston. Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219-232. Deno, S. L., Mirkin, P. K., & Chiang, B. (1982). Identifying valid measures of reading. Exceptional Children, 49 36-45. Elliot, S. N., & Fuchs, L.S. (1997). The utility of curriculum-based measurement and performance assessment as alternatives to traditional intelligence and achievement tests. School Psychology Review, 26, 224-233. Espin, C. A., & Deno, S. L. (1993). Performance in reading from content area text as an indicator of achievement. Remedial and Special Education, 14,47-59. Fewster, S.A. (2000). School-based evidence for the validity of curriculum-based meansurment norms in School District #57 . unpublished Master's thesis, 33 University ofNorthern British Columbia, Prince George, British Columbia, Canada. Fewster, S., & MacMillan P.D. (2002). School-based evidence for the validity of curriculum-based measurement of reading and writing. Remedial and Special Education, 23, (149-156). Fuchs, L. S., & Deno, S. L. (1992). Effects of curriculum within curriculum-based measurement. Exceptional Children, 58, 232-242. Fuchs, L. S., & Deno, S. L. (1994). Must instructionally useful performance assessment be based in the curriculum? Exceptional Children, 61, 15-24. Hintze, J., Shapiro, E., Conte, K., & Basile, I. (1997). Oral reading fluency and authentic reading material: Criterion validity of the technical features of CBM survey-level assessment. School Psychology Review, 26, 535-553. Hurlburt, R. T. (1998). Comprehending behavioral statistics (2nd ed., pp 376-439) California: Brooks/Cole. Laurier, M. (1999). The development of an adaptive test for placement in French. In M. Chalhaub-Deville (Ed.), Studies in language testing 10: Issues in computeradaptive testing or reading proficiency (pp. 122-129). Cambridge: University Press. Marston, D. B. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp.18-78). New York: Guilford Press. 34 Marston, D., Mirkin, P., & Deno, S. (1984). Curriculum-Based Measurement: An alternative to traditional screening, referral and identification. The Journal of Special Education, 18 (2), 110-117. McNamara, T. F. (1999). Computer-adaptive testing: A view from outside. In M. Chalhaub-Deville (Ed.), Studies in language testing 10: Issues in computeradaptive testing or reading proficiency (pp. 136-149). Cambridge: University Press. Messick, S. (1989). Validity. In R. L. Linn (Ed), Educational Measurement (3rd ed., pp. 13-1 03) New York: Macmillan. School District #57. (1996b). School Support Services: Practices, organization, and principles. Prince George, BC: School District #57. Shinn, M. R. (1989). Identifying and defining academic problems: CBM screening and eligibility procedures. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp.1-17). New York: Guilford Press. Shinn, M. R., & Habedank, L. (1992). Curriculum-based measurement in special education problem identification and certification decisions. Preventing School Failure, 36(2), 11-15. St-Pierre, S. (2002). Curriculum-based measurement norming for reading fluency and written expression for French Immersion students in School District #57. Master's project, University ofNorthem British Columbia, Prince George, British Columbia, Canada. Tilly III, W. D. & Carlson, S. (1992). Creating measurement materials. CBA Training Institute, University of Oregon. 35 APPENDIX A School District #57 Problem-Solving Model PROBLEM SOLVING APPROACH LEVEL IV E ~ Extended Problem Solving Team ..0 0 L.. 0.... ~ ~ 0 V) LEVEL II 0 +- School-Based Team "U ~ "U I ~ ~ z ,, I I I I I ~~~ LEVEL I Teacher - Parent I I I I I I I I I I I I I I I I I I Process Identify the problem Analyze the problem Determine strategies Write an action plan Implement plan/collect data Evaluate/Follow-Up Intensity of Problem "' Adapted from Heartland Education Agency 11 Division of Special Education - Iowa 10 37 APPENDIXB Approval School District #57 SCHOOL DISTRICT NO. 57 (PRINCE GEORGE) 1894 Ninth Avenue, Prince George, B.C. V2M 1L7 Phone: (250) 561-6800 • Fax (250) 561-6801 www.sd57.bc.ca January 13, 2003 Sherry Thibault 5205 York Place Prince George, BC V2N 2A4 Dear Sherry, This letter is to confirm our email discussion regarding your request to obtain access to schools in the Prince George School District for the purpose of educational research. As we discussed, our school district recognizes that research is an integral part of edUcation, and we support the research sponsored by our local tertiary institutes as a priority. Your project, analyzing the validity of the French CBM norms will provide valuable information for our French immersion schools and for our district. This letter's purpose is to indicate that you have district approval to proceed with your project. "District approval" allows the researcher to approach principals and subsequently teachers to request their permission to conduct research in their school/classroom. Researchers must understand that circumstances may not be appropriate and that school administrators have the final decision. Your next step will be to contact the principals of the schools involved to set up a meeting with each of them to discuss your project and obtain their permission to undertake the project in their school. A copy of this letter has been forwarded to the following principals: John Mclay, Janice Wharrie, Brian Pepper, Madeleine Crandell and Verne Thompson . If you have any questions, please do not hesitate to call me. Good luck with your project. I look forward to receiving a copy of the final report. Sincerely, ~ Bonnie Chappell Director Curriculum & Instruction CC: John Mclay, Principal, Austin Road Elementary School Janice Wharrie, Principal, Spruceland Elementary School Brian Pepper, Principal, Heather Park Middle School Madeleine Crandell, Principal, College Heights Elementary School Verne Thompson, Principal, Duchess Park Middle School BC/hg 39 APPENDIXC Approval UNBC Ethics UNNERSITY OF NORTHERN BRITISH COLUMBIA RESEARCH ETHICS BOARD MEMORANDUM To: Sheralee Thibault From: Henry Harder, Chair Research Ethics Board Date: April 26, 2005 Re: E2005.0322.035 School Based Evidence for the Validity of French Immersion CBM Norms in School District #57 Thank you for submitting the above-noted research proposal and requested amendments to the Research Ethics Board . Your proposal has been approved. Good luck with your research.