INVESTIGATION OF CURRICULUM BASED MEASUREMENT APPLICABILITY TO JUNIOR SECONDARY SETTINGS by Miranda M. G. Pastore B.Ed., University of British Columbia, 1983 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF EDUCATION In EDUCATIONAL COUNSELLING THE UNIVERSITY OF NORTHERN BRITISH COLUMBIA September 2004 © Miranda M. G. Pastore, 2004 1^1 Library and Archives Canada Bibliothèque et Archives Canada Published Heritage Branch Direction du Patrimoine de l'édition 395 W ellington Street Ottawa ON K 1A 0N 4 Canada 395, rue W ellington Ottawa ON K 1A 0N 4 Canada Your file Votre référence ISBN: 0-494-04701-1 Our file Notre référence ISBN: 0-494-04701-1 NOTICE: The author has granted a non­ exclusive license allowing Library and Archives Canada to reproduce, publish, archive, preserve, conserve, communicate to the public by telecommunication or on the Internet, loan, distribute and sell theses worldwide, for commercial or non­ commercial purposes, in microform, paper, electronic and/or any other formats. AVIS: L'auteur a accordé une licence non exclusive permettant à la Bibliothèque et Archives Canada de reproduire, publier, archiver, sauvegarder, conserver, transmettre au public par télécommunication ou par l'Internet, prêter, distribuer et vendre des thèses partout dans le monde, à des fins commerciales ou autres, sur support microforme, papier, électronique et/ou autres formats. The author retains copyright ownership and moral rights in this thesis. Neither the thesis nor substantial extracts from it may be printed or otherwise reproduced without the author's permission. L'auteur conserve la propriété du droit d'auteur et des droits moraux qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation. In compliance with the Canadian Privacy Act some supporting forms may have been removed from this thesis. Conformément à la loi canadienne sur la protection de la vie privée, quelques formulaires secondaires ont été enlevés de cette thèse. While these forms may be included in the document page count, their removal does not represent any loss of content from the thesis. Bien que ces formulaires aient inclus dans la pagination, il n'y aura aucun contenu manquant. Canada TABLE OF CONTENTS ABSTRACT................................................................................................................................. iv LIST OF TABLES........................................................................................................................v LIST OF FIGURES.................................................................................................................... vl ACKNOW LEDGEMENTS........................................................................................................vil CHAPTER ONE INTRODUCTION........................................................................................... 1 CHAPTER TW O REVIEW OF THE LITERATURE............................................................... 8 CBM at the Elementary Level................................................................................... 9 CBM at the Secondary Level.................................................................................. 14 CHAPTER THREE M ETH O D S.............................................................................................. 19 Subjects..................................................................................................................... 19 Subject Selection..................................................................................................19 Measures................................................................................................................... 21 Standardized Norm-Referenced Testing......................................................... 21 Curriculum Based Measurement Probes......................................................... 21 Student Final Marks.............................................................................................23 Procedures.................................................................................................................24 Training..................................................................................................................24 Data Collection.....................................................................................................25 Ethics..........................................................................................................................28 CHAPTER FOUR RESULTS............................................................ Means and Standard Deviations for CBM Measures Across Grades.................30 Growth...................................................................................................................31 Gender Differences.............................................................................................. 32 Coefficients of Equivalence and Stability for CBM Measures Across Grades.43 Reiiability.............................................................................................................. 43 Concurrent Validity.............................................................................................. 44 Predictive and Discriminant Validity for CBM Measures Across Grades 49 CBM Validity in Relation to Success in Core Academic Courses.................49 CBM Validity in Relation to “At Risk” or Advanced Designation....................50 CBM Validity in Relation to Program Placement............................................. 52 CHAPTER FIVE DISCUSSION AND CONCLUSIONS...................................................... 61 Norming...................................................................................................................... 61 Concurrent Validity of CBM Measures.................................................................. 61 Predictive Validity of CBM Measures.................................................................... 63 Discriminant Validity of CBM Measures................................................................ 63 Implications for Practice...........................................................................................65 Recommendations....................................................................................................66 REFERENCES..........................................................................................................................68 Appendix A Administration and Scoring Guidelines - CBM Reading Probes...............73 Appendix B Grade 10 Pronunciation Guide and Reading Probes................................... 82 Appendix C Grade 9 Pronunciation Guide and Reading Probes......................................87 Appendix D Grade 8 Pronunciation Guide and Reading Probes......................................92 Appendix E Administration and Scoring Guidelines - CBM Writing Probes (1 - 6 ) ........97 ABSTRACT Curriculum based measurement (CBM) has been described in the literature as valid and reliable for assessment of reading and writing skills, as well as for subsequent school interventions at the elementary level. This study was an investigation into the validity of CBM reading and writing measures in a junior secondary setting. CBM oral reading and written expression scores obtained from approximately 400 students in grades 8, 9, and 10 were compared with their comprehension and vocabulary scores on the Stanford Reading Diagnostic Test. This study also provides evidence of the validity of CBM in predicting success in English 12 and Social Studies 11 in addition to placement in regular, remedial/support, or advanced classes. Overall, the findings indicate that CBM reading and writing measures are valid assessment tools for use in junior secondary settings. IV LIST OF TABLES Table 1: Flesch Reading Ease and Flesch-Klncaid Grade Levels for Novels Selected 22 Table 2: Sequence for Administration of Reading Probes 27 Table 3: Reading Fluency (WRC) Scores for Each Testing Period by Grade and Gender 34 Table 4: Written Expression Fluency (TWW and WSG) Scores for Each Testing Period by Grade and Gender 35 Table 5: Mean Percentile Scores on SORT by Grade andGender 36 Table 6: Mean %WRC and %WSC for Each Testing Periodby Grade and Gender 37 Table 7: Cohen’s d Effect Size - Reading Measures 38 Table 8: Cohen’s d Effect Size - Written Expression 39 Table 9: Correlations - Probing Sessions 46 Table 10: Relations Between CBM Oral Reading and SORT (Comprehension and Vocabulary) 47 Table 11: Relations Between CBM Written Expression and SORT (Comprehension and Vocabulary) 48 Table 12: Relations Between CBM and SORT Reading Measures with Final Course Mark Percentages (%) 55 Table 13: Relations Between CBM Written Expression and Final Course Mark Percentages (%) 56 Table 14: Relations Between CBM and SORT Reading Measures with “At Risk’’ and Advanced Placement 57 Table 15: Relations Between CBM Written Expression with “At Risk” and Advanced Placement 58 Table 16: Relations Between CBM and SORT Reading Measures with Program Placement 59 Table 17: Relations Between CBM Written Expression with Program Placement 60 V LIST OF FIGURES Figure 1: Grades 8, 9 and 10 WRC means and best-fit lines 40 Figure 2: Grades 8, 9 and 10 TW W means and best-fit lines 41 Figure 3: Grades 8, 9 and 10 WSG means and best-fit lines 42 VI ACKNOWLEDGEMENTS A journey of a thousand miles must begin with a single step - Chinese Proverb There are many people who have contributed to the completion of this thesis and the steps necessary to bring closure to this journey of learning. I would like to extend my thanks and gratitude to School District #57, particularly Dr. Carl Anserello for providing funding and support for this study, and the staff and students of Prince George Secondary School for their participation in it. I would also like to thank UNBC for funding my travel to Halifax where I had the opportunity to present my research and learn from the feedback provided by various professors and educators at the CSSE conference. This thesis would not have been possible without the valuable input, effort, thoroughness and support provided by the members of the CBM team during the weeks of data collection. Thank you. A special thanks to my supervisor. Dr. Peter MacMillan for your commitment and effort in guiding me through this part of my life journey. Your support, reassurance and advice made the difference during those difficult times. Finally, to my family and friends - a huge thank you from the bottom of my heart for your patience, support and words of encouragement. You all believed... Perhaps love is the process of my leading you gently back to yourself. Antoine de Saint-Exupery VII CHAPTER ONE INTRODUCTION The debate over the use of standardized norm-referenced tests for student assessment has been a topic of discussion and research and continues to divide educators (Fewster & MacMillan, 2002; Ysseldyke & Christenson, 1988). These tests are sometimes criticized as being inappropriate because of the bias to cultural groups and gender. As well, the tests do not provide a clear indication of student strengths and weaknesses, necessary information for the planning of interventions and Individual Education Plans (Good & Jefferson, 1998; Marston, 1989). Thus, some educators argue that this type of assessment is not congruent with instruction or decision-making for appropriate interventions. However, many educators and psychometricians continue to support the use of these instruments because professionals recognize the tests, which facilitate communication. In School District #57 (SD #57), assessment philosophies and the resulting methods of student assessment have undergone tremendous change at the elementary school level. The district focus for the elementary schools has moved away from reliance on standardized and norm-referenced tests to identify students who may require extra support or special services, to the use of Curriculum-Based Measurement (CBM) to support the district problem solving model (Fewster et al., 2003; SD #57, 1996). General interest in CBM at the district level started in 1985, resulting in a TriUniversity Summer Institute in CBM held in Prince George in 1991. A School Support Services Task Force released recommendations in 1993 that a district problem-solving model be developed to “examine the service being provided in the district and to suggest ways to make these services more responsive to schools.” (SD #57, 1996, p. 3) The 1993 Task Force report stressed the need for a district focus allowing classroom teachers and schools to identify needs and make decisions regarding support. Further, 1 to support the new district focus, it recommended, “that simple informal and formal system for gathering information for the purpose of monitoring and evaluating the effectiveness and efficiency of service delivery is developed” (SD #57, 1996, p. 3). Several district staff and teachers conducted the background research and the district adopted CBM as the district-wide evaluation system as it “would support the new SD #57 collaborative problem solving model, which links functional assessments to effective interventions” (SD #57, 1996, p. 3). A norming project, conducted in conjunction with the University of Northern British Columbia (UNBC), was initiated in 1995. Norming procedures were developed and teachers from all the elementary schools in the district were trained in the administration and scoring of the oral reading and written expression probes. The collected data provided the first school district wide norms for CBM oral reading and written expression for grades one through seven to support the district problem-solving model (SD #57, 1996). District-wide norms for CBM mathematics computations were developed in 1999-2000 with the technical report released in 2000 (Walraven & MacMillan, 2000). In 2002-2003, the CBM oral reading and written expression measures for grades one through seven were renormed and school district wide Kindergarten through Grade Two Dynamic Indicators of Basic Early Literacy Skills (DIBELS) norms were developed. As implementation of the problem-solving model is now complete at the elementary level, district staff is now looking at secondary schools to determine if CBM and the problem-solving model can be extended to secondary students. At Prince George Senior Secondary School (PGSS), the Learning Center provides assessment for students in grades 8 to 12. There are generally three purposes for assessment. Students transferring from out-of-province schools or 2 entering the school system from home schooling, who were on different educational programs, complete the assessment to determine the most appropriate regular class placement. As well, school staff refers students to the Learning Center for assessment to fulfill requirements of the Ministry of Education. The Ministry dictates that any student receiving targeted funding or additional support services must have an Individual Education Plan (lEP) (Ministry of Education, 1995) which must include a recent school achievement assessment. In addition, the Ministry also requires a recent school achievement assessment to accompany an application for adjudication of provincial exams for learning disabled students. Most often, however, the assessment of students occurs because the student experiences difficulty or failure in the regular classroom. School staff, parent(s), or a student may make a request for an assessment. Assessment is necessary to determine if placement in a regular classroom with support and/or monitoring is an appropriate option for the student. If not, the assessment serves to screen students to determine placement eligibility for the three alternative programs provided by the school. The Pre-Employment Program (PEP) is a program available to students who are age appropriate for grades 11 and 12 but who do not have the academic skills to be successful with the school curriculum. These students work to develop job skills and leave school with a School Leaving Certificate, not a Dogwood Certificate. The Junior and Senior Alternate Programs are programs available to those students age appropriate for grades 9 and 10 (Junior Alternate) or grades 11 and 12 (Senior Alternate) who have the academic skills to be successful with the regular curriculum but whose placement in a regular classroom setting is not a possible or recommended option. Students completing the Senior Alternate Program graduate from high school with a Dogwood Certificate. The third alternative program is the Skills Support Program 3 for students age appropriate for grades 9 and 10 who do not have the academic skills to be successful in the regular classroom. The opportunity for academic skill remediation is provided and students who are successful in skill development re-enter the regular classroom. Those students unable to reach a level of skills necessary for the regular classroom enter in the Pre-Employment Program. Learning Assistance staff share assessment results with the student, parent(s), and school personnel to determine possible options for the student. This service is available to all students registered at PGSS, as well as those students applying to enter the school. The incentive to investigate the applicability of CBM to the junior secondary level originated from concerns regarding the validity of the reading assessment tools used by school staffs, the communication of student assessment results, and budgetary restraints (Anserello, personal communication). Motivation to pursue an alternative assessment arose from the concern regarding the validity of standardized and normreferenced assessment to determine reading strengths and weaknesses. The SORT (4^^ edition) is a reading test normed in the United States (both rural and urban). The test authors concentrated on the equal representation of “males to females. Whites to African Americans, and Whites to Hispanics “(Swerdlik, 1998). When considering the variety of student backgrounds, differences, and specific needs of students in Northern British Columbia, it is evident that the norming population used for standardizing and norm referencing the SORT is clearly not representative of our student population. In addition, the student population at PGSS differs from other schools in the district as some students come from elementary schools classified as “inner-city” schools. A higher level of dropouts illustrates the difference in student population when compared to both the district and province (Ministry of Education, Skills and Training, 1996). Definitely, the use of American normed standardized tests at PGSS is not appropriate. 4 Another factor contributing to our investigation into CBM is that our grade 8 population comes from elementary feeder schools where CBM is the fundamental and core component of the district problem-solving model. For students moving from elementary-level schools to junior secondary, continuation of practice and assessment provides consistency and may facilitate better understanding of student needs by school staffs. The third factor contributing to our investigation in CBM is budgetary restraints. Current assessment procedures at the junior secondary schools in the district involve the use of standardized and norm-referenced reading tests. The two reading tests used at PGSS to assess reading ability are the Formal Reading Inventory for students in the 8th grade and the Stanford Diagnostic Reading Test (SORT) for students in grade 9 and 10. The cost of updating these assessment tools and ordering protocols is an issue as the department budget is insufficient to purchase the new supplies. In addition to lower funds available for the purchase of testing materials, a reduction of staff available for administering the school assessments creates the necessity to pursue a more cost-effective assessment that can be administered more efficiently to the increasing numbers of students entering the junior high level with educational difficulties and needs. As the photocopying of CBM probes does not violate copyright regulations, significant savings could be realized. As well, due to the nature of the measurement, CBM could be administered in a short period of time, thus reducing the amount of staff time and involvement. SD #57’s interest and commitment to CBM is continuing as it has proved to be relevant and practical assessment. Research in the district is ongoing with a study on elementary reading and written expression probe scores as predictors of program placement In junior secondary grades (Fewster & MacMillan, 2002) as well as gender 5 differences on oral reading and written expression probes (Hedekar, 1997) and MacMillan (2000). Othier non-CBM research in SD #57 has included a study that investigated the importance of the early identification of low reading ability and its relationship to students dropping out of junior secondary school (Mothus, 1997). Adding to the body of literature pointing to poor achievement, absenteeism, and truancy contributing to school failure, Mothus (1997) noted that grade 8 students with poor reading abilities were more likely to fail one or more subjects. Her research highlighted that failure to achieve could be predicted, given reading ability, and that “failure to assist students who are poor readers appears to be detrimental to their school attendance” (Mothus, p. 95). More importantly, those students undetected by the school as poor readers but identified as such on the SORT and not offered learning assistance, dropped out at alarming rates. Her evidence shows that the assessment of a learner’s ability to read is crucial to identify high-risk students so that an intervention occurs prior to the student experiencing failure and then dropping out of school. Thus, research to determine if CBM is applicable in junior secondary settings could provide the district with assessment options to identify high-risk students in grades 8 to 10. Employed as a Learning Assistance teacher at PGSS, my role is to collect information and implement support strategies for students enrolled in regular classes. I have access to information, collect it, and share it with staff, parents, and students so that the students may benefit from the information and/or intervention. My goal is to find valid and reliable assessment which will provide data to assist staff in recommending placement options and/or possible interventions to support students at PGSS. The purpose of my study is to develop local school norms based on curriculum- based measures in order to determine if CBM is an assessment option that is applicable to junior secondary settings to provide relevant and necessary data to staff. The research questions in the first section address norming issues while those in the latter section focus on validity evidence: 1. Is CBM a useful tool at the junior secondary level? Specifically, do CBM oral reading and written expression scores have a ceiling effect within junior secondary grades that hinders their use? If the research finds a ceiling effect for the regular school population, is CBM still useful for students at lower levels of performance within the grades? Is development over grades, as measured by CBM, consistent with the trend found at the elementary levels? 2. Are the gender differences in the oral reading and written expression CBM scores reported by Hedekar (1997) continued at the junior secondary level? Do gender differences exist in the reading scores measured by the Stanford Diagnostic Reading Test at the junior secondary level? 3. Is there a relationship between reading scores on the Stanford Diagnostic Reading Test, senior English and SS 11 grades, and the CBM oral reading scores? 4. Do CBM oral reading and written expression scores screen accurately for the following students: Pre-Employment students? Skills Support students? International Baccalaureate students? CHAPTER TW O REVIEW OF THE LITERATURE Curriculum Based Assessment (CBA) is performance based assessment that allows direct observation and measurement of a student’s performance within a curriculum so that this performance can be used to make educational decisions. Curriculum Based Measurement (CBM) is a type of CBA. CBM is a “set of specific measurement procedures that can be applied to quantify student performance in reading, written expression, spelling and arithmetic” (Deno, 1989, p.15). The CBM model was developed by Deno in the 1970s in response to concerns by some educators regarding the reliance on commercially developed standardized and normreferenced tests to assess students to determine eligibility for support services. Concerns raised regarding the use of these instruments included the cost of the administration, interpretation of standardized and norm-referenced tests and their lack of validity, the limited information provided by these instruments to assist educators in making instructional decisions, as well as an “overrepresentation of minority students in special education” (Elliott & Fuchs, 1997, p.224). Other educators favour the use of commercial standardized norm-referenced tests. The Stanford Reading Diagnostic Test (SDRT) is an example of a commercial reading assessment frequently used by educators in Canada and the USA. The creators of the SDRT purport that the test can: diagnose students’ strengths and weaknesses in the major components of the reading process. Its results can be used to challenge students ... and provide special help for others who lack some of the essential reading skills. Results also can be used to identify trends in the reading levels of students ... provide information about the effectiveness of instructional programs, measure changes that have taken place over an instructional period and keep the community and 8 school board informed about students’ overall progress In reading (Karlsen & Gardner, 1996, p. 7). Swerdlik (1998) expresses concerns regarding the use of this test for determining reading strengths and weakness due to reliability issues and cautions users that they should use the SDRT4 only if the test closely matches the reading curriculum in which the students were instructed. Whereas traditional assessment has focused on the classification aspect of whether or not a student should receive support services, the needs of the education community are to have assessment tools sensitive to instruction and curriculum in determining whether support services are required, as well as to be able to document student growth and the effectiveness of the intervention provided. Educators and school psychologists lobbied for change to the assessment practices, the decision making process, and the service delivery systems so that assessment practices would more closely link with intervention for students (Deno, 1989; Jones, Wilson & Bhojwani, 1997; Marston, 1989; Marston & Magnusson, 1988; Shinn, Nolet & Knutson, 1990; Ysseldyke & Christenson, 1988). Assessment measures drawn directly from curriculum appear, at face value, to be reliable and valid assessment tools that provide this linkage between students, the curriculum and instruction. With CBM, students are required to respond directly to the measure rather than select their responses as with commercially developed assessment tools, thereby increasing the face validity of CBM (Tindal, 1988). CBM allows for repeated measurement of a students’ ability to interact with the curriculum, which increases CBM’s reliability and documents students’ growth. CBM at the Elementary Level The focus of much of the research conducted in CBM has been on the development and implementation of CBM oral reading and written expression at the 9 elementary school level. There Is considerable evidence demonstrating that CBM is a valid and reliable assessment method at the elementary school level (Deno, 1989; Fewster & MacMillan, 2002; Good & Jefferson, 1998; Shinn, 1989; Tindal, 1988). CBM oral reading measures have been found to have high reliability in all three reliability as assessed by techniques such as test-retest (reliability coefficients are reported as being above .90), parallel form (scores ranging from .84 to .96), and interrater agreement with coefficients at .99 (Marston, 1989). In the context of CBM, oral reading and written expression measures are defined as fluency measures as they are a combination of speed and accuracy. Potter and Wamre (1990) theorize the appropriateness of the use of reading fluency as a measure of reading ability based on Chall’s stages of reading development and LaBerge and Samuel’s 1974 model of automaticity. Both reading models consider reading a developmental process where a reader moves through stages of reading by building decoding skills, gaining reading fluency and moving into stages of developing comprehension skills. As a reader becomes more fluent, more attention is directed to the comprehension of the reading material. Potter and Wamre suggest “the number of words read within a specified time period may be an analogous measure of processing speed” (p. 22) and information processing speed impacts comprehension and reading ability. Marston (1989) states, “oral reading fluency contributes significantly to a model of reading and may be as valid a measure of comprehension as decoding” (p. 39). Marston (1989) and later Shinn, Good, Knutson, Tilly & Collins (1998) summarize extensive research into the criterion-related validity of CBM. Deno’s initial research determined that oral reading from basal readers was a valid measure of reading skill as correlated with norm-referenced reading tests (coefficients ranging from .73 to .91 with most coefficients over .80). Marston’s summary of 14 studies confirms Deno’s initial 10 research findings with coefficient ranges of .63 to .90 with most correlation coefficients over .80. Once the validity of oral reading as a measure of reading skill was determined, Deno investigated reading fluency as a valid measure of reading. He reported validity coefficient measures ranging from .53 to .91 with half of the studies having coefficients greater than .80. Marston confirmed Deno’s findings with subsequent studies reporting coefficients ranging from .57 to .86 with half of the studies over .80 and stating that the finding of the research “provides additional support for the criterion-related validity of curriculum-based reading measures as a predictor of global reading proficiency” (p. 33). Earlier research by Deno, Mirkin and Chiang (1982) demonstrated that the criterion validity of all CBM reading measures correlated highly with standardized and norm-referenced tests, r = .70 to .95, with words read orally most highly correlated with comprehension scores, r = .79 (as cited in Deno, 1989). Later, Shinn, Good, Knutson, Tilley & Collins (1992) found that reading fluency was more highly correlated with comprehension subtests (r= .92) than word study tests (r= .81). Deno’s validation research also investigated the relationship between reading fluency and teacher judgment of reading ability. Deno’s research demonstrates that reading fluency is highly related to teacher judgment, r = .86 (Marston, 1989). Subsequent research by Marston and Deno (1981) confirmed the strong relationship between reading fluency and teacher judgment and, when compared to teacher ratings with standardized and norm-referenced tests, the relationship between reading fluency with teacher ratings was significantly stronger. More recent research by Shinn et al. (1992) confirmed Deno’s research finding strong support for criterion related validity relationships between oral reading and standardized tests and teacher judgments (ranges of .60 to .90 with an average of .80). 11 In addition to assessment measures that are reliable and valid, educators require assessment measures able to distinguish those students requiring additional support from their peers to ensure appropriate support and placement decisions. Marston (1989) cites research into discriminant validity of CBM conducted by Deno, Marston, Shinn and Tindal (1983) and Shinn and Marston (1985) which found that CBM results were able to reliably separate regular students from mildly handicapped students with learning difficulties. Good and Jefferson’s (1998) review of the research cites several studies investigating the discriminant validity of the CBM oral reading measures. They report that these reading measures, administered to students in grades one to six, can differentiate special education students and students with academic concerns from their peers, learning disabled students from non-learning disabled students, and those with low reading ability from mildly handicapped students. As well as requiring sensitive instruments able to distinguish between students, educators require assessment tools that allow the monitoring of progress and evaluation of the effectiveness of teaching or curriculum changes. Compared to standardized reading tests, Marston (1989) reports that CBM oral reading measures “delineated greater growth in the reading performance of students and correlated much more closely with teacher perceptions of individual student improvement” (p. 40). Potter and Wamre’s (1990) study confirmed previous research demonstrating that there was consistent growth in rates of oral reading in higher grade levels. Hedekar’s (1997) research in elementary schools in SD #57 confirmed the stability of growth in both oral reading and written expression rates through all grades. In comparison to CBM oral reading measures, the reliability and validity coefficients of the different CBM written expression measures are smaller with greater ranges in coefficient scores. (Espin & Tindal, 1998; Fewster & MacMillan, 2002; 12 Marston, 1989). Marston & Deno (1981) and Tindal & Parker (1989) found reliability coefficients of r = .85 internally consistent, between judges (r= .95) with retest stability smaller in ranges of .48 to .71 (cited in Parker, Tindal & Hasbrouck, 1991). The written expression indices used at the elementary school level include the total number of words written, words spelled correctly, correct letter sequences, mature word choices, large words written, correct word sequences (Marston, 1989; Parker, Tindal & Hasbrouck, 1991), and uncommon words (Tindal, 1988). Parker, Tindal & Hasbrouck (1991) report a moderate correlation with total words written and total words spelled correctly and published assessments of writing. Marston (1989) reports correlations ranging from .41 to .84 for total words written and .45 to .92 (most over .70) for words spelled correctly. Correct letter sequences, a more complex index, yielded a narrower range of scores of .57 to .86. Several studies were reviewed by Good and Jefferson (1998) reporting “most validity coefficients in the lower end of the .60 to .80 range, providing modest support for the construct validity of CBM for written expression scored with correct writing sequences” (p. 66). Later Fewster and MacMillan (2002) found validity coefficients between oral reading, words spelled correctly and student subject marks, in courses three years after the probing sessions, to be significant and of medium effect size. The relationship between writing fluency as measured by CBM written expression measures and teacher perception of writing ability is strong. Marston (1989) reports a relationship between total words written and teacher marking (r= .85) and words spelled correctly with teacher marking (r= .84). Shinn and Marston (1985) conducted research demonstrating that the written expression measures can differentiate regular students from those requiring support or changes to their curriculum. In Shinn and Marston’s research, they found that the differentiation 13 coefficient was stronger in the lower elementary grades than in grades 5 and 6 (as cited in Marston, 1989). The information collected on CBM written expression measures demonstrates that “the data obtained can serve as vital signs of growth in basic written language skills” (Deno, 1989, p. 224). Marston (1989) summarizes studies in student growth reporting that there was a significant increase in the total words written, words spelled correctly and correct letter sequences in a student population grades one to six. The predictive validity of the CBM oral reading and written expression measures at the elementary level to future achievement in the junior secondary level has not been an area of investigation by Deno and associates. Fewster and MacMillan (2002) analyzed grade six and seven CBM oral reading and written expression scores (words spelled correctly) collected in the spring probing session of 1996 with course marks of the students in their grade 8, 9 and 10 years. They found a significant relationship between the oral reading and written expression scores with grade 8, 9 and 10 final marks in English and Social Studies. They found that although both words read correctly and words spelled correctly were significant predictors of student success, the oral reading was a stronger predictor. The CBM scores were predictive of student placement in the honors program. CBM at the Secondary Level Deshler’s research (Deshler, Schumaker et al., 2001; Schumaker & Deshler, 1994) focused on secondary students with learning disabilities and emphasized the difference in the expectations of student performance between the elementary and secondary school levels. At the elementary school level, the focus of instruction is on skill development and skill proficiency. The focus at this level is the use of assessment sensitive to detect skill differences between students, so that students with skill 14 deficiencies can be identified to receive support, and to evaluate the effectiveness of instructional support and its impact on skill acquisition. At the secondary level, the focus of instruction moves away from skill development to knowledge acquisition where students are expected to possess the skills to then be able to learn and demonstrate their learning, primarily through print. Reading fluency becomes a critical component of a student’s reading comprehension and impacts on student success. As a result, most of the assessment conducted at the high school level centers on classroom evaluation of the mastery of specific course learning objectives. Formal assessment is used to screen students whose skill deficiencies interfere with their ability to interact successfully with the curriculum to meet learning objectives. The research into the use of CBM oral reading and written expression measures at the junior secondary setting has been limited. The reluctance to pursue CBM assessment procedures at the junior secondary level is based on statements that the measures are too simplistic to be sensitive to the increasing levels of reading and writing proficiency at the junior secondary school. Chall’s developmental model of reading and LaBerge and Samuel’s model of automaticity suggest that measures of oral reading rates become less able to differentiate between students and predict academic success (Espin & Deno, 1993). Measures of writing rates at the junior secondary school are considered too simplistic to be useful given the complex interaction of variables involved in the writing process. Thus, at face value, many educators do not consider CBM oral reading and written expression measures appropriate for the junior secondary school level. The reliability of oral reading measures at the secondary level has been reported by Espin and Tindal (1998) to be as high as is found at the elementary level. Espin and Deno (1993) report test-retest reliability average coefficients of .91 (ranges .88 to .93), 15 parallel forms of .91 (r= .90 to .92), and alternate form reliability of .94. The correlation between student oral reading measures and results on standardized achievement tests, as well as final grades, ranged from r = .35 to .47 (Espin & Tindal, 1998). A further study to determine the relationship between CBM oral reading measures (words read correctly), reading achievement tests and text-based reading of grade 10 students was conducted by Espin and Deno (1993). They reported a “low-moderate” relationship between words read correctly and scores from textbook reading and a “moderately high” relationship between the CBM reading scores and the standardized reading test results with correlations ranging from .32 to .53 (p. 55). In addition, they noted, “reading was more strongly related to the academic success of low-level students than to that of high-level students” (p. 55). Fewster and MacMillan’s (2002) findings corroborate the strong relationship between the reading and academic success of less able students. In addition, they report that CBM reading scores are strong predictors of the academic success of honor roll students. A follow up study (Espin & Foegen, 1996) of middle school students investigated the relationship between reading aloud, reading maze, and vocabulary matching and content tasks. They found correlations between the CBM measures and tasks to range between .52 to .65, which were higher than the correlations with G PA and standardized test results. Research has supported the transition of the commonly used elementary reading indices of total words read and words read correctly to the high school level whereas the utilization of the commonly used CBM written expression indices (total words written, words spelled correctly, and correct word sequences) is considered unacceptable in a secondary setting (Espin & Tindal, 1998; Parker, Tindal & Hasbrouck, 1991). A study by Tindal and Parker (1989) found that indices such as percent words spelled correctly (r= .73) and percent of correct word sequences written (r= .75) were 16 more strongly related to teacher marking than total words written, words spelled correctly and correct word sequences. They found that production-independent (percent indices) factors are stronger predictors of student performance (r= .69) than production-dependent (“number of” indices) factors (r= .24) (as cited in Parker, Tindal & Hasbrouck, 1991). Parker, Tindal and Hasbrouck (1991) investigated the relationship between various written expression indices, teacher marking, and results on the Test of Written Language (TOWL), a standardized commercial writing assessment, with students in grades 6 to 8 with mild disabilities. Of the indices studied: total words written, correctly spelled words, percent correctly spelled words, number of legible words, percent total legible words, correct word sequences, and mean length of correct word sequences, they found the strongest relationship between the criterion measures were with percent legible words, correct word sequences and mean length of correct word sequences. In the literature, one finds sizable ranges in correlation coefficients and the reliability of the different written expression indices is no exception. Espin (1997) reported the results of analyses determining the alternate form reliability of CBM written expression measures with middle school students. He found that the reliabilities for total words written, words spelled correctly, and correct word sequences ranged between .61 to .77, with the reliabilities for mean length of correct word sequences and sentences written characters written per word too small to report (as cited in Espin & Tindal, 1998). Two studies of particular relevance and interest to my study are SD #57’s initial CBM reading and writing 1996 norming study (Hedekar, 1997), and the subsequent reading and writing re-norming study in 2002 as reported by Cook in Fewster, Fortier & al. (2003). The research for the first study was collected prior to the data collection for 17 this study, while the re-norming data collection occurred after the data collection phase of this current study. In all three studies, the researchers were consistent in following the same methods and timelines for collecting the CBM data. The studies involved students in the elementary grades (grades 1 through 7) at various school sites. The grade 6 and 7 data in both studies demonstrate that growth in reading writing fluency continues to occur in the upper elementary levels and the correlations indicate that the measures are consistent and reliable with reliabilities ranging from r = .77 - .89 for words read correctly and r = .58 - .74 for total words written. The prior research in the effectiveness of Curriculum Based reading and writing fluency measures at the elementary level has been confirmed by studies conducted in School District #57. Fewster and MacMillan’s (2002) study has started the investigation into the bridging of the elementary CBM results with the success of students in selected courses at the high school level. The purpose of this study is to use the methods and research designs already established at the elementary level to determine the reliability and validity of using CBM reading and writing measures at the high school level. 18 CHAPTER THREE METHODS Subjects The subjects in this study were students registered in grades 8, 9, and 10 at a secondary school located in School District #57 (Prince George). Prince George Secondary School (PGSS) is a secondary school of approximately 1400 students in grades 8 to 12. The sample population consists of 350 students split evenly by grade and by gender within the junior secondary grade. Grades 8, 9, and 10 are considered in SD #57 to be junior secondary grades. In the elementary CBM norming project (SD #57, 1996), student selection involved random sampling of students in all elementary schools in the district. Given the size of the junior secondary population at PGSS, all of the students enrolled in the three grades participated in the study, thus no random selection of students occurred. Subject Selection Student involvement in this pilot project was consistent with the exclusion criteria used in the elementary CBM norming project as described in the Guidebook for the Use of Curriculum Based Measurement In School District #57 (SD #57, 1996) Thus, students who met the criteria of the categories of English as a Second Language (ESL), hearing impaired, visually impaired, multiply disabled, or students with mental disabilities (SLR) did not participate in this project. In addition, those students enrolled in the District Alternate Rehabilitation Program who attended PGSS were not included as participants in the project. The District Alternate Rehabilitation Program is a district program that is housed at PGSS but accessible to students within the entire district. Students in the program meet specific screening requirements based on age, adaptive behaviour, and intellectual functioning. Thus, these students were not included in the study as they are unsuitable candidates for regular school enrollment. 19 Information regarding student placement was collected to allow the investigation into the relationship between student placement and the results obtained with the CBM oral reading and written expression probes as well as the SORT comprehension and vocabulary subtests. Students were identified as enrolled in regular classes, the International Baccalaureate (IB) program, Skills Support, the Pre-Employment Program (PEP), Reception or Transition 8. The IB program is an advanced program where students work to receive credit for university transfer courses when graduating from grade 12. Students in Skills Support are age appropriate for grades 9 and 10, needing to work on skill remediation. Many of these students end up transitioning to the PEP program. PEP students are age appropriate for grades 11 and 12 and work to develop job entry skills. These students leave school with a school leaving certificate, not a Dogwood certificate. Students classified as Reception students are age appropriate for grades 9 through 12 scheduled for a block in learning assistance. Some of these students are learning disabled students who require support with reading and writing activities to demonstrate their potential in their courses. The non-disabled students either access learning assistance for specific course subjects (i.e. math), or attempt to complete academic core courses at the grade 9 or 10 level. These students have a history of learning difficulties. Students in the Grade 8 program, who require an Individual Education Plan (lEP) for educational, emotional or social emotional difficulties, are classified as Transition 8 students. Students who transferred into the school during the school year were added to the project population for subsequent probing sessions. This norming procedure is consistent with the one used in the elementary CBM norming project. Data gathering was complicated by student absenteeism during the probing sessions. All students 20 absent during their specified probing time were scheduled during other probing sessions held that week. The activities in this project were common activities to students in the school setting, as reading aloud and writing are necessary and expected outcomes. Thus, the school board office granted permission that the data could be collected without signed consent by parents for either the inclusion or exclusion of their children in the project. This procedure is consistent with that used in the 1996 elementary CBM norming project where “consent forms were judged unnecessary by school officials” (Hedekar, 1997). The school newsletter advised parents of the study and instructed parents to call a contact person at the school if they were opposed to having their children involved in the study. Measures The instruments consisted of a standardized reading test and CBM oral reading and written expression probes developed following the CBM procedures adopted in the elementary CBM norming project. Standardized Norm-Referenced Testing The Stanford Diagnostic Reading Test (SORT) (4*^ edition) is a standardized and norm referenced timed reading test consisting of multiple-choice items to assess vocabulary, comprehension and scanning skills. Teachers in the Prince George school district frequently use the SORT and it is standard procedure for teachers of English at PGSS to administer these tests to students in the first week of each semester to collect data regarding student reading levels. Curriculum Based Measurement Probes CBM measurement probes are short, timed samples which are extracted from a student’s curriculum. Samples for developing the reading probes at each grade level 21 were selected from curriculum material used by the English teachers at PGSS. Teachers of English 8, 9, and 10 were surveyed to determine a common novel at each grade level that was approved for curriculum by the Ministry of British Columbia. Reading samples were selected quasl-randomly from Superbike! (Brown, 1981) for the grade 8 probes, The Outsiders (Hinton, 1967) for the grade 9 probes, and The Chrysalids (Wyndham, 1955) for the grade ten probes. In selecting the samples, members of the probing team flipped to a page In the novel. If the page contained minimal amounts of dialogue, as recommended by a district staff member Involved In developing the elementary procedures for sample development, the passage was selected from the page. The samples were checked by the Word writing program for Flesch Reading Ease (FRE) and Flesch-Klncald Grade Level (FGL) and three reading passages with similar readability and grade level were selected to become the probes for each grade level (Table 1). Table 1 Flesch Reading Ease and Flesch-Kincaid Grade Levels for Novels Selected Probe 1 Probe 2 Probe 3 Grade 8 Superbikel FRE FGL 84.2 4.4 87.8 3.6 88.2 3.5 Grade 9 The Outsiders FRE FGL 82.2 5.0 78.3 5.5 76.9 5.8 Grade 10 The Chrysaiids FRE FGL 68.3 9.9 55.2 11.8 55.2 11.8 The reading probes were constructed to be 250 words In length to allow sufficient text for advanced readers during test administration. To determine the necessary length of the passages to ensure sufficient text for advanced readers. 22 students from ttie grade 12 English International Baccalaureate class were asked to read the text samples for one minute. The highest number of words read in the one minute period served as a minimum length for the reading probes. The International Baccalaureate students were selected and recommended by their English teacher because of their fluent reading ability. It was determined that a practice probe, to prepare the students for their involvement in the project, was not necessary as it was demonstrated in the elementary CBM norming project that there was no difference between the practice probe and probe #1 (MacMillan, personal communication). New writing prompts were developed for the project as the English teachers indicated that the story starters used in the elementary CBM norming project would not be appropriate for junior secondary students and recommended that prompts that are more appropriate be developed. A senior English teacher developed the written expression probes, which were then approved by the project team. The format of the written expression probe remained consistent with the format used in the elementary CBM norming project. Three reading and writing probing sessions were scheduled during October, January, and April in the academic school year to correspond with the probing sessions completed for the 1996 elementary CBM norming project. Oral reading (see Appendixes B, C and D) and written expression probes (refer to Appendix E) were developed for each grade following procedures used in the elementary CBM norming project. Student Final Marks Final percentage marks were collected, as completed by students in each of the three grades, in Social Studies 11 and either English 12 or Communications 12. The 23 Social Studies 11 marks are the teacher assigned final mark, the English 12 or Communications 12 marks are the final Ministry mark, composed of the ministry government exam mark (40%) and the teacher assigned final mark (60%). Procedures Training The school CBM team consisted of the learning assistance teachers of the PGSS Learning Center, members of school staff with experience in testing and assessment, and members of district staff who either were involved in the elementary CBM norming project or had assessment positions in the district. The members of the school CBM team were trained by the member of district staff involved in the training for the elementary CBM norming project. This one day training session focussed on the administration and scoring of reading probes with advanced readers in order to prepare members of the team for the project probing sessions. As Hedekar (1997) pointed out, “the use of CBM for advanced readers may require more practice and skill on the part of the trained rater” (p. 50). A goal of 90% reliability was set for the training session. The 90% reliability is a commonly accepted standard in CBM statistics. Instructions for the administration of the reading probes were explained and demonstrated and team members received copies of the instructions for future reference (see Appendix A and E). Training for the marking of the reading probes consisted of volunteer students reading a probe aloud for one minute for the group of team members. While an individual student was reading, the school CBM team members recorded the reading errors made on the teacher copy of the probe. The number of total words read (TWR) and the number of errors made by the student were recorded so that the words read correctly (WRC) could be calculated by subtracting 24 the errors from the number of words read. This process was repeated with different volunteer students until the 90% exact agreement goal was achieved. The training for the marking of the written expression probes consisted of the volunteer students providing writing samples for team members to mark. The written expression probes were administered following the procedure used in the elementary CBM norming project. The volunteer students were given a writing prompt followed by one minute of silence to plan their writing. They then wrote for three minutes. School CBM team members individually marked each written expression probe, crossing out each spelling error made. All other errors (punctuation, capitalization, grammar, etc.,) were ignored. Each marker recorded the total words written (TWW ) and the number of spelling errors made and then subtracted the errors made from the total words written to calculate the number of words spelled correctly (WSC). This procedure was used until the 90% exact agreement goal was met. Data Collection A school CBM team member administered the comprehension and vocabulary subtests of the SORT to students in the first two weeks of the first semester. Students in grade 8 wrote the Stanford Brown Diagnostic Reading Test (4*^ edition) while students in grades 9 and 10 completed the Stanford Blue Diagnostic Reading Test (4*^ edition). The students had 50 minutes to complete the comprehension subtest and 20 minutes for the vocabulary subtest as per the test administration guidelines. The learning assistance teachers involved in the school CBM team scored, recorded, and then filed the SDRT scores, as is normal procedure for the district. Students who were absent during the scheduled testing times were scheduled to write their tests upon their return to school. 25 The oral reading and written expression probes were scheduled in October, January and April to correspond with the probing sessions completed for the 1996 elementary CBM norming project. Members of the school CBM team administered the probes over five consecutive days. The dates for each norming period were planned in consultation with administration, counsellors and teachers. Before the start of the first norming period, a member of the school CBM team visited classes to introduce the students to the purpose of the activity, to describe what they would be asked to do and to outline the organization of the activity. Classes were booked into the library to work on class assignments and during this time, students were randomly called one by one to the upper library mezzanine. In one of the three offices located on the mezzanine, the student was seated across a table from a member of the school CBM team. The team member read the instructions for the procedure to the student (see Appendix A). The student was then given a student copy of the reading probe. The team member waited and started the timing for the one minute reading period when the student began reading aloud. Each reading probe was marked after the student left the room and all readings were taped and double checked by a different member of the school CBM team. Where there was a difference in the marking and scoring of the total words read and/or of the words read correctly, a third member of the team remarked the student reading probe to confirm the correct number of words read correctly. The October probing period determined the order of the reading probes for each student during the study. The reading probes for each grade were administered as described in Table 2. 26 Table 2 Sequence for Administration of Reading Probes Fall Winter Spring 33 students #1 #2 #3 2"'^ 33 students #2 #3 #1 3'^'^ 33 students #3 #1 #2 To insure that all students were tested three times within the probing time available, all teachers of junior secondary courses were scheduled with a time for their students to be probed during a specific block. The written expression probes were administered during the five day norming period. One member of the school CBM team went to each of the grade 8, 9 and 10 classes, as scheduled with classroom teachers, for students to complete the probes. The same team member administered the written expression probes through all three probing sessions and classes were randomly assigned the probes. Instructions were read to the students (see Appendix E) and once the probes were distributed to class members, the school CBM team member read the story starter aloud. Students were timed for one minute to plan their writing and then for three minutes to do their writing. At the end of this time, the students were asked to turn the papers over and the written expression probes were collected. Two school CBM team members marked each probe, with a third marker used when there were differences in scores. Members of the school CBM team scored and recorded the results of the oral reading and written expression probes within the five day period. The computer systems manager printed off final student marks. 27 Ethics School district officials were interested in the study and funded for the release of district and school staff from their duties to participate in collecting the data. The district provided release time for those school CBM team members who required substitutes to cover their classes. SD # 57 approved the study and its methods and, as prior approval was granted by the UNBC Research Ethics Board (Hedekar, 1997), granted the permission to collect the data without written permission of parents. Students would be involved in the probing sessions during class time and engaged in activities that are expected learning experiences in the curriculum. Consent was given to the author to access the data collected for the norming project and subsequent final subject marks. As with the 1996 elementary CBM norming project, the students would be involved in activities that posed no threat or risk. As no possible harm could come to the students, parents were informed of the study through the school newsletter. Parents were given a name and number to contact if they were opposed to having their child participate in the study. All eligible students enrolled in grades 8, 9, and 10 were expected to participate. Students refusing to complete any one of the activities, with a written request from a parent and/or guardian, were withdrawn from the project. The school CBM team members were the only persons who had access to the students’ probe responses and SDRT scores. Results from the SDRT were shared with English teachers, when requested by the teachers, thereby eliminating the need for English teachers to administer the reading test in their classrooms. School CBM team members did not discuss any other information regarding probing or testing data with any student or staff. If a student or parent requested result information, pertaining to themselves or their child, that was provided to them. 28 Once the data had been collected, all information regarding the study and student paper copies were stored in the Learning Center office where all assessments completed during the academic year are stored under lock and key to ensure that only authorized staff has access to the room. Relevant computer data files will be maintained on a school computer located in the Learning Center office, and back up copies will be located in the supervisor’s locked office at UNBC. When data analysis is complete, all student names and identifiers will be deleted. When the study and thesis have been completed, all paper copy information gathered either will be destroyed or will become property of Peter MacMillan, thesis supervisor at UNBC. Data will be retained while the research is being submitted to an academic journal for possible publication. 29 CHAPTER FOUR RESULTS This chapter Is divided into two sections. The first discusses the results of the CBM probe and Stanford Diagnostic Reading test scores and the results relating to student growth in performance in CBM along with corresponding results of inquiries into gender differences. The second section discusses the reliability and validity of the CBM oral reading and written expression measures. The validity discussion reviews the results of analysis relating to the correlation between the CBM probe scores and the Stanford Diagnostic Reading test scores, and the predictive value of the CBM probe scores with student success and student placement. Means and Standard Deviations for CBM Measures Across Grades The means and standard deviations are reported for both genders at each grade level and are in Table 3 for Words Read Correctly (WRC), Table 4 for Total Words Written and Words Spelled Correctly (TWW and W SC), Table 5 for the Stanford Diagnostic Reading Test (SDRT), and Table 6 for the Percentage of Words Read Correctly (%WRC), and Percentage of Words Spelled Correctly (%WSC). The “Total score” represents the data of the entire population within each grade level. There are several contributing factors resulting in the increased number of grade ten students compared to the grade eight or nines. The alternate education settings at the junior level combine grade 9 or 10 students, working on individual education plans, and classify them as grade 10 students. Grade 11 students repeating English 10 were included in the grade 10 sample. As well, although the number of students feeding into PGSS from elementary schools is small, resulting in approximately one hundred students in grade 8 and 9, many parents chose to transfer their children into PGSS for grade 10 so that the student is familiar with the school when entering into their senior years. 30 The gender differences in reading and written expression are contained in Tables 7 and 8. The ranges of the standard deviations for WRC, TW W , and W SC are consistent with those ranges obtained at the upper elementary grades in the elementary project (Hedekar, 1997). WRC mean scores for the grade 9 students are lower than the grade 8 mean scores, indicating a lower reading skill level for the grade 9 student population compared to the reading level of the grade 8 students. To confirm that the grade 8 WRC means are not extremely high or uncharacteristic, the means were compared to the grade 7 WRC means for the three probing sessions as reported by Hedekar (1997). The grade 7 total means scores of 132, 136, 140 compared to the grade 8 scores of 129, 139, 145 show that the scores are representative of the grade 8 population. The mean percentile scores of the SDRT corroborate the findings of the W RC mean scores as the grade 9 percentile means are lower than the grade 8 mean scores. The lower means of the grade 9 population on WRC are therefore not a function of the reading probes. Growth Mean scores for WRC increase over the year for all grades and genders with growth diminishing as students advance to higher grades. Females in grades 8 and 9 show slightly more growth in WRC means than males. This trend is reversed in the grade 10 year as male students demonstrate a slightly higher increase in WRC means than the female grade 10 students. There is no growth between grades as the change in the number of WRC at the grade 8 level accounts for the growth noted between grades 8 to 10 (see Table 3). Mean TW W in grade 8 increases over the year although there was no change in mean scores from the second probing session to the third (see Table 4). There was no growth in the mean number of words written by grade 9 and 10 students over the year. 31 The mean number of WSC within grades shows some growth in grade 8, minimal growth in grade 9 with no growth in grade 10. The mean number of W SC of the grade 10 students does not change and there is minimal growth between the end of the grade 8 year and the end of the grade 10 year. Whereas the mean TW W for females is higher than the males in all three grades, the mean TW W of males in grades 8 and 9 increase slightly more when compared to the means of their female counterparts. The mean percentage ranges for WRC for all grades range from 95 to 98%. There is no growth in %WRC over the year for each grade and the mean %WRC shows no increase from grade to grade (see Table 6). The mean percentage ranges for %WSC range from 94% to 99% (see Table 6). There is a slight increase in %WSC in grade 8 to ten with no growth within each grade. Results of the WRC, TW W , and W SC probes are illustrated in Figures 1, 2 and 3. The best-fit lines, labeled “linear”, assist in displaying the trend in the data by taking each student score in each grade and drawing a line that best fits (linear regression) through these points. The trend lines clearly show that most of the growth in the probe results is accounted for by the increase of students in grade 8. Growth diminishes as students advance to higher grades with minimal growth between grades. Gender Differences The results of t - test analysis indicate that the means for W RC are consistently higher for the female students in all probing sessions through the three grades (p < .05). The one exception is the grade 8 W RC in the October probing session ( t =^ .786, off = 102, p = .077). The trend of females outperforming males established in the elementary project, as reported in Hedekar’s study (1997), extends into the junior high school. This trend on the CBM measures was not consistent with the SDRT results (see Table 5) where analysis of the variation shows that there is no significant 32 difference between male and females on the SDRT comprehension and vocabulary subtests (p > .05) in all grades. The one exception is the grade 10 results on the comprehension subtest where the females outperformed the males (f= 2.874, df = 186, p = .005). The mean WRC percentage ranges from 95 - 98; see Table 6. There are no differences in gender results in the grade 8 population through the three probing periods (p > .05). The October and January probing sessions produced no %WRC score differences between males and females in grade 9 but female %WRC scores were significantly higher than the males (f= 2.085, df = 87, p = .04) in the April probing session; see Table 6. The grade 10 %WRC scores of females were significantly higher than those of the males in the first two probing sessions, with no differences in gender noted in the April probing session (t= 1.725, df= 169, p = .09). The %WSC results (Table 6) show no gender difference in grade 8 with females in grades 9 and 10 scoring significantly higher than the males in all three probing sessions. Overall, the females outscored the males in %WRC and %WSC results. The gender differences are expressed in Cohen’s d effect sizes in Tables 7 and 8. The WRC are generally of medium size { d= 0.5 - 0.6) with a small effect of of = 0.34 in the grade 8 fall session. In contrast, the SDRT sizes range from trivial {d = 0.10) to small {d= 0.33). The written expression sizes are also generally of medium size but slightly larger than the W RC with 0.5 to 0.7 for TW W and WSC. The four exceptions are large (0.5 - 0.9) effect sizes in the fall and spring sessions. 33 Table 3 Reading Fluency (WRC) Scores for Each Testing Period by Grade and Gender October_______________ January_______________April Grade and Gender M SD n M SD n M SD n Females 135 39 58 147 43 55 154 41 51 Males 121 43 46 128 36 42 136 35 45 Total 129 41 104 139 41 97 145 39 96 Females 139 37 45 143 40 42 146 37 41 Males 117 41 55 122 45 51 122 43 48 Total 127 40 100 132 43 93 133 42 89 Females 157 43 100 159 42 77 160 40 73 Males 132 40 115 139 44 90 139 39 98 Total 144 43 215 149 44 167 148 41 171 Grade 8 Grade 9 Grade 10 34 Table 4 Written Expression Fluency (TW W and WSC) Scores for Each Testing Period by Grade and Gender October Grade and Gender Grade 8 January April M SD n M SD n M SD n Females TW W W SC 60 58 15 15 56 69 66 13 13 47 69 66 17 19 49 Males TW W W SC 47 45 12 13 38 57 54 19 19 41 56 54 13 14 40 Total TW W WSC 55 53 15 15 94 63 60 17 17 88 64 60 16 18 89 Females TW W WSC 66 62 16 16 37 67 66 16 16 41 68 66 16 15 30 Males TW W WSC 52 49 19 18 42 53 51 18 18 41 55 54 18 16 42 Total TW W WSC 57 55 19 18 79 60 58 19 19 82 61 59 18 17 72 Females TW W WSC 71 69 17 19 63 73 71 16 16 69 70 69 16 16 62 Males TW W W SC 61 59 18 18 71 64 62 17 17 60 62 60 16 17 68 Total TW W WSC 66 64 18 19 134 69 67 17 17 129 66 64 17 17 130 Grade 9 Grade 10 35 Table 5 Mean Percentile Scores on SDRT by Grade and Gender Comprehension Vocabulary Grade and Gender Grade 8 M SD n M SD n Females 42 28 56 41 30 57 Males 36 29 43 38 30 43 Total 39 29 99 40 30 100 Females 39 24 43 32 28 43 Males 31 24 42 30 27 42 Total 35 24 85 31 27 85 Females 53 28 90 54 32 90 Males 41 26 99 52 30 99 Total 47 28 189 53 31 189 Grade 9 Grade 10 36 Table 6 Mean %WRC and %WSC for Each Testing Period by Grade and Gender October Grade and Gender Grade 8 January April M SD M SD M SD Females %WRC %WSC 97% 96% 4 5 97% 96% 4 4 97% 95% 4 5 Males %WRC %WSC 97% 95% 3 5 97% 94% 3 6 97% 95% 2 6 Total %WRC %WSC 97% 96% 4 5 97% 95% 4 5 97% 95% 4 5 Females %WRC %WSG 96% 98% 4 3 96% 98% 4 2 97% 97% 3 2 Males %WRC %WSC 95% 95% 6 7 95% 96% 6 5 95% 95% 7 6 Total %WRC %WSC 95% 96% 5 5 95% 97% 5 4 96% 96% 6 5 Females %WRC %WSC 98% 99% 3 2 98% 98% 2 2 98% 98% 3 2 Males %WRG %WSG 96% 97% 4 5 96% 97% 4 3 97% 97% 4 4 Total %WRG %WSG 97% 98% 4 4 97% 98% 4 3 97% 97% 3 3 Grade 9 Grade 10 37 Table 7 Cohen’s d Effect Size - Reading Measures WRC Session F M SD Effect Size Grade 8 Fall Winter Spring 135 147 154 121 128 136 41 41 39 0.34 0.46 0.46 small medium medium Grade 9 Fall Winter Spring 139 143 146 117 122 122 40 43 42 0.55 0.49 0.57 medium medium medium Grade 10 Fall Winter Spring 157 159 160 132 139 139 43 44 41 0.58 0.45 0.51 medium medium medium Grade 8 comp vocab 42 41 36 38 29 30 0.21 0.10 small trivial Grade 9 comp vocab 39 32 31 30 24 27 0.33 0.07 small trivial Grade 10 comp vocab 53 54 41 52 28 31 0.43 0.06 small trivial SDRT d < .20 trivial .20 - .49 small .50 - .79 medium d > .79 large 38 Table 8 Cohen’s d Effect Size - Written Expression TWW Session F M SD Effect Size Grade 8 Fall Winter Spring 60 69 69 47 57 56 15 17 16 0.87 0.71 0.81 large medium large Grade 9 Fall Winter Spring 66 67 68 52 53 55 19 19 18 0.74 0.74 0.72 medium medium medium Grade 10 Fall Winter Spring 71 73 70 61 64 62 18 17 17 0.56 0.53 0.47 medium medium medium Grade 8 Fall Winter Spring 58 66 66 45 54 54 15 17 18 0.87 0.71 0.67 large medium medium Grade 9 Fall Winter Spring 62 66 66 49 51 54 18 19 17 0.72 0.79 0.71 medium large medium Grade 10 Fall Winter Spring 69 71 69 59 62 60 19 17 17 0.53 0.53 0.53 medium medium medium WSC d < .20 trivial .20 - .49 small .5 0 - .79 medium d > .79 large 39 Grade 8, 9 ,1 0 Means W ords Read Correctly 160 & 140 E 120 100 Fall Winter Probing Sessions Spring Grade 8 Grade 9 Grade 10 Linear (Grade 8) Linear (Grade 9) Linear (Grade 10) Figure 1. Grades 8, 9 and 10 W RC means and best-fit lines. 40 Grade 8 ,9 , 10 Means Total Words Written g I g I 60 Ï o 50 Fall Winter Spring Probing Sessions —♦—Grade 8 Grade 9 Grade 10 Linear (Grade 8) Linear (Grade 9) Linear (Grade 10) Figure 2. Grades 8, 9 and 10 TW W means and best-fit lines. 41 Grade 8, 9, 10 Means W ords Spelled Correctly .5) and significant (p < .01) correlations with the comprehension and the vocabulary subtests of the SDRT (Table 10). The WRC scores are more closely related with the comprehension subtest than the vocabulary subtests with the strongest relationship between W RC and comprehension scores in grade 8 (r= .72, .74, .71). These results demonstrate the same relationships between reading fluency and comprehension versus vocabulary as found by Shinn, Good, Knutson, Tilley & Collins (1992). The %WRC and SDRT correlation scores are significant (p < .01) but smaller than the W RC correlations with comprehension and vocabulary (r= .36 - .51) Analysis of WRC and %WRC with the total junior high student population shows significant (p < .01) relationships with the comprehension (r= .61 - .66) and vocabulary subtests (r= .40 - .44) of the SDRT. The correlations between the CBM written expression probe results and the SDRT subtests are presented in Table 11. The TW W probe scores show no correlation with either subtest of the SDRT except at the grade 8 level where there are small but significant correlations (p < .05) between October TW W and comprehension (r= .21), October TW W and vocabulary (r= .20) and April TW W and the vocabulary subtest (r= .22). The same pattern with TW W exists in W SC with the only significant correlations found at grade 8 in the October and April probing sessions. The relationship between the SDRT subtests and the October W SC scores is significant (p < .05) with comprehension (r= .25) and with vocabulary {r= .22). The W SC scores in the April probing session were significantly correlated with both comprehension (r= .27, p < .05) and vocabulary (r = .28, p < 01). The strongest relationships between the CBM written expression and the SDRT exist with the %WSC. All %WSC correlations are significant (p < . 01) except for the grade 9 January probes and vocabulary scores (r= .26, p < 44 .05) and the April probes which showed no correlation. Except for the October %WSC and vocabulary correlations at the grade 8 and 10 level, all probe scores had a medium correlation with the SDRT (/s ranging from .30 - .51). An interesting note is the correlations that exist in the data when the students are not split into grades. All written expression analyses have significant (p < .01) relationships with the SDRT. The exceptions are no correlations between the January TW W probe results with comprehension and the April TW W scores with both SDRT subtests, and small but significant (p < .05) coefficients between January TW W and vocabulary (r= .13), January W SC and comprehension (r= .15) and April W SC and the comprehension subtest (r= .16). The %WSC correlation coefficients are larger (.32 .39) than TW W (.15 - .17) and W SC (.17 - .21). 45 Table 9 Correlations - Probing Sessions Words Read Correctly (WRC) January April October January OctoberApril r n r n r n Grade 8 .91** 92 .93** 87 .93** 87 Grade 9 .92** 87 .91** 81 .90** 81 Grade 10 .63** 151 .59** 135 .55** 146 Grade 8 .61** 78 .67** 76 .63** 77 Grade 9 .63** 70 .64** 60 .72** 57 Grade 10 .65** 106 .74** 89 .63** 89 Total Words Written (TWW) Words Spelled Correctly (WSC) Grade 8 .63** .70** .66** Grade 9 .63** .63** .73** Grade 10 .66** .76** .64** N.B. n’s for TW W and W SC are the same, therefore not repeated. .1 < r < .3 small .3 < r < .5 medium r> .5 large **p < .01 46 Table 10 Relations between CBM Oral Reading and SDRT (Comprehension and Vocabulary) WRC Grade 8 Grade 9 Grade 10 Total (n = 96) (n = 76) (n = 169) (n = 341) .72** .68** .62** .59** .63** .57** .66** .61** (n = 9^) (n = 69) fn = 132) (n = 292) .74** .68** .65** .63** .56** .55** .64** .62** (n = 85) (n = 66) (n = 127) (n = 278) Comprehension Vocabulary .71** .66** .66** .57** .63** .61** .66** .62** Comprehension Vocabulary .42** .43** .51** .46** .40** .40** .43** .41** Comprehension Vocabulary .37** .44** .48** .45** .40** .41** .41** .44** Comprehension Vocabulary .36** .43** .46** .37** .41** .46** .40** .42** October Comprehension Vocabulary January Comprehension Vocabulary April %WRC October January April N.B. n’s for WRC and %WRC are the same therefore not repeated. .1 < r < .3 small .3 < r < .5 medium r > . 5 large * * p < .01 47 Table 11 Relations Between CBM Written Expression and SDRT (Comprehension and Vocabulary) TW W G rades Grade 9 Grade 10 Total r n r n r n r n October Comprehension Vocabulary .21* .20* 91 .12 .10 64 .12 .05 120 .17** .15** 275 January Comprehension Vocabulary .10 .12 84 .12 .19 63 .03 .00 109 .10 .13* 256 April Comprehension Vocabulary .20 .22* 79 .00 .07 55 .02 -.01 96 .10 .12 230 October Comprehension Vocabulary .25* .22* .17 .15 .15 .15 .21** .19** January Comprehension Vocabulary .15 .17 .15 .20 .06 .03 .15* .17** April Comprehension Vocabulary .27* .28** .03 .09 .07 .06 .16* .18** October Comprehension Vocabulary .32** .27** .51** .44** .30** .25** .36** .32** January Comprehension Vocabulary .34** .35** .36** .26* .39** .34** .36** .32** April Comprehension Vocabulary .39** .43** .27 .18 .39** .44** .38** .39** W SC %WSC N.B. n’s for TW W , W SC and % W SC are the same, therefore not repeated .1 < r < .3 small .3 < r < .5 medium r> .5 large *p < .05 **p < .01 48 Predictive and Discriminant Validity for CBM Measures Across Grades CBM Validity in Relation to Success in Core Academic Courses. To investigate the ability of CBM scores to predict student achievement (final percentages received in courses completed), scores from the CBM oral reading and written expression probes, along with the results on the SDRT comprehension and vocabulary subtests, were correlated with the final percentage marks in Social Studies 11 and either English 12 or Communications 12 (Tables 12 and 13). Correlations with Advanced English Placement (IB) are not included in the analyses, as there were grades available for only six students. The WRC probe results were generally all significant when correlated with English 12 and Social Studies 11 whereas %WRC results were not. The predictive validity coefficients indicate a medium (.40 - .48) and significant (p < .01) correlation between WRC and English 12 (see Table 12). The correlations between WRC and English 12 are smaller than the correlation between English 12 and the SDRT comprehension subtest (r= .50), but the WRC probe results are more strongly related to English 12 than the vocabulary subtest (r= .31). For Socials Studies 11, the WRC scores from all three probing sessions have a smaller (.21 - .27) but significant relationship (p < .01) with SS 11 final marks. The SDRT comprehension subtest has a small (r= .25) relationship with Socials Studies 11 and there is no significant relationship between the marks in this course and the vocabulary subtest. With %WRC, there is only one significant relationship (p < .05), which is with Social Studies 11 (r= .16). There is no significant correlation with the final marks in Communications 12 and either of the oral reading analyses or the SDRT subtests. Overall, W RC scores were better than the SDRT in predicting course marks in Social Studies 11 and slightly lower than the test in predicting English 12. 49 For the written expression results, all three analyses yielded significant results with English 12. There are medium correlations between final marks in English 12 and TW W (r median of .31), W SC (r median of .34) and %WSC (r median of .30) scores in the October and January probing sessions (p < .01 ; see Table 13). These results are smaller than the correlations of the comprehension subtest with English 12 (r = .50) marks but higher than the respective vocabulary correlations (r= .31). The only other relationships noted with English 12 final marks are small but significant (p < .05) correlations between both the April TW W and W SC scores (.28 and .27 respectively). With the Social Studies 11 final marks, the April TW W {r= .24) and W SC (r= .26) scores were the only significant relationships noted (p < .01). These correlations are the same as the correlations between the SDRT comprehension subtest and the final marks in SS 11. As with the oral reading probes, the written expression scores show no relationship with final marks in Communications 12. In all instances, CBM written expression scores were less able to predict course marks when compared to the comprehension subtest but either better or the same as the vocabulary. CBM Validity in Relation to “At Risk” or Advanced Designation. Correlations between the CBM probe results and identification of students as regular, at risk, or advanced are reported in Tables 14 and 15. Students enrolled in regular classes at the appropriate grade level with no access to remedial or academic support services were classified as “regular” students. Those students enrolled in remedial, support settings, alternate setting, or repeating English 10 were classified as at risk students, while students enrolled in IB were designated as “advanced” students for the purposes of this study. Negative correlations are possible with this analysis, which involves biserial correlations, as the variable for class membership or designation is dichotomous (0,1). Students who perform poorly on the measures are likely at risk 50 students so it is expected that their scores on the measures would yield negative correlations. The reading fluency results obtained indicate that all correlations with designation as a regular or at risk student are significant (Table 14). Coefficients were significant for W RC and students considered at risk and advanced as they were for %WRC with identification as a regular or at risk student. In predicting a student as regular, at risk, or advanced, there are small but significant (p < .01) correlations between WRC probe results and placement in a regular (r ranging from .17 - .26) or advanced class (r ranging from .25 - .29). The correlations between W RC and regular class placement are higher than the correlation between the comprehension subtest (r = .11, p < .05) and regular classes. The relationship between W RC and membership in advanced classes is the same as with the comprehension subtest (r= .29, p < .01). The correlations between WRC and designation as an at risk student are medium and significant (r ranges from .30 - .41). These correlations are higher than both the comprehension and vocabulary subtests (r= -.27, r = -.24, p < .01). The %WRC results show low and significant (p < .01) correlations with regular (r median of .21) and at risk class placement (r median of -.28). The %WRC and regular student correlations are higher than the comprehension (r= .11, p < .05) and vocabulary (.04) subtest correlations. The at risk correlations with %WRC are the same as the comprehension and higher than the vocabulary (r= -.24, p < .01). The SDRT comprehension and vocabulary subtests both were more highly correlated with advanced class placement (.29, .37, p < .01) than WRC (r median .27). The January %WRC scores had a small and significant relationship with student membership in advanced classes. The correlation between %WRC scores and students classified as at risk are slightly smaller than WRC and at risk students, but stronger than the relationship between W RC and 51 regular class designation. Overall, WRC was the same as the SDRT in identifying an advanced student and both reading probe analyses were better at predicting the identification of a regular or at risk student than the SDRT. Relationships between the three written expression indices and identification of a student as at risk or advanced are significant, as are all correlations with %WSC. TWW have small and significant (p < .01) correlations with students designated as at risk (-.21 to -.22) in all three probing periods and in the October (.21) and January (.16) sessions for advanced students (see Table 15). A coefficient of r= .13 (p < .05) is found with regular student placement and TW W results in the April session. The correlations between W SC and at risk designation are similar but slightly higher than those found with TW W . In addition, regular student results were correlated with the January W SC (r = .12, p < .05) as were advanced results in the April probing session (r= .13, p < .05). The relationships between %WSC indices and all designations are significant (p < .01) and generally larger than the correlations with TW W and WSC. The largest correlations are with at risk students (-.27, -.29, -.29) with the regular student indices of r = .16 to .19 and advanced .15 to .18. In comparison with the SDRT, TW W are the same or better as predictors of regular student designation and W SC are better at predicting this designation. %WSC generates higher correlations and is better than the SDRT in identifying regular and at risk students. CBM Validity in Relation to Program Placement. Further analysis into student placement includes the investigation into the correlation between the CBM probes and students placed in regular classes, the International Baccalaureate (IB) program. Skills Support, the Pre-Employment Program (PEP), Reception or Transition 8. There are significant correlations (p < .01) between all CBM oral reading results and the placement of students in program categories 52 (Tables 16 and 17). The only exceptions are October and April %WRC with IB placement, October W RC and April %WRC with Reception (p < .05). Both of the oral reading analyses result in small correlations with program placements except for with PEP. The W RC and %WRC variables demonstrate medium correlations (r median of .30 and -.31) with PEP placement and the coefficients are larger than those obtained with comprehension (r= -.24, p < .01) and vocabulary (r= -.25, p < .01). All correlations with WRC are larger than the comprehension and vocabulary subtests except for when correlated with placement in IB. The same applies to %WRC except for in addition to smaller correlations with IB; there are also slightly smaller correlations with Transition 8 (r= -.13). Both the W RC and %WRC indices are better at predicting placement in programs other than IB. The written expression analyses of TW W and WSC produced fewer significant (p < .01) correlations with program placement than the %WSC analysis. All IB, Skills and Transition 8 written expression indices are significant. All %WSC correlations are significant with the exception of Reception. This could be accounted for by the fact that students placed in Reception have varied skill and ability levels and are not as homogenous as the IB, Skills and Transition 8 groupings. Placement in regular classes has a small and significant (p < .05) relationship with TW W in January and April (.11 and .13) and W SC in January (.12). Small correlations with p values < .01 are in all three probing sessions with %WSC (.16, .16, and .19). The TW W and WSC correlations are the same as those found with the comprehension (.09, p < .05) and vocabulary (.04). The %WSC coefficients are significant (p < .01) and larger than the SDRT subtests. The analysis with IB membership generates a greater number of significant (p < .01) relationships than the analysis with regular class placement. All probe correlations are small and significant (p < .01) excluding April’s TW W and WSC 53 (p < .05). Compared with the Stanford comprehension and vocabulary correlations with IB placement {r= .31 and .37), all of the written expression analyses are lower. As with the analyses of placement in regular and IB classes, the coefficients of Skills Support placement with written expression probe results are small. The October and April correlations for all three analyses have a significance of p < .01 whereas the January correlations are all p < .05. Overall, all of the three written expression analyses are better than the SDRT in predicting placement in Transition 8 and regular classes. %WSC results are better at the prediction of placement in PEP than the SDRT and the same as the SDRT in predicting Skills membership. 54 Table 12 Relations between CBM and SDRT Reading Measures with Final Course Mark Percentages English 12 Communications 12 Socials Studies 11 SDRT n r n n Comprehension (percentiles) .50** 76 -.14 21 .25** 148 Vocabulary (percentiles) .31** 77 -.07 21 .13 148 October .44** 84 -.04 22 .27** 160 January .40** 78 .02 24 .21** 135 April .48** 83 -.21 22 .27** 146 WRC %WRC October .17 -.08 .16* January .21 -.11 .07 April .18 -.22 .11 N.B. n’s for TW W , W SC and % WRC are the same, therefore not repeated .1 < r < .3 small .3 .5 large *p < .05 **p < .01 55 Table 13 Relations between CBM Written Expression with Final Course Mark Percentages English 12 Communications 12 Socials Studies 11 TWW r n r n r n October .31** 77 -.36 17 .15 123 January .40** 75 -.41 20 .12 120 April .28* 71 -.32 19 .24** 119 W SC October .34** -.33 .15 January .42** -.41 .12 April .27* -.32 .26** October .30** -.01 .10 January .32** -.35 .06 April .04 -.09 .14 %WSC N.B. n’s for TW W , W SC and % W SC are the same, therefore not repeated .1 < r< .3 small .3 < r < .5 medium r > .5 large *p < .05 * * p < .01 56 Table 14 Relations between CBM and SDRT Reading Measures with “At Risk” and Advanced Placement SDRT n = 372 Regular At risk Advanced Comprehension (percentiles) .11* -.27** .29** .04 -.24** .37** October .17** -.30** .25** January .19** -.36** .29** April .26** -.41** .27** October .21** -.26** .09 January .20** -.28** .14** April .22** -.29** .10 Vocabulary (percentiles) WRC %WRC .1 < r < .3 small *p < .01 ** .3 < r < .5 medium r > .5 large p < .05 57 Table 15 Relations between CBM Written Expression with “At Risk” and Advanced Placement TWW Regular At risk Advanced October .07 -.21** .21** January .11 -.22** .16** April .13* -.20** .11 October .09 -.23** .22** January .12* -.25** .18** April .15* -.23** .13* October .17** -.27** .17** January .16** -.29** .18** April .19** -.29** .15** WSC %WSC .1 < r< .3 small *p < .05 ‘*p < .01 .3 < r< . 5 medium r > .5 large 58 Table 16 Relations between CBM and SDRT Reading Measures with Program Placement SDRT n = 371 Reg IB Skills PEP Rec T8 %ile C .09* .31** -.21** -.24** -.10* -.15** %lle V .04 .37** -.21** -.25** -.13** -.17** October .17** .25** -.28** -.28** -.11* -.20** January .19** .29** -.28** -.30** -.15** -.16** April .26** .27** -.29** -.34** -1 2 ** -.18** October .21** .09* -.26** -.29** -.14** -.13** January .20** .14** -.27** -.32** -.16** -.13** April .22** .10* -.31** -.31** -.10* -.13** WRC %WRC N.B. Reg - Regular .1 < r < .3 small * p < .05 * * p < .01 Rec - Reception .3 < r < .5 medium T8-Transition 8 r > .5 large 59 Table 17 Relations between CBM Written Expression with Program Placement TTW Reg IB Skills PEP Rec T8 October .07 .21** -.16** -.05 .03 -.20** January .11* .16** -.11* -.08 -.12* -.17** April .13* .11* -.15** -.07 -.01 -.17** October .09 .22** -.16** -.08 .02 -.21** January .12* .18** -.12* -.12* -.13* -.21** April .15** .13* -.17** -.12* -.01 -.22** October .16** .17** -.24** -.28** -.06 -.19** January .16** .18** -.11* -.31** -.09 -.43** April .19** .15** -.22** -.42** -.02 -.40** W SC %WSC N.B. Reg - Regular .1 < r < .3 small *p < .05 **p < .01 Rec - Reception .3 < r < .5 medium T8 - Transition 8 r > .5 large 60 CHAPTER FIVE DISCUSSION AND CONCLUSIONS This chapter Is divided into five sections. The first four sections consider the norming, concurrent, predictive and discriminant validities of the CBM measures. The last section discusses the implications of the study. Norming Analysis of the oral reading and written expression results indicate that there is growth in WRC, TW W and W SC at the grade 8 level, an extension of the trend at the elementary level (Hedekar, 1997). The mean W RC scores increase over the year for both sexes in all grades, but with the rate of increase diminishing in higher grades. There is minimal growth in reading and writing fluency within grades 9 and 10 and no growth between grades. As there is minimal to no growth in oral reading and written expression means between grades, a ceiling effect occurs after grade 8. These results indicate that there is no growth in reading or writing fluency in grades 9 and 10. The leveling off of reading and writing skill development after grade 8 corroborates Deshler’s research, stating that the focus of instruction at the secondary level moves away from the development of reading and writing skills to the use of these skills for knowledge acquisition. Overall, the findings indicate that the gender differences in the CBM reading and writing fluency reported by Hedekar (1997) are maintained at the junior secondary level with females continuing to outperform males at all grade levels. Gender differences (smaller than those in the CBM results) also exist in the reading scores as measured by the comprehension and vocabulary subtests of the Stanford Diagnostic Reading Test. Concurrent Validity of CBM Measures The results obtained in this study demonstrate that the CBM oral reading probe results are highly correlated with both subtests of the SDRT. These results confirm the 61 prior research completed by Dene, Mirkln & Chiang (1982) showing the strong relationship between the oral reading CBM measures and standardized normreferenced tests. Shinn, Good, Knutson, Tilley & Collins’ (1992) results, which reported that reading fluency is more strongly correlated with comprehension subtests than with vocabulary subtests, is corroborated by this study. The WRC scores are also more strongly correlated with the SDRT subtests than the %WRC scores. The strength of the WRC and %WRC correlations with the SDRT support Deno and Marston’s previous findings that CBM measures are more valid assessment instruments than standardized norm-referenced tests. This study proves that the CBM oral reading probes are valid reading instruments at the junior high school in grades 8, 9 and 10. The relationship between writing fluency and the results on the standardized norm-referenced reading test shows that CBM written expression is related to reading on the SDRT comprehension subtest at the grade 8 level regardless of the level of analysis used. The more complex analysis of % W SC is a better measure than TW W and WSC and is moderately correlated, at all grade levels, with both the comprehension and vocabulary subtests of the SDRT. The data obtained in this study shows that a student’s written fluency, analyzed by more complex indices, is a valid indicator of reading ability. As with prior research, it was discovered that the reliability and validity coefficients of the CBM oral reading measures were higher than the coefficients with the written expression measures. The results of this study show that the validity and reliability of the CBM probes measuring reading and writing fluency at the elementary school level extends to the junior secondary school level. These results refute prior statements made in the literature that the CBM reading and writing measures are too simplistic, and therefore, inappropriate for use at the junior secondary school level. 62 Predictive Validity of CBM Measures In predicting academic performance, W RC scores are as good as the SDRT in predicting marks in English 12 and better than the SDRT in predicting SS 11 marks. All three written expression analyses are the same as the SDRT in predicting marks in English 12. The TW W and WSC scores are the same as the comprehension subtest and better than the vocabulary subtest in predicting SS 11 marks. The more complex analysis of %WRC yields no relationships with courses and the only relationship that exists with %WSC is with English 12. These results indicate that the simpler analyses of reading and writing fluency (WRC, TW W , and WSC) are valid predictors of final marks in English 12 and SS 11. The results of the written expression analyses found in this study contradicts Parker, Tindal & Hasbrouck’s (1991) conclusions that percent indices (productionindependent) of writing are stronger predictors of student performance than “number of” indices (production-dependent). My research does confirm Fewster and MacMillan’s (2002) findings that the WRC and W SC scores of upper elementary students are valid predictors of marks in English and Social Studies 10 and of the two, oral reading was a stronger predictor. Discriminant Validity of CBM Measures In screening students as regular, at risk, or advanced, both the WRC and %WRC analyses of the oral reading probe scores are better than the SDRT for predicting students as regular or at risk, whereas the SDRT subtest results are better predictors of advanced designation than either the oral reading or written expression probe scores. The W RC scores are valid predictors of regular class placement and the placement of students with learning difficulties. Overall, %WRC is a more accurate analysis in differentiating low achieving students from regular students. 63 Written expression results follow the same pattern as the reading results with all the writing indices being as good as or better than the SDRT in differentiating regular versus at risk students. Of the three writing indices, %WSC demonstrates the strongest correlations with student designation. This study supports prior research by Fewster and MacMillan (2002) and Marston, Mirkin & Deno (1984) that found that CBM oral reading and written expression probes are valid tools for differentiating students as regular, low achieving or high achieving. In predicting more specific placement of students, this study found that CBM oral reading scores (both WRC and %WRC) are as good as or better predictors of student placement in regular classes, Skills Support, PEP, Reception and Transition 8 classes than the SDRT comprehension and vocabulary subtests. Once again, the reading fluency results are better than the SDRT in identifying regular, Skills Support, PEP and Reception students, but the SDRT is better in predicting IB placement. All the reading scores (SDRT and CBM) appear to have small correlations with Reception and there are no significant correlations noted between placement in Reception and all the written expression probe scores. These results may be due to the fact that the students placed in Reception are not homogeneous as there are differing skill ranges and learning potentials in the students placed for support in this setting. Analysis of the written expression probe results indicate that TW W and WSC are the same as the SDRT in predicting regular. Skills Support, and Transition 8 placement. The %WSC results are more accurate in predicting placement in regular class. Skills Support, PEP, and Transition 8 than TW W and WRC and are better than the SDRT in predicting regular. Skills Support, PEP, and Transition 8. Written expression scores screen accurately for placement of students in regular. Skills Support, PEP, Reception 64 and Transition 8 classes. Overall, the %WRC correlations are greater than WRC and the %WSC are greater than either TW W or WSC. Implications for Practice This study has investigated the usefulness of CBM oral reading and written expression probes at the junior secondary setting. The reading and writing fluency scores are valid and reliable predictors of student success in core academic courses and are effective in screening students who require additional support or placement in an alternative setting. The CBM measures have a ceiling effect that would limit the use of the probes to measure the growth of reading and writing fluency in the general population in grades 9 and 10. At the grade 8 level, the CBM measures are effective and reliable in detecting growth in students as well as documenting the effectiveness of interventions provided to students requiring extra support at this grade. The measures would be appropriate for use with students at the junior school level with lower skill levels (skill levels ranging from grades 1 to 8) who require remedial support or alternative classroom placement. The CBM oral reading and written expression probes can provide the documentation of student development and the effectiveness of instruction and curriculum provided by educators. The gender differences at the junior secondary level in oral reading and written expression skills may give educators an opportunity to provide learning opportunities to males to increase their skills in order to increase their ability to interact with their curriculum. The impact would be greatest if the interventions occurred in the earlier school years as growth in grades 9 and 10 levels off. The reliability and validity of the CBM oral reading and written expression scores and their strong relationship to the Stanford Diagnostic Reading Test offers educators an alternative to the use of costly and labor intensive standardized norm-referenced 65 reading tests. The SDRT is an instrument that teachers feel comfortable working with to provide them information on students in their classes. With training and experience in administering and interpreting the CBM oral reading and written expression probe results, educators can comfortably use CBM to identify students requiring additional support or possibly an alternative program placement. The validity of the CBM measures in predicting English 12 and Social Studies 11 may assist students and parents in realizing the importance of oral reading and written expression skills to future course success and to graduation. As with any assessment tool, caution is required in using the CBM measures as the only piece of information for educational decision making. The oral reading and written expression probe results are “snap-shots” of student ability on any given day. For effective decision making to occur, student history, teacher observations, student and parent input are all necessary requirements. Recommendations This study has demonstrated that the CBM oral reading and written expression measures are valid assessment tools for use in a junior secondary school setting. Based on the results of this study, I would make the following recommendations: 1. The use of the CBM oral reading and written expression probes, as part of the Problem Solving Model used at the elementary level, be extended into the junior secondary setting at the Grade 8 level. The extension of the Problem Solving Model into grade 8 would facilitate communication from the feeder schools to the receiving junior high school and would provide consistency to students and their parents, already familiar with the CBM probing process as a result of their exposure to CBM at the elementary level. At the grade 8 66 level, educators can use the CBM oral reading and written expression probes to identify students requiring extra support and measure their growth through the grade 8 year. 2. The use of the CBM oral reading and written expression probes with students who have lower reading and writing fluency and are placed In remedial settings. The use of the results from the CBM probes can assist teachers and students In setting goals for Individual Education Plans. The outcomes would be easily measurable and may provide students with motivation to improve their oral reading and written expression scores. 3. SD #57 complete a norming study at the junior secondary level to provide educators with a guidebook for the use of the CBM measures and the norming statistics to assist them In making educational decisions based on normed data. 67 REFERENCES Brown, J. (1981). Superbike! Toronto, Canada: General Paperbacks. Deno, S. L. (1989). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52(3), 219-232. Deno, 8. L., Mirkin, P., & Chiang, B. (1982). Identifying valid measures of reading. Exceptional Children, 49, 36-45. Deshler, D. D., Schumaker, J. B., Lenz, K., Bulgren, J. A., Hock, M. P., Knight, J. et al. (2001). Ensuring content-area learning by secondary students with learning disabilities. Learning Disabilities Research and Practice, 16{2), 96-108. Elliott, S. N., & Fuchs, L. S. (1997). The utility of curriculum-based measurement and performance assessment as alternatives to traditional intelligence and achievement tests. School Psychology Review, 26(2), 224-233. Espin, C. A., & Deno, S. L. (1993). Performance in reading from content area test as an indicator of achievement. Remedial and Special Education, 14(6), 47-59. Espin, C. A., & Foegen, A. (1996). Validity of general outcome measures for predicting secondary student’s performance on content-area tasks. Exceptional Children, 62(6), 497-514. Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.), Advanced applications of curriculum-based measurement (pp. 214-253). New York: Guilford Press. Fewster, S., Fortier, P., Foulds, B. J., MacMillan, P. D., Struthers, L., & Walraven, G. (2003). Curriculum-Based Measurement and dynamic indicators of basic early literacy skill in School District #57 Guidebook. Prince George, Canada: School District 57 (Prince George). 68 Fewster, S., & MacMillan, P. D. (2002). School-based evidence for the validity of Curriculum-based measurement of reading and writing. Remedial and Special Education, 23(3), 149-156. Good, R. H., & Jefferson, G. (1998). Contemporary perspectives on curriculum-based measurement validity. In M. R. Shinn (Ed.), Advanced appiications of curricuium-based measurement {pp. 6^-88). New York: Guilford Press. Hedekar, L. (1997). The effects of month of birth and gender on elementary reading and writing fluency scores using Curricuium Based Measurement. Master’s thesis. University of Northern British Columbia, Prince George, British Columbia, Canada. Hinton, S. E. (1967). The outsiders. New York: Dell Publishing. Jones, E., Wilson, R., & Bhojwani, S. (1997). Mathematics Instruction for Secondary Students with Learning Disabilities [Electronic version]. Journal of Learning Disabilities, 3 0 ,151-163. Karlsen, B., & Gardner, E. F. (1996). Stanford Diagnostic Reading Tesf (4**^ ed). San Antonio: Harcourt Brace & Company. MacMillan, P. D. (2000). Simultaneous measurement of reading growth, gender, and relative-age effects: Many-facet Rasch measurement applied to CBM reading scores. Journal of Applied Measurement, 1(4), 393-408. Marston, D. B. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it? In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp. 18-78). New York: Guilford Press. 69 Marston, D., & Deno, S. (1981). The reliability of simple, direct measures or written expression. (Research Report No. 50). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities. Marston, D., & Magnusson, D. (1988). Curriculum-based measurement: District level implementation. In J.L. Graden, J.E. Zins, & M. J. Curtis (Eds.), Alternative educational delivery systems: Enhancing instructional options for all students (pp. 137-172). Washington, DC: National Association of School Psychologists. Ministry of Education. (1996) School information profiie for school year 1995/96. Victoria, Canada: Author. Ministry of Education. (1995) Special education sen/ices’ manual of policies, procedures and guideiines. Victoria, Canada: Author. Ministry of Education, Skills and Training. (1996) School information profile for school year 1995/96: Communicating accomplishments of British Columbia students. Victoria, Canada: Author. Mothus, T. G. (1997). The effects of strategy instruction on the reading comprehension achievement of junior secondary school students. Master’s thesis. University of Northern British Columbia, Prince George, British Columbia, Canada. Parker, R., Tindal, G., & Hasbrouck, J. E. (1991). Progress monitoring with objective measures of writing performance for students with mild disabilities. Exceptional Children, 58(1), 61-72. Potter, M. L., & Wamre, H. M. (1990). Curriculum-based measurement and developmental reading models: Opportunities for cross-validation. Exceptional Children, 57(1), 16-25. 70 Schumaker, J. B., & Deshler, D, D. (1994). Secondary classes can be inclusive, too. Educational Leadership, 52(4), 50-52. School District #57. (1996). Guidebook for the use of curriculum based measurement in School District #57. Prince George, Canada: Author. Shinn, M.R. (1989). Identifying and defining academic problems: CBM screening and eligibility procedures. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children, (pp. 90-129). New York: Guilford Press. Shinn, M. R., Good, R. H., Knutson, N., Tilly, W. D., & Collins, V. L. (1992). Curriculumbased measurement of oral reading fluency: A confirmatory analysis of its relation to reading. School Psychology Review, 27(3), 459-479. Shinn, M. R., Nolet, V., & Knutson, N. (1990). Best practices in curriculum-based measurement. In A. Thomas, & J. Grimes (Eds.), Best practices in school psychology {pp. 287-309). Washington, DC: National Association of School Psychologists. Swerdlik, M. E. (1998). Review of the Stanford Diagnostic Reading Test (4^^ Ed.). In J. Impara & B. Plake (Eds.), The thirteenth mental measurements yearbook (pp 942-944). Lincoln, NE: The Buros Institute of Mental Measurement. Tindal, G. (1988). Curriculum-based measurement. In J.L. Graden, J.E. Zins, & M. J. Curtis (Eds.), Alternative Educational Delivery Systems: Enhancing Instructional Options for all Students {pp. 111-135). Washington, DC: National Association of School Psychologists. Tindal, G., & Parker, R. (1989). Assessment of written expression of students in compensatory and special education programs. The Journal of Special Education, 23(2), 169-184. 71 Walraven, G., & MacMillan, P. D. (2 000 / District Curriculum-Based Measurement norming project manual. Prince George, Canada: School District #57: Prince George. Wyndham, J. (1955). The Chrysalids. Harmondsworth, England: Penguin Books. Ysseidyke, J. E., & Christenson, S. L. (1988) Linking assessment to intervention. In J.L. Graden, J.E. Zins, & M. J. Curtis (Eds.), Alternative educational delivery systems: Enhancing instructional options for all students (pp. 91 -110). Washington, DC: National Association of School Psychologists. 72 Appendix A Administration and Scoring Guides CBM Reading Probes 73 Instructions To Probers Each testing room will consist of the following: tape recorder tapes 3 binders of scoreable reading probes file folder of resources Prior to starting any testing: 1. Write the date, block and the # of the testing room on the record sheet 2. Start your tape prior to the testing of your first student When a student enters your room: 1. Take the student information slip from the student and ask the student to be seated 2. Write the student name and student number on the record sheet 3. Record the probe # which will be given to the student 4. Read the student name and student number aloud so that the information will be recorded on the tape 5. Place the student information slip into the brown envelope 6. Read the instructions to the student 7. Start the probe 8. When one minute has elapsed and the student has stopped reading, the student may leave the room 9. Mark the probe indicating the number of words read, number of errors and the total number of correctly read words If there are any questions that you may have about pronunciation, etc. mark the probe and we can discuss at a later time. If you have any observations which are Important (such as the student was very reluctant, etc), please mark it on the scoreable probe. At the end of the morning, please place the used tapes and marked probes into the larger brown envelope for processing. Thank you for volunteering!!! 74 Hello, my name is . “When I say ‘begin’ start reading aioud at the top of this page. Read across the page (DEMONSTRATE BY POiNTING). Try to read each word. If you come to a word you don’t know, i’ii teii it to you. Be sure to do your best reading. Are there any questions?” (pause) Say ‘Begin’ and start your stopwatch when the student says the first word If the student falls to say the first word of the passage after 3 seconds, tell them the word and mark it as incorrect, then start your stopwatch. At the end of 1 minute, place a bracket (]) after the last word and say, “Stop ” 75 m in is tr a tio n a n d S c o r in g Directions fo r 1-M inute A d m in is tra tio n o f Reading Passages MaigriaJs: 1. Unnumbered copy of passage (student copy) 2. Numbered copy of passage (examiner copy) 3. Stopwatch 4. Tape recorder Direcdons: 1. Place the unnumbered copy in front of the swdent. 2. Place the numbered copy in front of you but shielded so the student cannot see what you record. 3. Say these specific direcdons to the student for each passage: When J say 'beg in ,' start reading alou d at the top o f this page. Read across the page (DEM O NSTRATE B Y PO IN TING ). Try to read each word. I f you come to a word you do n't know, I ' l l te ll it to you. Be sure to do yo u r best reading. A re there any questions?" (Pause) 4. Say “Begin" and start your stopwatch when the student says the first word. If the student fails to say the first word of the passage after 3 seconds, tell them the word and mark it as incorrect, then start your stopwatch.^ 5. Follow along on your copy. Put a slash ( / ) through words read incorrectly (see scoring procedures). 6. If a student stops or struggles with a word for 3 seconds, tell the student the word and mark it as incorrect. 7. At the end of 1 minute, place a bracket ( ] ) after the last word and say, "Stop. " ^Tape recorders facilitate error analysis. ^ On rare occasions the student may "speed read" (i.e., read the passage very fast and without expression). I f this occurs, tell the student, "This is not a speed reading test. Begin again, and be sure to do your best reading.” 76 n a m im s c r a iio n and üconng Scoring Procedures What is a “W ord” and W hat is a “ Correctly Read Word?” Ex. I cat TW = 1 read as; "cat" W RC= 1 Ex. 2 1 sat. TW = 2 read as: "1 sat." WRC = 2 W hat is a “Correctly Read Word?” Rule 1. Correctly Read Words Are Pronounced Correctly. A word must be pronounced correctly given the context of the sentence. Ex. 1. The word "r-e-a-d" must be pronounced "reed" when presented in the context of; He will read the book. not as: "He will rsd the book." W RC=5 W RC = 4 Ex. 2. The word "1-e-a-d" must be pronounced "led" when presented in the context of; She picked up a Isgd pipe. not as: "She picked up a leed pipe." WRC = 6 WRC=5 Rule 2. Self-Corrected Words Are Counted As Correct. Words miszead initially but corrected within 3 seconds are counted as correctly read. Ex. 1. The river was cold. read as: "The river was could...f2 sec)...cold." WRC=4 Matt cleaned the house ^ Mom. read as: "Matt cleaned the house flf...(l sec), cleaned the house fsx Mom." WRC =6 WRC=4 Ex. 2. WRC = 6 77 t u s L r a iio n a n a o c o n n g Rule 3. Repealed Words Are Counted As C orrect. Words said over again correctly are ignored. Ex. 1. Ted ran swiftly. read as: "Ted ran...Ted ran swiftly." W RC = 3 W RC = 3 Ex. 2. Sally saw a cat. read as: "Sally saw a...a cat." W RC = 4 W RC = 4 Rule 4. Dialect. Variations in pronunciation that arc explainable by local language norms are not errors. Ex. 1. They washed the car. read as: "They warshed the car." WRC = 4 W RC = 4 Ex. 2. Let’s go to the paikread as : "Let's go to the pawk." W RC = 5 WRC = 5 Rule 5. Inserted Words Are Ignored. When a student adds extra words, they are not counted as correct words nor as reading errors. Ex. 1. Sue was happy. read as: "Sue was very happy." W RC = 3 WRC = 3 Ex. 2. Kelly played the flute. read as: "Kelly played a the flute." W RC = 4 W RC = 4 What is an “ Incorrectly Read Word?” Rule 6. Mispronounced o r Substituted Words are counted as incorrect. Ex. 1. The ds2g ate a bone. read as: "The dig ate a bone." WRC =5 WRC = 4 78 « a m in j s c r a t io n a n a a c o n n g Ex. 2. Lynne has many hai.s. read as: "Lynne has many hat." W RC = 4 He wanted a new car. read as: "She wants a new car." W RC = 5 W RC = 3 Ex. 3. W RC = 3 Rule 7. Omitted Words are counted as errors. Ex. 1. Mario climbed the oak tree. read as: "Mario climbed the tree." W RC = 5 WRC = 4 Ex. 2. The king fought with an alligator in the moat. read as: "The king fought in the moat." W RC = 9 W RC = 6 Ex. 3. Sewing is my favorite hobby. 1cnioy scjving.dresses and suits- What is your favorite hobby? read as: "Sewing is my favorite hobby. What is your favtjritc hobby?" W RC = 16 W RC = 10 Rule 8. Hesitations. When a student hesitates or fails to correctly pronounce a word within 3 seconds.the student is told the word and an error is scored. Ex. 1. Mark saw an elephant read as: "Mark saw an ...(3 sec)" or read as: "Mark saw an elll-eee ...(3 sec)" WRC =4 W RC = 3 W RC = 3 - — IW R u le 9. Reversals. When a student transposes two or more woids, those words not read ^ in the correct order are»errors. Ex. I. Ot Charlie o n quickly. read as: "Charlie quickly on." W RC = 3 WRC = 1 79 Ex. 2. Shelly bought a beautiful Swcaiciread as: "Shelly bought a sweater beautiful." WRC = 5 W RC = 3 Special Scoring Rules Rule 10. Numbers Written As Numerals arc counted as words and must be read correctly within the context of the passage. Ex. 1. May 5,1222. Should be read as: "May fifth, nineteen eighty-nine.” not as: "May five, one nine eight nine." WRC = 3 W RC = 3 W RC = 1 Ex. 2. He was in grade i should be read as: "He was in grade three." not as: "He was in grade third." WRC =5 WRC = 5 WRC =4 Rule 11a. Hyphenated Words. Each morpheme separated by a hyphen(s) is counted as an individual word if it can stand alone. Ex. Fifty-seven Daughter-in-law WRC = 2 WRC = 3 Rule 11b. Hyphenated Words. If one or more of the morphemes separated by a hyphcn(s) cannot stand alone, the entire sequence is counted as one word. Ex. re-evaluate Spic-n-span Bar-b-que W R C = I WRC = 1 WRC = 1 Rule 12. Abbreviations arc counted as words, and must be read conecdy within the context of the sentence. Ex. 1. Dr. Adams received a promotion. should be read as: "Doctor Adams received a promotion." WRC = 5 W RC = 5 80 uuu oconng not as: "D-R Adams received a promotion." W RC = 4 Ex. 2. Jan lives on Fifth Ave. should be read as: "Jan lives on Fifth avenue" not as: "Jan lives on Fifth a-v-e" W RC = 5 W RC = 5 W RC = 4 . Ex. 3. Jan lives on Fifth Ave. also should not be read as: "Jan lives on Fifth ave" W RC = 5 WRC =4 Ex. 4. John watched T .V . can be read as: "John watched lee-vee" or as: "John watched television." W RC = 3 John watched television. should be read as: "John watched television." not as: "John watched tee-vee." W RC = 3 WRC = 3 W RC = 3 Ex. 5. WRC = 3 WRC = 2 81 Appendix B Grade 10 Pronunciation Guide CBM Reading Probes 82 Pronunciation Guide Grade 10 Probes acceptable pronunciation Probe 1 Probe 2 Probe 3 scarcely scarcely scarcly Rosaiind Rôseâlind Rds^lind Roselfiln Rbsâlin address afidress^ uhdress detestation detestation address ahdress uhdress offences uffences offences Daker's Dgt(er's Dakar’s peroration pair pur peer % (air) (are) (Ëdres) (adres) uh ' ra oh : shun 83 Probe #1 - grade 10 Student Name:_ Anne now transmitted nothing whatever, we caught no trace of her, but 12 whether she had the strength of will not to receive we were still uncertain. From 27 Rachel, her sister, we learnt that she would listen only to words, and was doing 42 her best to pretend to herself that she was a norm in every way, but that could 59 not give us enough confidence for us to exchange our thoughts with freedom. 72 And In the following weeks Anne kept it up, so that one could almost 86 believe that she had succeeded in renouncing her difference and becoming a 98 norm. Her wedding-day arrived with nothing amiss, and she and Alan moved 111 into the house which her father gave them on the edge of his own land. Here 127 and there one encountered hints that she might have been unwise to marry 140 beneath her, but otherwise there was little comment. 148 During the next few months we heard scarcely anything of her. She discouraged visits from her sister as though she were anxious to cut even that 160 174 last link with us. W e could only hope that she was being more successful and 189 happier than we had feared. 194 One of the consequences, as far as Rosalind and I were concerned, was 207 a more searching consideration of our own troubles. Quite when it was that we 221 had known we were going to marry one another, neither of us has been able to 237 remember. It was one of those things that seem ordained. In such proper accord 251 with the law of nature and our own desires, that we felt we had always known it. 268 The prospect coloured our thoughts even before we acknowledged it to 279 ourselves. To me, it had never been thinkable that anything else should 291 happen, for when two people have grown up thinking together as closely as we 305 had, and when they are drawn even closer together by the knowledge of hostility 319 all round them they can feel the need of one another even before they know they 335 are in love. 338 But when they do know they are in love they suddenly know, too, that there are ways In which they differ not at all from norms... 352 364 Total W ords Read:____________ Errors:_________ W ords R M d Correctly;_ 84 Probe #2 - grade 10 Student Name:_ My father included Aunt Harriet’s name in our prayers on the evening of the day 15 the news came, but after that she was never referred to again. It was as though 31 she had been wiped out of every memory but mine. There, however, she 44 remained very clearly, given form at a time when I had only heard her, as an 60 upright figure with a face drained of hope, and a voice saying clearly: I am not 76 ashamed— I am only beaten.’ And, too, as I had last seen her, looking up at the 93 house. 94 Nobody told me how she came to die, but somehow I knew that it had not 110 been by accident. There was a great deal that I did not understand in what I had 127 overheard, and yet, in spite of that, it was quite the most disturbing occumence I 142 had known yet—it alarmed me with a sense of insecurity far greater, for some 157 unperceived reason, than I had suffered over Sophie. For several nights I 169 dreamed of Aunt Harriet lying in the river, still clasping the white bundle to her 184 while the water swirled her hair round her pale face, and her wide-open eyes 199 saw nothing. And I was frightened... 205 This had happened simply because the baby was just a bit different in 218 some way from other babies. It had something, or lacked something, so that it 232 did not exactly accord with the Definition. There was the ‘little thing’ that made it 247 not quite right, not quite like other people... 255 A mutant, my father had called it....A mutant!.... I thought of some of the 270 poker-work texts. I recalled the address of a visiting preacher; the detestation 283 there had been in his voice when he thundered from the pulpit: ‘Accursed is the 298 MutantV 299 Accursed Is the mutant. ..The mutant, the enemy, not only of the human 312 race, but of all the species God had decreed; the seed of the Devil within, trying 328 unfiaggingly, eternally to come to fruition in order that it might destroy the divine 342 order and turn our land, the stronghold of God’s will upon Earth, into a lewd 357 chaos like the Fringes; 361 Total Words Read:_________ Errors:_________ Words Read Correctly:_________ 85 Probe #3 - grade 10 thing. Student Name:_ My father did not actually hit the inspector, but It must have been a near 15 He went on boiling with rage for several days and the next Sunday we 30 were treated to a searing address on the toleration of Mutants which sullied the 44 Purity of our community. He called for a general boycott of the owner of the 59 Offences, speculated upon immorality in high places, hinted that some there 70 might be expected to have a fellow-feeling for Mutants, and wound up with a 85 peroration in which a certain official was scathed as an unprincipled hireling of 98 unprincipled masters and the local representative of the Forces of Evil. 109 Though the inspector had no such convenient pulpit for reply, certain trenchant remarks of his on persecution, contempt of authority, bigotry, religious 120 131 mania, the law of slander, and the probable effects of direct action in opposition 145 to Government sanction achieved a wide circulation. 152 It was very likely the last point that kept my father from doing more than 167 talk. He had had plenty of trouble over the Daker’s cat which was of no value at 184 all: but the great-horses were costly creatures; besides, Angus would not be 197 one to waive any possible penalty... 203 So there was a degree of frustration about that made home a good place to get away from as much as possible. 217 225 Now that the countryside had settled down again and was not full of 238 unexpected people, Sophie’s parents would let her go out on rambles once 250 more, and I slipped away over there when I could get away unnoticed. 263 Sophie couldn't go to school, of course. She would have been found out 276 very quickly, even with a false certificate; and her parents, though they taught 289 her to read and write, did not have any books, so that it wasn’t much good to her. 307 That was why we talked— at least I talked— a lot on our expeditions, trying to tell 324 her what I was learning from my own reading books. 334 The world, I was able to tell her, was generally thought to be a pretty big place, and probably round.. 350 354 Total W ords Read:_ Errors: W ords Read Correctly:_ 86 Appendix C Grade 9 Pronunciation Guide CBM Reading Probes 87 Pronunciation Guide Grade 9 Probes acceptable pronunciation Probe 1 tach Probe 2 Ducati tack a'f'' Doo February Probe a ca cah te February Febuary Suzukis Kawasakis Honda C B X’s Montreal Montreal french pronunciation acceptable Marc Capa C ip a C^pa 88 Probe #1 - grade 9 Student Name;. This time it was easy. The bike started immediately and seemed eager to 13 continue, surging ahead. As Neii shot down the main straight, he watched for 26 Carol ahead. She gathered up her skirts when she heard him coming and took a 41 seat on a post near the edge of the tarmac. Blue wild flowers poked from her 57 straight brown hair. Her thumb was poised on the stop watch. She waved lazily 71 as Neii drew near, and Neil managed to wave back, quickly, before settling down 85 to business. 87 Somehow, he seemed to have assimilated the many thousands of bits of information which were giving him some sort of brain overload before lunch. 99 Ill Now everything was falling into place. By the time he’d taken the first two 125 comers he could tell that he didn’t need a lap or two to get into the swing of 143 things. It was just as if he hadn’t stopped for lunch at all. 156 The five laps at the old speed were uneventful, and he could barely wait 170 to see what the bike would do when he opened it up to seven thousand. Even 186 that wasn’t the maximum, he knew. After five iaps at seven thousand, he would 200 be able to take it all the way to eight thousand five hundred, if he dared. 216 As he rounded the final corner. Jack was standing out on the start line 230 holding up five fingers. Five iaps. Far down the track, Carol was ready with the 245 watch. It was time. 249 Slowly, he turned up the throttle, very gradually. But the bike didn’t speed 262 up gradually at all. At fifty-five hundred revs, it seemed to gather itself up and 278 hurtle into another time warp, the tach needle flicking up so fast he could hardly 293 follow it! He ripped beyond seven thousand before he could grab the next gear. 307 The msh of speed was like being flung forward from a catapult. The sound of 322 the bike changed, too. Gone was the chumping exhaust note he'd become used 335 to. In its place was a clean, crisp sound that was indescribably efficient, even a 350 little vicious. 352 Total W ords Read:____________ Errors :____________ W ords Read Correctly:____________ 89 Probe #2 - grade 9 Student Name:___________________________ For ttiree long weeks, Neil didn’t dare peep into the garage store room. Every 14 single night, he went up to his room instead and struggled through every scrap 28 of his homework as if making up for lost time. 38 In school, his dreams were now equally divided between Marsha Hoffman 49 and the big red Ducati waiting for him in the back of the garage. Now, however, 65 when he found his mind drifting, he remembered his mother’s warning. If his 78 marks did not improve at once, the bike would have to go. And if he could not 95 sell it, she would take it to the dump. Personally. Time and again he forced 110 himself to concentrate on what the teacher was saying. To keep his daydreams 123 in check. It wasn’t easy. 128 Almost at once, though, his marks took an upturn. Mini-tests held in class 142 put him on the honest side of fifty for the first time that year. As the weeks 159 dragged by, his marks continued to climb. His teachers scratched their heads in 172 bewilderment, wondering how it was that they were suddenly getting through. 183 Eventually, even his stepfather relaxed. 188 If his parents secretly hoped that he would somehow lose interest in the Ducati during this time, they were wrong. Waiting out those long cold weeks 201 214 made him want the bike even more. With his marks still improving, Neil knew at 229 the end of the month that it was at last safe for him to visit Gord’s shop. 246 Clutching the card Gord had given him, he took the bus downtown one February 260 evening after he had done his work. 267 "I was wondering what happened to you, ”Gord said. 276 “I’ve been busy,” Neil answered. “School stuff. You know?” 285 Gord’s establishment was not like any small bike shop he had ever seen. 298 It was a model of efficiency. The inevitable pile of junk motorcycles out back did 313 not exist here, the painted cement floors in the work room were spotless, the 327 work benches clean, the tools were hanging in their proper places. Special and 340 expensive machines had been carefully installed for doing special and 350 expensive jobs— electronic tuners, a hydraulic press, a small boring mill, a new 363 lathe. 364 Total W ords Read:____________ Errors:____________ W ords Read Correctly:____________ 90 Probe #3 - grade 9 Student Name:__________________________ Going Into the final, though, Neii was worried. If his mother thought this 13 racing came easy, that was okay by him. But the pavement had grown hot and 28 sticky. The Suzukis, the Kawasakis and the Honda CBX’s couid at last get all 42 their power onto the ground. Ripping down that long main straight, they were 55 really putting the motor on him. Aii season, they’d been getting faster and faster. 69 Now, it was becoming hopeless. Neil finished well down the field in the qualifier, 83 further down than he’d intended. Come Sunday, he knew he’d have to race his 97 heart out. 99 That night, he watched the last rays of sun dying in the mist and trees to 115 the west. He stretched in his canvas chair by the tent and yawned and stood up, 131 trying to overcome the anxiety that wouldn't go away. His parents had driven 144 back to Montreal for the night. He decided to look for his friends. 157 Here and there, he could see the flickering of campfires in the trees as 171 people settled around the flames to trade tall stories— bench racing, they called 184 it. Somewhere, he could hear people singing along with a guitar. A large ham 198 turned slowly on a barbecue as hungry friends gathered around. Neil ambled 210 down to the creek where Gord, Kate, Sandy and the others were camping and 224 he joined them around the fire as they roasted hot dogs, ladied out huge 238 servings of potato salad and rhubarb pie. It was that evening that he would 252 remember later almost as clearly as the race itself. Friends had always been 265 important to him. Singing with the others, he got to thinking again about Marc 279 Capa coming in the morning, and then about Carol although he’d promised 291 himself he wouldn’t. When he thought about her, even racing sometimes 302 seemed to be an empty exercise. 308 On a dozen radios around the camp the following morning the weather 320 man predicted a fine day for racing. And indeed the sky was dear blue and the 336 wind was easy. Neil’s big event was to be the last race of the day. 351 Total W ords Read;________ ___ Errors:____________ W ords Read Correctly:____________ 91 Appendix D Grade 8 Pronunciation Guide CBM Reading Probes 92 Pronunciation Guide Grade 8 Probes acceptable pronunciation Darry uairy Darry Probe 1 socs socks socbes (or socb's) Probe 2 reformatiory (air) (are) I I ' I w muh' tor i i ! tar I ê I Probe 3 Brumly Bru’mly /Y i Cl A (Broomly)- 1' img afas 93 Probe #1 - grade 8 Student Name:_ I looked through the door. Sodapop was giving Darry a back-rub. Darry 13 is always pulling muscles; he roofs houses and he’s always trying to carry two 27 bundles of roofing up the ladder. I knew Soda would put him to sleep, because 42 Soda can put about anyone out when he sets his head to it. He thought Darry 58 worked too hard anyway. I did, too. 65 Darry didn’t deserve to work like an old man when he was only twenty. 79 He had been a real popular guy in school; he was captain of the football team 95 and he had been voted Boy of the Year. But we just didn’t have the money for 112 him to go to college, even with the athletic scholarship he won. And now he 127 didn’t have time between jobs to even think about college. So he never went 141 anywhere and never did anything anymore, except work out at gyms and go 154 skiing with some old friends of his sometimes. 162 I rubbed my cheek where it had turned purple. I had looked in the mirror, 177 and it did make me look tough. But Darry had made me put a Band-aid on the 195 cut. 196 I remembered how awful Johnny had looked when he got beaten up. I 209 had just as much right to use the streets as the Socs did, and Johnny had never 226 hurt them. Why did the Socs hate us so much? We left them alone. I nearly 242 went to sleep over my homework trying to figure it out. 253 Sodapop, who had jumped into bed by this time, yelled sleepily for me to turn off the light and get to bed. When I finished the chapter I was on, I did. Lying beside Soda, staring at the wall, 1kept remembering the faces of 267 285 298 the Socs as they surrounded me, that blue madras shirt the blond was wearing, 312 and I could still hear a thick voice: "Need a haircut, greaser?” I shivered. 326 "You cold, Ponyboy?” 329 ""A little,’" I lied. Soda threw one arm across my neck. He mumbled 342 something drowsily. "Listen, kiddo, when Darry hollers at you...he don’t mean 354 nothin’. He’s just got more worries than somebody his age ought to. 366 Total W ords Read:____________ Errors:^ W ords Read Correctly:__ 94 Probe #2 - grade 8 Student Name:_ I climbed over ttie barbed-wire fence wittiout saying anyttiing else. I could 13 tiear Johnny laughing at me, but I didn’t care. I went strolling down the red dirt 29 road, hoping my natural color would come back before I met anyone. I wonder 43 what Darry and Sodapop are doing now, I thought, yawning. Soda had the 56 whole bed to himself for once. I bet Darry’s sorry he ever hit me. He’ll really get 73 worried when he finds out Johnny and I killed that Soc. Then, for a moment, I 89 pictured Sodapop’s face when he heard about it. I wish I was home, I thought 104 absently, I wish I was home and still in bed. Maybe I am. Maybe I’m just 120 dreaming... 121 It was only last night that Dally and I had sat down behind those girls at 137 the Nightly Double. Glory, I thought with a bewildering feeling of being rushed, 150 things are happening too quick. Too fast. I figured I couldn’t get into any worse 165 trouble than murder. Johnny and I would be hiding for the rest of our lives. 180 Nobody but Dally would know where we were, and he couldn’t tell anyone 193 because he’d get jailed again for giving us that gun. If Johnny got caught, they’d 208 give him the electric chair, and if they caught me, I’d be sent to a reformatory. 224 I’d heard about reformatories from Curly Shepard and I didn’t want to go to one 239 at all. So we'd have to be hermits for the rest of our lives, and never see anyone 257 but Dally. Maybe I’d never see Darry or Sodapop again. Or even Two-Bit or 272 Steve. I was In the country, but I knew I wasn’t going to like it as much as I’d 291 thought I would. There are things worse than being a greaser. 302 I met a sunburned farmer driving a tractor down the road. I waved at him and he stopped. 317 320 “Could you tell me where Jay Mountain Is?" I asked as politely as I could. 335 He pointed on down the road. “Follow this road to that big hill over there. 350 That’s it. Talking a walk?” 355 Total W ords Read:____________ Errors:____________ W ords Read Correctly:____________ 95 Probe #3 - grade 8 Student Name:_ I watched Darry going toward Tim and ttie leader of ttie Brumly boys. He 14 shouldn’t be here, I thought suddenly. I shouldn’t be here and Steve shouldn’t 27 be here and Soda shouldn’t be here and Two-Bit shouldn’t be here. W e’re 41 greasers, but not hoods, and we don’t belong with this bunch of future convicts. 55 We could end up like them, I thought. W e could. And the thought didn’t help my 71 headache. 72 I went back to stand with Soda and Steve and Two-Bit then, because the Socs were arriving. Right on time. They came in four carloads, and filed out 87 101 silently. I counted twenty-two of them. There were twenty of us, so I figured the 117 odds were as even as we could get them. Darry always likes to take on two at a 135 time anyway. They looked like they were all cut from the same piece of cloth: 150 clean-shaven with semi-Beatle haircuts, wearing striped or checkered shirts with 162 light-red or tan-colored jackets or madras ski jackets. They could just as easily 177 have been going to the movies as to a rumble. That’s why people don’t ever 192 think to blame the Socs and are always ready to jump on us. W e look hoody 208 and they look decent. It could be just the other way around— half of the hoods I 225 know are pretty decent guys underneath all that grease, and from what I’ve 238 heard, a lot of Socs are just cold-blooded mean—but people usually go by looks. 254 They lined up silently, facing us, and we lined up facing them. I looked for 269 Randy but didn’t see him. I hoped he wasn’t there. A guy with a madras shirt 285 stepped up. “Let’s get the rules straight—nothing but our fists, and the first to 300 run iose. Right?’’ 303 Tim flipped away his beer can. “You savvy real good." 313 There was an uneasy silence: Who was going to start it? Darry solved 326 the problem. He stepped forward under the circle of light made by the street 340 lamp. For a minute, everything looked unreal, like a scene out of a JD movie or 356 something. 357 Total W ords Read:____________ Errors;____________ W ords Read Correctly:____________ 96 Appendix E Administration and Scoring Guides CBM Writing Probes 97 A dm inistration and Scoring Directions for 3-M in ute Adm inistration of W ritten Expression Materials: 1. Story starter. 2. Stop watch , Directions: 1. Select an appropriate story stanin-. 2. Provide the student with a pencil and a sheet o f lined paper. 3. Say these specific directions to the students: “ You are going to write a story. F irs t, I w ill read a sentence, and then you w ill write a story ab ou t what happens next. You w ill have I minute to th in k about what you w ill write, a n d 3 m inutes to w rite y o u r story. Remember to do yo u r best work. I f you d o n 't know how to spell a word, you should guess. A re there any questions? (Pause). P ut y o u r pencils down and listen. F o r the next minute, th in k about ... (insert story starter)." 4. A fter reading the story starter, begin your stopwatch and allow 1 minute for students to "think." (M onitor students so that they do not begin wndng). A fte r 30 trcnnds sav: "Y ou should be th in k in g about (insert Story stancr)." 5. A t the end o f 1 minttte say: “Now begin w ritin g ." Restart your stopwatch. 6. Monitor students' attention to the task. Encourage students to work only if they are looking around or talking. 7. A fter QQ seconds sav: "Y ou should be w ritin g about (insert story starter)." 8. A t the end o f 3 m'nutes say: "Stop. P u t y o u r pencils down." 98 A d m in w ra n # w * nnrf S<®nrtit: m W ritten Expression Scoring Rules W h a t k A Word? Any lenir or group of letters separated by a space is defined as a word.evcn if the unnl is misspelledor^ a nonjensc word. _ U y < L A ,M ^ . g -J a , T W W .4 TWW. 4 T W W .4 ........ "44 TW W » 3 Rnle l: ffj^ h m a u d WardK Each morpheme s«pam«d^ * i g i|gMM# r ts counted as^ anintiividitalwoidifttca»9aadatoiMr.................................................. ........... ' ^ R u ie Z H y p h g M ted W o r^ U aakm vaom of tfemocpiieines separated by ahyphen(s) canamsaaAtdaoae attire SMHieaoe is ccMiiiBii as OOPswrL TWW. 6 R w k a . XAAMtAwkauk Qimattimiy toed mbbxevia&ms are countial as svotds T W W .3 Wanis writtoi m te jP e a r axair«KEtl| are counted T W W .IS 99 .... A dm inistration a n d S c » r in # ^ ^ 'lH m '■■■'■■ Rule 5. Numbers. With the exception of dates, numbeis that are not spelled out are no» counted as words. 3 -//nM/ny y ia /? v . TW W = 2 TWW. 3 T W W .5 Ruietf. Unum al C haraaers. Symbols used m writing sacb as not spelled outdare not counted as words. " ' ST that ■ ' .. jL fx -c rh , Q :^ ^ c < y r4 ^ n £ /y u :a e * rww-n Rule 4a. Womb written m ^ dde w as am«=#% are counted T W W . 15 100 Written Expression Probe 1 Write a story ttiat begins with; I was hiking with my family in Northern BC when I heard a loud crashing in the underbrush. I... School District #57 CBM Norming Project Tww_ wsc 101 Written Expression Probe 2 W rite a story ttiat begins witti: Last weekend I was at a party wtien a figtit broke out next to me. I... Sctiool District #57 CBM Norming Project TWW_ wsc 102 Written Expression Probe 3 W rite a story that begins with: Yesterday my best friend asked me to skip school with her and go to the park. I... School District #57 CBM Norming Project TWW_ WSC 103 Written Expression Probe 4 W rite a story that begins with: Yesterday when I got home from school, my mother started yelling at me as soon as I came through the door. I... School District #57 CBM Norming Project TWW_ WSC_ 104 Written Expression Probe 5 W rite a story that begins with: I didn't want to lie to my parents, but what else could I do? I.. School District #57 CBM Norming Project TWW_ WSC_ 105 Written Expression Probe 6 W rite a story that begins with: W e were late getting home from the party, and the road was incredibly slippery. W e... School District #57 CBM Norming Project TWW_ WSC 106