NORMING OF CBM READING AND WRITING AND DIBELS INSTRUMENTS FOR SCHOOL DISTRICT NO. 57 (PRINCE GEORGE) by John Edward Cook B.A., University of British Columbia, 1982 PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF EDUCATION m EDUCATIONAL COUNSELLING UNIVERSITY OF NORTHERN BRITISH COLUMBIA May 2005 © John Edward Cook, 2005 ~RSrrYofNORTHERN BRITISH COLUMBIA LmRARY PdDce Georp, B.C. ii ABSTRACT This study explored the development of a series of local norming Tables for Curriculum Based Measurement (CBM) reading and writing measures and Dynamic Indicators of Basic Early Literacy Skills (DIBELS) for use in Grades Kindergarten through 7 of School District 57 (Prince George). A total of 2420 students from 44 elementary schools participated in a total of three testing sessions that took place in the fall, winter and spring of the 2002/2003 school year. The method of sampling and data collection was explained. The quality of the data set was evaluated. Stability and equivalence coefficients were calculated for these measures. Equivalence of the probes used for both reading and writing subtests were assessed using Analyses of Variance procedures. A series of norm tables for Grades 1 to 7 for the fall, winter, and spring testing periods were generated for CBM measures entitled Words Read Correctly, Total Words Written, and Words Spelled Correctly. A series of norm tables for Grades 1 and Kindergarten were generated for DIBELS measures which included Letter Naming Fluency, Nonsense Word Fluency, Initial Sound Fluency, Phoneme Segmentation Fluency and Oral Reading Fluency. These analyses indicate that the CBM and DIBELS measures possess the technical qualities necessary for their use as intended by School District 57. The increases in the CBM norm values over their 1996 values illustrate the wisdom of the completion of this renorming study in 2003 and more generally the need for renorming studies to be done on a regular basis. --Ill TABLE OF CONTENTS Abstract 11 List ofTables v List of Figures VI Acknowledgment Vll CHAPTER ONE INTRODUCTION 1 Instruments 1 CBM Measures 2 Dibels Measures 2 Advantages of CBM and DIBELS measures 4 Rationale for Renorming 6 CHAPTER TWO METHOD Sample Sampling Procedures Probe Distribution Data collection 8 8 8 10 12 CHAPTER THREE RESULTS Descriptive Statistics CBM DIBELS Shape of the Distributions CBM Distributions DIBELS Distributions Probe Distribution CBM Reading CBM Writing Probe Differences CBM Reading CBM Writing Probe summary Measures of Stability and Equivalency CBM Coefficients of Equivalence and Stability DIBELS Coefficients of Stability Measures of Internal Validity CBM Validity Measures DIBELS Validity Measures Reliability CBM 14 14 14 14 15 16 17 19 19 19 19 22 23 25 25 25 27 28 28 29 31 31 IV CHAPTER FOUR DIBELS Summary of data set Norm Tables 32 32 33 DISCUSSION CBM Norm Tables CBM Comparisons CBM Reading CBM Writing Summary of CBM Changes DIBELS Implications for Practice Implications for Future Research 35 35 36 36 36 40 40 43 43 REFERENCES APPENDICES 45 Appendix A: Norm Tables CBM Reading Norms CBM Written Expression Norms DIBELS Kindergarten Norms DIBELS Grade 1 Norms Appendix B: Descriptive Statistics Appendix C: Comparison of 1996 and 2003 Reading Scores Appendix D: Comparison of 1996 and 2003 Writing Scores Appendix E: Examples of writing probes 48 49 56 63 69 71 77 85 93 v List of Tables Table 1: CBM DIBELS Norming Project Sample size 9 Table 2: Letter Naming Fluency Results for Kindergarten 16 Table 3: Reading Probe Distribution 20 Table 4: Writing Probe Distribution 21 Table 5: Analysis of Variance for Probe Differences in Words Read Correctly 23 Table 6: Analysis of Variance for Probe Differences in Total Words Written 24 Table 7: Pearson CmTelations for CBM Measures Between Norming Periods 26 Table 8: Pearson Correlation for DIBELS Kindergarten Scores Between Norming Periods 27 Table 9: Pearson Correlation for DIBELS Grade 1 Scores Between Norming Periods 27 Table 10: Pearson Correlation Between Total Words Written and Words Spelled Correctly 29 Table 11: Pearson Correlation Between Total Words Written and Words Read Correctly 30 Table 12: Grade 1 DIBELS Validity Correlations 30 Table 13: Grade Kindergarten DIBELS Validity Correlations 31 Table 14: CBM Reliability- Alpha Coefficients (Cronbach) 32 Table 15: DIBELS Reliability- Alpha Coefficients (Cronbach) 33 Table 16: Student Scores for Words Read Correctly and Total Words Written at the 20th Percentile from Spring Grade 1 to Winter Grade 4 35 Table 17: Comparison of 2003 and 1996 Lower Percentile Fall Writing Scores for Grade 2 to 4 39 Table 18: Comparison of Kindergarten DIBELS Scores at the 20th Percentile 42 Table 19: Comparison of Grade One DIBELS Scores at the 20th Percentile 42 VI List of Figures Figure 1: Example of typical CBM distribution 16 Figure 2: Change in Grade 2 reading distributions 17 Figure 3: Winter Phoneme Segmentation Fluency for Kindergarten 17 Figure 4: Change in distribution of Kindergarten Letter Naming Fluency over time 18 Figure 5: Ratings of Cronbach Alpha scores 31 Figure 6: Example ofNorm Table 34 Figure 7: Comparison of2003 and 1996 Grade 6 Reading Scores 37 Figure 8: Comparison of2003 and 1996 Grade 6 Writing Scores 38 Figure 9: Example ofDIBELS Table 41 vii ACKNOWLEDGMENTS I wish to thank Peter who advised me and encouraged me in completing this research. I also wish to thank Willy who provided an important sounding board and the occasional beer. Most importantly I wish to thank my loving wife Lynda who discussed stats over wine and prodded me to complete this study. I couldn't have done it with out her. Chapter One: Introduction Assessment of ability is a major component of any educational program (Deno, 1985). Types of assessment vary across a wide spectrum from the subjective to the objective and from the holistic to the atomistic. Currently, the Ministry of Education in the province of British Columbia places a strong emphasis on data collection as a measure of accountability within the school system. In their document entitled District Accountability Contract, the British Columbia Ministry of Education (2004) states, "Amendments to the School Act created an accountability cycle that requires each school, each district and the Ministry of Education to review performance measures and to plan, annually, for improvement" (p. 3). School Districts and individual schools across the province are preparing annual accountability documents to track student progress. In order to track student progress, schools need reliable assessment instruments. Two performance measures used to track student success are the atomistic assessments, Curriculum-Based Measurement (CBM) and Dynamic Indicators ofBasic Early Literacy Skills (DIBELS) scores (Spruceland Elementary School, 2004). For example, a school could keep records of students reading at or above a certain percentile. This study will focus on the development of a set of norming tables for CBM and DIBELS that School District 57 (hereafter SD 57) uses, in part, as data to support its contract with the ministry. Instruments This study focused on CBM measures of Reading and Writing Fluency and DIBELS measures of Letter Naming Fluency, Initial Sound Fluency, Phoneme Segmentation Fluency, Nonsense Word Fluency and Oral Reading Fluency. 2 CBM measures. Shinn and Bamonto (1998) define CBM as "a set of standard, simple short-duration fluency measures of reading, spelling, written expression, and mathematics computation" (p.l). The CBM Reading Test is a fluency measure of Words Read Correctly (WRC) in one minute. The students are given a passage to read and the scorer counts the number of correct words read in one minute. The CBM Writing Test is a fluency measure of Total Words Written (TWW) in three minutes. The students are given a sentence starter and asked to continue the starter. The scorer counts the number of words correctly written in three minutes. A refinement of the CBM Writing Test, Words Spelled Correctly (WSC) counts the number of words spelled correctly from that same three minute writing sample. DIBELS measures. Letter Naming Fluency (LNF) is a DIBELS measure that is administered for one minute. Students are presented with a printed page containing rows of randomly ordered upper and lower case letters and are asked to name as many letters as they can in one minute. The raw score is the total number of letters correctly identified in one minute. Initial Sound Fluency (ISF) is a DIBELS measure that requires children to identify from an array of four pictures, the word that begins with a target sound. For example, the examiner would say, "This is an egg, dice, spider and ladder. Which picture begins with /1/?" There is a total of 16 items on each probe. The ISF measure takes about 3 minutes to administer and has over 20 alternate forms to monitor progress. A calculation is applied to determine the number of initial sounds in one minute. Phoneme Segmentation Fluency (PSF) is a DIBELS measure that assesses a child's ability to fluently segment three- and four-phoneme words into individual 3 phonemes. In the simplest case each letter has a specific phoneme or distinctive sound. In the Kindergarten PSF test students are orally presented with three- to five-letter words and asked to repeat the word in segmented syllables or phonemes. In the example of "bad" the correct response are the sounds lb/, Ia! and /d/. In the word "beach" the correct response are the sounds /b/, leal and /chi. The number of correct phonemes segmented in one minute is the child ' s score. Nonsense Word Fluency (NWF) is a DIBELS measure that assesses a child's ability to decode nonsense words. The student is presented with randomly ordered vowel/consonant and consonant/vowel/consonant nonsense words such as et, dos , and tob. The student can reproduce the letter sounds or read orally the whole word. The student would receive a score of three for the word tob whether the student produced the word by letter sounds or read the complete word. The number of letter sounds produced in one minute is the student's score. Good, Wallin, Simmons, Kame'enui, and Kaminski, (2002) state, "Because the measure is fluency based, students receive a higher score if they are phonologically recoding the word and receive a lower score if they are providing letter sounds in isolation" (p. 8). Oral Reading Fluency (ORF) is a DIBELS measure that assesses a child ' s reading fluency. It is similar to the CBM reading measures and administered in a similar way. To ensure equivalent reading levels, "the Spache readability was used to revise and refine passages to keep the readability in a target range for each grade, but a broad range of readability estimates were considered in developing the passages" (Good et a!. (2002) p. 10). 4 As mentioned previously, scoring of CBM and DIBELS measures is objective. The CBM measures, in brief, rely on word counts. The DIBELS measures rely on counts specific to each instrument. The method of scoring all of the CBM and DIBELS instruments is described in the CBM I DIBELS guidebook available from SD 57. (School District 57, 2003) Advantages of CBM and DIBELS Measures One of the primary advantages ofusing CBM and DIBELS instruments is their ease and speed of administration. The testing procedures are clear and do not require a great deal oftraining to administer. The CBM and DIBELS instruments that this study describes take only a matter of minutes to administer to students. A CBM or DIBELS score is not a diagnostic measure on its own. Certainly, an experienced test administrator may learn where a child is experiencing errors but the real value of CBM and DIBELS lies in allowing large numbers of children to be screened quickly. Students' scores are indicators that identify children who might need to be followed up with additional time consuming and expensive diagnostic testing. The scoring of CBM and DIBELS instruments is, a much as possible, objective as a result of standardized rules. This eliminates subjective scoring and numerous different markers can obtain the same score. This very high inter-rater agreement is important (Sax, 1997) as many different teachers are going to administer and score the same instrument. It also increases the accuracy of student comparisons made based on score results. Traditional assessment instruments measure skills indirectly. Curriculum-Based Measurement and DIBELS instruments measure skills directly. For example a traditional 5 reading test may ask the student to read a passage and perform another task such as matching or selecting a multiple-choice answer and filling in a blank on a "bubble" answer sheet. The fine motor and organizational skill needed to fill in the bubble sheet correctly has little to do with reading but it does make the test easy to mark. In other words, the measure of the students' reading ability will be related to their ability to track questions and answers accurately when they fill in a bubble sheet. Additionally, the cognitive skills needed to select a correct response to a multiple choice question may not be related to measure of reading comprehension in text. Traditional types of assessment instruments also fall short when it comes to error analysis. All that is known is the student answered the question incorrectly but not where the student went wrong. CBM measures provide direct information to the scorer as to what type of errors the student is making. For example, when administrating CBM Reading Fluency the tester can notice if the child is stumbling over certain letter combinations. In another example, when administrating the DIBELS Letter Naming Fluency the tester can note which letters the child does not know. Traditional published assessment tools are useful for measuring individual differences between students but are not useful for measuring individual student learning over time (Hively & Reynolds, 1975). Marston, Fuchs and Deno (1986) also established this point with a sixteen week comparison of CBM measures and published normreferenced tests. Growth was far more evident using the CBM measures and was more in line with teacher evaluations as well. Marston and Magnusson (1985) reported similar findings in a ten week study. The DIBELS measures developed by Kaminski & Good ( 1998) were developed to monitor growth in the acquisition of critical early literacy 6 skills (Good, Gruba, & Kaminski 2001). DIBELS measures are designed to be used in a dynamic or ongoing way over time rather than as a summative measure. Through the use of CBM and DIBELS instruments a tester can track student progress with a direct skill measurement normed against the peers from the student's school district. This is more desirable than the use of national norms of other instruments which may not accurately reflect the population of a particular region. As well, national nonns often include grade equivalency scores which in my experience are sometimes misunderstood and misused by staff. Another problem of published tests relates to the test-retest phenomena (Sax, 1997). The frequency of testing must be reduced to prevent a student from learning the test or recalling some of the questions. The tests are not designed to be used frequently. CBM measures on the other hand can be administered on a weekly basis by using one of many alternate forms generated from the student's curriculum. CBM and DIBELS measures are designed to be used frequently with no loss of reliability due to test-retest. The advantages of ease and expediency of CBM and DIBELS assessments make them particularly useful for accountability purposes. Their direct measurement, error analysis and objective scoring provide further pedagogical advantages. Rationale for Renorming School District 57 (Prince George) has been using CBM atomistic measures district wide since 1996. Dr. Peter MacMillan of the University ofNorthem British Columbia and others developed the original sets of norm tables used to measure reading and writing fluency in 1995 (School District 57, 1995). These tables were developed before the CBM measures were in wide use in the district. 7 In light of the push for school districts to demonstrate accountability to the ministry through data collection, it was important that current norm tables were available. Given that the norm tables were over five years old and that CBM measurements are now in wide use in all of the schools in the district, district administrators decided it was time to renorm the tables that had been developed by SD 57 in 1996. To maintain the validity and reliability of any assessment tool it is necessary to renorm it on a regular basis and as with any norm-referenced test is important that the norms reflect the population that is being tested. (Sax, 1997) Renorming was also done to improve the delineation of student performance. In addition to the CBM instruments being used, SD 57 officials decided to introduce a new set of similar measures, Dynamic Indicators of Basic Early Literacy Skills (DIBELS), to be used for Kindergarten and Grade One students. A series of norm tables was required to increase the utility and local relevance of the DIBELS instruments. The purpose of this study was to create a series of norm tables for CBM and DIBELS measures used in SD 57. 8 Chapter Two: Method Sample Shinn (1989) states "Guidelines established by the American Educational Research Association and the National Council on Measurement in Education (1985) state that all tests used in education and psychology must be valid, reliable, and, if they are to be used in a norm-referenced manner, have adequate normative data" (p. 19). For the renorming project over 2400 children were randomly selected to be in the norming sample. This was 25 percent of the population of Grades K -7 in the school district that commissioned the study. Forty-four schools took part in three testing periods to make up the norming sample. Sample size is very important when developing district norms. Shinn (1989) suggests that 100 to 150 students per grade are needed in the sample group to develop district norms. The lowest number of students sampled in a grade in this present study was 258 with the highest being 353 . (See Table 1 for a description of the numbers of students sampled.) Sampling Procedures In September 2002 teachers from every elementary school in SD 57 attended a workshop on the selection procedures for the norming sample and administration of the CBM and DIBELS instrument. Students were selected for the norming sample from those who were registered in elementary schools by October 4, 2002. There were very few students excluded from the norming sample. The students who were excluded included level one and level two ESL students, students with mental disabilities, other hard-labeled students such as hearing- 9 impaired, visually impaired, or autistic students and students enrolled in French Immersion. Including students with a wide range of abilities in the norming sample was important to ensure that the norm tables were representative of the full population. Table 1. Norming Project Sample Size Grade Number of Students Sampled K 258 1 263 2 288 3 298 4 330 5 301 6 329 7 353 Total 2420 An a1phabetica11ist of students was generated for each grade of the school. Each school was provided a random start number that determined how to choose the first student to begin the selections of students from their alphabetized lists. After the first student was chosen every fourth student in the grade was included in the norming sample. For example if the first student selected was named Smith every fourth student after the name Smith would be chosen to be included in the norming sample. Upon reaching the end of the alphabetical list the students were then chosen from the 10 beginning on the alphabetical list while still following the "every fourth student" pattern until returning to the random start student. Directions were very specific about the remaining students on the list for each grade. If there was only one name remaining it was not included in the sample. If there were two names remaining the second name was included in the sample. If there were three names remaining the third name was included in the sample. Some procedures were developed in anticipation of problems that occur when dealing with large norming samples. For example if a target student was absent for the entire two week testing period they were not included in that norming period but they were included again in the next testing period. If the target student moved away from the school during the year, a list of students at that grade level that were new to the school was generated. The new students' names were put into a hat and one student was randomly selected to replace the missing student in the norming sample. If there were no new students at that grade level an alternate student was selected from the general grade population by flipping a coin. If the coin came up heads the student alphabetically above the target student who had left the school was selected. If the coin turned up tails the student alphabetically below the target student who had left the school was selected to be part of the norming sample. Probe distribution. The reading passage given to students for the CBM reading measure is called a probe. There were six different CBM reading probes administered to students within each grade. The probes were collected by SO 57 staff from grade level reading materials and were not newly developed for this norming study. The reading ll probes were chosen on the basis of mid-year readability level for each grade. (School District 57, 2003) Each school was assigned a probe number with which to begin its cycle of testing to ensure that all probes were used by all grades in all three testing periods. In the 1995 norming sample one probe was administered to all the students at that school. In other words, School A may have administered Probe 1 and School B may have administered Probe 2. This may have led to some school effects that could have affected the reliability of the norming Tables. Test procedures in the 2002 norming project were designed to eliminate this possible effect in the second norming sample. The story starter given to students for the CBM writing assessment is referred to as a writing probe. There were six writing probes that were given to all the students across the grades. These probes were developed by school district personnel. As with the reading probes, any one student would be exposed to three of the six possible probes during the testing cycle. It was a little more difficult to randomize the use of the writing probe. The 44 schools that participated in the norming sample were divided into six groups of relatively equal populations. All of the schools participated in the three nonning periods. The six writing probes were randomly yet equally divided among the six groups, grades and testing periods. Rather than the probes being administered to individual students, the probes were randomly administered to classes of students. The same six writing probes, as shown in Appendix E, were used for all the students in all grades. In the SD 57 1996 norming sample there may have been some economic bias or school effect because the probes that were used to collect the previous set of data were 12 not distributed randomly to the subjects in the sample. It was thought at the time that due to the similarity of the probes, the data collected would be similar. Some school effect was noticed after the data was collected. Although it was not considered significant in the creation of the norming Tables an effort was made to clear this new data set of any such problem. In this new norming sample all six reading probes were distributed randomly to all the students in all schools. A similar process to that used for student selection was used to determine which student got which probe. As mentioned previously this was not the case for the writing probes. The reading probes were administered individually while the writing probes were administered to groups of children all at the same time. Data Collection Each school recorded the CBM results for individual students on recording forms created in FileMaker Pro™ by SD 57. The complete forms were transmitted electronically to the central office where all the individual school files were combined into a large district database. The data were screened for data entry errors and these were corrected. Next, the data were examined for inconsistencies and outliers. Overly high scores, which appeared to be at first outliers in the data set, were checked with individuals at schools. Upon investigation these high scores were found to correspond with the students' performance and ability in class so were not dropped. Some zero scores were dropped from WRC from each testing period when it was felt by the recorder that the student was not trying to complete the task. Scores that were dropped this way had little to no effect on the sample size. The data were exported as tab 13 separated text to SPSS 9.0 and saved as an SPSS file where analysis was completed. The data were sorted by grade before the quality was analyzed. 14 Chapter Three: Results The data file was analyzed to evaluate its quality before norm tables were generated for CBM Words Read Correctly, CBM Words Spelled Correctly, CBM Total Words Written, DIBELS Letter Naming Fluency, DIBELS Nonsense Word Fluency, DIBELS Initial Sound Fluency, DIBELS Phonemic Segmentation Fluency, and DIBELS Oral Reading Fluency for three time periods; fall, winter and spring. Some of the analyses included descriptive statistics, the shape of the distribution of scores, distribution and differences of probes, the stability between testing periods, internal validity measures and the reliability of the data. Descriptive Statistics CBM. The descriptive statistics ofthe CBM instruments revealed skew and kurtosis values of intermediate students not greater than± 0.31 and 0.91 respectively. The large majority of the CBM results had skew values that were less than two times the standard error: that is, the distributions do not deviate from normality in the population (see Appendix B for the complete results). Standard deviations were somewhat higher in the early grades and became smaller as the students got older. This suggests there is more variability in reading and writing scores at the Grade 2 level than at the Grade 7 level. DIBELS. The descriptive statistics of the DIBELS instruments revealed that skew values were often more than twice than standard error which suggests that the distributions deviate from normality in the population. Kurtosis ranged from a high of 7.7 in fall scores of Grade 1 Nonsense Word Fluency to a low of -0.84 in spring testing on Kindergarten Phonemic Segmentation Fluency. In Kindergarten the standard 15 deviations were often greater than the mean which indicates a large variance in the scores. In Grade 1 the standard deviations were on a similar scale to those found in the CBM results. The high positive skew values in many of the DIBELS distributions were most likely the result of tasks presented to the children, prior to formal instruction of the skills of interest, which resulted in a large number of zero scores. The instruments used do not appear to be sensitive at the lower scores. As the children became more proficient at the skill, such as letter naming, (see Table 2) the skew and kurtosis values dropped. Standard deviation scores also dropped as a proportion of the mean. The drops in these values suggest the children were becoming more proficient at the task and that there was less variability in the scores. Shape of the Distributions Graphic displays of histograms, with a normal curve overlay, were observed for each instrument in each testing period. Normal data distributions are not required to generate the percentile ranks that make up the norming tables but given the large sample size, an instrument that is sensitive to the population should generate a normal data distribution. Nonnal distributions give a measure of reliability when using the norm tables with the larger population in the grade. The shapes of the distributions were observed for one additional reason. One of the assumptions used when performing an ANOV A is that the data are normally distributed. I wanted to check the distributions to make sure that this assumption was valid. 16 Table 2. Letter Naming Fluency Results for Kindergarten Mean SD Min Max 10.04 11.41 0 84 2.065 7.049 Winter 20.06 14.94 0 93 .905 1.503 Spring 15.78 0 84 .319 -0.002 Fall 29.85 Skew Kurtosis CBM distributions. The distributions of the CBM reading and writing scores were all essentially normal in the intermediate grades. This was not surprising given the skew and kurtosis results observed from the descriptive statistics of the instruments. An example is given in Figure 1 of the spring testing of Grade 4 Words Read Correctly. This example is fairly typical of all the CBM distributions from Grades 3 to 7. Spring WRC GRADE: 4 40,--------------------------, 30.4050.60.7080.90.101112131415161718 19 2021 22 o o o o o o o o.m.oo.oo.m.oo.oo.oo.m.oo.oo.m.oo.o Figure 1. Example of typical CBM distribution 17 The Grade 2 Words Read Correctly distribution started with a slightly positive skew in the distribution as indicated in Figure 2. As the children progressed in their abilities the distribution moved to a more normal curve although with a small positive skew. The Grade 1 distributions showed a similar distribution to the positively skewed distribution of fall Grade 2 Words Read Correctly seen in Figure 2. Fall Words Read Correctly Spring Words Read Correctly GRADE : 2 GRADE : 2 50 40 ~ - 30 20 >u c Q) 10 / / v ;rF- ~ '\ ::> 0" ~ u_ 0 0.0 20 . 40 . 60 . 80 . 0 0 0 0 ~ 10 12 14 16 18 20 0.0 0.0 0 .0 0 .0 0 .0 0.0 10 . 30 . 50 . 70 . 90 . 0 0 0 0 0 11 13 ~ ~ 15 0 .0 17 19 ~ ~ 21 0.0 Figure 2. Change in Grade 2 reading distributions DIBELS distributions . DIBELS distributions were in most cases not normal and more often positively skewed, sometimes dramatically so, as illustrated in Figure 3. The distributions, while remaining positively skewed, all reflected the growth in skills over WinterPSF GRADE: Kindergarten 100, - - - - - - - - - - - - - - - - - - , "" 60 40 [) fii i Q) ti 9d. Oev 20 =15.00 ~ 0 "::-:'--"::-:'--"::-:'--"-'--"-'--r..--.,._- N"' 259.00 5.0 15.0 25.0 35.0 45.0 55.0 65.0 75.0 Figure 3. Winter phoneme segmentation fluency for kindergarten 18 time. This trend is well illustrated in Figure 4, which shows the changes in Letter Naming Fluency in Kindergarten over the three testing periods. Fall Letter Naming Fluency Kindergarten 80 , - - - - - - -- -- -- - - - - - - - - -- - - - . 60 40 -n E 20 c:: 1/ I "'::J 0 '< ~~~~~~~~~~~~~ 0.0 10.0 20 .0 30 .0 40.0 50.0 60.0 70.0 80.0 5.0 15.0 25.0 35.0 45.0 55.0 65.0 75.0 85 .0 Winter Letter Naming Fluency Kindergarten 40 . - - - - - - - - - - - - - -- - - -- - - - - - - - . h 30 ; r- ,--!\; r20 1/ ~ Spring Letter Naming Fluency Kindergarten 4 0 , - - - - - - - - - - -- - - - - - - - - - - - - . - 0.0 10.0 20 .0 30.0 40 .0 50 .0 60 .0 70.0 80 .0 5.0 15.0 25.0 35.0 45.0 55.0 65.0 75.0 85.0 Figure 4. Change in distribution of Kindergarten Letter Naming Fluency over time 19 Probe Distribution To ensure that the CBM norm tables fairly represented the population to which they were being applied it was important to ensure a fair distribution of the probes used to generate the data. The DIBELS measurements did not use different probes during the same testing period. All children received the same DIBELS probes so no analysis of probe distribution was necessary. The Grade 1 students were tested with CBM probes only in the spring. CBM reading. Distributions for CBM reading probes presented in Table 3 indicate that there was a relatively even distribution of the probes in the three testing periods. The minor variations in the total number of probes distributed in each testing period are due to student absences. CBM writing. The CBM writing probe distribution presented in Table 4 appears to be less well distributed as compared to the reading probe distribution. The reason for the unequal disttibution is because the writing probes, as mentioned previously, were administered to entire classes. A small variation of distribution can make a large difference in the total number of scores for that probe because a two or three class difference in distribution can amount to a difference of 50 or 60 probes being administered. However, this variance in writing probe distribution was not considered large enough to have a meaningful impact on the norming tables. Probe Differences To discover if any one probe differed in difficulty from any other, an ANOV A was performed on each grade of the CBM probes using the options available in SPSS 9.0™. One of the difficulties in using six different probes is ensuring equality between 20 the probes. A question posed about the CBM writing task might be something like "What if the students are interested in one sentence starter over another? Wouldn't they Table 3. Reading Probe Distribution Probe Gr. 7 Gr. 6 Gr. 5 Gr. 4 Gr. 3 Gr. 2 Gr. 1 Fall Winter Spring Number N N N N N N N 1 55 52 46 52 44 44 2 56 51 45 49 44 41 3 54 50 43 50 46 42 4 53 50 43 52 47 44 5 55 52 50 52 47 47 6 55 51 46 52 49 44 Total 328 306 273 307 277 262 1 56 52 48 55 50 46 2 56 53 46 49 46 43 3 58 53 47 49 46 43 4 55 49 44 52 45 41 5 54 49 42 52 48 45 6 56 54 51 52 48 47 Total 335 310 278 309 283 265 1 56 56 50 53 48 45 45 2 56 53 49 53 46 46 44 3 58 55 44 53 47 49 42 4 56 52 48 49 47 43 37 5 56 49 43 50 45 41 41 6 55 49 42 51 48 42 39 Total 337 314 276 309 281 266 248 21 Table 4. Writing Probe Distribution ~ ~ ~ ~ ~ ~ Number N N N N N N 55 60 49 57 45 44 2 62 52 52 54 51 44 3 21 28 51 65 59 50 4 71 68 38 46 46 42 5 71 61 49 51 40 44 6 55 44 40 36 40 40 Fall Total ~ 59 37 38 39 42 2 57 55 49 55 45 46 3 70 59 58 53 44 39 4 19 50 52 65 66 54 5 71 28 44 48 52 46 6 62 64 44 51 40 44 Total Spring N 335 313 279 309 281 264 56 Winter ~ 335 315 284 310 286 271 69 59 45 54 43 45 38 2 67 55 43 46 47 45 44 3 61 59 48 54 44 44 37 4 56 so 52 51 43 39 50 5 20 28 53 65 60 49 53 6 64 64 42 46 48 43 29 Total 337 315 283 316 285 265 251 be inspired to write more?" If one set of scores is significantly better than another, then the starting sentence of the probe could be considered to be one of the reasons for the 22 difference. It was also important to ensure that the reading probes were of the same level of difficulty. Before running the ANOV A the data was tested for homogeneity of variance which is one of the underlying assumptions of an ANOV A. Of the 18 groups (seven grades and three testing periods) tested for Words Read Correctly there were some groups that showed some small departures from the assumption of equal variance. No violations of homogeneity of variance were repeated consistently between testing periods or in any grade. Significant findings using the Levene statistic included F (5, 241) = 3.59,p<.01 for the spring test of Grade 1 Words Read Correctly and F (5, 256) = 4.50,p<.001 in the fall testing of Grade 2 Words Read Correctly. I do not think the violations ofhomogeneity of variance affected the results ofthe ANOVA which is a robust test given the large similar sample sizes (balanced design) and normality of the distribution as mentioned previously. CBM reading. Due to developmental reading abilities, each grade was given a different set of six reading probes. After performing the ANOV A there were some probes in some testing periods that appeared to be discrepant from the others as shown in Table 5. An alpha level of .01 was considered appropriate, as the Bonferoni corrected alpha for 3 testing periods would have been .016. The significant findings are I believe, either a Type I error or a form of group effect. Type I errors are a false positive which means a difference is shown as significant when no there is no actual difference. If indeed a probe was different from the others the differences should consistently appear in each testing period. The suggestion of a group effect is based on the observation that Probe 4 and Probe 5 administered to Grade 5 students in the fall and winter respectively showed a 23 significant difference. Given the research design, Probe 5 would be given to students after Probe 4. The differences in the probes showed they were generating higher mean scores than the Table 5. Analysis of Variance for Probe Differences in Words Read Correctly Winter Fall Spring F p dfw F p dfw 7 326 2.03 .075 334 2.84 .016 334 3.79 .002* 6 305 3.65 .003* 309 .399 .850 311 1.23 .294 5 271 2.30 .046 276 3.96 .002* 273 1.88 .097 4 304 1.4 .224 308 2.64 .023 308 2.98 .012* 3 274 1.66 .144 281 2.77 .019 280 1.66 .144 2 261 3.58 .004* 263 1.25 .298 264 2.14 .049 246 2.36 .041 Grade dfw 1 (dfb was 5 for all cases) F p * p < .01 other probes. Perhaps, by chance, a capable group of students was given these probes in successiOn. CBM writing. The results in Table 6 show some significant differences using a conservative alpha of0.01 in Total Words Written in fall of Grade 5 and 6, winter of Grade 3 and spring of Grade 7. Further analysis showed that Probe 3 appeared discrepant, generating a lower mean score in Grade 6. Probe 4 appeared to be generating a lower mean score and was the cause of the discrepancy in the fall of Grade 5. In the winter of Grade 3, Probe 6 generated a higher mean score and appeared discrepant from 24 the rest of the probes. In the spring of Grade 7 Probe 4 was discrepant, generating a higher score than the other probes. I believe these were all false positive results or Type I errors. In one testing period Probe 4 generated a lower mean, while in another testing period the same probe generated a higher mean than the other probes. As mentioned Table 6. Analysis of Variance for Probe Differences in Total Words Written Fall Spring Winter Grade dfw F p dfw F p dfw F p 7 334 1.67 .142 332 2.47 .033 333 3.39 .005* 6 312 4.71 .000* 311 1.48 .196 310 .82 .537 5 277 3.96 .002* 279 .84 .552 276 .34 .890 4 306 .78 .563 306 1.59 .162 308 .45 .816 3 280 .73 .600 282 3.82 .002* 278 .85 .520 2 263 1.77 .120 266 1.60 .159 264 1.50 .191 246 2.40 .038 1 (dfb was 5 for all cases) * p < .01 previously the differences between probes should be repeatable in different testing periods and at the very least the probe should be discrepant in the same way generating a consistently higher or lower mean score. Replication of differences does not occur either across grades or testing periods; therefore I think the all the significant differences were the result of a Type I error. One of the reasons for the differences in probes might be attributed to class effect. Unlike the reading probes that were evenly distributed across the sample 25 population the writing probes were given to a whole class at the same time. Therefore, the means of a few poorly performing classes writing the same probe could influence the results. Probe summary In summary analysis of both the reading and writing probes show them to be equivalent due to their even distribution and similar scores. Actual distributions of the probes reflect positively on the procedures developed to ensure a wide, even distribution of the probes. Measures of Stability and Equivalency Equivalence and stability of scores over time are measures of reliability. The scores change because of variability in external contributing factors. The children also progress in skill performance but given the large sample used in this study, the progress of the groups is similar or equivalent. High correlations for CBM scores between testing periods observed in Table 7 suggest that the students are performing in a similar way on a similar task. CBM coefficients of equivalence and stability. Pearson correlations for the CBM instruments across norming periods were also performed (Table 7). They are a measure of equivalence because of the different probes used and a measure of stability because they compare different testing times. As expected, as the students improve over time, the spring scores are less highly correlated to the fall scores as compared to contiguous testing periods. The correlations are high, and given that they are a combined measure of stability and equivalence, a researcher might expect even higher correlations of either equivalency or stability when measured in isolation. Lower, yet consistent, correlations 26 between writing scores across testing periods suggest that the writing task has more variability for individual students. Table 7. Pearson correlations for CBM measures between norming periods Words Read Correct} y Grade r fall-winter r winter -spring r fall-spring 2 .87 .86 .84 3 .87 .89 .83 4 .86 .87 .86 5 .87 .86 .84 6 .89 .85 .81 7 .89 .89 .86 1 Total Words Written Grade r fall-winter r winter -spting r fall-spting 2 .62 .65 .59 3 .66 .65 .59 4 .62 .56 .60 5 .60 .62 .60 6 .74 .67 .65 7 .70 .63 .58 1 (All correlations are significant at the 0.01 level two tailed) 27 DIBELS coefficients of stability. Correlations for the DIBELS instruments across norming periods were also performed. The results are displayed in Table 8 and 9. Only one DIBELS measure, Letter Naming Fluency, was used over the three testing periods as compared to the CBM measures. The values are generally higher than CBM measures of TWW but lower than CBM WRC. PSF values of .69 are identical for winter-spring Table 8. Pearson Correlation for DIBELS Kindergarten Scores Between Norming Periods r fall-winter r winter -spting r fall-spring Letter naming fluency (LNF) .79 Nonsense word fluency (NWF) Initial sound fluency (ISF) .77 .65 .74 .70 Phoneme segmentation fluency (PSF) .69 (all correlations are significant at the 0.01 level, two tailed) Table 9. Pearson correlation for DIBELS Grade one scores between norming periods r fall-winter r winter -spring r fall-spting Nonsense word fluency (NWF) .67 .82 .65 Phoneme segmentation fluency (PSF) .70 .69 .55 Oral reading fluency (ORF) .90 (all correlations are significant at the 0.01 level, two tailed) for both Kindergarten and Grade 1. Again, as noted in the CBM values the correlations are lower when comparing fall-winter and fall-spring scores, varying for Kindergarten 28 LNF from .79 to .65 respectively. A similar change can be observed in the Grade 1 PSF measure that drops from .70 for fall-winter to .55 for fall-spring. The low fall-winter correlation in Grade 1 NWF (.67) may be due to the dramatic positive skew to the distribution of the fall sample. In other words, it is difficult to get strong correlations to different set of scores if one set has a large number of zero scores. Severely non-normal distributions will produce lower correlations so the correlations in Table 8 are good. The winter-spring correlation for NWF is notably higher (.82) because the distributions are less positively skewed and approach a more normal distribution. As the students improve over time, the spring scores are less highly correlated to the fall scores as compared to concurrent testing periods. This is similar to the CBM results. Measures of Internal Validity Correlations among measures that are related conceptually or theoretically are coefficients of internal validity. High correlations between similar skills suggest that the instruments are valid because they are both measuring what they are supposed to measure. Lower correlations between dissimilar skills can also be an indicator of validity. Measures of internal validity confirm for the researcher that the instruments are behaving as expected. CBM validity measures. Very high correlations ranging between .94 and .99 for Total Words Written {TWW) and Words Spelled Correctly (WSC) displayed in Table 10 are expected because they measure a skill in the same domain. If the correlations between these two measures were low it might indicate some problems with reliability, validity, or data collection. Lower, but relatively consistent correlations in each grade, between total words written and total words read correctly displayed in Table 11 suggest 29 that a different yet related skill is being measured. The consistency and stability of the correlations gives evidence of good reliability. DIBELS validity measures. Correlations between DIBELS skills remain fairly constant as displayed in Tables 12 and 13. High correlations (.74, .82) between Oral Reading Fluency (ORF) and Nonsense Word Fluency are expected as they both measure Table 10. Pearson correlation between total words written and words spelled correctly Grade Fall Winter Spring .94 1 2 .94 .94 .96 3 .97 .97 .97 4 .97 .98 .98 5 .98 .98 .98 6 .98 .98 .99 7 .99 .99 .99 (All correlations are significant at the 0.01 level two tailed) very similar skills. The correlation between Oral Reading Fluency (ORF) and Words Read Correctly (WRC) is even higher (.93) as the children are in both cases reading familiar words. Although ORF and WRC are both fluency levels in reading, the measures are not more highly correlated because of the slightly different reading levels of the instruments. Correlations ofPSF and ORF (.36) are expected to be low because the students are asked to perform different tasks and must produce letter sounds in PSF 30 Table 11. Pearson correlation between total words written and words read correctly Grade Fall Winter Spring .45 2 .48 .49 .45 3 .40 .40 .32 4 .34 .29 .38 5 .42 .27 .29 6 .39 .39 .43 7 .33 .27 .29 (All correlations are significant at the 0.01 level two tailed) Table 12. Grade one DIBELS Validity correlations Fall PSF- NWF .52 Winter Spring .56 .47 PSF- ORF .36 .36 NWF-ORF .74 .82 ORF- WRC .93 (All correlations are significant at the 0.01 level two tailed) compared to the ORF test where some students are able to sight-read entire words. The consistency and stability of the correlations gives evidence of good reliability. 31 Table 13. Grade Kindergarten DIBELS Validity Correlations Fall Winter Spring PSF- NWF .51 .52 PSF-LNF .39 .41 NWF-LNF .65 .69 (All correlations are significant at the 0.01 level two tailed) Reliability One of the issues concerning any measurement instrument is reliability. Cronbach's Alpha coefficients are reliability coefficients that examine internal consistency, based on the average inter-item correlation. Reliability coefficients were calculated for both the CBM and DIBELS instruments. To interpret the reliability coefficients George and Mallery (2003) suggest the rules of thumb presented in Figure 5. Alpha Score Rating > .9 excellent > .8 good > .7 acceptable > .6 questionable > .5 poor < .5 unacceptab Ie Figure 5. Ratings of Cronbach Alpha scores CBM. For the CBM coefficients the results from the three testing periods were compared to each other. The average intraclass measure correlation is reported in Table 14. All of the coefficients are above .80 and many are above .94 which is a good to 32 excellent measure of reliability. These measures should be considered as the lower bounds of reliability given that they span 6 grades over a time period of 6 months. DIBELS. For the DIBELS coefficients two and sometimes three results were compared. All ofthe DIBELS results presented in Table 15 are above .81 with the highest being .96. Considering the variability in the rates of student achievement and that some measures such as Grade K LNF (.88), Grade 1 PSF (.85) and Grade 1 NWF (.85) cover three testing periods over a span of 6 months, the results indicate a high degree of reliability for the scores. Table 14. CBM Reliability -Alpha Coefficients (Cronbach) Words Read Total Words Written Words Spelled Correctly (WRC) (TWW) Correctly (WSC) Grade 7 .94 .84 .84 Grade 6 .96 .86 .88 Grade 5 .95 .82 .83 Grade 4 .95 .81 .82 Grade 3 .95 .84 .85 Grade 2 .95 .82 .83 Summary ofData Set In conclusion the CBM data set can be considered to be of good quality and of high reliability because of the equivalency of the probes used in both reading and writing, the nonnal distributions, high correlations between testing periods, stable 33 validity correlations and good to excellent Cronbach alpha coefficients. The DIBELS data set is also of good quality and reliability. Although the DIBELS distributions are mostly non-normal there are good correlations between testing periods, stable validity correlations and good Cronbach alpha coefficients. Table 15. DIBELS Reliability -Alpha Coefficients (Cronbach) Phoneme Segmentation Fluency (PSF) Grade 1 .85 Grade K .81 Initial Sound Fluency (IS F) Nonsense Word Fluency (NWF) .85 .81 .82 Oral Reading Letter Naming Fluency Fluency (ORF) (LNF) .93 .88 Norm Tables The goal of SD 57's norming study was to develop a series of percentile rank tables or norm tables for both the CBM and DIBELS instruments. The complete tables are displayed in Appendix A. The percentile tables can be considered highly reliable due to factors mentioned previously in the summary of the data set. The CBM tables covered Words Read Correctly, Words Written Correctly, and Words Spelled Correctly for grades one to seven for the fall, winter, and spring testing periods. The DIBELS tables covered Letter Naming Fluency, Nonsense Word Fluency, Initial Sound Fluency, Phoneme Segmentation Fluency and Oral Reading Fluency for testing periods determined by SD 57 that followed the same fall, winter and spring pattern of the CBM 34 testing periods. The format of all the norm tables, which has been in use in the school district since the first set of norm tables were developed, is similar to the example given in Figure 6. Fall !'- Percentile 99 95 ~ 90 '- ·-·85 80 75 70 65 60 55 I50 ~ 45 ., 40 35 ,_ 30 25 20 15 1- ·- 10 ,._.,......... 5 1 GRADE FWE WordS ReiUICiimCtii' Winter Score Score 196 169 159 151 148 143 138 132 127 121 116 111 104 98 94 86 ill 81 '"' 75 ' l , _ 68 59 24 - ~ t- ~ 'h Figure 6. Example ofNorm Table 218 188 169 164 156 147 140 138 133 128 124 118 113 106 98 92 85 80 72 61 36 SPrinfl Score 215 191 180 173 166 158 151 146 140 135 131 125 122 117 112 103 95 ' 84 80 69 41 ~ ~ 'Ci .. ""f,§ Description .... 'ill Well Above r ~ Above Average Avera2e ~ Below Average "''' @ Wellt'elow Averag!,....._ ' 35 Chapter Four: Discussion The purpose of this study was to create a series of norm tables for CBM and DIBELS measures based on a large reliable data set. Comparison of the 1996 norm tables to the 2003 nonn tables created in this study shows some good reasons why the school district made the correct choice to commission new norm tables. Discussion of the CBM results will be followed by discussion of the DIBELS results. CBM Norm Tables As expected all of the norm tables generally show progressive growth across time and grade. Table 16 displays raw scores at the 20th percentile for both Words Read Correctly and Total Words Written over a period of three years beginning with the spring of Grade 1. The scores of the children do not show a drop from spring to fall until the fall of Grade 4 in Words Read Correctly and the fall of Grade 3 in Total Words Written. This might be surprising to some who anticipate the children will lose skills over the summer break and be less proficient at reading and writing in the fall. Table 16. Student Scores for Words Read Correctly and Total Words Written at the 20th Percentile from Spring Grade I to Winter Grade 4 Or 1 Gr 2 Gr 2 Or 2 Or 3 Gr 3 Gr 3 Gr 4 Gr 4 Spring Fall Winter Spring Fall Winter Spring Fall Winter Words Read Correctly 11 16 28 41 49 69 79 64 81 Total Words Written 6 9 14 18 18 21 25 25 28 36 CBM Comparisons CBM Reading. One of the interesting outcomes of this study was the noticeable difference in Words Read Correctiy between the 1996 norm tables and the 2003 norm tables. One of the larger differences in reading scores is illustrated in Figure 7. In grade 6 measures of Words Read Correctly, the fall reading scores at the 50th percentile have a raw score difference of 16 between the 1996 and 2003 norm tables. This means that the students are reading 16 more words per one minute sample in the 2003 sample than they were reading in the 1996 sample. A 1996 raw score of 130 would place a student at the 65th percentile while the same score in the 2003 tables would place them at the 50th percentile. Overall, reading scores were up about 10 percentile points over the year regardless of the grade or testing period. The complete reading comparison tables are available in Appendix D. Some exceptions were larger gains such as 15 percentile points in the fall of Grade 6 and smaller changes in the scores at or below the 25th percentile in Grade 2. These changes in the norm tables validate the school district's decision to carry out the re-norming project. CBM Writing. The overall differences between the 2003 and 1996 writing norms were not as dramatic as the reading differences. There was a slight positive change of about 5 percent overall in Grade 6 and 7 between the 1996 and 2003 writing norms. For example a score of 69 in the spring of Grade 7 fell at the 65th percentile on the old tables but fell at the 60th percentile on the new tables. The remaining grades show little overall change between the 1996 and 2003 norms aside from Grade 4 which shows a better than 5 percent positive change overall. 37 There was a much more noticeable change in the fall writing scores across the grades. These scores increased ten to twenty percentile ranks from the 1996 norm tables. (See appendix D for the complete tables) For example, in the fall Grade 7 writing tables GRADE SIX Words Read Correctly 2003 1996 2003 1996 2003 1996 Fall Fall Winter Winter S:Rring SJ!.rlng !Percentile Score ' 220 155 173 215 ~ 214 ~ 175 164 80 75 70 161 155 L 149 140 60, ~ -140 194 95 i 185 Score 215 195 Score Score 189 Score 217 196 165 179 170 Score 180 .175 ~ 185 154 135 166 159 154 125 146 ' 40 130 114 119 102 30 25 20 105 100 92 87 75 102 96 77 54 21 ! ! I I r 85 65 167 I 144 55 135 50 45 I 124 35 ~ ! ! ! 15 10 5 1 111 I 84 67 39 149 130 81 67 44 f) 181 Well Above Averaa;e 162 149 128 150 137 135 118 141 125 Avera2e 124 109 131 129 93 192 141 139 110 Description " 172 164 159 150 121 225 T 116 111 148 135 123 113 104 97 154 144 136 125 117 81 111 105 79 62 86 26 24 91 69 91 75 49 98 71 38 158 Above Average 142 ,, 130 119 113 108 101 95 89 78 69 55 ·,,· -,, ... '" ·; , .. Below Average Well Below Average ... 27 Figure 7. Comparisons of 2003 and 1996 Grade 6 Reading Scores a raw score of 50 ranked at the 45th percentile. A raw score of 50 in the 2003 tables only ranks at the 25th percentile. This change is also evident in the Grade 6 scores when comparing a 1996 raw score of 39 to a 2003 raw score of 39 as shown in Figure 8. The raw score of 39 changes ten percentile points between the 1996 and 2003 testing periods. Similar changes in percentile rank of the fall writing scores were present across all grades. 38 Although not as dramatic, the fall increase is also noted at the lower percentile ranks as noted in Table 17. CBM scores are used by SD 57 as an indicator of which children may need to be followed up with more diagnostic testing or be referred to --,., '""'· -GRADE SIX Total Words Written 1996 2003 1996 2003 1996 2003 Fall Fall Winter Winter Sprin2 Spring_ --l Score 91 67 Score 96 85 77 59 69 64 62 55 61 ~ ~57 50 65 62 60 57 56 54 52 49 47 !Percentile! Score 99___ 95 95 81 r75 ~ 85 70 80 67 ; I 75 f- f ! -t- 64 70 65 60 55 so I I 45 40 35 ~ i I !" 56 55 52 50 47 1 74 63 56 52 48 46 44 43 41 I 35 33 ·-JI 39 20 5 85 39 25 15 10 Score 44 42 30 I . ~ I 33 28 19 37 35 30 25 16 72 66 44 42 39 36 31 22 - ~ - ' Score 110 87 79 76 73 Score Description 77 Wen AbO..ve Averaae 65 50 68 65 63 61 59 57 46 52 52 79 72 67 62 58 56 55 52 47 44 42 40 37 35 28 19 70 JOT'" 84 73 69 67 61 59 57 55 53 so 50 45 41 38 33 16 Above Average 63 55 47 ._,. 48 45 42 37 33 AverJUle ..:: Below Average "' Well Below Averag4L-, 21 Figure 8. Comparisons of2003 and 1996 Grade 6 Writing Scores a learning assistant teacher for additional support. If, for example, the school district is using a cut score of the 20th percentile to refer for further assistance, a Grade 2 child who scored 8 Total Words Written may not have been not be referred using the 1996 norms as this score would have them placed at the 25th percentile. The same child scores at the 15th percentile using the 2003 norm tables and may therefore be referred for 39 assistance. There is a similar change in scores for other grades as noted in Table 17. Referral of students often occurs in the fall and recognizing students in the early grades who are in need of extra support at this time is critical. Table 17. Comparison of2003 and 1996 Lower Percentile Fall Writing Scores for Grades 2 to 4 Grade 2 Grade 4 Grade 3 2003 1996 2003 1996 2003 1996 Scores Scores Scores Scores Scores Scores 35 12 10 22 19 30 25 30 11 9 20 18 28 24 25 10 8 19 16 26 23 20 9 7 18 15 25 21 15 8 6 16 14 22 20 10 7 5 14 12 20 17 5 3 3 11 9 17 13 0 I 6 3 6 8 Percentile One of the reasons for the differences in the fall writing scores may be that the students are more familiar with the task now than they were when the first norming project was completed. If this were the only reason though, the pattern of increased output in Grade 2 would not occur as these students haven't had a great deal of practice with the task. Another explanation for the increased fall output in writing may be that the administration of the test may have improved for this norming sample. 40 Summary of CBM Changes The large changes to the percentile ranks in Words Read Correctly and slightly smaller but consistent changes to percentile ranks in Total Words Written give strong reasons for maintaining current CBM norm tables in SD 57. DIBELS The series of norm tables created for DIBELS included Letter Naming Fluency, Nonsense Word Fluency, Initial Sound Fluency, Phoneme Segmentation Fluency and Oral Reading Fluency. The norm tables are similar in design to the CBM tables used by the district as shown in Figure 9. The DIBELS tables are often missing scores from one or two of the three testing periods because the test was not administered to students in that period. One of the norm tables created in this study, fall scores for Kindergarten Initial Sound Fluency, compares very favourably with percentile tables created for the same measure by Good et al. (2002) System-wide Percentile Ranks for DIBELS Benchmark Assessment (Technical Report 9). The Good et al study had a sample size for fall Kindergarten of 3 7849 children. The SD 57 and the Good et al study percentile rank scores, for the fall , differ by a maximum of 4 percentile ranks below the 601h percentile. In other words, the same raw score in the SD 57 study would place the student at a slightly higher percentile rank than the same score in the Good et al study. This pattern of similarity is not repeated for the winter Kindergarten Initial Sound Fluency scores. The differences here are very large. A score of 5 in the winter SD 57 sample scores at the 20th percentile locally while only scoring at the 8th percentile in the Good et al study. -· 41 There were similar wide differences in other DIBELS percentiles generated by the Good et al study and this study. A sampling of the differences is provided in Table 18 and 19. Kindergarten Nonsense Word Fluency (NWF) -t-o- Percentile 99 65 ...... 55 - ·so !• ·-!- · ' - 45 38 75 27 20 - - 35 ' - 0 10 - 0 - 15 5 "'WI M f.;}' 18 16 13 ~ 2 1 0 - 0 0 0 .J.'. n Avea.ge .c 11 9 ,, 8 0 0 Above Average 19 5 3 ,_ 22 7 ~f ,,. WeD Above Average 28 25 6 30 25 20 - 33 9 --' Description 39 ,. 11 - - Score 16 14 - 40 .::. - 85 ,..--. 60 SP.ring_ Score - 90 ~ Winter - 95 80 75 70 FaD Score ,. £ - 6 5 3 0 0 0 0 ' .,_ -,- ''«! r ~ '':. ~ '+ "" Below Average t WeD Below Average +-. ··, l'V ' .:> w ' Figure 9 Example of DIBELS Table One possible reason for the widely discrepant percentile ranks between the Good et al study and this study may be that in the Good et al study many of the schools used in the sample had been using DIBELS measurements for three or four years (Good et al., 2002) and may have adjusted their academic programs to teach more of the skills sooner or in a systematic fashion. An example of this might be the Phoneme Segmentation Fluency scores observed in Table 17. There is a very large difference between the two studies. 42 Table 18. 1 Comparison ofKindergarten DIBELS Scores at the 20 h Percentile Winter Fall Spring Good SD 57 Good SD 57 Good SD57 score score score score score score Initial Sound Fluency 4.2 4.8 10.66 5 Letter Naming Fluency 2 1 14.5 6 29 15 Phoneme Segmentation Fluency 7 0 22 4 Nonsense Word Fluency 4.66 0 15 3 Table 19. Comparison of Grade One DIBELS Scores at the 201h Percentile Fall Winter Spring Good SD 57 Good SD 57 Good SD 57 score score score score score score 11 5 26 14.6 Oral Reading Fluency Letter Naming Fluency 25 17 Phoneme Segmentation Fluency 24 6 33 17 39.5 29 Nonsense Word Fluency 13 5 32.5 17.4 43 29.8 Perhaps the Kindergarten students from the Good et al study had been receiving instruction on this skill before the winter testing period. 43 Implications for Practice Development of the norm tables in this study allows teachers a current standard to evaluate student achievement in specific skill areas. Valid, reliable and stable norm tables allow teachers to be confident that the scores students are receiving are an accurate reflection of their ability. The changes in the norm tables from 1996 to 2003 demonstrate the need for up to date norm tables. Given the large changes in the CBM norming Tables from the old sample to the new sample and given the large discrepancies for DIBELS when compared to Good et al (2002) study this researcher recommends that the DIBELS norming Tables for SO 57 be re-normed in the future. It would likely be beneficial for this renorming to occur after teachers have had a chance to become familiar and comfortable with administering the new measures. Additionally, allowing time for the school district to fully implement early basic literacy skills into the Grade K and 1 curriculum is recommended before re-norming the DIBELS measures. Implications for Future Research Given that this study found large differences in percentile ranks when compared to the study by Good et al. (2002) System-wide Percentile Ranks for DIBELS Benchmark Assessment (Technical Report 9) follow up research could focus on the reasons for the large differences. Additionally benchmark scores referred to in the Good study appear unsuitable for the SD 57 population at this time. It would be beneficial to develop benchmark scores for the SD 57 population. Benchmark scores supported by a validity study that compares DIBELS scores to classroom achievement will give more validity and reliability to the DIBELS measures. 44 If SD 57 perfonns a similar CBM renorming study in the future every effort should be made to develop equivalent reading and writing probes before the norming study begins. Although this study found no real differences between the probes the result is more serendipity rather than good research design. If there had been differences in the probes the norming study would have been much more difficult and perhaps the results would be less reliable. 45 References British Columbia Ministry of Education. (2004). District Accountability Contract Guidelines: 2004-2005. Retrieved 03112/05 from I i http://www.bced.gov.bc.ca/schools/sdinfo/acc_contracts/guidelines.pdf Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219-232. ! . George, D., & Mallery, P. (2003). SPSSfor Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Boston: Allyn & Bacon. Good, R.H. , Gruba, J., & Kaminski, R. A. (2001). Best practices in using Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in an outcomes-driven model. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 679-700). Washington, DC: National Association of School Psychologist. Good, R. H., Simmons, D., Kame'enui, E., Kaminski, R. A., & Wallin, J. (2002). Summary of decision rules for intensive, strategic, and benchmark instructional recommendations in kindergarten through third grade (Technical Report No. 11). Eugene, OR: University of Oregon. Good, R. H., Wallin, J., Simmons, D. C., Kame' enui, E. J., & Kaminski, R. A. (2002). System-wide Percentile Ranks for DIBELS Benchmark Assessment (Technical Report 9). Eugene, OR: University of Oregon. Hively W. & Reynolds M. (Eds.) (1975). o r~f r testing special education. Reston, VA: Council for Exceptional Children. Kaminski, R. A., & Good, R. H. (1998). Assessing early literacy skills in a problem-solving model: Dynamic Indicators of Basic Early Literacy Skills. In M.R. 46 Shinn (Ed.) Advanced applications of curriculum-based measurement (pp. 113-142). New York: Guilford Press. Marston, D., Fuchs, L., & Deno, S. L. (1986). Measuring pupil progress: A comparison of standardized achievement tests and curriculum-based measures. Diagnostique, 11, 77-90. Marston, D., & Magnusson, D. (1985). Implementing curriculum-based measurement in special and regular education settings. Exceptional Children, 52(3), 266276. Sax, G. (1997). Principles of educational and psychological measurement and evaluation (4th ed.). Belmont, CA: Wadsworth Publishing. School District 57. ( 1995). Curriculum based measurement norming project manual. Prince George, BC: School District 57. School District 57. (1996). Guidebook for the use of curriculum based measurement in School District #57. Prince George, BC: School District 57. School District 57. (2003). Curriculum-based measurement and dynamic indicators of basic early literacy skills in school district 57: Guidebook. Prince George, BC: School District 57. Shinn, M. R. (1989). Curriculum based measurement: Assessing special children._New York: Gilford Press. Shinn, M. R., & Bamonto, S. (1998). Advanced applications of curriculum-based measurement "Big ideas" and avoiding confusion. In M. R. Shinn (ed.), Advanced applications of curriculum-based measurement (pp. 1-31 ). New York: Gilford Press. 47 Spruceland Elementary School. (2004). School plan for student success. Prince George, BC: School District 57. 48 Appendix A Norm Tables 49 GRADE ONE NORMS READING " Fall Percentile r- . GRADE ONE Words Read Co,.,.ectly Winter S~ r Score 99 95 90' .. 85 Score Score 126 99 ~ 'I 70 ""'\' 46 60 36 so 26 35 ••· --,._ :!! !·l" cc;· -, ii - .,-y, .,, ·,:;- 20 'i;f •; ~ c-:- ;_;:· --.:-,, ~ 20 10 5 1 ' '" Averaee 22 25 15 :.::¥··· ~ ~ 30 40 ··:m " . ~ ~T 42 45 t- 53 ,-· f Above Average - 55 ·- wen AboVe Averaae 80 65 ~ 82 70 60 75 Description ' ..,. ·;; T : ......-- 4. 18 15'; ·;: .. -,:- ~ rr ~ ___ ~ ·' ~ 13 Below Average 7 WeU BelQw Averaee 11 9 4 0 -,,- N.B. Grade One students were tested only once, during the spring norming period. 50 GRADE TWO NORMS READING ' h' Fall Score Percentile 99 ; 153 125 137 157 126 139 . 80 87 106 117 69 91 104 75 75 70 65 55 50 45 40 35 @; 30 25 20 15 10 5 1 84 62 56 55 '" 60 ' 78 70 • 98 86 63 78 32 52 65 38 54 28 21 18 16 13 10 7 1 56 31 28 23 18 12 3 . . Avera2e 73 45 >- Above Average 92 41 36 Wen Above Average itt 110 98 Description ,, 126 115 ' ..·t ' ~ 186 104 95 ~ ~ Soriru! Score 90 85 ~ .Winter Score 168 95 r GRADE TWO WordsRetidCorreCiiY M h 60 •:c 46 41 32 25 18 8 ~ ' '· Below Average -. ... " _ ' ,, ' Well Below Average .,_ i) '"'" 51 GRADE THREE NORMS READING Percentile ·--1- · GRA.DB-:.J.nHilR WtialsRMII."CA .. ' 9.9 95 90 85 80 75 70 65 !- - ~ 213 174 156 146 136 129 120 116 110 104 213 85 78 70 66 60 49 41 ,, 15 10 1 196 162 141 130 122 119 109 103 98 94 89 20 5 Whiter Score 60 55 50 45 ,_ , 40 35 30 25 Fall ·- SIJr!AI '' '" "' 36 .. 24 15 Score 99 94 89 83 79 76 69 58 44 34 12 ~ ~ ~ r Score 177 160 152 146 136 129 125 120 115 • .• . . ~ ~ ,A\. ... ~ ' Descri.ption ·•l ~ Above Average .-... Above Average .. 109 104 99 94 89 83 79 70 "62 42 27 .... ·' Averai!e ,, Below Average !t " WeD Below Average ' ' .§'", 52 GRADE FOUR NORMS READING GRADE FOUR . Words Read ConwCtly FaU Score Percentile 99 95 :,-_ , 90 85 80 75 70 65 ,. ~ q 55 so "' '" ,. 45 40 35 30 25 20 15 .. ,_ 5 1 ' . 190 173 162 147 138 132 127 121 114* ?{, ,, 108 103 ' 95 89 83 77 70 64 58 so 10' " [?:< · wWinter 37 26 s.,.-D...x . . l§i" ' .. ; ' 206 186 172 162 152 147 141 134 11130 124 ' t; .:ri Well Above Average ·"- Above Average ,,, p ; _, ~ .f $ Averaae 119 114 110 104 100 95 89 79 69 57 36 > .;_: f Description Score Score 208 181 168 160 148 143 133 128 123 117 112 108 100 94 89 85 81 74 62 47 29 '; '· ' 0 Below Average "' WeJI..JJ.eJow Avera•' ' ~f " " 53 GRADE FIVE NORMS READING GRADE FIVE Words Read Correctly Winter ~ r Score Score Score 196 218 215 169 188 191 ,, Fall Percentile 99 95 90 f- 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 r- 10 5 ,, .c.J 169 159 164 156 147 140 138 133 128 151 148 143 138 132 127 121 ,, '"" 68 59 24 173 166 158 151 146 140 135 124 116 111 "1041 98 94 86 81 75 180 -,_, 118 113 106 98 92 85 80 72 61 36 131 -,; 125 122 117 112 103 95 84 80 69 41 ~ Description " Ab()ve Avenae Above Average ·- l AV.e.raae ,- '; ; ' ' ~ ' '' Below Average Well Below Avera2e _, \ 54 GRADE SIX NORMS READING Percentile Score 214 99 95 194 Score 217 195 196 180 185 80 161 166 172 70 149 154 60 !::JI9 5D.. 130 40 119 155 65 135 45 124 35 25 20 '· 15 ' 84 67 39 135 tAl Aver.ae 131 ~ 136 " 117 111 96 105 79 86 26 38 91 69 l! 125 98 71 •-' ,, 144 r: iii '!: 150 116 111 102 ' Above Average )A 146 129 77 ti! 154 [, 124 92 10 159 139 111 105 100 30 164 150 ,, WeD Above Averue 179 159 144 55 5 1 173 167 Description Score 215 175 tt 75 ~ ki ' 90 85 ~ " GRADE SIX Words Read Correctly Fall Winter SRrin2 - ' l' T ,, ·. ~~ . Below Average ~ WeD Below Average ' "" ,- 55 GRADE SEVEN NORMS READING . ,_. 61UDESEJI'BN- ·-, ~ . . .~ Percentile FaD Score Wlnttr Score 95 207 209 ' 99 90 ~ 85 ·-f·- 193 70 155 60 144 148 so 133 139 40 121 65 55 45 35 30 150 138 127 11 8 113 25 105 15 93 20 10 5 1 101 84 72 46 213 197 185 172 159 175 166 ~ 160 ~ 154 87 73 51 170 165 158 148 145 139 ,'!;, o ~~r ·.,.; >' ~. rr ~ .: ..;; Description Well Above Avera2e Above Average .:;, 152 ' 143 133 127 123 117 110 104 97 r~ ~ ·.· 239 181 169 _'' Score 195 176 ~ ·. · sorinu 236 80 75 -r: ·- 230 Or " Average 134 129 122 116 109 Below Average -· "' 91 Well Below Average '" 101 w 74 60 ;u' -iii 56 GRADE ONE NORMS WRITTEN EXPRESSION ~ Percentile Score 99 9"5 '-- 90 1?1' 85 80 In GRADE ONE Written Expression Spring ~ r vvsc TWW vvsc Score !'1 37 ''"' ' ~ 75 70 " 65 60 r- - ~ 55 so 45 40 35 > " "" ' 30 25 20 0 ' >'@ 'ij.j VVeHAbove 17 16 14 12 11 15 14 5 4 2 ' 23 16 12 11 10 9 8 '\'' 20 23 21 18 7 15 \lf 32 29 6 ., , I 10 5 1 ' Description Score 25 13 il ~ ' Above Average 10 ' ,., 9 8 7 6 6 5 4 4 3 2 0 ,, Average ,, i [ii ·;: Below Average _,_ VVeHBelow 0 N.B. Grade One students were tested only once, during the Spring norming period. > '>\ 57 GRADE TWO NORMS WRITTEN EXPRESSION Fall . TWW wsc Percentile Score 99 95 90 85 , _ 80 ~ :- .. GRADE TWO Written Expression ¥! 75 70 65 60 55 so 45 40 35 30 25 ·,-;:-· 20 15 10 5 1 45 32 28 26 - 24 22 ....... 21 20 19 17 1s 14 "" 13 12 11 10 9 8 7 3 0 40 28 23 Winter wsc TWW Score g 55 "' 52 41 37 35 20 32 18 ·-- 30 17 28 16 26 14 25 24 13 12 22 11 21 10 20 19 18 9 8 17 7 15 6_·-· 14 12 5 4 11 2 7 0 3 30 -- ro ,. 27 24 22 21 20 19· 18 16 Spring wsc TWW Score 59 47 41 37 34 32 31 30 29 27 26 15 14 13 12 12 11 10 25 23 22 21 19 18 16 5 11 4 ,.l 14 56 45 37 32 30 27 26 _ 25 24 22 21 20 19 18 16 15 14 12 11 8 2 Description cw· WeD Above Above Average Avera2e 'M Average . Below . ,..,...,... Well -n ..•. 58 GRADE THREE NORMS WRITTEN EXPRESSION Percentile 99 95 90 k GRADE THREE Written Expression Spring Fall Winter wsc_ wsc wsc TWW TWW TWW 66 47 Score 41 37 85 37 80 ,_, 1- 35 75 32 30 70 65 29 bU 28 55 27 !_ ... 50 ''l 10 5 1 35 32 29 26 25 24 23 25 45 24 23 40 35 22 30 · -,....,... 20 25 19 20 18 15 16 14 11 "6 62 53 60 43 22 ~ ? 7 3 49 36 34 32 41 39 37 35 33 31 29 29 28 26 25 23 21 20 26 24 23 21 19 17 16 38 ·- 61 50 30 18 14 4 69 59 27 ''' 13 11 3 Score 51 44 45 42 39 20 19 18 17 16 15 13 10 Score - 46 35 44 442 41 39 36 33 32 33 ;;; 32 ·' 30 28 27 25 23 29 28 27 25 23 22 19 47 44 43 42 .:: 39 i- 64 54 37 34 21 17 0 .R. Description Well Above 13 0 .•. - Above Average m 31 17 f< h r ~ .' '{..:' .. ·,? . -•., .;;. ,,;: '"' Below Average % ~ o ~ .. 59 GRADE FOUR NORMS WRITTEN EXPRESSION Fall Percentile ~ T 7} 60 99 95 90 85 80 75 70 65 ,_, 60 1i 55 so 45 40 " 35 30 25 20 15 10 5 1 52 48 45 43 41 40 39 37 34 33 32 30 28 26 25 22 20 _,.- 17 6 GRADE FOUR Written Expression Spring Winter Score 64 54 49 45 42 40 39 37 35 33 31 30 27 26 25 23 21 20 18 13 4 70 63 Score 57 18 11 83 69 66 61 59 56 55 52 49 47 51 48 45 42 41 39 37 39 23 69 60 .. 55 55 51 47 45 44 42 40 37 35 34 32 30 28 25 wsc __ . wsc_ ,_rw\Y wsc _t. Tww 35 ~. 34 33 31 29 28 26 23 20 15 8 " Score ~~ ' 28 23 6 - (!2 . '0 WeD Above 59 56 45 43 42 40 38 36 34 32 77 67 Description ~ ""- 54 Above Average 52 49 ., 47 X "' 45 43 Average 41 39 'i . . 37 ' 35 x• ... 33 Below Average 31 28 ~ __ 24 19 '" 5 WeHBelow 60 GRADE FIVE NORMS WRITTEN EXPRESSION GRADE FWE 'Written EXPression Spring Winter Fan TWW wsc Score 1---, Percentile ~ I-' ,_ 99 95 9085 80 75 70 65 I-,--' 60 55 ·- so 45 40 35 30 25 r-20 15 ,_,....,.... 10 5 fo' 1 ·- - 80 69 63 59 56 52 f- 50 48 47 46 76 ·66 60 57 53 50 48 46 44 TWW wsc Score 81 72 65 62 58 55 53 52 50 48 47 45 43 41 42 40 38 36 42 40 "' :"' 39 37 35 40 32 ' 25 21 14 28 26 22 19 12 ; 44 :::rr 29 33 30 77 68 63 59 55 37 35 32 ' 29 24 11 53 51 B 49 47 45 43 41 40 38 36 34 32 30 27 23 8 ~ TWW wsc Score 93 80 71 67 90 77 69 65 61, 64 61 59 58 55 52 51 49 47 45 43 ..... ' 41 39 36 1-. 34 27 12 T -, ~ 58 56 39 37 34 30 25 10 Description ilo WeDAbove -· ~ -,,. Average 45 43 ~ Above Average 54 52 51 49 47 ~ r ,,· W· ,_- ·-· 1'\'i, " ,; ' _., • '- ! Below Average .,.,_ WeUBelow ~ ·- 61 GRADE SIX NORMS WRITTEN EXPRESSION GRADE SIX Written Expression Sprina Winter Fall I TWW wsc TWW wsc TWW wsc Percentile 99 95 95 81 85 80 75 70 65 60 55 70 67 64 62 59 57 56 90 ~ o 45 40 35 30 25 20 !- · 15 '- 10 5 1 Score 75 ~ r- 96 85 69 72 69 66 65 62 60 57 73 64 61 59 57 55 '" 53 51 55 52 so 47 44 42 -· 39 35 33 28 19 93 79 - 49 47 45 42 38 35 33 29 26 19 77 56 54 52 49 47 44 42 39 36 31 22 Score 94 82 110 87 70 66 64 62 59 57 55 76 ' 73 70 68 65 63 61 50 48 46 55 74 52 44 42 39 37 32 28 17 79 59 57 52 50 47 45 41 38 33 16 Score 103 84 78 74 72 68 Description Well Above -,, Above Average 66 63 60 58 57 55 53 50 47 45 42 39 36 30 13 Avera2e 11 .r Below Average ~ Well Below ,, -· ·,., 62 GRADE SEVEN NORMS WRITTEN EXPRESSION GRADE SEVEN Written Expression Fall wsc TWW Percentile Score ' 99 105 100 r·95 88 86 ,_. 1~ ~ 90 85 80 75 10 65 -· 83 80 .... 76 73 74 79 71 69 Whiter wsc TWW Score 105 104 93 89 119 95 84 82 85 77 74 72 75 .70 79 75 73 63 67 65 69 60' 65 76 71 69 67 ·-· 80 70 77 72 68 81 71 60 66 so 61 59 63 40 56 53 58 30 25 20 52 50 47 50 47 45 54 51 48 46 52 42 40,_ 46 26 24 25 55 45 35 15 10 5 1 63 59 54 45 41 35 27 60 56 52 42 39 33 23 65 61 57 46 37 63 58 56 54 52 49 43 35 ' . ' SPilil ' .. TWW wsc 67 63 62 59 57 54 48 40 <,' Score ~ Description 117 ·, ~ 91 84 WeD Above 76 73 7J Above Average 79 69 67 65 63 61 59 57 55 52 49 47 44 38 22 ·--' ., :. ~ 'lA !(' Below Average ' WeDBelow ... f ~ 63 KINDERGARTEN NORMS INITIAL SOUND FLUENCY Kindergarten I nidal Sound Fluency (/SF) " Percentile 99 95 ' - 90 85 r80 ""'·. 75 70 65 .,': 60 55 50 ,__, 45 40 35 30 25 20 15 JO 5 1 Fall Score 43.08 26.84 23.48 19.82 16.96 15.00 13.00 12.50 11.00 ~ 10.00 9.10 8.00 7.00 6.37 6.00 5.40 4.81 3.37 2.06 0 0 . Winter Score 49.86 33 .39 28.70 25 .00 23.00 20.13 17 61 16.00 ' "' t5J)() 14.00 11.92 "' 10.55 9.22 8.43 7.43 6.81 5.00 3.90 2.23 0 0 Sprin2 Score ~ Description - w Well Above Average :: Above Average - -- - : ,- ~ r;ru · ~ Average -- 'L 0' - 2\ - - - -- Below Average Well Below Average "' ,,, 64 KINDERGARTEN NORMS LETTER NAMING FLUENCY ' ' Percentile f- - 99 95 85 75 60 55 - ,__ ~ 45 40 35 30 25 I' 20 15 10 5 1 63 73 27 41 49 23 17 14 80 65 43 33 90 70 '" Kindergarten Letter Namin'g Fluency (LNF) Fall. Winter K SP.rinR Score Score Score Description 47 29 8 23 7 26 20 6 18 4"- 5 3 3 2 I 1 0 0 0 ,. 33 28 10 58 36 12 "" 45 41 39 37 35 33 13 25 -, 10 20 27 18 8 6 4 2 0 0 ' 22 11 i 15 12 9 5 0 H _, Av.era2e -' t Above Average 30 15 ...... Well Above Avera2e-}' 43 Below Average - it ' ""' Well Below Avera2e \j "-' 65 KINDERGARTEN NORMS PHONEME SEGMENTATION FLUENCY I Percentile Kindergarten Phoneme Segmmtlltion Fluency (PSF) I Fall Score Winter Score Sprin2 Score - 43 48 - 99 95 90 - 37 44 80 75 70 - 27 22 19 38 85 '!, ;_, ·- - - 65 60 55 50 35 30 1- 10 5 1 - 10 - 7 8 I'' 25 20 1- · 11 - 40 r- • - - - 43 34 31 16 14 - 45 15 33 - -,, "" -' .__, Well Above Avera2e . ~ '"" ~ Above Average 27 'i 25 21 17 5 3 15 12 10 8 1 6 0 0 0 0 0 .J' Description 57 67 "' 4 2 0 0 0 r ~ ,i, ., Below Average Well Below Average '" t\i ._, 66 KINDERGARTEN NORMS NONSENSE WORD FLUENCY _, .•• '-' ~ > Fall Score Percentile 1#"- 85 80 70 " [ ~ 65 60 ·55 45 ··:· 40 l.. ·, 20 15 5 1 ~ 14 9 7 19 18 16 · ·oc 6 -- 5 3 2 - - - - - - fi ·'!'. -- '' Score 75 39 33 28 25 20 11 - 25 f 27 - - 35 30 "'"' Score 38 - 50 f- Nonsense Word Fluency (NWF) Spring Winter - 75 F-· r - 99 95 90 ·<1:. r 13 0 12 11 9 8 0 5 1 0 6 "' Description " J!.L Above Average • t J •. Averag_e -"' Below Average ., 0 3 0 0 Well Below Averaae 0 0 ,,. 0 0 0 0 • Well Above Average 22 16 "' ~ <>·•"\ ~ 67 GRADE ONE NORMS LETTER NAMING FLUENCY 68 GRADE ONE NORMS PHONEME SEGMENTATION FLUENCY [ Grade One Phoneme Segmentation Fluency (PSF) Winter Fall Sorin2 Percentile f,-., l'<-· Score 99 95 90 85 80 75 70 65 60 55 50 45 _!0 35 30 25 20 15 10 5 1 70 61 50 - f- 47 43 39 36 33 29 25 21 18 14 12 10 8 6 4 2 0 0 Score 73 65 60 53 50 48 45 43 42 40 38 35 33 30 26 21 17 13 '"9 4 0 Description Score 78 73 62 59 51 54 Well o ~ Averae:e Above Average - 51 49 46 44 42 40 38 36 I 33 -, - 31 29 24 17 10 0 ~r i1!. Below Average ., j)L Well Below Average -.,+ . h . 69 GRADE ONE NORMS NONSENSE WORD FLUENCY Grade One Nonsense Word Fluency (NWF) 1\ Percentile 99 95 90 ,_ I' T 85 80 75 70 65 60 55 10 5 I Winter Score SDring Score 42 67 100 89 47 36 32 29 26 23 21 19 17 15 13 11 50 " 45 A 40 35 30 25 20 15 Fall Score 9 ·-f- 7 5 3 1 0 0 106 74 ~ 59 55 51 47 44 41 '<' 38 ' " -·', 141 117 • 41 21 17 15 11 5 1 Above Average 58 54 50 31 24 r 64 47 28 Well Above Averae:e 84 76 70 36 34 Description ~ 44 38 35 "' Averae:e ·' ' #£• .·' ·" w ·v,· ~ ~~ ~~ ~ " .....•. ,.. 33 Below A 20 Well Below Average 30 27 12 2 ' '"' :r r ~ j ,' ~ 70 GRADE ONE NORMS ORAL READING FLUENCY 71 Appendix B Descriptive Statistics 72 Descriptive Statistics of CBM Instruments Grade One Results Writing TWW Mean S.D. Min Fall Winter 13.45 8.28 0 Spring Max Skew Kurtosis 40 0.71 0.13 Max Skew Kurtosis 34 0.92 0.5 Max Skew Kurtosis 144 1.15 0.8 Grade Two Results Writing TWW Mean S.D. Min 8.84 16.8 0 Fall 22.21 8.05 1 Winter 26.84 10.98 0 Spring Max 53 58 74 Skew .72 0.66 0.73 Kurtosis 1.04 0.88 1.3 Grade Two Results Writing WSC Mean S.D. Min 12.72 8.05 0 Fall 17.98 9.47 0 Winter 22.63 10.83 0 Spring Max 50 54 71 Skew 1.21 1.01 0.97 Kurtosis 2.37 1.59 1.7 Grade Two Results Reading WRC Mean S.D. Min Fall 51.72 39.57 0 Winter 67.65 39.80 0 Spring 81.03 42.32 7 Max 210 162 209 Skew 1.00 0.31 0.33 Kurtosis 0.75 -0.93 -0.46 Grade One Results Writing WSC Mean S.D. Min Fall Winter 9.77 7.03 0 Spring Grade One Results Reading WRC Mean S.D. Min Fall Winter 36.02 29.60 0 Spring 73 Grade Three Results Writing TWW Mean S.D. Min 26.59 11.06 0 Fall Winter 31 .9 12.16 0 12.39 Spring 3 5.01 0 Max 78 72 74 Skew 0.96 0.34 0.23 Kurtosis 2.1 0.05 0.61 Grade Three R esults Writing WSC Min M ean S.D. Fall 2 3.00 10.94 0 2 8.34 Winter 12.19 0 Spring 31 .72 12.25 0 Max 70 70 70 Skew 0.93 0.39 0.32 Kurtosis 1.52 0.01 0.34 Grade Three R esults Reading WRC M ean S.D. Min 43.34 2 Fall 88.65 Winter 10 1.72 42.41 8 11 0.31 41.25 Spring 9 Max 217 216 225 Skew 0.47 0.26 0.18 Kurtosis -0.15 -0.27 -0.20 Grade Four Re sults Writing TWW M ean S.D. Min Fall 3 5.44 12.89 5 Winter 39.28 13.54 0 4 6.03 Spring 15.00 0 Max 77 79 95 Skew 0.36 0.10 0.05 Kurtosis 0.22 -0.02 0.31 Grade Four Results Writing WSC M ean S.D. Min 32.07 12.54 Fall 1 36.42 13.22 Winter 0 Spring 4 3.12 14.84 0 Max 76 75 91 Skew 0.30 0.12 0.02 Kurtosis 0.11 -0.05 0.12 Grade Four Re sults Reading WRC M ean S.D. Min 102.89 40.89 Fall 7 Winter 11 4.07 40.13 25 Spring 12 0.29 38.30 25 Max 198 225 224 Skew 0.15 0.11 0.10 Kurtosis 0.1 0.17 0.07 74 Grade Five Results Writing TWW Mean S.D. Min 4 43.73 14.37 Fall 45.53 14.26 0 Winter 51.64 15.77 0 Spring Max 85 82 110 Skew 0.23 -0.03 0.20 Kurtosis -0.17 0.25 0.90 Grade Five Results Writing WSC Mean S.D. Min 40.84 14.21 2 Fall 43.84 14.10 Winter 0 49.84 15.74 0 Spring Max 81 80 105 Skew 0.27 0.03 0.23 Kurtosis -0.20 0.07 0.78 Grade Five Results Reading WRC Mean S.D. Min 115.05 15 Fall 36.08 Winter 121.50 37.83 9 Spring 130.57 38.55 12 Max 216 230 233 Skew -0.02 0.08 -0.09 Kurtosis -0.37 -0.27 -0.32 Grade Six Results Writing TWW Mean S.D. Min 53.75 16.38 Fall 0 Winter 55 .71 15.87 11 59.14 Spring 17.19 0 Max 111 103 122 Skew 0.18 0.28 0.17 Kurtosis 0.25 0.06 0.89 Grade Six Results Writing WSC Mean S.D. Min Fall 51.01 16.54 0 Winter 53.36 16.17 11 Spring 56.96 17.33 0 Max 104 98 121 Skew 0.20 0.26 0.15 Kurtosis 0.07 -0.06 0.68 Grade Six Results Reading WRC Mean S.D. Min 128.01 42.47 16 Fall Winter 131.48 42.21 20 137.78 41 .28 Spring 19 Max 220 252 277 Skew -.07 -0.12 -0.21 Kurtosis -0.32 -0.18 0.22 75 Grade Seven Results Writing TWW Min Mean S.D. 61.82 16.62 24 Fall 16.99 21 63.20 Winter 11 16.77 65.40 Spring Max 118 127 134 Skew 0.28 0.36 0.39 Kurtosis -0.12 0.36 1.40 Grade Seven Results Writing WSC Mean S.D. Min 16.43 20 59.40 Fall 60.87 16.80 20 Winter 63.29 16.90 10 Spring Max 115 125 134 Skew 0.21 0.36 0.41 Kurtosis -0.12 0.37 1.38 Grade Seven Results Reading WRC Mean S.D. Min 40.49 40 135.32 Fall 139.16 40.66 35 Winter 143.93 40.18 40 Spring Max 248 263 278 Skew 0.29 0.18 0.14 Kurtosis -0.27 -0.22 -0.07 Max 84 93 84 Skew 2.065 .905 .319 Kurtosis 7.049 1.503 -.002 Max 47.6 9 53.0 0 Skew Kurtosis 1.340 2.243 1.000 1.046 - - - Max Skew Kurtosis 74 70 1.257 .471 1.371 -.843 Max Skew 43 100 1.628 1.930 Kurtosis 2.384 7.264 Descriptive Statistics ofDIBELS Instruments Grade Kindergarten Results LNF Mean S.D. Min 10.04 Fall 11.41 0 Winter 20.06 14.94 0 Spring 29.85 15.78 0 Grade Kindergarten Results ISF Mean S.D. Min 11.164 Fall 8.5667 0 8 Winter 14.086 10.419 0 1 2 Spring Grade Kindergarten Results PSF Mean S.D. Min Fall Winter 14.311 15.061 .0 Spring 20.65 16.41 0 Grade Kindergarten Results NWF Mean S.D. Min Fall Winter 7.01 9.08 0 Spring 14.89 13.87 0 - - - - - - 76 Grade One Results Reading LNF Mean S.D. Min 33.17 17.04 0 Fall Winter Spring - Max 77 Skew .245 Kurtosis -.576 Grade One Results Writing PSF Mean S.D. Min Fall 24.50 19.05 0 18.83 Winter 35.90 0 Spring 41.07 16.44 0 Max 72 78 74 Skew .550 .064 -.421 Kurtosis -.665 -.428 -.208 Grade One Results NWF Mean S.D. 19.77 17.06 Fall Winter 37.41 21.48 Spring 53.59 30.40 Min 0 0 0 Max 128 121 146 Skew 1.905 .654 .887 Kurtosis 7.703 .771 .543 Min Max Skew Kurtosis 0 0 107 145 1.930 1.030 4.174 .729 Grade One Results ORF Mean S.D. Fall Winter 19.73 20.79 Spring 39.24 28.29 - - - - - - - 77 Appendix C Comparison of 1996 and 2003 Reading Scores 78 GRADE SEVEN Words Reail Correctly 1996 2003 1996 2003 1996 2003 Fall Winter fYmter 8Rring Siirlnl!:. Fall Percentile Score ,_ I· I 99 95 90 85 75 70 ' 60 65 55 50 45 r= 4035 1- 30 I 25 20 15 ~ 207 l 193 176 80 ····l 169 ~ I 230 1D 5 '1 1 159 155 150 144 138 133 127 121 118 113 i 105 101 93 84 72 46 I Score 220 194 183 174 165 158 152 144 137 131 126 123 117 114 106 100 94 89 74 58 32 Score Score 236 214 199 185 177 170 209 195 181 172 166 162 160 154 148 143 139 133 ~ 123 117 110 104 97 c;87 73 51 155 148 143 136 132 127 121 116 111 105 101 92 79 66 34 239 213 197 185 175 170 165 158 152 148 145 139 134 129 122 116 109 101 91 74 60 ;p Score Score w A., "'· Description ' 218 ~ · · ·•: ' ···:. ·7·> ·:r .<' :;;:·. ~ ''i ·'' · 204 193 181 175 169 Well AbOye r~ ~ ~ rfj Above Average " 161 155 150 143 r}36 130 124 119 113 Averati . til 103 97 84 69 39 Well Below ~ r 109 ., ~ w :··· 't Below Average "" ,, i ~ 'L ,• . B: 11 79 GRADE SIX Words Read Co"ecdy 2003 Fall Percentilei Score 1 99 95 90 85 80 ,,; I t ! ! ~ 75 70 65 60 55 i 155 149 I 144 140 i 135 130 124 119 ! i 111 105 ·-r so 45 40 35 30 i 25 ~ / 15 10 5 1 214 194 175 167 161 I 100 92 ! 84 77 67 39 2003 1996 2003 1996 Fall Winter Winter Sprinf! Sorinll 1996 Score 215 185 164 155 149 140 135 130 125 121 114 110 102 93 87 81 75 67 54 44 21 Score 215 195 180 173 166 159 154 150 146 139 135 129 124 116 111 102 96 91 79 '!. 69 26 Score 220 189 175 165 154 148 141 135 128 123 118 113 109 104 97 91 81 75 62 49 24 Score 217 196 185 179 172 164 Score 225 ,, Description .-Ll ~ 192 181 WeD Above Averae:e 170 102 158 Above Average 149 -y 159 154 150 144 141 136 131 125 117 125 * Averae:e 105 98 86 71 38 " 78 69 ' WeUBelow Averae:e 55 111 1 142 137 130 "' ,- --- ' 119 113 108 101 95 Below Average 89 27 80 81 GRADE Four Words Rud Correctly 1996 1996 2003 2003 1996 2003 Sorinll. Fall Fall Winter Winter SPJ'i.Q.a. !') !Percentile! Score ~ I I 99 190 90 162 173 95 147 85 80 t. I I ! ! 70 132 127 121 55 l 45 ! 95 35 83 77 70 64 ! 58 65 60 114 i 108 S0.-4 103 40 30 25 I 138 75 20 15 f- ~ 1 Score 191 168 149 140 131 125 118 110 105 100 92 1 37 26 208 181 168 160 148 ,, 143 133 128 123 117 112 108 87 80 "" 100 73 94 67 89 62 85 89 ., 50 Score ~ 50 41 24 11 81 74 62 47 29 Score Score 206 199 176 186 161 148 138 132 125 120 115 107 103 96 91 88 81 75 66 58 47 34 13 172 162 152 147 141 134 130 124 119 114 110 104 ·' 100 95 '"' 89 79 69 57 ' 36 Score 211 188 "' Description "' 166 Well Above Avera 153 146 138 Above Average '" 134 128 124 118 111 Avera2e 105 99 93 88 81 Below A r ~ 72 64 58 Well Below Aver..ge 41 17 ' 82 --cc 'ifr GRADE THREE Words Read Correcdy 2003 199.6 1996 2003 1996 2003 ··,, ~~ Sotlill! FaD Fall Winter W"mter Description Score Score !Percentile: Score Score Score Score '\;\: "l 208 213 99 f 196 180 ' 213 190 1-i 175 171 177 163 174 95 ! 162 141 154 160 162 Well Above AveragL 90 139 156 ! 150 ! 152 123 146 140 85 : 130 rr·"ffw "' " 146 £ 138 80 122 113 136 126 i 130 Above Avera2e 75 I 119 104 129 118 136 123 99 120 113 129 70 109 11 7 65 103 93 116 109 125 ·rr --: ,, __<'\' __ •, F:all !Percentile! Score 99 80 ! 95 I 69 90 85 80 63 I 59 56 52 75 70 I 1.' I i 50 ~~ -+ 47 55 48 I I 45 ·t 50 40 40 30 36 20 , 31 35 38 25 ! I 46 44 42 33 15 ! 5 1 I 21 .14 10 29 25 1996 1996 2003 2003 1996 Fall Winter Winter Sprinf! SprlnJl, Score Score Score Score Score 90 70 81 80 93 78 53 72 68 80 56 65 48 58 52 46 44 42 41 40 38 36 35 33 31 30 62 55 55 53 52 50 48 47 45 43 41 ,40 37 28 26 23 29 10 11 19 63 60 57 35 32 24 ~ 52 50 49 47 45 43 42 40 38 36 34 32 29 24 17 71 67 73 68 ' m Description 64 51 59 57 56 54 52 Averaee 50 49 46 ,. 45 -/(, ). :: 49 47 45 43 41 39 36 34 27 12 61 40 37 33 27 22 ,(;i ~ u Above Average ;,: "% 11!. ,;; 43 c Well Above Averag!_ 64 61 59 58 55 52 ,c -'!\" , -_,,_ )_ ·-·-' . Below Average " Ill Well Below r~ ' ' 89 GRADE Four Totlll Woitls Written %003 1996 2003 ~ ~ Fan Fall Winter Winter ~S r ;r Percentile! Score ~ 71 ' 95 ! 60 l ! ~ 90 Score Score Score Score 54 63 65 69 64 l- s-2- -4:-9- - 85 ! 80- 45 40 43 55 75 43 38 47 70 48 10 41 36 45 39 33 42 40 65 34 44 37 31 40 55 :........... 50 -t-....;3;;;,;4;..._.__,;;; 2.:;.. 9 __ 39 i 45 1 33 28 37 40 I" 32 30 28 35 26 35 32 22 24 23 21 20 17 13 I 30 25 26 20 25 10 20 15 5 1 I ~ 6 25 ll 8 34 30 28 25 23 18 74 83 "' 1"'= Sn_rlnll Score 70 ~~~ ~ ~ o 61 61 47 56 54 Above Average 50 41 49 46 - 37 45 42 40 39 35 33 32 52 47 43 42 40 30 28 25 23 38 18 23 36 34 32 21 28 ~~ ~ r _ _ _ _ _ _ _...,... 55 43 Description 8.5 53 45 ' "> ,, 48 44 Average - - - - ~ _,.. 38 . .,..._._ 36 35 33 31 28 Below Average 25 21 ~~ Well Below AveragL ~ ~~ 90 GRADE THREE Total Words Written 2003 . ! Fall Percentile Score 99 - 66 95 90 ' -· ! 85 I 75 i 80 ] 70 i I 65 60 55 II e r I i I 29 27 37 35 30 ·.", 10 5 1 29_ Description 60 69 64 49 46 51 53 & Well Above 42 42 44 47 62 53 45 39 38 36 51 44 41 38 36 56 59 43 45 42 42 40 39 '" CiL ili . ,,. 39 25 23 30 31 34 35 Averue 23 20 28 27 32 33 ' 25 25 28 30 20 2S 18 .£15 2( ...;::; ~ 4 7 0 10 24 21 19 19 18 16 16 14 20 18 14 11 f! 6 15 12 9 3 32 29 26 23 21 20 14 32 29 26 23 17 13 37 35 34 33 31 30 27 28 23 24 26 17 ~ 18 ~ Above Average 37 24 r 50 47 34 I 22 ! ~ 34 35 25 20 15 1996 2003 26 I 30 1996 28 45 40 53 32 41 2003 Fall Winter Winter Sprin2 Sp_rlnJt Score Score Score Score Score 43 38 34 31 30 47 27 50 199.6 ,. . ~ ~ Below Average ~o ~ ~ ~ 91 2003 Fall Percentile Score 99 45 95 32 90 28 85 26 I,..,...-' 80 ,_ -- 1_.24 75 22 f:_ 70 21 65 20 60 19 55 17 GRADE TWO Total Words Written 1996 Fall Score 44 OS'•' 27 23 21 ·-· so 45 40 35 30 25 20 15 '-• ,_. 1.0 .,. ' 5 1 15 14 13 ·- 1112 10 9 8 7 3 0 - 19·- 18 17 16 15 14 12 12 10 10 ' -.,. 9 • 8 7 6 5 3 1 1996 2003 55 41 35 32 30 28 26 25 24 22 21 20 19 18 17 15 14 12 tl 7 3 2003 Winter Spring Score Score Winter Score ,, 50 38 33 30 27 26 24 23 22 21 20 19 59 47 41 , 37 34 32 31 30 29 27 26 11 25 23 22 21 19 18 16 6 3 11 4 18 16 15 ·- 14 12 9 14 1996 Spring Score ,, . ' . Description ~ 57 -'" 45 4()_ -- WeiiAJ\n.V;37 34 '"' 32 Above Average 31 30 28 26 25 24 > ~ '\ •, Avera2e "" . "' j 23 22 20 19 17 15 11 11 6 ~ Below Average .Well Below. '''· . ;;:-, 92 GRADE ONE Total Words Written 2003 1996 Fall 2003 Fall Winter "Percentile ·score Score ! Score ,_ 99 95 ;;. 90 85 -;- "' '· < i I 65 60 40 - 30 35 __ _- " 10 I• 5 1 ~ I '" 18 19 21 20 16 17 14 15 12 11 12 16 ·---,- - 'li ~ I I Above J Average I ' •1 r 9 11 10 9 !i 7 7 ! Below 8 6 6 5 4 2 0 ---- 13 8 I 15 23 10 - , ·DeseriPtion 25 13 25 20 29 25 23 15 ,_-, ; 1996 Spring SCI.Ire 33 .· 21 70 45 Score 37 Score 75 50 Spring _, _ 80 55 2003 1996 Winter w 5 -, 4 2 0 ~ % I i Average WellBelow '' v: ~ 93 Appendix E Writing probes 94 Name ------------------- Date - - - Grade - - - Written Expression probe 1 Write a story that begins with: I opened the door and SCHOOL DISTRICT # 57 C.B.M. NORMING PROJECT 2002-3 TWW _ _ __ wsc 95 Name ------------------- Date - - - Grade ----- Written Expression probe 2 Write a story that begins with: The cat climbed the telephone pole and ... SCHOOL DISTRICT# 57 C.B.M. NORMING PROJECT 2002-3 TWW _ _ __ wsc 96 Name -------------------- Date _______ Grade - - - - probe 3 Written Expression Write a story that begins with: Yesterday, a monkey climbed through the window at school and . .. SCHOOL DISTRICT# 57 C.B.M. NORMING PROJECT 2002-3 TWW ________ wsc 97 Name ________________ Date - - - Grade - - - Written Expression probe 4 Write a story that begins with: I saw strange footprints .. . SCHOOL DISTRICT # 57 C.B.M. NORMING PROJECT 2002-3 TWW - - - - - wsc 98 Name -------------------- Date ------- Grade ----- Written Expression probe 5 Write a story that begins with: I was walking my dog and all of a sudden a pack of wolves came running out and ... SCHOOL DISTRICT # 57 C.B.M. NORMING PROJECT 2002-3 TWW - - - - - - wsc 99 Name ------------------- Grade ----- Written Expression Date - - - probe 6 Write a story that begins with: In the middle of the night I heard some strange sounds. I got out of bed and ... SCHOOL DISTRICT # 57 C.B.M. NORMING PROJECT 2002-3 TWW - - - - - - - wsc