INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon th e quality of the copy sutxnitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bieedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. ProQuest Infonnation and Learning 300 North Zeeb Road, Ann Arbor, Ml 48106-1346 USA 800-521-0600 UMI' WORK HABITS EVALUATION; A CLOSER LOOK by Marion Hofmann B. A , Simon Fraser University, 1980 THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF EDUCATION in COUNSELLING © Marion Hofmann, 1999 THE UNIVERSITY OF NORTHERN BRITISH COLUMBIA March 1999 All rights reserved. This work may not be reproduced in whole or in part, by photocopy or other means, without the permission o f the author. MitkNiai Library of Canada Bibirothèque nationale du Canada Acqutsitions and Bibliographic Senrices Acquisitions et senrices bibliographiques agsWMHngtonStrMi CMm m ON K1A0T44 Canada SSS.niaWaWngton Ottawa ON K1A0N4 Canada Yaurm VdewrWkFie» OuilH AMra/Mrawa The author has granted a non­ exclusive licence allowing the National Libraiy of Canada to reproduce, loan, distribute or sell copies of this thesis in microform, p : ^ or electronic formats. L’auteur a accordé une licence non mcclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la forme de microfiche/film, de reproduction sur papier ou sur format électronique. The author retains ownership of the copyright in this thesis. Neidier die thesis nor substantial extracts from it may be printed or otherwise reproduced without the author’s permission. L’auteur conserve la propriété du droit d’auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation. 0-612-62478-1 CanadS APPROVAL Name: Marion Hofmann, B.A. Degree: Master of Education (Counselling) Thesis Title: Work Habits Evaluation: A Closer Look Examining Committee: Chair: Stan Beeler, Ph D. Chair, English Program University of Northern British Columbia Supervisor: Bryan Hartman, Ph D. Professor, Education Program University of Northern British Columbia Member: Peter MacMillan, Ph.D. Assistant Professor, Education Program University of Northern British Columbia Member: Colleen Haney, Ph D. Assistant Professor, Education Program University of Northern British Columbia External Examiner: BdbnAe Chappell, M.A. District Academic Achievement Administrator School District #57, Prince George, B. C. Date Approved: APR 2 6 1999 Abstract Secondary teachers were asked to indicate their agreement with statements regarding the work habit evaluation mark on the formal report to parents. They were asked three questions. (I) how they define student work habits for the purpose of evaluation, (2) their opinions of the purpose o f the evaluation, and (3) the process they used to complete this evaluation. Results from a survey indicated that there was a wide range of terms in use for the definition, that the purpose is unclear and that the evaluation processes are highly individualized. The conclusion is that using a letter grade as a means of evaluating student work habits is problematic. Recommendations are that the letter grade be dropped and available comments be increased in order to report more effectively on student work habits. TABLE OF CONTENTS Abstract................................................................................................................................. ii Table of Contents..................................................................................................................Hi List of Tables ....................................................................................................................... vi Acknowledgments.................................................................................................................viii Chapter One Introduction................................................................................................. I Rationale, Significance or Need for theStudy.............................................. 2 Statement of the Problem to beInvestigated................................................ 5 Elements to be Investigated......................................................................... 5 Delimitations and Limitations o f the Study................................................ 6 Definition of Term s...................................................................................... 6 Chapter Two. Review of the Literature.............................................................................. 8 Overview o f the Theory and Research Literature....................................... 8 Theory and Research Specific to the Topic....................................................15 Research in Cognate Areas Relevant to the Topic........................................ 19 Summary o f What is Known and Unknown about the T o p ic...................... 26 The Contribution This Study will Make to the Literature............................ 28 Chapter Three. M ethod........................................................................................................... 29 Research D esign.......................................................................................... 29 Specific Procedures..................................................................................... 29 iii Research Population.................................................................................... 36 Pilot Study.................................................................................................. 36 Data Collection............................................................................................ 36 Research Sample......................................................................................... 37 Treatment o f the Data................................................................................... 37 Chapter Four. Results......................................................................................................... 38 Analyses.........................................................................................................38 Elements relating to the Research Questions................................................ 41 Evidence that Supports the Research Questions........................................... 49 Unanticipated Results.................................................................................. 55 Chapter Five. Discussion.................................................................................................. 56 Strengths, Weaknesses and Limitations of the Study ...................................60 Implications for Professional Practice or Decision-making......................... 61 Implications for Future Research...................................................................62 Recommendations for Further Research....................................................... 62 Recommendations for Changes in Professional Practice..............................62 Recommendations for Modifications in Accepted Theoretical Constructs.......................................................63 Recommendations Concerning Changes....................................................... 63 Summary........................................................................................................ 64 References.............................................................................................................................. 65 IV Appendices A Student Report Sample................................................................................... 70 B ERIC search term s.......................................................................................... 71 C Survey........................................................................................................... 72 D Copy of letter that accompanied each survey........................................... E Copy of letter that went to eachschool confederate.................................... 76 75 TABLES Table 1. 2. Work Habit Criteria Policy from Three Secondary Schools................................................... 12 Work Habit Mark Assignment Policy in Four Secondary Schools..................................................... 13 3. Definition of Work Habit Constructs Leading to Survey Item s..................................................... 33 4. Purpose of the Evaluation Constructs Leading to Survey Item s..................................................... 34 5. Evaluation Process Terms Leading to Survey Item s........................................................................ 35 6. Item Means, Standard Deviations and Correlations of the Factors in Work Habits Definitions.......................................39 7. Item Means, Standard Deviations and Correlations of Teachers’ Opinions of the Purpose of Work Habits Evaluation...................................................... 39 8. Item Means, Standard Deviations and Correlations o f Evaluation Processes used by Teachers............................. 40 9 Item Means, Standard Deviations and Correlations o f Summary Statements.......................................................... 41 10. Work Habits Definition Factors....................................................... 42 11. Percentage Agreement Ratings o f the Work Habits Definition Factors.......................................... 43 12. Purposes of the Work Habits Evaluation Mark................................. 44 13. Percentage Agreements Ratings o f the Purposes of the Work Habits Evaluation.............................. 45 VI 14. Processes for Evaluating Students’ Work Habits.......................... 46 15. Percentage Agreements Ratings of the Work Habits Evaluation Processes....................................... 47 16. Summary Statements...........................................................................48 17. Percentage Agreements Ratings of the Summary Statements.......................................................... 49 18. Years of Teaching Experience by Agreement with Modal Groupings..................................... 50 19. Teaching Area by Agreement with Modal Groupings................................................................ 51 20. Number of Formal Measurement or Evaluation Courses Taken during Education by Agreement with Modal Groupings................................................................ 52 2 1. Gender of Teachers by Agreement with Modal Groupings................................................................ 53 22. Contributions from Surveys.............................................................. 53 Ml Acknowledgments Sincere thanks go to the many gifted and creative people who assisted me in the creation of my thesis. Thank you to Dr. Bryan Hartman, for his patience, enthusiasm and unflagging support. Thank you to Dr. Peter MacMillan, for his wise and good-humoured guidance through the statistical analyses. Thank you to Dr. Colleen Haney, for her interest and enthusiasm and for her attention to detail. Thanks to my colleagues Elizabeth, Faye, Chris, Rob, Julia and Carol, for taking time from their busy schedules to answer my constant questions. And thanks to my family for their support and encouragement. I truly could not have done this without the unique contributions of each one o f you. MU CHAPTER ONE Introduction All students who attend a public school in British Columbia (B. C.) receive at least three formal reports each school year (Guidelines for Student Reporting, 1994). In many secondary schools, a quarterly review of a student’s progress is provided during the school year. Most adults are familiar with student reports from personal experience either as students or parents. In British Columbia, the Ministry of Education has published guidelines that teachers and administrators are expected to follow when preparing and presenting reports to parents and guardians of B. C. students (Guidelines for Student Reporting, 1994). Achievement on academic outcomes is reported through letter grades, percentages or both. Student reports also carry attendance information, comments from teachers and a work habit evaluation. Academic letter grades range from A to F, which refer to percentages earned on learning outcome assessments. They are set by order of the provincial government (Ministerial Order 192/ 94). Work habit letter grades are organized on an ordinal basis, usually on a scale of three or five points. These ratings are subjective, and utilize terms such as "Outstanding", "Good", "Satisfactory or "Unsatisfactory". The letters correspond to the terms in use in that district, and may have some variation (Clarification of Student Reporting, 1994). The work habit mark that occurs on numerous reports in British Columbia appears to be the result o f a process that is not clearly defined. The Ministry of Education Guidelines for Student Reporting clearly states that teachers should use written comments about attitudes, work habits and effort. However, probably in the interest of efficiency, many districts have chosen to 2 include a letter grade for work habits. These letter grades are defined on the report card. The information would look similar to the following; • O Outstanding • G Good • S Satisfactory • U Unsatisfactory Not all districts use the same letters, but these are very popular. This terminology seems to have an intrinsic meaning, to be intuitively understandable. It is possible to assume that every teacher has the same notion of what "good" work habits are. In Appendix A (Student Report), close inspection reveals that one student’s work habits were evaluated differently despite the fact that the student’s achievement percentages were uniformly high. No comment was made to clarify what the differences were. It is evident that different teachers may have different standards, definitions and expectations when it comes to evaluating work habits. The Problem Rationale. Significance or Need for the Studv The report on a student’s academic achievement offers feedback on that individual’s progress in that course. Parents, guardians and students are encouraged to discuss the achievement with the relevant teachers for further information. Some letter grades indicate the possibility of failure in that course. Letter grades and percentages are used to determine honour roll placement for the high-achieving students, as well as credits on the Passport to Excellence. The Passport to Excellence is a B. C. government program that rewards the top thirty percent of students at a particular school with post-secondary financial credits. It is available to students from Grade 9 through 12 in each B. C. public secondary school. Work habit evaluations have 3 similar applications. A "lower”, or less than satisfactory, work habit grade may keep a student off the honour roll even if that student has qualified academically. It will also eliminate the student from the Passport to Excellence list for that school. These are not the only effects a report may have. There may be consequences at home, depending on the expectations of the parents and the student. Some financial institutions offer a small monetary reward for placement on the honour roll. A low work habit evaluation, which has eliminated the student from honour roll placement, will also prevent the student from earning the financial reward. Low achievement letter grades may be indicators of special considerations for student program placement or the need for modifications or adaptations. A low work habit letter grade may affect how other educators perceive the student. If he or she is not working hard to succeed, as indicated by the work habit mark, he may be viewed differently than a student who makes a greater perceived effort. On report cards, students are publicly rated as to academic achievement. They are also rated on the dimension of work habits. This information will be as public as a student wishes it to be, but there is no way that he may completely suppress the information. This publicity will have an effect on the student. It becomes a part of the overall view that the student will develop and maintain o f himself as a learner. It is a part of the evolving self-image of a young person in our society. Achievement letter grades are products of known processes, and the steps taken to determine those marks are clear and well defined in the secondary system. In each case, the teacher is expected to make sure that the student is well aware of how well he is doing on tests, assignments, homework or projects. Another question that this study focusses on is how the work habit letter grade is defined and explained to students by teachers. 4 We all remember a time when we opened our own reports with either anticipation or dread at what might be recorded on those pages. These emotional experiences are related to the sense of mystery connected with the marks inscribed on those sheets. Should mystery have a part in the reporting of a student’s achievement and progress? Evaluation that is mysterious or misunderstood cannot have a positive effect on a student’s awareness of what is needed to improve or maintain the relevant achievement. Students must understand and accept the process of assessment and evaluation that guides their progress (Guidelines, 1994; Stipek, 1995). Evaluation o f performance in the public school system has several purposes. It must both inform and instruct. It must inform in the sense of describing the student’s past achievement on learning outcomes and standards, and instruct in the sense of showing where further study is required. The method o f reporting those evaluations must be clear and concise (Guidelines for Student Reporting, 1994), and at the secondary level, must also be efficient. If there is a lack of clarity in the process anywhere, the evaluation loses its impact, its ability to instruct and inform. If the definition o f work habits varies from teacher to teacher, is this report clear and informative? An ordinal scale involves the ranking of objects, persons or traits strictly by order. The intervals between the scale values are not equal. The differences described are relative and frequently subjective. In ordinal scales, the dividing line between categories is also indistinct (Sax, 1989). This is in contrast to the commonly used interval scale, based on equal and regular differences between each of the points. The interval scale is used in measuring achievement. When student work habits are rated using letter grades on an ordinal scale, these limitations apply. Examining these limitations generates several questions. What is the difference between 'Good ' and "Outstanding” or between "Needs Improvement" and "Unsatisfactory"? 5 Does length of teaching experience lead to differences in the way work habits are evaluated? Are the differences the same for a math teacher and an art teacher? Concern about the rating scales is only one part of the question. There also appears to be differences in the way work habits are defined. The Guidelines document separates work habits from effort and attitude, but it does not define what work habits are. When work habits are defined for the purpose of guiding teachers in their evaluations, those definitions offer a wide range of possibilities. Anecdotal information indicates that each teacher has a personal definition of what work habits are and which behaviours are important. Educators at all levels assume that teachers know what work habits are and how to evaluate them. The purpose of this study is to discover how teachers define the concept "work habits”, what they perceive the purpose of the work habit mark to be, how it is assessed and evaluated, and what meaning they believe it has for students and parents. Statement o f the Problem to be Investigated What are work habit evaluations all about? Is it fair to ask teachers to use a method of assessment and evaluation that is "assumed" to be clear and valid? Are work habits clearly defined and described by teachers? What is the purpose of using a letter grade? Is there any other way to convey teachers’ observations and recommendations to students and parents? Elements to be Investigated There are several elements that are to be investigated. These elements are what factors of student behaviours are most important to teachers as they define and assess work habits, what they perceive the purpose of the evaluation is, and how teachers feel about the process. Delimitations and Limitations of the Studv This study will focus exclusively on teachers in B C who are presently working at a public junior secondary school. These are teachers who work with Grades 8, 9 or 10 students. The participants were restricted to keep the focus o f the study manageable and focussed. The participants are voluntary respondents to a survey that was mailed to their school. The schools involved are in eight school districts within B. C. These schools draw from a variety o f student populations, including urban, suburban, semi-rural and rural. These schools demonstrate the diversity of the B. C. population, in that there are students and teachers from numerous races and cultures at each. The teachers represent a wide range o f ages, years o f teaching experience, subject specialties and training. Definition of Terms Work habits presently are whatever it is that a teacher considers when preparing the work habit mark for a report to parents. Because work habits are neither clearly defined, nor fully described in any of the sources the researcher was able to discover, one purpose of the study is to define and describe in greater detail the term "work habits". Work habits mark refers to the letter grade given in conjunction with the achievement letter grade on a formal report. Recently, one school district in B. C. adopted a five-point ordinal scale for secondary teachers to use when evaluating work habits. The descriptive terms are "Outstanding", "Good", "Satisfactory", "Needs Improvement" and "Unsatisfactory". The letter grades which result from these terms and which are used on the reports are 0 , G, S, NT and U. Report is the official report to parents or guardians which is mandated by the British Columbia School Act and which must be prepared on a formal basis at least three times per 7 school year. At many B. C secondary schools, it is more usual to have four reports to parents during the secondary school year Assessment is "...the systematic process of gathering information about students, what they know, are able to do, and are trying to do" (Assessment Handbooks Series, Glossary). Evaluation is "...the process of making judgments and decisions based on the interpretation of evidence of student learning gathered throughout assessment. Evaluation might be done by the teacher or the student independently or in collaboration" (Assessment handbooks Series, Glossary). Teacher is an adult who is educated and certified to teach, and presently employed to do so in a B. C. secondary school. Student is a person between the ages of 12 and 16 years who attends a B. C. junior secondary school. CHAPTER TWO Review o f the Literature Overview of the Theory and Research Literature The assignment of letter grades to work habit evaluations has become standard practice in British Columbia schools. Although it is difficult to ascertain exactly when this practice was introduced, anecdotal reports and personal experiences indicate that it has been ongoing for at least fifty years. The Student Progress Report Order, issued under the authority of the School Act, by the Ministry of Education, Skills & Training, Legislation Branch offers this interpretation of the requirements for all written student progress reports. I. Written student progress reports for students in Kindergarten through Grade 12 must contain... (g) a description of the student’s behaviour, including information on attitudes, work habits and effort. (Student Progress Report Order, E-95, August 21, 1996). Currently, educators in B. C. are more specifically assisted by "Guidelines for Student Reporting for the Kindergarten to Grade 12 Education Plan", published by The Ministry o f Education o f the Province o f British Columbia in 1994. While the document focusses primarily on the process of reporting on academic achievement, there are some references to work habit reporting. The section on Specific Guidelines, Intermediate Reports, (Grades 8 to 10), reads Formal reports for each student in Grades 8 to 10 must; • Provide Ministry-approved letter grades as set out in the Provincial Letter Grades Order to indicate the student’s level o f performance as it relates to the expected learning outcomes for each course or subject and grade 9 • include written comments, where appropriate, that describe, in relation to the expected learning outcomes set out in curriculum a) what the student is able to do b) areas in which the student requires further attention or development c) ways of supporting the student in his or her learning # include written comments to describe student behaviour, including information on attitudes, work habits and effort. The Provincial Letter Grades Order sets out the letters that are associated with academic achievement percentages. Since 1995, these have been A 86-100 B 73-85 C+ 67-72 C 60-66 C- 50-59 [P In Progress. The student is making progress, but it has been determined that additional time is required to meet the expected learning outcomes for the course or subject or grade. F Failed. This grade may only beassigned after an IP has been assigned. W Withdrawal. SO Standing Granted. TS Transfer Standing. (Guidelines, p. 8) No letter grades are specified for work habits. 10 Finally, there is a section in the Formal and Informal Reporting chapter of the Guidelines document, which specifically addresses work habits. Reporting on Student Behaviour The School Act requires that teachers provide parents with information regarding their children’s behaviour, and policy for student reporting in British Columbia requires that in formal reports teachers do this using written comments, including information about attitudes, work habits and effort. Parents are interested in how their children get along with their peers, in their work habits and efforts toward learning, and in their attitudes toward, level o f interest in and motivation toward their studies. Assessing behaviour, effort, motivation and interest and including them in a grading system is problematic. Such traits are difficult to define and assess objectively. As much as possible, teachers should use systematic, dependable methods when assessing student attitudes and behaviour. Often a student’s achievement is affected by behaviour and, as an indirect result, the letter grade will be affected. However, reducing grades as a deterrent is unfair and self-defeating. Behaviours and personal traits are best reported to parents through written comments and in conferences. (Guidelines, p. 22) The Ministry o f Education clearly expects educators to report to parents on the effort, attitudes and work habits of their children. According to the Guidelines for Reporting, the preferred modes for this reporting are written comments and conferences. Using a grading system for work habits, behaviour and effort, while not expressly forbidden, is perceived as "problematic", ‘difficult to define" and "difficult. . .to assess objectively". 11 Report cards in B C carry comments from the student’s teachers that focus on the individual’s behaviour in class, as well as suggestions for ways to improve the academic achievement where necessary. Many also include a letter grade that purports to describe the student’s work habits, although this appears to be contrary to recommendations found in the Guidelines document. The authority for this may be found in another Ministry document, "Clarification of Student Reporting Policy and Answers to Commonly Asked Questions". This document is self-described as a summary of a question and answer session that took place in 1994. The participants are not identified. Question 18 reads "Can the comment on work habits for each subject at the secondary level be accompanied by such symbols as "0", "S", or "N"? May these abbreviations be used without individualized comments?" The response to this question is; Abbreviations of descriptions o f behaviour may be used if descriptions of these abbreviations are set out in a legend that parents can clearly understand. Abbreviations are sometimes inadequate and should be supplemented by more description. The format is a district decision, (p. 3) Districts may choose to use letter grades for work habit reporting. Each school district in the province has the right to decide what descriptions of work habits may be used and how they will be abbreviated. There is a possibility of numerous abbreviations and descriptions. The abbreviations cited above, "G", "S", and "N" are likely to be the most frequently used, as they relate to the apparently clear terms ""Good”, "Satisfacto/y " and "Meeds Improvement ”. Neither the School Act nor the Guidelines document defines work habits. Work habits are one item on a list that also includes the terms; effort, attitude and behaviour. It could be inferred that these are separate aspects of a student’s school life. There is a lack o f clarity on this issue. 12 which leads to a variety of assumptions and misunderstandings between student, parent and teacher. Some school districts have chosen to define the way work habits will be evaluated by their teachers. Given the authority to determine a grading system, and also given that teachers of secondary students may be required to evaluate as many as 200 students at reporting time, boards have opted for a stream-lined approach. In one school district, this issue has been addressed by the development of a district policy directing that behavioural abbreviations be utilized and suggesting that each secondary school develop a set of grading criteria. Some examples of school-based criteria are to be found in Table I. Table I Work Habits Criteria Policv from Three Secondary Schools Categories Responsibility Attitude Effort School# 1 School # 2 class work homework organization catch up after absences attendance punctuality preparedness sets own goals safety assignment completion well organized observes deadlines very punctual maintains an orderly notebook cooperative respectful positive enthusiastic attentive contributes to the class self-motivation active leamer/engaged self-improvement quality o f work productivity is cooperative and respects others School # 3 completes assignments punctually is organized (brings materials, prepares for class) is punctual cooperates with teacher and other students participates in class gets extra help promptly seeks help when needed is an active learner Note. These categories were taken from School # l ' s published suggestions. Published criteria from the other two schools were then sorted according to that set. 13 The published criteria are accompanied by suggestions for consideration when assigning work habit letter grades. They are given in Table 2. Table 2. Work Habit Mark Assignment Policv in Four Secondary Schools Letter Grades O OiUstandiitR G Good School tt I All Missing one School tt 2 School § 3 The student has ver\ good to excellent work habits and has mastered most or all of the items mentioned in the above criteria. In addition, the student must complete at least 95% of the assigmnents given. Good work habits are a goal for all students. School#4 Not in use Does his/her homework and assignments carefullv Works in class when given time to do so. Listens to the lesson carefully and asks questions when he/she doesn't imderstand. Comes to class prepared, with all the materials needed. Puts his/her best etTort into work, assignments, and studv. Shows a positive attitude in the classroom. Catches up work missed on days absenf in his/her own time. S Satistactorv EtTort has been consistent but more attention to details is required. Table 2 is continued on die iiexl page. The teacher believes that the student can improve in some or all of the areas listed above. In addition, the student must complete at least 73% oftlte assignments given. The student meets most of the criteria listed above but there is room for improvement Homework and assignments arc usually completed. Class time ollen used ertectivelv. Usually pays attention to the lesson, or seeks help Regularly comes to class prepared. Consistent etibrt in work, study, and assignment completion. Attitude is usually positive. Work missed on days absent is often completed._______ u NI Needs Improvement Improvement is needed in a number of areas listed above. An NI work habit mark in any term eliminates a student from Passport to Education (grade 9 and 10) Honour roll and recommendations Can be used both as "comment” and/or as encouragement This student would most likely be missing several points in each R-A-E categories (sic). Homework is poorly and/or rarely done. Often olT-task and disrupting in class. Does not pay attention to the concepts being taught Usually comes to class unprepared There is little or no eftbrt put forth in work, study habits. Attitude is usually positive, (sic) Work missed on days absent is not completed U Unsatisfactory If student is lacking in any one of attitude, or eftbrt categories, a "IT should be assigned Work habits are not satistbctory in areas listed above and the student's marks are being seriously aftected. There may also be a signiticant risk of failing the course. In addition, the student has completed less that 70% of the assignments given. Poor attitude, poor behaviour, and poor organization skills are often contributing factors. Work habits are not satisfactory in the areas listed above. An unsatisfactory work habit in any term elimmates a student from Passport to Education (grade 9 and 10, Honour roll and recommendations . It is recommended that students who exhibit unsatist'actory work liabits receive interim reports each term. Not in use. * Where the box is blank, no policy lias been published. The previous tables demonstrate the variety that exists within one district with regard to the definition and evaluation of work habits. Each school has developed its own set of definitions and criteria. There is some overlap, but there are also gaps and differences. Each school has used 15 terms which allow for flexibility in assessment such as "may consider", "most", "some or all o f the areas". Some schools are more directive than others are; some departments within schools are also more directive. Definition, assessment and evaluation o f work habits is ultimately allotted to the autonomy and responsibility of the individual teacher. Theory and Research Specific to the Topic Literature that focuses upon the topic of the definition and evaluation o f work habits is not extensive. A thorough search of ERIC, using a variety of terms, elicited few leads. Appendix B contains a list o f the terms used, and the number o f results under each heading. The Dissertation Abstracts International was equally silent on the topic, as was a close inspection of several textbooks used in teacher education. While reporting on students has been studied thoroughly, the focus of this reporting has been on academic learning. The manner in which a student goes about the business of being a student is frequently described with reference to learning styles, behaviour or physical circumstances. Work habits as a separate dimension of evaluation appear to have been overlooked. W. A. Napthali did one particularly relevant study in England. The subjects of this study were teachers of students aged 13 to 16 years, which is the same student age group o f the teachers in the present study. He tried to discover how teachers define work habits (Napthali, in Woods, 1987). Sixteen teachers identified the criteria each used to describe behaviours that would contribute to student success. They generated fifly-six distinctively worded descriptors. Some of the descriptors were subject-specific, for example, mapping skills in geography, and others were more general in nature, such as behaviour and work output. These were then grouped into twenty-two derived constructs. These constructs were classified into three psychological categories; cognitive, affective and motivational. In this study, there were more constructs in the 16 affective domain (n=10) and in the motivational domain (n=8) than in the cognitive (n=4). Napthali concluded that teachers were likely to differentiate between pupils on the basis of six derived constructs. These are, in no particular order of importance: a. The involvement of the pupil in the learning situation. b. The ability the pupil has in the subject. c. The overall ability of the pupil. d. The behaviour of the pupil. e. The quality and tidiness o f work presented. f. The interest displayed by the pupil in the subject, (p. 23) Napthali also concluded that teachers do vary considerably in what they look for and that what they see and what they value is crucial as well as idiosyncratic. Finally, in a statement that has direct bearing on the concept of subjective evaluation, he finds ' each teacher will mix the ingredients (ability and application) in a personal way and will differ over the weights they give to each” (Napthali, 1975, p. 24). Fuchs, Fuchs and Phillips, (1994), investigated the importance that teachers placed on student work habits, how those work habits are connected to achievement, and what effect student work habits might have on teachers’ planning. They revisited the notion o f the selffulfilling prophecy and extended it to include work habit behaviour. They were particularly concerned with the possibility of differential treatment and expectations as inclusion o f special needs and learning disabled students increased in mainstream classrooms. The authors had 121 generalist elementary school teachers complete a scale that included the following items: e students listen and comply with teacher instructions and directions 17 • students have good work habits (italics added) and make efficient use o f class time • students have independent study skills • students attend consistently to assigned tasks • students cope with failure in an appropriate manner and don’t give up • students assume responsibility for having materials • students produce work of acceptable quality given their skilllevels • students follow classroom rules • students work without disrupting the rest o f the class. (Fuchs et al., 1994, p.335) This appears to be a thorough and well-detailed list of student behaviours that should lead to academic success if followed. It is interesting to note that the authors appear to believe that work habits are activities distinct from the other behaviours that they describe. There is a question, then, as to what these work habits might be. Descriptors such as "listen to and comply with directions" or "make efficient use of class time" define specific, observable, and measurable behaviours. The authors have assumed that teachers know what work habits are, and how they are distinct from the described behaviours in the study. They did find conclusively that a teacher’s high standards and expectations for student "work habits" led to higher student achievement. • The authors of Guidelines for Student Reporting have made an assumption similar to the one mentioned above. In this document, teachers are asked to report to parents with information regarding their children’s behaviour. .. Including information about attitudes, work habits and effort (p. 22). There are no precise definitions for educators to follow, but again, work habits are 18 seen as separate from attitudes and effort. In this instance also, there is the assumption that teachers will both be able to identify and agree upon what work habits are. Most of the assessment of student learning is focused on academic objectives. How are these assessments and evaluations affected by student deportment? Pedulla, Airasian and Madaus, (1980) investigated how teacher ratings and predictions of achievement on standardized tests were affected by student behaviour. They supplied 170 elementary school teachers with twelve behaviour constructs and asked them to rate their students (n_= 2,617) on those behaviours and then predict scores for IQ, English and mathematics standardized tests. The teachers’ assessments were thus limited to the criteria that the authors had selected. The authors grouped these constructs into two factors. Factor 1 is described as relating to classroom behaviours and is similar to motivational behaviours mentioned in other discussion. Factor 2 is related to social or personal student behaviours and is similar to affective groupings found in previous studies. They found that teacher judgment of students’ IQ, English, and mathematics performance are confounded with their judgments of other academically related behaviours, such as attention span and persistence (p.307). Teachers indicated they believe that a student who seems to be paying attention or who works hard has innate ability to succeed academically. On the other hand, students who showed strength in social, or affective, constructs were not deemed as able as their more attentive peers (Pedulla et al., 1994). This finding could have widespread implications. For example, very social students who are more interested in interacting with their peers may be underrated on an intellectual scale. A student who appears diligent and hard working may be over-rated. Some students are less able to pay attention for sustained times or 19 have less interest in completing assignments. They may be perceived as less able to succeed and diverted into programs that do not offer them the challenges of which they are capable. Allai (1988) studied 45 elementary teachers in Switzerland to determine how they went about determining grades and making promotion decisions about their students. She discovered that there are qualitative and quantitative components to each mark given to students by teachers. No work habits marks are given as a separate evaluation, but the observations are an intrinsic component of the overall evaluation done. Adjustments in an achievement rating were frequently made if the teacher felt that the average, arithmetically derived, mark did not reflect the "true capacity" of the student. These adjustments were the "result of a more qualitative type of synthesis based on a variety of elements; assessment of effort or of perseverance ...unrecorded and intuitive observations of the child’s attitudes and work habits, global judgments, and so on." (Allai, 1988, p. 47). Teachers and supervisors in that system considered this practice reasonable, valid and reliable. The previous studies did not explicitly state that factors such as work habits, effort or behaviour would affect a student’s achievement grade, but it is possible that this "fudge factor" is a component of any evaluation that a teacher does on a student. Research in Cognate Areas Relevant to the Topic One term that recurs when defining work habits is "effort" (Fuchs et al., 1994; Guidelines, 1994; Napthali, 1987; Pedulla, 1994, Tables 1 and 2 above). How is effort defined? Effort is variously described as leading to fiill use of ability (Nicholls, 1976), perseverance and active engagement during instruction (Mac Iver et al., 1991), diligence (Jagacinski and Nicholls, 1990), how hard a person works (Ames and Archer, 1988) and persistence (Gayer, 1994; Schunk, 1982). There is general agreement that effort is under volitional control (Covington, 20 1980; Covington and Omelich, 1979; Jagacinski and Nicholls, 1990; Mac Iver et al., 1991; Nicholls, 1976; Schunk, 1982, Weiner, 1979). Effort is what a teacher perceives as effort, and is measured by observation. Teachers expect students to make an effort to learn enough at least to pass the course (Covington, 1980; Covington and Omelich, 1979; Nicholls, 1976; Pedulla, 1980; Schunk, 1982) and so do parents (Ames and Archer, 1987; Covington and Omelich, 1979; Mac Iver, 1991). Teachers are more likely to reward effort with praise and to accept the student’s level o f achievement if it appears that the student has been making an effort to succeed (Covington, 1980; Covington and Omelich, 1979; Darakjian and Michael, 1982; Nicholls, 1976). Teachers assess effort by comparing achievement to teacher perceptions of ability (Anderman, 1992; Benham, 1995; Hoge and Butcher, 1984; Pedulla et al., 1994; Swann and Snyder, 1980). Ability may be perceived as either fixed (Covington, 1980; Gayer, 1994; Schunk, 1982) or variable (Ames and Archer, 1988; Swann and Snyder, 1980). This definition varies within each individual according to past experience and relevant classroom orientation. Some teachers believe that an individual has intrinsic ability and will do well in the subject regardless o f instructional techniques or classroom orientation, while other teachers believe that specific instruction and support will enhance a student’s learning no matter what her (extrinsic) ability. Swann and Snyder (1980) studied male undergraduates in an effort to understand how teachers adapt techniques to meet the perceived needs o f their pupils. The "teachers" were led to believe that some o f the students were very capable of learning and others would need comparably more instruction. They found that the teachers maintained their beliefs about the students and somehow communicated these beliefs to the students, even in the face of contradictory evidence. Their findings confirm numerous others that are now well publicized in the field of education. 21 However, it seems important to once again point out the need to be aware o f the effect of labeling students. Students are aware of the need to look like they are making an effort (Covington, 1979; Jagacinski and Nicholls, 1990). Generally, a student will try harder to succeed if he enjoys or is interested in the subject (Maclver, 1991). Effort is also affected by a student’s perceptions of her own ability (Mac Iver, 1991; Nicholls, 1976). If a student is confident of success, then she will make a greater effort to overcome challenging tasks (Nicholls, 1976) but if the tasks are too easy, success will not lead to pride and the student’s confidence may not be reaffirmed (Mac Iver, 1991). Covington and Omelich (1979, 1985) and Nicholls (1976) offer some other points to consider when assessing effort. Although their studies were done with university students who likely had very little experience with failure and lots of confidence in their ability to learn, their conclusions are interesting to bear in mind when younger students with a greater range of academic success are the focus. What follows is a summary o f their conclusions. Covington and Omelich ( 1979, 1985) and Nicholls (1976) claim that students would rather be thought of as lazy than stupid. In our world people with high ability are praised more frequently and appear to be more valued. Even if a person has shown little in the way of success, it is better to be perceived as smart but lazy than not smart but hardworking. It is shameful not to have high ability. Therefore, a student will do everything possible to maintain a perception he would have done really well in that subject or that test, if he had wanted to. Failing the class or the test is due to lack of trying, not caring or wanting to pass, not having time to study for various reasons, being bored by the subject or claiming the teacher doesn’t like him. Students are motivated by a desire to "save face" or to protect a self-image. They are also motivated to avoid 22 punishment. For a student, failing is punishment. The trick in secondary school is to do just enough to satisfy a teacher, and to have plenty o f good excuses available to moderate any punishment. Unfortunately, these tricks may backfire. Too many failure experiences, for whatever reason, will have the effect of reducing the student’s ability to succeed. This could be due to lack of knowledge and practice rather than actual ability. What started as a failure-avoiding technique becomes confirmation that failure is inevitable due to lack of ability. The student becomes resigned to failing and continues to reduce his effort, as failing while trying hard would be the final confirmation that he is "dumb". These factors are diametrically opposite to teacher expectations. Teachers want to see a student work hard, even if she is unsuccessful. They dislike laziness, and are especially provoked by the laziness of an apparently capable student. Teachers want to believe that students are as eager to succeed as they are to have them succeed. They admire students who work hard no matter what their achievement. They also want to know if a student has lower ability so that alternate programming or learning assistance can be offered. For teachers, having low ability is not a shameful condition, but one that must be addressed through different means. There is an apparent mismatch between student and teacher perspectives on how to succeed in school. This mismatch may result in errors in estimation of ability or effort on the part o f the teacher. Hoge and Butcher (1983) found that teachers had a tendency to over-estimate the ability o f pleasant, cooperative students and to under-estimate the ability of less cooperative ones. This would be demonstrated on a report by a low work-habit mark or a comment about lack of effort, or both. 23 Educators and parents are concerned with issues around the effort and motivation o f their students and children. Numerous studies have been done which focused exclusively on either the students or the teachers. Studies of teachers of elementary students find that work habits are important and will affect the achievement letter grade (Allai, 1988; Anderman, 1992; Fuchs et al., 1994; Schunk, 1982). Studies of teachers of secondary students find that the work habits are seen as relevant to success and affect the teacher's expectations, but may not affect the letter grade for achievement (Napthali, 1987). Students in both elementary and secondary settings display attempts to confound and mislead their teachers, and feelings of cynicism and dismissal towards the teachers’ perceptions (Lapadat, 1997). Evaluation o f work habits is essentially a subset o f the spectrum o f evaluations that each teacher performs. Skill in evaluation in any area will naturally benefit that process in another. It is imperative to define what is important about the process of evaluation in order to achieve fair and objective outcomes. The issue of what teachers see and what they value is critical to understanding how work habits are evaluated. The Guidelines document states "(a)ssessing behaviour, effort, motivation and interest and including them in a grading system is problematic. Such traits are difficult to define and assess objectively. As much as possible, teachers should use systematic, dependable methods when assessing student attitudes and behaviour” (p. 22). Assessment and evaluation of student progress and achievement are integral aspects of education. Given this importance, one would expect a great deal o f time and energy would be spent on training teachers in these processes and that there would be a great deal o f literature devoted to the topic. A cursory examination of the shelves of one university library's education 24 collection show numerous shelves devoted to texts on either general or particular dimensions of education, but only two sections contained volumes devoted to assessment and / or evaluation. An ERIC search showed similar proportions of other educational topics to assessment and evaluation within the literature. A further search through indices of several assessment and evaluation texts revealed no discussion o f the topics o f work habits or effort and only occasional references to general attitudes or behavior. Three universities in British Columbia are presently involved in the education of teachers. At the moment, two require some formal assessment and evaluation instruction, either as a full course or as part of a course. Instruction in defining and evaluating work habits is included in these courses on an informal basis (Simon Fraser University and University o f British Columbia instructors, personal communication). Generally, there seems to be few hours of study in work habit assessment and evaluation taken by student teachers during their university education. Several authors have expressed concern about this perceived gap. Although they are not speaking about the Canadian system and circumstances, it is likely that there are numerous similarities. Richard J. Stiggins, (Stiggins, 1985; 1991; 1995) speaks eloquently on the need for improved assessment and evaluation literacy on the part of educators. He advises that educators need more education in assessment methods, and is concerned that there is no clear definition of academic success at the moment. The consequence o f this lack o f clarity is the confusion that abounds around how to assess and evaluate learning, achievement and success. Another advocate for increased education in assessment and evaluation methods is J. R. Hills (1991). He contends that teachers and administrators apparently do not know or do not attend to what are sound and proper assessment practices. Hills comments that grades have been 25 used for disciplinary purposes, something which is a by-product of using letter grades for work habits on reports. He also suggests increased instruction in assessment and time to develop improved instruments. There is considerable documentation that teachers receive little formal preparation for the process of assessment and evaluation during preparation or while working (Allai, 1994; Plake and Impara, 1997, in Phye, 1997; Stiggins, 1991). Allai interviewed forty-five teachers in Geneva. She found that teachers were not provided with a systematic approach towards understanding or utilising evaluation methods or theory either in their undergraduate education or from their supervisors. She recommended increased instructional time in assessment and evaluation practices. According to Hills (1991), approximately 20% of teachers have taken measurement and evaluation courses while at university, and those were taken at least 10 years previously. More have participated in workshops or other professional development opportunities, but these tend to be more informal and based on sharing personal experiences and opinions (Plake and Impara, in Phye, 1997). Plake and Impara conclude that teachers are ill equipped to successfully undertake one o f the most prevalent activities of their instructional program, student assessment (p. 67). Some authors argue that teachers do know how to evaluate the achievement of their students (Allai, 1988; Baker, Mednick, and Hocevar, 1991; Hoge and Coladarci, 1989; Pedulla et al, 1980). These studies and others, demonstrate that teachers can be extremely sensitive both in assessing ability and predicting achievement. It is felt that, although many may not have formally studied assessment and evaluation practices, experience, intuition and informal instruction result in teachers practicing reliable and valid evaluation techniques. 26 We are left, then, with something o f a dilemma. Do teachers assess and evaluate work habits consistently and reliably or not? Common practice prompts a positive assumption that is supported by tradition and long-standing practice, but the current trend in favour o f increased accountability seeking may require more rigour. It may no longer be acceptable to assume that this is a valid process. Summarv of What is Known and Unknown about the Topic There are some basic tenets that guide good evaluation. These are: 1. The grading system should be clear and understandable. 2. The grading system should be communicated to all stakeholders. 3. The grading system should be fair to all students, regardless of gender,race, class or socioeconomic status. 4. The grading system should support, enhance and inform the instructional process. (Holmes, 1993) Government documents indicate evaluation o f work habits is required without defining what is to be evaluated. Each teacher is left to decide the purpose of the work habit mark. Achievement reporting, in contrast, has a clear and universally understood main purpose - to report the achievement on performance or product-based curricular learning outcomes within a specific time span, as determined by the number o f terms in each school year. Each term a report on academic progress and achievement, as well as work habits, is issued. If the purpose of this evaluation is to simply report on past behaviour, an instructor may use such objective criteria as frequency of homework assignments completed, preparedness for class, punctuality, attendance, or meeting o f deadlines. 27 Anecdotal comments from various professionals indicate that on occasion a particular dimension of the student’s behaviour will dominate the evaluation. If a student has been particularly annoying, the work habit mark could reflect this, with no direct measure being used. A teacher may use the mark to indicate that a student is doing well academically although, homework is rarely done. Perhaps the capable student is talking too much to classmates, or is cheeky, or arrives late to class. This student could then receive a less than satisfactory rating for work habits. In contrast, there is the example of a student who is perceived as average or less than average in ability, but who fulfills all the teacher’s behavioral expectations. This student may be awarded a higher work habit letter grade than the previous student. To the adults who read this report, it seems clear that the student has been working well and should be proud. To the student, the inference is that the teacher believes the student has low ability and is only capable o f very low quality work (Ames and Archer, 1988; Covington and Omelich, 1979; Jagacinski and Nicholls, 1990). Certainly, it is unlikely that the purpose o f the teacher was to say anything derogatory or injurious to the student, yet this could be the result. What is unknown about the topic of using letter grades for work habit evaluation is how these habits are defined by junior secondary teachers, how they are measured and how they are then evaluated. We do not know if there is consistency between teachers at the same school, let alone across districts or the province. It is also unclear if this is a necessary grade to assign. It may be that, given the range o f possibilities that exist, this assessment may be more effectively conveyed in another format. Ultimately, evaluation must assist a student in the learning process. 28 The Contribution This Study Will Make to the Literature At the moment, it seems that without a clear set o f criteria, the work habit letter grade as a mark has dubious value. Professional educators feel that they know what a work habit mark means, but there is no research which supports this feeling. Teachers have the ability to make meaningful judgments about their students and recommendations about their behaviour. While teachers may feel they understand the process of assessment and evaluation, they may not be applying this expertise to the work habit dimension with the same rigour that they are applying it to achievement. After evaluation, it is important to communicate results and planning decisions effectively and efficiently. At the moment, this does not appear to be the case with work habit evaluation. This study will offer some information on how teachers define work habits, what they perceive the purpose of evaluating work habits to be, and how that determines the way they are evaluated. It will show what teachers value in student behaviour. Some understanding may emerge as to the impact this procedure has on students, and whether there is a better way to communicate praise and concern. 29 CHAPTER THREE Method Research Design The purpose of this study was to discover (a) how secondary teachers define and evaluate student work habits, (b) what they perceive the purpose of this evaluation process to be, and (c) if there is similarity among the work habit definitions and the evaluation processes used. The design for this study followed a description in Miles and Huberman (1994, p.41). This design begins with a qualitative exploration o f the general topics to be studied. This field data is analysed, and then a survey is developed. The data from the survey is then subjected to statistical examinations. The findings from the quantitative component may then be used in further qualitative exploration. This study may also be described as a combination o f two components. These components are analytical (Mauch and Birch, 1993, p. 114) and opinion polling (Mauch and Birch (1993, p. 118). Specific Procedures It was necessary to begin by determining which terms, or constructs, teachers use for each of the constructs (Miles and Huberman, 1994; Napthali, 1987). Six teachers were interviewed in order to elicit current terminology and expressions. They were selected according to the level of interest expressed when the topic was introduced, and readiness to be interviewed. These six were all employed at secondary schools and have three or more years o f teaching experience. Their teaching specialties include Humanities, Music, Art, Physical Education (P. E ), Learning Assistance and Science. The interviews were guided by eight specific questions, although more were asked depending on the interviewer’s perceived need for clarification. Each interview took approximately one hour. The questions were 30 1. How do you define work habits? 2. What criteria do you use? 3. What is the purpose of this evaluation? 4. Do you believe that work habit marks have an effect on student behaviour? 5. Have you ever discussed with your students what work habits are and how you would evaluate them? 6. Suppose a student gets a “C-/G” (on his or her report). How do you think he or she feels? 7. Were you ever taught how to define and/or evaluate work habits? 8. Are you satisfied with the system we are using now? Four teachers were interviewed individually; two were interviewed together due to time constraints. They had been contacted prior to the interview, so were able to think about the subject ahead of time. The interviews were audiotaped and then transcribed. The elicited constructs were then sorted according to the major themes o f interest. These themes were: 1. definition of work habits, 2. purpose of work habit evaluation, and 3. processes used to evaluate. Fifty-five terms were elicited from the six teachers in the ‘definition of work habits” category, thirty-four terms in the ‘‘purpose of work habit evaluation” category and twenty-terms in the “purpose of the evaluation of work habits” category. These constructs were sorted and “chunked” (Miles and Huberman, 1994). After the constructs had been chunked according to semantic content similarity, constructs from the literature were compared to the derived constructs (Napthali, 1997) in each category. Statements were developed that were succinct. 31 discrete and concise. A conscious effort was made to keep the number of derived constructs to a minimum in order to create a survey that would be attractive to busy teachers (Palys, 1992). Surveys allow for quantification of the data, elicit information from a greater number and wider range of respondents, and increase generalizability of the results and conclusions. Surveys also allow responses from a larger number o f respondents in the least amount of time (Palys, 1992). Survey responses are less likely to be influenced by a desire to please or impress, as there is no interaction between subject, or respondent, and researcher. The specificity of the questions focusses the respondents’ attention on the topic. Surveys allow one researcher to work efficiently with a great deal of data (Palys, 1992; Sax, 1989). The limitations of a survey are that ambiguities or misunderstandings are difficult to clarify, the anticipated response rate is low and there is a potential for volunteer bias (see Palys, p416). As the surveys were going to a well-educated population, literacy was not a concern. It was important, however, to ensure that questions were worded clearly, showed no bias and were succinct enough that the responses would be unambiguous. Every effort was made to develop directions that were easy to follow. Space was made available for respondents to state their opinions and add to the rating scale response information. ^This allowed for additional or alternate responses to be considered after the survey has been completed (Miles and Huberman, 1994, p. 41). Allowing individual responses improved the communication between respondent and researcher, and gave the respondent increased impetus to return the surveys (Palys, 1992). Fuchs et al. (1994) developed a Likert-type scale that allowed teachers to rate the importance of certain work habits for student academic success. Their scale does not examine 32 how well individual students are developing these habits. Other researchers (Napthali, 1987; Pedulla et al., 1980) also studied what habits were considered to lead to success, but did not include individual teacher ratings on student work habits. The instrument developed for use in this study includes components from both the literature and interview data. Table 3 illustrates the development of survey items from the original teacher constructs for work habit definition. In some cases, the survey item is taken directly from the teacher constructs, as it seemed the clearest statement about that construct. In other cases, the item is summative. The middle column shows links to the three studies cited. Constructs from Napthali (1987) are indicated by (N), constructs from Pedulla (1980) are indicated by (P) and constructs from Fuchs et al. (1994) are indicated by (F). 33 Table 3. Definition o f Work Habit Constructs Leading to Survey Items Constnicu trom leach# interviews Handing in junk Orderly presentation o f work Legible Neatness Neat presentation o f work Dating work Quantity o f class work completed Assignment completion rate Completes all the parts o f each task Homework has been completed Finish all work L’sing time efltciently and elTectively Time on task Uses time elTectively Stays on task Speed o f production (Wanders around room) Organization o f book and binder Keeping track o f papers Organizational skills Learning to prioritize Brings materials to class Having a writing "stick " Participation in class discussions Pays attention Focusing Arriving to school on time Arriving to class on time Talking at the wrong time ■too much Treating someone (badly) Consideration and awareness o f other people's needs Working with group Learning manners Not physically hurting anybody Making an eflbrt to achieve the goals 1 have for them Solid elTurt Try to do the things I teach Trying to do the work 1 expect Degree o f eflbrt Trying to improve If student needs a push Attitude Hard work Sportsmanship Showing independence Seeking help when needed Becoming self-directed Achieving within capabilities Trying to do his or her best (Not) doing the best work possible (Not) doing what I want Doing what I ask (Quality o f class work Working too quickly to produce good work Linked to achievement Does more than required Corresmnding constructs ftom literature Presentation o f work (N) Survey item 1. Student's work is easy to read. Neatness in schoolwork (P) 2. Student's work is neat. Students attend consistently to assigned tasks (F) 3. Student's work is completed on time. Pupil involvement, class participation (N) 4. Student uses class time elTectively. Students assume responsibility for having materials (F) S. Student has necessary materials available. Class participation (N) Speech/use olTanguage (P) Concentration, interest (N) .Attention span (P) Attendance (N) (P) Students work without disrupting the rest o f the class: Ibllow classroom rules (F) Likeability. behaviour, aggressiveness, maturity (N) 6. Student participates in class discussions Students have good work habits and make eflicient use o f class time (F) Persistence in school work (P) 7. Student folloyvs classroom rules. X. Student cooperates with peers. Manners; behaviour in school : getting along with other children (P) Perceived eflbrt Pupil involvement (N) Students cope with failure in an appropriate manner and do n 't give up 9. Student makes an eflbrt to improve. (F) Keenness to get on (P) Pupil involvement, reliability (N) 10. Student has a positive attitude. Students have independent study skills (F) Working with limited supervision (P) Pupil involvement. natural ability (N) Students produce work o f acceptable quality given their skill levels: listen and comply with teacher directions (F) Perlbrmance in subjecL perception o f subject. (N) 11. Student works to the best o f his/her ability. 12. Student succeeds in class. 34 The next section o f the survey focused on what teachers believe is the purpose o f the evaluation. Teachers are required to inform parents about the behaviour, attitudes, work habits and effort of the students (Guidelines for Student Reporting, 1994). Teachers are further directed by their districts to perform this process, but a more specific purpose was not included in any document the researcher was able to locate. Teachers apparently develop their own rationale for this required activity. Table 4. Purpose of the Evaluation Constructs Leading to Survev Items Constructs elicited from teacher interviews I use it as a warning that the student has been displaying unacceptable behaviour To wake a student up If student needs a push To tell the student there is room for improvement To give them a message to do more To get the idea across that they aren't performing as well as they should To help them perform Positive boost Tell how they measure up to my standards Give a message to keep on try ing To reward hard work Appeal to their pride Congratulations (for performing at the best level for a student in that class) Let the student know if he or she has achieved a satisfactoiy amount of production Student is exhibiting habits of a good worker Student is showing attitudes of a good worker To set them up for success in later life To connect behaviour in school to the world of work Message to parents Parents redct strongly sometimes Parents are concerned about their child's behaviour I was told to do it I was told it was part of my job Administrators have told me to do it Survev item 15. describe past behaviour 16. alert the student to concern about his/her progress 17. encourage improvement 18. encourage continuation of satisfactory student behaviour 19. praise acceptable behaviour 20. demonstrate the link between school and workplace behaviour 21. communicate with parents 22. follow school district requirements 23. follow B. C. Ministry of Education requirements 35 A third grouping was made from descriptions o f how these work habits are evaluated for the purpose for reporting. Table 5. Evaluation Process Terms Leading to Survev Items Constnicts elicited from teacher interviews I record task completion rate I keep some very objective records I use a visual image of the student I call up a mental image of the student in my class 1 think of my impressions of tlie student over time It is a totally subjective evaluation It is kind of a niceness thing Includes how much 1 like tliis kid It is linked to the personalitv o f the student Includes the way my teaching stvie and management style interact with the kid's learning and behaviour style I take an objective measure of the percentage o f tasks completed The mark is always linked to acluevement Even if a student is doing liis or her best, a low academic achievement will lead to a satisfactory work habit letter grade, never any higher 1 take a holistic view of the student It is linked to my opinion if that student can do better The mark is an individual thing I take other variables into consideration 1 considered he was trying to stay on a team I knew the parents would overreact to a low grade I knew the parents would give me a bad time if I gave him a bad (work habit) grade I discuss with the kids what 1am looking for. but there is overlap with academic achievement Not discussed as well as I should Never described the work habit criteria Althouglt-I didn't do it. 1 can see that it would be necessary to express criteria at the beginning of a course Survey item 26. rely on recorded data 27. rely on my memory 28. evaluate subjectively 29. evaluate objectively 30. combine subjective and objective criteria 31. consider the smdent holistically 32. using criteria that have been described to the students 33. assume students know what criteria are used Note. The gaps in the numbering occur as two spaces were left blank at the end of each section to accommodate write-in suggestions. 36 The final five items in the Summary section were intended to offer some insight as to the confidence teachers have in their methods o f assessment and evaluation and whether they believe the process is useful. The questions were set up in a Likert-type scale that rated each item in seven gradations 4 from such polarities as “Never to Always”, “Least Important to Most Important” or “Strongly Disagree to Strongly Agree”. A seven-point scale was used to increase the range o f possibilities ' and to reduce any tendency to cluster around the midpoint (Sax, 1989). Every effort was made • to develop directions that were easy to follow. Research Population The population studied was secondary teachers in British Columbia. A sample of the population was selected by means o f a random purposeful strategy (Miles and Huberman, 1994, p. 28). Another aspect of the selection was convenience. The study involved only those sites that included a contact person known to the researcher. Pilot Studv A pilot study was done in a junior secondary school. Seven teachers completed the draft version of the survey. Following their suggestions and comments, modifications were made. A new survey form was shown to three of these teachers and no further changes were suggested. A copy o f the final survey is found in Appendix C. Data Collection Two limitations of using a survey were considered. In that most teachers are required to evaluate work habits for the purpose of ascertaining a letter grade for reports, it seemed likely that volunteer bias would elicit more volunteers who had strong feelings about the process, either positively or negatively. The other concern was the low return rate for surveys (Palys, 1992; Sax, 37 1989). It was decided to address these concerns by locating a confederate at each school. This confederate was asked to locate possible respondents. The researcher and the confederates discussed the most effective means o f improving the return rate. The variables deemed relevant to the confederates were (a) the likelihood of the subject completing the survey responsibly and (b) looking for subjects with a range o f experience and subject specialties. Each confederate was asked how many surveys he or she would be willing to distribute. The number ranged from five to ten. This enabled the researcher to bundle surveys and have them bundled in return mail, keeping the mailing cost down. The surveys were sent in large envelopes, accompanied by stamped, pre-addressed envelopes. With each confederate taking responsibility for distributing and collecting a few surveys, the process became more personal and more manageable at each site. An envelope accompanied each individual survey. A copy o f the letter that also accompanied each survey is found in Appendix D. A copy o f the letter sent to each confederate is found in Appendix E. The large return envelopes were addressed to an assistant, who removed the completed surveys and discarded the return envelopes. This ensured that no survey could be traced to its school of origin, so respondent confidentiality was secure. Research Sample The surveys were distributed to twelve schools in eight school districts in B. C These schools range from large urban schools to small rural schools. The student populations are multi­ cultural, and include students ranging in age from twelve to eighteen years. A total o f one hundred-fourteen surveys were distributed. Seventy-eight were returned—a 68 % return rate. Treatment of the Data All o f the returned surveys were numbered for later ease o f identification. The data were entered on a Microsoft Excel spreadsheet. 38 CHAPTER FOUR Results One hundred fourteen surveys were distributed. Seventy-eight surveys were returned, a 68% rate of return. All were usable. Fourteen respondents added one or two personal terms in the category o f definition factors, two included their own criteria for the purpose of the evaluation and five suggested additional processes. Forty included further comments about the process either overall or specific aspects. These comments will be discussed following the presentation of the numerical data. Analvses The first statistical treatment was an item analysis. This was done to determine if any items were ambiguous or nondiscriminating (Sax, 1989). Miskeying is not an issue as there are no correct or incorrect responses. The discrimination between items relates to agreement with the other respondents rather than agreement with a key. The data were analyzed in two steps. One step was to do a within-groups item analysis. The mean and the variance were calculated on that dimension. Next, an item analysis was done on the entire set o f questions. The numbers have been allocated to match the frequency tables that follow. The survey was numbered from “ 1” to “40”, with each section including two blank numbered places for respondents to add their own contributions. The following tables show the mean score and variance for each item, as well as a correlation for each item within the scale and one for the whole survey. 39 Table 6. Item Means. Standard Deviations and Correlations o f the Factors in Work Habits Definitions Item 1. Student's work is easv to read 2. Student's work is neat. 3. Student’s work is completed on time. 4. Student uses class time effectively. S. Student has necessary materials ayailable. 6. Student participates in class discussions. 7. Student follows classroom rules. 8. Student cooperates with peers. 9. Student makes an effort to improve. 10. Student has a positive attitude. II. Student works to the best of his/her ability. 12. Student succeeds in class. Item mean Standard deviation Item-Scale correlation 4.65 4.74 6.22 6.54 6.31 5.34 6.09 5.97 6.17 5.97 6.45 4.87 1.83 1.60 1.02 0.89 1.09 1.42 1.32 1.28 .57 .63 .19 .53 .52 .49 .62 .72 .57 .70 .49 .36 I.IO 1.28 1.08 1.80 Wholescale correlation .52 .59 .21 .43 .40 .41 .54 .67 .54 .60 .47 .20 In the item-scale correlations in Table 6, there were no small effects {d = .2 tp»=. 10). Medium effects {d= .5 i^=.24) were found in one item, and all others showed large effects {ct= .8 rp*=.37) (Cohen, in Kirk, 1996, p. 751). In the whole scale item correlation, two items (3 and 12) showed medium effects, while the other ten showed large effects. Item 3 showed the lowest correlation but a high frequency of agreement, so it was retained. Table 7. Item Means. Standard Deviations and Correlations o f Teachers’ Opinions of the Purpose of Work Habits Evaluation Item Item mean Standard deviation Item-Scale correlation 1. to describe past behaviour 2. to alert the student to concerns about progress 3. to encourage improvement 4. to encourage continuation of satist'actorv' behaviour 5. to praise acceptable behaviour 6. to demonstrate the link between work and school 7. to conununicate with parents 8. to follow school district requirements 9. to follow B. C. Ministry of Education requirements 4.53 4.73 6.16 6.49 6.27 5.25 5.99 5.84 6.04 1.94 1.55 1.07 0.96 1.02 1.50 1.40 1.43 1.19 .63 .66 .30 .53 .66 51 .66 .72 .57 Whole scale correlation .55 .60 .24 .43 .44 .39 .58 .69 .56 40 In Table 7, the correlations between the statements within the scale and among the entire set showed medium to large effects (Kirk, 1996). The mean scores for items I, 2, 6 and 8 place these items within the middle range of ratings. Item 7 is included in the range of highly rated items. No items had means in the low range. Item 3 showed very low correlation values for both within-scale and whole-scale correlations, although it showed high frequency o f agreement. Table 8. Item Means. Standard Deviations and Correlations of Evaluation Processes used bv Secondary Teachers. Item Item Mean Standard deviation Item-Scale correlation 1. I rely on recorded data. 2. I rely on my memory. 3. I evaluate subjectively. 4. I evaluate objectively. S. 1evaluate using a combination of subjective and objective criteria. 6 . 1 consider the student holistically 7 . 1 use criteria which 1 ha\ e previously described to the students. 8. 1 use criteria that 1assume is understood bv the students. 5.30 4.66 5.03 5.30 5.57 1.80 1.86 1.64 1.41 1.70 .36 .45 .63 .25 .68 Whole scale correlation .43 -.03 .18 .37 .38 5.65 5.65 1.31 1.59 .51 .47 .30 50 4.19 2.27 .40 .03 In Table 8, two items show extremely small effects for whole scale correlations, although they were within medium to large effect range for the within-scale correlation. These were (2) I rely on my memory, and (8) I use criteria that I assume is understood by the students. Item 3 dropped from a large within-scale correlation to a small whole scale correlation. The mean for all items places these factors within the middle range o f ratings. 41 Table 9 Item Means. Standard Deviations and Correlations o f Summary Statements. hem hem Mean Standard deviation hem-Scale correlation I. Work habit letter grades help students with the task of learning good work habits. 2. Work habit letter grades help teachers with the task of teaching good work habits. 3. Work habit letter grades help parents with the task of teaching good work habits. 4. The method 1 use to evaluate students' work habits is valid. 3. The method I use to evaluate students' work habits is etTicient 4.29 1.91 .84 Whole scale correlation .51 4.31 1.80 .81 .36 4.45 1.73 .84 .44 5.69 5.54 1.25 1.30 .58 .53 .44 .37 Table 9 treats two distinct concepts, one related to the usefulness of the evaluation mark and the other related to the process. Items 1, 2 and 3 performed similarly, with similar means and within-scale correlations The difference in overall correlations is significant for items 1 and 2. It is greater than the difference between a large and medium effect (Cohen, in Kirk, 1996, p. 751). The means for all items place the overall ratings within the middle range. Elements Relating to the Research Questions The data were grouped into four sections in the survey. These sections are; factors in work habit definition, purpose of the evaluation, process used and summary statements. The data were grouped from a seven-point scale into three discrete units. These are low (ratings o f 1 or 2), middle (ratings of 3 ,4 or 5) and high (ratings o f 6 or 7). This was done to give a clearer picture o f trends that may occur, and to simplify interpretation (Kirk, 1990; Sax, 1989). The results are in the following tables. 42 The first section was concerned with the factors used in defining work habits and how important each was to individual teachers. The factors are presented in Table 10. Table 10 Work Habits Definition Factors Number Statement 1 The student’s work is easy to read. 2 The student’s work is neat. 3 The student’s work is completed on time. 4 The student uses class time effectively. 5 The student has the necessary materials available 6 The student participates in class discussions. 7 The student follows classroom rules. 8 The student cooperates with peers. 9 The student makes an effort to improve. 10 The student has a positive attitude. 11 The student works to the best o f his/her ability. 12 The student succeeds in class. Table 11, on the following page, illustrates the percentages in each category for each of the factors. 43 Table 11 Percentage Agreements Ratings o f the Work Habits Definition Factors Facton in W ork Habit Definition 0»o jsN ol Important ■1> * • i#&)mewha( Impurhinr OVety Important 3l*a 3 4 i 9*. : IS IS ! IS < 6 % I5S 8". i U S 31% : S IS ' OIS I 5 : 6 8 6 S 1 3S 49*i : 46S i j , 7 ! 3*. . 8 i 3S 1 9 10 IS IS i 3"é 1I0"i 1 i l l ; 11 I 1: ; ::s i 28". IS*. :s*. ; I:*. , 46". 67", S IS 69*. , 86*. ! 40*. t 76". The factors that were rated “Very Important” by the greatest number of teachers were (4) using time effectively, (5) “has necessary materials”, ( II) “works to the best of his/her ability”, (3) “completes work on time”, and (9) “makes an effort to improve”. At least two-thirds of teachers rated (7) “follows class rules”, (8) “cooperates with peers”, and (10) “has a positive attitude” as very important. Factors that were considered “Somewhat Important” were (I) “work is easy to read” and (2) “work is neat”. Items (6) “participation in class discussions” and (12) “succeeding in class” were rated somewhat to very important. The range used for rating the 44 factors as very important was from 31% to 91%. Factors 3,4,5,9 and 10 are seldom considered to be “Not important” . The next section focusses on the opinions of teachers about the purpose of the mark. There are nine suggested reasons. These are presented in Table 12. Table 12 Purposes of the Work Habits Evaluation Mark Number I believe that the purpose of evaluating work habits is to . . 15 describe past behaviour. 16 alert the student to concern about her/his academic progress 17 encourage improvement. 18 encourage continuation of satisfactory behaviour. 19 praise acceptable behaviour. 20 demonstrate the link between school and workplace behaviour. 21 communicate with parents. 22 follow school district requirements. 23 follow Ministry of Education requirements. Table 13, on the following page, illustrates the frequency percentages in each category for each of the factors. 45 Table 13 Percentage Agreements Ratines o f the Purpose o f the Work Habits Evaluation Purpose o f Work Habit Evaluation 8U“o 70“o 30»ii , 20“o I0»o - IS ; 16 : 17 ! 18 : 19 : 20 jSSlran^y Dinayce .12". 9% 0"6 ■ Agfec 33“o 45". 31". jO Snin^y .Vgn* 29". 41". 67". 3". 3">> 9". 27". 28". 31". 68". 67". 56". 21 3". 26». 24». 29". 50“b 51"b ' 67». 23". 22". ! “Strongly Agree” was selected by the greatest number of teachers for the following factors; (18) “to encourage continuation of satisfactory behaviour’, (17) “to encourage improvement”, (2 1) “to communicate with parents”, and (20) “to demonstrate the link between workplace and school behaviour”. Teachers showed moderate agreement with the suggestion that the purpose is to follow (22) “school district” or (23) “Ministry requirements”. Teachers showed moderate to strong agreement with (16) “alerting the student to concerns about academic 46 progress” and (20) “demonstrate the link between school and workplace behaviour”. No trend emerged for (15) “describing past behaviour”. The range for strongly agreeing with the suggested purposes for the evaluation is from 22% to 68%. The third section examined ways teachers evaluate a student’s work habits. The options in this section are presented in Table 14. Table 14 Processes for Evaluating Students’ Work Habits Number When I give the report mark for work habits for a student, 1 26 rely on recorded data. 27 rely on my memory. 28 evaluate subjectively. 29 evaluate objectively. 30 evaluate using a combination o f subjective and objective criteria. 31 consider the student holistically. 32 use criteria which I have previously described to the students. 33 use criteria that I assume is understood by the students. Table IS illustrates the frequency percentages in each category for each of the factors. 47 Table 15 Percentage Agreements Ratines o f the Work Habits Evaluation Processes Processes Used to Evaluate Work Habits 80“o . 70«6 H 50“o , 0“o f. 26 |0Ne«edBMi'eicdchieiti. "When 1 ghrc the report mark for work habits for a Student, I 26. rely on recorded data. 27. rely on my memory. 28. evaluate subjectively. 29. evaluate objectively. 30. evaluate using a combination of subjective and objective criteria. 31. consider the student holistically. 32. use criteria which I have previously described to the students 33. use criteria that I assume is understood by the students. 34. other? 35. other? N ew New New New New N ew N ew N ew N ew N ew 1 2 1 2 1 2 I 2 I 2 1 2 f 2 ! 2 1 2 I 2 3 4 3 4 3 4 3 4 3 4 3 4 3 4 3 4 3 4 3 4 S 6 S 6 S 6 S 6 S « 5 6 5 6 5 6 S 6 S 6 7 7 7 7 7 7 7 7 7 7 Ahrayt Almys Ahoyt Ahnyi Ahnyi Ahwyi Ahrayi Ahnyi Ahnyi Ahnyi Comments Som yF w w l rop2ef 3 Sept.DaeUH 74 Appendix C Survey (page 3) Uiietniqr eTNttlbin Bhdeb W«fc B M tt EnkuliM S v n ; V. Sumniary qucstkms PkMc iadicaie ihe eneM 10 «Mch yon i p v mid) ihe K H canis b d o « by did in g die oppoeiieeechi 36. Wofk habit lenergndes help ttudentt with the task of leaniing good work habiu. 37. Wofkbabitleilcrgiadesbelpteacben with the task of teaching good work habits. 38. Work hahit letter grades help parents with the task of teaching good work habits. 39. The method I use to evaluate students’ work habits is valid. 40. The method I use to evaluate students'work habitt is efficient I dHt R p m ta n your opiaicB. Diwpee i 2 3 D iaine i 2 3 Diairae i 2 3 Dinpee 1 2 3 DUagne i 2 3 5 6 7 5 6 7 5 6 7 5 6 7 5 6 7 Apue Afiee Apue Apee Apee Summary Comments Please note bdow any aammary or gcaeral comments that you would like to add to this survey Thank you for your time and assistance. Survey Perm! PigeSsf 3 Sept -D ec!?•§ 75 Appendix D Copy o f letter that accompanied each survey Dear colleague. Thank you for the interest you liave shown and for taking the time to support my research. This sur\ey. when completed, will provide me witli important data for my graduate thesis. I am a graduate student at UNBC in Prince George. 1am also a full-time tcacher-counsellor at Blackburn Junior Secondary School. My tliesis question is "How are work luibits defined and evaluated for the purpose o f reporting in B. C.?" As you know, work habit marks have a number of implications for a student's future. My interest is in how professionals define the particular behaviours tliat they perceive as work habits and how they then assess and evaluate these behaviours. In addition to the sur\cy. you have been given an envelope. .After completing it. please place the sur\ey in this envelope and return it to the person who gave you the survey Be assiucd that I will do everything possible to maintain your confidentiality. Your name will not be passed on witliout your pennission. All envelopes will be discarded and all the returned surveys will be mixed together. Teachers from several school districts are participating in this study and no surv eys will be identified by subject, school or district. Tlic surveys will be destroyed after my thesis has been defended. I would be happy to share the results of this study after it is completed. I will be preparing a synopsis of the results. If you would like to receive a copy at your school, please ask the person who contacted you about this survey to request a copy on your bclialf. I w ill liav e the requested number of summaries deliv ered to your school. Again, thank you for participating in my research. Sincerely. Marion Hofmann 76 Appendix E Copy o f letter that went to each school confederate Dear— ; Thank you for agreeing to assist me in my research. As we discussed over the telephone, the enclosed survey is the final step of the research I am conducting to complete the requirements of my graduate thesis. I am deeply appreciative o f your assistance. I plan to ensure that no completed survey will be traceable to either the district or the respondent. Therefore, there are some steps that I ask you to follow in order to preserve the acceptability and confidentiality of these documents. You have been given a stamped brown envelope that has been addressed to me in care of my advisor. Dr. Bryan Hartman at UNBC. As this envelope arrives at the university, it will be opened, discarded, and the smaller envelopes will be set aside. A white envelope that has been addressed to me should accompany each survey form. If your staff or administration are uncomfortable with the process or document, do not distribute the surveys. Simply return them to me in the pre-addressed envelope with a note of explanation. If there are questions, I may be reached at 250-963-7474 (school), 250-963-8237 (fax) or hofmann@bc.svmpatico.ca (e-mail). When approval has been granted, distribute the survey and envelope to volunteer respondents. Please keep a list of those who have taken a survey, and mark off when they have returned it to you. After completing the survey, each teacher has been asked to place it in the envelope and then seal the envelope. When all surveys have been collected, place all of them in the large brown envelope and mail it. Once again, thank you for taking the time and energy to assist me in this research. Sincerely, Marion Hofmann