FVA I l lA I I() OF IIII-- 1--FFI ·C I l V I· l SS () I I JJ F Sl· C'() Ds\RY SCifC)Ol EC II J\KC) VA l L I· Y 1A IIJI \L\ ti CS J> RC)CiR \i\1. \ Sl ·RII ·S \ I I f·RRl JPT I·D ll f\11 · \ 1 YSIS B.I d .. I o u '-~ h bo ro u .....e. h lJ n t \c r~t tv. . 1977 'Ti ll SIS PRC)PO. AL .'lll3'v11TTI D I ~ P,\R I IAL FL' LI·I LI Jv11 ·N J () J· TI IE REQUIRI:MEN f S l·C)R 1'1 fE DI ~ GR E E C) I· M '\.'Tl :R () J· I DliCA l l C)N . 111 Cl JRRICULlJM A D INSTRl 'CTION © Lynn Makysn1chak. 1998 T ill: UNIVER ITY OF NC)R l I IFRN BRI riSI I COl UMBI \ J unc 1998 All ri ghts rcser\ ed. I hi s \vork n1 ay not he reproduced in \vhole or in part. b; photocop) or other 111CU11S, without the pcttn ission or the nuthor .. 11 1\BS I RJ\C r fhis StUd}. a progra111 C\ aluation, Ill\ C~ttgatcd hrogran1 to teach n1athcn1 at 1cc.., to all students in the school affected the n1ean e'\atninatlon ~core and the partic1patton rate for Math 12. The stud) C'{UJnincd pr<)\ lnci,ll e~c.lll1111at i 011 stattstics ror Math 1 2~ CheJnJstr) 12 and Ph} sics 12, and the GPJ\ scon~c.., olthc c..,tuclcnh tal-..1ng Math 12. I in1e ~ene~ anal) ~i~ \\a~ u~ecl to tn\e"itigatc trend~'' tth1n the data. Re~ults dtd not support the l\\O n1ai n h} pothe~e~ that the introduction of the '\'~ ~ \ lathen1at1c~ Progran1 t111 pt o\ ed n1ean score and partic1patton rate. llO\\e\er \\hen C\ 1dencc of cohort cJfectc..;, c..,c]f c..,clect1on e{Tccts and dtstrict trends \vas con~iclcrcd, the prognun. \\a~ judged to. at \\orc..,t. y icld results no eli rrcrcnl than the traditional approach. Son1e e\ idcncc suggc~ts the re 111ay ha\c been a slight in1proven1ent particularly in the relati\c increase in participation rate. . .. 1II ·11\BLI· OF('() IF IS Abstract I able of Contents I i~t of fable~ .. II ... I 11 \ . List of Figure~ VI 1\ck.nO\\ lcdg1ncnt \ III PlJRPOS I· OF I I IE S l lJDY Mathcn1atic~ Curnculun1 in Bntic.,h ( 'olu1nbia rhe echako Valle\• Second an• School \ 1athcn1attcc., 'urnculun1 I esting \\r ithin I he VSS 1\ lathcnulllcc., J>rogran1 I est re\v rite ~ \\ ithin the progran1 I \.Ception ~ to the cut c.,core rule Goab of the "'\;VSS Progran1 Mathen1atics 12 1nodi fications I'he provincial C\:Hl11ination Concerns and I s ~ucs t\1edia C<)\ crage Progn1n1 Evaluation Quantitative Methods Qual itati \ e Methods Which Method i ~ Appropriate? Quas i-Experin1cntation Causation Cohort design Interrupted tin1e series analys is Outcon1c n1 easures Research Questions rhe NVSS Mathcn1attcs Mean Score The NVSS Participat ton Rate Stati stical I Iypothcscs 1 11 12 13 14 14 15 16 16 17 17 1g 18 18 18 19 LI I ERJ\'I U RF Rbl I\ I I._ D TO 1'1 II· S flJDY 20 [._ va luation of l ~ducational Progran1s Selecting an Appropriate Fvaluation Model Evaluation Modele., Detennining the Model for Tht ~ Stud) Jud ging Success C)uas i -Expc rinu~n tati o n - /\ n Approp riat e Method l·,x Post l·acto F\aluation 1 4 7 8 <) 9 10 I1 -I} '6 '") 0 _ () 28 ~(} tV fhe uitahilit) of fitne Serie~ De~ign s Ach antages of the Destgn Cautions J\d n1ini stering Thi s I in1e Series I·"pen n1ent ~11 ri IOD 16 . ubject~ Measures Mean Scores Participation Rates Procedure ,\l pha I C\el R I· S lJ I ~I S Question I : Wa~ there a change in the NVSS n1can ~core? Question 2: \\' a~ I here a Change 111 the VS') Parti cipation Ratc'J Other I· ~planations Partict pat ion Rate an <.I \lean Prov1nc tal Exan1i nation ~cot c Grade Point A\ eragc Score~ Cotnpari so n of l:xmnination Data Acros5 School Courses Mean Scores Participation Rate Con1pari~on \Vith Other Schools in the San1c School Dt ~tnct ,'UMMARY AND CONCLU IO Sun1n1ary of the Study Conclusions Mean Scores Participation Rate GPA Scores C'on1paring GPA Scores and Mean l:xmnination Scores Con1paring Chetnistry 12, Math 12 and Physics 12 Con1paring NV. S, FSJSS and I· LFSS Sun1n1ary Of Interpre tation of Result~ Conclu5ion Luni tattons Deficiencies The Evaluation Model Rccon1n1endations for Practice and Research /\ PJ>l ~ N DJ X 10 12 13 15 1 Pcrn1ission to usc data 16 17 38 1R 19 40 4I 41 47 49 50 5I 56 56 58 62 67 67 68 71 71 71 72 72 71 71 75 75 76 77 78 v I tst of Tables I. Math I 2 Exan1ination Data 42 I...... Math 12 Participation Rates 47 3. Values Assigned to GP ,\ 'cores 52 4. Mean GP/\ Scores for NVSS Students Taking Math 12 51 5. V Mean l· xatnination Score~ for Chetni~tn• 12. Math 12 and Physics 12 6. NV Participation Rates for Chctnistry 12. Math 12 and Physics 12 59 7. Math 12 l·,atnination Mean Score~ for 62 8. Math 12 Participation Rates for 9. Utnmary or Results VSS. I S.JSS and I· LI ·SS VS.'. fSJSS and FLESS 64 69 • VI List or Figures VSS f\.1ath 12 pro\ incial e\.anlination n1ean ~ --;hOught fronl a\ anct~ or intcre"'>t group'> tncluding teacher'> and uni vcrsit\ facultY. rhc f'OCLI S of thi" fC\ t"ed curriculu111 \\aS "'the need for all "tuclcnts to r "' be provided V\ ith a hal(.lnce 0 r tnathen1at teal e'\ pcttenccs .. (M inistr} 0 r l:d ucatt on. I 98 g p. \ i). 1 he curriculun1 gu1de \\Cnt on to "'> uggc"t that all n1etnhcr~ of "octet; need 111athctnatical skills and that tnathen1atic"'> I'> an 1111portant cotnponcnt or education. It \\(1<) also rccon1tncndcd that ever) student recct \ c 1n~tructt on appropriate to thci r needs and abili tic~. '1he rationale behind these beliefs \\US ba"'>ed on a nun1bcr ol points. It \\as suggested that n1athcn1atic ~ pcrn1eates other curriculun1 areas and that n1athen1atic(.) i "'> a necessary tool. Jn order to understand n1athen1atics. a student needs to be able to think. analytically and to reason logical!} . Po"i trv c attitudes tO\\ ards n1athen1atic ~ arc developed when students are provided\\ ith experiences \vhich de\ clop thetr understanding or n1athcn1atics and which arc appropriate to the level or ahil ity of the ~tudent. It is ··critical that ~tudcnts have successful experiences" because success is a powerful reinforccn1cnt (Ministr} ofFducation, 1988 p. \iii). fhe"c C\..pcnenee~ ~hould be dcveloptncntal in nature and should he organi/ed scqucntiall). 'I he curriculunl guide also etnphasizcd th at th e 111alhcn1atical abilit~ f~lll ~ along a continuun1 or ~tudenh Therefore the "content and prcsenlatton 1 tn tnath cltlS~es J ~hould be appropriate to the liH.:rca<.,tngl} dt\Cr"e need~ of all ~tudent"" (\ lt nt"'tr~ of' l dUCc.lltnn, 3 1988 p. viii). The guid e inforn1cd teachers that en1phasis should be plnced on the dcvelopn1 cnt of n1athcnH1tical ski II~ in students because these skills \Vere seen as necessar) for the student to function in soc1ct; and for the student to pursue rurthcr st ud y in areas invo lving n1athen1atical applications. All ~tudenh ~hou ld he encouraged to ro~ter positive attitudes towards I ife-long lcarni ng. I he curriculun1 gu1de then outlined the patl1\vays \\ hich students cou ld take through the n1athcn1atic~ courses orrcrcd llntd the end of Grade R all '-ltudent~ vvcre required to follO\\ the san1c n1athetnatics curriculun1. ~ \ftcr Ciradc R. the path ~p lit ~ Into two. On one route. a student \Vas directed through Math 9. Math I 0 and Math I I. all or vvhich cn1phasi/c abstract concepts and S)l11bolic notation. In Briti ~h Colun1bia. Grade 12 n1athctnat1cs courses arc clectJ\ cs and arc not required. After Math 1L the student was given a choice of Math 12 or Survey Math 12. The other route took the ~tudcnt through Math 9/\, Math 1OA and Math 11/\. fhcsc courses place less emphasis on abstractions and notation. and allo\v students to cover the required n1aterial at a slovver rate (Ministry of Education, 1988 p. xiii). The student who took the second route was provided an opportunjty to re-enter the first route. Introductory Math 11 was designed to allow students \Vho had cotnpleted Math l 0/\ to qualify for Math 1 I. As Math 11 i~ the fifth n1ath course, it wou ld be taken in the Grade 12 ..vcar. The Math 12 cour~c is then not accessible to students who have chosen this route. 'I he who le Briti sh Colllln bta n1athen1atics curri cu lllln \Vas re\ ie\ved lollt)\\ inl.!.'- the 1990 Provincial /\ssessn1 ent 0 f Mathetnatics (Min tSlt') 0 r I. clueation. 1996) In 1996, new Integrated Resource Packages (IRPs) replaced the old curricul un1 gut de~ 4 Mathcn1atics 8 is still tuught to every student acro~s the pnn incc. Arter (irade R, th e IRPs outline t\\tO different patlnvays for all grades 9 to 12. I hcsc path\\lays arc nnn1ccl Principle ~ 0 r 1ath en1a ti c~ and \pph catto n ~ or \1 athenlatic~ I he Pn nci pic ~ 0 r I Mathetnatics path\\U) for grades 9 through 12 CO\ cr~ the ~a ln e content a~ the rornlcr Math 9, Math l 0, Math 11 and rv1ath 12 d1d . I here ~ ~ an tn c rea ~cd cn1phas1s on prohlcn1 ~0 1\ ing and the u ~e 0 f gt a ph I ng ca leu! ato1~. but the t o piC ~ I CllHllll th e ~a Ill C. i\pplicattons or 'I he 1 athe tn atic~ path\\H) C<.ll1110l he C0111pared to the IO llll Cr Mttth 9;\, t\1uth IOA, and Math I I;\ ccnt r~cs <:; ince it d oc~ not cover the ',UlllC tnatcrial. Prcs~ure fro n1 individual school di ~ trict s has pe r~uaded the Min1 ~ try ofl ·ducation to allcnv ~c hool s to continue offering these 1\ cour~e~ until the) too can he e\:an11ned and updated . Princi plc~ of Mathen1atic ~ 12. \\ htch repl ace', the old Mathcn1atics 12. n1u~t be in1plen1cnted in all schools in Septen1bcr 1998. 'I he provinc ial cxatnination based on thi s course \Vill be offered for the first titne in Januar; 1999. 1 he Ncchako Valley Secondary School Mathernatics Curriculun1 In 1993. the 1nathen1atics teachers at ec hak.o Valle} Secondary School ( V~S). a Grade 8 to Grade 12 schooL developed an individuali7cd. n1odulari/cd. se lf-paced n1athenutti cs progrmn for their students. 'I he prognun was itnplen1cntcd for a nun1bcr or rea~o n s n1 ost or \\lhich can be rel ated to the philo ~o ph } and goals of the Bntt ~h Collunbta Curricu lutn . The progn1n1 was used to teach all students in the school so that all \\ere exposed to the san1c n1athcn1attcal expen cnccs. 'I he progrmn \vas dcstgned ~o that s tudent ~ earned n1arks no IO\\er than a B (71° o). 'I he progttltll aiiO\\cd for uH.ll\ rdu~tl 5 differe nces in learning styles and speeds since students vvot"k.cd individually and independc nll y. NVS n1athen1atics teachers developed the progran1 bel iev 1ng that n1athc tnatics is an i1nportant part of education and that all ~tu d e nt~ ~hould take as n1an} n1ath course<.; as they arc able to con1plete. I'hesc 111athen1atics cour~es sho uld also he at the highest level offered. By usi ng this progran1. it \\a') intended that the student set th eir Ovvn tin1cline for stud; ing n1athen1atics. Thts in1pl1ed that ~o rne ()t udents \\Ottlc.l need n1orc, or less, tJn1e than \\as suggested b; the curriculun1 guide. Students \\ere allo\\ed to take a<.; n1an y setnestcrs as they needed to cover all of the curriculun1. In the NVSS progran1 , all learning outcon1cs specified b) the curriculun1 guide arc included. This n1eans that students n1ust CO\ er all of the topic~ listed in the curriculun1 guide instead of being exposed only to those topics for\\ hich the \\hole class had tunc. In traditional elassroon1s, the pace at which content is delivered tnust n1atch the pace at which the average student is able to absorb the n1aterial. Traditionally, n1athcn1atics classes did not cover the \Vhole curriculun1 because of tin1e constraints. There \\as too n1uch n1atcrial in the curriculun1 to be covered in the tin1e scheduled . rhe decision as to which areas to n1i ss out \vas left with the teacher. In the NVSS progn.1n1, students covered all topics and so were actually exposed to n1ore content than had pre\ iousl~ been the case. The average student was sti ll ab le to n1anage this but students \vith le ~s than average n1athen1atical abi lity did find that the scheduled tin1c vvas not enough. The progran1 allowed ror this by letting students take 1110re tin1e to fini sh a COUr~c 6 Positive atli tudes tO\\ards n1ath arc fostered \\hen ~ tudents understand the content and arc able to cope \\'ith the acaden1ic requiren1cnts. ·yhe NVS l rnathetnatics progran1, required all students to earn a pcclficd grade 111 order to pa~~ a te ~ t. and each tec.,t n1u c., t be passed before an) c.,tudent \\a ~ allt)\\ed to 111 0\ eon. ·1hc~e 111Jntn1UI11 cut c.,co rc~ ensured that the student got good g rad e~ \\htch 111 tutn tna) ha\e tnotl\ ated the student to keep stud; ing. f\1ath teachers Icit that ~t udent s UI H.kr~tood tn on: of the content \\hen the) passed the tests. I he VSS curriculun1 ''a~ organ t;ed cle\elopnlenta lly and c.,equcntiall y. Studenh arc first tested on 1nd i vidual lcan11 ng outco n1cs. then on larger and larger clusters of outcon1cs. I his ~)~tc n1 aiiO\\Cd the student to \\Ork \Vith the ha~tc 1cleas before being asked to appl; then1 . Once the ha~ic concepb \\ere acquired, group1ng o f co ncept<; \\ould have allo\ved the student to see ho\v concepts fit together and how they can be applied. NVSS n1athen1atics teachers also believed that the instruction a student received should be appropriate to the student 's need~ and ab d ities. ~~h e NVSS n1athen1 ati c5 progran1 allo~ed s tud e nt~ to \\Ork. at their 0\\11 pace. I cachers pro\ idcd instruction a-... 1t was needed by the student. This learning environn1cnt should have provided all students with the opportunity to learn and to progress. ·r he presentation 0 r 111atenal happened in a varicty 0 r \vays. studcnt~ \\ ork.cd f ron1 textbooks and used teacher-prepared n1atcnals. Student~ could c.Kces~ tutoritll-.... pre- tests and review exercises. Fach student was prov1cled \\ ith a I ist or re so urce~ crossreferenced to the appropriate grade or tnathetnatics. 'I he te~tbook..s u ~ed "ere not c.l h\ c.l\-... the ones recon1n1cndcd hy the Mtnt c., lt) or I 'ducation ror that grade but \\Cre u-...ed beCdll\C 7 of the su itabilit) of their content and for their case of use. /\I I te-xtbooks used \vcrc approved by the Ministr) or l·ducation for usc in schools. VSS rnathcn1at1c ~ classe~ arc organ11cd to "u rt ~t ud e nts' "chedulrng. Mo ~ l classes arc con1po~ecl of"tudent~ fron1 different grade~ \\tlhin the ~chool takrng tn grade 11. Unit tests were designed to assess the s tudent' ~ undcr:-,tanding or a group or core topics given together and to aid the student in linking concept~ ·I he unit tc"t "~1" developed using tnore cornpl cx questions \Vhich required several steps to so h e Unit k'"t q ucs tion~ rcq u1red the app lication of a concept as \\ell a~ UtH.Ierstand 1ng. 'I he cut "cote for a unit tc<;t \va~ "ct at 70° o for gtade~ R through l 0 and ()0°/c>101 grade 1 1 ~ lodulc exan1s \\ere designed to assess the ~ tudent · c;; g ra ~ p or the concepts fi·on1 se\ era I core topics and unit tests. The tnodule exan1 replaced traditional 1111dtenn and final c'atns but \\as not cun1ulati"e in content. For e"\mnple, th e Modul e 4 F"\an1 covered only the n1aterial in that n1od ul e and not tna teri al fron1 the first th rce n1od ulcs. The 1n tent behind the n1odule exan1 \vas to gt\ c the student the chance to relate several principles and to usc thcn1 to sol\'e n1ore in\ oh eel questions. 'Jhe cut score on th1~ e'\anl \vas set at 60°'<> fo r all grades. Generall; four vc rstons of each te~t \\ere de\ eloped. rl he dlf ferent \ er~JOilS we re \vritten b) the san1e teacher but ha\e not been Cs onl) tho~c distncts \vh rch arc rnatchecl by cletnographics. then the stor) \\ otdd have had 111l1Ch 1nore crcd ihi 1i ty. Ir the reporter had read the cautionary notes fron1 the Ministr) of I·ducation. a cleaner picture 1ni ght have e1ncrged. In order to counteract this negative publicity, the NVSS n1athcn1atics teacher~ began an asscssn1ent of the tnathenHltics progrmn. The teachers \Van ted to kno\\ \\ hcthcr the progran1 was doing the job for which 1t was developed and whether 1t \Vas tn ~cttng the goals which had been establi shed. Progrmn Evaluation One o Cthe fun cti o n ~ of educational research i~ to tn1pnn c pract tcc. I 'a lu ~tt ion research exarnincs practice at a particular ~ 1te and ts concerned \\ ith ,l""e~-., rn g the n1ent 01 14 \VOrth of a spcci fie practice. rhc le\ cl or gene ral izabll it) is IC)\\ since the rc ~ ults arc usually specific to the site under study and to the specific practice at that site. Fvaluation research is often used to ''advance the research and n1cthodology ora specific practice Jand toJ aid in dcci sion-n1aking at Ia! gi\'cn site" (McMillan & Schlllnacher, 1997, p.20). T\VO dif'f'crcnt paradigtns can be applied \\hen C\aluating progran1~. C)uantitativc and qualitative tnethods arc both used in thi s field . Bcf'ore beginning any progran1 evaluat ion, the evaluator n1ust dcctdc \vhich n1cthod best fits the si tuation under cxan1ination. Quantitative Methods Quantitative n1ethods are concerned\\ ith dctenntning the relationship between \ ariablcs, \v ith con fi nning or discon firn1 ing hypotheses, and \\ ith identifying con1n1on properties (Guba & Lincoln, 1981 ). In this paradign1, e\aluators arc interc~ted in getting the ans\\ers to speci fic questions. and 111 verif) ing their \Vorking theories. 'I he cn1phasi~ is on change and growth. The evaluator seeks to ascertain \vhat would have happened without the treatment. The tnethod can be applied \Vhcn con1paring past performance with current when a progratn has been applied universal! y. Qualitative Methods Qualitative inquiry is n1orc oriented to activities that to intents (Guba & Lincoln. 198 I ). In thi s paradigtn, the identification of the issues drives the research design \vhich then provides the inforn1ation to he studied . Responsive e\ aluation gi\es 1ts audience the infonna ti on it requires. 'I he contc' used. The 1nathen1atics teacher~ sought to develop a progratn that \VOUld in1provc the learning of the ~ tudcnt~ in their classroon1s. Following the 'I ylcrian n1odel. used in developing n1athen1atics curricula for n1any years, the NVSS n1athe1naties teachers developed a progran1 \vhich \Vas obJecti\ e based. Speci fie outeo1nes \Vere expected and tested for. Progression through the progrmn was based on asscssn1ent or student learning after e>..posurc to each or the"e concept~. l·valuation of SUCh a progranl should be llllUertak.en ll~ln g the satne objectl\ C bc.l~t.Xf criteria. 16 Guba and Lincoln (I 981) pointed out that progran1 C\ aluation is only usefu l 'A-hen j t deals \\ i th the actual concerns and issues 0 f the 93(1110) the pr<)\ ll1Cial cxan1tnation 111Can pnor to 1991 ro r Mathcn1altCS 12. J' ,u"' 19,n = J[ po\tl'N3< ma 1 H I: J[ prd 993( IIIli) :f. J[ f10\tl 1J91( 11/(/) Jr p rdCJCJl( mo) was the proportion ofNVS students taking Mathcn1atics 12 prior to 1993. Jr p m tt sfu l tn affecting the \vay that N VS ., ~ tudenh learned mathen1atics. 1 he particular question that can1e fron1 tbi~ global v tew vva~ whether the NVS provincial exan1ination n1can score changed \\ tth the introduction of the prograrn. Did this progran1 do a better j ob than a traditional approach? Secondar; to thi s was the other question. Did the patiicipation rate increase? That is, did n1ore students take Math 12 as a Grade 12 elective? These questions fonn the iten1s of a progran1 evaluation. It was necessary to detennine \Vhether progran1 evaluation tn ethods were suitable to this study and which of these n1ethods was n1ost suitable. As \vas indicated in Chapter 1, the method assun1ed to be n1ost suitable is a quasi-expcri tnentaL tin1e-scrics design. Thi s chapter develops the points already n1adc 111 order to strengthen the case ror the usc of thi s 111ethod. 22 E' aluation of Educational Progratns E\aluation is an integral part or the curriculun1 sequence \\hich also includes planning and dcvelopn1ent (Madaus t~ .._ tufflcbeatn. 19R9 ~ r; ler. 1949) Progrmn evaluation is a purpose rather than n tnethod (13abbie & Wage naar. l9R9). The purpose or progran1 evaluation 1s to asse~~ the progrmn'c.., product. It ic.., an appraisal of the progran1' s worth or qunlity (l laug. 1996: Lan1. 1995: J>ophan1 . 19RR). I he info rn1ation collected can be used to itnprovc, n1a1ntain or tcnntnatc the progratn (Conrad & Wibon, 19R6 ~ Provus, 1973: Worthen. 1995 ). I·valuation ~hou ld co ntri butc to the edu cati onal proccs~ (Worthen & Sanders. 1991 ). The evaluator n1ust consider \\hat ,,ac.., Intended b; the progran1 'c.., introduction, "' hether this intention can be n1casurcd. and. i r it can. ho"" (8 abb ic & Wagenaar, 1989: Hedrick & hipn1an. 1988). The ai1n i') to find out "hO\\ flll· the learning experiences a~ developed and organi?cd are actually producing the desired results'' (T; lcr, 1949. p. I 07 ). The NVSS progrc.un was devel oped in respo nse to British Colun1bia's n1athen1atics curricultun 's en1phases. Its educational acco n1plishn1ents or in1provi ng the way n1athematics is taught and its effect on student outcon1es should be as hi gh as possible (Fraser, Walberg, Welch & I Tattie, 1987). In this study, the evaluation is investigative (Sn1ith & I Iaucr, 1990). The organization of the NVSS Mathen1atics Prognu11 facilitates ongo ing n1oddicatJons \\ htch n1eans that this evaluation is ronnativc in nature. 'I he progran1 ts at a stage'' here it 1" itnportant to ask vvhat change~ have taken place as a res ult or the tnten cn tton and whether n1odifications arc needed . l'ylcr ( 1949) \\rotc that at th1 " point tn the 21 dcvcloptnent and use o r a progran1, evaluati on is a key step since it provides "inforn1ation about the success of the school to the school's clientele" (p. 125). The purpose or tht s stud; , and of other prograrn ev aluat i o n ~, was to collect inforn1 ati on ''hich \\ Ould addre~~ quec::, tt nn~ abo ut rnathcn1 ati cs teaching. It \vas abo in1portant to be rcspon<.; i\ c to th e concern 'i of the stake holders, the larger con11nunity (Atneri can L valuati on Association, 1996) . electing an /\ppropnate Eva lua ti on Model Progrmn cvaluators need to con~H.Ier care fu ll; .. hov\ to C\ aluatc d y nan1 ic, u n ~lab l c progran1S Or e ffo rts \\ hen there C'd ~l~ l1ttlc prior kilO\\ ]edge about thc n1 or ahout ho\v to evaluate then1' (Sn1ith & I Iaucr. 1990, p. 490). DiCfcrcnt conclusions may be reached depending on vv hich evaluation n1odel is used (I Iaug, 1996 ). Evaluati on Models The n1odel of progrmn evaluati on chosen to a large extent depends upon the curriculun1 orientation of the progratn. /\oki (1985) identified three curriculun1 orientati ons: the En1pirical-Analyti c (Technical), the Situati onal and the Cri ticalRefl ecti ve ·rhe En1 pirical-/\nalytic ()rientati on IS the don1 inant curriculun1 n1odel. It I\ ba\ed on an explanation of the way the wo rld V\Orks and cn1 phnsi/CS tcchntcal kno\\ ledge Curri culun1 rcscnrch in this orientation is scientific in nature and tn akes usc or cxpcritncntation. It is co ncerned \v ith ca use and encct and cn1p1t 1cal k.nO\\ ing. 24 Evaluation is goal based and ach ie\ cn1en t ori ented . .T udgctnent is tnadc ba~ed on e'\ ternal standards. ·1he Situational Orientation ts concerned\\ tth the tn eaning that students gt\e to situattons. lt\ cn1pha~t\ t\ on con1n1unication <1nd phcnorncnologtcal de\crtptlon. Progratn C\ aluat ton is tntcrprctt\ c and encourage'-~ each per~on to tn ake thc11 O\'v n tneaning. ·r hcrcf'orc the rc~carchcr nlU\t enter into dtaloguc \\tlh people tn the si tualton. Vv'hat i~ needed arc fir~t order dc\crtpllon\ '' htch arc unrncdiate tntcrprctattons of the situation. The progran1 C\ aluator 1~ concerned \\ tth the qua lity or tht~ li\ ed expencncc The Critical-Reflective ()ricntation i~ ba~ed on reflection and the intent to bring about change. 'I he C\ aluator in thi~ conte\.t ts ~ec kin g to dt~covcr the underlying asstunptions. interests and \aluc\ \\hich undcrptn the ~tu clcnt' s c'\perience and \\htch underpin the curriculun1 as deli\ered. In this orientation. the re~carcher n1U\t quc~tion the descriptive accounts provided and encourage critical re1lection about the progran1. 'I he individual perceptions lead to tnultiple interpretations of the situation. Worthen and Sanders ( 19R7) out! incd four tnethodological approache~ to progrmn evaluati on: the Experitnentalists, the Eclectics, the Describers and the Benefit-Co\t Analy/crs (p. 55). The first three of these can he linked directly to J\oki's curriculun1 orien tat ions. ·I he I· \.pcrin1entaltsts seck to tdcnttl). cau\nl link\ and etnplo} e\.pct uncntal and quasi-cxpcrin1cntal designs. They n1i ght sec curri culwn Crotn an l·tnpirical <\nal} ttc ori entation. Progran1 evaluation could be based on te st~ "cores (c g. D tlltn. 1974. Evaluatton Upda te. l99X : Sander". 199 q, cau~al tnodcltng (c g. l tchclhetgct. 1974: 25 Wang & Wal bert.1983 ), or the anal) sis of data den\ ed Cor other purposes (e.g. Lee, Croninger & .. n1ith. 1997: • chtnidt & lv1cKnight. 1995: Wcstbur; & I l~u. 1996). The Fclectics ~eek. to enhance the search for causal ltnks b} C'XO) In thi ~ situation the; are natncd ~ tnglc ca~c tin1c ~cries and have a ~inglc situation \\ ith tnultiplc tin1c points (Sa\\) Cr. 19R7) . ..1he) arc al~o appropnatc to u~c \\hen dealing \Vith archival data \\hich ha~ been routtncl) collected fot adtnini<;trati\c rather than cxpcrirncntal purpose~. I in1e scncs C'\.pcritnents can be u~ed \\hen different group~ cannot be treated tn different \\a}~ (G la~s. 1997: P atter~on et al.. 1992: Sn11th & C1la~~- 19X7) . .., he; do not require the \\ithholding ol trcattnent fron1 an; one (f\lcConnelL 19R2). Thi~ latter po1nt ~~ particular!) itnportant in an ethical cnterpn~e ~uch a~ cducc.ttional progran1n1tng. '1in1c series arc easily understood. cas) to follow and produce relevant inforn1ation (Can1pbclL 1976: Worthen & Sanders. 1987). I he design can be used in objective-based evaluation . In fact this p(nverful evaluation tool can analy7c the interaction between til11C and the introduction of U progran1 \Vhich is a further recon1111endation when evaluating 'Whether the initiation of a progran1 had an} cfTcct 011 exatn scores. The design can be used to separate intcncntion efTccts rron1 long tcnn trends already present (Glass. 1980). Intcrruptcd tin1c "eric" dcstgns arc bcttct than pre tc"t po~t-tc'-ll dc'-~tgns as the ~e ric~ of 111Ca~U fC ~ tcprc"Cnttllg the ~i tualion bciOI'C ptogran1 in1plcl11L'11tation fnrt1l ~l 12 control for the quasi-e\pcri rnent (Cilass. 1980~ Kratocln\ilL 1978~ McConnelL 1982). In C\ aluating the success or an; progran1 it 1s C\)<.;Cntial to detcnninc \\hat \\a<.; happcni ng before the progran1·~ introduction Cl) lcr. 1949) Rcpeatecl lnca"iurcs in a tin1e se ric~ allO\V the e\ aluator to JUdge \\ hethcr the in ten cntion changed the pattern. I' here i ~ therefore an extension into the pa"t and the Iuture be; ond that of the pre - te ~ t/ po s t-tcsl design. Cohort group~. as long as the} represent the san1c background (McConnelL I 982 ), can be tak.en as the san1e subJect ( Kratoclnvi I L 1978 ). T'hi s gt \ cs 1epcatcd rncasurcn1ent <.; under base ltne. be fore the I11 itiation 0 f the progratn, and 1nler\ cntion. after the I 11 itiation , condition~. Thc<.;e successt\ c potnts 111 tin1c arc unit replication"\\ ith the ~clect 1on critena the san1c for each successi\ e group. Th1s Increases the chance of sJnlilarit; betvveen the groups. The con1parability of rc"ults con1es fron1 the consistent application of 1ncasuren1ent standards (Lan1. 1995). 'I he British Colun1bia Ministry of I·ducation has not changed its adn1inistration tcchniq ucs O\ er the course or the ti n1c series. I\.., I an1 noted, thi s assun1ption that the trcattnent or data has been constant leads to htgh conl.)truct validity. Advantages of the Design ri he 1110St in1portant reqt11ren1ent is the fit of' this de<.;ign to the C\}1Crlll1CI1t~ll criterion: the C0111parison o(' the dependent vnriah}c be Core nnd after the tntroduct1011 of the trcatn1ent. or educat1onal prognun (KratoclnvilL 1978) 'I he purpose ol'thc e\ nluatton 11 is to shovv the n1ain effect of one\ ariable rather than the interaction of t\\O others (Can1pbell & ._ tanlcy, 1963). ~~h e te~t~ n1o~t ~uitah lc for n1ea~uring indt\ iclual differences \\ill difTercntiate \\hen e\er)one 1 ~ e\:po~ed to the san1 c trcattnent a~ i~ the case \Vith this progrmn (U iascr. 1961) Son1e of the n1ajor threat~ to \altdit) arc not a concern in tht~ ca~o.,e In tht~ n l paradign1, the periodic n1casuretnent~ lul\e undergone no change tn unit or acltn tnistratton and so the tnstn1n1cnts arc reliable (C,unpbciL 1976). ' l here is no threat ortesttng since a different group \\a~ tested each t1n1c. i\laturation i~ not a concern ~incc the students in\ oh eel \\ere all the s~unc age b; group. 'I he Inter\ cntion \\as Introduced randon1ly and not \\hen the trend of the ~cne~ \\as n~tng \\tthout any inter\cntion (Kratoch\vdL 197X). Cautions In using a tin1e series design, the evaluator is assun1ing that it is possible to evaluate a progratn using a san1plc of ~ituation~. It i~ 1111portant to a~k V\hether the evaluation device actually provides e\ tdcnce of the behav tor desired (I) lcr, 1949). It 11.., espcctally 1n1portant that the \ ariable~ be n1easured \ alidl; and rei iabl) ( Kratoch\\ II L 1978). The threat ofhi~tory i~ a prin1e concern. 'I he tunc \ariablc rcprc~o.,ent~ c.lll the other things that happened as titne pa~scd (Vcne;. 1993) 'I here ~~a ri~o.,k of ht~o.,torical confoundin g (KratochvviiL 1978). ()thcr e\ e nt ~ could c'-:plain an) etTech found (C'an1pbcll, 19 76~ Can1pbcll & Stanley, 1961: McConnelL 1982) Cook and Catnphcll ( 1979) suggested that the cvaluntor should C\:(1111 inc '"'hat ought to be a fleeted b\ an\ 14 progran1 inter\ ention and an) cotnponcnts that should not be affected. If ~upplcn1ental tneasures arc also affected. it i~ probabl) not the in ten ention that cau~cd the ob~cr\ cd change (Babbie & \\'agcnaar. I 989). l·\aluatton de~igns f~1 il \\hen the intervention i~ seen as the onl; e\ cnt happentng in the scric~ (Coo le) & I cinhardL l9HO) '[he obscn ational ~e ric "' ~hould be arranged ~o that the C) clc~ arc held constant (Catnpbell & Stanley. I961). 'I hi~ i~ a nccc~~ary co ndition ~incc Jt i~ in1portant to distinguish bct\vccn real effect~ and regular fluctuations in the scnes, between randon1 displacen1ents and dctcrnltntsttc one~ (Babbtc & Vvagenaar, 19H9: C1las~. 1980, 1997). frend s alread} present tn the data n1ay gi\ e the appearance of an ef feet vvhcn there 1~ none and ~o. ideally. the data ~hould be "'table before the tntroductton of an) Jntcrventton (Kratocb\\ i IL 1978 ). It is also incun1bent upon the progran1 e\ aluator to ~peci f) 111 advance the expected tin1e relationships bet\\Ccn the in ten ention and the rnani re~tation of the effect (Catnpbcll & tanlcy. 1963). 'I he evaluator needs to describe an} expectation~ about how the series wi II change and to keep these uppcnnost "'hen exan1 ining the serie<.:> a~ a protection against tnaking Type I errors (Glass, 1980). It is necessary to predict \vhether the effect be imn1ediatc or delayed, t11nc lagged (Can1pbelL 1976: Glass, I 997: Vcne), 1991 ). Students \Vho wrote the Math 12 provincial e'{an1inatron tn 1994 had not been exposed to the treatn1ent or the progran1 for as long as st udent ~ \vho \\rotc th~ exwn tnallon in 1996. Delayed effect~ arc tnorc dif'ficult to interptet and gt\ ~ nHHC roon1 ror alternative explanation<.; (Conk & CarnpbelL 1979). 35 The degree of stationarity in the series is another concern. 1\. stationar) series fluctuates tninitnall} bet\\ een t\\ o Iin1i Ling points. In other \\ ord~. there is no sy stcn1ali c increase or decrease in the le\ cl of the series as it dri ns up and dO\\ n. I )(nvevcr, tnost tin1c scricc;; produced by data collections fron1 the soc ial sc iences arc non-stationar}. 1\. non-stationary series fluctuates across the whole grid at n11H.lon1 (C'ook & C'an1pbclL 1979 ~ Glass. 1980). A stationar} sene~ 1~ the one to \\Ork with for detection purposes since an} effect caused by the tntcn cntton is easd) tdcntlfied (C) la~s. 1997). I lowcvcr a non-stationm-; ~c rie s ts n1ore con1n1on '' tth clas<, <, t/C gro ups. Adrninistering This Tin1e Series E" Progran1, the concerns and issues are centred on exc.uninati on tnean ~ and rmrti c1pation rates. 'I he evaluati on is a function of the stati sties e'pcci fi ed. The cun1 ul ati\ c) carl; report \\a~ u ~cd in ord er to accon1n1oclatc the di fTercnccs in cxan1 difficulty at each individual ses~io n and the difTcrences in the nun1ber of students wri ling at each session. 'I he yearly su1nn1ary also cq uali:ted the changes in tin1ctab li ng at the school because this elitninated the need to con1 pare results taken fron1 different exan1ination sessions. On a linear (year long) tin1etable all ~tude nt s wo uld have written the exan1ination in June. On a sen1ester (half year) tin1etable stud ents n1ay have written the exan1 at the end of scn1ester one. in Januar;. or at the end of setnester t\\ 0, in June depending on the placen1cnt of Mathe1natic<) 1:2 in the tin1ctable. Participation Rates The exan1 participation rate is calculated by the British Colun1b1a Ministr) or Education and is public;; hcd with cxan1 results. fh c participation rate is calculated b) dividin g the nun1bcr of untquc c'\an1 \vritcr~ by the sc h oo l ·~ Septctnber 10 Clradc 1:2 enro lln1ent fi gures (Mini~try or Fducati on, 1997a). Unique C\.Uil1 \\ ritcr~ I~ the designation given to the 11l1111bcr of incJi\ iuuaJ students \ VhO \\rOtC a prO\ tncial 19 exan1 ination during the selected 1ear. I·ach ~tudent is counted on I; once no n1atter hing the plausibiltt)' of ri\al hypotheses which n1ight account for obscr\ eel change~ in the variables prin1arily of interest in this study. Alpha level An alpha level of . 10 \vas chosen for all stati ~ tical testing. 'I his le,el i~ appropriate ~ince the study Vvas based on a stnall lllll11ber or data points. \\'hen 11 is ~tnall there is the possibi Iity or excessive Type fi error. failure to reject II" \\hen it i ~ 1~11 ~e 41 CI f/\Pl ER 4: RhS UL rs 'I his section sun1n1an;cs the stud; 's results. '] he n1atcri aJ is o rgan i ~:cd according to the research questions. Question 1: Was there a change in the NYSS n1ean score? Table I di splays the n1can scores on the Math 12 provincial cxwn ination for the NV students \Vho \Vrotc the exmnination and for al l students in the province. It also gives the I.-SCOre relattOl1Sl11p for these sets of ll1Ctll1S. I he NVS, Mathcn1atics progran1 \\US in1pl cn1entcd with ~ tudcnts taking Math 12 in the 1992-93 school )Car. Year 6 (*)is th erefore the ;car of tn1pJ cn1entation . Students Vvho Vvrotc the pro\ incial c~atni nation in Year 7 had recet\ cd t \\O ; ca r~ of treatn1 ent fron1 the program. Students in Ycar 10 had had fiye ) cars of treatn1ent. The Mini stry of Education a~serts th at ··provincial leve l exan1ination results reflect stability over tin1e" (Ministry of fducation. 1997b. p. I). HoVvcver. 1t ts clear fron1 the table below that the provincialtneans 1is ted do show variation fron1 ) ear to ; car. The lowest n1 ean score is 63.36 and the highest is 66.57. a range or ] .2 1. For thi s reason. the / -scores shown are used to provide a stanclard i?cd \Vay or relating NVSS scores to provincial scores. 7.-scores also acco unt ror variation in studcnl abil ity over the} cars co rrected for vari atio n with tn the level of difficulty or th e provincial c:-...anlllHltion and for sarnple s i ~e. 42 Table 1 Math 12 E'\an1ination Data Year' 987-88 988-89 989-90 990-91 991-92 ' 992-91 993-94 ' 994-95 995-96 : 996-97 NV~ ( I) (2) (1) (4) (5) (6*) (7 ~*) (8***) (9****) ( 10*****) Math 12 c\arn 111t1t ron Pn)\ mcial Mc1th 12 l.!\tll11111atron mean mean ';8 s1 ')9 ')9 )9 86 '-) ) 9) 64 6 ~ 66 57 61 J? ~ ') I ') II 63 q) 61 71 56 20 52 4) 51 69 64 92 6).20 64 02 6) 76 6) '72 66 3') 66 51 17 01 I 7 SO 18 )1 18 )4 17 86 16 20 10 25 21 41 14 18 00 13 6-l Provrncral Math I 2 c \ <.1111111 at 1on standt1r d de\ rat1on 16 16 I ') 61 16 91 Number of NV~<:; c;tudcnts takrng Math 12 IR II .__ /-score rc lntronshrp bcl\\CCn NY~~ and provrncral mcanc; -2 00 -2 09 -0 ~p -2 12 -0 46 -0 6) -2 16 - ~ SR -4 46 -0.32 1. The nun1bers in parentheses indicate the correspondi ng x value on all graphs. The * symbols indicate the nun1bcr of) cars or treatn1cnt to \\htch each group \\Ia~ exposed. The VS. n1cans suggest an increase tn scores in the } car~ 1987 - 88 ( Ycar I ) to 199 1-92 (Year 5), the }Car before the VSS prognun \\a~ introduced, although 1990-91 (Year 4) docs not foliO\\ this trend. Follo\ving the in1plen1entation of the NV';S progran1 in 1992-93 (Year 6), the n1cans scen1 to decrea~e until 1996-97 (Year 10) \Vhen there is another Increase. Years 8 and 9 sho\v the greatest enrolln1cnts and also the lo\vest 7"""' scores. The patte rn can be seen n1ore clear!; vvhcn th is data is graphed (see t igurc 1). 42 Table 1 Math 12 I· xan1 i nation Data Year 1 1987-88 1988-89 1989-90 1990-91 1991-92 1992-91 1993-94 1994-95 1995-96 1996-97 (I) (2) (3) NV. S Math 11 c\amtnttlton mean <\8 51 59 59 59 X6 ( 4) ')') l)') (5) ( 6 *) 61 4 ') 6171 5620 51 45 (7*") (8*,..*) (9**" *') (I 0* * *:t *) 52 69 6-4 92 Provmcml Math I I c \ tlll1111a llOil mean 6I6l 66 ') 7 61 1! 64 ~ ts 65 :?0 64 02 -65 76 65 7? 66 15 66 S I Numbct of Pro\ tn ctal Math 12 c~ammat10n NV<-\5 ~ tudcnls c;ttlndard de\ mtton taktng Mctth 12 - I'16 16 'X I " 61 16 91 15A I 17 .OJ 17.50 IX 51 IX5l 17 86 IX 00 ;-c;cot c rclattonsl11p bel\\ ccn NV<..;<; and ptOVIIlC ICIIllleans -'") 00 --16 ')'") -2.09 -0 X? -217 -0 16 -0 65 -2 16 -4 )8 -4 46 -0 17 20 20 ,_) 21 41 "'4 13 ~) 1. 'I he nun1bcrs in parentheses tnclicate the corresponding x value on all graphs. 'I he * S)'l11bOlS indicate the 11Ut11bCr Of} car~ of trcattnent to \\ htch ec.lCh g1Ollp \\US e'posed. 'I he VS, n1cans ~ u gge~ t an 1ncrea"c 1n "co re ~ in the) car" 1987- 88 (Y car 1) to 1991-92 (Y car 5 )~the ; ear before the NVSS progran1 \va~ introduced, although 1990-91 (Y car 4) does not foliO\\ thts trend. Folio\\ 1ng the in1plctncntation of the VSS prograrn in 1992-93 (Year 6). the n1eans ~een1 to decrease until 1996-97 (Year 10) \\hen there i~ another tncrea~c. Y cars 8 and 9 ~hO\\ the J.!,reate~t cnrolltnent~ and also the It)\vest /~ scores. r[ he pattern can be seen 1110rc c Iearl) \\hen thi~ data IS graphed (\CC Fi gurc I ). 41 c("j Q) ~ 65 63 61 59 57 55 53 51 49 47 45 1 2 3 4 5 6 7 8 9 10 Ycar b\"' '< \'aluc Figure 2. NVSS Math 12 provtncwl CXtlllltnatton n1can l.i shovving a ~ inglc trend . 1\s the cxan1ination i~ adn1 ini stercd at the provincial le\ eL the tncan<.; vvcre analy?cd in relation to the provincial lllCans. Fi gure 3 C0111pares the Jl1eans ror NVSS and provincial candidates. This graph indicates that the provtncial n1can i(.) tnuch Jnorc constant. The 1l uctuations in the n1cans frorn yea r to year arc not as great at the • provincial level as the; arc for the NVSS ~co re ~. rhis observation is to be expected since the provincial statistics arc based on a n1uch larger san1ple ( VS.._ n 240, Pro\ incial N = 143,315) and so are n1ore likely to he stable over titne. It can also be ~ecn that the NVSS n1eans rctnain belO\\ the provincwl average for all of the year~ ~ hO\\n . 45 70 65 .,..,.. , , ' ' ' , ...-- ----.......... -...- .,.- ,.,. - - - - -- - - - 60 - - - NVSS ____ Prov 55 50 45 N Ycar h\• x value Figure 3. Con1 pari son of N VS... and pnn incia1 1\rl ath 12 cxwn i nation rneans. J\lthough the Ministry of l·ducation ( 1995b) clain1s that the cxan1inations arc stable over tin1c, the actual prov incial CX(llnination changes fl·on1 session to session . The abilities of the various groups of studenb tested do not rernain constant either. ·r here fore the z-score relationship bet\\ccn the VSS and provincial scores \va~ cxan1ined. ,.1he/- • scores \Vere calculated using the provincial exan1inati on n1ean and standard deviation scores publi shed by the Ministry or Education in their report to schoo ls at the end or each acaden1ic year and NV. n values which \verc provided b) the school. The equation used was: X - Jl -- - - Fron1 Figure l, it can be seen that the N VSS n1ean is ah\ays hnver than the pro\ inc tal 1nean resulting in negative z.-scorcs throughout (sec Table 1) (haphing these scores shows again the rluctuations in the le\ el or the NVSS n1ec.ll1 scores over tin1c (sec Fi gure 4). The difTcrcncc bcl\vecn pre- and post 1991 /sco re ~ is 46 no t significant (t = -0.380, elf 8, p > • I 0). 'I hi s is the smnc resu lt as the one obtained by testing the actual n1ean scores (sec page 42). 0 1 2 -1 1'.) 1--. -2 0 u (/) I N -3 -4 -5 figure 4. The l.-score relationship bct\\cen n1 eans. VSS and pro\ incial Math 12 exan1ination In cotnparing the graphs of tncan exatnination ~cores and of /'-~cores, it can be • seen that an increase fro1n Ycar 2 to Ycar 3 is follo,vcd by a dip in Ycar 4. Both graphs then display a rise in Year 5 followed by consecutive declines until Year 8. Fron1 Year R to Year 9 there is a slight ri<:;c in both graphs. This i~ Collo\ved by a sharp rise fron1 Year 9 to Year 10. The only difference in the graphs occurs bct\vecn Ycars 1 and 2. '] he graph of n1ean cxatnination scores rises sli ghtly frotn Year 1 to Ycar 2. The /.-score graph shows a sli ght decline bet\veen these tvvo) cars. 1 he usc of both t-tcst~ and regression anal) sis on rav.. n1ean <:;core~ and on standardized z-sco res indicate no clifTerence pre- and post-in1pktncntation or the NVSS Mathcn1atics Progran1 . llo\vever thc~c result~ do not take account of the f~lct that othct variables tnay be influcnctn g the trend~ tndi cttted t)\ graphtng. \lterntlll\ c c'\pltlnation-... arc reported later in the chapter. 47 Question 2: Was there a change in the NVSS participation rate? Table 2 displays the pat1icipation rate for both NVSS and for the provincial situation. 1he data beg in~\\ ith the 1989-90 (Y car 3) acadcn1ic year as this \Vas the first year in \\hich the Jfini ~ tr) of J·ducation kept ',llCh record ~. J\n atten1pt \\HS Jllacle to calcu late participati on rate~ ror the }Cars pri or to this but the necessary inronnation co uld not be obtained fronl either the \11111 ~ tn or l·ducati on or fron1 the school. .; Table 2 I\1ath 12 Part icipation Rate ~ Year 1989-90 1990-91 1991-92 1992-93 1993-94 1994-95 1995-96 1996-97 "' 4 5 6(*) 7(**) 8(* * *) 9(****) 10(*****) _) NYSS patiicipati on rate 17 _qcyo 16. 1°o __ _ () ?"') .')0 10.4°o 9.5° 0 16.7°o 1 I .9° o 6.2°o Pro vincial patiici pati on rate 3 1.8° o 11.9% 14.4% "')_). "' 1o;; 34.6o/c) 36. 1° o 38.l 0 o 38.3°/o () The provincial participation rate is hi gher than that of NVSS for all the years exatnined. There is son1e lluctuation in the NVSS rate (see Ftgure 5) 'N ith the hi ghest rate occurring in 199 1-92 (Ycar 5) and the lov\ e"t in 1996-97 ( Y car 10). 'I he [1\ c1 age rate across the years listed is 13. 9o/o and the range is 17.2° o. I [o\vcver. the Iinc of regrc"sion (b - 1.43) shows a steady decline in the percentage of ~ tudcnts taking the Math 12 co urse. The dirfercncc bctvvcc n pre- and po..., t- 1993 pa rti ctpatton rate~ 1" not ~tg nilic ~.lnt ( r' = .41 , d r 6. p I0). 48 25 ,.--..., ~ ....._... 20 0 c: ..... ..... cj 15 .....u0.. 10 ·~ cj c... 5 0 1 2 3 5 4 6 7 8 9 10 Year by x \aluc Figure 5. NY ~ participation rate ror Math 12 provincial cxan1ination. The provincial rate is 1norc steady (sec l·igurc 6) Vvith a high in 1996-97 (Year I 0) and a lo\v in 1989-90 (Year 3). The ~l\crage rate aero~~ the years listed i~ 34.8°o vvith a range of 6.5%. While the NYSS participation rate is clearly dec lining, there would secn1 to be a steady increase in the provincial level of participntion. 'I he line of regression (h 0. 99) disp lays this increase. 40 ,.--..., 38 0 0 ....._... 36 c: 34 0 ....... 32 ..... cj 30 0.. .....u 28 ..... 26 ~ 0... 24 22 20 1 2 3 4 5 6 7 8 9 10 Ycnr by ' \ c.ll uc _Fi gure 6. Prov incial participation rate ro r Ma th 12 pn)\ Jncial CXH11111Hll1011. 49 Statistical testing indicates that there is no difference 111 the participation rates preand post-in1ple1nentation of the NYS ivlathen1atics progran1. In light of the increase in provincial participation rates. the decrease in the NVSS rate seen1s quite serious. llowever other results ha\e not) et been e'\an1ined and so there 1nay he alternative explanations for these results. ()ther I '\planation~ It is possible that the YSS i\lathen1atic~ Progran1 had ~on1c effect on the type of student who enrolled in Math 12. In order to dctcnninc vvhethcr the characteristics of the students taking Math 12 had changed as a result or the change in the way in which the course was delivered. other relationships n1ust he considered. The relationship between pmiicipation rate and exan11nation n1ean score \va~ exmnined. J\n analysis was also n1ade • of the relationship bet vveen n1ean provincial cxan1 inatJon score and the n1can GP J\ ~co re for the s tudent~ taking Math 12. In order to establish v~hether an) differences in NVSS Math 12 provincial examination results were a functi on of the treattncnt con1parisons \vere also n1ade for Chen1istry 12 and Physics 12 results for the san1c years. ' fhe~e cour~cs \Vcre ~c lec tecl since they attract si1nilar students and no special treatn1ents were used in curriculllln delivery during the san1e period. Ir the NVSS Mathetnatics Progn1n1 had been responsible for any significant eli f fercnccs in Math 12 re s ult~. there should he no indication or a change OVCI ttn1e in the n1enns ol Chen11Sll") 12 or Ph~ SlCS 12 NVSS is one or three high "chool s in School I)tstnct ~6 ( cch,lko) rl he t'ther" arc Fort St. .h.uncs Secondar} School (I S.JSS) ,lJH.l Fra"c1 I akc l · knll~nt,ln Sccondttt \ 50 Schoo I (I· I E~ S). In order to anal) /e the O\ crall chan ge 1n tn can exan1 ination scores at VS . , the exatnination scores for J\ 1ath 12 fron1 th ese other tVvo ~c h oob were obtained in order to detern1 1ne i r there 1~ a d 1 ~ trict \\ 1de pattern to th e exan11nati on rcsul ts. Participation Rate and tv1can Provinciall-:xatni natt on Score Fi gurc 7 display s the rclatJonsh 1p bet \\Cen partie 1pati on ra tc and n1 can exmninati on score for the Nv.~s iv1ath 12 pnn incw l c'\an1 inati on rc<..; u}t<..; 'Jhe correlati on across Year 3, Year 4. and Year ~ I<> po~1ti'c ( r before introduc ti on of the 0 94. dr = I. p I 0). 'I his ~ u ggests that VSS Mathctnatic~ progran1. an increa~c in the participation rate \\as reflected b\ an increase in the 1ncan exan11natton c;,co re. ()n the other hand, the ~ correlatio n acros<..; the treattncnt ) car~. Y car 6 to Ycar 10. <..;hen\<., a ~i gnilican t i11\ er~c reIations hip ( r = -0 .R1. d r _ "'.,. p .1 0). Th1~ ..,uggcst~ that arter the introductt on of the NVSS Mathen1<1tic A scores 2. R6 2. 80 3.'8 3.21 .., 4"" _). l 1987-88 19R8-89 1989-90 1990-91 1991-92 1992-91 1993-94 1994-95 1995-96 1996-97 -J.., ) 4 5 6(*) 7(**) 8(***) 9(****).· I 0(*****) ) 3.1 1 2. 85 3.0 I 3.22 3.40 Figure 9 di splays the CiPA data in graph fonn. 'Jhis graph appears to roughl y follo\v the pattern of the NV S tncan cxan1ination sco res with co rresponding dips in • Ycars 4 and 7 and corresponding highs in Ycars 5 and l 0 . Q) l-; 0 u (/) ~ 0.... 0 36 34 32 3 28 26 24 22 2 1 2 3 4 6 5 7 8 9 10 Yca r hv " value ~ Figure 9. Mean GPA sco res ofNVSS " tudcnt~ tnk.tng f\Jlath 12 'I he sitni lari ty bet\\vCcn the pattern o l tncan e\.atn ination scores c.UH.l (i P \ scot c . . can be seen tnorc clca rl; 111 Figutc 10 \\here both ~c t s or dattl c.lrC '·dl0\\11 54 (l) L., 0 u Cl) c: ro <1.) ~ 66 64 36 34 62 60 58 56 32 <1.) 1-< 3 0 u 28 Cl) Mean 26 ~ 24 0 54 52 50 CIP J\ 2.2 2 1 2 3 4 5 6 7 8 9 10 Ycc.u h; \. \'alue I· igurc I 0. C'on1panson or tncan C\.C.U11111atton ~co re~ and tn can (jJ> 1\ scores. I· igurc 11 di~pla) ~the relatton ~ htp het\\een CiP \ ~co re ~ and the cxan1ination n1ean scores. Linear regrc~sion aero~~ all data potnh (h VSS pnn incial 8.29) indicate~ that as the CrP J\ ~co re in c rea~c~ the n1can c:\an1ination ~co re also incrca~cs. 'I he correlation bet\\ccn CiP 1\ ~co re <:> and VSS tnean ~co re ~ i<) not significant (r 2 = 0 2 1, • df : 8, p ..> . l 0). Jt i ~ also noted that there I ~ no clear pattern forn1cd h) the trcattncnt years. I Iowcvec Ycar L Ycar 2 and Ycar 7 all had G P1\ sc ore ~ less than 3.0 ( B) but the correlation between CiPA ~co re s and NV S tnean 5Corcs for thc~c) ca r~ 1 ~ not ~ 1 gnificant (r"' = .39. elf = 1. p . I 0). On the other hand. Year<:> J . ·-L 5. 6. 9 and 10 al l had GP \ scores greater than 1.0 (8) and the co rrelati on bet\-veen GP/\ scores and NVSS n1can scores for these years is stgnt fi eant ( r2 = . 83. d r 4, p . I 0). 'I ht ~ \\OUid ~ee n1 to 1nd iCc.l te that the second group is 11Ulclc up of co horh \\ Jth larger 1Hl111bet'-' of able ~ tudcnt s. 55 v1-< 0 u (/) c: C'd v ....~ 66 64 62 60 58 56 54 52 50 - Yea r 10 - Year ':; - Yca t 6 - Year l - Ycay2 - ca r I - Yca1 7 - 'r car4 - ~ <.:,11 8 2.5 2.7 29 - 't etlr9 31 3.3 35 CiPJ\ ~co re Figure 11. I he re1attonshtp bet vvccn (j P1\ ~co re and NVSS Math 12 provincial cxatnination n1ean'i. Figure 12 displays the relation ~ hip between C1PJ\ scores and the NVSS patiicipation rate. Linear regression (b 5.66) indicate ~ that as the GP 1\ ~co re increases the participation rate also increases. I Jo\\e\ er. the correl ati on bct\veen C.JP J\ scores and the NVSS participation rate is not significant (r) 0.04, elf 8, p .;> 10). 25 35 3 (lJ 20 .._., 25 c: ~ 0 u (/) 2 < 0... 1.5 v 0 15 ...... ........ r"j c.. 10 ·u ...... • Rate GPA t 1 C\l 5 0... 05 0 0 1 2 3 4 5 6 7 8 9 10 Ycar b't- x \ aluc Figure 12. rl he relationship bcl\vee n C1PJ\ ')CO te and the NVSS ~L.lth 12 participation rate. 56 Con1pnrison of Exan1ination Data !\cross School Courses Mean ~ cores ext_ Math 12 n1ean ~co res \\Cre con1pared with the exan1i nation scores for Chctnistr) 12 and Ph) ~ics 12 'I able 5 ~ hO\v S the VSS tncan C'{cunination scores for Chcn1i stry 12. Math 12 and Ph; sics 12. Figure 11 rc pre~c nts this data graphical ly. Table 5 NVSS Mean Exan1ination Scores for Chen1istr; 12, Math 12 and Ph y~ics 12 Year 1987-88 1988-89 1989-90 1990-91 199 1-92 1992-93 1993-94 1994-95 1995-96 1996-97 Chetn istn"' 12 62.54 56.04 56.4 1 54. 12 60.37 59.75 53.09 57.52 72.00 71.20 (1) (2) (3) (4) (5) (6*) (7**) (8***) (9****) ( 10*****) 75 70 OJ 1-; 0 u C/) c ("j Q) 65 "" " ' 60 I \ \ \ 55 ... . . . -.. • . . . . . . . Chems try 12 \ Math 12 (' \ I , ,• \ ( • • '·, 1.' I I \ ' I \ \ '\ \ I \ \ \ ,.l\ \ .. I· I \ ~ ?-. " I \ ........... \ • • Physics 12 67.08 70.08 72.00 59.90 74.()7 66.)3 61.90 58.50 7"'_) .....I"') 61.00 Math 12 58.53 59.59 59.86 55.95 () l A·5 61.73 56.20 52.45 52.69 64.92 ~ I•, . • - - - - AlySICS 12 ' I· \ • •' 50 0 .- Ycar b) '< value .I·i.g_ure 13. CotnpatJ SOI1 or NVSS Chetnistt \ II ' t\ lath I I c.llld Ph: SIC'-' l I ~'\"ln1inatton n1cans. 57 ·I he patterns tn aJl three ca<.;cs follo\\ each other. 1\ high in Y ca r 3 in all subjects is follovvcd b) a dip in Year 4 \\hich i ~ then follo\\Cd b; another high in YearS. J\11 three l incs continue dO\\ n\vard in Y car 6 and Ycar 7. I here \\ otlld seen1 to be a cohort effect. '" I he d i fTc renee bel\\ cen Chctn i5lt') 12 Jl1ean ",COrC'-' pre- and PO'-'t-introduction 0 r the NVSS Mathctnntic ~ Progran1 1 ~ not signifi cant (t = 0 2R9. elf 8. p . I 0) rhc di fTcrcncc bet \veen Ph) ~ics 12 tnean sco res pre- and post-introduction of the NVSS Mathcn1atics Progran1 1s also not ~ • gn i fie ant ( t = -0.25 5, d r 8. p > • I 0 ). ·r he resu Its indicate that there arc no "'ign1 ficant c ha n ge~ tn the~e tncan~ ove r t1n1e a~ there \Vas no stgnlf1cant change in the \.lath 12 n1ean O\C r tin1e Linear re grc~s ton (b 1.16) on the Chen11~tr; 12 tncan ~ "'uggcsts an Increase in Chen11str; 12 n1cans O\er tin1e (sec Figure 14). 'I he tncrea~e tn not ~ i g nificant hovvever • , (r = 0.28. df = 8, p .> . l 0) 75 70 65 60 55 50 45 ~ 0 N ..-- Ycar b) \. \ aluc Figure 14. Chetni stry I '2 pto vtncwl C' • 10). 75 70 Q) ~ 0 u V) c ro llJ 65 60 ......... ?. 55 50 0 ...- Ycar by ' value Figure 15. Physics 12 pro\ incial exarnination n1canl-i. In con1paring the three subjects. it ~een1~ that\\ hile Chen1istr) rncan e\:an1ination scores have shoV\In a slight overall increase \\ ith tin1c. the n1ean exan1ination scores for both Physics 12 and Math 12 have decreased. None of the trends. hovvcvcr, differ significantly fron1 zero (no change). Participation Rate i\s for exan1ination tneans. participation rates ror the three subject~ \\Crc al\.)o con1parcd . I able 6 displa) s the partictpatton rate for Chcn1tslt) 12. f\ lath l' and Ph) ~•c~ 12. I· igure 16 displa) s this ~an1c data graph1<:all). 59 '1able 6 Year 1989-90 1990-91 1991-92 1992-93 1993-94 1994-95 1995-96 1996-97 Chen1 is try 12 14.0°/o 20.2° 0 ,0 -o 0 -' . ) J 4 5 6(*) 7(**) 8( ** *) 9(****) I 0( * * * * *) Math 12 17 4°fo I6. I o/o 21.2°o I 0.4% 14.0% 9 5° 0 7. 7cVo 5 C)O 0 7. Iex) C) ')0 0 16.7°/o 1 I .9o/o (?o/c ),_ 0 Physics 12 5. 8° 0 R.9o/o 15.9° () 5.9° 0 4.5° 0 1.8° 0 5.5° 0 2.4° {) 35 30 ,........_ 0 0 25 • '--"' c: 20 ('j 15 0__, ....... , \ \ , • ·-·-u 10 ~ 5 \ · · · · · · - Chem1stry 12 . • 0.. • ., ·' . / ---Math 12 ' / ----PhySICS 12 . . __ '' __ - . -. ..... • ........ 0... 0 1 2 3 4 5 -- ..,- 6 7 8 Yca r bv '< 'alue 9 10 ~ Figure 16. NVSS participati on rate~ for Chcn11~try 12. 'v1ath 12 and Ph ys ic ~ 12. /\gain, the patterns 111 all three subj ects ro li O\\ each other. A I O\\ 111 y ec.lr .f I\ followed by a hi gh in Ycar 5 foll o\\ed b) d dip 111 Yeat 6 The pattern contin ue') downward in y ca r 7 but the rvtath 12 rate goes again~t the trend in y c,lr 8 ......... ng \\ htlc the ratec.; for Chen1 istr) 12 and Ph) c.;tc~ 17 fa ll. Linear regresc.; ton on the Chetn isl t) 12 pat ttcipation rate ( b overall decrease in the perce ntage ol students tak. tng Chctn istn 12 O\er tin1e ("'\Ce 1 t ~urc 17). Tht s dee rcase ts signtlt cdnt ( r , = 0.4(>, df 6, p . I0). 60 35 ,..-...... 30 '..........., cf?. 25 c 20 ·-...... 15 0 ("j 0.. ...... ·-t (.) 10 ~ 5 0... 0 1 2 3 4 5 6 7 10 9 8 Year b\- \. \aluc participation rate'-, for Chctnistr) 12. Linear rcgres'-,ion on the Ph] ~ic~ 12 participation rate (h -0.93) '-,ttggc~t '-, a decrease in the percentage or student~ taking Physics 12 over the years also (sec Figure 18). This dccrca~c i\ not ~tgnilicant (r 2 = 0.30. JJ 6, p .10). 9 10 16 14 ,..-...... 0 0 ..........., c0 ·-...... ("j 0.. ...... (.) ...... ~ 0... 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 Y car b; \. \ <.lluc Fi gure 18. NVSS participation rdtc~ for Ph] \tC" 12 In all th ree courses. Math 12. Chcn1 tStt) 12 <.HH.I Ph]stcs I'· the pttrttctpation rate htt'-, decl ined 0\Ct' tin1c fv1ath 12 (h - l..f 1. base = 17.4'X,) droppL·d appt0\.1111<.\lL'h ) () 61 percent O\ er the data se ries. Both Chen1IStt') 12 (b (b -2.29, base = 14o/o) and Ph ysics 12 5.R 0 o) dropped approx1n1ately 16 percent. -0.93, base It is \vorth; of note that in Year R. the participation rate for Chen1i~tr} 12 is higher than ror the other l\\ 0 ~ubjcch. I or the ~al11C ) car. the Chcn1i~ tl) 12 cxa n1lnation 111ean sco re is clo~cr to the other l\\o than at c.ll1) other tin1e It \\Ou ld see n1 that the relat ionship displayed In Figure 7. c.h participation tdte tncrea<;,e~ tnean c\.an1 1nat ton sco re decrca~cs, i ~ lath 12 ("ce Figure X) and Cherni~tr} 12 (~ec Figure 19). true for both 80 70 60 1J $-., 50 u0 40 c: (\j 30 1J 20 ~ 10 0 35 30 ~ 0 ...._,. 25 c: 20 0 0 Cl) --......co 15 0.. (.) ...... 10 ro 5 ~ 0... 0 3 4 5 6 7 8 9 Parllci pation • Mean 10 Year by '< val uc Figure 19. Con1pari~on or patiicipation rate and tnean ~core fo r Chen1istr; 1I Figure 19 shcnv~ that rronl Year 7 to Year 9. the ll1ean ~core for Chen1l'-,tl) 1') I'-, inc rca~ing \\bile the participation rate ts dccreac;;tng In Year 10. the pattern hc.l" tC\Ct"ed. the n1can ~co re t c.., n10\ tng do\vl1\\ard \\htlc the parttctpc.ltton lc.lte •" tHO\ ing UJ1\\drd 62 Con1pari so n With Other School s ln The San1c , chool Di strict Jable 7 displays the Math 12 n1can c\.an11nation scores for three schools. NVSS, F ~ JSS and FLESS. \\ hich arc al l part of School Distr ict 56 (Ncc hako). ri gurc 20 displays the san1c data graph tcall). Table 7 Math 12 Exmninatt on Mean Scores for NVSS, 1 S.JSS and FLr ~ss 4 vss rvtath 12 Year 1987-88 1988-89 1989-90 1990-9 1 199 1-92 1992-93 1993-94 1994-95 1995-96 1996-97 I 58.53 59.59 59.86 55.95 63.45 6 1.73 56.20 52.45 52.69 64.92 ') ._, 3 4 5 6(*) 7(**) 8(* * *).· 9(****) 10(*****) 80 ~ 0 u (/) ~ co (1) ~ ~ FLFSS Math 12 68.00 64. 16 66.33 53.23 44.46 47 .26 56.07 51.5 8 52.72 62.25 • 75 v I· SJSS tv1ath 12 61.55 70.89 64.88 55.72 63.72 64.00 78.93 59.22 64.70 59.46 • • • • 70 • 65 ' . ..... ,_ _.__, \ • • • \ • ' ' 60 55 50 , \. \. \ .......... 45 - , / '- . • •• NVSS rvlath 12 - .. . _. _. FSJSS rvlath 12 ____ FLESS rvlath 12 40 ~ N ~ lO • 10). 70 65 1J l-o 0 60 u r./l c cj 1J ~. 55 50 45 40 ..- N v 1.() tO ,..... 0 ..- Yettt h\• '- \~.lluc Figure 22. FLFSS Math 12 pto\ JIH.:w l C'\(llninatJ on 1ncans 64 ·rhe situation appears to be the san1e at al l three schools. In all cases the trend over tin1c is a decrease in Math 12 pro' incial exan1ination scores. In con1paring the three schools, it would scen1 that the overall decrease in scores is slightl y less at NVSS (b -0.17) than it \\as at the other t\vo ~choolc:; (rSJSS b = -0.21. FLE S b - 1.12). Table 8 clispla) s the participation rates for the three schoob. NVSS. FSJSS and FLESS . Figure 21 rcprcsenh th1 ~ data graph1call ). Table 8 Math 12 Participatio n Rates for NVSS, FSJSS and FLLSS. Year 1989-90 1990-9 1 1991-92 1992-93 1993-94 1994-95 1995-96 1996-97 NV SS Participati on Rate 17 .4o/o 16.1o/o 13.1° 0 10.4o/o 9.5% 16.7° 0 1 1. 9° 0 6.2o/o 3 4 5 6(*) 7(* *) 8(* * *) 9(****) 10(**** *) I· SJSS Participation Rate 34.6% 27.3% 19.0% 11 .4% 12.9o/o 13.3% 9.3° 0 9.1% 45 40 ,...--.. 35 0 0 ....__, 30 c: 25 0 ........ +-' 20 C"V 0.. 15 . (.) ....... 10 t ~ p... 5 0 I I h ._ S Part1c1pation Rate 26.7°/o 35. 1% 40.7o/o 22.4% 12.7% 17. 9° 0 15.3o/o 18.4o/o - - NVSS -- -- ·- - FSJSS - - - - FLESS • ---1 1 2 3 4 5 6 7 Year b\ x \ !llue 8 9 10 l· igurc 21. Partici pat1on rate 1n 1\tlath I ! fo1 NVSS. FSJSS and FI I 'SS 65 Linear regression (b -3 .18) on the J·SJSS participation rate data suggests a dran1atic decrease O\ er titnc (sec Fi gure 24 ). Tht~ decrease is significant (r·., = 0.80. df 6, p ~ .05). 40 ,...-.. 35 0 0.....__, 30 c 25 0 ·+-' C\l 20 c.. .....(.) ..... 15 t('j 10 0.... 5 0 1 2 3 4 5 6 7 8 9 10 Ycar b; x 'al uc Figure 24. F. JSS participati on rate for Math 12. Linear regression (b = -2.80) on th e FI I·SS participation rate data al~o ~ u ggests <.l dran1atic decrease over tin1e (see Figure 25) 'I hi ~ decrease is signi fi cant (r = 0.48. df - 6,p < .l0). 66 .,....-.... 0 0 ............ c: 0 ·~ +-' ro 0.. ·~ (.) ·~ ~ P-c 45 40 35 30 25 20 15 10 5 0 • 1 2 ~ 3 4 5 6 7 8 9 10 Year by' value Pigure 25. FLf: SS participation rate ror Math 12. In all three ~chools, therefore, the part icipation rate he1s declined over tin1c. Again, the overall decrease in the participation rate for NVSS (h - -1.4)) is lc~~ than that of the other two schools (FSJ. s b -3.38. I· LESS b -2.80). I lO\\CVCL the decline or approxi1nately one point per year in the Math 12 participntion rate at NVS con1pared with three points per year decline at the other school s 111USt be regarded in light of the lower initial values for NVSS. NVSS (b - -1.43. bnse = 17.4%) dropped an a\erage or eight percent. Both FSJSS (b = -3 .38. base 34.6o/o) and FLESS (b -2.80, base = 26.7%) dropped an average of 10 percent. 1 he results arc sun1n1art zed 111 tabular fonn tn Chuptcr 5. 'I hercfnrc the) ll rc not presented in sumn1ary forn1 here 67 CllJ\P'J l·R 5: Sllf\1MJ\RY J\ D ('() CI USIONS 'fhe purpose of' this chapter i ~ to prO\ 1de a fon1111 in \vhich the study' s findings can be interpreted and placed in the context or the hypotheses original ly stated . It is also the ~ec tion ''here the 1111plication~ and lln1itatton~ of the ~tud; ca n be e\.mnined critically. Sun11narv . or the Stud\. Thi~ ~tud; C\aluatcd the VSS f\ l athe1natic~ Progran1. I he progran1 \\H~ developed b) the 1nathen1atic~ teacher<.; at NVSS in re~pon~e to the belief that the curricuhun. a~ pub!J'-.hed b) the Brttl~h Collllnhia Iv1inl~tr; or l·clucation, did not n1CCt the needs of the local ~i tuation. In parttcular the ~VSS n1ath teacher~ \van ted to increase the 11lll11ber or students stud} ing higher lc\ el n1athen1atic<.; and to in1prove their rnath ~k ill s<.;() that they \vould do \veil at th1~ higher le\cl. The NV. S \llathen1allc~ Prog1an1 \\a~ adopted 111 the Spring of 1993 and wa<.; u~ed \Vi th all 111ath c l as<.;e~ in the ~chool fron1 the Fall () r 1991 on. Th I ~ Inc an~ that ~tudent~ who took Math 12 in the 1991-94 school year \vere e'\poscd to the NVSS Mathernatics progratn in both Math 11 and \!lath 12 cour~e~ \\he rea~ ~tuJenh \\ho con1pleted l\ lath 12 in the 1996-97 school ) ear \\Crc C\.pO~cd to the NVS') \1athenult1CS Progran1 for all or thci r high sc hool 1nath courses 'Jhe efTect1veness of the progra111 'v\ a~ e\ aluated In tcn11~ 0 r the \\ U) 111 \\ hlC h 1t tnct the goa ls or the tcachet"' \Vho de\ L'loped it. DH.l thi~ progran1 en~lhk. ~l nd l'\ en encourage. n1ot e students to tttke f\ 1ttth 12'' Did the tnean '.COt L' on thL· pt O\ tnct~tl \ ttth 12 exaxninat1on tnlplo vc as a result ol the 1111pkn1entlltion ol the progr4un ·.> 68 ·I o evaluate a progran1 of thi s type. tin1e series anal ys is was dctern1in cd to be the best n1ctbod. Titne series anal) sis i~ used fo r intac t progran1s \\h ere archi val evidence is cn ailable and u ~efu l. In th i ~ ca~e the progratn \\ a~ alreacl; in usc \\hen evaluati on \va~ fi r~ t co nduc ted and archi\ al C\ tde nce. pn)\ inc tal e'\a nl i nati on n1ean scores, and partieipati on rat e~. fro tn both ~c h on l records and r¥1in is try of I:d ucati on pu hii cati ons. was available. In tin1 c seri es anal) \,)JS. the progra n1 e\ nluato r cha rts da ta points to look ror trends and changes to the~c trends 'I he uni t of ana l )~l~ \\a~ fVlath 12 cla~~c~ lor the )Car~ 1987-88 (Yea r 1) to 199697 (Y car l 0). !\ lthough C.\ din ercnt \.)Ct 0 I ~tuden t~ C0111prised each or t hc~e classe\.), they \\ ere representative cohort group~ and ~o could be u~ccl in a lt tne ~eric\,) anal) <:> i ~. f he e l assc~ \\ere ~ itnilar dcn1ographical l) C0111ing fron1 the ~a lllC location and heing e'po~ed to sin1ilar cd ucattonal experiences. C ' onc l u\,)ion~ As there are no sitnple trend') in the data. the t\\O 1nain h) pothe~es arc fir~t cxan1incd. This is fo ll C)\Ved by inte rpretation or the n~~u l ts of testi ng the alternate hypotheses. Onl y the n i~ a conc lus1on nhout the NVSS Mu thcn1attcs progntnl reached . Fxan1 ination tnean~. participation rates and G P '\~co re~ al l fluctUtlle \\ tth titnc and c.lppe~.lr to tntcract in con1ple' ways. '1he rcsulh are ~un1n1arued in .. I c.tble 9. 69 Table 9 Sun1n1ary of Res ults 1l'r Qothcsi~ Result Tentati\ e Conclusion - Main I J) potheses 1. achic\ en1cnt -· participation I t = -0.18~ ns~ b -0.17 t -0. 15: n ~. b -1.41 I here is no sign ificant (u I 0 ) d i fTc rc ncc in uch tevcn1ent bet ween the pre- and post-in tervention While there is no dilTerence In participation between the pre- and post-in terven ti on. there is an in1portant do\\11\\ard trend in NVSS Math 12 participation _\1 ternate I I) pothcse~ Provincial Trends • 1. ac hicvetnent tz 0.38: ns 'Jhere is no sign iJicant (u. .10) difference I11 achic\ en1ent bet ween the pre- and post-intervention \\hen pro\ i11cial trends arc taken into account I ._. participation hJil \ :..: () 99 'I he do\\n\\ard trend 111 NVSS Math 12 participation i~ 111 contrll~t to a provinctalup\\ard trend Achicven1cnt vs. Part icipatton Rate Interaction r2 = .004: n~. b -0.0~ l ncn:ll~ing the pool or stLH.k11ts \\ ho coinpletcd ~ l ath 12 did not lo\\CI O\ era II nellie\ en1cnt 70 Interacti on \\ ith J\b iIity as ~1 easu rcd b; CJ P!\ ( ~cl r selection - cohort effects) I . achicveincnt r 2 = .2 I : ns (u. ~ . I 0) b 2. participati on 8.29 scores and tncan exan1 c..;core!-!. r 2 = .04: n~~ b 'I here is a posi ti vc, d ircct relationship bet vvcen G PJ\ 5.66 'I here is a positi\ e, direct relat ionship between CiP/\ c..;cores and the participation rate. J\b ilit) a~ iv1 ea~u rcd b; ~\'SS C h cn11~t r) and P h }~ I C!-! (',clfc..;clectJon- cohort effects) 1. achic\ cn1ent tchcm = 0.29: n~: h 111= 1.16: t ph)s- -0.26~ ns: bph)S = -0.62 While Chen1 istr; cxan1 scores increased sligh tl y O\ er ti n1c. the n1can score c..; fo r Math and Ph ysics dec!incd. ~--------------------------------+-~--------------------------------------------+--------------------------------------·-- 2. parti cipation r)chcm = 46: 'I he decline acroc..;', tin1c in l') I rn = - -"') ·')- <) • partici pat1 on occurred in all three ~uhject area',. I he dec] ine in Math 12 participation \vas slo\\Cd b; the in teraction. l .zphys -- · '1) 0 ·' 11 •s '· bphs= -0.93 Cotnpari son ofNVSS Math 12 \\ith I·SJSS and FLESS Math 12 1. a c h ic \ e111en t r' 1 S JSS = .0}: 11~: h1 , 1ss= -0.21: r\ 1 1 .18: ns: \s hI I I S\ = -1 • 11.... 2. pat1icipation In al l 1 schoo l', there •~ an overall dec ]inc in achic\ en1ent. I he decltnc 1~ less at NVSS than at the other ~choob. •2 I I s JSS - <) oo. • In al l 1 schools there is an - '1 '10. O\ erall decline in l1 1 JSS - - ) · ) <"' · .. ~I I I SS = ·4 8' blli ·SS - - 2 8(} part JctpatJon l he decltnc t" less ttl N\ SS than at the other "chonl" 71 Mean Scores 'I here arc no ~ignlficant eli !Terence ~ in the n1 ean<., before or after the tntrocluction of the NV, s rvlathetnatics Progran1. I lcn\e\er there i" an O\erall d0\\11\Yard tendency to the n1 eans across ti tnc. I· '\(.ll11 inati on or th e L-"cores \\ h ich cotn pare schoo l and provincial exan1ination tnean ~ con finn the"e nuctuation" and the lack of "igni flcant d ifTercnces. Participation Rate There arc no "igni fic ant ddTerence~ in the participation rates before or after the introduction of the NVSS 1v1athenu1tic" Program. I here is an overall c.lownvvard tendency to the participation rate aero~~ tnne. It was discovered th at the relation~hip" bet\\lecn parttcipation rate and achievcrnent i ~ an 111\ crse one. /\~the participation rate increa~cd, the n1ean score decreased. I 10\\C\ er thi~ decrease \\aS not ~tatt~ticall] ~ign tficanl. This i ~ or intcre~t <.,incc the increase 111 the pool of <.,tudent" t<1king f\1ath 12 \\oulcl ha\ e hcen C<.ltt<.,cd h) the ~elf <.,C lcction of students \\ hol)e rnath <.,kIll" had pre\ lOllS I) kept thcn1 out () r the cour~c Ir it is true that ~tudcnh v. ith \\eak.er 1nath ~kilb \\ere encoura~ed to take f\..lath 1' bec(.1u~c '--' of the way in which the curriculurn had been deli\ cn:d to then1 in pre\ IOU" tnath cour"e". then it is natural that the n1ean sco res vvoulcl be hnver because these students \\Ould not do as wel l on the final exan11nat ton a~ ~t ud ent~ \\ tlh ~tronger 1nath "k tlls (j P1\ Scores ·r he (; p !\ ~co tes to a huge degt ee Io llo\\ ed the patll'tll or the rnean L''\atn inat ion ~co re~ 1here \Vcte co JJ c~potH.Iing highs in both nlettns ~u1d Clccti ng to ltlh.C l\ ltlth I_. 75 Conclusion While these concluc..,ions suggest that the in1plcn1cntation of the progran1 \vas good for the NVSS situation the) n1ust be seen in tern1<., o r the ori ginal hy pothe se~ defined in Chapter l . Consequently the !ir~ t h; pothcsis, th at the n1ean c..,corcs on the Math 12 provincial e~an1inat1on alter in1plen1entat1on \vould increase 111 con1parison to the n1ean scores before itnplernentatron, 1s reJected. Furtherrnorc, the second hypothesis, that the participation rate 1n \1 ath I 2 alter itnplctnen tat ton \\ ould increase 1n con1pa n so n to the participation rate hclorc Hnplenlcntation, ,..., alc..,o tcj ectcd. \I though the tnain h; pothec..,e<., 111 their on gtnal forn1 arc rc.JcctecL th e inlet vcn llon \\aS ~ UCC ess ful for a lllllllber of rec.lc..,onc.., ~~ he tn lroductJOll o r the progran1 brought about a decrease in the rate of decltnc for both achtc\enl ent and parttctpation. 'I here \\a<., an • increase in the pool of students taking Math 12 and thi~ increac..,c did not affect achicvetncnt. Math 12 re~ ult s at NVSS arc in better ~hapc th an resu lts in Chen1istr; 12 or Physics 12 and also better th an Math 12 result~ at the other secondary schools in the distri ct. For these reasons, it \Votll d appear that the chan ge tn curri cul un1 dcli\ery and th e in1plcnlcntation or th e NVSS MathenlattCS Progranl \\tlS a successful \ cnt ure for the <;chool and its student<., Li 111 i lations It ~ hould he ren1cn1 be red that the N\'~S l\ lathetnattc" Prngran1 \\ ll" de\ t: lopL'd b~ teac h e r~ on s ite \Vit h particular g(ldh Ill tnind and lot ll\L' 111 unL' pdt ticulat "chool. IlK' ~c hool 'v\as not \L'kc tcd al ra ndonl but tt \\d\ not "ekctL'd hL'L.ntse it \\a\ 1-.no\\11 tn lK' dirlerent c1thc1 'T'he school is a typ1cal \L'condtt t\ o..;clH)OI in the ptov1nce of British 76 Coltunbia. It offers the courses offered at other ~eco ndary schools and it has a n1ix of students like those at an) other ~chool. The generali;ations rn ade arc reasonable \Vithin the lin1itations expressed. The progran1 \\as rntroducecl to "tudents a~ 1t \\a ~ being de, eloped by the teaching stafr. I he rnathen1atics teachers did not choose a particular point in th e tirnc sc ric ~ at vvhich to in1plcrn cnt the progn1n1. In f~tcL the point of In1plcn1entatron was at a tin1e when the trend in the tirn e se ne ~ ror rncan C'\ain111att on sco re~ and for participation rate \Vas dO\\n\\ <.u d. \\'i thout the outlying data points ol Yeai '), thi~ do\vn\\ard trend 'V..oulcl have been e'en n1orc pronounced fhcre is a diluted treatn1ent effect due to the lag tin1c \\ith in the progran1 . 'I he inter\ ention did not happen O\ erni ght. Student~ had to \\ ork their \\H) through the progran1 at the eli ftc rent lc\ els and then ~elect to take Math 12 f'ron1 a \\ide variet) of course offerings. '1he c\ aluation conducted in this ~ tud) focu~ed on only l \\O di~cretc aspec ts of the itnplcn1entat ron of the progran1 and did not anafy /e the effect of the progran1 in other areas. l·or exan1pl e. the study did not cxan1rnc the nun1her of st ud ents who took Math 1 1 hut opted not to take i'vlath 12. Deficiencies '1he t JJne se n cs c'\a n1 i ned (trc ~ho rt . 'I he longer senes co nttt in on!\ len d,lta points: fi ve points on etther ~ td e of the trcatll1L'I1t. '(he ltl11L' ~Cite~ li~L'd rot p,ll ttctptttion rate anal y~ t ~ ate e\ en shorter. It \\ ,\s not po~~ ~ bk to obt,l in patltu pal ion r,ltL'~ l()t '\ L'ars I and 2 since th e nccessarv data \Vas not availttbk. ·1hL· u...,c nfannualtL'~ults as a unit nf "' 77 anal; sis resulting in onl) I 0 data point ~ n1eans that ~ tati s tica l tcst5 arc extrctne ly prone to Type II error. A short tin1c ~e ncs docs pr<)\ ide a better picture than a Sltnrlc pre- and po ~ t- te~ t situalion (Cook & C'an1 pbe I L 1979). I IO\\ C\ cr. the l a~t data point used in this study is also the flrsl data p0111l to represent a group o r students \VhO have taken aJ J of their 111ath courses using the VSS ~lathcn1at1c~ Prognun . 'I ht~ IS th e group or students who have been in1n1cr~cd in the ain1~ and procedut e~ o l th e progn.1n1 throughout high ~c hool. Indi' 1dual rc~ult~ \\Cre not available for iv1ath 12 ~t udcnts. 'I hcrclore it \Vas not pos~tblc to 1natch the ~tLH. knt<, \Vith cat Iter ~tandard i /cd tc~t 1 c~ulh ~uch as the Canadian rest 0 r Ba ~ ic Sk.i Ib ((' I B s) This l11Cc.l l1~ that It \\a<, not po~~l hle to H':>Certai n \\I hcthcr trends in the data \\Cre a runction or indi\ ldual group~ In particular. and noted ~C\e ra l • tin1es. the rc ~ ults acrose;; the boa rd 111 Yca1 5 arc c\:ccptional With C I BS ~co re~ 1t \\Ottld have hccn possible to detcnninc \v hethcr this group of ~o.,t ud e nt ~ differed ~•gn ifi can tl; fron1 the other groups. S in1ilarly. in Yca r 4 al I of the scri c~ plotted sho\v a lo'v point and C T8S sco res could have hccn used to dctcrn1ine the nature or this group or ~ tu dc nt s. ~I he Evaluation Model I his stud y ctn ployed one tnodcl or progt an1 C\ aluat ion. an ob,ccll \ c based one. Thi ~ was selected because o ltts lit \\ith both tnathcn1at1cs Clllltcul tt gcncldlly and \\tlh the way 111 \vh tch the NYSS M(tt hetnatics Prognun in p~u tH... ttl~ll \\d~ de\ eloped This 1110<.k I \vas a Iso able to provt de the i 11 ronnat ion that the ~takcho ldL' I" ll10Sl \\ antL'd. 78 1lo'Acvcr_ b} using a different tnodel of curriculun1 developn1ent and evaluation. a di rrerent picture of the progranl and 1ts iinplenlentation tni ght have enlerged. I r the progran1 had been de\eloped rronl the Situational Orientation, the evaluation tni ght have focu ssed on process and context and could ha\e exan1incd the vicvvs of the stud ents in1n1crscd in the progran1 . If th e progran1 had been developed frotn the Critical Re nccti\ c OrientatiOn, the C\ nlualion tnigh t ha\ e focussed on what chan ges need to be tnade to the progran1 and couJJ June exc.un incd alternate tneth ods of curriculun1 de li very. Reco n1n1 endations for Practi ce and Research J\ ") the progran1 C\ aluat1 on \\a~ done in re~pon~c to question~ po~cd \vi thin the cotnn1unit}. it is itnportant to point out that there i<; a dange r in interpreting the effecti\ eness or the • VSS f\ Iathen1atics Progran1 fron1 too narro'" a stance. ·r he ana l ys 1 ~ tnade in this stud; shO\\ ~ that n1erel; Jocu'-IJng on a Ii 111 ited " ic"" or one or t \\ o aspec ts of the progratn \\ ithout evaluating the conte~t and all the other factors 'Ahich arrect the success of progran1 in1plctnentation \\oulcl lead n1ost obscn ers to ~ugge~t that the progran1 should not be used further. llus ~tud} ~upports the staten1ent that th1~ is not the case. At wo rst. the NVSS Mathen1attcs progran1 perlonned nodi fTercntl} than the ot her n1ath progran1s or the sc tencc course~. In fact. it would seen1 that the decl1ne in Math 12, Chctni stJ } 12 and Ph ystc"i 1'"' w ithin the sc hool and in Math 12 across the disltJct \\'arrants di rect inter\'cntton ThL' NVSS Mathctnati cs progn.tnl n1ny have helped to siO\\ the decline (lJH.I "iO tt could be co ncluded th at further e\.a tn inntton o l the \\i.l) 111 \\hic h the"ie suh1ech dtL' tau12,ht is a u ~c rut e'\c rctsc in itnpn)\ tng teac hing pracltce and slulknt results. 79 I· urther research is needed. ()ne particular a' enue that ~ hould be follo\vecl i ~ the n1atching or individual student results to scores on pre\ ious a~"essn1ent tools. [he correlation between C I'BS scores and Math 12 statistics would be \vorthy or study. MacMillan (in progress) has begun exan1tnin g the relati onship between C'I BS scores and pro\ inc tal exan1tnat1on n1ark. ~. 'I he NVSS \ l nthen1atiC~ J>rogran1 I ~ ~ td I in ll~C 'r he 111at hetnatics teachers continue to reline and rC\\ork the defin1t1on~ \\ 1th1n the progran1 and the \vay in \Vhtch tnaterial is pre~entcd to 5tudcnts. 'I he) ha\ e patd attention to I; ler' " ad\ icc that tn an) progran1 there "hould be an .. ongoing C'\atnination of' ObJCCti\ C". COlli "e n1atcrials. learning experience~ and "tudent outcon1c" "o that both the cour"ie and ~t udcnt learntng can be 1n1pnn ed" ( I949. p. 5 ). • RO References An1encan I·\ aluat1on Association. ( 1996 ). I he progran1 C\ al uation standards. Kalan1azoo. MI: author. Aokt, T T. ( 1985) 'I O\\ard curn cu lun1 llK]Utr) in a nevv ke). ()ccasional paper No.2 (Revised). Departn1ent of Secondar) l:ducation. University orAlberta. Babbie, I ·. & 'A' agennar. 'I .C. ( 19R9) Practic 1ng soc tal research (5 th eel.). Beltnont. C/\ : Wads\vorth. Berk. R. \ & Ro~~~. P.I I. ( 1990). 'Think ing about progran1 e\ aluation. Ne\\bllt) Park. CA: Sage. Bertrand. J T. Stl\ cr. J. <.~ Porter. R ( 1989). f'v1 c th oclo l ogic~ for evaluating th e in1pact orcontracepli\C ~Octal n1arket tng pto g ran1~. l·\alualt011 rcviCvv, }] (4), 123- 154 Bcs\\ick. R. ( 1990). L\ a lu ati n ~ educational prograrn~ II :RIC Di gest Series urn bcr l·l\ 54] . l·ugcne. () R: I nc Clean nghou~e on I ducat1onal Ivlanagc n1 cn t !Producer and Distnbutorl . Bruckerhof'L C & BruckcrhofT. T. ( 1996 ). ·1he Connecticut M u~cun1 Collaborative for ~cience ed ucation: 1995 - 1996 annual report. I FRJC Di gc~ t ED40 11 19] . \\' a~h i ngton. DC : F Rl C C learn ingho u ~c on J\s~es<.,tncnt and Evaluation I Producer and D i<.,L ri butor 1. • Catnpbcll. D .T. (I 976). /\~scss in g the in1pact or planned social change [ERI C F 0 3035 12]. Occa~ional paper scri c<.,. #8. \\'a<.,hington. DC· I· RIC database I Producer and distri hutor ]. Can1pbeJl. D.T. & Stanle). J.C. (1961). l:\.peri n1ental and qua ~ i-c\.pcr11ncntal designs for research on teaching. Bo<.,ton. I Iough ton i'v1ifflin. Conrad. C I·. & Wi I ~o n . R.W. ( 19X6 ). /\cadc1ni c progran1 rc\ ic\vS [I· RI C Digc" t l· 0 284522 1 Washington. DC. r RIC ( Jearn 1nghou~c on I Itghcr Ed uc,tt 1on I Pt oduccr c.t nd Di stributor] . Cook. T D <-X Can1pbcll. D 1'. ( 197<)) ()uao;;i- I \.pcntncntt.ltion: Destgn and a nal ysi~ ~ ~~ uc ~ l(l!:. fiel d ~etttng~. Bo<.,ton: I Ioughton fVlt f1l1n. Cooley. W W & I ctnhardL (i (I <>XO) '"I he t11 "\ltucltonnl dinlCil'-.1011" -.,tudy. l·ducational I·v aluallon and Po ltD \nal; "•~\..2..: ( 1). 7 '') Cro ~hic . .1. ( 1991) Interru pted tin1c ~cncs anttl) ~~~ '' tlh bncl single "llhiL'Ct data. Journal of Consulting and ( ltntcal Ps\cholog\. 6 1 (6). 966 -974. Dalla, L ( I <)74) '"I he 1111pacl or the 'A CSll n ghou~c ()hto C\ a luttl l011 011 the de\cloptnent of Pt o1cct I lead ~t,ut \n c\.,ltlliiWtton ol the tnlnH:~dttllL' tliH.llonnL'r-tcnn eff ec t ~ and ho\\ they ca n1c about In ( ( \ ht (Ed.).l!Jc L'\tdttation h ttH?.ton. DC: I' R I(' Database I Prod ucet and D 1'->tributor ). Iledri ck. T .F & ShtpnHltL S I . ( 1988) \1ulttple quc'-.ltons tequtrc tnultipk dcstgns: An evalualton of th e 19X I chan ges to the \l ·DC ptognun. l \ '"tluatton rc\ tC\\. I :2 (4 ), 427 - 44X X2 Kratoch\vi ll. T.R. (1978). Foundations oftin1e-serics research. fn T .R. Kratochvvill (Eel.). Single subject re~earch: Strategies for e\aluating change (pp. I - 94). Nevv York : 1\cadetnic. l atn. T .C.M. ( 1995). lndi' icluali1ation or ohjecti\c~ : !\ tnodcl of educati onal progran1 e\ aluatio n. E\aluation Practice, 16 ( I). 11 - 27. La1arus. M. ( 1982). Eva luating educati onal progran1s. J\rl ington. V!\: /\n1c ri ca n Association of Schoo l J\dn11ni strator". Lee. V.E .. Croningcr. R.G. & Sn1ith. J.B. ( 1997). Course-taking, equity, and 1nathctnat ics learning: 1 e"t 1ng the con'-ltra1 ned curnc ulun1 h) pothesi" in U.S. ~ccondar} ~c h oo l s. l·du ca tion al L\aluat1on and Pol1c\ \nah ~ i ~, 19 (2) 99- I 2 1. Iv1acMillan. P. (1998) 'I he 'alid1t\ of the C I BS in prediction of Grade 12 course ~elec ti on and Grade 12 tnark ~ f\ hm u"cn pt 111 preparation. l · n1\ ersity or Northern British Colun1bia. Madaus. Ci 1 . & Stul"lle bean1. D . ( l9X9 ) I ducationa l C\aluat1on. Cla"">lcal \Votk" or Ralph W. 1\ !cr. B o~ton: KIU\\er J\cadcln ic. ~1au ser. G . '\. & I lo ln1cs. R A (1992). An C\<.duation of the 1977 Canadian firearn1s legisla ti on. E\a luatio n rc\ ie\v. I 6 (6). 601- 617. McConne II. B .8 . ( 1982 ). I \ aluat1 ng bihngual educat1on u"1 ng a ti n1 c "erie~ design. In G. r ore hand (Ed.). C\\ direc tion ~ for progran1 C\ al uation . Application~ o I tin1c series analysis to eval uati on. no. 16 (pp. 19 -12) San I·rancisccr .l ossey-Bass. Mci\1il1an. J.I I. & Schun1achcr. S. ( 1997 ). Rc ~carc h in cducat1on. J\ co nceptual introduction. New York.: Longn1an . Miller. R.L . ( 1976). lncli \ iduali1ed in<:,truct1on in tnathctnatt c~ Arc\ IC\\ of research. Mathetnatics Teacher, f\la y 1976. Reston. VA: NC I M. Pal; s, T . ( 1992). Research d cci~ ion s: Quantitati\ c and qualitat i\ c per~pect l\ c" I oronto, ON : I Iarcourt Brace Jo\ a no\ tch . Patterso n, B.J J.. Kc ~-.; Jer. I (j . \\'ax. ) . Bet n-.; te1n. \ . Light. L.. ~lidthunc. D '\ . Portnoy, B. , fenncy, J. & luck.e11nant). h. ( 1992) I '~.1luat1on or ,1 su pcnnarkct interven tion : 'I he NC I-Ci l(ll1t l·ood cat for health s lLt<.l ) 1·\ al.!_!ltlt on re\le\\ .. J() ("'~). 464 - 490 . Pophan1, W..J. ( l9X8). l·duca ti onq_! C\ ,tluat lon (2nd cd.). Engk\\ood Clin~. '\I Prenti ce llall. Pt ov tnce ol Bnti sh Coltunhia. i\l1ni!'lt\ nf l ducc.ttion. (J<)RX) ( iradc 7 - 12. V tel on a, BC ': /\ ulhor l\ttti1L'l11~lltc..., J>rov1ncc o l Bt1t ish ( 'o llltnbia f\ l1nistry ofEd uc"'~d) J)t...,l tt Ll 111fO_rnHll1011 J2!0fiJe fo t ~L h oo i~'~ll· J <)<)4 f <)')·..:..:_Colll11H1111Cllllll_g dCC0111pltshn1L'Ilh nf Briti sh Colwnbi 't students". Dt ~ lti c t )6 (NcLhako) cd ttton \ 1ctotta. BC: <.lut hor. R3 Province or British Colun1bi a, Mini stry o r Edu cation. ( 1995 h). Provincial exatninati ons: Cauti onar) notes on interpretati on or results. Vi ctori a. BC: Author. Prov ince or Briti sh Co lun1bi a. Mini stry of Educati on. ( 1996). Mathen1atics 8 to I 0: Integrated reso urce package 1996. Victori a. BC: author. Province of Briti sh Co h.tn1bi a, Mi nistry or I·ducati on. ( 1997a). Report nun1hcr 502 1L : I li slo ri cal distrib utio n of lette r g rade~ for ~e le c te d pu blic schoo ls. Victori a. BC· author. Province of British Co lun1bia. Mi ni '-, tr; or J·ducation. ( 1997h). Standard report 1274P. Victori a. BC. author. Provus, M. ( 1973 ). Fval uati on or ongo ing progran1s in the publ ic sc hool sys tc1n In B.R. vVo rthc n & J.R. Sander'-, (I· d~. ). t~ ducationa l e\ al uat io n: 'I he or; and prac tice ( pp 170 - 207). \\' orthi ngton. 0 I I: Charles ,\ J o n e~. Sa n der~. J. R. ( 1994 ). 'I he progran1 e\ aluation ~tanda rd~: I lo\\ to as~es~ e\ a l uat1 on~ of educational prou.ran1'-, (2nd ed .). I housand Oak'-,, CA. Sage. Sawyer. D. ( 19R7). Anal y;ing pscudo-ti n1c se ric~ i ntervcn ti ons. Evaluati on revie\\ . II (2). 178 - 196 • Schn1id t. W. IT. & McKnighL C.C. ( 1995). Survey ing educational opportunity in tn athen1 atics and ~cicnce· J\n international pe rspccti\ e Educational I \ aluat ion and Policy Analysis. 17 (3) 337- 353 Scri\ en. M. ( 1991 ). I Iard-\\O n le~~o n s in progran1 e\ aluation San 1· ranc isco : J ossev.. B a~s . Stni th. M. L. & Glas~. G.V. ( 1987) Research and e\c.duation in education and the social sc iences. I.. n~le\\Ood Cliff~. NJ: Prentice-! Jail. .... Sn1i th, N. I . & J lauer. D.M. ( 19<-JO). l·\al uation retlect1on: I he app l1cabilit) ol ~e l ec ted eva!uati on tnode I'-, to e' oh 1ng in\ c~tigati' e dc~igns Studtc'-, 1n F duc~.ltJonc.ll !:va luation, 16. 4X9 - 500. nr 'I yle r. L.L & Kle in. M. l·. (I 974 ). Fvaluation \\ itht n the con te\.l CUliiCUllllll dcvcloptncn t and 111'-,truct tonal n1atc ttc.tls. In ( i D Bo11ch (Ed.). l '~.d Uilt 1ng cd uc~.lttonill nrogran1s and prod ucts pp 120- 119 T) ler. R ~ ( 1949) Ba'-,JC princtplc\ or curricu lunl tll1d 111~ltl1Cl11.)11 ( htc~.lgo: Universit) ofChtcago. Va nderhoo I Challl her () f Conl nlcrcc. ( 199()) \' c.UH.I ct hoo r \ diH.krhon r. B( au thor. Veney . .I .E. ( l <)<)l) Evaltwtion Dpp ll cat l Oll~ or rcgrcssinn an.lh "1"' tinlc "('Ill'S data. !.. va luation Pntcticc, 14 (3). 259- 274. 84 Wang, M.C. & Walbert II.J. (1983). Fvaluating educational progratns: J\n integrative ca usal-tnodeling approach. Eclucationall ~val uat io n and Pol icy Analysis, 5 (l) 347- 366. Westbury. I. & l Isu, C S. ( 1996). Structures orcurricu lun1 governance and classroon1 practice in n1athen1 ati cs. Ed ucati onal Evaluation and Policy Analys is, 18 (2) 123 - 13 9. White. P.J\., Gan1oran. /\. .. Stnithson. J. & Porter, /\.C. ( 1996). Upgradi ng the hi gh sc hool n1ath curricul utn : Math cou rse-takin g pattern s in seven hi gh schools in California and e\\ York. Educational l·\aluation and Polic) Ana lysis, 18 (4) 285-307. Wolf. R.M. ( 1974). Data analysi~ and reporting considerati ons in evaluation. In \\' .J. Pophan1 (Ed.), E' aluation in educatton: Current appli cations (pp. 203 -242). Berke ley, C/\: McCutchan. Worthen. B. R. ( 199 5 ). Son1e obse rvations about the instituti onalization or e\ aluation. I·' aluati on Prac ti ce, 16 ( 1), 29 - 36. Worthen, B.R.. Borg. \V.R. & White. K.R. ( 1993). Mea~urernent and e\aluation in school s. New York: 1.ongn1 an. Worthen. B.R .. & Sanders, J.R. ( 1987) Educational eval uati on: Alternative approaches and practical guidd ines Ne\\ York.- Longn1an Worthen, B.R. & Sanders, J. R. ( 199 1). The changing 1~lce of educati onal evaluation. rheory into Practice. 30 ( 1), 1 - 12. P 0 Box 129 Vanderhoof, 8 C VOJ 3AO Servtng Fort Fraser Fort St. James Fraser Lake Vanderhoof Telephone (604) 567-2284 Fax (604) 567-4639 June 7, 1996 Ms Lynn Maksymchak P 0 . Box 1433 Vanderhoof, B.C. VOJ 3AO • Dear Lynn· • I am wnting to extend support for your research work towards your Masters program at Un1vers1ty of Northern British Columbia. I understand your research enta1ls us1ng arch1val test data on Mathematics 12, Physics 12 and Chemistry 12 provincial exam1nat1on results and on the students who have taken Mathematics 12 smce 1988 Anonymity of students and score results are essent1al 1n mamtammg the confidentiality of the students I look forward to rece1v1ng a copy of your thes1s find1ngs Best of luck 1n your stud1es Yours truly, Lou1se Burgart Supenntendent of Schools LB/cp •