WASHBACK AND STAKEHOLDER INFLENCE ON CONTEMPORARY
COMMUNICATIVE LANGUAGE TEACHING:
IMPLICATIONS FOR CURRICULUM DEVELOPMENT
by
Steven Knowles
B.A., University of Auckland, 1999
M.A.A.L., University of New England, 2005

THESIS SUMITTED IN PARTIAL FULFILLMENT OF
THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF EDUCATION
IN
MULTIDICIPLINARY LEADERSHIP

THE UNIVERSITY OF NORTHERN BRITISH COLOMBIA
April, 2018
© Steven Knowles, 2018

Abstract
This research study explored the influence of summative assessment and stakeholders on
contemporary Communicative Language Teaching (CLT) in high-stakes educational settings.
The participants comprised of second language instructors that were instructing university
preparatory English as a Second Language (ESL) courses. Each participant had advanced
qualifications, had extensive teaching experience, and self-identified as a contemporary CLT
instructor. The study utilized a phenomenological methodology to examine the participants lived
experience of their respective working environments. A fuller understanding of the desirable and
undesirable effects of summative assessment and stakeholder influence on contemporary CLT
practice emerged from the study. The study concludes by synthesising the findings with current
research to reveal practical knowledge for those involved in proficiency based second language
program design to consider.

Acknowledgements
I would like to I would first like to thank my thesis supervisor Dr. Whalen from the
department of Education at the University of Northern British Colombia. Dr. Whalen was always
contactable whenever I ran into a trouble spot or had a question about my research or writing.
She consistently allowed this paper to be my own work, but steered me in the right the direction
whenever she thought I needed it.
I would also like to extend my sincere gratitude to the five English language instructor
participants who made this research study possible. Each participant was open and forthright as
they articulated the influences that came to bear on their instruction.
Finally, I must express my very profound gratitude to my family who provided me with
unfailing support and continuous encouragement throughout my years of study and through the
process of researching and writing this thesis. This accomplishment would not have been
possible without them. Thank you.

Table of Contents
Abstract .......................................................................................................................................... ii
Acknowledgements ...................................................................................................................... iii
Table of Contents ......................................................................................................................... iv
List of Tables ............................................................................................................................... vii
Associated Definitions and Reference Terms .......................................................................... viii
Chapter One .................................................................................................................................. 1
1.1 Statement of the Problem ................................................................................................... 3
1.2 Significance of the Research .............................................................................................. 3
1.3 Purpose of the Study and Research Question .................................................................. 3
1.3.1 Research questions. ...................................................................................................... 4
1.4 Background of the Study .................................................................................................... 4
1.5 Researcher Context (Personal Location) .......................................................................... 6
1.6 Conceptual Lens (Theoretical Orientation)...................................................................... 7
1.7 Limitations and Challenges ................................................................................................ 8
1.8 Delimitations ........................................................................................................................ 9
1.9 Thesis Overview .................................................................................................................. 9
Chapter Two: Review of the Literature.................................................................................... 11
2.1 Section 1: History, Theoretical Underpinnings and Application of SLA Principles .. 11
2.1.1 Historical path of second language acquisition. ...................................................... 12
2.1.2 Current assumptions about the nature of second language acquisition. .............. 13
2.1.3 Second language acquisition theory to communicative language teaching
principles.............................................................................................................................. 13
2.1.4 Rate of language acquisition. .................................................................................... 14
2.1.5 Cultural factors affecting student academic learning. ........................................... 15
2.1.6 Professionalism in communicative language teaching. .......................................... 15
2.1.7 Transformative learning. .......................................................................................... 17
2.1.8 Outcomes based education. ....................................................................................... 19
2.2 Section 2: Communicative Language Assessment Strategies ....................................... 19
2.2.1 Standardized assessment strategies. ......................................................................... 20
2.2.2 Alternative assessment. ............................................................................................. 21
2.3 Section 3: Assessment Washback .................................................................................... 21
2.3.1 Alignment. .................................................................................................................. 21
2.3.2 Motivation and performance. ................................................................................... 22
2.3.2.1 Heuristic / algorithmic tasks. ............................................................................. 22
2.3.2.2 Anxiety: Debilitating and facilitating. ............................................................... 22
2.4 Section 4: Review of the Research of ESL Washback on Teachers and Students ...... 23
2.4.1 Lack of studies controlling for instructional professionalism in communicative
language teaching. ............................................................................................................... 23
2.4.2. Washback on teaching and learning curricula. ..................................................... 24
2.4.3. Washback on teaching strategies, methods and approaches. ............................... 26
2.4.4 Washback on classroom assessment. ....................................................................... 27
2.4.5 Washback on student learning. ................................................................................ 27
2.4.6 Washback on the mental state of instructors (attitudes, and feelings). ................ 29
2.4.7 Washback on attitudes and feelings of learners. ..................................................... 29

2.4.8 Washback on student motivation toward exams. ................................................... 30
2.5 Section 5: The Fourth Way: The Inspiring Future for Educational Change.............. 31
2.6 Summary of the Literature Reviewed ............................................................................. 32
Chapter Three: Research Methodology.................................................................................... 33
3.1 Qualitative Research ......................................................................................................... 33
3.2 Interpretive Description Methodology ............................................................................ 34
3.3 Ethical Considerations...................................................................................................... 35
3.3.1 Confidentiality and anonymity. ................................................................................ 35
3.3.2 Vulnerable populations. ............................................................................................ 36
3.4 Research Procedures ........................................................................................................ 36
3.4.1 Recruitment of participants. ..................................................................................... 36
3.4.1.2 Voluntary participation. ..................................................................................... 37
3.4.1.3. Recruitment process. ......................................................................................... 37
3.4.1.4 Consent................................................................................................................. 37
3.4.2 Data collection process. ............................................................................................. 37
3.4.2.1 Data collection and storage. ............................................................................... 38
3.5 Data Analysis ..................................................................................................................... 38
3.5.1 Triangulation. ............................................................................................................. 38
3.5.2 Components of data analysis. ................................................................................... 39
3.5.3 Data analysis process. ................................................................................................ 39
3.6 Research Credibility ......................................................................................................... 40
3.6.1 Epistemological integrity. .......................................................................................... 40
3.6.2 Representative credibility. ........................................................................................ 40
3.6.3 Analytic logic. ............................................................................................................. 40
3.6.4 Interpretive authority. ............................................................................................... 41
3.7 Summary ............................................................................................................................ 41
Chapter Four: Data, Themes and Discussion........................................................................... 43
4.1 Professional Context of Research Participants .............................................................. 43
4.1.1 Instructional background of participants................................................................ 44
4.1.2 General interpretation of the principles governing communicative language
teaching. ............................................................................................................................... 45
4.1.3 Learning context. ....................................................................................................... 46
4.1.4 Learners’ background context. ................................................................................ 47
4.1.5 Assessment structure utilized at place of employment. .......................................... 48
4.2 The Experience of Dealing with Assessment and Stakeholder Influence .................... 49
4.2.1 Experience of assessment alignment. ....................................................................... 50
4.2.2 Experience of student influence on instruction. ...................................................... 51
4.2.3 Experience of academic course instructor influence. ............................................. 53
4.2.4 Experience of institutional/administrative influence. ............................................. 54
4.2.5 Experience of alternative assessment washback. .................................................... 56
4.2.6 Experience of assessment washback. ....................................................................... 57
4.2.6 Overall effects of influences on instruction. ............................................................ 57
4.2.7 Instructional desires. ................................................................................................. 58
4.3 Conclusion ......................................................................................................................... 59
Chapter Five: Implications and Recommendations ................................................................ 61

5.1 Study Implications ............................................................................................................ 61
5.1.1 Standardized assessment is difficult to implement successfully. ........................... 61
5.1.2 Incorporate alternative assessment with caution. ................................................... 62
5.1.3 Direct washback on instructional approach is negligible....................................... 62
5.1.4 Stakeholders’ influence impedes successful implementation of communicative
language teaching. ............................................................................................................... 63
5.2 Recommendations ............................................................................................................. 64
5.3 Future Research ................................................................................................................ 66
5.4 Conclusion ......................................................................................................................... 67
References .................................................................................................................................... 68
Appendix A: Summary of Research on Washback ................................................................. 79
Appendix B: Guiding Questions ................................................................................................ 86
Appendix C: Participant Recruitment Email Brochure ......................................................... 87
Appendix D: Information Letter / Participant Consent Form ............................................... 89
Appendix E: Transcriber Confidentiality and Non-Disclosure Agreement .......................... 94
Appendix F: Codebook - Jeffery ............................................................................................... 96
Appendix G: Codebook - Gary ................................................................................................ 108
Appendix H: Codebook - Jaden............................................................................................... 120
Appendix I: Codebook - Tim ................................................................................................... 132
Appendix J: Codebook - David ................................................................................................ 143
Appendix K: Interview Transcript: Jeffery November 02, 2017 ......................................... 149
Appendix L: Interview Transcript: Gary November 23, 2017 ............................................. 169
Appendix M: Interview Transcript: Jaden November 29, 2017........................................... 192
Appendix N: Interview Transcript: Tim December 05, 2017 ............................................... 206
Appendix O: Interview Transcript: David Dec 08, 2017....................................................... 218

List of Tables
Table 1 The Roles of a Teacher: A Summary............................................................................... 17
Table 2 Instructional Background of the Research Participants ................................................... 44
Table 3 Research Concerning Washback on the Teaching and Learning Curriculum ................. 79
Table 4 Research Concerning Washback on Teaching Strategies, Methods and Approaches ..... 80
Table 5 Research Concerning Washback on Classroom Assessment .......................................... 81
Table 6 Research Concerning Washback on Student Learning .................................................... 82
Table 7 Research Concerning Washback on the Mental State of Instructors ............................... 83
Table 8 Research Concerning Washback on Attitudes and Feelings of Learners ........................ 84
Table 9 Research Concerning Washback on Student Motivation toward Exams ........................ 85

Associated Definitions and Reference Terms
Assessment: Assessment is a process which endeavors to continually monitor learner progress
and provide feedback to support learning (Sousa, 2015).
Assessment Alignment: Refers to the alignment of learning activities and assessment tasks. In
an aligned system the learner is said to be 'trapped' without an avenue to succeed without
learning what he or she is intended to learn (Biggs, 2003).
Communicative Language Teaching (CLT): CLT is the term used in the second language
teaching field to denote the teaching of a second language for the purpose of communicating in
that language (Lee & VanPatten, 2003).
High-stakes Assessment: Refers to the status of the assessment. Assessments on which
participants’ careers or study plans hangs are most likely to heavily influence teaching and
learning behaviors (Alderson & Wall, 1993).
Language Acquisition: The development of the linguistic system in the mind/brain, this is
accepted as a largely unconscious process (VanPatten, 2003).
Language Learning: Signifies the process of consciously learning aspects of language, usually
associated with traditional language classrooms (Krashen, 1982).
Negative Washback: Undesirable influence of assessment on classroom practice (Alderson &
Wall, 1993).
Positive Washback: Desirable influence of assessment on classroom practice (Alderson & Wall,
1993).
Reliability: Refers to the degree to which assessment produces stable and consistent results on
repeated trials (Carmines & Zellar, 1979).

Testing: Testing is utilized to measure what a learner knows at a given moment. Testing is a
singular event. Testing differs from assessment in its purpose. (Sousa, 2015).
Summative Assessment: Refers to assessment with a primary goal of assessing quality (Taras,
2005).
Validity: Refers to how well assessment measures what it is purported to measure (Carmines &
Zellar, 1979).
Washback: Denotes the influence of assessments on classroom practice (Alderson & Wall,
1993).

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Chapter One
The immense desire for an opportunity to have access to an English education has
reverberated throughout the world. Inadvertently, this trend has caused preparatory language
departments at universities to become the major gatekeepers of who gets admission into
university (Ryan, 2010). It is vital that language departments ensure instructors have the ability
to enable students to efficiently gain the English language skills they need to support their future
studies with consideration of diverse cultural learners’ needs, desires, and expectations.
Second language instruction has emerged through the growth of understanding of second
language acquisition and general education learning theory (VanPatten, 2003). The relatively
new field of Second Language Acquisition (SLA) can be traced back to the early 1960s when
Noam Chomsky’s theories on first language acquisition (1965 on) were applied to ideas about
second language learning (VanPatten, 2003). The role of the teacher in the field of general
education, and language education has been undergoing a major shift away from the traditional
and behaviorist view of teachers as passive technicians. Contemporary education conceptualizes
teachers as reflective practitioners who facilitate knowledge through an integrated approach, and
teachers as transformative intellectuals who embrace the sociopolitical role of language
(Kumaravadivelu, 2003). The shift can be exemplified through the introduction of Core
Competencies in the British Colombia’s K-12 curriculum. Core competencies consist of
intellectual, personal, social and emotional skills used engage in deep and life-long learning.
Teachers utilize classroom activities that promote students use of thinking, collaboration and
communication to solve problems, address issues, and make decisions (British Colombia
Misinstry of Education, 2017). Ontario’s K-12 curriculum assessment and instruction guidelines
evidence the transition towards teaching professionalism as the curriculum endorses individual

1

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
and collective ownership of learning, achievement, and student well-being (Ontario Ministry of
Education, 2013).
Language institutions require assessment strategies that address the incongruent nature of
utilizing traditional standardized testing to assess the process-based learning in contemporary
second language teaching. Researchers have called for standardized assessment that seeks to
gauge communicative proficiency rather than linguistic knowledge (Bachman & Palmer, 1996)
and process-orientated assessment strategies, such as portfolios, presentations and written
products to address this incongruence (Coombe, Davidson, O'Sullivan, & Stoynoff, 2012) .
The objective of this research study was to gain an understanding of the phenomenon of
assessment and stakeholder influence on contemporary Communicative Language Teaching
practitioners. The study was designed to elicit practical knowledge for the advancement of
effective second language learning curricula. The phenomenological research methodology was
used to explore the lived experience of highly qualified instructors with extensive experience of
working in the high-stakes testing environment of university preparation placement courses. The
pre-eminent second language teaching qualification is a Master degree in the field of second
language teaching in Canada. Master of Second Language Teaching degrees offered throughout
the world are: Master of Education (MEd) in TESOL, Master of Arts (MA) or Science (MS) in
Applied Linguistics, Master of Arts (MA) in English with an emphasis in TESOL and Master of
Arts (MA) in Teaching (MAT) in ESL.

2

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
1.1 Statement of the Problem
The literature revealed gaps in the understanding of the nature of assessment and
stakeholder influence on professional second language instruction, which is practiced in
accordance with current Communicative Language Teaching (CLT) principles. The study sought
to highlight these gaps in knowledge by exploring the influence of current summative assessment
strategies and stakeholders on contemporary CLT.
1.2 Significance of the Research
The results enabled a greater understanding of the desirable and undesirable effects of
summative assessment and stakeholder influence on advanced Communicative Language
Teaching (CLT) practice. The findings revealed practical knowledge for the advancement of
curricula. The study also elucidated the need for further research into the area of influences that
inhibit the successful implementation of CLT in high-stakes settings. The study results may be of
interest to language institutions striving for excellence and for skilled instructors seeking to
promote systemic change toward conditions that facilitate optimal language acquisition. I was
interested in conducting this study to help establish Communicative Language Teaching (CLT)
as a practice with unique qualities that cannot be addressed with strategies solely taken from
general education and yet that fits well with the constructivist and social learning approaches that
prevail in Canadian higher learning environments.
1.3 Purpose of the Study and Research Question
The overarching purpose of the study was to examine the lived experience of
contemporary language instructors. I was interested in exploring the influences that inhibit the
successful implementation of CLT within high-stakes English as a Second Language (ESL)
course settings. I opted to study influence with regards to highly qualified instructors with

3

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
extensive experience as a means to identify second language teachers that have applied
knowledge of the current CLT principles and emerging hypotheses in their course design and
class delivery.
A phenomenological qualitative research method was used to gain meaning and
understanding of the complex forces that summative assessment and stakeholder influence bring
to bear on skilled instruction. The study illuminated the influences that may impel skilled
instructors to alter their instructional content and methodologies by analysing the lived
experience of the study’s participants.
1.3.1 Research questions. I sought to reveal the lived experience of contemporary
second language instructors working within a high-stakes second language instructional setting
though this study. The central questions guiding the study were the following: how does
summative assessment influence advanced second language classroom instruction; how do
stakeholders influence advanced second language classroom instruction; and what input can be
gleaned from instructor responses to better inform institutions to address the undesirable effects
of assessment and stakeholder influence on contemporary second language instruction?
1.4 Background of the Study
Current Communicative language Teaching (CLT) theory has not fully transferred into
practice. Teaching methodologies have become more communicative in many cases, but testing
largely follows more traditional structures that emphasize lower-order thinking with a focus on
form rather than meaning (Brown, 1994). Jacobs and Farrell (2003) asserted that traditional
testing structures create a washback effect that deters teachers from adopting current
methodologies. Moya and O’Malley (1994) maintained that such undesirable washback effects
stemmed from utilizing product-orientated assessment in order to measure the process-orientated

4

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
instruction associated with contemporary CLT. Messick (1996) reiterated the necessity of
aligning instruction with assessment:
Ideally, the move from learning exercises to test exercises should be seamless.
As a consequence, for optimal positive washback there should be little if any
difference between activities involved in learning the language and activities
involved in preparing for the test. (p. 241)
Communicative Language Teaching is the teaching of a second language for the purpose
of communicating in that language. CLT advances instructional principles and approaches drawn
from the fields of Second Language Acquisition and Educational Theory (Lee & VanPatten,
2003). CLT currently involves the notion of providing learners with large amounts of meaningbearing and comprehensible input through activities that are meaningful in nature (Barcroft,
2016). Lee and VanPattern (2003) stated: “[CLT] cannot be equated with first learning some
vocabulary, then learning some grammar, and then finding something to talk about to use the
grammar and vocabulary” (p.1). Kumaravadivelu (2003) stated that traditional second language
teaching, based on the notion of teachers as transmitters of information, is no longer considered
adequate to facilitate contemporary CLT. The second language instructor’s role has become one
of a facilitator as instructors endeavor to facilitate the maximizing learning potential through
active engagement in communicative activates. As contemporary CLT seeks to promote active
learner engagement, traditional extrinsic learner motivators such as standardized tests have
become problematic (Noeles, Clement, & Pelletier, 1999).
A number of teaching methodologies have consequently emerged in order to cope with
the communicative orientation of language instruction and the facilitative role of the second
language instructor. The post-method era began to emerge after the turn of the century as a
movement away from the limiting dogmatism of methodology (Kumaravadivelu, 2003).
Kumaravadivelu (2003) suggested that post-method orientated researchers are challenging the

5

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
concept of teaching methods, asserting that blind adherence to any one methodology is too
limiting to meet the dynamic nature of the linguistic, conceptual, and communicative capacities
of a given group of learners.
A contemporary second language instructor is able to select and adapt teaching strategies,
methods, and approaches in order suit the dynamic nature of any given classroom situation
(Kumaravadivelu, 2003). Kumaravadivelu stated that an instructor draws from his or her Second
Language Acquisition theoretical knowledge base in order to assess the suitability of any
instructional methodology or approach upon application in the classroom.
The goal of this research study was to consider the inherent benefits and challenges that
current summative assessment practices and stakeholder influences place on contemporary
second language instructors. An in-depth account of the aforementioned theories is provided in
chapter two as part of the literature review.
1.5 Researcher Context (Personal Location)
I am a second language instructor with 13 years of experience in the field. My related
educational background includes a Master of Applied Linguistics and a Teaching English to
Speakers of Other Languages (TESOL) diploma. I have also attended a multitude of language
teaching conferences and workshops specific to language-teaching issues. I am currently situated
in Canada, but have the experience of instructing in university settings in Mexico, Colombia,
Saudi Arabia, and the United Arab Emirates. I have sought to understand the dynamic nature of
the language classroom while striving to adhere to the overarching theoretical principles of
current communicative language instruction throughout my career. I believe strong institutional
support and curriculum design is vital to enable learners to efficiently acquire language
proficiency. I am a strong advocate of the movement toward instructor professionalism and

6

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
autonomy within the second language instructional context. Instructors acting as professionals
are not bound by the dictates of a prescribed textbook or syllabus that cannot take into
consideration the fluid dynamics of a particular class setting.
I have instructed under the following summative assessment structures: the standardized
assessment schemes of the Cambridge English Language Assessment exams and the Educational
Testing Service’s (ETS) Test of English as a Foreign Language (TOEFL®) test; purely portfolio
based assessment schemes; combinations of alternative assessment and standardized assessment
strategies; and text book-based testing and teaching systems. Instructing under differing
assessment schemes has enabled me to experience the complex array of positive and negative
pressures on instruction inherent in each assessment scheme.
1.6 Conceptual Lens (Theoretical Orientation)
The theoretical framework for this study was guided by the interpretive description
qualitative research approach developed by Sally Thorne (2008). Interpretive description arose
from the phenomenological tradition, utilizing themes and patterns to draw out knowledge
pertinent to practical application (Thorne, 2008). Interpretive description strives to answer the
question so what and is driven by the overriding aim of gathering practical useable knowledge
(Thorne, 2008). Interpretive description does not dictate adherence to a prescribed set of
procedures. It consists of guidelines that enable the mindful integration of theoretical and
technical devices into the researcher’s study (Thorne, 2008).
I chose to follow the interpretive description research approach as I wanted to explore,
understand and explain the phenomenon of assessment washback and stakeholder influence as it
directly affects practicing advanced second language instructors; as influence is rooted in human
experience, dealing with influence on human endeavor; as little is known about the phenomenon

7

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
of washback and stakeholder influence on advanced second language instructors; and as I sought
to put forward hypotheses relating the advancement of second language learning curricula.
The interpretive description methodology allowed me to reveal the lived experience of
the participants experiencing the phenomenon of assessment and stakeholder influence and
reveal information to improve curricula. Interpretive description’s hermeneutic underpinnings
allowed me to acknowledge and work with the inherent biases and assumptions I hold as I am an
experienced ESL instructor.
1.7 Limitations and Challenges
Several limiting factors emerged as I conducted this research study. I was unable to limit
my study to the Canadian university context as I could not secure a sufficient number of
participants. Consequently, I decided to widen my study to include university contexts worldwide. Despite reaching out to thousands of instructors I was able to secure five of the six
participants I initially set out to obtain. Many potential participants stated that they shared the
concerns the study set out to highlight but were unable to participant due to time constraints. I
acknowledge that this widening of context and lowering of participants negatively impacted the
amount of data collected. However, I believe that my study retained sufficient representative
credibility as: the Teaching English to Speakers of Other Languages (TESOL) context is an
international phenomenon; and I was confident that my categories were saturated as I saw in the
data similar instances over and over again. As I explored the experiential context of the
participants, maintaining interpretive authority proved difficult. I found myself constantly
reflecting on and analysing my own bias and experience in an effort to ensure that my
interpretations were trustworthy. During the interview process I made an effort to ensure my
questioning was impartial and I limited my conversational input as much as possible.

8

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
1.8 Delimitations
In a contemporary second language classroom the instructor emphasizes the learning of
the linguistic system through the process of communicating (Lee & VanPatten, 2003). University
second language courses commonly include additional skills specific to the academic
environment and the students’ needs. Academic skills include formal writing, critical thinking,
study habits, understanding of cultural norms, and educational perspectives. Assessment schemes
generally do not clearly delineate between linguistic competence and academic skills. This study
was primarily concerned with assessment and stakeholder influence on the instructors’ core
pursuit of facilitating linguistic competence. Due to the lack of clarity in differentiating between
linguistic competence and academic skills, I examined assessment and stakeholder influence
from the assessment contexts as found in practice in their entirety. This study looked at how the
assessment of language proficiency and additional academic skills, along with stakeholders,
influence classroom instruction within a university preparatory program context.
The study was solely concerned with the impact of assessment washback and stakeholder
influence directly on contemporary CLT, and sought practical knowledge that can be of use to
language departments and curriculum designers seeking to advance curriculum. Mitigating
factors such as the institutional realities of creating a marketable language program that
curriculum designers and department management must consider when developing curricula
were not considered in this study.
1.9 Thesis Overview
My intention in conducting this research was to explore summative assessment and
stakeholder influence on contemporary second language teachers. The study explored the extent
these influences impact instructional practices. The study further considered the desirable effects

9

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
of washback and stakeholder influence to reveal how instructors experience the interplay
between positive and negative washback.
The thesis is comprised of five chapters. Chapter One introduces the topic of the study,
outlines the study rationale, situates the researcher in the context of the study, outlines the
theoretical and conceptual framework, and considers the limitations of the study. Chapter Two
presents a review of current literature relevant to the research study. Chapter Three outlines and
provides supporting rationale for the qualitative methodology that I utilized for the study;
presents the stages of the research study until completion with the inclusion of ethical
considerations; and outlines the background of each participant. Chapter Four presents the data,
themes, and patterns uncovered through the process of categorization and findings discussion in
light of the current literature. Chapter Five details the implications and suggests
recommendations for the development of second language learning curricula.

10

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Chapter Two: Review of the Literature
The study investigated the influence of summative assessment and stakeholders on
contemporary Communicative Language Teaching (CLT) in the high stakes environment found
in university preparation courses for international students. An understanding of the related
literature is needed in order to better appreciate the impact of these influences on CLT curricula.
The literature reviewed in this chapter is delineated into five sections. Section 1 outlines
the current state of CLT by summarizing the historical journey and the theoretical underpinnings
and practices of CLT. This section serves to illustrate the unique nature of the field of Second
Language Acquisition (SLA) and how that uniqueness impacts curriculum design and classroom
practice. Section 2 outlines the current research pertaining to the nature of language assessment
with a specific focus on the nature of assessment alignment within the CLT context. Section 3
outlines the theoretical framework relating to the phenomenon of positive and negative forms of
assessment washback. Section 4 presents an overview of current research on washback in the
context of second language learning. Lastly, section 5 explores alternatives to ESL curricula
based on standardized testing. Of all the literature reviewed, Hargreaves and Shirley’s (2009)
call to embrace post-standardization in education best reflected the study’s findings.
2.1 Section 1: History, Theoretical Underpinnings and Application of SLA Principles
Communicative Language Teaching (CLT) is the instruction of a second language for the
purpose of communication (Richards, 1996). CLT comprises of a collection of approaches
informed by current Second Language Acquisition (SLA) theory (VanPatten, 2015a). However,
as theory has evolved, it has not smoothly translated to practice leaving the field largely in a
quasi-behaviorist state (VanPatten, 2015a). The current situation is best explicated with an
understanding of the history of second language theory and practice.

11

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
2.1.1 Historical path of second language acquisition. The field of Second Language
Acquisition (SLA) began to emerge during the 1950s and 60s when behaviorism dominated the
field of psychology. At the time, the US military adapted learning methodologies that promoted
habit forming for use in the second language learning classroom. Errors were singled out during
production and corrected, dialogues were memorized, and language further rehearsed though
language drills. Chomsky’s (1957) work on first language acquisition began to challenge the way
researchers thought about how first languages were learned (VanPatten, 2003). Chomsky
postulated that habit forming alone would be woefully inadequate to account for the rate and
amount of language children acquire when internalizing language. He theorized that humans
must instead have an innate internal language system that allows them to learn language (as cited
in VanPattern, 2003). This system later became known as the Language Acquisition Device
(Gass & Selinker, 2001). Chomsky’s (1957) ground breaking work was soon picked up by SLA
researchers and applied to second language learning. A seminal paper entitled The Significance
of Learners’ Errors (Corder, 1967) suggested foremost that acquiring a second language
involved innate processes and mechanisms that are unique to language. That paper and the
subsequent pioneering work by prominent researchers in the early to mid-1970s ushered in the
contemporary era of SLA research (VanPatten, 2003). Researchers since the 1970s have
primarily concerned themselves with: what learners do when they are learning a second
language; what stages of learning they progress through; how second language differs from first
language; how errors occur; how extraneous factors affect acquisition; and how learners access
and make use of the internalized linguistic system in order to communicate (VanPatten, 2003).

12

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
2.1.2 Current assumptions about the nature of second language acquisition. Lee and
VanPattern (2003) outlined the uncontested areas of agreement that is currently seen in the field
of SLA. The areas can be summarized as: SLA is the creation of an unconscious linguistic
system through a variety of complex processes in a dynamic manner that takes time; skill
acquisition is different from the creation of an unconscious linguistic system; and most second
language learners are unable to achieve native like competence.
SLA researchers contend the degree and manner in which the unconscious linguistic
system can make use of the explicit learning of grammatical and lexical forms. This contention is
termed the interface debate within the field of SLA (Han & Finneran, 2014). Proponents of the
no-interface position hold that explicit learning has no relevance in the acquisition of language.
Advocates of the strong-interface position maintain that language learning is largely, if not
completely, regarded as a conscious process. Traditional approaches to second language teaching
can be equated to the strong-interface position. Proponents of the weak-interface position hold
that linguistic forms made salient, for example via highlighting or repetition, can enable a greater
chance for the linguistic system to uptake these forms. A stronger form of the weak-interface
positon holds that the conscious noticing of grammatical forms and lexical items by the learner
can expedite uptake (Han & Finneran, 2014). The majority of researchers in the field of SLA
favor the weak-interface position (Han & Finneran, 2014). Rod Ellis (2005) claimed that, given
the insufficient evidence for each positon, instructional curricula should incorporate instructional
methodologies derived from each interface position.
2.1.3 Second language acquisition theory to communicative language teaching
principles. Research has revealed that language acquisition cannot be isolated from
communication. Communication must be both comprehensible and meaning-bearing to be of

13

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
value in the creation of the mental representation of a second language (Lee & VanPatten, 2003).
A working definition of the nature of communication is first needed due to the central role of
communication in CLT. Lee and VanPatten (2003) put forward the definition, extrapolated from
Savignon’s (1997) work on communicative competence: “Communication is the interpretation,
expression and negotiating of meaning in a given context” (p. 53). Communication in a CLT
classroom setting therefore involves students being constantly involved in what is being talked
about. In this sense they are not being talked at but, talked with (VanPatten, 2016a).
Jacobs and Ferrell (2003) stated current thinking in CLT has led to eight major changes
in approaches to language teaching: a) greater learner autonomy; b) allowing for social nature of
learning; c) the curricular integration of wider subject matter; d) focus on meaning; e)
diversification of learning strategies; f) development of higher-order thinking skills; g) teachers
becoming co-learners; and, h) incorporation of alternative assessments. To accommodate for this
paradigm shift CLT instructional curricula are constructed around communitive tasks, replacing
grammar and vocabulary (VanPatten, 2016a). However, it should be made explicit that that this
does not mean that classroom practices need always be task based or that language forms cannot
be incorporated into the classroom (VanPatten, 2016b).
2.1.4 Rate of language acquisition. Cummings (1979) introduced the terms Basic
Interpersonal Communication Skills (BICS) and Cognitive Academic Language Proficiency
(CALP) to highlight the differing time periods immigrant children typically needed to obtain
conversational fluency compared to grade-appropriate academic proficiency in the target
language. Collier, (1987); Klesmer, (1994); and Cummins, (1981) found that immigrant children
required approximately two years of classroom instruction to develop BICS and a further three
years to develop CALP to the level of their target language speaking peers.

14

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
2.1.5 Cultural factors affecting student academic learning. Cultural influences play a
significant role in second language preparatory courses at Canadian universities as classes are
made up of international students from various cultural backgrounds. Institutions and language
instructors need to consider the cultural perceptions and expectations regarding education before
designing and implementing the instructional curriculum. Contemporary Communicative
Language Teaching (CLT) principles can often be at odds with many of these assumptions. To
exemplify, Huang’s (2005) study on student perceptions of learning academic listening in the
classroom found that Chinese students tended to question the value of a classroom that focuses
on discussion rather than lecture, deviates from following a textbook, and emphasizes group
work. Institutions can mitigate cultural and other learner influences on instruction by aligning
assessment with desirable instructional approaches (Biggs, 2003). For example, the
internationally recognized International English Language Testing System (IELTS™)
standardized language proficiency test follows a communicative orientated format.
2.1.6 Professionalism in communicative language teaching. Professionalism is defined
in the Canadian Oxford Dictionary Online (2016) in the following way:
The combination of qualities and conduct regarded as essential to professional
practice. These include knowledge, skills, relevant competence, behavioral
qualities, and values of honesty, integrity, ethical probity, and capability of
working well with patients or clients, colleagues, and representatives of the
public.
The concepts of critical reflection and inclusion into a wider discourse about expertise within the
field are prominent within the literature concerning professionalism in education (Englund, 1996;
Hedgcock, 2002; Kumaravadivelu, 2003). Kumaravadivelu (2003) called for language
instructional practices to go beyond relying on mechanical teaching strategies and methods, and
towards critical thinking about practice. The post-method era requires language instructors to

15

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
become strategic practitioners with the ability to read and reflect on the dynamic nature of the
communicative language classroom. Kumaravadivelu (2003) noted that the paradigm shift could
further include the current movement, in the wider field of education, toward the notion of
teachers as engaged transformative intellectuals (Kumaravadivelu, 2003). The literature
described teachers who are transformative intellectuals as:
Professionals who are able and willing to reflect upon the ideological
principles that inform their practice, who connect pedagogical theory and
practice to wider social issues and work together to share ideas, exercise power
over the conditions of their labor, and embody in their teaching vision of a
better and more humane life. (Giroux & McLaren, 1989, p. 23)
In juxtaposition to the current view of teachers as strategic practitioners and/or
transformative intellectuals is the traditional view of teachers as passive technicians. Passive
technicians are charged with transmitting content knowledge to the student without alteration.
The aim of the traditional approach is to promote the understanding of content knowledge and, as
such, it is rooted in behaviorist educational theory (Kumaravadivelu, 2003). Table 1, presented
on the next page, illustrates the functions and features of these three viewpoints of teaching.
The proposed research study concerned itself only with participants who identify as
reflective practitioners and/or transformative intellectuals as outlined above. Second language
instruction through a passive traditional approach was purposely excluded from this study as it is
not compatible with current CLT principles.

16

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

17

Table 1
The Roles of a Teacher: A Summary
Teachers as passive Teachers as
technicians
reflective
practitioners
conduit
facilitator

Techers as
transformative
intellectuals
change agent

Primary source of
knowledge

professional
knowledge +
empirical research
by experts

professional
knowledge +
teacher's personal
knowledge + guided
action research by
teachers

professional knowledge
+ teachers personal
knowledge + selfexploratory research by
teachers

Primary goal of
teaching

maximizing
content knowledge
through prescribed
activities

all above +
maximizing learning
potential through
problem-solving
activates

all above + maximizing
sociopolitical awareness
through problem-posing
activates

Primary orientation discrete approach
of teaching
anchored in the
discipline

integrated approach,
anchored in the
classroom

holistic approach,
anchored in the society

Primary players in
the teaching
process (in rank
order)

teacher + experts +
learners

teachers + learners +
experts + community
activists

Primary role of
teacher

experts + teachers

Note. Adapted from “Beyond methods: Macrostrategies for language teaching” by B.
Kumaravadivelu, 2003, Yale University Press. Copyright 2003 by Yale University.
2.1.7 Transformative learning. Transformative learning theory has offered educators an
approach to elicit a deep form of learning. Transformative learning requires learners to go
beyond thinking about knowledge in terms of acquisition and application and to further consider
how knowledge is internally reflected. It enables emancipation from life experience rooted in the
unquestioned assumptions layered onto us by the societies and cultures we grow up in (Mezirow,
1991). Merrian and Bierema (2014) concluded that the overarching goal of adult education is to

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
enable a more liberated, socially responsible, and autonomous society. The author’s stated goal is
achieved by encouraging learners to strive for making more informed choices. Hargreaves
(2003) argued that teachers need to prepare students for success in the knowledge based society
of today by advancing learning principles such as ingenuity, advancement of collective
intelligence, creativity, problem-solving, cooperation, flexibility, networking skills, and life-long
learning. He claimed that curricula developed around soulless standardization and market
fundamentalism are inadequate to achieve this goal. Transformative learning involves education
that creates opportunities for reflective thought, imaginative problem posing, and discourse,
involving group deliberation and group problem-solving of real-life experience (Mezirow, 1997).
Students identify, then critically examine their own and others assumptions and deeply held
beliefs through the transformative learning process (Mezirow, 1991). To facilitate a
transformative learning classroom the instructor ensures these principles: a) full access to
information; b) freedom from coercion; c) opportunity for all to voice beliefs, challenge others,
defend oneself, explain, assess evidence, and judge arguments; d) assumptions are critically
reflected upon; e) all participants are empathetic and open to other view points; f) an active
listening environment; g) a clear search for common ground; and h) a democratic discussion
environment. Assistance from the teacher gradually dissipates as the class becomes more selfdirective until, ultimately, the teacher assumes a co-learner role (Mezirow, 1997).
Derince (2012) pointed out the trend in university organizations to structure themselves
on behaviorist models where students are often trained to take an endless stream of exams. She
called on teachers to focus on transforming the future society. Derince (2012) postulated that this
could be done by finding ways to implement critical learning into the classroom. Gardner (2008)
made note of the wave of restrictive legislation that affects many Western education systems by

18

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
forcing standards-driven curricula in education. Despite this, she saw opportunity in turbulent
times because as systems change they can leave spaces for new ideas.
2.1.8 Outcomes based education. Curricula designed around outcomes are making
headway into the Teaching English to Speakers of Other Languages (TESOL) industry as a
method of accommodating the needs of Communicative Language Teaching (CLT) (Burns,
2003). Curricula designed with Outcomes Based Education (OBE) framework focus on what
students can actually do after they are taught or have learned (Spady, 1994). This translates to a
set of can do statements which illustrate what a language learner can do with language in the real
world in the TESOL field (VanPattern, 2015b).
Spady (1994), outlined a description of the theory of OBE, in stating that it was built on
the following assumptions: a) every student is able to succeed as a learner, b) successful learning
leads to more success; and c) that educational institutes have control over the conditions which
make it possible for success. However, OBE has not come without its issues. Schlebusch and
Thobedi (2004) concluded through their study of OBE implementation in South Africa that grade
eight ESL learners experienced problems with OBE due to insufficient use of advanced
strategies for teaching and learning by the instructors. Burns (2003) similarly concluded that
language learners may view the subjective nature of a proficiency orientated system as both
demotivating and weak. He claimed that the subjective nature and pass/fail benchmarking
inherent in the OBE framework does not allow for straight forward integration with traditional
grading systems, which present a multitude of application issues.
2.2 Section 2: Communicative Language Assessment Strategies
The aim of a communicative language assessment is to determine to what a learner can
do with the language as opposed to establishing how much of the grammatical and lexical

19

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
knowledge is understood by the test-taker. Assessment is made up of a set of purposeful tasks
that are both relevant in subject matter and appropriate in level for the test-taker (Coombe et al.,
2012). The authors stated five major issues inherent with communicative assessment: a) how to
gauge one overall language proficiency from multiple language skill tests (speaking, reading,
writing, and listening) given that results may greatly vary; b) how to account for human factors
such as personality, mood, culture, that may impede performance on a given communicative
task; c) how to test perceptive language skills (reading and listening) without a clear notion of
what understanding consists of; d) how to test productive language skills (speaking and writing),
as it is not clear how to ascertain when a piece of writing or spoken production has achieved its
purpose; and, e) that language tests can only be made up of sampling aspects of language,
because the enormity of purposes language is used for, only a fraction could be selected for the
creation of tasks presented in the test. The designing of any communicative language proficiency
assessment strategy currently involves compromises as these issues remain largely unresolved
(Coombe et al., 2012). A test developer needs to balance test qualities of reliability, construct
validity, authenticity, interactivity, impact, and practicality for the given situation (Bachman &
Palmer, 1996).
2.2.1 Standardized assessment strategies. Moya and O’Malley (1994) stated that
process-orientated curricula and instruction associated with CLT is largely viewed as being
incongruent with product-orientated assessment and as such standardized testing. The high stakes
language tests such as IELTS™ and TOEFL®, nevertheless, remain prevalent within the
industry. These tests strive for improved reliability and validity in the face of the multitude of
confounding factors inherent in the testing of communicative tasks (Coombe et al., 2012).

20

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
2.2.2 Alternative assessment. Alternative assessment consists of an ongoing interactive
process between the learner and the instructor in assessing learner performance (Hancock, 1994).
Alternative assessment methods include portfolio collation and task-based language assessment
(Coombe et al., 2012).. Additional benefits of this form of assessment include better higher-order
thinking and life-long learning skills (Hancock, 1994).
Alternative assessment is largely a qualitative approach and subjective in nature, thus the
test validity and reliability tend to become difficult to establish and maintain. Maintaining
validity and reliability can lead to a potentially costly and time-consuming situation creating the
impetus for many institutions to avoid incorporating alternative assessment strategies (Moya &
O'Malley, 1994). Alternative assessment need not be implemented in isolation from standardized
testing. Coombe et al. (2012) stated that it is best implemented in conjunction with standardized
tests as part of a multiple-measures assessment scheme.
2.3 Section 3: Assessment Washback
The term washback has come to denote the manner and extent a test influences teaching
and learning practices (Alderson & Wall, 1993). Test influence on teaching and learning
practices may occur in a myriad of desirable or undesirable ways, and need not necessarily
correlate with test validity. A poor test could conceivably encourage good study habits, while a
good test may cause undue anxiety (Alderson & Wall, 1993). This independent nature of
washback gave rise to the term washback validity (Messick, 1996). The extent to which
washback will be greater in tests with perceived important consequences, as with high-stakes
testing, will differ among individuals (Alderson & Wall 1993).
2.3.1 Alignment. A curriculum is said to be aligned when the assessment represents the
curriculum well (Biggs, 2003). Curriculum built around communicative activities and tasks,

21

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
reveals alignment when an assessment scheme is communicative in nature. However, effective
standardized assessments that are communicative in nature are problematic to design. Spolsky
(1985) noted that the reason for this difficulty is that “any language test is by its very nature
inauthentic, abnormal language behavior, for the task is not to give so much as to display
knowledge” (p.1). The rise in the inclusion of alternative assessment practices has largely
resulted from the challenges of creating effective communicative standardized assessment
schemes (Haney & Madaus, 1989).
2.3.2 Motivation and performance. The relationship between motivation and
performance is very complex. One should not assume that extrinsic motivation provided by the
test will translate into desirable behaviors (Alderson & Wall, 1993).
2.3.2.1 Heuristic / algorithmic tasks. Pink (2011) argued that motivation acts differently
on algorithmic tasks than on heuristic tasks. He defined algorithmic tasks as those that comprise
routine steps leading to success, while heuristic tasks have no clear path to success.
Communicative Language Teaching (CLT) has evolved into a heuristic undertaking. Pink
outlined the danger of making use of extrinsic motivators such as if-then rewards, goal setting,
and deadlines to encourage heuristic tasks. He used the analogy of an artist being only able to
create sub-optimal art when external pressures such as the possibility of monetary gain came to
bear. Pink noted performance is not hindered if the external motivator is perceived as enabling.
He defined an enabling motivation as one that creates a feeling of empowerment to do something
exciting or interesting by the motivator giving useful information and feedback about ability. The
motivator must ensure that the task has a clear connection to a larger purpose.
2.3.2.2 Anxiety: Debilitating and facilitating. Alderson and Wall (1993) stated that the
effect of assessment washback on performance may be additionally viewed through the

22

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
consideration of anxiety on a learner or teacher. Anxiety may be experienced as debilitating or
facilitating depending on the interrelation of many factors specific to the teacher or learner, the
assessment, and the perceived consequences of the assessment.
2.4 Section 4: Review of the Research of ESL Washback on Teachers and Students
I have found relatively few washback studies of note in relation to the field of Teaching
English to Speakers of Other Languages (TESOL). Alderson and Wall (1993) noted a lack of
washback studies in general. I found no study of washback in the field of TESOL that
specifically focused on contemporary CLT instructional practices. The existing literature was
divided into five major themes: a) Washback on the teaching and learning curriculum; b)
Washback on teaching methods; c) Washback on classroom assessment; d) Washback on student
learning; Washback on the attitudes, view points, and feelings of second language instructors;
and, e) Washback on student motivation.
2.4.1 Lack of studies controlling for instructional professionalism in communicative
language teaching. I found a deficit of literature that focused on instructional professionalism in
the application of contemporary CLT principles. The majority of studies focused on second
language instruction as it exists in practice. Varying degrees of CLT instructional
professionalism can be gleaned from each study. This is of interest as the distance between
theory and practice in the wider TESOL environment is evident.
Cheng’s (1997) research into washback influence on secondary ESL teaching in Hong
Kong sampled teachers to represent language teaching as it is being practiced there. Her study
revealed an overall lack of applied knowledge of current CLT practices, not only in second
language teachers but in all stakeholders concerned. The lack of applied knowledge is evidenced
through her outline of the over reliance on textbooks as direct teaching support and through

23

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
interviews where it was revealed that the majority of teachers were grappling with very basic
CLT concepts such as integration of skills and learner participation. Cheng noted that some
schools use solely test-practice papers for classes preparing for the final exit test. Qi’s (2007)
study on testing as a pedagogical change agent in China found that instructional practice did not
follow test developers’ intentions. Communicative tasks were not utilized in the classroom.
Instead instructors focused on the testing situation and the assumed preferences by the markers.
Studies situated in Western universities include Shohamy’s (1993) study which focused
on second language education at the secondary level in Iowa. She noted a difference between
novice and experienced language instructors. Novice language instructors were more embracing
of testing formats that encouraged communicative strategies as they better aligned with their
instructional practices. Experienced instructors on the other hand saw the test as a burden that
they sought to manage without changing their approach. A Canadian study of teachers’
perspectives on assessment change in TESOL at the secondary level comprised of teachers who
all had at a minimum a Bachelor of Education and a TESOL (Teaching English to Speakers of
Other Languages) training certification of some nature and ranged from novice to veteran
teachers. The study showed the varying professional stances and beliefs that lead to inconsistent
findings (Turner 2006).
2.4.2. Washback on teaching and learning curricula. The following studies found
washback on curricula from various types of high-stakes standardized tests. By washback on
teaching and learning curricula I refer to the influence of assessment on the instructor’s choice of
what material and language related skills to focus on in class. Erfani (2012) made a comparative
study of the market-leading standardized proficiency university entrance tests TOEFL iBT ® and
IELTS™ in the Iranian context. The study found both positive and negative washback. However,

24

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
negative washback was more substantial in both test formats. The study concluded that negative
washback acted to push class activities away from communicative activities and toward
unproductive test-taking strategies. A number of related studies have found similar results and a
summary of their breakdown can be located in table 2.
Li’s (1990) research into the introduction of the Chinese National Matriculation English
Test (NMET) followed a similar pattern. Li considered this to be positive washback as Li’s study
found a movement away from the focus on the formal linguistic properties of language and
toward the practice of language skills. Wall and Alderson’s (1993) highly cited Sri Lankan study
came to similar conclusions, but lamented that the test encouraged teachers to focus only on the
skills that appeared in the test. A number of related studies have come to similar conclusions and
are summarized in table 2.
The following studies found no washback on curricula from various types of high-stakes
standardized tests. Wersdorp’s (1982) study looked at the effect of introducing multiple-choice
tests on language instruction of the writing skill. The questionnaire data gathered found that no
significant narrowing of the curriculum occurred. Watanabe’s (2000) study found evidence of no
washback. He surmised that this was due to an inability of teachers to change traditional
practices what-so-ever. The researcher, however, did note that the lack of movement away from
traditional practices could also be due to a multitude of other reasons from lack of test
specifications to imbedded cultural traditions.
Studies relating to washback on the curriculum have revealed that washback is a strong
force for shaping the instructional curriculum, influencing what teachers decide to teach in the
second language classroom. Studies revealed that washback from standardized tests can be an
effective tool to move teachers away from traditional linguistic-based instructional practice.

25

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
However, the studies also indicate that washback leads teachers to narrow their instructional
practice, encouraging teachers to focus only on what is necessary for passing the test.
2.4.3. Washback on teaching strategies, methods and approaches. Several washback
studies on teaching strategies, methods and approaches were discovered in the literature.
Shohamy (1993) found that the test looked at in her study influenced only novice teachers’
methodologies. She noted that additional communicative activities were employed by novice
teachers when preparing their students for oral tests. Stecher et al.’s (2004) study of the writing
section of the Washington Assessment of Student Learning test revealed that teachers
incorporated processes and strategies intended by the tests into their instruction. Alderson and
Hamp-Lyons’ (1996) study found that while assessment did influence how teachers taught, the
effect was in no way uniform from teacher to teacher. A number of related studies came to
similar conclusions as Alderson and Hamp-Lyons’ study and are summarized in table 3.
The following studies found no washback on teaching strategies, methods and approaches
utilized by the instructor. Qi (2007) conducted a study to ascertain whether instruction followed
test developers intent within the high stakes secondary school National Matriculation English
Test (NMET) in China. Qi found that high-stakes tests are not effective tools for bringing about
instructional change. Turner’s (2006) Canadian study looking into Professionalism in the context
of high-stakes tests found that when second language instructors were faced with mandated
assessment and curricula changes they managed realignment of their classroom practices only
within their personal beliefs and professional stances concerning teaching methodologies. Qi’s
(2005) study found that tests designed for both the change in instruction and to measure language
proficiency tend to fall short of either or both goals due their conflicting nature. Numerous other

26

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
studies have found testing to have little to no influence on strategies, methodologies, and
approaches and are summarized in table 3.
Overall, the studies revealed a complex picture of washback on instructors’ strategies,
methodologies and approaches. The research has holistically shown that a direct relationship
cannot be established. Any washback appears sparse and not uniform in nature. It appears that
test reform by itself is unable to effect methodological change of language instruction.
2.4.4 Washback on classroom assessment. Wesdorp (1982) studied the use of multiple
choice question type testing in the high-stakes language learning environment. The researcher
found no significant evidence to suggest in-class assessment was changed to match the final test
structures. The conclusion was that this study did not reveal washback on classroom assessment.
Wall and Alderson’s (1993) Sri Lankan study found that the newly implemented test had
a major effect on in-class testing design. Their findings revealed that instructors changed the
content of their class work, and that in class testing tended to mimic the question style of the
final test. Wall and Horak (2006) looked at washback from the TOEFL ® test, noting that
screening, diagnostic and practice tests purposefully followed the TOEFL® test format.
Washback in relation to how in-class assessment changes as a result of assessment has
been studied in a limited manner. The literature has not produced consistent results, and therefore
it is premature to glean tendencies.
2.4.5 Washback on student learning. The following studies found washback on student
learning. Hughes (1988) studied a newly introduced exam in a Turkish university language
program, and concluded that student learning was found to have increased. However, as the
results were compared to tests with no resemblance to the newly incorporated test, his
conclusions were not robust. Saif’s (2006) study looked at test washback on students after

27

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
teachers had received training to prepare them for the new exam. In this circumstance, positive
washback was found as student learning had improved. Negative washback was found in
Tsagari’s (2011) study, which found that instructors devoted large portions of class time to
intensively prepare for the test despite believing that the exam created a lack of student
autonomy and ability to communicate.
The following studies found no washback on student learning. Wesdorp’s (1982) Dutch
study found that no negative or positive washback was found on student learning with the
incorporation of multiple choice orientated testing. Cheng (1998) found similarly, concluding
that student learning strategies remained largely unchanged.
Pan’s 2014 study looked into how the exit tests (the local GEPT ® test, and the
internationally recognized TOEIC® test), influenced student learning strategies, and test
performance in Taiwan. He found that the test had no significant effect on the learning strategies
employed out of class by the students. Students were found to fall back on traditional learning
strategies, such as studying grammar patterns, despite the communicative orientation of the tests.
Pan’s study concluded similarly with studies by Chang (1998), and Xie (2013) in its finding that
study habits are not easily coerced by a test. Watanabe’s (1992) study looked into the effect the
high-stakes university entrance English language examination has on language learning
strategies for subsequent language study. The study lacked enough vigor to substantiate any
findings. It did reveal the complexity of the phenomenon of washback, raising further questions
for future studies.
Jinsong, Peiying, and Xiaomei’s (2014) study looked into student learning washback
from the Fudan English Test after its introduction in Fudan University in China. They found
limited evidence of positive washback on student learning practices. They concluded that in

28

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
order for positive washback to occur additional factors such as resources, support, and
information need to be developed and incorporated alongside the test.
Research into the area of student learning is sparse and somewhat haphazard. However, it
can be generally gleaned from the studies that students tend to fall back on traditional learning
strategies such as studying grammar patters, despite the communicative orientation of any test.
2.4.6 Washback on the mental state of instructors (attitudes, and feelings). Shohamy
(1996) found that overall negative reactions, including stress, anger and humiliation as a result of
newly introduced tests, largely stemmed from the fact that teachers had not been consulted about
the test in advance, but also because of the pressure they felt to teach in ways dictated by the
exam. Li’s (1990) study into the National Matriculation English Test (NMET), showed teachers’
feelings of unease toward the exam abated as time passed. Cheng’s (1998) study reviled that the
avoidance of feelings of guilt motivated one teacher to focus heavily on test preparation.
Alderson and Hamp-Lyons (1996) similarly found negative attitudes including frustration being
fermented toward the use of the TOEFL® test.
Not all teachers were found to harbor negative feelings, some felt positive and felt
enjoyment in their teaching experience. Read and Hayes (2003) documented positive feelings of
teachers toward the IELTS™ test orientation in their New Zealand study. A number of related
studies have found similar results and a summary of their breakdown can be located in table 7. A
range of both positive and negative feelings and attitudes of teachers towards standardized
testing has been observed suggesting that many additional factors need to be considered.
2.4.7 Washback on attitudes and feelings of learners. Li (1990) and Read and Hayes
(2003) noted positive student feelings toward the revised exam, which coincided with heightened

29

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
motivation. However, Shohamy et al. (1996) found that students tended to mirror teachers in
feeling anxious about the imposed test changes.
Studies have shown that students’ attitudes need not necessarily reflect that of the
teachers. Alderson and Hamp-Lyons’ (1996) study found no evidence to suggest students’
insistence of methodology, use of practice tests, or test specific strategies (in a TOEFL ®
orientated testing situation). However, other studies found this not to be the case. Through
teacher interviews, Wall and Horak (2006) revealed that student expectations of classroom
activities pressured teachers’ instructional decision making. Other related studies have found
similar results, and are located on table 7 (appendix A) in a summarized format.
The conflicting results indicated that washback on the learners’ attitudes and feeling is
not straight forward. The inconsistent results from the studies indicate that many forces appear to
play a role in shaping the attitudes of learners towards assessment.
2.4.8 Washback on student motivation toward exams. Zhan and Wan (2015) looked
into the positive washback effect of the high stakes English language exams found in China.
They looked at student practices outside the classroom, and found that the motivation generated
from the test was largely short-term, peaking in the weeks before the exam. Tsagari’s (2011)
study indicated that students felt a strong motivation to pass the test as passing was associated
with opportunities for professional and educational advancement. However, instructors also
reported that students found the intense exam-orientation of class resulted in students becoming
demotivated and bored. A number of other studies have found similar results and a summary of
their findings can be located in Table 9 (Appendix A). Watanabe (1992) concluded that exams
motivated students only to pass the exam which perhaps best sums up research findings
associated with exam washback on motivation.

30

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
2.5 Section 5: The Fourth Way: The Inspiring Future for Educational Change
Hargreaves and Shirley (2009) utilized the term The Fourth Way to denote the poststandardization era in the field of K-12 education. They stated: “We are entering an age of poststandardization in education. It may not look, smell, or feel like it, but the augurs of the new age
have already arrived and are advancing with increasing speed” (2009, p1). Hargreaves and
Shirley (2009) argued that the coming post-standardization era follows the three major phases of
educational changes found within developed nations. In the first phase (1950s – mid 1970s)
teachers were afforded much trust and autonomy enabling them to largely teach as they saw fit.
The first phase resulted in an innovative time period, but it also came with a lack of consistency.
The second phase (mid 1970s – late 1980s) saw a systemic reversing where instructors were
intensely mistrusted. High importance was placed on standardization, and uniformity. The
second phase had the net result of diminished quality, depth, and breadth of learning. The third
phase (late 1980s – present) sought to balance professional autonomy with accountability.
Support networks and training were incorporated alongside vigorous performance data. The third
phase, despite its promise, has not encouraged learning, creativity and change, but rather habit
and compliance, while narrowing the focus to short-term issues. Hargreaves & Shirley (2009)
asserted that, “As The Third Way [the third phase] has evolved, it has not put the passion back
into teaching nor the pleasure into learning” (p. 45).
The principles put forward by Hargreaves and Shirley (2009) include: a) the creation of a
vision that is inspirational, inclusive and compelling in order to guide learning and achievement;
b) investment into high-quality teacher attraction and retention; c) creation of professional
cultures of trust, cooperation, and responsibility geared toward improvement; d) promotion of
evidence-informed change as opposed to data-driven reform; e) maintenance of trust

31

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
relationships between all administrating bodies; and, f) community engagement, development,
and empowerment (Hargreaves & Shirley, 2009). Central to the development of these principles
was the recognition of the tendency for distracters in the educational system to divert teachers
away from effective instruction. Distractors included mandated targets, excessive testing,
formulaic curricula, multitudes of rubrics, droves of standards, and an overwhelming emphasis
on the basics. Professionalism must be fostered and then trusted in order to lower the need for
such distractors as previously stated. Professionalism is achieved with high-quality teachers,
positive and powerful professional associations, and lively learning communities combined with
indirect accountability measures such as test sampling, which promote responsibility over
accountability (Hargreaves & Shirley, 2009). The post-standardization era in the field of K-12
education was essentially about relinquishing autocratic control of curricula in favor of a
democratic professional path toward improvement (Hargreaves & Shirley, 2009).
2.6 Summary of the Literature Reviewed
The reviewed literature outlined the current knowledge pertaining to the impact of
assessment on CLT curricula. The literature illustrated how the unique nature of the field of
Second Language Acquisition (SLA) impacts curriculum design and classroom practice,
illuminating the complex nature of assessment and stakeholder influence. Hargreaves and
Shirley’s (2009) depiction of the emerging post-standardization age in the wider field of
education was briefly examined as its principles of democratic professionalism may be relevant
in the search for a way forward for the Teaching English to Speakers of Other Languages
(TESOL) industry. The following chapter discusses the research methodology used within the
study.

32

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Chapter Three: Research Methodology
3.1 Qualitative Research
The phenomenological qualitative research methodology best suited this study because I
wanted to explore, understand and explain the phenomenon of assessment and stakeholder
influence as it directly affects practicing advanced second language instructors. The fundamental
purpose of a phenomenological study is to gain understanding and meaning of human experience
regarding a phenomenon through a description of the underlying universal meaning or essence
(van Manen, 1990).
Qualitative research is borne from a constructivist perspective. Guba and Lincoln (1989,
cited in Waters and Mehay, 2010, p.3) demonstrated that the philosophical paradigm of
constructivism is based on relativist ontology and a subjectivist epistemology. Guba & Lincoln
outlined that “In Constructivism: The nature of truth about ‘what is what’ is socially negotiated
(with others) AND; That the true meaning of knowledge is then internally constructed” (cited in
Waters and Mehay, 2010 p. 3).
van Manen (1990) described phenomenological research in the following way:
“phenomenological research is the description of the experiential meanings we live as we live
them” (p. 11). The phenomenological researcher endeavors to understand the reality of others by
allowing the participants to guide the process (Creswell, Hanson, Clark Plano, & Morales, 2007).
A phenomenology creates opportunity for the researcher to immerse her/himself into the
phenomena being explored (Creswell, Hanson, Clark Plano, & Morales, 2007).
Hermeneutic phenomenology contends that researchers’ biases and assumptions cannot
effectively be set aside by bracketing (Gadamer, 1960, 1998). Hermeneutic phenomenology
views researchers’ assumptions and biases as embedded and essential to the interpretive process.

33

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
The researcher routinely considers how his or her own experience relates to the issues that arise
during the period of research (Allen, 1996, as cited in Laverty, 2003). Interpretive description
follows the hermeneutic view (Thorne, 2008):
No human being is immune from theoretical influence (and bias), and
therefore the way in which this susceptibility is handled involves explicit
recognition and understanding, so that it influences the research in ways that
are consistent with integrity to an inductive reasoning process that generates
findings that are well grounded within data. (p. 71)
I utilized the interpretive description methodology as it allowed me to explore the
phenomenon of assessment and stakeholder influence with an orientation toward revealing
information for practical application. Interpretive description’s hermeneutic underpinnings
allowed me to acknowledge and insert my point of view, as an experienced ESL instructor, from
a non-biased perspective.
3.2 Interpretive Description Methodology
The interpretive description research approach draws from phenomenology as it is rooted
in the study of human experience of phenomena. Interpretive description strives to answer the
question so what and is driven by the overriding aim of gathering practical useable knowledge
(Thorne, 2008). Thorne stated that she developed this strategy to address the applied knowledge
orientation of the health care industry. The analytic procedures of interpretive description go
beyond sorting, coding, and describing to additionally involve synthesizing, theorizing and recontextualizing in order to understand the phenomena and ultimately generate a theory (Thorne,
2008). The interpretive description methodology suited my study as I sought practical knowledge
that was applicable to the advancement of second language instruction and curriculum design.
The interpretive description methodology does not require the researcher to adhere to a
prescribed set of procedures. The methodology presents guidelines that enable the mindful

34

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
integration of theoretical and technical devices into the researcher’s study (Thorne, 2008). The
guidelines for interpretive description are outlined by Thorne:
Interpretive description studies: a) are conducted in as naturalistic context as
possible in a manner that is respectful of the comfort and ethical rights of all
participants; b) explicitly attend to the value of subjective and experiential
knowledge as one of the fundamental sources of clinical insight; c) capitalize on
human commonalities as well as individual expressions of variance within a
shared focus of interest; d) reflect issues that are not bounded by time and
context within which the current expressions are enacted; e) acknowledge a
socially constructed element to human experience that cannot be meaningfully
separated from its essential nature; f) recognize that, in the world of human
experience, reality involves multiple constructed realities that may well be
contradictory; and, g) acknowledge and inseparable relationship between the
knower and the known, such that the inquirer and the object of that inquiry
interact to influence one another. ( p. 74)
3.3 Ethical Considerations
This study conformed to the UNBC policy on research involving human participants and
did not commence without the approval of the UNBC Research Ethics Board (REB). The study
was guided by the Tri-Council Policy Statement on the Ethical Conduct for Research involving
Human Subjects (CIHR, NSERC & SSHRC, 2014). This policy promotes research that is
conducted according to the highest ethical standards.
3.3.1 Confidentiality and anonymity. I understood the importance that I maintained
confidentiality and anonymity for all participants who took part in the study. I have kept the
names and contact information stored on a password protected computer accessible only to
myself. I asked each participant to choose a pseudonym for identity protection. I replaced all
references to participants with their corresponding pseudonyms in the thesis and subsequent
publications. The interview transcribed data was accessible only to me while follow-up email
threads were accessible to supervisory committee members who were requested to keep all
correspondence on a secure password protected computer.

35

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
3.3.2 Vulnerable populations. This study had minimal risk as there was not any part that
could harm a participant or a participant’s students. The signed consent form used to inform the
participants of the research process and expectations was reviewed for understanding and signed
as consent to volunteer in the study. Participants were informed they did not have to answer any
question if they do not wish to without the need to explain their choice. Participants were
volunteering and were informed they could withdraw from the study if at any point they felt
uncomfortable or upset. I would have respected his/her wishes without challenge.
3.4 Research Procedures
3.4.1 Recruitment of participants. Participant criteria for the study were that they had
worked as second language instructor within a high-stakes university or college setting; held a
Master’s degree in second language teaching; self-identified as an advocate of current
Communicative Language Teaching (CLT) principles; and, were willing to volunteer their time
to be part of the research study. I distributed the Participant Recruitment Email Brochure
(Appendix C) to each viable participant that I could determine from online searches at
educational institutions across Canada.
I widened my search to universities world-wide due to insufficient numbers of Canadian
participant commitment to the study. The interviews were from approximately 45 to 60 minutes
in duration. I did not require additional email discussion with any of the participants. I conducted
five interviews before obtaining sufficient saturation of the data.
The interview process was divided into two sections with the initial section structured to
draw out data on the instructional background of each participant and the specific context in
which each worked. The second section of the interview process garnered information relating to

36

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
the lived experience of the participants’ regarding the influence of summative assessment and
stakeholders on their instruction.
3.4.1.2 Voluntary participation. The participants in this study were chosen to contribute
on a purely voluntary basis. Any participant could have withdrawn from the study at any time
without consequence. If a participant did choose to leave the study, all information collected
from that participant would have been deleted or shredded.
3.4.1.3. Recruitment process. I began the recruitment process by analyzing the publicly
available information from the language departments in Canadian universities. I searched online
and identified viable candidates and contacted their department chair(s) to assist with distributing
the Participant Recruitment Brochure (Appendix C). The brochure stated the research questions,
research opportunity, research purpose, research goals, and the participant commitment
necessary to successfully accomplish the study. I set up a participant recruitment booth at the
Teachers of English as an Additional Language (TEAL) conference in Vancouver held on 18
November 2017 to find enough candidates to perform the study due to lack of response through
the previous attempt. I eventually had to widen my participant search to include international
universities and colleges.
3.4.1.4 Consent. Each interested viable participant was sent the Participant Consent
Form (Appendix D) for review and signature indicating they wished to participate. Interviews
did not commence until I obtained the signed consent form.
3.4.2 Data collection process. The data collection process began with finding eligible
participants and collecting the signed consent forms. The next phase included the scheduling of
interviews. Each participant interview was approximately 45 to 60 minutes via the Microsoft
Skype application. I reminded each participant about using the Sync cloud system for future

37

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
contact and discussion after completion of their interview. The purpose of follow-up discussion
was to enable further engagement with participants should themes emerge that required
additional input.
3.4.2.1 Data collection and storage. The research participant interviews were conducted
through the Microsoft Skype application and the audio was recorded with the MP3 Skype
Recorder application. I did not conduct any face-to-face interviews. However, I had prepared to
do so in a quiet public space such as a coffee shop and by using the Olympus VN-541PC Digital
Voice Recorder. I did not utilize a transcriber. I would have employed a research transcriber if
unforeseen time constraints had arisen. A transcriber would have signed the Confidentiality and
Non-Disclosure Agreement Form (Appendix E) if he/she was needed for the research study.
Names and location identifiers were replaced by a pseudonym. Physical and electronic copies of
transcripts and recordings were stored in a locked filing cabinet at the residence of the
researcher. Electronic files were accessed only on a password-protected computer. The data was
used for writing this thesis and future publishable academic articles. All information will be
deleted and shredded by May 31, 2028.
3.5 Data Analysis
Patton (1999) stated qualitative analysis is fundamentally a creative process that relies on
the insights and conceptual capabilities of the analyst in recognising patterns, linkages and
plausible explanations. Patton maintained in order to have credibility and validity the analysis
must also show that it is analytically rigorous, mentally replicable, and explicitly systematic
(Patton, 1999).
3.5.1 Triangulation. Triangulation of methods of data collection and analysis strengthen
the research study (Patton, 1999). Patton (1999) stated “The logic of triangulation is based on the

38

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
premise that no single method ever adequately solves the problem of rival explanations” (p.
1192). This research study consisted of a combination of interview and document qualitative
analysis. The kinds of triangulation this study permitted were triangulation of sources in which
curriculum documents were examined for consistency with interviews and triangulation of
perspective in which I interpreted the data through multiple perspectives. Patton (1999) noted
that inconsistencies that arise through the process of triangulation should not be viewed as a sign
of weakness in the credibility of the results, but rather as offering insights into the complex
relationship between inquiry approach and the phenomenon under study.
3.5.2 Components of data analysis. The components of data analysis consist of coding,
categorization and conceptualization (Richards & Morse, 2013). Raw data is organized into
ideas while depicting what is going on through coding. The process of categorization begins after
the data is coded. Researcher categorization reveals concepts, themes, and patterns which
eventually lead to conceptualization for the researcher. At this point the study is moved from
description to analysis and the researcher moves beyond identification of patterns toward
exploration of possible concepts. The researcher’s knowledge, informed by literature, enables the
interpretation and explanation that derives and justifies understanding (Richards & Morse, 2013).
3.5.3 Data analysis process. The data analysis process of this research study started after
all the participants had verified and approved their interview transcripts. I then went through the
abstraction process of coding, categorization and conceptualization for each interview transcript.
I subsequently created a codebook of the illuminated themes and concepts derived from each
participant’s interview transcript (Appendix F though J).
I compared and contrasted themes and concepts from all participant interviews to identify
larger categories, themes and concepts with the goal of illuminating the essence of the

39

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
phenomenon and drawing out practical advice. I summarized the analysis of the data, outlined
the findings and presented the resulting implications and recommendations that are all
culminated in this particular thesis report.
3.6 Research Credibility
Thorne (2008) proposed that researchers consider the criteria of epistemological integrity,
representative credibility, analytic logic, and interpretive authority in order to address rigor in the
qualitative nature of interpretive description studies. Leininger (1994) argued that the
quantitative criteria of validity and reliability are not well suited to substantiate qualitative
findings as they serve the quantitative needs of the logical positivism ideology used in a
scientific method.
3.6.1 Epistemological integrity. The researcher must write the research question and
interpret the data in accordance with the stated epistemological standpoint of the study (Grey,
2013). An epistemological standpoint denotes the nature of knowledge and how it is acquired
(Waters & Mehay, 2010). As noted above, I conducted this study from a subjectivist standpoint,
which maintains the fundamental position that knowledge cannot exist without individuals to
construct it.
3.6.2 Representative credibility. A qualitative study should demonstrate representative
credibility. The claims the study makes should be consistent with the manner the study was
sampled in order for a qualitative study to show representative credibility (Thorne, 2008). The
representative sample of this study consisted of contemporary communicative language
instructors teaching in high-stakes settings.
3.6.3 Analytic logic. A study should show evidence of the inductive reasoning process to
be analytical. Erlandson (1993) advocated an adherence to the principle established by Lincoln

40

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
and Guba (1982) in which the researcher creates an audit trail in which another researcher or
inquiry auditor could follow the reasoning process. The audit trail establishes the dependability
of a qualitative study and the confirmability of its data and conclusions (Lincon & Guba, 1982).
Chapters 4 and 5 detailed the researches inductive reasoning process.
3.6.4 Interpretive authority. The study should demonstrate an interpretive authority.
Interpretive authority is the assurance that the researcher’s interpretations are trustworthy,
revealing some truth beyond his or her own bias and experience (Thorne, 2008). The context of
the researchers experience and bias were outlined in sections 1.5. I reflected on my own biases
and assumptions by maintaining a journal of reflections throughout the research process. During
the interview process I made an effort to ensure my questioning was impartial and I limited my
conversational input as much as possible.
3.7 Summary
This study followed a qualitative research design methodology to explore the lived
experience of the phenomenon of assessment and stakeholder influence on advanced second
language instructors. The study utilized the interpretive description approach to further advance
the research goal of collecting useable knowledge for the advancement of curricula inclusive of
contemporary communicative language teaching (CLT) practices. I paid attention to and was
cautious of participant confidentiality and anonymity throughout the research process.
All sensitive data was stored on password-protected systems or kept in a lockable cabinet
accessible only to those formally permitted access to the data. Data will be deleted and/or
destroyed by May 31, 2028.
Data analysis was carried out with attention to maintaining credibility. I took into
consideration epistemological integrity, representative credibility, analytic logic, and interpretive

41

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
authority throughout the data analysis process. The following chapter discusses themes from the
data analysis.

42

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Chapter Four: Data, Themes and Discussion
This chapter is broken down into two subsections: a) data collected from the interview
process on the instructional background of each participant within the specific context in which
each worked; and, b) the emerging themes garnered from the interview process relating to the
lived experience of the participants’ dealing with the influence of summative assessment and
stakeholders on their instruction. The categorised information is presented in each section in this
chapter and then discussed in relation to the reviewed literature.
4.1 Professional Context of Research Participants
I asked the participants questions to illuminate the instructional context in which they
work. Information was gathered regarding the following topics: a) the instructional background
of the participants; b) the general interpretation of the principles governing Communicative
Language Teaching; c) the learning context; d) the learner’s background context; and, e) the
assessment structure utilized at the participants place of employment.

43

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
4.1.1 Instructional background of participants.
Table 2
Instructional Background of the Research Participants
Participant
Gary

Location
University, Canada

Employment experience
Gary was a language instructor with over 10 years of
experience and an assessment developer. Previously he had
the role of associate chair of language training. He has a
Master's of Educational Psychology Degree

Jeffery

University, Canada

Jeffery was a language instructor with over 10 years of
experience. He had the additional role of curriculum writer
and support. He has a Master’s degree in English Literature
and Film studies, and a Certificate in TESOL.

Jaden

University, Canada

Jaden was a language instructor with over 10 years of
experience various post-secondary institutions around the
world. He was on a committee that creates assessment. He
has a PhD of Education with a focus on assessment, Master
of Applied Linguistics and was an IELTS examiner. He
was instructing IELTS Prep classes utilizing only a little
CLT. However, he had extensive experience in the gatekeeping proficiency-based programs offered at his
institution.

Tim

University, U.A.E.

David

University, Qatar

Tim was an instructor with over 30 years of experience in
language instruction. He had instructed under alternative
and standardized assessment structures. He has a Master’s
degree in the field of TESOL.
David was an instructor and coordinator of an academic
writing gatekeeping program. He had extensive experience
in TESOL instruction. He has a Master’s degree in the field
of TESOL.

I was grateful to be able to interview a group of professionally upstanding second
language instructors. Each participant had many years of experience and was highly qualified in
the area of Teaching English to Speakers of Other Languages (TESOL). They all instructed
English for academic purposes in a university setting either in Canada or the Middle East. Most
of the participants had advanced in their careers accepting additional duties such as curriculum

44

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
development, assessment development, and program coordination. Each participant clearly
articulated the complexities of their experience with washback and stakeholder influence within
their professional context in addition to their professional portfolios and speaking about their
knowledge in the area of contemporary Communicative Language Teaching (CLT). Their
credentials clearly placed them as advanced practitioners in their fields.
4.1.2 General interpretation of the principles governing communicative language
teaching. I asked each participant to outline the major principles they use to guide their
instruction. Gary described the informed eclectic instructional approach that guides his second
language instruction. He stated he kept up with related research in the field. Gary’s instruction
combined a focus of forms with task-orientated instruction. Jeffery’s interpretation of CLT was
based around communication rather than accuracy. He used meaningful tasks to overcome the
inauthentic context of the classroom. He found instructor/student rapport essential in developing
authentic communication with learners. Jeffery saw the role of the language instructor to also
include enabling students to self-question adopted learning strategies. Jaden favored the use of
authentic texts in his teaching. He maintained a student-centered class. Jaden ensured student
buy-in by giving students the reasoning behind classroom activities. He stated that grammar
teaching made up part of class structure at least in the beginning. Tim advocated studentcentered methodologies that enable learner encoding of language rather than decoding
methodologies that break language down into teachable items. David refrained from describing
his interpretation of CLT.
Each participant’s description of the principles that guided their instruction correlated
with Jacobs’ and Ferrell’s (2003) account of the current thinking in CLT (see section 2.1.3) and
with Kumararvadivelu’s (2003) description of the post-method era where language instructors

45

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
need to become strategic practitioners with the ability to read and reflect on the dynamic nature
of the communicative language classroom. Tim was the only instructor who advocated
approaches exclusively guided by the no-interface position (see section 2.1.2). The other
participants endeavored to include more traditional elements in their instruction such as direct
grammar instruction and role-play activities. It was clear that these traditional elements did not
lead their instructional curriculum. In my view, all participants could be classified as reflective
practitioners of CLT (as outlined in section 2.1.5).
4.1.3 Learning context. I asked each participant to describe the context in which their
instruction took place in order to substantiate the high-stakes nature of their instructional
situation. Jeffery was part of the gatekeeping program for second language learners wanting to
enter the university. Gary’s instruction took place in the context of an English for Academic
Purposes (EAP) high-stakes environment where many students were under the threat of losing
funding. He was required to instruct EAP and the Language Instruction for Newcomers to
Canada (LINC) program to students simultaneously. A lot of acculturation education took place
at the start of the English as a Second Language (ESL) program to ensure students would be able
to function in universities within the Canadian context at the undergraduate level in Jaden’s
institute. Tim instructed a foundation program where there was a strong standardized exam focus
that created a teaching to the test context. Once learners passed the foundation program they
continued their English language studies alongside their undergraduate courses. Tim’s previous
employment context was strongly student-centered utilizing a purely alternative assessment
structure. He stated “I just worked on what I saw. Authentic needs of the students, so I start
interacting with them, I find what their problems are and I develop something based on that as
we go along”. David instructed the second level writing course at his institute in which passing

46

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
students gained full access to the academic program. The curriculum at his institute was recently
revised to add more time for teaching academic skills.
Alderson and Wall (1993) stated that the extent of washback will be greater in tests with
perceived important consequences as is the case with high-stakes testing. All of the participant
instructors’ courses were gate-keeping courses as learners that failed were denied full entry to
academic programs. Two participants noted additional pressures on their learners that served to
increase the high-stakes nature of their language assessment. Gary pointed out that many of his
learners are additionally under the threat of losing funding for their studies. Jeffery noted that for
many students cultural and family pressure to pass the test can be immense.
4.1.4 Learners’ background context. The research participants were questioned about
the wider context of their students. Jeffery noted that extrinsic learning was expected and valued
exclusively by learners that came into the program and as such learners had a strong focus on
passing tests. He noted that learners were under great pressure from their families to get to the
degree program as soon as possible; many students entered the program with an inflated belief of
their language proficiency level; and, learners did not know how to successfully learn a second
language. Jeffery mentioned that when students received a failing grade or were placed in a
lower level than they expected they generally externalized the blame with a focus away from
language proficiency. All of the participant instructors helped students come to the realization of
their proficiency level by enabling learners to see what was needed in terms of language
outcomes. Gary noted that the language learners he interacted with generally did not have an
adequate understanding of CLT. He stated that learners tended to arrive with an expectation of a
textbook focus, an expectation of teaching to the test, and a belief that grades are a negotiation

47

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
between instructor and learner. When Gary’s students did not find what they expected, many
became frustrated.
Alderson and Wall (1993) stated that due to extraneous factors assessment washback
effects individual learners and teachers differently. The participants’ experience was consistent
with Huang’s (2005) study which found that that Chinese students tend to question the value of a
classroom that focuses on discussion rather than lecture; deviates from following a textbook; and
emphasizes group work.
4.1.5 Assessment structure utilized at place of employment. Each participant outlined
the assessment structure of their institution. Jeffery’s institute utilized standardized assessment
mixed with horizontally uniform in-class standardized testing. Gary’s institution employed the
Canadian Language Benchmarks (CLB), which were modified in-house for their particular EAP
(English for Academic Purposes) context. The benchmarks were modified to include the nonholistic grammatical and vocabulary competencies as the holistic nature of CLB was considered
insufficient to assess readiness for entry into undergraduate study. Standardized assessments
were aligned to the benchmarks. Gary mentioned the rubric for the standardized test attempted to
avoid holistic interpretations by focusing on grammatical forms and in-class isolated skill-based
quizzes comprised a small part of the summative grade. Jaden’s institute utilized a purely
standardized assessment structure utilizing established themes developed in-house with a focus
on English for Academic Purposes (EAP) that included higher-order thinking tasks and
questioning. He stated that in-class standardized tests made up a small portion of the final grade,
but were not subjective in nature. Tim’s institute was strongly standardized where students were
prepared for one standardized test with no use of in-class summative grading. A purely
alternative assessment system was used at Tim’s previous employment location and instructors

48

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
utilized learner-generated portfolios to derive the summative grade. David instructed the writing
skill course that utilized standardized testing of writing tasks that were graded with non-holistic
rubrics. These rubrics itemized academic competency and language mechanics separately. The
writing course utilized mid-terms and in-class quizzes to make up part of the final grade.
The curricula at Gary’s place of employment and Tim’s previous place of employment
were consistent with Spady’s (1994) description that an Outcomes Based Education (OBE)
framework focuses on what students can actually do after they are taught or have learned and
VanPattern’s (2015b) contention that in TESOL this translates to a set of can do statements
which illustrate what a language learner can do with language in the real world. Standardized
testing prevailed in all of the participants’ educational settings despite Moya and O’Malley
(1994) stating that process-orientated curricula and instruction associated with CLT is largely
viewed as being incongruent with product-orientated assessment and as such standardized
testing.
Five of the six instructional contexts examined in this study can be equtated with
Hargreaves and Shirley’s (2009) third phase of general education (section 2.5) in that
professional autonomy was balanced with accountability. Tim’s previous employment can be
equtated with Hargreaves and Shirley’s fourth phase of general education (section 2.5) in that
responsibility was promoted over accountability.
4.2 The Experience of Dealing with Assessment and Stakeholder Influence
Each participant discussed how they experienced the pressure stemming from assessment
on their instructional practice. Influence from assessment and stakeholders that shaped the
instructional context was discussed. Seven major themes emerged from the information: a)
Experience of assessment alignment; b) Experience of student influence on instruction; c)

49

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Experience of academic course instructor influence on instruction; d) Experience of
institutional/administrative influence on instruction; e) Experience of alternative assessment
washback; f) Experience of assessment washback; g) Overall effects of influences on instruction;
and, h) Instructional desires.
It was obvious that each of the participants was dedicated to the successful learning of the
students as they ardently articulated their experience with assessment and stakeholder influence.
It became clear that the complexities of human influence lay at the root of many of the longstanding issues they experienced.
4.2.1 Experience of assessment alignment. Most of the participants did not find that the
standardized assessments were significantly out of alignment with Communicative Language
Teaching (CLT). Jeffery had a positive experience with standardized assessment in finding that it
worked well with CLT as long as CLT was introduced and conducted correctly. He noted that
formative assessment helped to pull learners toward the wider goal of improving academic
language proficiency. Gary noted that standardized assessment was achievable only when
created and maintained by those qualified in assessment. He determined that the advantage of a
well aligned standardized test was that it helped to isolate gaps that need addressing. Jaden found
in-class tests particularly advantageous as they are more easily aligned to classroom practices.
He found that topics could be tailored to learner needs and interests, and that the reasoning
behind in-class testing became clearer. David sought to improve standardized assessment
alignment through consistency with the themes and topics of the undergraduate courses. He
found that as the assessment had been sufficiently aligned, teaching to the test successfully
becomes part of the course structure. David also believed that team work among instructors
helped to overcome subjectivity issues in standardized rubrics.

50

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Tim believed standardized testing, in itself, induced artificial learning environments
which prevented the alignment of assessment to CLT. Tim noted that instructors end up teaching
grammar directly as it is an inherent need of the standardized test. Teaching grammar directly
happened despite his institute not giving instructors access to the standardized test as a measure
to ensure instructors avoided teaching to the test.
An overall acceptance of standardized assessment as a successful tool was observed from
most of the participants in the study. These four participants indicated that creating a valid and
reliable standardized test that was aligned with CLT was difficult, but could be adequately
achieved. Every participant’s place of employment had or was about to undergo curricular
change indicating a possibility that all of the participants’ institutions had not yet obtained an
adequate standardized assessment structure. This fact would endorse Coombe et al. (2012) claim
that standardized tests strive for improved reliability and validity in the face of the multitude of
confounding factors inherent in the testing of communicative tasks.
4.2.2 Experience of student influence on instruction. Each participant depicted their
experience with test-focused learners and how they push for more traditional instructional
approaches as beneficial to their short-term goal of passing the test. Two participants noted that
in some cases this pressure was extreme, amounting to students harassing teachers or dropping
their courses in favor of self-study. Jeffery found that learners tended to look for shortcuts to
passing tests as they see ESL as secondary to academic studies in stating “they just want to get to
it”. He complained that learners see grammar instruction as a shortcut and pressure instructors to
teach this way. Jeffery noted that, in extreme cases, learners will even drop a course to study for
the IELTS test if they feel their test focus is not being met by the instructor. Jeffery mitigated
learner pressure by ensuring that learners bought in to the approaches he utilized in class. He

51

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
found that student buy-in can generally be achieved if CLT is introduced correctly, with more
success if the learner comes from a Canadian-based curriculum. Gary mentioned his students’
frustrations of experiencing a different learning environment caused them to pressure the
instructor to move toward the traditional teaching and assessment practices they are familiar
with. These frustrations include: a lack of understanding of an outcomes-based curriculum; an
inflated belief in their proficiency level; a belief that their grade is negotiable; an expectation of
being taught to the test; and an expectation of a textbook focus. He stated that the coercion from
students was intense and at times escalated to the verbal bullying of instructors, which caused a
lot of stress for instructors. Assigned student advisors also often gave learners poor advice that
aligned with their misguided belief of their learning situation. Tim felt that learner pressure
combined with a standardized assessment structure and institutional dynamics had the effect of
negating the successful implementation of CLT. Tim noted that within the alternative assessment
orientated curriculum in his previous place of employment students did not resist CLT once they
were able to see that they were actually improving their language proficiency. Jaden did not
experience significant student influence on CLT. He stated “My students are from China and I
think that [the] common perception is ‘teacher knows best’ and they decided to come to Canada
because of the change in teaching style”. Jaden stated that in order to minimize learners’
insistence for teaching to the test, learners needed to know the reasoning behind assessment:
You can't do one without the other. If the test is good but the students don't
know why they are doing the test or the students don't notice the goals of that
test or what the test is doing, then it doesn't work because it doesn't motivate
the students. The students don't have the realization that there could be a
positive washback to testing.
David stated that his writing assessments were aligned with CLT and that he did not experience
significant learner washback.

52

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Learner influence on instruction via pushing for the traditional teaching practices they are
familiar with and by insisting their instructor teach to the test was a major concern for most of
the participants. All participants felt that the effects of student influence could be overcome
through the following solutions: a) Ensuring buy-in from students by introducing CLT correctly
and giving the reasoning behind classroom activities; b) Ensuring an understanding of why they
are doing the test and what the goals of the test are; c) Providing institutional support of CLT,
and, d) Ensuring improved alignment. These findings support the small amount of research
reviewed in section 2.4.5 on the influence of assessment on student learning, which generally
finds that students tend to fall back on traditional learning strategies such as studying grammar
patterns, despite the communicative orientation of any assessment.
4.2.3 Experience of academic course instructor influence. Each participant described
the pressures that came to bear from instructors in the undergraduate programs in which the
passing language learners advance into. Most of the participants commented on issues generated
from a lack of understanding of what academic language learning involves and how long it takes.
Gary noted that academic course instructors do not see their role as facilitators of communication
for language learners, do not respect language departments on equal terms as academic
departments, and expect unrealistic language proficiency gains from the time learners spend in
the language program. He highlighted his point by recalling a conversation in which he had to
convince preservice high school teachers of their responsibility to teach all of their students
inclusive of their ESL students how to effectively communicate within their subject area.
Most of the participants spoke about the importance of their communication with
academic course instructors. They regarded academic course instructor feedback as invaluable
information that could be used to improve the course and assessment structures. Gary further

53

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
noted that academic course instructor feedback was important in reducing tension between
academic course instructors and second language instructors. Jaden outlined an example of
important information coming back from academic course instructors, which was their desire for
learners to be able to communicate in higher domains of thought. He noted academic course
instructors were less concerned with English language learners’ inaccuracies in grammar and
lack of advanced vocabulary.
The research findings indicated that academic course instructors generally question the
integrity of ESL instructors. Nevertheless, the participants valued academic course instructors’
feedback when it could be used to improve the course and assessment structures. To the best of
my knowledge no research has looked into the area of academic course instructor influences on
the delivery of preparatory programs.
4.2.4 Experience of institutional/administrative influence. All participants lamented
the lack of understanding by the administrative staff of the unique dynamics an ESL program
brings. Jeffery felt that the business model pressures the program to push students through when
they are not yet able to succeed in an academic environment. He also noted that the complexity
in the hierarchical structure of the ESL department within the university structure caused
difficulty in communication with stakeholders. Gary similarly stated that within his institution
there was little understanding and respect for CLT and the structure needed to support it. He
noted that his university structure formally designated a lower professional status for language
instructors. This amounted to an intense and stressful relationship within the university. Gary
also stated that there was a lack of institutional understanding of how stakeholders influenced the
curriculum, and a lack of institutional understanding of the time required for academic language
proficiency to develop. In Jaden’s experience, the administration endeavored to limit academic

54

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
freedom for standardization. “Their justification is that we have so many new teachers and that
the new teachers need to be on board; it's easier if everything is standardized”. He continued,
“Management frowns on alternative testing because they are more worried about students
comparing their assessment techniques and complaining about it”. Overall, he found
management to be unsupportive of CLT in practice:
I find that there is less and less academic freedom, because it's easier to have
ten teachers teaching exactly the same thing. It's easier for assessment of both
the teacher and the students and I think that works a bit against
Communicative Language Teaching because you don't have the freedom to
do what you want based on the students’ needs and desires.
David’s frustrations with the difficulties in coping with management caused him to surmise that
management had a surface level understanding, looking only at student pass and fail rates as they
did not understand the process of second language instruction. His frustration included
management insisting on testing on different days with multiple versions with little regard for the
difficulties in test creation. Tim found that management overestimated what could be obtained by
learners. He postulated that institutional and student influences act in unison: “It's not just the
students, it's the institution as such. [It] has the effect of negating the implementation of CLT”.
Tim summed up the feelings of all participants with his poignant comment: “If
management would actually understand. We need somebody who knows [communicative]
language teaching at the top”.
The lack of understanding departmental management had of the unique dynamics an ESL
program brings created an intense and stressful relationship between the participants and the
university intuition. The participants found their institutions to be in practice unsupportive of
CLT. To my knowledge no research on the effect of an uninformed management structure on
CLT has been conducted.

55

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
4.2.5 Experience of alternative assessment washback. All of the participants had had
some experience with the introduction of alternative assessment as part of summative assessment
or, as in Tim’s experience, the entirety of assessment. All participants felt that incorporating
alternative assessment to be of value, but cautioned in its implementation in stating that the right
context needed to be present for it to be effective. Jeffery stated that he believed there is a place
for alternative assessment. He asserted that instructors must be careful that language is assessed
in terms of grammar, vocabulary, and language structure. Gary described his previous experience
with alternative assessment as a “hot mess”. He stated that “instructors were not capable of
creating and assessing tasks”. He found that the lack of understanding and implementation of
learning tasks lead to strong negative student washback that amounted to verbal harassment at
times. He added the notion that accountability is hard to achieve with the inconsistent nature of
alternative assessment. Gary’s opinion revealed alternative assessment is not viable without
knowledgeable qualified instructors and with institutional support. Jaden found the standardized
assessment structure at his institution to be effective. He found this was the case as for many of
his students, learning English is seen as an obstacle rather than a goal, and the tests provide some
extrinsic motivation. He viewed his experience with alternative assessment as precarious because
of its subjective nature. He cautioned that if alternative assessment becomes standardized then it
defeats its purpose. Despite Jaden’s stance on alternative assessment he desired a less
standardized assessment environment in stating, “Our mandate is not just about language, it's
about academic readiness to enter university. I think teachers should be allowed to select their
expert judgement in those cases and because of over standardization we are losing some of that”.
Tim noted all teachers need to be following same approach for alternative assessment to be
effective. He felt CLT was successfully able to be implemented based on his instructing

56

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
experience in a fully alternative assessment-based curriculum. David’s institute had
unsuccessfully experimented with introducing alternative assessment through portfolios. He
stated that alternative assessment was abandoned as management, parents and students all
complained. He commented, “The people here don't support that type of alternatives that much.
Especially the top management”. Nevertheless, alternative assessment continued for some
instructional activities such as blogging, albeit with very small summative grading.
The participants’ descriptions of their experience with alternative assessment indicated
that they all felt that alternative assessment is not viable without knowledgeable instructors that
act in a professional manner and institutional support because of its subjective nature and
inability to hold instructors accountable. Moya and O'Malley (1994) stated that alternative
assessment is largely a qualitative approach and subjective in nature, thus the test validity and
reliability tend to become difficult to establish and maintain. This finding correlates with all of
the participants’ expereinces.
4.2.6 Experience of assessment washback. Of particular note is that a theme of direct
washback did not emerge during the interview process. None of the participants felt their
instructional approach as was directly driven in any way by the assessment structure. The claim
that washback from traditional testing structures deters teachers from adopting current
methodologies (Jacobs & Farrell, 2003; Moya & O’Malley, 1994) and the findings from studies
relating to washback on teaching strategies, methodologies and approaches appear not to apply to
skilled communicative language instructors.
4.2.6 Overall effects of influences on instruction. Several wider effects of washback
and stakeholder influence were illuminated through the interview process. The fact that all of the
participants spoke of recent or upcoming changes to the assessment structure at their place of

57

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
employment is worth noting. Jeffery described what he felt as an underlying uneasy dualism
between communicative outcomes and language outcomes, which complicates language
instruction. Gary stated that his unhealthy working environment where instructors were unable to
perform optimally stemmed in part from the fact that management did not seriously consider
washback and assessment influence. Tim noted that standardized assessment structures
constantly are rearranging/changing due to the socio-economic triangle of competing pressure –
Institution, instructor, and learner. He outlined how this socio-economic triangle plays out:
A student will have certain expectations and expect the teachers to meet those
expectations, so obviously teachers modify their way of teaching, their
approach to teaching. They betray their own way of understanding of how
they should learn to simply meet both the management and the student.
He added:
I know what I need to do as a teacher for you to learn. So how can I meet
your expectations given my whole knowledge and at the same time I'm
keeping management happy and eventually they will keep paying me to keep
doing my work. So, it's a tricky, it's a tricky little thing. It's complicated. Most
of us just end up betraying ourselves basically.
Tim’s experience with a fully integrated alternative assessment curriculum led him to believe
that there is a deeper instructor satisfaction with an alternative assessment based system.
4.2.7 Instructional desires. I asked what each participant desired to support their
language learners’ paths to success toward the end of each participant interview. Jeffery voiced
his desire for language supports for language learners for the duration of their educational
journey. Additionally, he wanted to see communication improved between all stakeholders:
There has to be really good communication between the instructor and the
student. But also between the program and the students coming in. Students
[that] are enrolling in your program should have a very clear understanding of
what they are going to learn, why they are going to learn, and how this
learning is going to help them meet and fulfill their future learning goals.
Then washback in assessment can be maximized for benefit.

58

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Gary wanted to see more reflection system-wide and a confidence to take on any systemic
changes that are required. He noted instructors did not want to teach to the test despite pressure
to do so from stakeholders. He would have liked to see academic course instructors understand
their role in ongoing communication development for language learners, greater learner
involvement in curriculum, and more class hours for instruction and assessment. Gary summed
up his frustrations with his working environment:
Our leadership doesn't seem interested in asking the questions or finding the
answers to the type of questions that you’re asking me today that are really
important. I think that is a very crucial, critical missing piece to what is
happening at least with our institution, possibly within many others.
Jaden would have liked to see less standardization and professional development in assessment
for instructors and management. Finally, Tim stated, “If management would actually understand.
We need somebody who knows language teaching at the top. You know, that actually knows”.
4.3 Conclusion
This chapter presented the findings of the participants’ lived experience of assessment
and stakeholder influence on their instruction. The findings were then discussed in relation to the
current literature. The study’s findings were unique in the fact that they stemmed from a study
controlled for highly experienced and qualified Teaching English to Speakers of Other
Languages (TESOL) instructors working within a high-stakes environment. I found these factors
to be lacking in much of the current research on the phenomenon of washback in a TESOL
setting.
The findings showed that, in the cases observed, the combined effect of washback and
stakeholder influence results in a continually changing, unhealthy working environment in which
skilled CLT instructors are unable to perform optimally. Tim summarized the findings well:

59

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
I know what I need to do as a teacher for you to learn. So, how can I meet
your expectations given my whole knowledge and at the same time I'm
keeping management happy and eventually they will keep paying me to keep
doing my work. So it's a tricky, it's a tricky little thing. It's complicated. Most
of us just end up betraying ourselves basically.
Contemporary communicative language instructors are tasked with embracing
professionalism within this unconducive environment as outlined in Kumaravadivelu’s (2003)
call to go beyond relying on mechanical teaching strategies and methods, and towards critical
thinking about practice (see section 2.1.5).

60

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Chapter Five: Implications and Recommendations
The aim of this phenomenological research study was to elicit practical knowledge for
the advancement of effective second language learning curricula in a high-stakes environment.
The study was intended to be of interest to language institutions that strive for excellence and for
skilled instructors seeking to promote systemic change toward conditions that facilitate optimal
language acquisition.
This chapter examines the potential implications and highlights several recommendations
for language institutions to take into consideration for the implementation of a curriculum
inclusive of contemporary Communicative Language Teaching (CLT) practices. It is not my
intent to generalize the findings into the wider Teaching English to Speakers of Other Languages
(TESOL) field. These findings may enlighten and benefit others in the TESOL field who can
relate to the experiences of the participants and inspire much needed further research.
5.1 Study Implications
By comparing and contrasting the findings with the existing literature I drew out four
significant implications for curriculum development in high-stakes ESL settings. The section is
divided into the following implications: a) Standardized assessment is difficult to implement
successfully; b) Alternative assessment should be incorporated with caution; c) Direct washback
on instructional approach is negligible, and; d) Stakeholders’ influence impedes successful
implementation of CLT. The implications indicate that there is no simple solution to the issues
raised.
5.1.1 Standardized assessment is difficult to implement successfully. Moya and
O’Malley (1994) stated that standardized assessment is product-orientated and as such cannot be
fully aligned to the process-orientated instructional curricula required in CLT. Their claim

61

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
implies an instructional system that encourages direct teaching to a standardized test is not viable
in the CLT context. This study, however, claims that programs can utilize standardized
assessment successfully provided the assessment is adequately aligned to the outcomes. Most of
the participants’ experiences led them to understand that creating adequately valid and reliable
assessment is a difficult task that requires specialized training.
5.1.2 Incorporate alternative assessment with caution. Alternative assessment is
largely a qualitative approach and subjective in nature thus the test validity and reliability tend to
become difficult to establish and maintain. Maintaining validity and reliability can lead to a
potentially costly and time-consuming situation creating the impetus for many institutions to
avoid incorporating alternative assessment strategies (Moya & O'Malley, 1994). The
participants’ experiences revealed in this study concurred with Moya and O’Malley’s claim. The
participants noted the following implications: a lack of contemporary CLT and assessment skills
with their colleagues; an institutional unwillingness to foster and trust professionalism of the
ESL instructors in the program; and an institutional unwillingness to reflect CLT professionalism
in the institutional structure. These findings made alternative assessment impractical for their
institutions to fully adopt.
5.1.3 Direct washback on instructional approach is negligible. The assessment
structure does not directly influence the contemporary communicative language instructors’
instructional approach regardless of its format. A more complex picture has emerged regarding
pressures from other stakeholders that make informed instruction problematic in that some
instructors choose to modify their instructional approach to the detriment of informed language
instruction.

62

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
5.1.4 Stakeholders’ influence impedes successful implementation of communicative
language teaching. The study revealed that learners who were unable to continue language
learning practices they were familiar with or did not see a strict adherence to teaching to the test
questioned the integrity of the language program and their instructor. The participants strived to
overcome this issue by promoting learner buy-in from students by introducing CLT and giving
the reasoning behind classroom activities. The participants were clear in stating there was no
institutional support given in overcoming this issue.
The experience of the participants revealed that academic course instructors and other
institutional stakeholders who did not see the proficiency gains in learners that they expected
tended to question the integrity of the language program and the instructors. Management was
unable to react in a supportive manner as they were not sufficiently knowledgeable in CLT
practice in these cases.
The study also showed that academic course instructors and the institution, as a whole,
placed unrealistic expectations on the language program and failed to understand that Cognitive
Academic Language Proficiency (CALP) is a complex task that is generally recognized within
the field of SLA that requires five to seven years of instruction to develop (Roessingh, Kover, &
Watt, 2005). The findings also indicated that academic course instructors were not aware of or
knowledgeable in the role they had in developing second language learner communicative
ability.
Communication between instructors and the institutional stakeholders was strained with
the study participants experiencing a lack of professional respect and disregard for the unique
knowledge they possess. An intense and stressful relationship developed between the CLT
instructor and those in the university system above them.

63

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
This study revealed the combined effect of unchecked washback and stakeholder
influence that resulted in a continually changing, unhealthy working environment in which
skilled CLT instructors were unable to perform optimally. The net result was language programs
stagnating in mediocrity. I suspect skilled staff would seek alternative employment as it arises
due to this type of work environment.
5.2 Recommendations
The discussion of the findings and literature review point to several recommendations for
language institutions to consider for the implementation of a curriculum inclusive of
contemporary CLT practices.
1. Institutions need to invest in time and specifically trained professionals to undertake the
complex task of creating valid and reliable standardized assessments. Alternatively,
intuitions may choose to outsource all or part of their assessment by incorporating
established assessments such as the Cambridge suite.
2. Institutions need to empower instructors to teach to the outcomes of the test rather than
directly teaching to the test in order to ensure that the standardized test maintains its
validity. Empowerment could be achieved through: in-house training of Outcomes Based
Education (OBE) (Outlined in section 2.1.8); institutional support for students struggling
with the concept of OBE; and educational programs aimed at ensuring the buy-in of all
stakeholders to OBE.
3. Curriculum developers should incorporate alternative assessment with caution.
Alternative assessment should be incorporated to the degree that instructor
professionalism and institutional trust of that instructor professionalism allows.

64

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Employment of highly credentialed staff and consistent in-house training would be
invaluable to advance instructor professionalism and institutional trust.
4. Institutions need to facilitate professional communication between all stakeholders.
Communication between academic course instructors and English for Academic Purposes
(EAP) instructors needs to be respectful of the unique professional context CLT takes
place in. Learners enrolling in the language program should have a very clear
understanding of what they are going to learn, why they are going to learn, and how this
learning is going to help them meet their future learning goals.
5. Institutions need to place realistic expectations on the language program, given that
Cognitive Academic Language Proficiency (CALP) is a complex task that takes five to
seven years to develop with instruction (Roessingh, Kover, & Watt, 2005). Sufficient
academic language proficiency cannot realistically be achieved through an intense
language program alone. Institutions may overcome the shortfall by extending language
preparatory programs and by incorporating specifically designed language supports
throughout the learning pathway of the language learner.
6. Institutions need to define and make explicit the role academic course instructors have in
developing second language learner communicative ability.
7. Institutions need to foster professionalism through professional development of
contemporary CLT and language assessment creation. Professionalism is achieved with
high-quality teachers, positive and powerful professional associations, and lively learning
communities combined with indirect accountability measures such as test sampling,
which promote responsibility over accountability (Hargreaves & Shirley, 2009).

65

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
8. Management of language departments need to mitigate the questioning of CLT instructor
integrity by stakeholders as informed arbitrators. This would entail that management
knows and can articulate how principles emerging from the field of Second Language
Acquisition (SLA) constrain the practice of CLT.
9. Institutions need to consider the negative effects of the professional marginalization
experienced by CLT teachers within the university setting. Respect for the professional
practitioner role of the second language instructor needs to be fostered. MacDonnald
(2017) called for the embrace of a Third Space of professionalism within the Canadian
universities for the EAP sector. Fostering postmodern professionalism focused on
engagement, service and collaboration.
10. Institutional hierarchy needs to be informed in best practices in the field of CLT. I
recommend institutional stakeholders read the book While We're On the Topic: BVP on
Language, Acquisition, and Classroom Practice (2017) by Bill VanPattern. It is concisely
written to a non-technical audience and reasonably priced.
5.3 Future Research
This study was unique in the fact that it looked at influences that inhibit the successful
implementation of contemporary CLT in high-stakes settings. The lack of research in this
specific area indicates the need for further research to substantiate the study’s findings before
generalisations can be inferred to the field of TESOL. The research study revealed further
research is required in the areas of: a) Standardized assessment and alternative assessment within
an outcomes based framework; b) Stakeholder communication; c) The role of academic course
instructors in second language learning; d) Professionalism in TESOL, and, e) Stakeholder buyin.

66

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
5.4 Conclusion
The lived experiences of the five skilled instructors are testimonies to the need to break
away from the dystopian second language systems that many university language departments
inadvertently follow and embrace a structure that successfully supports contemporary
Communicative Language Teaching for the sake of the learner.
The significant finding of this study reveals the systems the participants work for do not
allow for advanced practice resulting in the frustration and alienation of knowledgeable CLT
practitioners. The revealed participant experiences did not find assessment to directly cause their
frustrations, regardless of the assessment type. The study revealed that this untenable situation
arose due to: the difficulty of successful implementation of standardized assessment; the
condition-dependent nature of alternative assessment; and the power of stakeholder influence to
impede successful implementation of CLT. The findings will assist to bridge the gap in the
understanding of the influences that affect the implementation of CLT in high-stakes settings and
set the stage for future research in the field of study.

67

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
References
Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback.
Language Testing, 13(3), 280-297.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115-129.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and
developing useful language tests. Oxford: Oxford University Press.
Barcroft, J. (2016). Vocabulary in language teaching. New York: Routledge.
Barnes, M. (2016). The washback of the TOEFL in Vietnam. Australian Journal of Teacher
Education, 41(7), 157-174. Retrieved from http://ro.ecu.edu.au/ajte/vol41/10
Berwick, R., & Ross, S. (1989). Motivation after matriculation: Are Japanese learners of English
still alive after exam hell? Japan Association for Language Teaching Journal, 11, 193210.
Biggs, J. (2003). Aligning teaching for constructing learning. Higher Education Academy, 1-4.
Brown, H. D. (1994). Teaching by principles: An interactive approach to language pedagogy.
Cliffs, N.J: Prentice Hall Regents.
Burns, A. (2003). ESL curriculum development in Australia: Recent trends and debates.
Regional Language Centre Journal, 34(3), 261-283.
Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect
inthe Australian adult migrant English program. Washback in language testing: Research
contexts and methods, 113-128. Retrieved from
http://faculty.mu.edu.sa/public/uploads/1347487499.950171132987-Washback-inLanguage-Testing.pdf#page=136

68

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
British Colombia Misinstry of Education. (2017). Building Student Success - BC's New
Curriculum. Retrieved from BC's New Curriculum: http://curriculum.gov.bc.ca
Carmines, E. G., & Zellar, R. A. (1979). Reliability and validity assessment (Vol. 17). Thousand
Oaks, Calif.: Sage publications.
Cheng, L. (1997). How does washback influence teaching? Implications for Hong Kong.
Language and Education, 11(1), 38-54.
Cheng, L. (1998). Impact of a public English examination change on students' perceptions and
attitudes toward their English learning. Studies in Educational Evaluation, 23(3), 279301.
Cheng, L. (2004). The washback effect of a public examination change on teachers’ perceptions
toward their classroom teaching. Washback in language testing: Research contexts and
methods, 147-170.
Cheng, L. (2005). Changing language teaching through language testing: A washback study
(Vol. 21). Cambridge University Press.
CIHR, NSERC and SSHRC (Canadian Institutes of Health Research, Natural Sciences and
Engineering Research Council of Canada, and Social Sciences and Humanities Research
Council of Canada). (2014). Tri-Council Policy Statement: Ethical Conduct for Research
Involving Humans (TCPS).
Collier, V. P. (1987). Age and rate of acquisition of second language for academic purposes.
TESOL Quarterly, (21), 617-641.
Coombe, C., Davidson, P., O'Sullivan, B., & Stoynoff, S. (2012). The Cambridge guide to
second language assessment. Cambridge University Press.

69

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Creswell, J. W., Hanson, W. E., Clark Plano, V. L., & Morales, A. (2007). Qualitative research
design: Selection and implementation. The Counselling Psychologist, 35(2), 236-264.
Cummins, J. (1979). Cognitive/academic language proficiency, linguistic interdependence, the
optimum age question and some other matters. Working Papers on Bilingualism, 19, 121129.
Cummins, J. (1981). Age on arrival and immigrant second language learning in Canada: A
reassessment. Applied Linguistics, 2, l32-l49.
Derince, Z. M. (2012). Reflections on teaching practices through conditionings in Turkey.
Journal for Critical Education Policy Studies, 10(1), 248-262.
Ellis, R. (2005). Principles of instructed language learning. Asian EFL Journal, 9.
Englund, T. (1996). Are professional teachers a good thing? In Goodson, I. F. (2002). Teachers'
professional lives.
Erfani, S. S. (2012). A comparative washback study of IELTS and TOEFL iBT on teaching and
learning activities in preparation courses in the Iranian context. English language
Teaching, 5(8), 185.
Erlandson, D. A. (1993). Doing naturalistic inquiry: A guide to methods. Sage.
Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and
learning. Washback in language testing: Research contexts and methods, 191-210.
Gadamer, H. (1998). Truth and method (2nd ed.). New York: Continuum.
Gardner, S. (2008). Transforming talk and phonics practice: Or, how do craps clap? Teachers of
English to Speakers of Other Languages, 42(2), 261-284.
Gass, S., & Selinker, L. (2001). Second language acquisition: An introductory course (2nd ed.).
Mahway , N.J.: Erlbaum Associates.

70

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Giroux, H. A., & McLaren, P. (1989). Schooling, cultural politics, and the struggle for
democracy. Critical pedagogy, the state, and cultural struggle. Retrieved from Retrieved
from http://www.sunypress.edu/pdf/50168.pdf
Green, A. (2007). Washback to learning outcomes: A comparative study of IELTS preparation
and university pre‐sessional language courses. Assessment in Education, 14(1), 75-97.
Grey, D. E. (2013). Doing research in the real world. Sage.
Hancock, C. R. (1994). Alternative assessment and second language study: What and why?
ERIC Digest. Retrieved from http://files.eric.ed.gov/fulltext/ED376695.pdf
Han, Z. and Finneran, R. (2014), Re-engaging the interface debate: strong, weak, none, or all?.
International Journal of Applied Linguistics, 24: 370–389.
Haney, W., & Madaus, G. (1989). Searching for alternatives to standardized tests: Whys, whats,
and whithers. Phi Delta Kappan, 70(9), 683-687.
Hargreaves, A. (2003). Teaching in the knowledge society: Education in the age of insecurity.
Berkshire, United Kingdom: Open University Press.
Hargreaves, A., & Shirley, D. (2009). The fourth way: The inspiring future for educational
change. Thousand Oaks, Calif.: Corwin Press.
Hargreaves, E. (1997). The diploma disease in Egypt: learning, teaching and the monster of the
secondary leaving certificate. Assessment in Education, 4(1), 161-176.
Hayes, B., & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the
IELTS academic module. Washback in language testing: Research contexts and methods,
97-111. Retrieved from
http://faculty.mu.edu.sa/public/uploads/1347487499.950171132987-Washback-inLanguage-Testing.pdf#page=120

71

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Hedgcock, J. S. (2002). Toward a socioliterate approach to second language teacher education.
The Modern Langauge Journal, 86(3), 299-317.
Hughes, A. (1988). Introducing a needs-based test of English language proficiency into an
English-medium university in Turkey. Testing English for university study (ELT
Document No.127 pp. 134-153). London: Modern English Publications.
Huang, J. (2005). Challenges of academic listening in English: Reports by Chinese students.
College Student Journal, 39(3), 553-569.
Jacobs, G. M., & Farrell, T. S. (2003). Understanding and implementing the CLT
(Communicative Language Teaching) Paradigm. RELC journal, 34(1), 5-30.
Johnson, K. E., Jordan, S. R., & Poehner, M. E. (2005). The TOEFL trump card: An
investigation of test impact in an ESL classroom. Critical Inquiry in Language Studies:
An International Journal, 2(2), 71-94.
Jinsong, F., Peiying, J., & Xiaomei, S. (2014). Washback of university-based English language
tests on students' learning: A case study. The Asian Journal of Applied Linguistics, 1(2),
178-191.
Klesmer, H. (1994). Assessment and teacher perceptions of ESL student achievement. English
Quarterly, 26(3), 5-7.
Krashen, S. (1982). Principles and practice in second language aquisition. Retrieved from
http://aces.ir/attachments/22d1286622494-communicative-approach-stephen-crashen.pdf
Kumaravadivelu. (2003). Beyond methods: Macrostrategies for language teaching. Yale
University Press.

72

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Lam, H. P. (1994). Methodology washback-an insider's view. In Bringing about change in
language education. The International Language in Education Conference (pp. 83-102).
Hong Kong: University of Hong Kong.
Laverty, S. M. (2003). Hermeneutic Phenomenology and Phenomenology: A Comparison of
Historical and Methodological Considerations. International Journal of Qualitative
Methods, 2(3), 21-35.
Lee, J. F., & VanPatten, B. (2003). Making communicative language teaching happen. Boston:
McGraw-Hill.
Leininger, M. M. (1994). Evaluation criteria and critique of qualitative research studies. In J. M.
Morse, Critical issues in qualitative research methods (pp. 95-115). Sage.
Li, X. (1990). How powerful can a language test be? The MET in China. Journal of Multilingual
& Multicultural Development, 11(5), 39-404.
Lincoln, Y. S., & Guba, E. G. (1982). Establishing Dependability and Confirmability in
Naturalistic Inquiry Through an Audit. Paper presented at the annual meeting of the
American Educational Research Association. New York.
Lumley, T., & Stoneman, B. (2000). Conflicting perspectives on the role of test preparation in
relation to learning. Hong Kong Journal of Applied Linguistics, 5(1), 50-80.
MacDonald, J. (2017). The Margins as Third Space: EAP Teacher Professionalism in Canadian
Universities. TESL Canada Journal, 34(1), 106-116.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241-256.
Mezirow, J. (1991). Transformative dimensions of adult Learning. San Francisco, CA: JosseyBass.

73

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Mezirow, J. (1997). Transformative Learning: Theory to practice. New Directions for Adult and
Continuing Education, 74, 5-12.
Moya, S. S., & O'Malley, J. M. (1994). A portfolio assessment model for ESL. The Journal of
Educational Issues of Language Minority Students, 13(1), 13-36.
Noeles, Clement, R., & Pelletier, L. G. (1999). Perceptions of teachers’ communicative style and
students’ intrinsic and extrinsic motivation. The Modern Language Journal, 83(1), 23-34.
Ontario Ministry of Educaetion. (2013). Learining for all: A guide to effective assessment and
instruction for all students, kindergarten to grade 12. Ontario. Retrieved from
http://www.edu.gov.on.ca/eng/general/elemsec/speced/LearningforAll2013.pdf
Pan, Y. C. (2014). Learner washback variability in standardized exit tests. TESL-EJ, 18(2), 2.
Patton, M. Q. (1999). Enhancing the quality and credibility of qualitative analysis. Health
services research, 35(5 Pt 2), 1189-1209.
Pink, D. H. (2011). Drive: The suprising truth about what motivates us. New York, N.Y.:
Riverhead Books.
Qi, L. (2005). Stakeholders' conflicting aims undermine the washback function of a high-stakes
test. Language Testing, 22(2), 142-173.
Qi, L. (2007). Is testing an efficient agent for pedagogical change. Foreign Language Teaching,
6(1), 49-54.
Read, J., & Hayes, B. (2003). The impact of IELTS on preparation for academic study in New
Zealand. IELTS International English Language Testing System Research Reports, 4,
153-206.
Richards, J. (1996). Communicative language teaching today. New York: Cambridge University
Press.

74

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Richards, L., & Morse, J. M. (2013). Readme first for a user's guide to qualitative methods (3rd
ed.). Sage.
Ryan, P. (2010, December). Patricia Ryan: Don’t insist on English! [Video file]. Retrieved from
https://www.ted.com/talks/patricia_ryan_ideas_in_all_languages_not_just_english/transc
ript?language=en#t-341000
Roessingh, H., Kover, P., & Watt, D. (2005). Developing Cognitive Academic Language
Proficiency: The Journey. TESL Canada Journal, 23(1), 1 - 27.
Saif, S. (2006). Aiming for positive washback: A case study of international teaching assistants.
Language Testing, 23(1), 1-34.
Savignon, S. (1997). Communicative competence: theory and classroom practice: texts and
contexts in second language learning (2nd ed.). New York: McGraw-Hill.
Schlebusch, G., & Thobedi, M. (2004). Outcomes-based education in the English second
language classroom in South Africa. The Qualitative Report, 9(1), 35-48.
Shohamy, E. (1993). The power of test: The impact of language tests on teaching and learning.
National Foreign Language Center Occasional Papers. Washington, DC: National
Foreign Language Center.
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect
over time. Language Testing, 13(3), 298-317.
Smith, M. L. (1991). Put to the test: The effects of external testing on teachers. Educational
Researcher, 20(5), 8-11.
Sousa, D. A. (2015). Brain-Friendly Assessments: What They Are and How to Use Them. West
Palm Beach, FL: Learning Sciences International.

75

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Spady, W. (1994). Outcome-based education: Critical issues and answers. Arlington, Va.:
American Association of School Administrators.
Spiegelberg, H. (1982). The phenomenological movement (3rd ed.). Boston: Mearinus-Nijhoff.
Spolsky, B. (1985). The Limits of Authenticity in Language Testing. Language Testing, 2(1), 3140.
Stecher, B., Chun, T., & Barron, S. (2004). The effects of assessment-driven reform on the
teaching of writing in Washington State. Washback in language testing: Research context
and methods, 191-210. Retrieved from
http://faculty.mu.edu.sa/public/uploads/1347487499.950171132987-Washback-inLanguage-Testing.pdf#page=76
Taras, M. (2005). Assessment–summative and formative–some theoretical reflections. British
journal of educational studies, 53(4), 466-478.
Thorne, S. (2008). Interpretive Description. Walnut Creek, CA: Left Coast Press.
Thorne, S., & Darbyshire, P. (2005). Land mines in the field: A modest proposal for improving
the craft of qualitative health research. Qualitative Health Research, 15(8), 1105-1113.
Tsagari, D. (2011). Washback of a high-stakes English exam on teachers' perceptions and
practices. In E. Kitis, N. Lavidas, N. Topintzi, & T. Tsangalidis (Ed.), Seleted papers
from the 19th International Symposium on Theoretical and Applied Linguistics (pp. 431445). Thessaloniki: Department of Theoretical and Applied Linguistics, Aristotle
University of Thessaloniki, School of English.
Tsushima, R. (2012). The mismatch between educational policy and classroom practice: EFL
teachers' perspective on washback in Japan (Master’s Thesis). Montreal: McGill
University.

76

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Turner, C. E. (2006). Professionalism and high-stakes tests: Teachers' perspectives when dealing
with educational change introduced through provincial exams. TESL Canada Journal,
23(2), 54-76.
van Manen, M. (1990). Researching lived experience: Human science for action sensitive
pedagogy. Albany, N.Y.: University of New York Press.
VanPatten, B. (2003). From input to output: A teacher's guide to second language acquisition.
Boston: McGraw-Hill.
VanPatten, B. (2015a, October). teawithbvp - Episode 1 - The State of Language Teaching
[Audio podcast]. Retrieved from http://www.teawithbvp.com/
VanPatten, B. (2016a, March). teawithbvp - Episode 20 - Principle 2: The Nature of
Communication [Audio podcast]. Retrieved from http://www.teawithbvp.com/
VanPatten, B. (2016b, September). teawithbvp - Episode 29 - Ask Us Anything! [Audio
podcast]. Retrieved from http://www.teawithbvp.com/
VanPattern, B. (2015b). Theories in second language acquisition: An introduction (2nd ed.).
New York: Routledge.
VanPattern, B. (2017). While We're On the Topic: BVP on Language, Acquisition, and
Classroom Practice. ACTFL.
Wall, D. (2000). The impact of high-stakes testing on teaching and learning: Can this be
predicted or controlled? System, 28(4), 499-509.
Wall, D. (2005). The impact of high-stakes examinations on classroom teaching: A case study
using insights from testing and innovation theory. Cambridge University Press.
Wall, D., & Alderson, J. C. (1993). Examining washback: the Sri Lankan impact study.
Language Testing, 10(1), 41-69.

77

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Wall, D., & Harak, T. (2006). The impact of changes in the TOEFL examination on teaching and
learning in Central and Eastern Europe: Phase 1, The baseline study. ETS Research
Report Series, 1, i-199. Retrieved from
http://onlinelibrary.wiley.com/doi/10.1002/j.2333-8504.2006.tb02024.x/pdf
Watanabe, Y. (1992). Washback effects of college entrance examination on language learning
strategies. JACET Bulletin, 23, 175-194.
Watanabe, Y. (1996). Does grammar-translation come from the entrance examination?
Preliminary findings from classroom-based research. Language Testing, 13(3), 318-333.
Watanabe, Y. (2000). Washback effects of the English section of Japanese university entrance
examinations on instruction in pre-college level EFL. Language Testing Update, 27, 4247.
Waters, M., & Mehay, R. (2010). A deeper look at constructivism – ontology and epistemology.
Retrieved from
https://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEw
jayOKIjpvQAhUHxlQKHfzPCd8QFggbMAA&url=http%3A%2F%2Fwww.essentialgpt
rainingbook.com%2Fresources%2Fchapter_10%2FA%2520Deeper%2520look%2520at
%2520Constructivism.doc&usg=AFQjCNEK_6FA6TW
Wesdorp, H. (1982). Backwash effects of language-testing in primary and secondary education.
Journal of Applied Language Study, 1(1), 40-55.
Xie, Q. (2013). Does test preparation work? Implications for score validity. Language
Assessment Quarterly, 10(2), 196-218.
Zhan, Y., & Wan, Z. H. (2014). Dynamic nature of washback on individual learners: The role of
possible selves. Assessment & Evaluation in Higher Education, 39(7), 821-839.

78

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix A: Summary of Research on Washback
Table 3
Research Concerning Washback on the Teaching and Learning Curriculum
Author(s)

Context

Significant Conclusions
Studies that Found Significant Washback

Barnes (2016)

Vietnam

Influence found but mediated by the use of the selected test
preparation text book.

Cheng (2005)

Hong Kong

Test played a role in moving traditional teaching toward
more communicative activates.

Erfani (2012)

Iran

Negative washback out-weighted positive washback.

Green (2007)

UK

Test-prep classes were not beneficial in raising test scores.

Lam (1994)

Hong Kong

Teachers significantly modified their curriculum and
instruction to match the content of the exam.

Li (1990)

China

Positive washback found as test promoted language
practice over linguistic class orientation.

Shohamy, DonitsaSchmidt, and
Ferman (1996)

Israel

A narrowing of the curriculum to tested areas.

Stecher, Chun, and
Barron (2004)

USA

Only the genres found on the test were practiced in class.

Tsagari (2011)

Greece

Intense focus on test prep found.

Tsushima (2011)

Japan

Teachers favored teaching to the test.

Wall and Alderson
(1993)

Sri Lanka

Teachers focused only on the skills that appeared in the
test.

Wall (2005)

Sri Lanka

Teachers focused only on the skills that appeared in the
test.
Studies that Found No Significant Washback

Watanabe (2000)

Japan

No washback due to inability of teachers to change
traditional practices what so ever.

Wersdorp (1982)

Netherlands

No narrowing of the curriculum was found.

79

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Table 4
Research Concerning Washback on Teaching Strategies, Methods and Approaches
Author(s)

Context

Significant Conclusions

Studies that Found Significant Washback
Alderson and Hamp-Lyons
(1996)

U.S.

Effect varied within instructors.

Burrows (2004)

Australia

Effect varied within instructors.

Read and Hayes (2003)

NZ

Effect varied within instructors.

Saif (2006)

Canada

Greater teacher understanding lessened
variability.

Shohamy (1993)

Israel

Only novice teachers’ methodologies were
influenced by the test.

Stecher, Chun, and Barron.
(2004)

U.S.

Teachers incorporated processes and strategies
intended by the tests into their instruction.

Watanabe (1996)

Japan

Effect varies within instructors.

Studies that Found No Significant Washback
Cheng (2005)

Hong Kong

Methodological change initiated through testing
happened only very slowly, reluctantly, and with
difficulty.

Hargreaves (1997)

Egypt

Teaching methods were independent of testing.

Qi (2005)

China

Tests designed for the both change in instruction
and to measure language proficiency tended to
fall short of either.

Qi (2007)

China

High-stakes tests were not effective as tools for
bringing about instructional change.

Tsagari (2011)

Greece

No change from traditional methodology despite
communicative focus of test.

Turner (2006)

Canada

Teachers manage washback through their
personal beliefs and professional stances.

Wall and Alderson (1993)

Sri Lanka

Washback on teaching methodology was
inhibited for a multitude of reasons.

Wesdorp (1982)

Netherlands

Influence of test on methodology was so small
that it is of no consequence.

80

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Table 5
Research Concerning Washback on Classroom Assessment
Author(s)

Context

Significant Conclusions

Studies that Found Significant Washback
Wesdorp (1982)

Netherlands

No significant evidence to suggest in class
assessment was changed to match the final test
structures.

Studies that Found No Significant Washback
Wall and Alderson
(1993)

Sri Lanka

In class testing tended to mimic the question style
of the final test.

Wall and Horak (2006)

Central and
Screening, diagnostic and practice tests
Eastern Europe purposefully followed the final test format.

81

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Table 6
Research Concerning Washback on Student Learning
Author(s)

Context

Significant Conclusions

Studies that Found Significant Washback
Hughes (1988)

Turkey

Concluded that student learning was found to
have increased.

Saif (2006)

Canada

Positive washback was found after teachers
had received training to prepare them for the
new exam.

Tsagari (2011)

Greece

Intense prep for test despite believing that the
exam created a lack of student autonomy and
ability to communicate.
Studies that Found No Significant Washback

Cheng (1998)

Hong Kong

Students’ study habits were not easily coerced
by a test.

Jinsong, Peiying, &
Xiaomei (2014)

China

Limited positive washback found on student
learning.

Pan (2014)

Taiwan

The test had no significant effect of the
learning strategies employed out of class by
the students.

Watanabe (1992)

Japan

Revealed a multifaceted nature of the
phenomenon of washback on learning.

Wesdorp’s (1982)

Netherlands

No positive washback was found on student
learning with the incorporation of multiple
choice orientated testing.

Xie (2013)

Hong Kong

Student study habits were not easily coerced
by a test.

82

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Table 7
Research Concerning Washback on the Mental State of Instructors
Author(s)

Context

Significant Conclusions

Studies that Found Negative Washback
Alderson and HampLyons (1996)

U.S.

Found negative attitudes including frustration being
fermented toward the use of the TOEFL® test.

Cheng (1998)

Hong Kong

Avoidance of feelings of guilt motivated one
teacher to focus heavily on test preparation.

Li (1990)

China

Teachers’ feelings of unease toward the exam
abated as time passed.

Shohamy (1996)

Israel

Overall negative reactions due to lack of
consolation and pressure to adhere to teach to the
test.
Studies that Found Positive Washback

Johnson, Jordan, and
Poehner (2005)

U.S.

Instructor was willing to doubt his own evaluations
in light of results from standardized test.

Read and Hayes
(2003)

NZ

Documented positive feelings of teachers toward
the IELTS™ test orientation.

83

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Table 8
Overview of the Research Concerning Washback on Attitudes and Feelings of Learners
Author(s)
Alderson and HampLyons’ (1996)

Context
U.S.

Significant Conclusions
No evidence to suggest students’ insistence of
methodology, use of practice tests, or test specific
strategies.

Ferman (2004)

Israel

Individual difference in attitudes and feelings toward
exams.

Li (1990)

China

Positive student feelings toward the revised exam
coincided with heightened motivation.

Lumley and
Stoneman (2000)

Hong Kong

Students were more exam orientated than the
teachers.

Read and Hayes
(2003)

NZ

Positive student feelings toward the revised exam
coincided with heightened motivation.

Shohamy (1993)

Israel

Individual differences in attitudes and feelings
toward exams.

Shohamy, DonitsaSchmidt, and Ferman,
(1996)

Israel

Students tended to mirror teachers in feeling anxious
about the imposed test changes.

Wall and Horak
(2006)

Central and
Student expectations of classroom activities
Eastern Europe pressured teachers’ instructional decision making.

84

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Table 9
Research Concerning Washback on Student Motivation toward Exams
Author(s)
Berwick and Ross
(1989)

Context
Japan

Significant Conclusions
Marked increase in the intensity of student
motivation in the final year of high-school.

Jinsong, Peiying, &
Xiaomei (2014)

China

Students perceived test as giving them
motivation to learn English.

Pan (2014)

Taiwan

Learners’ intrinsic and extrinsic
motivations were enhanced by formal
testing.

Tsagari (2011)

Greece

Exam-orientation of class resulted in
students becoming demotivated and bored.

Watanabe (1992)

Japan

Exams motivated students only to pass the
exam.

Zhan and Wan (2015)

China

Motivation generated from the test was
largely short-term.

85

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix B: Guiding Questions

Guiding Interview Questions
1. Contemporary communicative language teacher is a mouthful and not that well defined in
the literature. Can you first talk a little about your instruction? Specifically how you
differ from more traditional approaches.
2. What is the assessment strategy utilized in your department? How is it evolving?
3. How aligned are assessment strategies at your educational institution with content and
teaching methodologies that you utilize?
4. In your opinion, how appropriate is standardized proficiency based testing such
as…….for your practice?
5. In your opinion, how appropriate are alternative assessment schemes such as………in
your practice?
6. In your view, what is the importance of stakeholder buy-in in your practice?
7. How do the assessment strategies at your educational institution impact institutional
support for the incorporation of contemporary CLT?
8. In your opinion, how can contemporary CLT be best supported practically institution
wide?
9. What other issues related to washback on your instruction that were not covered above
have you experienced and what additional related insights can you offer?

86

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix C: Participant Recruitment Email Brochure

Research Participant Recruitment Email Brochure
Advanced TESOL Instructors Required for Interview Based Research Project
Dear, xxxx
I have contacted you as you are a practicing master’s prepared ESL instructor.
Would you like to participate in a research study that explores the influence of assessment on
the advanced second language instructor?
The Question
What is the experience of advanced language instructors dealing with assessment washback
within high-stakes University preparatory ESL course settings?
Research Opportunity
Are you interested in sharing your experience as an advanced ESL instructor? As a Master of
Multidisciplinary Leadership student in the Education Department at the University of Northern
British Colombia, I am currently conducting a phenomenological research study under the
supervision of Dr. Catherine Whalen.
Research Purpose
i.
ii.

To examine the lived experience of master’s prepared instructors, dealing with
assessment washback within high-stakes University preparatory ESL course
settings.
To gain a fuller understanding of the desirable and undesirable effects of
standardized assessment on advanced practice in order to elicit practical
knowledge for the advancement of curricula.

87

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Research Goals
i.
ii.
iii.
iv.

Interview and document advanced ESL instructors experience with assessment
washback within their educational setting in order to elicit common themes.
Provide a possible way forward based on current practice and practicing
professionals.
To be of interest to language institutions striving for excellence, and skilled
instructors seeking to promote systemic change toward conditions that facilitate
optimal language acquisition.
To reach a larger audience by, additionally, publishing the study in an academic
journal.

Participant Commitment
You will need to participate in a Microsoft Skype interview of approximately one hour in
duration. Additionally, you will need to further engage in electronic dialog should themes
emerge that require additional input.
The Sync cloud service will be utilized for the transference of files and electronic dialog. This will
require you to create a personal Sync account (Free for up to 5GB of use). Sync is a fully
encrypted, zero-knowledge cloud service that makes it easy to store, share and access your files
from everywhere with privacy guaranteed.
Your name will not be revealed at any time. Pseudonyms will be used to replace name and
location identifiers.
You will be informed not to use professional or personal identifiers that would identify your
place of employment. Should you accidentally reveal any identifiers, I will remove them from
the transcript of the interview.
You will be asked to speak only in general terms about your curriculum to ensure that no
proprietary or sensitive information is revealed to me. I will remove any proprietary or sensitive
information accidentally revealed by you from the transcript of the interview.
Please reply to knowless@unbc.ca to show your interest in sharing your experience as an
advanced ESL instructor.
Regards,

Steven Knowles

88

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix D: Information Letter / Participant Consent Form

Information Letter/Participant Consent Form
The Phenomenon of Assessment Washback on Contemporary Communicative Second
Language Teaching
Student Researcher: Steven Knowles
As a Master of Multidisciplinary Leadership student in the Education Department at the
University of Northern British Colombia, I am currently conducting a phenomenological
research study under the supervision of Dr. Catherine Whalen.
You can contacted me by cell phone at 250-981-5909 or email at knowless@unbc.ca.
Alternatively, you can contacted my supervisor by office phone at 250-960-5639 or email at
catherine.whalen@unbc.ca
In addition to the UNBC graduate degree, I intend to use the data for future academic
publications and in conference presentations.
Purpose of Project
I am inviting you to take part in this research study because you are a highly qualified practicing
ESL instructor working within a high-stakes University preparatory ESL course setting.
I will synthesize your contributions with contributions from other advanced instructors.
I am conducting this study to gain a fuller understanding of the desirable and undesirable
influence of assessment on advanced instructional practise.
The goal of the research project is to elicit practical knowledge to enable a better
understanding of the assessment requirements needed to support advanced instruction at an
institutional level.
The participants in this study will choose to contribute on a voluntary basis. Any participant
may withdraw from the study any time without consequence. If a participant should choose to
leave the study, all information collected from that participant will be deleted or shredded.

89

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
What will happen during the project?
If you agree to be part of the study, you will need to participate in a Microsoft Skype interview
of approximately one hour in duration. Additionally, you will need to further engage in email
dialog should themes emerge that require additional input.
You will be informed not to use professional or personal identifiers that would identify your
place of employment. Should you accidentally reveal any identifiers, I will remove them from
the transcript of the interview. Additionally, you will be asked to speak only in general terms
about your curriculum to ensure that no proprietary or sensitive information is revealed to me.
I will remove any proprietary or sensitive information accidentally revealed by you from the
transcript of the interview.
I will conduct the interview through the Microsoft Skype application and record the audio with
the MP3 Skype Recorder application. Either I, or a research transcriber, will transcribe the
interview verbatim. Any transcriber will first have signed the UNBC Confidentiality and NonDisclosure Agreement Form. I will replace names and location identifiers with a pseudonym
before the transcriber receives the recordings. I will store physical and electronic copies of
transcripts and recordings in a locked filing cabinet. Any data collected and transcribed will be
done so on a password-protected computer and encrypted storage device.
After coding, I will return the results to you to ensure there is agreement in the revealed
experience of the studied phenomenon. Each participant will be able to amend or delete their
individual transcribed document where he or she deems fit. I will only utilize the final agreed
upon coded document for analysis.
The Sync cloud service will be utilized for the transference of files and electronic dialog. This will
require you to create a personal Sync account (Free for up to 5GB of use). Sync is a fully
encrypted, zero-knowledge cloud service that makes it easy to store, share and access your files
from everywhere with privacy guaranteed.
Risks to participating in the project
I am predicting there is minimal to no risk to any participant interested in participating in this
research study. I intend to strictly follow ethical practices as stated in the UNBC Policy on
Research Involving Human Participants and conduct the research according to the Tri-council
Policy Statement: Ethical Conduct for Research Involving Humans 2014 (TCPS2). If participants
want to withdraw from the study for any reason, their information will be destroyed and no
portion of their data will be used in any final reporting of this study or in any future studies.
Participants' confidentiality will be protected by the use of pseudonyms. I will store all raw data
collected on an encrypted storage device and keep it in a locked file cabinet and save it on the
Sync encrypted cloud server. The transcriber will only utilize the Sync encrypted cloud for

90

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
accessing and storing data. A password protected computer will be only used when working
with the data.
As a research participant, you do not have to answer any question if you do not want to. If at
any point in the study, you feel uncomfortable or upset, and wish to end your participation,
then I will respect your wishes.
Benefits to participating in the project
By being part of the study, you will be helping language institutions that strive for excellence;
and providing an academic voice to skilled instructors seeking to promote systemic change
toward conditions that facilitate optimal language acquisition.
Confidentiality, Anonymity and Data Storage
All names and contact information of participants will be strictly confidential through the use of
pseudonyms agreed upon by the participant. The interview sessions will be recorded and
transcribed by a transcriber who will have signed a confidentiality and disclosure form previous
to receiving any interview recordings. The transcriber will have access to the interview
recordings which address participants as their chosen pseudonyms, thus the transcriptions will
not have the real identity of each participant nor their affiliated institution. I will store the data
and all forms with identifying names and contact information on an encrypted storage device in
a locked file cabinet and in the Sync encrypted cloud storage. The transcriber will only utilize
the Sync encrypted cloud for accessing and storing data. A password protected computer will
be only used when working with the data for security measures.
I will ensure that data will only be kept on hard-drives and/or memory sticks that are encrypted
with either BitLocker or AES Crypt software.
Microsoft Skype will be utilized for conducting interviews in this study. Microsoft Skype utilizes
encryption to ensure security. However, the extent to which Microsoft itself and governmental
agencies may have access to Skype content is unsure. Skype has also been shown to be
vulnerable to malware designed to monitor your calls and videos.
I will be retaining the data for a period of 10 years after the final thesis report if approved in
order to use it for future publications and conference presentations. The data will then be
permanently destroyed/deleted/shredded.

91

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Compensation
All participants in the study will choose to contribute on a purely voluntary basis.
Study Results
I will report the results of the study in my graduate thesis. Additionally, I intend to publish the
findings in academic journal articles and present the findings at academic conferences.
Questions or Concerns about the project
If you have any questions about what I am asking of you, please do not hesitate to contact me
by email at knowless@unbc.ca or by cell phone at 250-981-5909. Alternatively, you can contact
my supervisor Catherine Whalen by email at catherine.whalen@unbc.ca or by office phone at
250-960-5639.
If you have any concerns or complaints about your rights as a research participant and/or your
experiences while participating in the study, contact the UNBC Office of Research at 250 960
6735 or by e-mail at reb@unbc.ca.
Participant Consent and Withdrawal
Taking part in this study is entirely up to you. You have the right to refuse to participate in this
study. If you decide to take part, you may choose to pull out of the study at any time up to the
completion of the study without giving a reason and without any negative impact on you [for
example, employment commitments].
If you withdraw from the study your data will be removed from the study and destroyed.
CONSENT
I have read or been described the information presented in the information letter about the
project:
YES

NO

I have had the opportunity to ask questions about my involvement in this project and to receive
additional details I requested.
YES

NO

92

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

93

I understand that if I agree to participate in this project, I may withdraw from the project at any
time up until the report completion, with no consequences of any kind. I have been given a
copy of this form.
YES

NO

I agree to be recorded.
YES

NO

Follow-up information (e.g. transcription) can be sent to me at the following e-mail or mailing
address:
____________________________________________.
YES

NO

Signature (or note of verbal consent):
Name of Participant (Printed):
Date:

Your signature below indicates that:



You have received a copy of this consent form for your own records.
Your signature indicates that you consent to participate in this study.

__________________________________________
Participant Signature
__________________________________________
Printed Name of the Participant signing above

________________________
Date

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

94

Appendix E: Transcriber Confidentiality and Non-Disclosure Agreement

Confidentiality and Non-Disclosure Agreement
This study, The Phenomenon of Assessment Washback on Contemporary Communicative Second
Language Teaching is being undertaken by Steven Knowles at the University of Northern British
Columbia (UNBC). The objective of the study is to examine the influence of high-stakes assessment
on advanced second language instruction.
Data from this study will be used to complete the requirement for the Master of Multidisciplinary
Leadership. Additionally, the researcher intends to publish the study in an academic journal.
I, ________________________________, agree as follows:
1.
2.
3.
4.
5.

To keep all the research information shared with me confidential by not discussing or
sharing the research information in any form or format (e.g. disks, tapes, transcripts)
with anyone other than the Principal Investigator(s);
To keep all research information in any form or format secure while it is in my
possession;
I will not use the research information for any purpose other than creating translations;
To return all research information in any form or format to the Principal Investigator(s)
when I have completed the research tasks;
After consulting with the Principal Investigator(s), erase or destroy all research
information in any form or format regarding this research project that is not returnable
to the Principal Investigator(s) (e.g. information stored on computer hard drive).

Recipient

(Print name)

(Signature)

(Date)

(Signature)

(Date)

Principal Investigator:

(Print name)

If you have any questions or concerns about this study, please contact:
Dr. Catherine Whalen
(250) 960-5639 and catherine.whalen@unbc.ca

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

This proposed study has been reviewed by the Research Ethics Board at UNBC. For questions
regarding participant rights and ethical conduct of research, contact the Office of Research by email
at reb@unbc.ca or telephone at (250) 960-6735.

95

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

96

Appendix F: Codebook - Jeffery
Themes Relating to the Context of the Instructional Experience
Tables 1 through 5 depict the context of Jeffery’s instructional experience.
Table 1.
Theme Category: Instructional Background
Page Line
4

3

Textual Data

Code

They are complementary, they balance each other yes.
Curriculum writer
Uhh, this is one area, like in terms of my position as a
curriculum person I don't have as much, I'm less familiar
with the internal diagnostic tests (Em hem). Ahh, that's
one area of the curriculum that I have not been able to
get to yet so

Summary. Additional to language instruction Jeffery is also involved in curriculum writing.
Table 2.
Theme Category: CLT Interpretation
Page Line

Textual Data

Code

1

22 Communicative Language Teaching like from me tries
CLT based around
to get the student, no matter what proficiency level that
communication rather that
they are at to begin using the language in a real
accuracy.
authentic kind of context or way as soon as possible.
Instead of trying to teach say perfect pronunciation or to
teach them grammatical structures as they used to do,
you would write endless mounts of drills perfecting your
grammar, before you ever spoke a single word in
English.

1

27 obviously in the modern classroom it's all about
authenticity

Authenticity

1

29 our focus is more narrow, to the context to the language,
to the skills that they need to be successful in a

EAP focused

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

97

university environment.
1

33 I know the language classroom is always sort of
inauthentic, ha, right, (by it's nature, ha) it's a cocoon
isolated from reality, (yeah) in some ways. Um, so you
try to do your best to provide meaningful tasks and get
them using those tasks in authentic ways.

Use of meaningful tasks to
overcome inauthentic
context of the classroom.

1

44 . Focuses on intelligibility, can you communicate your
message effectively?

Focus on communication

6

1

Rapport

7

13 teachers have to be there and be able to present students
with some of this information and maybe construct their
classes somewhat around somebodies ideas of
metacognition, self-regulation, being able to analyze
your strategies that you apply to your learning in
different context and be able to, sort of self-question.
Arh, is it a strategy actually working for me right now
(yeah). ‘Do I need to switch to something else to be able
to be successful or to meet the goal that I have for
myself’.

So the instructors tone, umm, your attitude and general
environment that you've created in the classroom, is it
one that where students feel safe and secure in making
mistakes (Yeap). Now, if they do, then washback is
much, much different and much more likely to result in
the learning outcomes that you want and the learning
that your students are, are desiring.

Student learning how to
learn.

Summary. Jeffery’s interpretation of CLT is based around communication rather that
accuracy. He uses meaningful tasks to overcome inauthentic context of the classroom. He finds
rapport essential in developing authentic communication with learners. Jeffery sees the role of
the language instructor to also include enabling students to self-question adopted learning
strategies.
Table 3.
Theme Category: Learning Context
Page Line

Textual Data

Code

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
2

12

If students pass our highest level they are granted entry
into university degree setting

98
Gate keeping program

Summary. Jeffery is part of the Gatekeeping program for second language learners wanting to
enter the university.
Table 4.
Theme Category: Learner’s Background Context
Page Line

Textual Data

students arrive with, Ahhh, maybe an inflated belief in
(Em hem) their language proficiency

Code

3

9

Inflated belief in language
level

3

13 this often results in a backlash of sort of externalizing
blame. (yeah, yeap) There is environmental blame,
umm, ‘the test room was too cold, too hot, too dry, too
loud’, ‘the audio wasn't, wasn't, good enough’, umm, ‘I
was under slept’ or ‘I had jet lag’ (yeap). Umm, I, So
there is all kinds of blame, but it's all externalized, it's
very rarely it’s self-reflective. (Em hem) .Umm, Not
‘what could I have done to improve the outcome’, it's
some other accommodation had to happen, um

Externalizing blame for low
test scores with focus away
from language proficiency.

3

28 Some students they, they will accept it. They will realize
that within the couple of weeks of classes that they are
placed properly. (Em hem), Um, that they realize that
‘no my English proficiency is not enough to be in higher
levels, and certainly, ‘I'm not ready to, to deal with the
share weight and volume of, of course work of a
university program’. (Em hem). So quite often, like our
instructors were, were able to do a good job of
convincing students that you need to be where you are
and this in the long term will benefit you.

Instructors able to help
students come to realization
of proficiency their
proficiency level once they
see what is needed in terms
of outcomes.

3

Cultural belief about passing
40 Yes, and, and this, this attitude or this belief system
about failing a test, or not doing as well as anticipated it tests
serves them very poorly with language learning, because
the victories in language learning are so slow. And it
can be very hard to see progress, and they want to see
fast progress, they want to get to their business degree.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

99

3

Pressure to get to degree
43 There is so much parental pressure and financial
pressure, they have invested in a lot of resources coming
across the ocean and settling in a new strange small city
in the middle of Canada. ‘I just want to get to it’, ‘I just
want to get to the real learning right’, and language
learning is somehow secondary strangely, ha ha.

5

21 being predominantly mainland Chinese, assessment is
sort of an inherent motivator for these students, (Em
hem) for good and bad, Umm, so, in general, I find that
no matter, in no matter what skill I'm teaching
assessment is kind of the stick and the carrot,
simultaneously (Yeap). Umm, it's, it motivates students
to complete assignments, umm, it motivates them to do
or complete tasks that perhaps they otherwise wouldn't
do on their own, ahh, so there is almost a necessity in
the program to have assessment sort of on an ongoing
bases in order to get students to do what you want them
to do because a lot of the time they are not intrinsically
motivated to complete this, uhh, complete the task that
is set for them. Ahh, they require some kind of mark as
a validation for their work

Extrinsic learning expected
and valued exclusively.

7

7

Don’t understand how to
learn a language (expect
through skill building).

7

23 many of our students expect the very traditional student
teacher relationship (Yeap). You have like the sage on
the stage delivering perils of wisdom that the students
collect, right, and that and that is very much the Eastern
pedagogical model or (yeap) or students don't have
questions, they just wait for the information to be
delivered to them and they studiously write it down

Yeah, I think how to learn, is a huge question for many
students, umm, it that they don't have frameworks to
apply to their own, uh, learning management. I think I
mentioned self-regulated learning in our, a little bit last
week, when we talked (yeap). They are not able to
reflect on, umm, their strategies,

Behaviourist style
expectations

Summary. Extrinsic learning is expected and valued exclusively by learners that come
into the program and thus have a strong focus on passing tests. They are under great pressure to
get to the degree program as soon as possible. Many coming with an inflated belief of their
language proficiency level. Learners do not know how to learn a second language successfully.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

100

When they receive a failing grade or are placed in a lower class than they expect they generally
externalize the blame with a focus away from language proficiency. Teachers are able to help
students come to the realization of their proficiency level once they see what is needed in terms
of outcomes.
Table 5.
Theme Category: Assessment Structure
Page Line

Textual Data

Code

2

30 there is no, there is no summative test at the end. It's just
based on their accumulative assessment throughout the
term.

Standardized mixed with inclass testing

2

40 that final level would share that same test, which would
make up some of the grade.

In-house Standardized
testing (horizontal)

Jeffery: That is correct, yes. So for example in writing,
the writing instructors would get together, they would
decide on a specific, Uuh, test question, and then the
students would all write on the same question. Then
(Yeap) same with listening, they would all listen to the
same lecture material and all complete the same
standard test, (Yeap, yeap)
Summary. Standardized assessment mixed with in-class standardized testing
(horizontally uniform).
Themes Relating to the Experience of Washback
Tables 6 through 11 depict Jeffery’s experience of washback.
Table 6.

Theme Category: Experience of Assessment Alignment

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Page Line

Textual Data

101
Code

3

45 ‘I just want to get to it’, ‘I just want to get to the real
learning right’, and language learning is somehow
secondary strangely, ha ha.

ESL is secondary to learning
‘just want to get to it’

5

48 It depends a little bit on student back ground, Umm. So,
we do have a certain percentage of students that go to
say a Canadian based curriculum high school in China.
And they tend to arrive here much better prepared,
acclimatized little, but more familiar with the social
cultural components of learning English in a Canadian
context because they had Canadian trained teachers.

Student Buy-in – better if
the come from Canadian
based curriculum in China.

6

16 Now if that is all been done well and students feel that,
Positive from standardized
you know, the channel of communication that they can
tests if CLT done right.
ask questions they can say 'I don't understand, what this
means’, ‘I don't know how to complete that component
of the assignment'. The channel of communication are
open and explanations are provided in a clear and really
supportive (yeap) way. Then washback and the potential
for the benefits for assessment are great.

6

24 students would buy in to the fact that what they are
learning in the class is valuable to them in the future
(yeap), and that, in fact, the writing assignments that
they will face in university are going to be far more
complex and with multiple projects going on
simultaneously, right. So, to me, a lot to do with
assessment in the classroom and the results of
assessment have a lot to do with the student teacher
relationship, classroom management and the learning
environment that the instructor creates

6

Students looking for short
37 they come to you after they receive say an initial
assignment back and the feedback and they come to you cuts see grammar as one of
and they say ‘teacher can you give me, more grammar
them.
exercises’, ‘Can you teach us, can you teach us, whether
it’s this structure, whether it's passive voice (yeap)
whatever it is’. And I sort of, I look at them, ‘Well, ok,
but are you ready to do the work?’, ‘I can teach you,
and I can lead you down this path, but me just
presenting the material to you doesn't mean any learning
is going to happen’, right (em hem), ‘you have to meet
me half way and do the work’. And very often I find
students don't want to do the work. (Yeah, they are

Buy in – if CLT done right.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

102

looking for shortcuts) They want you to come to them
with some grammar worksheets, and they will do the
grammar worksheets, perfect them. Then you never see
any implementation in their actual writing (yeah). So I
think the…, to come back to your original point, it that,
um, feedback or assessment on very specific things can
be beneficial, but the students have to be willing to put
in the work (yeah) and to be self-reflective of like their
progress (yeap yeap).
10 46 there's this kind of interesting phenomena that
sometimes happens especially if students get placed in
one of the lowest levels. Umm, there can be, I
mentioned about how washback can be, can result in
this sort of, existential crises with in the student (yeap),
where there's, they are very bitter. So, some students
will take this to the extreme, where they will, just a very
small number, but it does happen where they actually
stop coming class altogether in order to study for say
the IELTS exam (em hem) they will withdraw from the
program and attempt, they say that are going to study
and then they go write the IELTS in hope of being able
to jump levels. Very rarely works, though students
persist in doing this all this has pretty negative
consequences. Obviously there no coming to class, so
they are short of defaulting of four months of valuable
learning experience to go study for an (I agree) exam

Students will drop course to
study for test if they feel
their test focus is not being
met.

10 18 I see formative assessment like within the program.
Like within the discrete skill like listening, speaking,
reading, writing. Formative assessments I think students
tend to value as long as there is clear communication
between the instructor and the student about how these
learning tasks are applicable, (yes) how they will
benefit them (Yeap). So, there is like a tangible, umm,
message in something that students can understand and
project into the future as opposed to oh right I just have
to jump through this hoop (Yeap). I don't really
understand why I have to jump through this hoop. But
I'll jump through it because it's worth 30% of my mark
(yeah).

Formative assessment helps
to pull to wider goal of
improving.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

103

Summary. Learners tend to look for shortcuts to passing tests as they see ESL is
secondary to undergraduate studies they ‘just want to get to it’. They see grammar instruction as
a shortcut and pressure teachers to teach this way. In extreme cases learners will drop a course to
study for IELTS test if they feel their test focus is not being met. Student buy-in becomes
important and can generally be achieved if CLT introduced correctly, with more success if the
learner comes from a Canadian based curriculum in China. Standardized tests work well with
CLT as long as CLT is introduced and conducted correctly. Formative assessment helps to pull
to wider goal of improving.
Table 7.
Theme Category: Experience of Indirect Academic Course Instructor Washback
Page Line
9

9

Textual Data

A lot of misunderstanding of what can be accomplished
in, ahh, in four months in a language program. So, I
have a lot of conversations with or I'm in discussion
groups with university professors and they are mystified
by how students who have you know come through our
program or come in to the university with a specific
IELTS mark and that they see, all they see is like
deficiencies, and they are mystified as to how these
students arrive here (Yeah). And I find that that this sort
of a misunderstanding about how students acquire
academic proficiency. That it results in a lot of finger
pointing and blaming and results in a lot of spinning
dialogue about how do we support these students.
‘Shouldn't they be able to do this already when they
arrive in first year business or second year business?’,
‘Shouldn't they be able to do XYZ and be perfectly, you
know, capable of making a presentation to the class?’
So, in terms of like providing support to communicative
teaching I actually think it needs to extend far beyond
the walls of the discrete English language program
(yeap).

Code
Lack of understanding of
what language learning
involves and how long it
takes.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

104

Summary. Instructors from undergraduate courses lack understanding of what language
learning involves and how long it takes.
Table 8.
Theme Category: Experience of Institutional/Administration Washback
Page Line

Textual Data

Code

9

28 I think all across Canada like English language
programs such as ours they are housed in very different
units within the universities (yeap), and so they have a
very different political arrangement, different levels of
formal acknowledgement, uhh, within the institution,
uhh, so there is a lot of complexity in terms of where
that English language institute is housed in the umbrella
of the University and how it is able to communicate
with stakeholders throughout the university. To actually
understand what, what happens in an English language
centre

Complexity in the
hierarchical structure of the
ESL department within the
university structure causes
difficulty in communication
with stakeholders.

8

36 International students are seen as a way to maintain the
bottom line (yeap). So, that's one very uneasy
relationship between higher university administrations,
administrative like desires and the desires and hopeful
outcomes of a language program imbedded within that
institution

Business model pressures
program to push students
through.

Summary. Complexity in the hierarchical structure of the ESL department within the
university structure causes difficulty in communication with stakeholders. Business model
pressures program to push students through.
Table 9.
Theme Category: Experience of Alternative Assessment Washback
Page Line
7

Textual Data

39 What alternative? Umm, I think within a program such
as ours which is a gatekeeping program (yeap) there,

Code
Must be careful in that
language is being assessed in

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

8

5

105

one has to tread somewhat carefully with say the
portfolio model in in writing, because in our program
we want to get, make sure it's possible, to be as sure as
possible that we are assessing language, that we are
assessing language components as opposed to merely
jumping through hoops (em hem) merely say, ‘Ok I able
to hand in X number of pages therefore I get a grade for
that’ (yeap), not even assessing what is happening
within the document, in terms on say vocabulary, uh,
grammar, uh, language structure of all kinds, um, like
are they, is the form of the writing recognisable within
an institutional setting (em hem), as opposed to just like
free writing. Which has its place as a fluency based
activity, but in terms of like assessing we have to be
very careful to balance between assessing language and
assessing other things that are extra linguistic.

terms of grammar and
vocabulary.

I think there is definitely a place for alternative forms of
assessment like that. It’s just it has to be, um, looked at
carefully and make sure that it is in balance with the
learning objectives and our obligations to, um, the
university over all in terms of student preparation in
terms of being sufficiently ready to enter degree study

Carefully implement in
balance with learning
objective and obligations

Summary. There is a place for alternative assessment. However, we must be careful in
that language is being assessed in terms of grammar, vocabulary, language structure (task based
but not purely task assessed).
Table 10.
Theme Category: Instructional Desires
Page Line
9

Textual Data

Code

21 I think that within university administration there needs
to be a recognition of the fact that learning academic
English, whether that be writing, reading, speaking
vocabulary, listening to lectures and what have you
takes a long time and these students require supports all
along the way. Umm, and it can't just be in the form of
like an academic writing center where students go and

Language support for
duration of study

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

106

see a tutor. There has to be social-cultural components
maybe even like mentors that help students begin to like
navigate the terrain of a Canadian University. Umm, so,
in terms of communicative language teaching, yeah.
The supports need to be robust.
10 30 So, there has to be really good communication between Quote of the interview
the instructor and the student. But also between the
program and the students coming in. Students are
enrolling in your program they should have a very clear
understanding of what they are going to learn, why they
are going to learn, and how this learning is going to help
them meet and fulfill their future learning goals (em
hem, ok). Then washback in assessment can be
maximized (yeap) for benefit.
Summary. Language support is needed for the duration of second language learner’s
study.
There has to be really good communication between the instructor and the student. But
also between the program and the students coming in. Students [that] are enrolling in your
program should have a very clear understanding of what they are going to learn, why they
are going to learn, and how this learning is going to help them meet and fulfill their future
learning goals. Then washback in assessment can be maximized for benefit.

Table 11.
Theme Category: Overall Washback Effects
Page Line
8

Textual Data

20 I think largely within our program, I think there is an
uneasy an uneasy, what is the word and uneasy dualism.
Uneasy relationship between this notion of the
communicative outcomes the language outcomes (yeap)
and students passing. Um from the curriculum writers

Code
Uneasy dualism between
Communicative outcomes vs
language outcomes Teaching to pass or to

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
side. Primarily, like when we are developing materials.
We are very much looking at language based learning
outcomes. What will the student take away in terms of
communicative language proficiencies, skills from these
assignments, um, because ultimately, because we don't
have a dedicated exit standardized test. Where we have
sort of, we have final exams to each of the discrete
courses and our highest level that contribute to the
overall mark, um, student just passing, yeah they make
it through the program. It's assumed that they have
absorbed and meet the learning objectives to a certain
degree. So from the point of view of the language
program, we hope that they have learned enough,
improved enough to be able to kind of survive at least
within the university context

107
outcomes

Summary. Uneasy dualism between Communicative outcomes vs language outcomes Teaching to pass or to outcomes.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix G: Codebook - Gary
Themes Relating to the Context of the Instructional Experience
Tables 1 through 5 depict the context of Gary’s instructional experience.
Table 1.
Theme Category: Instructional Background
Page Line
Textual Data
Code
2 16 We offer the same midterms and final across levels. Umm, I've been
Assessment
the primary designer or those assessments and for the rubrics that
developer
accompany them.
Summary. Additional to instructing Gary is an assessment developer.
Table 2.
Theme Category: CLT Interpretation
Page Line
Textual Data
1 11 I've never stuck to one particular kind of approach or
methodology in class
1 14 I've always found that text books don't always,
they're not always designed with a process, a
teaching process in mind, or necessarily a good
teaching practice in mind.
1 23 I tend to pull in from a lot of different approaches to
learning
1 32 And I like to experiment, too like if I see something,
or I'm like, oh inspired by something that I've read or
I've seen
5 17 so that I can build, so if they are good with, ahh,
simple present simple past whatever I can maybe
work on to perfect tenses or, ahh, these other things
(yeah) so it's actually helped us organise a little more
though it doesn't necessarily make us grammar um
dependence.

Code
Eclectic instructional style
Textbooks are ill-aligned
to practice.

Informed Eclecticism
Continuous Informal research

Focus on forms
instruction combined
with task orientated
instruction.

Summary. Gary uses an informed eclectic instructional approach to his ESL instruction.
He keeps up with related research in the field. His instruction combines a focus of forms with
task orientated instruction.

108

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

109

Table 3.
Theme Category: Learning Context
Page Line
Textual Data
6 15 they negotiate for marks on the test, cause it's a
negotiation, 'I give you, I give you this mark' rather
than, um, you earning it or whatever it is right (yeah) so
it's a constant fight, I mean I've had colleagues go on
stress leave because of the abuse of the students in this
regard. The relentless bullying and hostility and
comments it's never ending. The the lines that are
crossed are (yeap), you know, just mind blowing
sometimes.
6 25 a lot of ours are funded, they have limited government
funding (uh huh) to take their ESL
6 44 It's part of the providing a quality learner experience is
that somebody should be able to come into our ESL
program and they go smoothly efficiency into a nursing
program, let's say. You know, that's the way it should
be they shouldn't have to have an external test,
theoretically to be able to go into this thing. So our
administrators our leadership want this. Our
international students come in, they don't want it to be
these other things that have to happen, it should happen
in-house
7 13 and the students feel the heat because they have, um,
these governmental funding deadlines, timeline, you
know, where it is going to expire.
12 19 Our leadership doesn't seem interested in asking the
questions or finding the answers to the type of
questions that you’re asking me today that are really
important. Umm, and I think that is a very crucial,
critical missing piece to what is happening at least with
our institution, possibly within many others
41 3 Our counter parts, because we have the ESL intensive
program and we have a LINC program as well

Code
High stakes pressure coming
to bear.

High stakes treat of losing
funding
All stakeholders want
departmental/internal
assessment for gate-keeping
purposes.

High stakes threat of losing
funding.
Leadership not interested in
asking the right questions or
finding right answers.

ESL intensive program

Summary. Gary instruction takes place in the context of an EAP high-stakes
environment where many students are under the threat of losing funding. Gary is required to
instruct EAP and LINC learners simultaneously. He has experienced extreme pressure to

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

110

increase grades escalating to bullying from students at times. All stakeholders want
departmental/internal assessment for gate-keeping purposes into the academic program, yet
leadership do not seem interested in finding answers to important washback related questions.
Table 4.
Theme Category: Learner’s Background Context
Page Line
Textual Data
10 18 well number one none of the students think that the can pass
IELTS and they are probably right.
6 15 they negotiate for marks on the test, cause it's a negotiation, 'I
give you, I give you this mark' rather than, um, (you earn a
mark, yeah) you earning it or whatever it is right (yeah) so it's
a constant fight, I mean I've had colleagues go on stress leave
because of the abuse of the students in this regard. The
relentless bullying and hostility and comments it's never
ending. The the lines that are crossed are (yeap), you know,
just mind blowing sometimes.
7 1 they don't have an understanding of what good teaching
practice is they don't know what our tests should or should not
look like. Um, they don't know what good assessment is in
ESL
10 33 we do have student advisory panels (em hem) that are ESL
leadership, where there is a representative from each class
who comes forward. All they talk about is tests, you know, all
that talk about is tests, the teacher doesn't prepare me for the
test. What they really mean is that the teacher it not teaching
to the test
10 42 The other thing they complain about is the text books. You
know, we don't have enough time to finish a text book. ‘No we
don't’. The textbook doesn't align with the outcomes, ‘no it
doesn't’, you know, And the teacher doesn't always use the
textbook, ‘no she doesn't or no he doesn't, because it's not
good enough right’

Code
Student IELTS test
perceived as difficult
Student belief that
grade is a negotiation.

Students don’t have
understanding of CLT
or proficiency
assessment.
Student frustrations of
not being taught to the
test.

Expectation of
textbook focus.

Summary. The language learners Gary’s interacts with generally don’t have an adequate
understanding of CLT. They tend to arrive with an expectation of a textbook focus with

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

111

corresponding teaching to the test, grades are seen a negotiation between instructor and learner.
When they do not find this many learners become frustrated.
Table 5.
Theme Category: Assessment Structure
Page Line
Textual Data
4 5 not everybody else is, is confident and capable to do those
final assessment pieces

4 16 We actually have maps for each exam that show which
test items align to specifically which outcome
4 32 we actually had to map grammar outcomes to the CLBs

2 6

2 10
2 15
2 18

2 42

3 19

we are COB based we work with the Canadian Language
Benchmarks which suggest that any of our, um
assessments should also be, therefore be tasked based
On top of that though, we have obligations to, sort of
academic obligations
our mid-terms and our finals are are standardized. We
offer the same midterms and final across levels
our quizzes and then our projects that are offered within a
level are entirely teacher based. So they could focus on a
particular skill
The rubric that assesses whether or not they were able to
complete the task, whether or not they were actually able
to write the email. But there are also elements on the
rubric that look at pragmatics, the use of, umm, correct
expressions, umm, there is a grammar piece for level
appropriate, umm, grammar expectations
CLB tasks, umm, that are designed by, umm, Canadian
Centre for Language Benchmarks (em hem) are holistic
they don't, um, evaluate the grammar or the pragmatics or
the vocabulary. They just look at them holistically which,
so we couldn't, when it came to providing an ELP like an
English Language Proficiency level for, ahh, in-house or
even external mattering, it wasn't sufficient because we
needed to be able to say academically speaking this
persons got the grammatical, vocabulary skills

Code
Standardized assessment
designed by the few that are
qualified to contend with
the complexities of
assessment creation.
Standardized assessment is
mapped to outcomes.
Grammar points extracted
from ‘can do’ statements for
rubric development.
Outcome based with Task
based assessment.
Outcomes directed to EAP
context
Standardized testing
In-class isolated skill based
quizzes comprise small part
of summative grade.
Rubric for standardized test
attempts to avoid holistic
interpretation. (focus on
forms)

Holistic nature of Outcome
model was not regarded as
sufficient to assess
readiness for entry into
undergraduate study. And
therefore was modified.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
5 24 when the assessment actually comes they see on the
rubric like, umm, must demonstrate mastery or control or
developing control of this particular form
5 16 a learner wont progress up into a 6 unless they have
demonstrated a certain degree of competency with that
form

112
Outcomes directly state
explicit grammatical forms.
Reverting back to forms for
passing requirements

Summary. Gary’s institution utilizes an Outcomes Based curriculum created in-house
for their particular EAP context. The outcomes are modified to include the non-holistic
grammatical and vocabulary competencies as the holistic nature of outcome based curriculum is
considered insufficient to assess readiness for entry into undergraduate study. Standardized
assessments are aligned to the outcomes. The Rubric for the standardized test attempts to avoid
holistic interpretation by focusing on grammatical forms. Additionally, in-class isolated skill
based quizzes comprise small part of the summative grade.
Themes Relating to the Experience of Washback
Tables 6 through 12 depict Gary’s experience of washback.
Table 6.
Theme Category: Experience of Assessment Alignment
Page Line
Textual Data
4 11 Strongly, powerfully, really really good. Um, it took
us. I had to have a bit of a stern talking to. I went on
mat leave after developing our first round of exams
and when I came back, umm, they had the tests, the
teachers had gone in and altered exams
5 9 Like grammar is included as part of a lesson perhaps
as a pre-task or a post-task analysis for example if if a
teacher is doing an actual task cycle

5 33 The students want it and this is, the issue is that the
teachers prefer not to. Umm, I can say collectively we
want to be able to teach how to communicate well in
English. What the students want and ask regularly is

Code
Assessment alignment
achievable but only when
created and maintained by
those qualified in assessment.
Teaching to the Test - Belief
that instructors are not using
skill building methodologies
despite assessment having a
forms component.
Teaching to the Test – Students
strongly conditioned to focus
only on test.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
‘Is what you're teaching now teacher on the test’, ‘no’,
‘then I don't want to learn this’ (yeap). And this is
where we get arguments or, um, students don't buy-in,
they fight, they check out mentally, mentally check out
whatever it is. If it's not on the test many of them have
no interest in learning it
6 1 Oh they and we could talk ourselves blue in the face
they they don't have the same goal as we do when it
comes to the classroom and so it's a battle
9 28 , whether it's the workplace or an academic program
it's nice to know what the gaps are because we want to
be able to address them. Like 'oh we weren't even
testing for that' do you know what I mean, like, um.
Now we can see if they can do it, right (em hem). Um,
or roll it back and looking at our outcomes and saying
‘well our outcomes will never actually meet the thing
that you're doing so do we need to develop another
program for it’.

113

Teaching to the Test - Students
strongly conditioned to focus
only on test.
Positive washback – a well
aligned standardized test help
isolate gaps that need
addressing. OBE allows gaps
to be addressed systematically.

Summary. Assessment is achievable only when created and maintained by those
qualified in assessment. Neither Gary nor other instructors are not using grammar focused
methodologies despite assessment having a forms component and the fact that students are
strongly conditioned to focus only on the test. The advantage to a well aligned standardized test
is that it helps isolate gaps that need addressing. Additionally, utilizing the outcomes based
model allows gaps to be addressed systematically.
Table 7.
Theme Category: Experience of Indirect Student Washback
Page Line
Textual Data
6 9 I'm having this fight with a lot of learners this year who are
not um achieving what they would like to achieve because
they think they are awesome
6 12 ‘You know at the end of the day these are the outcomes that
we have, that you have to show me you have to demonstrate
to me that you can do’, 'well I can do that' 'well then why
why aren't you doing' right (em hem).
6 15 they negotiate for marks on the test, cause it's a negotiation,

Code
Student inflated belief in
proficiency level.
Student frustration of
Outcomes based system.

Belief that grade is a

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
'I give you, I give you this mark' rather than, um, (you earn a
mark, yeah) you earning it or whatever it is right (yeah) so
it's a constant fight, I mean I've had colleagues go on stress
leave because of the abuse of the students in this regard. The
relentless bullying and hostility and comments it's never
ending. The the lines that are crossed are (yeap), you know,
just mind blowing sometimes.
6 31 So um, It is intense over here so I guess that students just are
not on the same page, umm, don't care to be on the same
page and it's, um, really stressful (yeap yeap I understand).
7 1 they don't have an understanding of what good teaching
practice is they don't know what our tests should or should
not look like. Um, they don't know what good assessment is
in ESL
8 19 It's not helping right, cause they the students the navigators
said I should try and skip the next level and I'm like ‘why
would you want to do that?’ (em hem). So, yeah, largely a
gap between what is realistic and what their expectations
are, so. (you can't power study a language, its..) No, it's like
math, figure out memorized formulas like [sarcastic tone].
So yeah, very big disconnect.
9 4 We'll if they are not blaming the teacher then I, they are
blaming themselves so. ESL teachers will take the hit. But
you're right the self-depreva.. they beat themselves up. like
it's my fault, well, no maybe you don't have good
partnership between, you know, whoever is instructing you,
like it is a partnership in a sense
9 7 a learner can't be expected to know all the principles of
language learning, they don't, that's not their job to know
how it works. They need a good mentor, they need a good
facilitator who can be a part of that process for them
10 30 They perceive it as such, but it is not
10 33

we do have student advisory panels (em hem) that are ESL
leadership, where there is a representative from each class
who comes forward. All they talk about is tests, you know,
all that talk about is tests, the teacher doesn't prepare me for
the test. What they really mean is that the teacher it not
teaching to the test
10 42 The other thing they complain about is the text books. You
know, we don't have enough time to finish a text book. ‘No
we don't’. The textbook doesn't align with the outcomes, ‘no
it doesn't’, you know, And the teacher doesn't always use

114
negotiation.

Dealing with learner
desires is stressful.
Students don’t have
understanding of CLT or
proficiency assessment.
Talking navigator’s poor
advice as it aligns with
their belief of their
learning situation.

Self-Blaming or teacher
blaming for failure.

Students can’t be
expected to understand
principles of language
learning.
In-house test perceived
as easier than IELTS.
Student frustrations of
not being taught to the
test.

Expectation of textbook
focus.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

115

the textbook, ‘no she doesn't or no he doesn't, because it's
not good enough right’
Summary. Learner’s frustrations of experiencing a different learning environment cause
them to pressure instructors towards the traditional teaching and assessment practices they are
familiar with. These frustrations include: a lack of understanding of an Outcomes based
curriculum, an inflated belief in their proficiency level, a belief that their grade is a negotiation,
being taught to the test, and an expectation of a textbook focus. The coercion is intense and
amounts to bullying causing much stress for instructors. Learners can’t be expected to
understand principles of language learning and so tend to blame themselves or their instructor
teacher for their failure. To compound this assigned student advisors often give them poor advice
that aligns with their misguided belief of their learning situation.
Table 8.
Theme Category: Experience of Indirect Academic Course Instructor Washback
Page Line
Textual Data
7 15 , we have pressure from within to be able to provide effective
language instruction for them to ladder in
7 38 Oh my God yes, yes, there is so little respect from them for
what we do. And as language instructors even within the field
we are not even considered professionals like they are
8 30 , some preservice high school teachers who had opted to take
this intro to ESL methodology and they had this attitude ‘well
I'm teaching science, it's not my job to teach that to teach
English. And I lost it. I said that yes it is, I said that ‘you are
teaching everybody in that room how to communicate in a
way that is expected within the field of science, right, the
terminology the when your doing scientific reports, how to
report on your results or talk about your experiment’. Like
that is language, ‘You’re building academic language
proficiency for everybody whose in-house first language or
second language’. ‘It is your responsibility to teach them how
to communicate about science in a way that is expected in
English’. ‘It is your job to do that so you need to know what

Code
Pressure to perform well
from academic course
instructors.
Lack of respect

Don’t see their role as
facilitators of
communication for
language learners.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

116

that means, right, not just for a second language learner but
for anybody whose in their..’. ‘It just might be easier or faster
for the native speaker to pick it up than your non-native
speaker’. ‘But yeah you're part of that, um, specific language
learning process and if you don't recognise that, then your
only serving a certain number of students
9 42 getting the feedback after we developed that new assessment
has been really good because we stopped getting, ah, there
was less screaming than ha ha than before about learners
being ready or not ready. So that's been better, you know,
communicating about, um, what they want and whether or
not we can account for it.

Feedback important to
reduce tension and
increase
communication.

Summary. Academic course instructors who teach the courses that language learners
feed into: don’t see their role as facilitators of communication for language learners, do not
respect language department on equal terms as academic departments, and expect unrealistic
language proficiency gains from the time learners spend in the language program. However,
listening to their feedback has been important to reduce tension and increase communication
Table 9.
Theme Category: Experience of Institutional/Administration Washback
Page Line
Textual Data
7 17 we are serving too many masters. Including our own
institution who have little understanding and often times
little respect for what it means to be an ESL professional
and what it means to be a good ESL program.
7 32 I sat in meetings where we were just getting screamed at
7 41 But, you know, it's even professionally we are not
considered on an equal level our program isn't taken
seriously because we are foundational and our learners
language proficiency is always called into question. Um,
you know we are not act.. we are not strictly an academic
program for goodness sakes
7 45 , ‘you know, we have all of these other influences going
into our programming our curriculum development and
our assessment so no we are not making you happy
either’, they scream at us, they scream.
8 12 ‘Well, they have been here for four years, why can't they

Code
Little understanding and
respect for CLT and the
structure needed to support
it.
Intense and stressful
Lack of respect via
professional status.

Lack of institutional
understanding of other
stakeholder influences to
curriculum.
Lack of institutional

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
do it?’, ‘Cause it's not, it's a skill set that takes a heck of a
lot longer to learn’

117
understanding of time
required for academic
language proficiency.

Summary. Institutionally there is little understanding and respect for CLT and the
structure needed to support it. The structure formally denotes a lower professional status for
language instructors. This amounts to an intense and stressful relationship within the university.
There is a lack of institutional understanding of how stakeholders influence the curriculum, and a
lack of institutional understanding of time required for academic language proficiency to
develop.
Table 10.
Theme Category: Experience of Alternative Assessment Washback
Page Line
Textual Data
3 31 The portfolios were a hot mess. Mostly because our
team, the majority of our team, umm is not quali,
they don't make good tasks, they don't know how to
create a performance either like it's. So the
portfolios failed us and we ended up teaching to the
test
3 34 if the students didn't do well in a test they would
bully teachers into making a new portfolio task that
is simpler or to just repeat the task itself until they
finally got it right
4 49 thank God my colleagues, you know, have the
humility to say to say like I'm not qualified to make
an assessment task that is weighted such that it, you
know, and it shouldn't, you know, not without
development or support or whatever.
4 3 It wasn't consistent, you know, there was, no, you
know, there was no way of checking

Code
Alt. assessment “A hot mess”. Felt
instructors were not capable of
creating and assessing tasks.
(confusion of alternative
assessment – trying to standardize
alt. assessment)
Alt. assessment encourages strong
negative student washback ‘Bullying’.
Alternative assessment not viable
without knowledgeable qualified
instructors with institutional
support.
Assessment consistency not
achievable in institutional
situation in which accountability
is needed.

Summary. Previous alternative assessment structure created ‘a hot mess’. Instructors
were not capable of creating and assessing tasks. It encouraged strong negative student washback

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

118

that amounted to ‘Bullying’. It is not viable without knowledgeable qualified instructors and with
institutional support. Accountability hard to achieve with inconsistent nature of alt. assessment.
Table 11.
Theme Category: Instructional Desires
Page Line
Textual Data
5 39 I can say with a great deal of confidence that we don't
want to teach that way
7 26 I think they need to go away, ha ha.

9

13 they need input that they can understand

11 5

there would be more time in order to teach and to
assess.
11 6 Ideally, learners would have input at some point along
the way on content curriculum outcomes and
assessment to a point
12 9 there is not enough reflection in the institution or
departmentally on the effect of all these masters of the
of the washback of the student expectations too. Our
instructional team brings it up at meetings actually
quite regularly but leadership often just throws it's
hand in the air and I feel like were chasing our tails.
We have the same conversations all the time.
12 13 About how we'd like these very powerful systemic
changes so that we don't have these, um, problems
anymore so that it is better for the students.
12 19 Our leadership doesn't seem interested in asking the
questions or finding the answers to the type of
questions that you’re asking me today that are really
important. Umm, and I think that is a very crucial,
critical missing piece to what is happening at least
with our institution, possibly within many others

Code
Instructors do not want to
teach to the test.
Emotive response to
annoyance with stakeholders’
lack of respect and knowledge.
Comprehensible Input needed
from academic course
instructors.
More class hours for
instruction and assessment.
Learner involvement in
curriculum (desire for student
centeredness)
More reflection system wide,
systemic changes needed

Systemic changes needed

Leadership not interested in
asking the right questions or
finding right answers.

Summary. More reflection system wide and Systemic changes are required. Instructors
do not want to teach to the test despite pressures to do so as stakeholders lack respect and
knowledge. Academic course instructors need to understand their role in ongoing communication

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

119

development for language learners. Greater learner involvement in curriculum (desire for student
centeredness). More class hours for instruction and assessment.
Our leadership doesn't seem interested in asking the questions or finding the answers to
the type of questions that you’re asking me today that are really important. Umm, and I
think that is a very crucial, critical missing piece to what is happening at least with our
institution, possibly within many others.
Table 12.
Theme Category: Overall washback effects
Page Line
Textual Data
12 16 It's not healthy for our instructors because we are not doing
our jobs well and so therefore we are not serving our students
very well or serving the institution well and we don't have
enough time to have very good reflective meaningful
conversations about it.
7 9 , we are not doing anything well, we are doing most things
moderately effectively

Code
Not healthy working
environment and no
real fix is being looked
at.
Inability to preform
optimally.

Summary. Unhealthy working environment where instructors are unable to perform
optimally. Issues are not seriously being looked at.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

120

Appendix H: Codebook - Jaden
Themes Relating to the Context of the Instructional Experience
Tables 1 through 5 depict the context of Jaden’s instructional experience.
Table 1.
Theme Category: Instructional Background
Page Line
2
3
3
5

Textual Data

48
1
1
18

I was in the committee who made it
my PhD is in assessment
I am an IELTS examiner trainer too.
No, (Oh, ok) Because we work in [City] our low
season is the winter, so we drop by more than 50% in
January, February and March, April (oh ok) it's so
cold. Because of that we have a lot of turn over. I
think we have more than like 100 teachers right now
and only 19 is full time and like another is part-time.
A lot of turnover in the part-time sessionals
1 18 in my current situation there is not much
communicative teaching, we are just doing prep

1 19 but when I teach research writing or I teach four skills
or a bridging program I target, I use, authentic text, so
I bring what is interesting for the students into the
classroom. For example, I take, um, give the students
a sense of ownership to the program (em hem) for
example, umm, I get students to bring their own
material.

Code
On committee that make test
PhD in assessment
IELTS examiner
No training given as high
amount sessional instructors are
employed.

Currently instructing IELTS
Prep classes only with little
CLT.
Experience in other programs
at University.

Summary. Jaden is an instructor at a Canadian University, and is also on committee that
creates assessment. He has a PhD in assessment and is an IELTS examiner. Currently he is
instructing IELTS Prep classes utilizing only a little CLT. However, he has had experience in
the gate-keeping proficiency based programs offered at his institution. His institution employs a
large number of sessional instructors and does not provide in-house training to current
instructors.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

121

Table 2.
Theme Category: CLT Interpretation
Page Line

Textual Data

1 19 but when I teach research writing or I teach four skills
or a bridging program I target, I use, authentic text, so
I bring what is interesting for the students into the
classroom. For example, I take, um, give the students a
sense of ownership to the program (em hem) for
example, umm, I get students to bring their own
material.
1 24 I get them to create student lead discussions every
week (em hem). For example, they would lead a
discussion for a given topic that they like to discuss in
their class. And also, um, it is very student focused
2 1 for grammar. I actually have the students teach me a
grammar point. In day one I get them to write
something, and I look at their writing and I pick up
two grammar points that they, uh, they are systemic,
that they have problems with and to teach the class.
And I get them to teach the class on those two
grammar points. I get them to make the exercises.
They are just grammar exercises or editing exercises
where they actually have to find a text and they have
to purposely find those two grammar points that they
do (yeah) and so their, so they take owner
2 14 So, I one thing I do in the class, I tell them the reason,
my reason behind why I do it (em hem). Students have
to be an active participant in the classroom to

Code
Use of authentic texts.

Student centered

Grammar teaching makes up
part of class structure at least in
the beginning.

The reasoning behind
classroom activities is given as
a tool to enable buy-in from
students.

Summary. Jaden favors the use of authentic texts in his teaching. He maintains a student
centered class. To ensure buy-in from students he gives the reasoning behind classroom
activities. Grammar teaching makes up part of class structure at least in the beginning.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

122

Table 3.
Theme Category: Learning Context
Page Line

Textual Data

1 27 They just, like when I see them, they have just arrived,
like one day before (yeap). There is a lot of
acculturation that needs to take place. So we do lots of
things, we do role-play, we do needs assessment and
we make sure that students will be able to function in
universities in Canadian context at the undergraduate
level. In their countries is, there is not group work,
there is no mediation of any sort. So I need to focus on
that to be able to make sure that students can function
in that, can be able to function within that context of
masters of PhD program where they have to work in a
small groups or work in pairs to be able to do
collaborative assignments and activities as such.
2 10 And those people, um, before they finish their
bachelor they, and then they study English for 14
years. So, they have already been taught grammar.
What they have problems with, is that internalizing
those lessons, right.

Code
Acculturation needed at start of
program

Students come with 14 years of
grammar lessons

Summary. At the start of the ESL program a lot of acculturation education takes place to
ensure students will be able to function in universities within the Canadian context at the
undergraduate level. Students come with 14 years of grammar lessons, what they have problems
with is internalizing those lessons.
Table 5.
Theme Category: Assessment Structure
Page Line

Textual Data

2 23 Well for the gatekeeping program the assessment is,
um, we actually only have one standardized
assessment. The exit test is anything that we deem

Code
Purely standardized testing

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

2 29

2 43
3 12

3 32

3 36
6 23

6 37

7 46

appropriate for the level and for the student needs, for
example we have standard listening test which is a 20
minute of a university lecture
in IELTS there is no synthesis or, um, and synthesis or
analysis it's actually a purely a listening task, right (em
hem). So, in the class, in the gate keeping course I'm
more interested in, are they, not just about do they
have the language skills but do they have the language
and the academic skills to be successful in the
university.
Yeah, it's like two or three or four different versions of
I think because I am aware of those issues when I was
designing the test for our needs. Because the needs
that we had in the, in the class was not just about
language skills it was about more about, are they able
to function in an academic background
we believe that part of the test construct is to be able to
mimic what they will see in their stages at a university
level
are there any sort of alternative assessments that are
part of the summative grade? (No)
No, we'll no, I have bi-weekly, every two weeks I have
a test, (Yes, but that's not summative. That's just part
of the class?) Part of the class, Well it is progressive so
it's all part of their final score. (Yeah) right, um, so it'
summative for the language points that we've been
using for those two weeks
So with the test that I construct myself sometimes, you
know, it doesn't have to be a test. Or I just need to
understand that they understand the points of those
two weeks. So sometimes I do a class crossword
puzzle
No, because I don't have the freedom to push their
grade through because all the scores that they see, they
see almost immediately so as soon as. Like the
students can calculate their scores on average. So there
is no pushing through of students (Ok, Ok) right
alternative assessment like if I do a crossword puzzle
is it for like a vocabulary test. It is still mainstream test
presented in a different way. (Yeah, Yeah) It is not an
actual engaging their skills in an alternative method.
(Yeah, It's not a subjective grade, yeah) Yes, so there
is no subjectivity in the test except for when there is

123

Includes higher order thinking
tasks and questioning.

Developed in house
EAP focused

Testing proficiency within
standardized themes.
No Alt assessments make up
summative grade.
An in-class portion of final
grade

An in-class portion of final
grade

In-class tests are not subjective

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

124

synthesis for an argumentation for when I am grading
their style of language use
Summary. The institute utilizes a purely standardized assessment structure utilizing
standardized themes. It is developed in-house with a focus of English for Academic Purposes
(EAP), and thus includes higher order thinking tasks and questioning. In class tests make up a
small portion of the final grade, however they are standardized and are not subjective in nature.
Themes Relating to the Experience of Washback
Tables 6 through 14 depict Jaden’s experience of washback.
Table 6.
Theme Category: Experience of Assessment Alignment
Page Line
6 1

Textual Data

Code

sure that happens for the final test. But for the test that
I'm am allowed to design was a smaller test for what I
am doing inside the class. I think they are more
motivated because they, um, they are using what they
learned in the classroom in a meaningful way that is
immediately identifiable for them, um, the goals and to
see their own progress of whose in the classroom,
(yeah) like I mentioned with my grammar presentation

In-class tests able to be better
aligned to classroom practices.

Summary. In-class tests able to be better aligned to classroom practices particularly as
topics can be tailored to student interests.

Table 7.
Theme Category: Experience of Indirect Student Washback
Page Line

Textual Data

1 42 There is no push back because a lot of my students is,
my students are from China and I think that common
perception is teacher knows best and they decided to
come to Canada because of the change in teaching

Code
No pushback towards CLT.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

125

style.
6 1

sure that happens for the final test. But for the test that
I'm am allowed to design was a smaller test for what I
am doing inside the class. I think they are more
motivated because they, um, they are using what they
learned in the classroom in a meaningful way that is
immediately identifiable for them, um, the goals and to
see their own progress of whose in the classroom,
(yeah) like I mentioned with my grammar presentation

In class tests able to be better
aligned to classroom practices.

6 10 Most of the time they are saying I, that they see that I
am being inclusive of their needs. and the topic that
they might be interested in (yeap) and so in that
positive washback also. (Yeap, so because it's
meaningful, therefore it's engaging for them, um,
because it's well aligned to the test they can see that
the wider goal is met as well as the immediate goal of
the test.
6 15 (So I think you're telling me that it all needs to come
together.) Yes, you can't do one without the other right,
if the test is good but the students don't know why they
are doing the test or the students don't notice the goals
of that test, what the test is doing, then it doesn't work
because it doesn't motivate the students. There is no,
ah, the students don't have the realization that there
could be a positive washback to testing
6 15 (So I think you're telling me that it all needs to come
together.) Yes, you can't do one without the other right,
if the test is good but the students don't know why they
are doing the test or the students don't notice the goals
of that test, what the test is doing, then it doesn't work
because it doesn't motivate the students. There is no,
ah, the students don't have the realization that there
could be a positive washback to testing

Inclusive to needs in classroom
tasks and tests. Particularly as
topics can be tailored to
student interests.

Students need to know goals of
test in order to motivate.

Students need to know goals of
test in order to motivate.

Summary. No direct washback on CLT “my students are from China and I think
that common perception is teacher knows best and they decided to come to Canada because of
the change in teaching style”. Less student washback with in-class testing as it can be better

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

126

aligned with learners’ needs and interests and reasoning behind in-class testing is clear.
However, students need to know reasoning behind assessment.
Yes, you can't do one without the other right, if the test is good but the students don't
know why they are doing the test or the students don't notice the goals of that test, what
the test is doing, then it doesn't work because it doesn't motivate the students. There is no,
ah, the students don't have the realization that there could be a positive washback to
testing.

Table 8.
Theme Category: Experience of Indirect Academic Course Instructor Washback
Page Line

Textual Data

4 13 No, um, this is why that test was revamped last year
and this is why we did, so we had dialogue, dialogic
process between the professors of the different
facilities, because we keep track of all those students
who enter the university through the program and we
have seen the trend that their success has been
dropping (em hem), and this is why the course was
just revamped, this session the course was revamped
for that 7.5 level
4 24 a lot of professors are, they are not native speakers
either. (yeap) Right so I think they have an
understanding, um, so I think there are some aspects
that they are more lenient with their language skills. If
they can understand what their student is trying to
communicate then they are fine with that. Even if they
are all systematic mistakes and there's patches that
they can't understand. What they have a problem with
is the, um, the academic skills themselves. About, can
they synthesize information are they just regurgitating
what they, you know, what they heard. Are they
understanding the key, are they only being able to
identify key words in the text based on the question
types (em hem) and be able to just copy the whole
thing and being able to get the points, as such, (em
hem) Right, so it's more of the communicative intent
of the curriculum and the task design itself that I think
people are more concerned about, are they able to, are

Code
Academic course instructors
not happy with proficiency of
students, dialog with academic
course instructors during
revamp of course.

Need for higher order thinking,
while fine with grammar in
accuracies.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

127

they able to complete with the students, with the
students with a Western style of education, and be able
to discuss ideas in that level.
Summary. Academic course instructors voiced concerns about second language learners
failing to cope with academic studying environment. Which caused course structural changes to
be implemented. Academic course instructors found a greater need for communication in higher
domains of thought, while were fine with grammar in accuracies.

Table 9.
Theme Category: Experience of Institutional/Administration Washback
Page Line
5 7

Textual Data

Well, if you're teaching credit courses, even if you're
taking like four, I don't know, four different sections
of the same course. Every teacher teaches it totally
differently. The assignments are not going to be the
same. (yeap) And so it's, my feeling is that if they
trusted their instructors who had academic freedom
enough. (yeah, Do they trust?) We'll when I see more
and more standardization I don't see it happening like
that, but, their justification is that we have so many
new teachers and that the new teachers need to be on
board. And it's on board and pretty much easier if
everything is standardized (yeah) and that's the
justification
5 24 It's good for the new teachers like they are saying. But
for the ones who have experience, for the ones who
have knowledge and a higher academic background
get a little bit trapped. (Yes) so it's hard to envisage
how you could move forward in that setting,
7 5 So yeah, there is no alternative test and I think that the
management looks, you know frowns on alternative
testing because they are more worried about students
comparing their assessment techniques and
complaining about it.
5 44 I mean our institute is going towards more and more
standardization, standardization of tests,

Code
Limiting academic freedom for
standardization.

Over bearing control.

Management frowns on alt.
testing as opens door for
student complaints.

Moving more and more
standardized in content.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

128

standardization of curriculum, standardization of even
homework assignments, um, there is. I find that there
is less and less academic freedom. Because it's easier
to have 10 teachers teaching exactly the same thing,
then it's easier for assessment of both the teacher and
the students (yeap) and I think that works a bit against,
um, Communicative Language Teaching because you
don't have the freedom to do what you want based on
the students’ needs and desires
Summary. Administration endeavors to limit academic freedom for standardization.
“Their justification is that we have so many new teachers and that the new teachers need to be on
board. And it's on board and pretty much easier if everything is standardized and that's the
justification”. “Management frowns on alternative testing because they are more worried about
students comparing their assessment techniques and complaining about it.”
Management unsupportive of CLT in practice:
I find that there is less and less academic freedom. Because it's easier to have 10 teachers
teaching exactly the same thing, then it's easier for assessment of both the teacher and the
students and I think that works a bit against, um, Communicative Language Teaching
because you don't have the freedom to do what you want based on the students’ needs
and desires.
Table 10.
Theme Category: Experience of Alternative Assessment Washback
Page Line

Textual Data

7 19 I would like to see less standardization. I would like to
see more teacher adjustment, more respect for teachers
expert judgement that would allow students to pass.
Because even when assessment says that a student
passes, you know that this student is not going to do
well, (Yeah) at university or sometimes when a
student passes numerically, you know you see what I
mean right. Even if they fail then you think that they,

Code
Less standardization

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
you know that they have the motivation and they are
able to excel. Because if our mandate is not just about
language it's about academic readiness to enter
university I think teachers should be allowed to select
their expert judgement in those cases and I think
because of over standardization that we losing some of
that
5 31 Well because a lot of my students are Chinese, testing
in a way has positive washback because I guess it's
that the students is familiar with that type of extrinsic
motivation, right, like some students like tests (yeap)
like they, if their score is a 10 out of 10 they. Like
some students are motivated. And I think it has a lot to
do with intrinsic and extrinsic motivation (em hem) of
assessment, um, and it depends on the class a lot of the
students are not, you know, are not intrinsically
motivated, you know, their parents told them that they
needed to study, they needed to do a master’s or they
needed to do a bachelors and if you know anything
with the Chinese system is that they could get into a
job if they went to a good Chinese university than
[unclear] anywhere else, (but) because of, they didn't
the scores that the needed for their university entrance
exam their parents decided ok next best thing is for
you go get a [unclear] English and have a degree
outside of the country. So some of them are lacking
that intrinsic motivation and the tests provide some
extrinsic motivation
3 40 There is very little standardization (em hem) like for
example, um, I had a teaching literacy workshop the
other day, um, and the teacher was saying that you
could use poetry or painting or things to assess their
language ability right. (yeah) But a lot of those things
are very subjective (yeap) and a single rubric might
not be able to encapsulate all the different
representations of alternative assessment. (yeap) So
unless it was a standardized alternative assessment it
kind of defeats the purpose of it. (em hem) Supposed
to do it in a classroom setting where it is shaky.

129

Alt. assessment. testing culture
and students are extrinsically
motivated, learning English is
seen as an obstacle not a goal.

It’s ‘Shaky’ as so subjective –
if made too standardized then it
defeats its purpose

Summary. Due to testing culture and students are generally extrinsically motivated. For
many students, learning English is seen as an obstacle rather than goal. And the tests provide

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

130

some extrinsic motivation (that Alt. assessment can’t provide). It’s ‘Shaky’ as so subjective, if
alt. assessment is then standardized then it defeats its purpose (Standardized alternative
assessment does not work).
Table 11.
Theme Category: Instructional Desires
Page Line

Textual Data

7 19 I would like to see less standardization. I would like to
see more teacher adjustment, more respect for teachers
expert judgement that would allow students to pass.
Because even when assessment says that a student
passes, you know that this student is not going to do
well, (Yeah) at university or sometimes when a student
passes numerically, you know you see what I mean
right. Even if they fail then you think that they, you
know that they have the motivation and they are able
to excel. Because if our mandate is not just about
language it's about academic readiness to enter
university I think teachers should be allowed to select
their expert judgement in those cases and I think
because of over standardization that we losing some of
that
7 38 I think that professional development of how to make
a test, (em hem) you know as something as simple as,
you know, if you want to allow teachers to make tests
you need to have a standardized understanding of what
a test, you know, the test construct. (em hem) and be
able to understand inference and validity argument for
why a test is valid or not valid. um, (is it reliable and
all of this, yeah) or the practically of the reliability or
the or be able to assess the target use of the language
domain and be able to even understand something like,
you know, what is the meaning of a test specification.
But there is no professional development provided to
teachers
8 45 and even managers themselves I don't think they
understand um how much work it takes and at one
person cannot do, create a test specification for a test,
it has has to be done in a consensusitive in an
interpretative way and it takes a lot of time to create a

Code
Less standardization

Professional development in
assessment.

Management don’t understand
what is involved in making a
test.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
single specification for one expert test right. So I think
they just don't understand
Summary. Less standardization, Professional development in assessment for instructors
and management.

131

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

132

Appendix I: Codebook - Tim
Themes Relating to the Context of the Instructional Experience
Tables 1 through 4 depict the context of Tim’s instructional experience.
Table 1.
Theme Category: Instructional Background
Page Line
1

Textual Data

11 I started formal teaching back in 1990. Bilingual
school in [home country] and uh uh, it's 100%
communicative or that was. Then I moved down
to Saudi Arabia where as you know we worked
together, (yeap) and it was a basically
communicative student and I mean really
students centered methodology. However when
I got here to the UAE. The [Company] final
assessment called the IELTS and the teachers
basically prepared their students for the IELTS
exam.

Code
CLT under both extremes of
assessment structures.

Summary. Instructor with extensive experience in CLT instruction under a purely
Alternative assessment structure and purely a standardized assessment structure.
Table 2.
Theme Category: CLT Interpretation
Page Line
2

Textual Data

44 I started formal teaching back in 1990. Bilingual
school in [home country] and uh uh, it's 100%
communicative or that was. Then I moved down
to Saudi Arabia where as you know we worked
together, (yeap) and it was a basically
communicative student and I mean really
students centered methodology. However when
I got here to the UAE. The [Company] final
assessment called the IELTS and the teachers
basically prepared their students for the IELTS
exam.

Code
Advocates student centered
methodology but bends his CLT
instruction to the needs of the
institute.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

133

Summary. Standardized testing induces artificial learning environments which induce
alignment of assessment to CLT (memorizing, etc.). Advocates student centered methodology
but bends his CLT instruction to the needs of the institute.
Table 3.
Theme Category: Learning Context
Page Line
5

5

1

1

6

Textual Data

they have, they continue. They have to continue
with academic speaking academic writing umm
etc etc. because there level. Right now the
education department I think is requesting, It's
still requiring 6.5 IELTS.
35 But you need, It's proportional to the amount of
time in input. (yeap) assuming you're highly
motivated and you actually want to learn the
language (yeah) so you can't go beyond that,
beyond human capability so I think, I think
sometimes you over estimate what students
show be able to learn in a year in a two year
academic course at university we need to I don't
know what 500 hours (yeah exactly) I don't
know how many hours you actually do (yeah)
you know what I mean, 20 hours a week I don't
know how many weeks multiply that. I don't
think it goes beyond 400 or 500 hours. So what
can you expect (ha ha exactly) sometimes we
over es.. we don't over estimate we over expect
their level of English, you know, it's
proportional to the number of hours and the
motivation etc etc.
16 the IELTS and the teachers basically prepared
their students for the IELTS exam. So in that
sense I personally didn't break my tradition as
being communicative but I was forced to
modify my teaching strategy or my approach so
we could actually prepare students for the
IELTS exam
22 we fell back into the grammar teaching because
as you know the IELTS has a huge grammar

Code
Continue English studies alongside
undergraduate courses.

Over expect what can be obtained
by students.

IELTS prep.

Do best to make the standardized
nature communicative.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

3

134

component (yeap) so and we prepare them for
the writing, but again it was artificial because
they had to be prepared for this kind of task 1,
task 2, kind of writings. (yeap) so it becomes
very artificial really. and I can't help it because
the institution it linked to the IELTS and the
IELTS is linked to that and so your forced to do
as they need (em hem). Though I make it as
communicative as possible.
Yeah, I miss that too because I didn't even in
Deeper teacher satisfaction with alt.
fact prepare classes, I mean we, I just worked on assessment system.
what I saw. Authentic needs of the students
(yeah) so I start interacting with them, I find
what their problems are and I develop
something based on that as we go along.

8

Summary. Continue English studies alongside undergraduate courses. Standardized
context – a strong focus on preparing for IELTS exam, teaching to the test vs. Alt. assessment
content: “I just worked on what I saw. Authentic needs of the students (yeah) so I start
interacting with them, I find what their problems are and I develop something based on that as
we go along.”
Table 4.
Theme Category: Assessment Structure
Page Line
1

1

Textual Data

16 the IELTS and the teachers basically prepared
their students for the IELTS exam. So in that
sense I personally didn't break my tradition as
being communicative but I was forced to
modify my teaching strategy or my approach so
we could actually prepare students for the
IELTS exam
22 we fell back into the grammar teaching because
as you know the IELTS has a huge grammar
component (yeap) so and we prepare them for
the writing, but again it was artificial because
they had to be prepared for this kind of task 1,
task 2, kind of writings. (yeap) so it becomes

Code
IELTS prep.

Do best to make the standardized
nature communicative.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

135

very artificial really. and I can't help it because
the institution it linked to the IELTS and the
IELTS is linked to that and so your forced to do
as they need (em hem). Though I make it as
communicative as possible.
Summary. IELTS focused. IELTS has a huge grammar component.
Themes Relating to the Experience of Washback
Tables 5 through 10 depict Tim’s experience of washback.
Table 5.
Theme Category: Experience of Assessment Alignment
Page Line

Textual Data

Code

2

44 I think it's artificial, like when they write right
now they just memorize certain, what they have
to do in the, in the introductory paragraph in the
body one and body two and then the conclusion.
Most of the language is just memorized. It's
artificial.

Standardized testing induces
artificial learning environments.

1

28 . I mean when I teach grammar I don't teach the
simple present as such. I do present situations
where you tend to, you know, to repetitive
actions, you know, I work everyday, uh, 'do you
brush you teeth?', 'yeah, I brush my teeth'. 'But
are you brushing your teeth right now'. 'Oh, no
teacher you're not you're sitting in the class'. (em
hem). So I do try to make grammar as
communicative and a, I would say, functional as
possible. But anyway I'm constrained by the
needs of the classroom, anyway.
46 So, now with the new test, it's becoming even
worst that IELTS in that it is limiting you
further.

End up teaching grammar as it is
need of the test.

1

2

1

So as the teachers as such we are all. Basically,
teaching for the test. So what we have done is,
we go around looking at the students taking the

New curriculum is more limiting.
And thus forces more pressure to
teach to the test despite measures to
avoid this (not showing teachers the
test structure of the test or have
them mark it) (double coded)
Teaching to the test.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

2

1

test and we try to remember the kinds of
questions we can see
3 Because we don't have access to the test itself.
We can see the kinds of questions and obviously
most of us have been teaching for a while so
based on that we say ok we have got these kinds
of questions, we got this kinds of questions.
We've got multiple choice questions on this type
of grammar we've got ta ta ta. So we design
something exercises drilled based on the kinds
of questions they are going to get (em hem). But
we are not teaching English as such do you see
what I mean we just preparing the students for
an exam.
37 Well, yes because that's what they're, that's what
the will be tested on. And it's not just the
students it's the institution as such

136

No access to test causes more
teaching of test taking strategies
instead of language.

Not just student but relates to
institution.

Summary. Standardized testing induces artificial learning environments which negate
alignment of assessment to CLT (memorizing, etc.). End up teaching grammar as it is need of the
test. New standardized assessment is more limiting. And thus forces more pressure to teach to
the test despite measures to avoid this (not showing teachers the test structure of the test or have
them mark it). No access to test causes more teaching of test taking strategies instead of
language.
Table 6.
Theme Category: Experience of Indirect Student Washback
Page Line
1

3

3

Textual Data

37 Well, yes because that's what they're, that's what
the will be tested on. And it's not just the
students it's the institution as such
26 But anyway basically we are almost back to the
same old grammar. (yeah) we can't do grammar
translation because we don't speak Arabic but
(ha ha) but if we could that's what we would do
to keep employed, you know what I mean.
35 Well, I think that you don't really need to

Code
Not just student but relates to
institution.
Reverting back to behavioural
grammar teaching.

Able to convince student learning is

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
convince them. I think that after one or two
weeks they realize that they can actually
understand and they can actually produce
something so it's more like a demonstration of
method. The student, realizes that he is actually
learning (yeap) so. He is not spending time to
memorize a script and spit it out like a parrot.

137

happening.

Summary. “Well, yes because that's what they're, that's what the will be tested on. And
it's not just the students it's the institution as such”. Has the effect of negating the implementation
of CLT.
Table 7.
Theme Category: Experience of Indirect Academic Course Instructor Washback
Page Line
5

Textual Data

22 Steve: do feel that um at [OPE] that the students
that the teachers above were aware of of what
level we brought could bring them up to and
their journey in English and sort of how far they
had gone?
Tim:I think there is a problem with this I mean.
What really matters is if you assume that the
student is highly motivated ok (yeap) you know
he actually engages the 400 or 600 or 500 hours
of English he takes at the college level (yeap)
What you need, you know, research tells us you
need between 700 or 1000 hours (yeah) ok

Code
Academic course instructors not
fully aware of how long language
learning takes.

Summary. Academic course instructors are not fully aware of how long language
learning takes.
Table 7.
Theme Category: Experience of Institutional/Administration Washback
Page Line
5

Textual Data

35 But you need, It's proportional to the amount of
time in input. (yeap) assuming you're highly

Code
Over expect what can be obtained
by students.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

1

1

6

3

motivated and you actually want to learn the
language (yeah) so you can't go beyond that,
beyond human capability so I think, I think
sometimes you over estimate what students
show be able to learn in a year in a two year
academic course at university we need to I don't
know what 500 hours (yeah exactly) I don't
know how many hours you actually do (yeah)
you know what I mean, 20 hours a week I don't
know how many weeks multiply that. I don't
think it goes beyond 400 or 500 hours. So what
can you expect (ha ha exactly) sometimes we
over es.. we don't over estimate we over expect
their level of English, you know, it's
proportional to the number of hours and the
motivation etc etc.
46 So, now with the new test, it's becoming even
worst that IELTS in that it is limiting you
further.

37 Well, yes because that's what they're, that's what
the will be tested on. And it's not just the
students it's the institution as such
38 If management would actually understand. We
need somebody who knows language teaching
at the top. (yeah) You know that actually
26 But anyway basically we are almost back to the
same old grammar. (yeah) we can't do grammar
translation because we don't speak Arabic but
(ha ha) but if we could that's what we would do
to keep employed, you know what I mean.

New assessment is more limiting.
And thus forces more pressure to
teach to the test despite measures to
avoid this (not showing teachers the
test structure of the test or have
them mark it)
Not just student but relates to
institution.
Management needs to be
knowledgeable in CLT
Reverting back to behavioural
grammar teaching.

Summary. Management over expects what can be obtained by students. Institutional
and student washback act in unison “And it's not just the students it's the institution as such. Has
the effect of negating the implementation of CLT. “If management would actually understand.
We need somebody who knows language teaching at the top. You know that actually”.
Table 8.

138

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

139

Theme Category: Experience of Alternative Assessment Washback
Page Line
4

2

3

3

3

Textual Data

23 obviously do to the encoding thing everybody
needs to work in the same way. You all need
need to be on the same page. (yeap) at least in
the approach, the methodology has to be
uniform that that I agree (yeap) but the problem
is you know to have enough teachers who
actually know what they are doing, you know,
that understand.
33 Well, I'd rather go for the [OPE] thing. (Yeah)
for many reasons, um, I think you do more
justice to the reality. I mean it becomes more
authentic teaching the the skills in learning
would allow them to confront random
interaction, you know what I mean, (yeap) well
the students here don't seem to be prepared for
that. You break the norms and you break away
from what they are used to, they are completely
lost, (yeap) So [OPE] I think we were process
deve.. we were assessing process (yeah) so I
think it's a more, it's a better way of doing it
because it's. I mean the student learns a lot more
4 We'll that's true. So in the process, what your
mentioning what you really need very
experienced highly qualified teachers and I don't
mean qualified by having degrees, by actually
knowing what they are doing. (yeah) So I was
really really happy at [OPE] when I was
teaching because that is the way I think people
actually learn
35 Well, I think that you don't really need to
convince them. I think that after one or two
weeks they realize that they can actually
understand and they can actually produce
something so it's more like a demonstration of
method. The student, realizes that he is actually
learning (yeap) so. He is not spending time to
memorize a script and spit it out like a parrot.
4 We'll that's true. So in the process, what your
mentioning what you really need very
experienced highly qualified teachers and I don't

Code
All teachers need to be following
same approach.

Better way to go supported
institutionally.

Alt. assessment needs very
experienced teachers.

Able to convince student learning is
happening.

Alt. assessment needs very
experienced teachers.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

140

mean qualified by having degrees, by actually
knowing what they are doing. (yeah) So I was
really really happy at [OPE] when I was
teaching because that is the way I think people
actually learn
Summary. All teachers need to be following same approach. You are able to convince
student learning is happening and therefore get student buy-in. institutionally supported Alt.
assessment enable better CLT implementation. However, Alternative assessment needs truly
qualified teachers.

Table 9.
Theme Category: Instructional Desires
Page Line
6

Textual Data

38 If management would actually understand. We
need somebody who knows language teaching
at the top. (yeah) You know that actually

Code
Management needs to be
knowledgeable in CLT.

Summary. “If management would actually understand. We need somebody who knows
language teaching at the top. (yeah) You know that actually”.
Table 10.
Theme Category: Overall washback effects
Page Line
1

2

Textual Data

Code

38 So now we changed we dropped the IELTS and Changing of Curriculum.
now we have a new state designed test called the
[name of test] (oh really). So but it is basically
like a placement test. It's just like an IELTS
there is no interview though
14 No. We worry ourselves because we know we
Triangle of competing pressure.
are not doing it. (but it doesn't seem to go
above) exactly. You kind of fall in that triangle.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

3

3

6

6

2

There's the institution that has some objectives
and you have you're own objectives, and the
student has his own or her own objectives. So
you are kind of pulled. you know the usual
thing. But then the final exam is [exam name]
so, you know, you have to prepared them, you
know, whether you like it or not. I mean that's
what they are going to see
8 Yeah, I miss that too because I didn't even in
fact prepare classes, I mean we, I just worked on
what I saw. Authentic needs of the students
(yeah) so I start interacting with them, I find
what their problems are and I develop
something based on that as we go along.
26 But anyway basically we are almost back to the
same old grammar. (yeah) we can't do grammar
translation because we don't speak Arabic but
(ha ha) but if we could that's what we would do
to keep employed, you know what I mean.
12 I think it basically, is that um, when someone
pushes you to take an exam. A student well you
have certain expectations. (yeah) you expect the
teachers to meet those expectations so obviously
teachers modify their way of teaching, their
approach to teaching they they betray their own
way of understanding of how they should learn
to simply meet both the management and the
student. (yeah, the pressure is enourmus) We are
assessed by the students and by the
management, see what I mean. So we end up
modifying our own way of teaching to to meet
the needs. I mean it's a socio-economic triangle
27 I know what I need to do as a teacher for you to
learn. So how can I meet your expectations
given my whole knowledge and at the same
time I'm keeping management happy and
eventually they will keep paying me to keep
doing my work. (ha ha) so it's a tricky, it's a
tricky little thing. (it is) it's complicated (yeah).
Most of us just end up betraying ourselves
basically.
1 So as the teachers as such we are all. Basically,
teaching for the test. So what we have done is,

Deeper teacher satisfaction with alt.
assessment system.

Reverting back to behavioural
grammar teaching.

Socio-economic triangle.

Quote of the day.

14 overall effects - 6 Assessment
Alignment – (double coded)

141

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
we go around looking at the students taking the
test and we try to remember the kinds of
questions we can see

Teaching to the test

Summary. Over all effects of Standardized assessment structure: Constant
rearranging/changing of standardized assessment structure born from the Socio-economic
triangle of competing pressure (institution, instructor, and learner).
A student well you have certain expectations, you expect the teachers to meet those
expectations, so obviously teachers modify their way of teaching, their approach to
teaching they they betray their own way of understanding of how they should learn to
simply meet both the management and the student.
I know what I need to do as a teacher for you to learn. So how can I meet your
expectations given my whole knowledge and at the same time I'm keeping management
happy and eventually they will keep paying me to keep doing my work. So it's a tricky,
it's a tricky little thing. It's complicated. Most of us just end up betraying ourselves
basically.
Overall effects of Alternative assessment structure: A deeper teacher satisfaction with alt.
assessment system.

142

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

143

Appendix J: Codebook - David
Themes Relating to the Context of the Instructional Experience
Tables 1 through 3 depict the context of David’s instructional experience.
Table 1.
Theme Category: Instructional Background
Page Line
1

8

Textual Data

I work for the foundation program (yep) but within the
foundation program we have three sub-programs. We
have the academic writing program, I'm a part of the,
actually, I am the coordinator or facilitator for that
program, for the 2 writing courses (em hem) this
program serves all the colleges here which use English
as their medium of instruction. We prepare them for
research writing, presentations (em hem), just like an
academic program in North America. Academic writing
for undergraduates (yep). So this is what I teach, I teach
the advanced level, Writing 2

Code
Instructor and coordinator of
academic writing gatekeeping
program.

Summary. Instructor and coordinator of academic writing gatekeeping program.
Table 2.
Theme Category: Learning Context
Page Line
1

2

Textual Data

22 because I am academic writing 2, if they pass their
writing 1 they can take some classes in the college, they
are allowed to do that, (yep). Then they are waiting to
pass our course to take other classes. So there are some
prerequisites.
25 No, no writing also, so for example, you know, last
time we did a summary test where the students the
students were supposed to read an academic article and
then summarize. (yep) so then after the test, we all
looked at the grades and problems and issues we had to
review the rubric a little bit more (em hem) but the test
itself was ok, then the other thing, we had to add like an
extra week of teaching them summary, preparing them

Code
Partial entry to undergraduate
program.

Changing curriculum to add
more time to teach academic
skills.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

144

how to write summary, (ok, ok) so basically the
curriculum effective than the test itself
Summary. Partial entry to undergraduate program allowed. Must pass writing course to
gain full entry. Changing curriculum to add more time to teach academic skills.
Table 3.
Theme Category: Assessment Structure
Page Line
1

1

1

2

Textual Data

16 We have summary writing, response writing, and then
at the end of the semester they submit a term paper (em
hem) a mini research paper (em hem) and, with a
presentation
36 Um, well we have a standardized testing, so we run our
test our stream, and well, all our tests are writing based
and the rubrics are not holistic rubric
39 Yes, so we have content, so we only look at content,
then we have organization, we look at their paragraph
structure (em hem) then we have the AP or research
and documentation element where we look at their
citation, referencing, paraphrasing, integration, (yeah,
but not direct, you don't assess grammar or vocabulary
directly) Oh yeah, we have one element which is
grammar, that is what I was going to say, grammar and
vocabulary (sorry) where we look at their mechanics.
1 So there are mid-terms as well and there are some in
class tests, quizzes and in our course in my course
specifically we don't have a final exam because of the
research paper but in writing one they have a, they have
two in-class tests which are equal to mid-term exam
and a final exam.

Code
Task based.

Standardized testing with not
holistic rubrics.
Academic portions separated
out in rubric alongside
mechanics (grammar /
vocab).

5 Assessment Structure –
mid-terms and in-class
quizzes as part of summative
grade.

Summary. Standardized testing of writing tasks with non-holistic rubrics. Academic
portions are separated from mechanics in rubric. Mid-terms and in-class quizzes as part of
summative grade.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

145

Themes Relating to the Experience of Washback
Tables 4 through 10 depict David’s experience of washback.
Table 4.
Theme Category: Experience of Assessment Alignment
Page Line
2

4

Textual Data

35 Ah, well we assign them a lot of practice and then we
show them how language is assessed, how that element
is assessed (Ok) and then practice the thing, we need a
lot of practice
13 Yes, even the articles we select for our assessment we
try to address the major we are serving like sometime
we go to business major, medical sciences. So we try
our best. (yep) like bigger learning outcomes in terms
of their course

Code
Teach to the test.

Try to align to academic
needs and topics of
undergraduate courses.

Summary. Teaching to the test is part of the course structure. David tries to align to
themes to topics of undergraduate courses.
Table 5.
Theme Category: Experience of Indirect Student Washback
Page Line
4

3

Textual Data

21 Yeah, sometimes it does because you cannot control
that event at all but what we do here is we do a lot of
learning sessions or moderation (Yep) with articles and
then so like we take sample papers and then we grade
them and then we discuss together. So we try our best
to be on the same page but sometimes of course there
are issues with students (Of course) we introduce a
second grader or third grader. (Yeah, of course) an
extra marker, yeah) It's in control it's like on or two
every semester, which is good
1 Oh yes that is a possibility yes. (em hem) but they can
memorize again, they can memorize some phrases to

Code
Team work to overcome
subjectivity issues in rubrics
to which lowers student
complaints.

Memorizing phrases not a
concern

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

146

how to, that they could use possibility in summary
writing or linking phrases, not more than that, (ok so)
only article (Yeah) because they would summarize
Summary. Team work among instructors helps to overcome subjectivity issues in rubrics
which in turn lowers student complaints. Learners memorizing of phrases not a great concern.
Table 6.
Theme Category: Experience of Indirect Academic Course Instructor Washback
Page Line
3

4

Textual Data

46 In preparing the curriculum we did a need analysis, like
in the colleges and we met other stakeholders like some
of the organisations outside of [institute] and we
considered their feedback input and then we design our
curriculum. So we did talk to a lot of people outside of
our program
4 Yes they, as of right now, they are happy (em hem)
because we are giving them sort of the skills they need
to use when they go to the colleges like writing reports
they need all these skills they are learning now and
writing like a project or a short thesis they need all of
these things. So we are just aligning basically our
curriculum and assessment to the college needs

Code
Communication with
undergraduate teachers when
writing curriculum.

Academic course instructor
washback not an issue

Summary. Communication with undergraduate teachers when writing curriculum is
important step. “We are just aligning basically our curriculum and assessment to the college
needs”.
Table 7.
Theme Category: Experience of Institutional/Administration Washback
Page Line
3

Textual Data

15 Pressure from the management that there are logistic
pressure on us because they are more worried, our
management, top management is more worried, and
they are not from a linguistic language teaching
background that's the other thing, they are from

Code
Management wanting testing
on different days with
multiple test. Not
understanding difficulties in
test creation.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT

3

3

sciences (Yep) or business background, so the pressure
we face from this type of management, number one
right now, we are suffering actually, is that we have to
have multiple versions of our tests, because there are
classes which are on a Sunday and Tuesdays and there
are classes which are there are classes on Monday,
Wednesday, and Thursdays. (Sorry you have to have
monthly?) Multiple version of this (oh, multiple
versions, Yep) that is one problem for us because they
we have to go through a lot of things readability, to
many factors to make sure the, there is some reliability
there in the test. (Yep, and you feel that they don't,
because they are not from a language testing
background, they don't understand what) They don't
understand anything, we actually, it was very tough for
me because I am the person in charge for exams, so to
convince them to allow us to have a common mid-term
exam day. So that they are can go and write a two hour
exam, they didn't want that, logistic pressure is, is, a lot
on us
30 Pressure about proficiency or the level of the test itself,
not that much because they don't understand anything
there. (em hem) So they give us that type of comment.
But at the end of the day if there are too many passing
with A grades then we do have to write a report and we
do have to do a program evaluation, why there are too
many As and vice versa as well. If there are too many F
grades there
39 : I don't think so, (yeah) we had been lamenting about
that, (yeah) it is very difficult to cope with those things.

147

Surface level only as they
don’t understand anything.

Difficult to cope with lack of
understanding from admin.

Summary. Management wanting testing on different days with multiple test versions.
Management not understanding difficulties in test creation. Management has a surface level
understanding only, looking only at numbers passing and failing only as they don’t understand
anything about second language instruction. Difficult to cope with lack of understanding from
admin.
Table 9.
Theme Category: Experience of Alternative Assessment Washback

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Page Line
4

5

148

Textual Data

Code

40 We did have portfolio. But it didn't work well enough
in our context. Because of the culture I think (Yep) so
we had to give up on that and we had, we have some
online elements as well (yep) we have readings,
journals blogs, we do a lot of other alternatives as well
but the weighting is low because some people, the
people here don't support that type of alternatives at
that much. Especially that top management
4 It was easier actually to come up with a portfolio but
for most of the students and parents they were
complaining about it. (Yeah, not understanding how,
how it works)

Not in use as not culturally
appropriate.

Management, parents and
students all complained.

Summary. Institute experimented with introducing alternative assessment through
portfolios. It had to be abandoned (Alternative assessment continued for activities with very
small summative grading) as management, parents and students all complained. “the people here
don't support that type of alternatives at that much. Especially that top management”. The
cultural context didn’t allow for the successful implementation of alternative assessment.
Table 10.
Theme Category: Overall washback effects
Page Line
2

Textual Data

Code

15 because we just changed our curriculum like a year ago

14 Overall Effects - Recent
changes to curriculum

Summary. Recent changes to curriculum.

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix K: Interview Transcript: Jeffery November 02, 2017
Part 1
Steve:

Great! So you're at work obviously,

Jeffery: Yes I am, yeah.
Steve:

Ok, shall we get going.

Jeffery: Let’s go.
Steve:

Ok, hope I don't go over too much. So to start off, I thought that, umm, my title is quite
a mouthful. Contemporary Communicative Language Teaching. umm, so I and I know
it's not that well defined in the literature, what it is, so but I wanted, in in the study to
sort of separate, sort of, more experienced teachers from those ones that aren't. And I
found that in the research they are not very well separated. So that is sort of the angle I
wanted to go. So first if you could just sort of speak about how your instruction differs
from the traditional approaches.

Jeffery: Well, Communicative Language Teaching I think differs primarily from say Translation
Method, you know, like going way way back in the tradition, you know, like all kinds of
different pedagogical approaches which come and go with the seasons it seems. Umm,
Communicative Language Teaching like from me tries to get the student, no matter
what proficiency level that they are at to begin using the language in a real authentic
kind of context or way as soon as possible. Instead of trying to teach say perfect
pronunciation or to teach them grammatical structures as they used to do, you would
write endless mounts of drills perfecting your grammar, before you ever spoke a single
word in English. (Yeap) Right, so obviously in the modern classroom it's all about
authenticity, a lot about trying to, like for us because the program we work in is an

149

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Academic English program and not for the purpose of say settlement or Immigration.
Umm, where our focus is more narrow, to the context to the language, to the skills that
they need to be successful in a university environment. (Em hem) So we try to expose
students to as much of the language of those grammatical structures. The contextual,
socio-cultural components of language learning, umm, and try to get them just using it
as much as possible, like, I know the language classroom is always sort of inauthentic,
ha, right, (by it's nature, ha) it's a cocoon isolated from reality, (yeah) in some ways.
Um, so you try to do your best to provide meaningful tasks and get them using those
tasks in authentic ways. (Em hem) But, I don't know if I need to say too much more
than that.
Steve:

Well I think you have it covered. Probably, a key word that I hear you say is ‘tasks’, a
lot of it build around tasks and sort of I'm hearing you saying more communication less
on accuracy things like this.

Jeffery: Yeah, for example to give a concrete example: say pronunciation, uh, the goal in a
modern pronunciation class is not to rid a student of their native accent. Focuses on
intelligibility, can you communicate your message effectively? (yeap) Uhh, even if
there are grammatical mistakes, even if there are errors in, in segmentals, for example,
segmentals don't tend to be disastrous in terms of communication, uhh, super
segmentals likely result in more miss communication with native speakers so there is
things that you can specifically focus on to ensure that peoples message is being
conveyed, (yeah, on communication, as the prime…), Yeah.

150

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve:

Ok, so I'll move on. Thanks for that. Umm, so Ahh, Let’s talk more about assessment.
So what… what is the assessment strategy that is used in your department? Sort of, if
you could give me a, a very loose idea of what it is and how it is sort of changing.

Jeffery: Sure, Umm, so initially we, when students apply to the program, Umm, Everyone is
generally subject to the initial, Umm, placement battery test. So assessing reading
proficiency, writing and listening and speaking the whole gambit, Umm, So and then
based on those results, Umm, because our program is multi-level, from a low, say, low
intermediate to, Ahh, high intermediate/advanced level like, umm. So, If students pass
our highest level they are granted entry into university degree setting . So we are a
program that's kind of a gate keeping program. So based on the result from this test
battery we place them in one of five levels. And, Uhh, we are currently looking at
modifying our placement test so we're, this is sort of in development right now. But for
the last 8 years we have been using the same standardized test. (Um hem) It's not
delivered, umm, it's a pen and paper test so it's not an online test. Many universities in
Canada use an online test, so we may be a little unusual in that we are still using a pen
and paper test. (em hem). Uhh, Now in terms of, do you want me to speak about the
washback effect of that?
Steve:

yeah, or could you tell me about the exiting test? Umm, specifically the final exiting test
that that the gate keeping test is it a similar test? Like, how is it structured?

Jeffery: We actually, umm, we don't use, ahh, a final exit test. So based on them achieving a
minimum standard in their course work at the, are final exams in most of the skills, not
all of them, and so the combined total mark, if they achieve the minimum standard of
what were set level then they receive a pass. (Yeap) So in in they higher them at the

151

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
university they don't care about, umm, a letter grade or a percentage grade. They just
care about (a pass) P or F (yeap). So (If they gain the proficiency, yeah), so essentially,
yeah, there is no, there is no summative test at the end. It's just based on their
accumulative assessment throughout the term.
Steve:

And that is wholly teacher driven, and the teacher decides on the testing methods?

Jeffery: Umm, well there, we do have standard curriculum for some of these tests. Like the tests
are essentially, would be approved by committee.
Steve:

Ok, so yeah, that whole level, that final level would share that same test, which would
make up some of the grade.

Jeffery: That is correct, yes. So for example in writing, the writing instructors would get
together, they would decide on a specific, Uuh, test question, and then the students
would all write on the same question. Then (Yeap) same with listening, they would all
listen to the same lecture material and all complete the same standard test, (Yeap, yeap)
Steve:

Which is created in-house, basically.

Jeffery: Which is created in-house that is correct.
Steve:

Excellent, well that is good to know. Umm, so yeah if you could continue with the
washback of that.

Jeffery: So we, we find that especially the, the entrance placement test, and am sure that you
will find this in the literature and in many other places. That, with placement there is
quite a few negative washback effects that we find especially when students find
themselves placed in a level that is considerably lower than what they anticipated.
Umm, So this seems to result from, in, I don't know, a number of different factors.
Uhhh, students arrive with, Ahhh, maybe an inflated belief in (Em hem) their language

152

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
proficiency , (hum hem). And when they receive, so for example, a student arrives, they
write the test, they think ‘Oh I'm fairly advanced’, um, and they get placed very low,
like say, in our level 1 or 2 (Em hem). Umm, this often results in a backlash of sort of
externalizing blame. (yeah, yeap) There is environmental blame, umm, ‘the test room
was too cold, too hot, too dry, too loud’, ‘the audio wasn't, wasn't, good enough’, umm,
‘I was under slept’ or ‘I had jet lag’ (yeap). Umm, I, So there is all kinds of blame, but
it's all externalized, it's very rarely it’s self-reflective. (Em hem) .Umm, Not ‘what could
I have done to improve the outcome’, it's some other accommodation had to happen, um
. So what results often, in that students initially on the first week of the course
classwork, there is a lot of grumbling, complaining that this is not fair - ‘I should not be
here’, ‘I should be in a much higher level’. And we do subsequent, ahh, diagnostic
testing in the first 4 levels to ensure proper placement, so there is additional in-house
testing once the students are placed, to make sure that they are, where, where they
should be. (Em hem) Ahh, It is very very rare that students will move up based on
subsequent writing tests or reading comprehension tests.
Steve:

And do those subsequent tests carry their own washback as well?

Jeffery: They do, ha ha, so there is another washback. Some students they, they will accept it.
They will realize that within the couple of weeks of classes that they are placed
properly. (Em hem), Um, that they realize that ‘no my English proficiency is not enough
to be in higher levels, and certainly, ‘I'm not ready to, to deal with the share weight and
volume of, of course work of a university program’. (Em hem). So quite often, like our
instructors were, were able to do a good job of convincing students that you need to be
where you are and this in the long term will benefit you. (Em hem). But because many

153

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
of our students are very recent graduates of, like Chinese high schools, that is where
most of our students are from, Main land China (yeah), Umm, they take failure very
poorly. They see it as a commentary on their overall intelligence (Em hem), and so
taking this this failure that is almost catastrophic, Umm, that has undermined all of their
goals, (eh hem) that, the time line that they have. (Yeah, that they have internalized,
what they have told their parents, I imagine, ha) Yes, and, and this, this attitude or this
belief system about failing a test, or not doing as well as anticipated it serves them very
poorly with language learning, because the victories in language learning are so slow.
And it can be very hard to see progress, and they want to see fast progress, they want to
get to their business degree. There is so much parental pressure and financial pressure,
they have invested in a lot of resources coming across the ocean and settling in a new
strange small city in the middle of Canada. ‘I just want to get to it’, ‘I just want to get to
the real learning right’, and language learning is somehow secondary strangely, ha ha.
Steve:

My experience is in, in the Middle East. And I can completely relate to all of that ha.

Jeffery: Yeah, and we, we do have significant, we have fewer Middle Eastern students that we
used to but, ah, one second here. (ok) ok
Steve:

Umm, so, ahh, in, your let’s talk about the standardized portion of the proficiency test.
Umm, in your opinion, how, ahh, appropriate is it as, as a tool to, to… In your system
you have used it for part of the system, that, that you've used it for part of the
assessment, but how appropriate do you find it as a tool for the gate keeping, or the
passing on of the students?

Jeffery: Um I think it's generally useful and adequate, umm, you know, as with any, uhh,
standardized test there is always imperfections in the system. Right? Weather it is with

154

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
your assessors, umm, consistency (Yeah, yeah, Reliability, Validity all of this, yeah,
yeah). So, generally speaking, I think it’s been adequate to our needs. Um, though I
think that the inadequacies of the placement test have been properly dealt with mostly
by the subsequent internal diagnostic testing to ensure that proper placement has
happened. It does happen occasionally that you find that, for whatever reason, a student,
students like their placement mark doesn't really reflect what they demonstrate in their,
what they demonstrate in the classroom. That, so very occasionally you do get a student
who you wonder how they preformed so poorly on the placement test (Em hem) when
they are showing you something very different in your classroom. You’re wondering,
‘yeah this seems like they should be in our top level’. (Yeah) And it hasn't happened
very often, but usually there is these anomalies. Right? (yeap, so that helps to iron it
out). Yes it does, the internal diagnostics try to ensure that those inconsistencies are
ironed out like you said.
Steve:

And the, the combination of the standardized test with the alternative assessment
strategy, do you find that this, the combination works? Like if you could speak about
the alternative part or the in-house part that is not standardized. Does one balance the
other or do they fight together?

Jeffery: They are complementary, they balance each other yes. Uhh, this is one area, like in
terms of my position as a curriculum person I don't have as much, I'm less familiar with
the internal diagnostic tests (Em hem). Ahh, that's one area of the curriculum that I
have not been able to get to yet so I, in my teaching, um, I did not teach in the levels
that did the internal diagnostics. I typically taught in the highest level. So once you’re
placed in the highest level there is no need for internal diagnostics, you just assume that

155

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
the student is placed correctly, you just assume that there is no way to demote
somebody. (Yeah, ok) I can't speak with too much knowledge about the makeup of
those internal tests. I just know that instructors have created these standardized
assessments and that they are managed by some of the admin. (Ok)
Steve:

Umm, and if you could speak about stakeholder buy in. Like how, how, you have
spoken a little about this, about the students not believing their standard, so what about
them buying into the more modern way of teaching?

Jeffery: Oh, yes, Ahh, I think generally speaking, It depends a little bit on student back ground,
Umm. So, we do have a certain percentage of students that go to say a Canadian based
curriculum high school in China. And they tend to arrive here much better prepared,
acclimatized little, but more familiar with the social cultural components of learning
English in a Canadian context because they had Canadian trained teachers. (Yeap)
Part 2
Steve:

Um, so, I finished last night transcribing our interview, so you've given me a lot to think
about. um perhaps even in directions that I wasn't anticipating but it's good. (ok) I
wanted to...

Jeffery: Tell me if I'm going too far off course or umm…
Steve:

No, I actually think that, that’s a lot of the aim of the phenomenological study, is to sort
of, give you scope and see where it takes you because it's about your experience. (Ok)
And then I'm gonna line all that up with the others and see what correlates. But, umm, I
wanted to sort of bring it back now more to your teaching and how assessment effects
your teaching (Em hem). So, uum, so we, you, you spoke about stakeholder buy-in and

156

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
the importance you know as an overall issue, but how, in your view, do you find it
important? How, how important do you find stakeholder buy-in in your practice? Yeah.
Jeffery: Umm, Well to me, it's hard to generalize, but I'm gonna make one general statement, it,
that considering our student demographic, being predominantly mainland Chinese,
assessment is sort of an inherent motivator for these students, (Em hem) for good and
bad, Umm, so, in general, I find that no matter, in no matter what skill I'm teaching
assessment is kind of the stick and the carrot, simultaneously (Yeap). Umm, it's, it
motivates students to complete assignments, umm, it motivates them to do or complete
tasks that perhaps they otherwise wouldn't do on their own, ahh, so there is almost a
necessity in the program to have assessment sort of on an ongoing bases in order to get
students to do what you want them to do because a lot of the time they are not
intrinsically motivated to complete this, uhh, complete the task that is set for them. Ahh,
they require some kind of mark as a validation for their work , (Yeah). Umm, so that is
sort of like a general picture about assessment. Umm, I find that assessment has
different impacts on my teaching, different effects on the in-classroom environment,
perhaps based on the skill that is being taught or even the subskill (Yeap) within a
course. So, say, you take for example, maybe a contrast might help. So you have a
speaking course on one hand and perhaps writing on another (Yeap). I find that within a
conversational class where they are learning sort of social-cultural skills for carrying on
group discussions for agreeing/disagreeing maybe working on a bit of pronunciation,
maybe doing some, ahhh role plays for specific context that they will need in university
(Yeap) you often find that students are much more intrinsically motivated by that
context, by learning those, that particular skill set. They see how those tasks are directly

157

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
related to their future goals and to benefiting them in the context of University
culture/environment in the classroom and outside the classroom. So they really value
the feedback (Yeap). They can say ‘Ok, I can immediately apply this, I see how it can
be apart’, and they can more rapidly see improvements over time. So they are in the
speaking context, I find that, that, washback can be generally very positive while like, it
somewhat depends on the pedagogical approach of the teaching, like how do they
provide feedback, in terms of the assessment. So, I think, feedback is a really important
sort of component of assessment, right like, your assessing a student’s performance.
Now in the speaking environment, often the assessment comes quite quickly. There
might be some immediate feedback or assessing of a student’s pronunciation. So the
instructors tone, umm, your attitude and general environment that you've created in the
classroom, is it one that where students feel safe and secure in making mistakes (Yeap).
Now, if they do, then washback is much, much different and much more likely to result
in the learning outcomes that you want and the learning that your students are, are
desiring. Ok, I'm going to come back, so we have the speaking where, where I said
feedback often seems like assessment seems to have more positive washback effects.
Writing on the other hand, you are working on really kind on, like, oral assignments
with kind of very tight deadlines. (Yeah) and assessment here, in writing, I think is more
highly variable, umm, (More subjective I imagine) What's that? (more subjective the
way you sort of…) perhaps, like we, we have quite a few standardized rubrics that we
use when we are assessing our assignments (Yeap). Umm, but weather it's very
interesting to see how students absorb the feedback from your written comments from
the rubrics (em hem) and again, I think here, this has a lot to do with classroom

158

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
management, how instructors explain assignments, how they explain (Yeah) the use of
the rubrics, what the rubrics mean and what students need to do in order to fulfill the
expectations of the assignment and the sort of distinct criteria that's outlined on the
rubrics (yeah). Now if that is all been done well and students feel that, you know, the
channel of communication that they can ask questions they can say 'I don't understand,
what this means’, ‘I don't know how to complete that component of the assignment'.
The channel of communication are open and explanations are provided in a clear and
really supportive (yeap) way. Then washback and the potential for the benefits for
assessment are great. Um, so, in my experience I was always the kind of teacher that
had great high expectations of my students. I expect them to produce material and not
just be, you know, mailing it in, right. (yeap) And, I never had problem with that,
students would buy in to the fact that what they are learning in the class is valuable to
them in the future (yeap), and that, in fact, the writing assignments that they will face in
university are going to be far more complex and with multiple projects going on
simultaneously, right. So, to me, a lot to do with assessment in the classroom and the
results of assessment have a lot to do with the student teacher relationship, classroom
management and the learning environment that the instructor creates . (yeap, I
understand, I agree). Yeah.
Steve:

I would would do you find that your assessment there is a cause for the students to ask
for more traditional type of teaching? Like, they would, might ask for some grammar
instruction?

Jeffery: Ahh, yeah that can happen so for a good example would be in writing. You’re talking
about grammar (um hem), ah I have very strong opinions about this. Ahh, it's interesting

159

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
that students often they come to you after they receive say an initial assignment back
and the feedback and they come to you and they say ‘teacher can you give me, more
grammar exercises’, ‘Can you teach us, can you teach us, whether it’s this structure,
whether it's passive voice (yeap) whatever it is’. And I sort of, I look at them, ‘Well, ok,
but are you ready to do the work?’, ‘I can teach you, and I can lead you down this path,
but me just presenting the material to you doesn't mean any learning is going to
happen’, right (em hem), ‘you have to meet me half way and do the work’. And very
often I find students don't want to do the work. (Yeah, they are looking for shortcuts)
They want you to come to them with some grammar worksheets, and they will do the
grammar worksheets, perfect them. Then you never see any implementation in their
actual writing (yeah). So I think the…, to come back to your original point, it that, um,
feedback or assessment on very specific things can be beneficial, but the students have
to be willing to put in the work (yeah) and to be self-reflective of like their progress
(yeap yeap).
Steve:

So, it's a big, it's a learning curve for them and it's a challenge to, to have them towards
that system. So, I can imagine it's a group effort, that all teachers need to be on board in
some respect to move these students toward, where they should be in terms of thinking
about grading and thinking about how to learn.

Jeffery: Yeah, I think how to learn, is a huge question for many students, umm, it that they don't
have frameworks to apply to their own, uh, learning management. I think I mentioned
self-regulated learning in our, a little bit last week, when we talked (yeap). They are not
able to reflect on, umm, their strategies, so a lot of our students are excellent rote
memorizers. But when that comes to learning the vocabulary, we know that from the

160

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
research that rote memorization of vocabulary may be a good way a good place to start
but it doesn't result in depth of learning (yeah). Ahh, so, so I agree, yes teachers have to
be there and be able to present students with some of this information and maybe
construct their classes somewhat around somebodies ideas of metacognition, selfregulation, being able to analyze your strategies that you apply to your learning in
different context and be able to, sort of self-question. Arh, is it a strategy actually
working for me right now (yeah). ‘Do I need to switch to something else to be able to be
successful or to meet the goal that I have for myself’.
Steve:

And, and these are all things that, they come in not realizing that they are going to be
doing that. (Right) so um.

Jeffery: Many many of our students expect the very traditional student teacher relationship
(Yeap). You have like the sage on the stage delivering perils of wisdom that the students
collect, right, and that and that is very much the Eastern pedagogical model or (yeap) or
students don't have questions, they just wait for the information to be delivered to them
and they studiously write it down , (yeap). And, of course the Western education model
is far more Socratic in nature and, uhh, your, your statement or even a question that you
ask maybe answered with another question (em hem). And students find, can find that
very disconcerting (sure).
Steve:

Um, if I can rewind. Next, I wanted to ask about positive washback. But you answered
this, in that you were telling me how important, how, how standardized testing brings
amazing positive washback. Uh, (It can yes). So, with the alternative style of testing,
um, some schools are moving towards less standardized testing they are putting more

161

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
weight of alternative tests or even completely doing alternative testing, like with
portfolios and work like this. What, what do you feel about that?
Jeffery: What alternative? Umm, I think within a program such as ours which is a gatekeeping
program (yeap) there, one has to tread somewhat carefully with say the portfolio model
in in writing, because in our program we want to get, make sure it's possible, to be as
sure as possible that we are assessing language, that we are assessing language
components as opposed to merely jumping through hoops (em hem) merely say, ‘Ok I
able to hand in X number of pages therefore I get a grade for that’ (yeap), not even
assessing what is happening within the document, in terms on say vocabulary, uh,
grammar, uh, language structure of all kinds, um, like are they, is the form of the
writing recognisable within an institutional setting (em hem), as opposed to just like free
writing. Which has its place as a fluency based activity, but in terms of like assessing
we have to be very careful to balance between assessing language and assessing other
things that are extra linguistic. Um, the value of like a portfolio is that perhaps students
see what they are working on as being personal or self-directed and it acknowledges
principles of like you know adult learning. Where students have input into their own
learning until there is certainly benefit there in terms of motivation and maybe learning
and being able to work on some of those regulated learning characteristics that are very
desirable, um, so I think there is definitely a place for alternative forms of assessment
like that. It’s just it has to be, um, looked at carefully and make sure that it is in balance
with the learning objectives and our obligations to, um, the university over all in terms
of student preparation in terms of being sufficiently ready to enter degree study (yeap).

162

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve:

I think the key word I hearing you say is balance (Balance yeah) Yeah. Um, if I could
move on to something that I've had in my experience is with the standardized test I've
had, um, almost push back from the administration. Where I seeking support for
Communicative Language Teaching but what I'm hearing from the administrative staff
above me is ‘a focus on the test’. How many students passed, instead of how many
students have the proficiency and and so I'm feeling, I felt, I have felt this pressure this
washback from the administration. Have you had experience with this or do you agree
with this in any way? (Ahh) Being part of the administrative team yourself, ha ha.

Jeffery: Yeah exactly, Um, I think largely within our program, I think there is an uneasy an
uneasy, what is the word and uneasy dualism. Uneasy relationship between this notion
of the communicative outcomes the language outcomes (yeap) and students passing.
Um from the curriculum writers side. Primarily, like when we are developing materials.
We are very much looking at language based learning outcomes. What will the student
take away in terms of communicative language proficiencies, skills from these
assignments, um, because ultimately, because we don't have a dedicated exit
standardized test. Where we have sort of, we have final exams to each of the discrete
courses and our highest level that contribute to the overall mark, um, student just
passing, yeah they make it through the program. It's assumed that they have absorbed
and meet the learning objectives to a certain degree. So from the point of view of the
language program, we hope that they have learned enough, improved enough to be able
to kind of survive at least within the university context . Now some other university
administration (yeap) its, to be quite frank, there is the pressure just to get students into
the university because it's dollars and cents (of course) international students are

163

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
valuable, right, and so you don't want there to be too many barriers to admission (em
hem) um because right, not right with super competitive market in Canada and North
America and the dire financial situation that many big institutions. International students
are seen as a way to maintain the bottom line (yeap). So, that's one very uneasy
relationship between higher university administrations, administrative like desires and
the desires and hopeful outcomes of a language program imbedded within that
institution.
Steve:

I've actually. In presenting this thesis to my supervisor. I had to defend against that
because she was speaking to me about oh it's about dollars and cents, you know. Like
‘No this is about academic, um, this is about how to best provide a language to a
proficiency level language to students, I'm not concerned with the dollars and cents part,
that another department, sort of’(exactly). So, I always need to push that.

Jeffery: There is a really interesting book that you might be, maybe your familiar with it. Arrh,
It's called ‘ESL readers and writers and higher education’ (oh ok, whose that by) It's an
edited book by Norman Evans, Neil Anderson, and William Eggington it's a Routledge
publication from 2015 (Ok), but they discuss a lot of this very topic, um. They very kind
of dynamic between the large institution and the interests of the English language
learner.
Steve:

Oh, I will definitely look that up. Thanks. Um, if we move along to, ahh. Are there any,
um, suggestions or ways that you think that communicative language teaching can be
supported or better supported by the institution?

Jeffery: Like the larger institution. Ahh. That is a complex question. Ahh, to me there is an
assumption. Where there is just a lot of misunderstanding or preconceived notions of

164

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
what language learning does. A lot of misunderstanding of what can be accomplished
in, ahh, in four months in a language program. So, I have a lot of conversations with or
I'm in discussion groups with university professors and they are mystified by how
students who have you know come through our program or come in to the university
with a specific IELTS mark and that they see, all they see is like deficiencies, and they
are mystified as to how these students arrive here (Yeah). And I find that that this sort of
a misunderstanding about how students acquire academic proficiency. That it results in
a lot of finger pointing and blaming and results in a lot of spinning dialogue about how
do we support these students. ‘Shouldn't they be able to do this already when they arrive
in first year business or second year business?’, ‘Shouldn't they be able to do XYZ and
be perfectly, you know, capable of making a presentation to the class?’ So, in terms of
like providing support to communicative teaching I actually think it needs to extend far
beyond the walls of the discrete English language program (yeap). I think that within
university administration there needs to be a recognition of the fact that learning
academic English, whether that be writing, reading, speaking vocabulary, listening to
lectures and what have you takes a long time and these students require supports all
along the way. Umm, and it can't just be in the form of like an academic writing center
where students go and see a tutor. There has to be social-cultural components maybe
even like mentors that help students begin to like navigate the terrain of a Canadian
University. Umm, so, in terms of communicative language teaching, yeah. The supports
need to be robust. Like I can speak to some very practical, I think all across Canada like
English language programs such as ours they are housed in very different units within
the universities (yeap), and so they have a very different political arrangement, different

165

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
levels of formal acknowledgement, uhh, within the institution, uhh, so there is a lot of
complexity in terms of where that English language institute is housed in the umbrella
of the University and how it is able to communicate with stakeholders throughout the
university. To actually understand what, what happens in an English language centre .
So. (Yeah I agree with all of that. Yeah.) So, I hope I answered your question there.
Um, in terms of like, it's a very wide ranging question and, umm, there is no easy
answer. (no) There is a lot of competing ideologies there is competing like politics and
there is economics all wrapped up in the question. (Yeap, and the teachers at the bottom
doing their best) The teachers in the trenches, Yes exactly.
Steve:

Um, Just one final question, is just to sort of throw it to you, if there is any other points
or areas about washback that you feel we have missed. (Umm) or further insights that
you could offer.

Jeffery: Well there is one thing that I did think about, it terms of like our our earlier discussion
about umm the placement test umm there's this kind of interesting phenomena that
sometimes happens especially if students get placed in one of the lowest levels. Umm,
there can be, I mentioned about how washback can be, can result in this sort of,
existential crises with in the student (yeap), where there's, they are very bitter. So, some
students will take this to the extreme, where they will, just a very small number, but it
does happen where they actually stop coming class altogether in order to study for say
the IELTS exam (em hem) they will withdraw from the program and attempt, they say
that are going to study and then they go write the IELTS in hope of being able to jump
levels. Very rarely works, though students persist in doing this all this has pretty
negative consequences. Obviously there no coming to class, so they are short of

166

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
defaulting of four months of valuable learning experience to go study for an (I agree)
exam . So, I don't know they seem, it's just a fascinating phenomenon to me. That, I
think, crystalizes this deep misunderstanding within so many of our students, of the
many layers of, many valuable layers that exist within coming into a language program
such as ours (yeap yeap).
Steve:

I always feel that too, especially with academic language it's just, ‘Sure you can
communicate you can buy a coffee but ‘can you do something academic?’, it's a whole
other level.

Jeffery: It's a whole other level yeah, yeah exactly. So, umm, so looking into my notes here if
there is anything else that I can think of. Yeah, so maybe I'll just finish, my last
comment would just be. Like I see formative assessment like within the program. Like
within the discrete skill like listening, speaking, reading, writing. Formative assessments
I think students tend to value as long as there is clear communication between the
instructor and the student about how these learning tasks are applicable, (yes) how they
will benefit them (Yeap). So, there is like a tangible, umm, message in something that
students can understand and project into the future as opposed to oh right I just have to
jump through this hoop (Yeap). I don't really understand why I have to jump through
this hoop. But I'll jump through it because it's worth 30% of my mark (yeah).
Steve:

So what you're saying, you’re using formative assessment to pull into the wider goal of
improving (exactly) Yeah.

Jeffery: So, there has to be really good communication between the instructor and the student.
But also between the program and the students coming in. So that students are enrolling
in your program they should have a very clear understanding of what they are going to

167

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
learn, why they are going to learn, and how this learning is going to help them meet and
fulfill their future learning goals (em hem, ok). Then washback in assessment can be
maximized (yeap) for benefit. (Yeap, yeap, excellent, nice way to finish.)
Steve:

Umm. Ok I think that wraps it up. I have to thank you for all the information you've
given me. (My pleasure). One other thing. Would you like to choose a pseudonym that I
could use for your name?

Jeffery: H Ha, My pseudonym, how creative can I get. arhh. How about Jeffery.
Steve:

Jeffery.. Excellent No thank you for your time.

168

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix L: Interview Transcript: Gary November 23, 2017
Steve: Ok, so I know my title is a bit of a mouthful. Umm, ‘Washback on contemporary
communicative second language teaching’. So umm, but what I wanted to do is sort of
separate it from more traditional teaching. Because I found on the research, there is
actually not a ton of research on washback like I expected, so umm. And it wasn't defined
especially how communicative teaching is evolving quite quickly and it's not really, I
didn't find it was catching up. So I felt that this was a good avenue to go. Umm, but first
sort of to begin and to give a bit of background I wondered if you could sort of speak
about how your instruction differs from the traditional instruction?
Gary: Me personally, I, I've always been, interested in, I pull from a lot of areas like whether it
for my research or my instruction or whatever. Umm, I've never stuck to one particular
kind of approach or methodology in class . Umm, I like to look for ways to innovate or
build on what exists or combine just depending on what I'm trying to teach. Umm, and I,
and that has always imposed my material development as well, because, umm, I've
always found that text books don't always, they're not always designed with a process, a
teaching process in mind, or necessarily a good teaching practice in mind. They're done
in a way that's, I don't know, sometimes they align with older research, or older ideas
about how things should be taught, verses what people are actually doing, so I pull from,
for example, drama pedagogy as in not how to teach drama, but how to use drama to
teach for example. Or inquiry based learning or other approaches that are used in
different, in different ways, even like sort of that whole cognitive idea about like, umm,
for example, when we teach like, moving from a concrete concept to for example to an
abstract one, when it comes to language or prepositions of time and place for example. So

169

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
I tend to pull in from a lot of different approaches to learning , umm that are proven
sound across many different areas, fields and then pull them into language instruction,
and see how we can make them work in that particular area. Umm, that where I, I don't
know, it's I'm a bit of a mishmash of, (I agree) in terms of learning design.
Steve: I, I'm sort of a, like like, I think what I hear you saying is is you don't you're not rigid on
one methodology that sort of the flavor of the month or the year (no). You sort of
Gary: And I like to experiment, too like if I see something, or I'm like, oh inspired by
something that I've read or I've seen . I want to see how that might go down in my class
(em hem) and you know if, and it's never a onetime thing. It might not work the first time
or I my mind it may not go, I might not be meeting whatever outcomes I've got in mind
for that thing but I'll try it again later with a different way, or a different group or a
different, maybe not with grammar this time maybe I'll try it with vocabulary or, you
know, tweak it along that way (em hem) um to see how it, how it goes. I sometimes get
feedback from the learners on, if we are trying something weird or new (em hem) to see
how they felt about an approach or a particular teaching methodology that I've brought in
and that also helps umm not only with the teaching itself but eventually with materials
design as well they are my secret guinea pigs I guess (ha ha). Yeah, yeah, I try to elicit
feedback from students who actually understand that I'm not a dancing monkey in front
of the class doing random things. Usually there is a, a plan or an end game in mind and
(em hem) you I know.
Steve: But you, It can seem like like when you start doing um some things.
Gary: 'Teacher you plan things? it's not just you walk in and say things in English and hope that
it all works out?' like no we actually, methods to the madness and stuff (ha yeap).

170

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: So, if we could sort of move on to the, umm, the strategy the assessment strategy that
that's being used. If you could just loosely tell me about it. Umm. at at your place of
employment.
Gary: Umm, because we're a, we are COB based we work with the Canadian Language
Benchmarks which suggest that any of our, um assessments should also be, therefore be
tasked based (yeap). Umm and they have to aligned to the Canadian Language
Benchmarks which describe what a learner can do, umm, with their English. And there is
different streams and different strands. On top of that though, we have obligations to, sort
of academic obligations . If we want programs within the college to accept our, umm,
language assessment scores and entry, you know, proven ELP essentially for nursing or
whatever. We also have to have a fairly academic focus, which speaks to some of the
performance expectations let’s say in nursing or pharmacy, or those academic type
courses. So our assessments are largely based around CLB outcomes. They are, our midterms and our finals are are standardized. We offer the same midterms and final across
levels. Umm, I've been the primary designer or those assessments and for the rubrics that
accompany them. But then our quizzes and then our projects that are offered within a
level are entirely teacher based. So they could focus on a particular skill. Let's say, umm
you know, skimming and scanning 'I want you to demonstrate that you can skim and
scan', or a particular, like form. like 'I'm gonna test you on this particular grammar’, they
don't have perhaps. So they are more teacher, umm, they go with whatever the teacher is
doing in the moment or whatever skill that they might be working on. Umm, eventually,
hopefully leading up to, a learner’s ability to complete the task that is in the final
assessment. So the mid-term assessments.

171

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: Ok, and are those skills that teacher gives. Are they part of the final grade?
Gary: Yeah, the rubric for sure and the quizzes themselves are calculated. Our mid-term and our
final. Umm, I think they make up about 50% something percent of the grade. Um,
quizzes make up I think 15-20% off the top of my head and there is a project as well
which I think is 15-20 somehow mathematically it all comes together (yeah). But yeah
the quizzes are counted. They are not weighted they are not worth as much as the bigger
pieces that are standardized but they are, (yeap) part of that final grade.
Steve: Yeah, so I think you’re saying that the quizzes, the quizzes are more academic skills
based. They are looking at the academic skills. And then sort of the final standardized test
or including the mid-term as well it's more proficiency, language proficiency based.
Gary: Yeap, so umm one of the CLB standards for example social interaction umm, CLB 5 they
have students demonstrate that they can write an email. Pragmatically appropriate, umm,
email to, umm, either a friend, employer or a colleague or something like this. And that
would be task (yeap). The rubric that assesses whether or not they were able to complete
the task, whether or not they were actually able to write the email. But there are also
elements on the rubric that look at pragmatics, the use of, umm, correct expressions,
umm, there is a grammar piece for level appropriate, umm, grammar expectations . So for
a CLB 5 they should be able to use simple tenses correctly, for example (yeap) umm ahh,
and then things like punctuation and formatting so the rubric breaks down um, elements
of the task (yeap), umm, holistically is the task completed appropriately and that it breaks
down academically by grammar, by vocabulary and by formatting and so that would all
then comprise then comprise the score. 75% congratulations or whatever it is, on that
particular task.

172

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: and and for the, umm, the listening skills, and the reading skills?
Gary: Same idea, (same idea, yeap) yeah you would have a listening task, an audio, umm, for
example that fits the strand of the CLB so. Presentation information that's a really easy
one you have find somebody giving a presentation or, you know, having a meeting or
something like this and you, we develop, umm, questions that address the CLB outcomes,
the learner can identify facts and details, main idea, umm, connectors, sequential
connectors for example within or organizational connectors within the audio itself, you
know, therefore, consequently or whatever, or able to infer or predict content. That would
be a CLB 6 for example. Ahh, so the questions would be designed to be to assess whether
the learner can do, umm, all of those, meet all of those outcomes (yeap yeap, that sounds
good).
Steve: So, these tests, they are created in-house?
Gary: In house yeah, cause they, umm, CLB tasks, umm, that are designed by, umm, Canadian
Centre for Language Benchmarks (em hem) are holistic they don't, um, evaluate the
grammar or the pragmatics or the vocabulary. They just look at them holistically which,
so we couldn't, when it came to providing an ELP like an English Language Proficiency
level for, ahh, in-house or even external mattering, it wasn't sufficient because we needed
to be able to say academically speaking this persons got the grammatical, vocabulary
skills, whatever it is to be able to go into your program and succeed. So we develop
everything in-house (Yeap yeah).
Steve: Ok, so, umm. So it's mainly standardized testing, not a lot of the alternative testing. There
is no portfolios or things like this?

173

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Gary: We used to, we did. The portfolios were a hot mess. Mostly because our team, the
majority of our team, umm is not quali, they don't make good tasks, they don't know how
to create a performance either like it's. So the portfolios failed us and we ended up
teaching to the test (em hem). Or if the students didn't do well in a test they would bully
teachers into making a new portfolio task that is simpler or to just repeat the task itself
until they finally got it right (yeap) um
Steve: I do like that word you've come out with, Bullying, I've I've felt that myself and that it
sort of outlines the pressure that you can have from students.
Gary: Our counter parts, because we have the ESL intensive program and we have a LINC
program as well. Our LINC program is doing portfolio testing right now and it's...my
husband is working in it right now, it's it's such a mess the students are absolute bullies.
They do nothing but test over at LINC it seems, and it's just the, you know, if a student
doesn't perform well on that particular task they just put the heat on the teacher to to keep
re-testing and my husband honestly feels that the majority of his job is just. He lives, his
class just lives for the test and so, umm, we got out of that, the ESL testing side because it
wasn't working for us (umm hem, yeap, you've got the option to get our at least. I hear
that at LINC it's across country.) Yeap, and there we were, thank God my colleagues, you
know, have the humility to say to say like I'm not qualified to make an assessment task
that is weighted such that it, you know, and it shouldn't, you know, not without
development or support or whatever. It wasn't consistent, you know, there was, no, you
know, there was no way of checking . So, it's one of the reasons why the assessment
development now has fallen on to just a couple of us within the ESL department, ahh,

174

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
intensive department, because not everybody else is, is confident and capable to do those
final assessment pieces .
Steve: Ok, so um, with the testing that you have the standardized testing how aligned do you
find it with the outcomes?
Gary: Strongly, powerfully, really really good. Um, it took us. I had to have a bit of a stern
talking to. I went on mat leave after developing our first round of exams and when I came
back, umm, they had the tests, the teachers had gone in and altered exams . And they
were going ‘It's really not working’ and I'm like ‘of course it's not working you changed
the test ok, it no longer aligns to the outcomes’. ‘You're testing things that are not
actually outcomes’. Umm, so in terms of what we are supposed to be testing they're
they're mapped. We actually have maps for each exam that show which test items align to
specifically which outcome (yeap yeap).
Steve: I always find outcomes can be kind of difficult because it's kind of like. ‘Can write a
sentence’, like you know, ‘have very good communication’. or something like this. it's
quite.
Gary: The CLBs aren’t that vague. The CLBs are really specific so that's, and we are lucky in
that regard that the CLBs are written in such a way that, you know, ‘Can write a
paragraph’ that is not what is. It's really great itemized ‘can do’ statements and so it's (em
hem) helpful for us as as designers or developers to be able to have that level of clarity,
so (yeap yeap)
Steve: Um, well you've answered that one (sorry).
Gary: You now it's funny, we actually had to map grammar outcomes to the CLBs . The CLBs
hint at particular grammar forms or vocabulary or, you know, phrases and things that

175

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
learners should be able to use. But they don't specifically, very explicitly provide a list of
ability. I actually had, we ourselves internally actually had to create grammar and, umm,
pragmatics maps to align with CLB tasks and CLB outcomes. (Yeap, Ok) which has
helped as well on the rubrics.
Steve: So, you mean like points of grammar that they might pass?
Gary: Well for example, if you look at the CLB four they are supposed to be able to give for
example sequential instructions, (yeah) you know, in order. So when it comes to grammar
what you need to be able to do in order to utter an instruction (Ok) you need to be able to
have imperatives for example right, you need to have certain sequential time markers,
first, second, third, next, after that, which also implies that you need to be able to do time
causes in a compound or complex sentence and so we actually had to go through the
outcomes and look at which grammar forms were implied or almost explicitly, you know,
within a each, umm, each task or within each outcome within the CLBs. So we actually
have a grammar map now that, that, internally mind you, that accompanies the umm
outcomes that we have. Which has also helped with our rubric development, and is also
helping, you know, I'm not just teaching this one thing over here it has to happen, um um,
at this particular CLB level and they, you can build and scaffold on it. Or spiral it even
throughout the different levels so.
Steve: Do you find that that has an effect of of encouraging teachers to go back to more like start
teaching grammar. And focusing their classes around a grammar point.
Gary: Um it has, what it's done is actually um yeah not exactly, luckily no. Like grammar is
included as part of a lesson perhaps as a pre-task or a post-task analysis for example if if
a teacher is doing an actual task cycle (yeap) the grammar might appear in a pre-task or a

176

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
post-task analysis but it also helps the, umm, it's helped us anyways target or identify or
agree upon which grammar forms should be offered at certain levels and must be, and
then there is the nice to be, so then if I'm teaching a CLB 6 for example, I have the
understanding that my colleague has covered something already at CLB 5 or CLB 4 and
so a learner wont progress up into a 6 unless they have demonstrated a certain degree of
competency with that form or there is an expectation that they have covered it, so that I
can build, so if they are good with, ahh, simple present simple past whatever I can maybe
work on to perfect tenses or, ahh, these other things (yeah) so it's actually helped us
organise a little more though it doesn't necessarily make us grammar um dependence.
But, we also sometimes provide lists of the forms to the students at the beginning of the
semester saying, ‘you're responsible’, you know, for these forms. We have on-line
grammar checks through Pearson now, through our Moodle, whatever. So they can
actually access the chapters themselves if they want to do self-study as well (ok). So
when the assessment actually comes they see on the rubric like, umm, must demonstrate
mastery or control or developing control of this particular form , which is already
identified right from the very beginning. The learner knows that that's one of the
expected, umm, things that they have to be able to to show by the end of semester right,
so,(yeap)
Steve: So if we could sort of move on the teaching to the test. That's sort of what you're, umm,
I'm hearing it's not too much of a problem, the way you have it set up.
Gary: The students want it and this is, the issue is that the teachers prefer not to. Umm, I can
say collectively we want to be able to teach how to communicate well in English. What
the students want and ask regularly is ‘Is what you're teaching now teacher on the test’,

177

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
‘no’, ‘then I don't want to learn this’ (yeap). And this is where we get arguments or, um,
students don't buy-in, they fight, they check out mentally, mentally check out whatever it
is. If it's not on the test many of them have no interest in learning it . Umm, and if we as a
group I can say with a great deal of confidence that we don't want to teach that way . Um,
the exams are there to provide a snapshot of where that learner is at that point in time in
terms of proficiency but our overall goal and mandate is to provide language instruction,
um, and intercultural and all of this so that learners can use language in the workplace or
academically or, you know, whatever the next step will be. And ‘that is not to teach you
what is on the mid-term, it is not to teach you specifically what is on, a final exam, that is
not what it is’.
Steve: So to bring the students to the wider goal of.., yeah.
Gary: Oh they and we could talk ourselves blue in the face they they don't have the same goal
as we do when it comes to the classroom and so it's a battle . Like ‘teacher what’s on the
exam’, (yeap) it's like ‘I don't care what's on the exam’, right.
Steve: Do you find that they have an inflated, ahh, value of of when they evaluate themselves,
they actually feel that they are much better than what they are doing in the test?

Gary: Absolutely yes, they, yeah and that's also, umm, people, oh, I'm having this fight with a
lot of learners this year who are not um achieving what they would like to achieve
because they think they are awesome and I like ‘Well you're not demonstrating to me
that you can do these things’, you know, and that's not I have to. ‘You know at the end of
the day these are the outcomes that we have, that you have to show me you have to
demonstrate to me that you can do’, 'well I can do that' 'well then why why aren't you

178

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
doing' right (em hem). It's a lot of them thinking that they are better than they are or they
they negotiate for marks on the test, cause it's a negotiation, 'I give you, I give you this
mark' rather than, um, (you earn a mark, yeah) you earning it or whatever it is right
(yeah) so it's a constant fight, I mean I've had colleagues go on stress leave because of the
abuse of the students in this regard. The relentless bullying and hostility and comments
it's never ending. The the lines that are crossed are (yeap), you know, just mind blowing
sometimes.
Steve: And I think that that highlights just how high stakes things are for these kids for the
young adults.
Gary: And a lot of ours are funded, they have limited government funding (uh huh) to take their
ESL and so for them they think ‘if I go to the next level that means I am that level’, I'm
like 'no it doesn't, like you could be a CLB 1 in a CLB 8 class it doesn't make you a CLB
8 it doesn't mean your better just because you’re in that class. And we are not going to
put you there, because we are not going to give like a… specially like our credibility is
certainly on the line it took us a lot of time for our external, internally and externally for
people to be able to accept our CLB scores over IELTS scores, that's a fight (yeap). So
um, It is intense over here so I guess that students just are not on the same page, umm,
don't care to be on the same page and it's, um, really stressful (yeap yeap I understand).
Steve: Umm, what about from not so much sort of indirect washback from above from
administration, even though I guess you’re part of that, even above, like, um, from the
university structure and from sort a lack of understanding of where ESL or academic
English fits in to a university structure. How does that effect, come all the way back to
the classroom?

179

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Gary: Our program to be honest with you is serving too many masters. Which is really I think,
umm, made it difficult for us to actually do our jobs well. Ah, part of the pressure from
the, whether it's the administration or whatever, I, we what to have laddering, internal
laddering, we want to have some and so does the institution. It's part of the providing a
quality learner experience is that somebody should be able to come into our ESL program
and they go smoothly efficiency into a nursing program, let's say. You know, that's the
way it should be they shouldn't have to have an external test, theoretically to be able to go
into this thing. So our administrators our leadership want this. Our international students
come in, they don't want it to be these other things that have to happen, it should happen
in-house but, umm, they don't have an understanding of what good teaching practice is
they don't know what our tests should or should not look like. Um, they don't know what
good assessment is in ESL, um, the fact that we are doing CLB. The CLBs are never,
they are not supposed to be used for high-stakes entry, like that is not what they are
designed for, it is explicit, that is not what they're known for. Um, task, like the stuff that
we are doing. We have these community workplace schools and now we also have this
academic goal that we are supposed to be academically prepared to go into nursing or
pharmacy or whatever and it's really watered down what we are able to do in the amount
of time that we are given, um, to do it in. And, um, we are not doing anything well, we
are doing most things moderately effectively but, um, and our text books don't align, ha
ha. So, um (even though the text books have written on them nicely you know
communicative approach, you know this is what we need and the advertising is amazing).
The power of PPP (ha ha). So we are not doing anything, um, to the best of our ability
really, and the students feel the heat because they have, um, these governmental funding

180

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
deadlines, timeline, you know, where it is going to expire. Ah, we have pressure from
within to be able to provide effective language instruction for them to ladder in . We have
literacy learners, we have refugees who have never held a pencil before in their lives,
who were not ever going, you know, so yeah we are serving too many masters. Including
our own institution who have little understanding and often times little respect for what it
means to be an ESL professional and what it means to be a good ESL program. We are
not good at what we do because we are combining literacy instruction, workplace ESP,
you know, English for Specific Purposes and academic, English for Academic Purposes
all within the same classroom and it doesn't work.
Steve: Yeah, too many, too many stakeholders with their own agenda.
Gary: I think they need to go away, ha ha.
Steve: That is what I've always said in my classes (Give us more money and leave us alone)
Yeah, tell me what you want and go away, ha ha.
Gary: It's hard to be accountable to them. I used to be the associate chair for my program and I
sat in meetings where we were just getting screamed at (em hem), you know, and it's just
ahhh.
Steve: Do you find that also from the instructors of the university program saying your students
you can bring to us are not prepared.
Gary: Oh my God yes, yes, there is so little respect from them for what we do. And as language
instructors even within the field we are not even considered professionals like they are .
Because, you know, we are teaching language which is apparently hippy dippy. And I'm
like ‘we all have a master’s degree thank you for coming’. But, you know, it's even
professionally we are not considered on an equal level our program isn't taken seriously

181

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
because we are foundational and our learners language proficiency is always called into
question. Um, you know we are not act.. we are not strictly an academic program for
goodness sakes , ‘you know, we have all of these other influences going into our
programming our curriculum development and our assessment so no we are not making
you happy either’, they scream at us, they scream. Those meetings are really fun (sure).
Steve: Do you think the, basically your saying they don't have an understanding of just how
advanced academic language is. And it's not just you can go, I can order something at
Starbucks therefore I speak English, therefore...
Gary: It takes learners CALP, to learn CALP twice as long as to learn BICS if not three times
longer, right. You learn your basic interpersonal communication skills within like five or
six years of being within the country that you’re in. CALP like the Cognitive Language
Proficiency takes twice as long if not more. If you are a literacy student right or if you've
never had that academic experience, you know, how long did it take for us to learn. From
kindergarten they start us off with show and tell. That is an actual academic skill,
presenting information within the socio-cultural context that you’re in, right. They don't
have that. It takes twice as long to get. ‘Well, they have been here for four years, why
can't they do it?’, ‘Cause it's not, it's a skill set that takes a heck of a lot longer to learn’
(em hem, yeah). Even our navigators, we have a system of what are called navigators.
They are assigned to a student form the second that they enroll into the college and they
are designated to advise the student on what they should be doing, and which road or path
they should be taking. They keep telling our ESL students that they should be trying to
get through ESL as fast as they can (Yeah). That they should be trying to jump levels.
And I'm like 'shut up, like stop telling them this' (yeah), you know, It's not helping right,

182

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
cause they the students the navigators said I should try and skip the next level and I'm
like ‘why would you want to do that?’ (em hem). So, yeah, largely a gap between what is
realistic and what their expectations are, so. (you can't power study a language, its..) No,
it's like math, figure out memorized formulas like [sarcastic tone]. So yeah, very big
disconnect.
Steve: Umm, there was actually a very interesting presentation at the last um conference about
teaching, giving teachers, ahh, mainstream teachers the awareness of language principles,
so they could understand it. They are still teaching English in their class..
Gary: Well on of the this I, I was in a class once with some, um, some preservice high school
teachers who had opted to take this intro to ESL methodology and they had this attitude
‘well I'm teaching science, it's not my job to teach that to teach English. And I lost it. I
said that yes it is, I said that ‘you are teaching everybody in that room how to
communicate in a way that is expected within the field of science, right, the terminology
the when your doing scientific reports, how to report on your results or talk about your
experiment’. Like that is language, ‘You’re building academic language proficiency for
everybody whose in-house first language or second language’. ‘It is your responsibility to
teach them how to communicate about science in a way that is expected in English’. ‘It is
your job to do that so you need to know what that means, right, not just for a second
language learner but for anybody whose in their..’. ‘It just might be easier or faster for the
native speaker to pick it up than your non-native speaker’. ‘But yeah you're part of that,
um, specific language learning process and if you don't recognise that, then your only
serving a certain number of students in your room and your choosing to not teach a
certain number of students in your room’ (yeah). So that was a soap box I went there but

183

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
it was just (no I agree). That was the attitude that was prevalent among the, the, umm, K12 or these other, you know, teachers, you know, um, teachers about what they felt that
their job was that their role was with these, um, non-native speakers of English in their
room. It's prevalent.
Steve: I agree with it, that I've found. I've found that students are almost trained to blame
themselves on their short comings.
Gary: Absolutely yes, we'll if they are not blaming the teacher then I, they are blaming
themselves so. ESL teachers will take the hit. But you're right the self-depreva.. they beat
themselves up. like it's my fault, well, no maybe you don't have good partnership
between, you know, whoever is instructing you, like it is a partnership in a sense . Um, a
learner can't be expected to know all the principles of language learning, they don't, that's
not their job to know how it works. They need a good mentor, they need a good
facilitator who can be a part of that process for them . I want to develop learner autonomy
and a good, um, science instructor should also want to develop learner autonomy, but
that's you teach that, um, in whatever way that you’re doing it so that a learner eventually
can take responsibility for their own learning or not learning but it doesn't happen (and
they need input that they can understand , so).
Steve: It's amazing that as soon as I open up the the question of assessment it goes like so far
and it goes in all directions. (I can't wait to read your paper man) It's gonna be hard to put
together (I'm going to send it to all of my leadership people,, just read this please.) oh
great. I should do two, one in academic sort of language and one in more simple sort of
ha ha (plain language so it's like here kids ha ha). Um, if we pull it back to assessment,

184

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
we talked a lot about the negative washback so the positive washback of the standardized
assessment.
Gary: I mean, we know, I mean, knowing what the gaps are from when our students go form
our program from to the next is awesome. Or within the business community cause we've
developed whole programs, um, and therefore assessments within them, for ESP, more
specific purposes workplace English. Is good to know what is actually happening when
our learners are in those, um, whether it's the workplace or an academic program it's nice
to know what the gaps are because we want to be able to address them. Like 'oh we
weren't even testing for that' do you know what I mean, like, um. Now we can see if they
can do it, right (em hem). Um, or roll it back and looking at our outcomes and saying
‘well our outcomes will never actually meet the thing that you're doing so do we need to
develop another program for it’. Which is, which is possible. An online one off course or
something like this. Or do we need to look at, um, what we are assessing and make sure
that that thing that we want to see happening is getting plugged into our extraction and
therefore into our assessment so we can actually say yeah, we tested for it and the learner
is demonstrating that they can or can't do it and make sure then that the people that are
coming out of our program and going into yours have the level of readiness or
preparedness or keeping validity to manage what is being asked of them. That has been
very helpful. I mean we don't know what we don't know. And if you're upset about
learners who can't do things then tell me what they can't do alright, or tell us what is
working well and we'll we'll keep assessing for that thing, um, getting the feedback after
we developed that new assessment has been really good because we stopped getting, ah,
there was less screaming than ha ha than before about learners being ready or not ready.

185

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
So that's been better, you know, communicating about, um, what they want and whether
or not we can account for it.
Steve: And students like, you've mentioned that they, it's on paper. So you can't argue. They say
I can do this but you say we'll look no you can't. Do you find that the washback with the
students is quite positive in that you can...
Gary: We'll yes and no there are students who are like ‘I don't understand, we don't need a test’.
I'm like ‘yeah you do. Because if we don't give you this test it means that you have to
take IELTS (em hem) in order to get into this program’, ‘Oh’, you know what I mean. So
we have two different types of students. One who like they really love the test, they want
everything to be taught to the test the whole thing and others who don't, they find better
value in just getting feedback rather than having a test and I'm like ‘like we are stuck
between a rock and a hard place guys’, ‘you want to be able to go directly into this
program from ESL you want to have some sort of certificate of piece of paper that says
that your capable. But at the same time you don't want to have the test how are we going
to do this? Right, (em hem) right, in a way that is satisfying all of your needs, we can't’.
Again serving too many masters too many ideas. Ideally, we'd just be handing out
feedback left right and center (yeap). But then everybody would have to take IELTS
(yeah), nobody wants to take IELTS. So nobody's happy.
Steve: Yeap, Why why do you find that that nobody wants to take IELTS?
Gary: It's money, it's well number one none of the students think that the can pass IELTS and
they are probably right. (yeah). And it's expensive. And with IELTS it's as much about
the English as it is about knowing how to take IELTS (yeah) like how to play that game,
right. And so it's for many of the students, they are fully aware that they are going to take

186

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
IELTS prep courses in order to be able to manage that test and they are right. And most
of our guys because of the strict nature of the IELTS test itself are not able to pass it
compared to a CLB test which has the task as opposed to the multiple choice or whatever
right or whatever so.
Steve: So do they feel that the CLB is easy it's an easy it’s easy, it's an easy way sort of?
Gary: They perceive it as such, but it is not . (yeah, yeap) They are never happy, we have never
been able to make the students happy with, um, which is where I guess that a lot of the.. I
mean they're very pleased to spend an entire day bullying a teacher, um, and, you know,
to change a test, to change a mark, to, to just scream, rant, and rave and we do have
student advisory panels (em hem) that are ESL leadership, where there is a representative
from each class who comes forward. All they talk about is tests, you know, all that talk
about is tests, the teacher doesn't prepare me for the test. What they really mean is that
the teacher it not teaching to the test (yeap, Yeap).
Steve: But they are getting listened to aren't they.
Gary: They are getting listened to, what we can do with the information I don't know, but you
know. The other thing they complain about is the text books. You know, we don't have
enough time to finish a text book. ‘No we don't’. The textbook doesn't align with the
outcomes, ‘no it doesn't’, you know, And the teacher doesn't always use the textbook, ‘no
she doesn't or no he doesn't, because it's not good enough right’ .
Steve: Um, so let’s move to some solutions that that you've found, that in a perfect world, ha ha.
Gary: We want to stream, literacy learners for example, we really need to stream them. Um,
learners who are bound for workplace, not academia, we want to stream them, Ah,
learners who are bound for academics, we want to steam them, ah, because then we've,

187

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
you've got then you can tailor everything to it's intended outcome and to the body that's in
the room, right, which is, um, ideally, ah, that would be ideal right. Um, there would be
more time in order to teach and to assess. Ideally, learners would have input at some
point along the way on content curriculum outcomes and assessment to a point , to a
point but, um, for us anyways streaming would be magical. Um, at [our institute] of the
things that is theoretically supposed to be changing is we have a new curriculum
framework policy thing that's going to be launched. It's just been developed it's supposed
to change the whole process of assessment and curriculum development so that it's not
done in stages and it's not done in silos where it's a more organic process within all
stakeholders within the institution or the community itself, so that it's not like ‘Hey Gary
we need to develop a test’, ‘Ok now that you've developed the test now you are going to
meet with the intercultural person’ or, you know, it's like all of those people will be at the
table, (em hem) from the beginning of the development piece itself. Which I think it is
going to change.
Steve: I'm glad that you have brought that up because it's been one of my issues in my previous
employments that the curriculum experts go into a room and close the door and out
comes the curriculum. It's like..
Gary: Well yes and that is what happens with us and then money disappears but there is still
work to be done and it gets done off the side of the desk after that and people are losing
their minds. In our case, what often happens is that a manager often comes to the
developer and says ‘we've decided the things that you need to develop please go and
make..’ And I'm like ‘where was the subject matter expert in this conversation and and
then these train stops along that way that everybody be on the same train at the same time

188

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
developing the piece or, you know, so yeah, this theoretically on paper, it has changed
because now we have this new understanding of how we are going to look at
development and assessment, um, we also just assigned colleges in Canada's Institutes,
Canada's Indigenous protocol, which means the way in which we assess also must change
to adopt an indigenous paradigm of teaching, learning, instruction and assessment. So
that is going to be a new master at the table [yeah]. So, um, which is good, but at the
same time it is adding another (another dimension) another dimension to it so (yeap).
Steve: I quite liked, um ahh, who was I listening to, I can't remember who it was not, talking
about the, ahh, IELTS the outcomes were so British orientated and they were working at
changing the ones for Canada to make it more equality and saying that immigrants, like
why should they be tested on their knowledge of literature of the dominant culture.
Gary: We, oh my God yes, we developed a, um, our very own EAP course because, um, as one
of the ways to address the laddering assessment issue and it's now a in lieu for English
[level 1] so that, um, if an immigrant requires an English [level 1] as a either academic
requirement lets say for whatever U of A. They don't have to take that, they don't have to
read Hamlet, um, they can come into the EAP course and do 'It' and if they get the right
scoring or percentage then they don't have to do Shakespeare. So, we have the exact same
thing (cool) yeap, like it's not relevant, it's totally not relevant oh yeah.
Steve: Um so um, I think we have covered a lot of ground. (sorry I know) No, it's great, though I
will have to slow your voice down when I transcribe though, ha ha (I know I know). Um,
are there any things that we haven't covered that, that sort of come to your mind about
washback instruction?

189

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Gary: I don't know like not off the top of my head. Like your questions definitely get me going
but like it's something I've never, I mean I don't sit there and think about it very often
mostly because I'm in the trenches, um, reflecting on it is really hard. And I think, I think
that that is maybe a missing piece too, is that there is not enough reflection in the
institution or departmentally on the effect of all these masters of the of the washback of
the student expectations too. Our instructional team brings it up at meetings actually quite
regularly but leadership often just throws it's hand in the air and I feel like were chasing
our tails. We have the same conversations all the time. About how we'd like these very
powerful systemic changes so that we don't have these, um, problems anymore so that it
is better for the students. It's not a healthy environment for our students, right, not
because all they want to do is learn about the task. It's not healthy for our instructors
because we are not doing our jobs well and so therefore we are not serving our students
very well or serving the institution well and we don't have enough time to have very good
reflective meaningful conversations about it. Our leadership doesn't seem interested in
asking the questions or finding the answers to the type of questions that you’re asking me
today that are really important. Umm, and I think that is a very crucial, critical missing
piece to what is happening at least with our institution, possibly within many others (em
em definitely, yeap).
Steve: Ok that's good. I think that about wraps it up, thanks for your input, it’s good. It will be
very interesting to delve into it and pull out the themes. I think that many similar things
that you're bringing up are definitely, a lot of the agitations are similar (It wouldn't
surprise me). I will be nice to get this down formally into a study. Thank you for your
time.

190

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Gary: Thanks for contacting me, when I saw what you were doing I went yeah I would talk
about this. Cause somebody needs to be doing it, somebody needs to be chasing this and,
you know, um, helping the field out so, yay (for sure. oh thanks).
Steve: Oh, one more thing, um, can you choose a pseudonym for your name if you'd like. (A
female or male), you can choose (Can I be Gary). Gary sure.

191

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix M: Interview Transcript: Jaden November 29, 2017
Steve: So, first to sort of start off and to give us some sort of background to this study. I know
that my title is quite a mouthful. Contemporary washback on contemporary
Communicative Language Teaching. So if I could just get an idea of your style of
teaching and then we will move on to sort of how assessment effects that style of
teaching. So just give a brief overview of your style of teaching.
Jaden: Umm, so I teach at [institute] and I think it is [deleted] institute in Canada (oh really). I
teach in all the programs. Right now I teach in their graduation student preparation
program where they, um, students who finish their bachelor who are going to apply for
graduate studies in their masters or PhD so that's my students right now.
Steve: Ok, you're teaching English to get them up to graduate level?
Jaden: Um, kind of, their English level yeah. Um, Right now, it's interesting, I usually teach
business writing. I teach them how to write a research paper, how to write a research
proposal as such (yeap). But right now I'm actually teaching IELTS (Oh, with the same
set of students?) Yes, with the same students. (What score are you trying to bring them up
to?) 7.5 (Ok, yeap), um, yeah so in that case, in my current situation there is not much
communicative teaching, we are just doing prep . So, but when I teach research writing or
I teach four skills or a bridging program I target, I use, authentic text, so I bring what is
interesting for the students into the classroom. For example, I take, um, give the students
a sense of ownership to the program (em hem) for example, umm, I get students to bring
their own material. I model what kind of things I'm looking for. I get them to model, um,
the materials from the class, I get them to create student lead discussions every week (em
hem). For example, they would lead a discussion for a given topic that they like to

192

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
discuss in their class. And also, um, it is very student focused , where we do a lot of
things like, um, role-play. So, these are students that are coming to Canada. They just,
like when I see them, they have just arrived, like one day before (yeap). There is a lot of
acculturation that needs to take place. So we do lots of things, we do role-play, we do
needs assessment and we make sure that students will be able to function in universities
in Canadian context at the graduate level. In their countries is, there is not group work,
there is no mediation of any sort. So I need to focus on that to be able to make sure that
students can function in that, can be able to function within that context of masters of
PhD program where they have to work in a small groups or work in pairs to be able to do
collaborative assignments and activities as such.
Steve: So, I hear you say, perhaps lots of tasks, you get them to do. You get them to um roleplay and lots of things to acculturate them to expectations in Canada. Um, imagine that
these things are not what they are expecting, when they come to this country. And having
lots of pushback on that.
Jaden: There is no push back because a lot of my students is, my students are from China and I
think that common perception is teacher knows best and they decided to come to Canada
because of the change in teaching style.
Steve: Thanks good to know, so, um, perhaps when you don't do grammar focused lessons and
rote memorization of vocabulary. What? They don't seem to to complain?
Jaden: No, they don't, um, for example, for grammar. I actually have the students teach me a
grammar point. In day one I get them to write something, and I look at their writing and I
pick up two grammar points that they, uh, they are systemic, that they have problems
with and to teach the class. And I get them to teach the class on those two grammar

193

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
points. I get them to make the exercises. They are just grammar exercises or editing
exercises where they actually have to find a text and they have to purposely find those
two grammar points that they do (yeah) and so their, so they take owner .. I think that
number one, this test is directly relevant to the mistakes that they make. and number two,
all these students you know, especially from, a lot of places, you know, they have been
studying English for 10,12,13,14 years (Yeap). And those people, um, before they finish
their bachelor they, and then they study English for 14 years. So, they have already been
taught grammar. What they have problems with, is that internalizing those lessons, right.
Which is why they can teach it to somebody else, right (good and you can get that across?
that that is what's necessary?) So, I one thing I do in the class, I tell them the reason, my
reason behind why I do it (em hem). Students have to be an active participant in the
classroom to .
Steve: So lots of reasoning with them? for for what needs to happen. and that is how you get
buy-in for (yeah) Ok, um, so the assessment, what is the assessment sort of if you could
tell me loosely that's being used. Well, firstly in the gate-keeping program and the
program just before they enter the undergraduate program and then for the one that you're
doing now.
Jaden: Well for the gatekeeping program the assessment is, um, we actually only have one
standardized assessment. The exit test is anything that we deem appropriate for the level
and for the student needs, for example we have standard listening test which is a 20
minute of a university lecture and they have, you know, gap fill, multiple choice stuff
like that (sort of IELTS style), pardon (kind of IELTS style?), No, because well, no,
because IELTS only has a maximum of 3 words right, you know, also long sentence with

194

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
two questions and paragraph questions and in IELTS there is no synthesis or, um, and
synthesis or analysis it's actually a purely a listening task, right (em hem). So, in the
class, in the gate keeping course I'm more interested in, are they, not just about do they
have the language skills but do they have the language and the academic skills to be
successful in the university.
Steve: Yeap, yeap so you're assessing both academic skills and language proficiency, general
language proficiency.
Jaden: For the gate keeping course and for the, my current IELTS course I teach just IELTS
(yeap, ok that's very nice).
Steve: So, with the gate keeping course, is that test. It's just one standardized test for all of the
teachers that is developed in-house, is that correct?
Jaden: Yeah, it's like two or three or four different versions of it (em hem, ok).
Steve: And what um, is that test change the way you would teach or would like to teach in any
way?
Jaden: Um well, [pause] it's difficult to say. I was in the committee who made it (yeap, ok)
right. I was in the committee who made it and my PhD is in assessment (em hem) and so
because my PhD is in assessment . We'll I don't think I told you about my background I
am an IELTS examiner trainer too. (no you didn't so it's good to have you on board) So, I
am an IELTS examiner trainer, I've been doing IELTS for like since 1992, (em hem) and
I've been [unclear] I don't actually quit. Because I have too much on my plate. Um, and
I'm doing my PhD in assessment (oh excellent, can I ask you your topic) So, my topic is,
[personal conversation] That is kind of what I'm interested in.

195

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: But washback is a, do you think with those automated systems, that's what I worry about
is the washback that comes from that, because once you start isolating small areas that a
computer can test. You can hone in onto that, a teacher can and a student can.
Jaden: I think because I am aware of those issues when I was designing the test for our needs.
Because the needs that we had in the, in the class was not just about language skills it was
about more about, are they able to function in an academic background (yeah). So, I
mean, I was creating the test by analysing the target language use domain like looking at,
um, interviewing professors within the university and different faculties. And we
identified that and we identified what we, though were would be good types of class
judging that feedback from professors (em hem) and there was a lot of feedback, back
and forth. It was just too difficult like maybe as you know it, it was too difficult for the
student [unclear] and as such. And we, because there is also practically of these tests, you
know, into ease of marketing and to ease of assessment for your also looking at not just
the language ability but communicative ability right (Demonstration of critical thinking,
there, yeah). And this applies to a test that we create that is not so much like an IELTS
test or a [unclear] test and test is not just a test it's also, the test is not just about that topic
at that time because it is scaffolded with what we've been doing in the class. So, we've
been studying a theme, so when we a. so a test is on facial recognition, so for two weeks
before the test, just like how they would do it within the university. In a university class
they wouldn't just give you a topic they, you don't know, about and so go and here's a
test. We build up to that point. (yeap) alright, um, so, so the design of the test, they are all
multiple choice questions, we don't actually go over multiple choice so there, um, we
added how to do a multiple choice question how to a true or false and now give them

196

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
questions in the class we cover those, um, we have those in the class for the test but we
believe that part of the test construct is to be able to mimic what they will see in their
stages at a university level (Yeap).
Steve: So you're doing tasks that happen at university and then from that graded their academic
ability. Um, so, are there any sort of alternative assessments that are part of the
summative grade? (No) how do you feel about alternative assessments in general?
Jaden: Um, I think it sounds great, but in theory, but it's difficult when you have so many
students with so many. There is very little standardization (em hem) like for example,
um, I had a teaching literacy workshop the other day, um, and the teacher was saying that
you could use poetry or painting or things to assess their language ability right. (yeah)
But a lot of those things are very subjective (yeap) and a single rubric might not be able
to encapsulate all the different representations of alternative assessment. (yeap) So unless
it was a standardized alternative assessment it kind of defeats the purpose of it. (em hem)
Supposed to do it in a classroom setting where it is shaky.
Steve: Yeah, I'm hearing a lot of that where that the alternative assessment your letting go a lot
of positive washback. If you'd agree with that. Um, so sort of to continue on, um, if I
could talk more about the program that your using IELTS 7.5. (yeap) to get them up to
that level. So you're saying you're doing a lot of test prep. Are you doing a lot of
academic skills work as well?
Jaden: No actually, the class is split into two, one is academic communication and one is IELTS
prep. I only do IELTS prep and that other does the four skills.
Steve: yeah ok, that's fine and, um, the, do you feel that the institute, you've spoke about you
have a lot of communication with the, the teachers in the graduate courses. Are they on

197

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
board, are they happy with the level of students coming in are they able to work with
them at their level, once they enter the program.
Jaden: No, um, this is why that test was revamped last year and this is why we did, so we had
dialogue, dialogic process between the professors of the different facilities, because we
keep track of all those students who enter the university through the program and we
have seen the trend that their success has been dropping (em hem), and this is why the
course was just revamped, this session the course was revamped for that 7.5 level .
Steve: Ok, do you feel that, do you feel that they have an understanding of just what it takes to
learn language.
Jaden: No, well I, where are you from (New Zealand) ok, well I live in [Large city] it's the most
multicultural, supposedly the most multicultural city in the world. (I'm in BC now) Oh
ok. So a lot of professors are, they are not native speakers either. (yeap) Right so I think
they have an understanding, um, so I think there are some aspects that they are more
lenient with their language skills. If they can understand what their student is trying to
communicate then they are fine with that. Even if they are all systematic mistakes and
there's patches that they can't understand. What they have a problem with is the, um, the
academic skills themselves. About, can they synthesize information are they just
regurgitating what they, you know, what they heard. Are they understanding the key, are
they only being able to identify key words in the text based on the question types (em
hem) and be able to just copy the whole thing and being able to get the points, as such,
(em hem) Right, so it's more of the communicative intent of the curriculum and the task
design itself that I think people are more concerned about, are they able to, are they able

198

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
to complete with the students, with the students with a Western style of education, and be
able to discuss ideas in that level. (em hem, for sure).
Steve: Even, if you look at their vocabulary, it's compared to a native speaker. I mean how do
you, How do you compare the two, that what the feeling I get. Um, so if we move back to
communicative language teaching. How do you feel it could be supported on an
institutional level, what's do you feel it's being supported enough? Do you feel that, um, it
needs more supports?
Jaden: Um, I think part of it, I mean our institute is going towards more and more
standardization, standardization of tests, standardization of curriculum, standardization of
even homework assignments, um, there is. I find that there is less and less academic
freedom. Because it's easier to have 10 teachers teaching exactly the same thing, then it's
easier for assessment of both the teacher and the students (yeap) and I think that works a
bit against, um, communicative language teaching because you don't have the freedom to
do what you want based on the students’ needs and desires , (yeap).
Steve: Could this be something to do with such a large institution that that needs to be a
semblance of control so they tend to limit teachers ability to..
Jaden: Well, if you're teaching credit courses, even if you're taking like four, I don't know, four
different sections of the same course. Every teacher teaches it totally differently. The
assignments are not going to be the same. (yeap) And so it's, my feeling is that if they
trusted their instructors who had academic freedom enough. (yeah, Do they trust?) We'll
when I see more and more standardization I don't see it happening like that, but, their
justification is that we have so many new teachers and that the new teachers need to be

199

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
on board. And it's on board and pretty much easier if everything is standardized (yeah)
and that's the justification (yeap).
Steve: Is there training? to to help bringing these teachers up during the summer?
Jaden: No, (Oh, ok) Because we work in [City] our low season is the winter, so we drop by more
than 50% in January, February and March, April (oh ok) it's so cold. Because of that we
have a lot of turn over. I think we have more than like 100 teachers right now and only 19
is full time and like another is part-time. A lot of turnover in the part-time sessionals.
Steve: So, I'm sort of hearing that because of this control, because of this standardization. It's
good for the new teachers like they are saying. But for the ones who have experience, for
the ones who have knowledge and a higher academic background get a little bit trapped.
(Yes) so it's hard to envisage how you could move forward in that setting, That's partly
why I've chose this topic because I've felt that frustration in being sort of, you know, you
do what I need to do. Ok, ok what else have I got here? So, while I read through my notes
are there other areas of washback that concern you that I haven't brought up?
Jaden: Well because a lot of my students are Chinese, testing in a way has positive washback
because I guess it's that the students is familiar with that type of extrinsic motivation,
right, like some students like tests (yeap) like they, if their score is a 10 out of 10 they.
Like some students are motivated. And I think it has a lot to do with intrinsic and
extrinsic motivation (em hem) of assessment, um, and it depends on the class a lot of the
students are not, you know, are not intrinsically motivated, you know, their parents told
them that they needed to study, they needed to do a masters or they needed to do a
bachelors and if you know anything with the Chinese system is that they could get into a
job if they went to a good Chinese university than [unclear] anywhere else, (but) because

200

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
of, they didn't the scores that the needed for their university entrance exam their parents
decided ok next best thing is for you go get a [unclear] English and have a degree outside
of the country. So some of them are lacking that intrinsic motivation and the tests provide
some extrinsic motivation (Yeap, something they can get their hands on).
Steve: So that would bring me to the point of bringing their focus back to wider goal of actually
learning English rather that their fixed goal of just passing the test. Do you find that that
is quite difficult to lead the students towards a wider goal all the time?
Jaden: Um, sure that happens for the final test. But for the test that I'm am allowed to design was
a smaller test for what I am doing inside the class. I think they are more motivated
because they, um, they are using what they learned in the classroom in a meaningful way
that is immediately identifiable for them, um, the goals and to see their own progress of
whose in the classroom, (yeah) like I mentioned with my grammar presentation . I don't
let the students. I I design, you know, sometimes some teachers use the same test over
and over again. but I always change the test depending on the topics of the assessment
current events, you know, and because of that there is, those are our subject test and they
have the meaningful for students contact at that moment and they can relate it back to
language learning. So, that I think they feel they feel a sense of achievement. Most of the
time they are saying I, that they see that I am being inclusive of their needs. and the topic
that they might be interested in (yeap) and so in that positive washback also. (Yeap, so
because it's meaningful, therefore it's engaging for them, um, because it's well aligned to
the test they can see that the wider goal is met as well as the immediate goal of the test.
(So I think you're telling me that it all needs to come together.) Yes, you can't do one
without the other right, if the test is good but the students don't know why they are doing

201

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
the test or the students don't notice the goals of that test, what the test is doing, then it
doesn't work because it doesn't motivate the students. There is no, ah, the students don't
have the realization that there could be a positive washback to testing (yeap yeap).
Steve: Do you have a mid-term tests as well?
Jaden: No, we'll no, I have bi-weekly, every two weeks I have a test, (Yes, but that's not
summative. That's just part of the class?) Part of the class, Well it is progressive so it's all
part of their final score. (Yeah) right, um, so it' summative for the language points that
we've been using for those two weeks (Yeah, Yeah).
Steve: So, um, I've had experience with that where we've had two mid-terms and then a final so,
what ends up happening is it just becomes test-prep. Because there is no time for
anything else. Um, how do you feel about mixing a little bit of a little bit of alternative
assessment with a standardized test? I would say it's the most common form is to have
say 20%, 30% classwork and then the rest on this final test.
Jaden: Um, I think the position of the test in my class is 20% of the final and everything else has
class so, um, so in the class there are homework assignments and there are I think, there
are four homework assignments and four tests, because we are an 8 week program. (yeap)
So with the test that I construct myself sometimes, you know, it doesn't have to be a test.
Or I just need to understand that they understand the points of those two weeks. So
sometimes I do a class crossword puzzle (yeap).
Steve: And that's not, sometimes, I fear that that is sometimes a way for a border line students to
find a a way, to find an avenue to bully to push to get over that line. And do you have
experience with this, or do you worry about that? (About boarder line students being able
to) to use the alternative assessment side to push their grade up just to get through?)

202

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Jaden: No, because I don't have the freedom to push their grade through because all the scores
that they see, they see almost immediately so as soon as. Like the students can calculate
their scores on average. So there is no pushing through of students (Ok, Ok) right
alternative assessment like if I do a crossword puzzle is it for like a vocabulary test. It is
still mainstream test presented in a different way. (Yeah, Yeah) It is not an actual
engaging their skills in an alternative method. (Yeah, It's not a subjective grade, yeah)
Yes, so there is no subjectivity in the test except for when there is synthesis for an
argumentation for when I am grading their style of language use , which is all subjective
right. You know, It’s a normal idea which is all subjective. So yeah, there is no
alternative test and I think that the management looks, you know frowns on alternative
testing because they are more worried about students comparing their assessment
techniques and complaining about it.
Steve: Yeah, yeah, and also what I'm hearing you saying is that it is a good thing. So you're
removing as much subjectivity as possible, allows or creates a situation where where
washback is minimized or at least negative washback is minimized.
Jaden: I understand that sometimes when you produce negative washback you also produce that
positive one right. (Right, yeah, so it's a balance)
Steve: Um, what would you like to see sort of to change in your place of employment? In terms
of testing.
Jaden: I would like to see less standardization. I would like to see more teacher adjustment,
more respect for teachers expert judgement that would allow students to pass. Because
even when assessment says that a student passes, you know that this student is not going
to do well, (Yeah) at university or sometimes when a student passes numerically, you

203

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
know you see what I mean right. Even if they fail then you think that they, you know that
they have the motivation and they are able to excel. Because if our mandate is not just
about language it's about academic readiness to enter university I think teachers should be
allowed to select their expert judgement in those cases and I think because of over
standardization that we losing some of that (em hem, no trust for the teachers to do what
they need to do). Like for example, in the past like if a student is bad in the final exam
then a teacher could give them, you know, the test is only written once right, a different
test and say oh that student just had a bad day (yeah). But now, no, the student has to get,
go to the manager, you have to get approval, you know, and it's it's a whole bunch of, you
know, is a long chain of different people to go to and talk to before that happens, you
know, (yeah).
Steve: Yeah, so um, yeah so greater support for the teachers, greater trust of the teachers. I
imagine support for what the teachers are doing would be quite important as well.
Jaden: I think that professional development of how to make a test, (em hem) you know as
something as simple as, you know, if you want to allow teachers to make tests you need
to have a standardized understanding of what a test, you know, the test construct. (em
hem) and be able to understand inference and validity argument for why a test is valid or
not valid. um, (is it reliable and all of this, yeah) or the practically of the reliability or the
or be able to assess the target use of the language domain and be able to even understand
something like, you know, what is the meaning of a test specification. But there is no
professional development provided to teachers (em hem) and even managers themselves
I don't think they understand um how much work it takes and at one person cannot do,
create a test specification for a test, it has has to be done in a consensusitive in an

204

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
interpretative way and it takes a lot of time to create a single specification for one expert
test right. So I think they just don't understand. (No, and they wouldn't want to invest the
money if they don't understand, Yeah, or the time)
Steve: Ok, I think that pretty much wraps up what I wanted to talk to you about unless you have
any other points to add.
Jaden: I don't think so. I'm on a lot of flu medications, unless you have more questions.

205

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix N: Interview Transcript: Tim December 05, 2017
Steve: So first if you could just give me an outline of your teaching style. Of sort of what it's
like, what you do just sort of generally and how it relates to Communicative Language
Teaching. As a sort of a background.
Tim:

Oh so you want to know how, what I basically do inside my classes?

Steve: Yeah, sort of like, just just so I can line it up and say all of these teachers are
communicative language teachers. All I want you to say is something formally that says
I'm not a traditional, I use textbooks kind of thing.
Tim:

Ok. I've been teaching for quite a while, quite some time now, (yeap) I started formal
teaching back in 1990. Bilingual school in [home country] and uh uh, it's 100%
communicative or that was. Then I moved down to Saudi Arabia where as you know we
worked together, (yeap) and it was a basically communicative student and I mean really
students centered methodology. However when I got here to the UAE. The [Company]
final assessment called the IELTS and the teachers basically prepared their students for
the IELTS exam. So in that sense I personally didn't break my tradition as being
communicative but I was forced to modify my teaching strategy or my approach so we
could actually prepare students for the IELTS exam. So the only communicative aspect
the IELTS had was the interview Again, it was an artificial where students were brought
through a series of topics and were asked a series of questions regarding flying eating
going to a restaurant, what do you think of this, what do you think of that. (yeap) and we
fell back into the grammar teaching because as you know the IELTS has a huge grammar
component (yeap) so and we prepare them for the writing, but again it was artificial
because they had to be prepared for this kind of task 1, task 2, kind of writings. (yeap) so

206

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
it becomes very artificial really. and I can't help it because the institution it linked to the
IELTS and the IELTS is linked to that and so your forced to do as they need (em hem).
Though I make it as communicative as possible. I mean when I teach grammar I don't
teach the simple present as such. I do present situations where you tend to, you know, to
repetitive actions, you know, I work everyday, uh, 'do you brush you teeth?', 'yeah, I
brush my teeth'. 'But are you brushing your teeth right now'. 'Oh, no teacher you're not
you're sitting in the class'. (em hem). So I do try to make grammar as communicative and
a, I would say, functional as possible. But anyway I'm constrained by the needs of the
classroom, anyway.
Steve: And you feel the pressure from the students to always pull it back to the test?
Tim:

Well, yes because that's what they're, that's what the will be tested on. And it's not just the
students it's the institution as such . So now we changed we dropped the IELTS and now
we have a new state designed test called the [name of test] (oh really). So but it is
basically like a placement test. It's just like an IELTS there is no interview though. (Ok)
so the, the language portion of it, the spoken language, the spoken aspect of it is gone. So
we only do reading, writing and basically grammar and vocab. And the vocab is the
maximum is like a C1 the European C1 and that is about as high as the vocab goes. So all
the grammar and all the reading and all the writing is based around that vocab maximum
C1.

Steve: So, now with the new test, it's becoming even worst that IELTS in that it is limiting you
further.
Tim:

Exactly. So as the teachers as such we are all. Basically, teaching for the test. So what we
have done is, we go around looking at the students taking the test and we try to remember

207

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
the kinds of questions we can see . Because we don't have access to the test itself. We can
see the kinds of questions and obviously most of us have been teaching for a while so
based on that we say ok we have got these kinds of questions, we got this kinds of
questions. We've got multiple choice questions on this type of grammar we've got ta ta ta.
So we design something exercises drilled based on the kinds of questions they are going
to get (em hem). But we are not teaching English as such do you see what I mean we just
preparing the students for an exam.
Steve: And is it, does it come. Does anyone. Like are they worried that English is not really
being taught it's just teaching to a test.
Tim:

No. We worry ourselves because we know we are not doing it. (but it doesn't seem to go
above) exactly. You kind of fall in that triangle. There's the institution that has some
objectives and you have you're own objectives, and the student has his own or her own
objectives. So you are kind of pulled. you know the usual thing. But then the final exam
is [exam name] so, you know, you have to prepared them, you know, whether you like it
or not. I mean that's what they are going to see (Yeap)

Steve: Well um, Ok so are there any advantages to the [test]? like positive washback?
Tim:

Well I guess, the only positive thing is like maybe 60 or 70% of the new comers are
actually passing the levels, now I don't know whether they have a higher level as they are
coming from high school now than the did before or it's really us actually succeeding,
you know what I mean (yeap). So I don't have, I don't have the limits to tell you whether
it's us or it's that they have a higher level that they had before (yeah).

208

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: Ok, so if you compare this with the, the. It's good to sort of speak to you because you've
got the two extremes from here and [previous place of employment [OPE] so [OPE] was
totally opposite.
Tim:

Well, I'd rather go for the [OPE] thing. (Yeah) for many reasons, um, I think you do more
justice to the reality. I mean it becomes more authentic teaching the the skills in learning
would allow them to confront random interaction, you know what I mean, (yeap) well the
students here don't seem to be prepared for that. You break the norms and you break
away from what they are used to, they are completely lost, (yeap) So [OPE] I think we
were process deve.. we were assessing process (yeah) so I think it's a more, it's a better
way of doing it because it's. I mean the student learns a lot more , yeap. You see what I
mean, he can be. Basically, that's what I see they can, they confront a random interaction
which is what really happens in real life (Yeap) I mean you go to the classroom you find
somebody and he goes 'yeah, what's up?'. So, that random interaction in more developed
in the students centered communicative processes assessed that the product we are trying
to produce here. Yeap. I think it's artificial, like when they write right now they just
memorize certain, what they have to do in the, in the introductory paragraph in the body
one and body two and then the conclusion. Most of the language is just memorized. It's
artificial.

Steve: Do you find, that with the [OPE] that sure the the teachers that know how to teach in that
way, they are fine. But then was this gap where teachers could come in and sort of not
know what they are doing and just not being able to assess what level they are at all sorts
of problems like this?

209

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Tim:

We'll that's true. So in the process, what your mentioning what you really need very
experienced highly qualified teachers and I don't mean qualified by having degrees, by
actually knowing what they are doing. (yeah) So I was really really happy at [OPE] when
I was teaching because that is the way I think people actually learn . (Yeah, I'm the same
as you, I I miss that). Yeah, I miss that too because I didn't even in fact prepare classes, I
mean we, I just worked on what I saw. Authentic needs of the students (yeah) so I start
interacting with them, I find what their problems are and I develop something based on
that as we go along.

Steve: And I found that you could get them engaged in what, you could find topics that they
liked and you could just go with it.
Tim:

Exactly, Yeah. They could develop their own curriculum in a sense. 'This is what we
need, this is what we will work on', (yeah) over here very thing is prescripted, I mean we
have a textbook which are lousy by the way, because we've been innovating we've been
using this online e-texts. And we are at the crest of the wave. So they are experimenting
with us so they give us this lousy books which are basically being piloting by us (oh)
when we have 6 students who need to pass an exam. (oh no) and that thing is on
backboard, excuse my language anyway, so these things all work on blackboard so we
have, (ha ha) yeah I know, the listenings don't work the interactive portions doesn’t work.
Oh so, do you know what I mean, we are testing the books for this publishers, (em hem)
so this guys want to be, you know, be at the crest, you know, of the wave. (yeah) so we're
being guinea pigs. It's a whole bunch of this that come up with that. But anyway basically
we are almost back to the same old grammar. (yeah) we can't do grammar translation

210

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
because we don't speak Arabic but (ha ha) but if we could that's what we would do to
keep employed, you know what I mean. (ha ha one step away from that, oh dear)
Steve: So what about Ok, I think it is more interesting to talk about the to talk about the [OPE]
system and I think, what about buy-in in the [OPE] system? Like getting students to
believe that this is how you learn. and it's not just through grammar exercises.
Tim:

Well, I think that you don't really need to convince them. I think that after one or two
weeks they realize that they can actually understand and they can actually produce
something so it's more like a demonstration of method. The student, realizes that he is
actually learning (yeap) so. He is not spending time to memorize a script and spit it out
like a parrot. He can actually see his language developing, his skills developing (yeap) so
he's actually looking for interaction. He's almost asking you Ok so I have a problem here
what can I do how can I solve this (em hem) you see the way that [OPE] is taught the
student knows what he wants to say he is encoding language he wants to say what he has
in his in his mind, he just doesn't know how to do it. But he knows what he wants to say
(yeap) in our system here the student is decoding language you give him a piece of
language and he has to decode it understand it and then spit it out again (yeah) and that
makes it very, very difficult so I think the real way is that you encode. That is what
[OPE] was doing. You encode (yeah) you know, you know what to say you just don't
know how. So what yo do is facilitate it by a saying, you want to say this, this is how you
say it. So he doesn't need to understand. He knows, he understands what he is saying
(yeah, yeah for sure) see what I mean, it's all the way around its flipped completely
backwards. (no I totally agree) so I, so that is what I learned after going through the
whole thing. One is encoding language and the other one is decoding language. Most

211

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
people teach, getting the student to decode what you are trying to explain to him, you see
what I mean (yeah), so this perplex in the class this monster is talking to me in a language
I can't even understand and he expects me to understand me I can't do that. (yeah) Do you
know what I mean, and the other one he doesn't need to understand what he wants to say
(yeah for sure) he knows what he wants to say. (I totally agree, I've never thought about it
like that, but totally agree) Thanks that way I see it.
Steve: Do you [OPE] was set up that all the teachers taught the same way so the students got to
go through that flow and they always had the example of the class before to build on. But
what is happening in a lot of places that I talk to is each teachers in a silo. and they do
what they do. So even if you do teach a very communicative method it's gonna be from
the beginning. Do you feel that you really need that everybody on board?
Tim:

I really don't know. (Ok) are you talking about that we need to have everybody teach the
same way?

Steve: Umm, yeah, everybody like with the [OPE] you can have one teacher saying this is what
we do, we have grammar translation and the next class up they say...
Tim:

Oh Ok, I see what you mean. Oh no no obviously do to the encoding thing everybody
needs to work in the same way. You all need need to be on the same page. (yeap) at least
in the approach, the methodology has to be uniform that that I agree (yeap) but the
problem is you know to have enough teachers who actually know what they are doing,
you know, that understand. It's like teaching with a silent movie (yeah) you see what I
mean (yeah) that student understand what's happening. He doesn't know how to say it but
he knows, he understands the movie immediately. So what you need to do is teach him
how to say what he understand in his mind (yeah) see what I mean, (yeap) so he doesn't

212

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
have to decode anything he is just encoding. So that, I think is the best way to learn (yeah
for sure) because you know what you're saying.
Steve: Um, well what about the, institution, like do they really know what it is. Like the
management and then the teachers (no) where they feed into. Do they need to know?
Tim:

Just think about this. There was oil well right now [Capital city] took over very thing it’s
been centralized. There is new management now. and everything that was centralized by
[Capital City]. Now we don't write a single test. We don't write a single test for the
students, we don't mark a single test except for the writing that come random. They put
them up on blackboard and then you might be marking from another another country, do
you know what I mean. So we don't do anything, the assessment are not, we don't assess
our students in any sense. That's it [Capital City] has centralized everything so basically
they give them a grammar placement test and the end of every cycle which we have every
6 or 7 weeks and you prepare them for that (wow) so it's basically vocab, reading writing
test. (yeah) it's like a placement test I think. They use that same exam for all the levels
and according to your score. you wind up level 1 or level 2 or level 3. But I think that is
what they have they have a placement test, so it's something like another little [test name]
(yeah) or another TOEFL kind of you know it's a ranking. I think it's a placement test. So
that is what we have.

Steve: So but students obviously, so who ever passes and [capital city] and [city] when they pass
and they go in to do their academic courses. What are the teachers saying? are they
saying they speak enough English. Are they complaining? What are they saying?

213

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Tim:

Oh no they have, they continue. They have to continue with academic speaking academic
writing umm etc. etc. because there level. Right now the education department I think is
requesting, It's still requiring 6.5 IELTS.

Steve: Oh so they start their undergrad course but they are also doing a language course at the
same time.
Tim:

Yes they continue. Yes, they call it general studies. So the engineers have to take certain
subjects in general studies. So one of them is for example is academic speaking (Yeap
yeap) so they have to practice debates, they have to learn to take notes from a video or
from a lecture and then they are quizzed on them. So they do continue their English
studies after the foundations level.

Steve: If we go back to [OPE] I remember that we had a um a teacher that was doing the
graduate courses give us a meeting as he basically yelled at us and told us we don't know
how to teach and that the students aren't good enough at their English Do you remember
that meeting? (No exactly, but what was the point?) The point was he was telling us we
don't know how to teach English and the students aren't prepared. Um do feel that um at
[OPE] that the students that the teachers above were aware of of what level we brought
could bring them up to and their journey in English and sort of how far they had gone?
Tim:

I think there is a problem with this I mean. What really matters is if you assume that the
student is highly motivated ok (yeap) you know he actually engages the 400 or 600 or
500 hours of English he takes at the college level (yeap) What you need, you know,
research tells us you need between 700 or 1000 hours (yeah) ok, If you're really 100%
engaged if you divide that by man hours 8 hours a day it ends up being 4 months.
Understand me. If you divide 8 hours up, you know like, 8 hours a day it's what it's 100

214

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
days (yeah) 1000 days whatever it is. 100 120 days. It's 4 months, so how much, you can
not go beyond the human compacity (yeah) you know to speak like I do you need like
15000 hours of English, and I'm not a native speaker of English, so obviously. But you
need, It's proportional to the amount of time in input. (yeap) assuming you're highly
motivated and you actually want to learn the language (yeah) so you can't go beyond that,
beyond human capability so I think, I think sometimes you over estimate what students
show be able to learn in a year in a two year academic course at university we need to I
don't know what 500 hours (yeah exactly) I don't know how many hours you actually do
(yeah) you know what I mean, 20 hours a week I don't know how many weeks multiply
that. I don't think it goes beyond 400 or 500 hours. So what can you expect (ha ha
exactly) sometimes we over es.. we don't over estimate we over expect their level of
English, you know, it's proportional to the number of hours and the motivation etc etc.
Steve: We'll it annoys me because I personally I think that they just say just do a course and why
can't you speak? It's silly.
Tim:

It's a question of development in humans, it develops (yeah) so people here have been
trying to speak English for 40 years and they still can't say I went they still I go
yesterday. they still you know the simple present tense (yeap) just for 40 years. We don't
know, you know, nobody really knows what the solution is. (no, just yeah exactly)

Steve: Ok no that's awesome Tim Thanks I think I've go enough. It's good.
Tim:

We weren't talking about washback really.

Steve: We'll we are it's sort of it's more indirect washback.
Tim:

I think it is. I think it basically, is that um, when someone pushes you to take an exam. A
student well you have certain expectations. (yeah) you expect the teachers to meet those

215

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
expectations so obviously teachers modify their way of teaching, their approach to
teaching they they betray their own way of understanding of how they should learn to
simply meet both the management and the student. (yeah, the pressure is enourmus) We
are assessed by the students and by the management, see what I mean. So we end up
modifying our own way of teaching to to meet the needs. I mean it's a socio-economic
triangle.
Steve: Well that's true. That's what I'm kind of doing that this topic because that hopefully that I
come out with something A piece of paper that teachers can just put on managements
desk and say you know read this sort of thing.
Tim:

So I think you should go for. I don't know, what we as teachers do regarding the problem.
I mean we are in that triangle expectations of the students regarding given examples you
know x and y characteristics. How can I break away from, you know, my own way, you
know. I can't explain it. I know what I need to do as a teacher for you to learn. So how
can I meet your expectations given my whole knowledge and at the same time I'm
keeping management happy and eventually they will keep paying me to keep doing my
work. (ha ha) so it's a tricky, it's a tricky little thing. (it is) it's complicated (yeah). Most
of us just end up betraying ourselves basically. If they play rock and roll I dance rock
and roll. If they play tango play tango. (It's a job at the end of the day, we all need the job
so, of course)

Steve: No, I think that in a round about way the other people have said similar thing, so it's good
so it's all good.
Tim:

If management would actually understand. We need somebody who knows language
teaching at the top. (yeah) You know that actually .

216

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: Don't you think though that there is very few teachers that know how to teach properly
anyway.
Tim:

I've been teaching like 50 years for God sake and I'm yeah exactly [personal story]

217

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Appendix O: Interview Transcript: David Dec 08, 2017
Steve: So I understand that you are at a very large institution. Um, if we sort of first start off
with, ah, sort of a background of your class. Sort of, if you could walk me though,
without giving..
David: Just my class not the whole program right?
Steve: Yeah, I'm more sort of giving, to start off to give a background of your teaching.
David: Well in [institution] basically I work for the foundation program (yep) but within the
foundation program we have three sub-programs. We have the academic writing
program, I'm a part of the, actually, I am the coordinator or facilitator for that program,
for the 2 writing courses (em hem) this program serves all the colleges here which use
English as their medium of instruction. We prepare them for research writing,
presentations (em hem), just like an academic program in North America. Academic
writing for undergraduates (yep). So this is what I teach, I teach the advanced level,
writing 2 , and, umm, class capsize usually 20 to 25 (yeap). We have been fighting for 15
but it didn't work, and we teach basically them. We have summary writing, response
writing, and then at the end of the semester they submit a term paper (em hem) a mini
research paper (em hem) and, with a presentation (em hem).
Steve: So um, unless the pass your class they cannot go in to the academic programs?
David: Ah, because I am academic writing 2, if they pass their writing 1 they can take some
classes in the college, they are allowed to do that, (yep). Then they are waiting to pass our
course to take other classes. So there are some prerequisites (yeah)
Steve: And alongside with your writing there are other classes with a, a academic, um, um
reading I imagine, or.?

218

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
David: All the other classes other than us, other than my sub-program, one of the other subprogram is basically teaching all the colleges or other major which are in Arabic, because
now the University has and Arabic streams and this is our biggest stream. (Ok, yeah that's
good).
Steve: And how does the assessment work? What kind of tests do you do?
David: Um, well we have a standardized testing, so we run our test our stream, and well, all our
tests are writing based and the rubrics are not holistic rubric . (Not?) No we don't use
holistic rubrics, (Ok) they are divided into different elements as you know. (yep, can you
give me an example of one element?) Yes, so we have content, so we only look at
content, then we have organization, we look at their paragraph structure (em hem) then
we have the AP or research and documentation element where we look at their citation,
referencing, paraphrasing, integration, (yeah, but not direct, you don't assess grammar or
vocabulary directly) Oh yeah, we have one element which is grammar, that is what I was
going to say, grammar and vocabulary (sorry) where we look at their mechanics. (yep)
like this in general.
Steve: Yep, put it under mechanics, Yep, ok and there is only one test at the end of the semester
or there, um, mid-terms as well?
David: So there are mid-terms as well and there are some in class tests, quizzes and in our course
in my course specifically we don't have a final exam because of the research paper but in
writing one they have a, they have two in-class tests which are equal to mid-term exam
and a final exam.
Steve: Yep ok excellent, um, two mid-terms and a final and the, the stuff that you do in-class the
in-class tests do they, are they, do they make up part of the final grade?

219

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
David: Yes, everything is included in the final grade (yep, yep ok)
Steve: Ok, so let’s move on, now I have a good sense of the background. Um, if we move on to
direct washback. Um, do you find that the test forces you to do things in a different way
that you would normally perhaps normally do things in a class?
David: Um, the only thing that we had to do differently and we keep changing on, because we
just changed our curriculum like a year ago . There was a different type of testing before,
there was a different, now a new team (yep). So, then, well the only thing is that we
consider giving them more input time or teaching time to be increased (em hem) We
didn't have to change our test much, because of this (Ok). But we did have to add like
extra days of teaching, so to prepare the student better because most of them are coming
from Arabic medium background (em hem) they need that extra time of input.
Steve: And you talking input as in speaking and reading?
David: No, no writing also, so for example, you know, last time we did a summary test where the
students the students were supposed to read an academic article and then summarize.
(yep) so then after the test, we all looked at the grades and problems and issues we had to
review the rubric a little bit more (em hem) but the test itself was ok, then the other thing,
we had to add like an extra week of teaching them summary, preparing them how to write
summary, (ok, ok) so basically the curriculum effective than the test itself (Yeah ok).
Steve: And with the language components of the the mechanics, how to you tackle that in-class?
David: Ah, well we assign them a lot of practice and then we show them how language is
assessed, how that element is assessed (Ok) and then practice the thing, we need a lot of
practice (yep, ok, no that's good)

220

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
Steve: Um, so if we could look at, sort of, indirect washback, so (em hem) things like student
pressures how they, they might want to, ah, um, go in a certain direction to get, like a
shortcut to get a gooder, better grade, is there any of this kind of pressure?
David: Um, well in our course, no, because it's the structured type of writing and they will have
to go through that practice and training so, I don't think so. But there are other tests like
only grammar tests, they are shortcuts, but not in my program.
Steve: What I'm sort of getting at is perhaps, students wanting to memorize phrases that they can
just plug in.
David: Oh yes that is a possibility yes. (em hem) but they can memorize again, they can
memorize some phrases to how to, that they could use possibility in summary writing or
linking phrases, not more than that, (ok so) only article (Yeah) because they would
summarize (yep)
Steve: Do you feel that the tests helps avoid a lot of that?
David: Yes. (ok)
Steve: Um, and pressure from perhaps other teachers what they are doing in their classes,
pressure from um the departments above you that you're feeding that students into, are
they saying we want a higher level of writing, this is not good enough, um what are the
pressures that you are feeling?
David: Pressure from the management that there are logistic pressure on us because they are
more worried, our management, top management is more worried, and they are not from
a linguistic language teaching background that's the other thing, they are from sciences
(Yep) or business background, so the pressure we face from this type of management,
number one right now, we are suffering actually, is that we have to have multiple

221

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
versions of our tests, because there are classes which are on a Sunday and Tuesdays and
there are classes which are there are classes on Monday, Wednesday, and Thursdays.
(Sorry you have to have monthly?) Multiple version of this (oh, multiple versions, Yep)
that is one problem for us because they we have to go through a lot of things readability,
to many factors to make sure the, there is some reliability there in the test. (Yep, and you
feel that they don't, because they are not from a language testing background, they don't
understand what) They don't understand anything, we actually, it was very tough for me
because I am the person in charge for exams, so to convince them to allow us to have a
common mid-term exam day. So that they are can go and write a two hour exam, they
didn't want that, logistic pressure is, is, a lot on us . (Do, do you feel that, sorry continue)
No, but I'm, to answer your question more directly. Pressure about proficiency or the
level of the test itself, not that much because they don't understand anything there. (em
hem) So they give us that type of comment. But at the end of the day if there are too
many passing with A grades then we do have to write a report and we do have to do a
program evaluation, why there are too many As and vice versa as well. If there are too
many F grades there (yeah).
Steve: Do you think that management has a good sense of just what it takes to learn a language?
David: I don't think so, (yeah) we had been lamenting about that, (yeah) it is very difficult to
cope with those things. (yep, you not alone that other people that have participated that
seems to be one thing coming through)
Steve: Um, what about teachers from the graduate courses do you do you communicate with
them in writing the exams in preparing the curriculum?

222

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
David: In preparing the curriculum we did a need analysis, like in the colleges and we met other
stakeholders like some of the organisations outside of [institute] and we considered their
feedback input and then we design our curriculum. So we did talk to a lot of people
outside of our program.
Steve: And are they happy with the level of students coming through?
David: Yes they, as of right now, they are happy (em hem) because we are giving them sort of
the skills they need to use when they go to the colleges like writing reports they need all
these skills they are learning now and writing like a project or a short thesis they need all
of these things. So we are just aligning basically our curriculum and assessment to the
college needs (yep).
Steve: And your finding they are getting, yeah, like you're saying the academic skills, um,
critical thinking and all of these skills have managed, you’re managing to build them up.
David: Yes, even the articles we select for our assessment we try to address the major we are
serving like sometime we go to business major, medical sciences. So we try our best.
(yep) like bigger learning outcomes in terms of their course (em hem).
Steve: Now with, um, your testing, because it's writing and you use rubrics. Do you find that
there is a lot of subjectivity? I imagine there is subjectivity but do you find that that
allows a student to then in-turn argue and push for a higher grade?
David: Yeah, sometimes it does because you cannot control that event at all but what we do here
is we do a lot of learning sessions or moderation (Yep) with articles and then so like we
take sample papers and then we grade them and then we discuss together. So we try our
best to be on the same page but sometimes of course there are issues with students (Of

223

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
course) we introduce a second grader or third grader. (Yeah, of course) an extra marker,
yeah) It's in control it's like on or two every semester, which is good (yep).
Steve: So, you’re working as a team and you're able to overcome those issues.
David: Before, every test or before any graded element we meet for an hour or sometimes more
than an hour and we grade sample papers use the same rubric and we discuss each
element of the rubric to make sure. And that is the other thing, we try our best not to
make a lot of revisions or changes on the rubric for at least more than a year, three to four
semesters.
Steve: Now isn't a question like with you have standardized testing. Would you personally like
to see more alternative style testing, portfolios um other kind of things that are alternative
come in?
David: We did have portfolio. But it didn't work well enough in our context. Because of the
culture I think (Yep) so we had to give up on that and we had, we have some online
elements as well (yep) we have readings, journals blogs, we do a lot of other alternatives
as well but the weighting is low because some people, the people here don't support that
type of alternatives at that much. Especially that top management (em hem)
Steve: So you had yep, Can you so you said perhaps the culture so you're talking the culture of
management? the culture of the students?
David: Students as well because in the high schools here as opposed to North America they don't
even learn any Microsoft skills, so software skills at all, they don't even know how to use
a Microsoft word. Some of my students, they don't even know how to login (em hem) to
their blog. (ok) So there were issues with technology and they were not able to. It was

224

ASSESSMENT AND STAKEHOLDER INFLUENCE ON CLT
easier actually to come up with a portfolio but for most of the students and parents they
were complaining about it. (Yeah, not understanding how, how it works)
Steve: Um, Ok I think I've pretty much covered what I want to cover with you, um, are there any
points that you wanted to bring up that interest you with washback?
David: I think, it's all the same things that we all agree upon. (Yeah) nothing new at this time,
Ok.

225