Assignment 8
Summary Beyond Test: Alternatives in Assessment

BEYOND TESTS: ALTERNATIVES IN
ASSESSMENT

The defining characteristics of the various alternatives in assessment that have been commonly used across the profession were aptly summed up by Brown and Hudson (1998, pp. 654-655). Alternatives in assessments

1. require students to perform, create, produce, or do something;
2. use real-world contexts or Simulations;
3.  are nonintrusive in that they extend the day-to-day classroom activities;
4.  allow students to be assessed on what they normally do in class every day;
5. use tasks that represent meaningful instructional activities;
6. focus on processes as well as products;
7. tap into higher-level thinking and problem-solving skills;
8. provide information about both the strengths and weaknesses of students;
9. are multiculturally sensitive whenpropedy administered;
10. ensure that people, not machines, do the scoring, using human judgment;
11. encourage open disclosure of standards and rating criteria; and
12. call upon teachers to perform new instructional and assessment roles.

THE DILEMMA OF MAXIMIZING BOTH PRACTICALITY AND WASHBACK

The principal purpose of this chapter is to examine some of the alternatives in assessment that are markedly different from formal tests. Tests, especially large-scale standardized tests, tend to be one-shot performances that are timed, multiple-chOice, decontextualized, norm-referenced, and that foster extrinsic motivation. On the other hand, tasks like portfolios, journals, and self-assessment are

• open-ended in their time orientation and format,
• contextualized to a curriculum,
• referenced to the criteria (objectives) of that curriculum, and
• likely to build intrinsic motivation.

Formal standardized tests are almost by definition highly practical, reliable instruments. They are designed to minimize time and money on the part of test designer and test-taker, and to be painstakingly accurate in their scoring. AJternatives such as portfolio, or conferencing with students on drafts of written work, or observations of learners over time all require considerable time and effort on the part of the teacher and the student.

PERFORMANCE-BASED ASSESSMENT

O'Malley and Valdez Pierce (1996) considered performance-based assessment to be a subset of authentic assessment. In other words, not all authentic assessment is performance-based. One could infer that reading, listening, and thinking have many authentic manifestations, but since they are not directly observable in and of themselves, they are not performance-based. According to O'Malley· and Valdez Pierce (p. 5), the following are characteristics of performance assessment:

1. Students make a constructed response.
2. They engage in bigber-order tbinking, with open-ended tasks.
3. Tasks are meaningful engaging, and authentic.
4. Tasks call for the integration oflanguage skills.
5. Both process and product are assessed.
6. Depth of a student's mastery is emphasized over breadth.

Performance-based assessment needs to be approached with caution. It is tempting for teachers to assume that if a student is doing something, then the process hasfulfilled its own goal and the evaluator-needs only to make a mark inthe grade book that says "accomplished”  next to a particular competency. In reality, performances as assessment procedures need to be treated with the same rigor as traditional tests. This implies that teachers should

• state the overall goal of the performance,
• specify the objectives (criteria) of the performance in detail,
• prepare students for performance in stepwise progressions,
• use a reliable evaluation form, checklist; or rating sheet,
• treat performances as opportunities for giving feedback and provide that feedback systematically, and
• if possible, utilize self- and peer-assessments judiciously.

PORTOFOLIOS

According to Genesee and Upshur (1996), a portfolio is "a purposeful collection ofstudents' work that demonstrates ... their efforts, progress, and achievements in given areas" (p. 99). Portfolios include materials such as

• essays and compositions in draft and fmal forms;
• reports, project outlines;
• poetry and creative prose;
• artwork, photos, newspaper or magazine clippings;
•  audio and/or video recordings of presentations, demonstrations, etc.;
• journals, diaries, and other personal reflections; .
• tests, test scores, and written homework exercises;
• notes on lectures; and
• self· and peer-assessments comments, evaluations, and checklists.

Gottlieb (1995) suggested a developmental scheme for considering the nature and purpose of portfolios, using the acronym CRADLE to designate six possible attributes of a portfolio:

1. Collecting
2. Reflecting
3. Assessing
4. Documenting
5. Linking
6. Evaluating

The advantages of engaging students in portfolio development have been extolled in a number ofsources (Genesee & Upshur, 1996; O'Malley &Valdez Pierce, 1996; Brown & Hudson, 1998; Weigle, 2002). A synthesis of those characteristics gives us a number of potential benefits. Portfolios

• foster intrinsic motivation, responsibility, and ownership,
• promote student-teacher interaction with the teacher as facilitator,
• individualize learning and celebrate the uniqueness of each student,
• provide tangible evidence of a student's work,
• facilitate Critical thinking, self-assessment, and revision processes,
• offer opportunities for collaborative work with peers, and
• permit assessment of multiple dimensions of language learning.

At the same time, care must be taken lest portfolios become a haphazard pile of "junk" the purpose of which is a mystery to both teacher and student. Portfolios can fail if objectives are not clear, if guidelines are not given to students, if systematic periodic review and feedback are not present, and so on. Sometimes the thought of asking students to develop a portfolio is a daunting challenge, especially for new teachers and for those who have never created a portfolio on their own. Successful portfolio development will depend on following a number of steps and guidelines.

1. State objectives clearly.
2. Give guidelines on what materials to include.
3. Communicate assessment criteria to students.
4. Designate time within the curriculum for portfolio development.
5. Establish periodic schedules for review and conferencing.
6. Designate an accessible place to keep portfolios.
7. Provide positive washback-giving final assessments.

JOURNAlS

A journal is a log (or "account") of one's thoughts, feelings, reactions, assessments, ideas, or progress toward goals, usually written with little attention to structure, form, or correctness. Learners can articulate their thoughts without the threat of those thoughts being judged later (usually by the teacher). Sometimes journals are rambling sets of verbiage that represent a stream of consciousness with no particular point, purpose, or audience. Fortunately, models of journal use in educational practice have sought to tighten up this style of journal in· order to give them some focus (Staton et al., 1987). The result is the emergence of a number of overlapping categories or purposes in journal writing, such as the following:

• language-learning logs
• grammar journals
• responses to readings
• strategies-based learning logs
• self-assessment reflections
• diaries of attitudes, feelings, and other affective factors
• acculturation logs

It is important to turn the advantages and potential drawbacks of journals into positive general steps and guidelines for using journals as assessment instruments. The following steps are not coincidentally parallel to those cited above for portfolio development:

1. Sensitively introduce students to the concept ofjournal writing.
2. State the objective(s) of the journal.
3. Give guidelines on what kinds oftopics to include.
4. Carefully specify the criteria for assessing or grading journals.
5. Provide optimalfeedback in your responses.
6. Designate appropriate time frames and scbedules for review.
7. Provide formative, wasbback-giving final comments.

CONFERENCES AND INTERVIEWS

Conferences are not limited to drafts of written work. Including portfolios and journals discussed above, the list of possible functions and subject matter for conferencing is substantial:

• commenting on drafts of essays and reports
• reviewing portfolios
• responding to journals ,
• advising on a student's plan for an oral presentation
• assessing a proposal for a project
• giving feedback on the results of performance on a test
• clarifying understanding of a reading
• exploring strategies-based options for enhancement or compensation
• focusing on aspects of oral production
• checking a student's self-assessment of a performance
• setting personal goals for the near future
• assessing general progress in a course

Discussions of alternatives in assessment usually encompass one specialized kind of conference: an interview. This term is intended to denote a context in .. which a teacher interviews a student for a designated assessment purpose. Interviews may have one or more of several possible goals, in which the teacher

• assesses the student's oral production,
• ascertains a studenfs needs before deSigning a course or curriculum,
• seeks to discover a student's learning styles and preferences,
• asks a student to assess his or her own petiormance, and
• requests an evaluation of a course.

OBSERVATIONS

All teachers, whether they are aware of it or not, observe their students in the classroom almost constantly Virtually every question. every response, and almost every nonverbal behavior is, at some level of perception, noticed. All those intuitive perceptions are stored as little bits and pieces of information about students that can form a composite impression of a student's ability. Without eyer administering a test or a quiz, teachers know a lot about their students. In fact, experienced teachers are so good at this almost subliminal process of assessment that their estimates of a student's competence are often highly correlated with actual independently administered test scores. (See Acton, 1979, for an example.)

Designing a system for observing is no simple task. Recording your observations can take the form of anecdotal records, checklists, or rating scales. Anecdotal records should be as specific as possible in focusing on the objective of the observation, but they are so varied in form that to suggest format nere wouId- be counterproductive. Their very purpose is more note-taking than record-keeping.The key is to devise a system that maintains the principle of reliability as closely as possible. Checklists are a viable alternative for recording observation results. Some checklists ofstudent classroom performance, such as the COLT observation scheme devised by Spada and Frohlich (1995), are elaborate grids referring to such variables as

• whole-class, group, and individual participation,
• content of the topic,
• linguistic competence (form, function, discourse, Sociolinguistic),
• materials being used, and
• skill (listening, speaking, reading, writing),

SELF- AND PEER-ASSESSMENTS

Self-assessment derives its theoretical justification from a number of wellestablished principles of second language acquisition. The principle of autonomy starids_ qut as one of the primary foundation stones of successful learning. The ability to set one's own goals both within and beyond the structure of a classroom curriculum, to pursue them without the presence of an external prod, and to independently monitor that pursuit are all keys to success. Developing intrinsic motivation that comes from a self-propelled desire to excel is at the top of the list of successful acquisition of any set of skills.

Peer-assessment appeals to similar prinCiples, the most obvious ofwhich is cooperative learning. Many people go through a whole regimen of education from kindergarten up through a graduate degree and never come to appreciate the value of collaboration in learning-the benefit of a community oflearners capable of teaching each 'other something. Peer-assessment is simply one arm of a plethoIa of tasks and procedures within the domain of learner-centered and collaborative education.

• Types of Self- and Peer-Assessment

It is important to distinguish among several different types ofself- and peer-assessment and to apply them accordingly. I have borrowed from widely accepted classifications of strategic options to create five categories of self- and peer-assessment:
(1) direct assessment of performance, (2) indirect assessment of performance, (3) metacognitive assessment, (4) assessment of socioaffective factors, and (5) student self-generated tests.

1. Assessment offa specific} performance. In this category, a student typically monitors him- or herself-in either oral or written production-and renders some kind of evaluation of performance. The evaluation takes place immediately or very soon after the performance. Thus, having made an oral presentation, the student (or a peer) fills out a checklist that rates performance on a defined scale. Or perhaps the student views a video-recorded lecture and completes a self-corrected ·comprehension quiz. A journal mayserve as a tool for such"self-assessment. Peer editing is an excellent example of direct assessment of a specific performance.
2. Indirect assessment of[general) competence. Indirect self- or peer-assessment targets larger slices of time with a view to rendering an evaluatioIl'of general ability, as opposed to one specific, relatively time-cortstrained performance. The distinction between direct and indirect assessments is the classic competence-performance distinction. Self- and peer-assessments of performance are limited in time and focus to a relatively short performance.
3. Metacognitive assessment [for setting goals}. Some kinds of evaluation are more strategic in nature, with the purpose not just of viewing past performance -or competence but of setting goals and maintaining an eye on the process oftheir pursuit. Personal goal-setting has the advantage offostering intrinsic motivation and of providing learners with that extra-special impetus from having set and accomplished one's own goals. Strategic planning and self-monitoring can take the form of journal entries, choices from a list of possibilities, questionnaires, or cooperative (oral) pair or group planning.
4. Socioaffective assessment. Yet another type of self- and peer-assessment comes in the form of methods of examining affective factors in learning. Such assessment is quite different from looking at and planning linguistic aspects of acquisition. It requires looking at oneself through a psychological lens and may not differ greatly from self-assessment across a number of subject-matter areas or for any set of personal skills.
5. Student-generated tests. A final type of assessment that is not usually classified strictly as self- or peer-assessment is the technique of engaging students in the process of constructing tests themselves. The traditional view of what a test is would never allow students to engage in test construction, but student-generated tests can be productive, intrinsically motivating, autonomy-building processes.

• Guidelines for Self- and Peer-Assessment

Self- and peer-assessment are among the best possible formative types of assessment and possibly the most rewarding, but they must be carefully designed and administered for them to reach their potential. Four guidelines will help teachers bring this intrinsically motivating task into the classroom successfully.

1. Tell students the purpose ofthe assessment. Self-assessment is a process that many students-especially those in traditional educational systems-will initially fmd quite uncomfortable.
2. Define the task(s) clearly. Ifyou are offering a rating sheet or questionnaire, the task is not complex, but an open-ended journal entry could leave students perplexed about what to write. Guidelines and models will be of great help in clarifying the procedures.
3. Encourage impartial evaluation ofperformance or ability. One of the greatest drawbacks to self-assessment is the threat of subjectivity.
4. Ensure benefictal washback through follow-up tasks. It is not enough to simply toss aself-checklist at students and then walk away. Systematic follow-up can be accomplished through further self-analysis, journal reflection, written feedback from the teacher, conferencing with the teacher, purposeful goal-setting by the student, or any combination of the above.

• A Taxonomy of Self-and Peer-Assessment Tasks

An evaluation of self- and peer-assessment according to our classic principles of assessment yields a pattern that is quite consistent with other alternatives to assessment that have been analyzed in this chapter. Practicality can achieve a moderate level with such procedures as checklists· and questionnaires, while reliability risks remaining at a low level, given the variation within and across learners. Once students accept the notion that they can legitimately assess themselves, then face validity can be raised from what might otherwise be a low level. Adherence to course objectives will maintain a high degree of content validity. Authenticity and washback both have very high potential because students are centering on their own linguistic needs and are receiving useful feedback.

References:
Brown, H. Douglas. 2004. Language Assessment: Principle and Classroom Practices. New York: Pearson Education