Assignment 5

Summary Designing Clasroom Language Tests

A. Test Types
1. Language Aptitude Tests

One type of test-although admittedly not a very common one-predicts a person's success prior 'to exposure to the second language. A language aptitude test is designed to measure capacity or general ability to learn a foreign language and ultimate success in that undertaking. Language aptitude tests are ostensibly designed to apply to the classroom learning of any language.

2. Proficiency Tests

A proficiency test is not limited to anyone course, curriculwn, or single skill in the language; rather, it tests overall ability. ProfiCiency tests have traditionally consisted of standardized multiple-choice items on grammar, vocabulary, reading comprehension, and aural comprehension. A typical example of a standardized proficiency test is the Test of English as a Foreign Language (TOEFL) produced by the Educational Testing Service.

3. Placement Tests

Certain proficiency tests can act in the role of placement tests, the purpose of which is to place a student into a particular level or section of a language curriculum or school. A placement test usually, but not always, includes a sampling of the material to be covered in the various courses in a curriculum; a student's performance on the test should indicate the point at which the student will find material neither too easy nor too difficult but appropriately challenging.

The English as a Second Language Placement Test (ESLP1) at San Francisco State University has three parts. In Part I, students read a short articre and then write a summary essay. In Part II, students write a composition in response to an article. Part III is multiple-choice: students read an essay and identify grammar errors in it. The maximum time allowed for the test is three hours.

4. Diagnostic Tests

A diagnostic test is designed to diagnose specified aspects of a language. A test in pronunciation, for example, might diagnose the phonological features of English that are difficult for learners and should therefore become part of a curriculum. Usually, such tests offer a checklist of features for the administrator (often the teacher) to use in pinpointing difficulties. A writing diagnostic would elicit'a writing sample from students that would allow the teacher to identify those rhetorical and linguistic features on which the course needed to focus special attention.

A typical diagnostic test of oral production was created by Clifford Prator (1972) to accompany a manual of English pronunciation. Test-takers are directed to read a ISO-word passage while they are tape-recorded. The test administrator then refers to an. inventory of phonological items for analyzing a learner's production. After multiple listenings, the administrator produces a checklist of errors in five separate categories, each of which has several subcategories. The main' categories include.

• stress and rhythm,
• intonation,
• vowels,
• consonants, and
• other factors. 

5. Achievement Tests

An achievement test is related directly to classroom lessons, units, or even a total curriculum. Achievement tests are (or should be) limited to particular material addressed in a curriculum within a particular time frame and are offered after a course haS focused on the objectives in question
Achievement tests are often summative because they are administered at the end of a unit dr term of study. They also play an important formative role. An effective achievenlent test will offer washback about the quality of a learner's performance in subsets of the unit or course. This washback contributes to the formative nature of such tests.
The specifications for an achievement test should be determined by

• the objectives of the lesson, unit, or course being assessed,
• the relative importance (or 'weight) assigned to each objective,
• the tasks employed in classroom lessons during the unit of time,
• practicality issues, such as the tinle frame for the test and turnaround time, and
• the extent to which the test structure lends itself to formative washback. 

B. Some  Practical Steps To Test Construction

1. Assessing Clear, Unambiguous Objectives

In addition to knowing the purpose of the test you're creating, you need to know as specifically as possible what it is you want to test.

2. Drawing Up Test Specifications

Test specifications for classroom use can be a simple and practical outline of your test.

3. Devising Test Tasks

You begin and end with nonscored items (wann-up and wind down) designed to set students at ease, and then sandwich between them items intended to test the objective (level cbeck) and a little beyond (Probe).

4. Designing Multipie-Choice Test Items

In the sample achievement test above, two of the five components (both of the listening sections) specified a multiple-choice format for items. Multiple-choice items, which may appear to be the Simplest kind of item to construct, are extremely difficult to design correctly. Hughes (2003, pp. 76-78) cau￾tions against a number of weaknesses of multiple-choice items:

• The technique tests only recognition knowledge.
• Guessing may have a considerabIe effect on test scores.
• The technique severely restrict what can be tested.
• It is very difficult to write successful items.
• Washback may be harmful.
• Cheating may be facilitated. 

The two prinCiples that stand out in support of multiple-choice formats are, of course, practicality and reliability.
Since there will be occasions when multiple-choice items are appropriate, consider the following four guidelines for designing multiple-choice items for classroom-based and large-scale situations (adapted from Gronlund, 1998, pp.60-75, and]. D. Brown, 1996, pp. 54-57).

• Design each item to measure a specific objective.
• State both stem and options as simply and directly as possible.
• Make certain that the intended answer is clearly the only correct one.
• Use item indices to accept, discard, or revise items

C. Scoring, Grading, and Giving Feedback 

1. Scoring

As you design a classroom test, you must consider how the test will be scored and graded. Your scoring plan reflects the relative weight that you place on each section and items in each section. The integrated-skills class that we have been using as an example focuses on listening and speaking skills with some attention to reading and writing.

2. Grading

Your first thought might be that assigning grades to student performance on this test would be easy: just give an "A" for 90-100 percent, a "B" for 80-89 percent, and, so on.

3. Giving Feedback

A section on scoring and grading would not be complete without some consideration of the forms in which you will offer feedback to your students, feedback that you want to become beneficial washback. Washback is achieved when students can, through the testing experience, identify their areas of success and challenge.

References:
Brown, H. Douglas. 2004. Language Assessment: Principle and Classroom Practices. New York: Pearson Education

Assignment  4
Summary  principles of  language assessment include: practicality, reliability, validity, authenticity and washback .

• Practicality

Affective test is practical. This means that it

• is not excessively expensive,
• stays within appropriate time constraints,
• is relatively easy to administer, and
• has a scoring/evaluation procedure that is specific and time-efficient.

A test that is prohibitively expensive is impractical. A test of language proficiency that takes a student five hours to complete is impractical-it consumes more time (and money) than necessary to accomplish its objective. A test that takes a few minutes for a student to take and several hours for an examiner to evaluate is impractical for most classroom situations. A test that can be scored only by computer is impractical if the test takes place a thousand miles away from the nearest computer. The v~lue and quality of a test sometimes hinge on such nitty-gritty, practical considerations.

• Reliability

If you give the same test to the same student or matched students on two different occasions, the test should yield similar results. The issue of reliability of a test may best be addressed by considering a ·number of factors that may contribute to the unreliability of a test.

• Student-Related Reliability 

The most common learner-related issue in reliability is caused by temporary illness, fatigue, a "bad day," anxiety, and other physical or psychological factors, which may make an "observed"score deviate from one's "true" score. Also included in this category are such factors as a test-taker's "test-wiseness" or strategies for efficient test taking (Mousavi, 2002, p. 804).

• Rater Reliability 

Human error, subjectivity, and bias may enter into the scoring process. Inter-rater reliability occurs when.two or more scorers yield inconsistent scores of the same test, possibly for lack of attention to scoring criteria, inexperience, iriattention, or even preconceived biases.

Intra-rater reliability isa common occurrence for classroom . teachers because of unclear scoring. criteJ;"ia, fatigue, bias toward particular "good" and "bad" students, or simple carelessness.

• Test Administration Reliability

Unreliability may also result from the conditions in which the test is administered. I once witnessed the administration of a test of aural comprehension in which a tape recorder played items for comprehension, but because of street noise outside the building, students sitting next to windows could not hear the tape accurately. This was a clear case of unreliability caused by the conditions of the test administration.

• Test Reliability

Sometimes the nature of the test itself can cause measurement errors. If a test is too ,long, test-takers may become fatigued by the time they reach the later items and hastily respond incorrectly. Timed tests may discriminate against students who do not perform well on a test with a time limit.

• Validity

By-far the most complex criterion ofan effective test-and arguably the most important principle-is validity, "the extent to which inferences made from assessment results are appropiate, meaningful, and useful in terms of the purpose of the assessment" (Gronlund, 1998, p. 226). We will look at these five types of evidence below.

• Content-Related Evidence
• Criterion-Related Evidence
• Construct-Related Evidence
• Consequential Validity
• Face Validity

• Authenticity

A fourth major principle oflanguage testing is authenticity, a concept that is a little slippery to define, especially within the art and science of evaluating and designing tests. Bachman and Palmer, (1996, p. 23) define authenticity as "the degree of correspondence of the characteristics of a given language test task to the features of a target language task," and then suggest an agenda for identifying those target language tasks and for transforming them into valid test items.

• Washback

A facet of consequential validity, discussed above, is "the effect of testing on teaching and learning" (Hughes, 2003, p. 1), otherwise known among language-testing , specialists as washback. In large-scale assessment, washback generally refers to the effects the tests have on instruction in terms of how students prepare for the test.

"Cram" courses and "teaching to the test" are examples of such washback. Another form of washback that occurs more in classroom assessment is the information thAt "wa,shes back" to students in the form of useful diagnoses of strengths and weaknesses.Washback also includes the effects ofan assessment on teaching and learning prior to the assessment itself, that is, on preparation for the assessment. Informal performance assessment is by nature more likely to have built-in was~back effects because the teacher is usually providing interactive feedback. Formal tests can also have positive washback, but they provide no washback if the students receive a simple letter grade or a single overall numerical score.

References:
Brown, H. Douglas. 2004. Language Assessment: Principle and Classroom Practices. New York: Pearson Education

Assignment 3
Analysis of the 3 principles of language assessment (practicality, Reliability, and validity)

Latihan Soal Ujian Nasional Bahasa Inggris SMP

A. Topik : Announcement
Questions 1-2 are based on the following text.

To All English Literature Class Students
(1) Exam will be held on Wednesday, October 23rd 2014, at 10 a.m. in room 143. Please read A Clean and Well-Lighted Place by E. Hemingway and Queenie by Alice Munro as they will be the subject materials. Copies of both are available in my locker, feel free to grab them!
(5) It will be an open book exam. You can bring books as many as you can, but no internet and cell phones! Cheaters will be expelled!
Professor Smith

1.To whom the announcement is made?
A. Students of English Literature Class
B. Professor Smith
C. E. Hemingway and Alice Munro
D. The readers

2. Which statement is correct based on the text?
A. Students cant open their books in the exam.
B. Using internet from the cell phones is allowed.
C. Prof. Smith put copies of subject materials inside her locker.
D. The exam will be held at 8 a.m.

B. Topik : Notice
Questions for number 3 to 5!

3. The notice above means.....
A. wash hand before work
B. wash hand before eat
C. use the shop for washing hand
D. wash hand in the bathroom

4. The statement based on the text is
A. The notice is for parents.
B. The notice is for students.
C. The notice is for employees.
D. The notice is for the guest.

5. Where you can find the notice?
A. In the market
B. In the class
C. In the company
D. In the street

C. Topik : Narrative
Questions 6-9 are based on the following text.

(1) A hen was so jealous at her friend, a goose, who could lay golden eggs. The farmer was very caring with his goose and he always feed the goose the best corn and wheat, as opposed to other livestock which were provided only with the usual fodder.
(5) The goose is so lucky. She lays golden eggs so she gets more affection from our master, said the hen.
(7) One day, the farmer came into the barn and said greedily, I wonder if there are many golden eggs inside the goose." Then, he captured the goose, took her to his home, and butchered her. The hen, knowing the fate of the poor goose, said, I am so lucky! If only I had laid golden eggs, I would have been dead instead of the goose!

6. What is the appropriate title for the text?
A. The Goose
B. The Jealous Hen
C. The Greedy Farmer
D. The Pursuit of Golden Eggs

7. What is the main idea of paragraph 3?
A. The hen was grateful because of her luck
B. The goose laid more eggs
C. The farmer butchered the hen
D. The hen was jealous with the goose

8. Which of the following statement is NOT mentioned in the text?
A. The goose was dead in the end.
B. The farmer always feed the goose the best food.
C. The goose was the farmers favorite.
D. The hen never felt unlucky.

 Then, he captured the goose, took her to his home, and butchered her.
9. What is the closest meaning of the underlined word?
A. Intimidated
B. Fed
C. Cared
D. Slaughtered

D. Topik : Advertisement
Questions 10-11 are based on the text below.

10. What is advertised from the text above?
A. Tea
B. Beverages
C. Perfume
D. rinks

"The Most Treasured Name in Perfume".
11.What is the meaning of the underlined phrase?
A. Most valuable
B. Most expensive
C. Most helpful
D. Most desired

Analysis of practicality, Reliability, and validity

• Practicality

In this test the level of practicality has been fulfilled, namely the problem is economical and does not require funding from students, the discipline of this question is easy to follow and easily understood by students and remains within the right time limits and the correct test procedures and is specific and time-saving.

• Reability 

is an index that shows the extent of an instrument reliable and reliable gauges. A test is said to have provisi on if reliable, consistent / stable and productive whenever the test used. From this problem can in dicate the level of stability of 9th grades tudents in junior high school. This means that the test can be used to express or measure the level of student knowledge of English subjects.

• Validity

validation in this problem is how we determine whether or not a problem can and distinguish groups in the aspects measured according to differences. The validity of the questions here shows an index of discrimination in distinguishing between high-ability test takers from low-ability test takers.

References:
https://blog.ruangguru.com/latihan-soal-ujian-nasional-bahasa-inggris-smp-dan-pembahasannya

Assignment 2
1. Explain types and objectives of assessment (achievement, diagnostic, placement, proficiency, and  aptitude tests)

• achievement assessment/test is to determine whether course objectives have been met with skills acquired by the end of a period of instruction. Achievement tests should be limited to particular material addressed in a curriculum within a particular time frame. Achievement tests belong to summative because they are administered at the end on a unit/term of study. It analyzes the extent to which students have acquired language that have already been taught.
• Diagnostic test the purpose is to diagnose specific aspects of a language. These tests offer a checklist of features for the teacher to use in discovering difficulties. Proficiency tests should elicit information on what students need to work in the future; therefore the test will typically offer more detailed subcategorized information on the learner. For example, a writing diagnostic test would first elicit a writing sample of the students. Then, the teacher would identify the organization, content, spelling, grammar, or vocabulary of their writing. Based on that identifying, teacher would know the needs of students that should have special focus.
• The purpose of placement test is to place a student into a particular level or section of a language curriculum or school. It usually includes a sampling of the material to be covered in the various courses in a curriculum. A students performance on the test should indicate the point at which the student will find material neither too easy nor too difficult. Placement tests come in many varieties: assessing comprehension and production, responding through written and oral performance, multiple choice, and gap filling formats. One of the examples of Placement tests is the English as a Second Language Placement Test (ESLPT) at San Francisco State University.
• Proficiency test is to test global competence in a language. It tests overall ability regardless of any training they previously had in the language. Proficiency tests have traditionally consisted of standardized multiple-choices item on grammar, vocabulary, reading comprehension, and listening comprehension. One of a standardized proficiency test is TOEFL.
• Aptitude test is to predict a persons success to exposure to the foreign language. According to John Carrol and Stanley Sapon (the authors of MLAT), language aptitude tests does not refer to whether or not an individual can learn a foreign language; but it refers to how well an individual can learn a 
• foreign language in a given amount of time and under given conditions. In other words, this test is done to determine how quickly and easily a learner learn language in language course or language training program.

2. Identify issues in language assessment

• Behavior

In the middle of 20 century, teaching and testing were influenced by behaviorism. Testing focused sentence structure, translation from L1 to L2, grammar and vocabulary items.

• Integrative 

The integrative approach refers to a test that seeks to integrate knowledge of systematic components of language (pronounciation, grammar, vocabulary) with an understanding of contex (McNamara, 2000). In an integrative test language is not viewed in a discrete component and isolated from its context. According to Heaton (1989), the integrative approach does not separate the skills like in discrete tests but requires students to use more than one skill simultaneously.

• Communicative language

Communicative language test is a test of learners language performance in meaning/real life situations. The test does not only test the learners competence, that is, what the learners know about the language and about how to use it, but also to the performance, that is, to what extent the learners are able to actually demonstrate in a meaningful or real life situations.

• Performance based assessment

Performance based assessment measures student ability to apply the skills and knowledge learned from a unit or units of study. Typically, the task challenges students to use their higher-order thinking skills to create a product or complete a process (Chun,2010).

3. Identify hot topics relating to classroom based assessment

Gardner groups student capabilities into eight broad categories (each student's unique learning style is a combination of these intelligences):

• Logical/mathematical (uses numbers effectively)
• Visual/spatial (is artistically or spatially perceptive)
• Bodily/kinesthetic (excels at tasks that require physical movement)
• Musical (perceives and/or expresses musical forms and patterns)
• Linguistic (uses words effectively)
• Interpersonal (responds well to others)
• Intrapersonal (is reflective and inner-directed)
• Naturalist (makes distinctions in the natural world)

According to Muller (2008), traditional assessment is an assessment that refers to choosing a response and more to measuring students' memories related to information obtained. This can be done through measurement of multiple-choice tests, cloze tests, true-false tests, matching and the like. Students typically choose an answer or memorize information to complete the assessment.

Authentic assessment is a form of task that requires learners to demonstrate meaningful performance in the real world, which is the application of the essence of knowledge and skills. Authentic assessment emphasizes the ability of learners to demonstrate their knowledge in a tangible and meaningful way. Assessment activities are not just asking or tapping knowledge that has been known to learners, but the real performance of knowledge that has been mastered.

Computer Based Test is a test conducted using a computer. Computer-based test items in the form of multiple choice (objective). For certain conditions, we can use this test to assess students.

References:

www.google.com/amp/s/thejoyoflanguageassessment.wordpress.com/2012/12/19/kind-of-test/amp/
https://www.slideshare.net/mobile/RahilaKhan6/assessments-concepts-and-issues
http://www.proenglishteacher.com/2015/04/asesmen-pengertian-asesmen-asesmen.html?m=1