DEVELOPING an ENGLISH PERFORMANCE TEST for INCOMING INDONESIAN COLLEGE STUDENTS

Abstract

Abstracts This study constructed, developed and validated an English Performance test as a complementary to an existing placement test in Universitas Klabat, Manado, Indonesia. It is designed to provide a valid criterion-based measurement within the placement programs of the said university. The study aimed to answer the following questions: (1) What test tasks and items are needed in a performance test based on current language testing theory? (2) Is the performance test valid? (3) Is the scoring of the performance test reliable? (4) Is the performance test practical and predictive? And (5) What are the ratings of the resultant performance test? The steps in developing a performance test involved conducting needs analysis, establishing attributes of good performance test, and constructing test specification based on current language testing theory. The speech event chosen which would generate language use was applying for financial assistance. From this speech event, four activities were elicited: (1) writing a letter of inquiry, (2) completing an application form, (3) making an appointment for interview, (4) giving an oral presentation/interview. These activities represent the four authentic tasks in which the desirable modes/channel of communication, language functions and skills, genre, and topic are integrated. The developed test is divided into four sections corresponding with the elicited tasks: (1) Formal letter, (2) Application form, (3) Making Appointment, and (4) Oral Presentation. The test was validated in several ways: (1) face validation compared the scores of Indonesian studying in the Philippines and in Indonesia, and found that the scores are highly correlated at Spearman ρ = .85. (2) The content validation relied on the evaluation of expert informants. The finding shows that the content coverage and relevance of the test is highly satisfactory. (3) The concurrent validation was conducted to the existing placement test and found that the two tests are highly correlated at .964 using gamma test. (4) The construct validation was conducted in two ways: item analysis for objective section of the test, and Rasch Analysis for subjective section. The item analysis found that only three items are “fair”, while the remaining are either “good” or “very good”. Rasch Analysis found that the raters could interpret the rating system and that the test is well fit and accurate. In general, the proposed performance test is valid. The reliability of the test was established using the internal consistency index. The findings revealed that all sections of the test are highly consistent at Kendall’s Coefficient of Concordance of .927. The correlation between sections to overall scores was also calculated, and found that each section is highly correlated at.942 (Formal Letter), .934 (Application Form), .917 (Making Appointment) and .862 (Oral Presentation) using Kendall’s tau-b test. The practicality of the test was established by analyzing the percentage distribution of score of the pilot tests and field test. The findings showed that the scores are well distributed in all four levels, and a pattern of distribution exists at Spearman’s ρ .85 correlation coefficient. Therefore, the rating system is found practical and predictive. The output of the study is a resultant and valid English Performance Test for Incoming Indonesian College Students. The test is named: Academic English Performance Test or ACCEPT.