The Second Language Proficiency Test (TPLS) is an educational resource created by and for the CEGEP network. Initially, the project was implemented by the Ministère de l’Éducation, du Loisir et du Sport (MELS) when it agreed to develop, along with the Official Languages Program, a scale of second language proficiency levels and related evaluation mechanisms. The general objective of the project was to improve the second language proficiency of college graduates, as recommended by the Commission d’évaluation de l’enseignement collégial (CEEC). The project led to a long collaboration between the members of the coordinating committees responsible for the production of French and English tests: a large number of teachers and professionals in the French and English CEGEP networks participated in the first phase of development of the linguistic scales and items associated with the resulting tests.

In the fall of 2012, the Collegial Centre for Educational Materials Development (CCDMD) was entrusted with a new mandate to develop a second language proficiency test. The CCDMD set up a production team (project managers, content specialists, programmers and technicians) after having consulted some twenty professionals and experts in the college and university networks. When the technical analysis was concluded, the CCDMD proposed an adaptive test model and a linguistic scale based on new criteria.

 

The primary objective of the Second Language Proficiency Test (TPLS) is to determine the proficiency level of the respondent in order to propose appropriate courses as well as tailored learning activities within those courses.

At the end of the test, the result is given as a number from 1 to 10 corresponding to a level on the linguistic scale.

The teacher and student can then consult the proficiency level descriptions for information on what the student is able to do at that level. The teacher can also use this information to plan learning activities to help the student progress to the next level. As a result, the TPLS goes further than a traditional placement test, such as the eTCALS, which is designed strictly to assign a student to a course level according to his or her results. Note, however, that a test manager in a subscribing college has the option in the management module to associate a course number with each of the 10 levels. 

The proficiency of students taking an upgrading (mise à niveau) course is at the lower levels of the scale. A level 10 result means that the student may be more proficient than the maximum level measured by the test. Ideally, college graduates should reach level 8: subsequent university training should enable students to fine-tune their proficiency and achieve level 9 or higher during their university studies.

The TPLS is made up of two categories of items: written comprehension and oral comprehension. There are five types of items: associations, multiple choice, sequencing, cloze text with drag and drop, and cloze text with menus. Each item includes:

• general instructions, provided in words and images;
• a text (written comprehension) or audio recording (oral comprehension);
• a question;
• a choice of answers.

The TPLS is an adaptive test, which is non-linear: each correct or incorrect answer affects the progression of the test. The test gradually estimates the provisional skill level of the respondent (θ value) and dynamically selects the next item most likely to assess the respondent’s actual skill level. The test continues until the estimated skill level becomes stable, and the final θ value is translated into a proficiency level on the language scale that is the reference for the measurement system.

An adaptive test, in comparison with a non-adaptive test, makes it possible to reduce the number of items presented and decrease the test administration time, which can increase the reliability of skill assessment.

The written instructions for each item, intended to guide the respondent in his or her approach, are accompanied by visual instructions. As a result, a respondent who is not sufficiently proficient to read the written instructions should be able to understand the task and answer the question. Visual instructions are provided for all items, including those in the advanced levels. This strategy creates a predictable virtual environment suitable for various learning styles.

Images have also been included with the text or audio recording in the majority of items in proficiency levels 1 to 3. In some cases, the answer choices consist of numbers, images or even colours.

Visual instructions that accompany the written instructions

1. You will have 15 seconds to read the instructions, the question and the answer choices.
2. After the tone, you will hear a dialogue.
3. Answer the question.

Accommodation for students with special needs

Since the objective of the TPLS is to determine reading and listening proficiency, no writing support measure (screen reader) or oral support measure (audio transcription) has been provided for in the test. The allotted answer time for each item is regulated by the administrator.

The test is an objective measure of second language proficiency throughout the college network. We therefore request that educational institutions do not adapt it, as we wish to avoid distortion of the data collected. 

The CCDMD production team would be pleased to discuss the development of an untimed replacement test, hosted on a separate virtual environment, to meet the needs of colleges wishing to accommodate groups of students with special needs.

The CCDMD can be reached at 514 873-2200 or info@ccdmd.qc.ca.

The TPLS is intended to be administered twice during a student’s college studies: once at the beginning and once before the diploma is awarded. The test manager of a subscribing college may, however, decide to use it differently. The test can be used as a placement test under certain conditions, as a test within a language course to determine group level or as a measure of the student's level compared to an expected level for certain professional careers. However, since students do not progress rapidly to a higher level of proficiency, it is not recommended they take the test repeatedly during their studies.

The management module provides an institution's test manager with all of the necessary features to manage a subscription. From this module, the manager can create groups, register respondents and define the testing schedule. The manager has access to both the overall results and detailed data. The manager can also configure the test as needed - for example, by associating linguistic scale levels with specific course numbers, reformulating certain messages, choosing whether or not to display results at the end of the test and defining the export format for results to a spreadsheet, for statistical analysis or to the institution's educational platform.

The test can be taken online from home, in class or in a lab. Students can access the test using an ID and a password sent by the test manager. The monitoring of tests administered in class or in a lab is facilitated by the fact that the test progression (the type and order of items) varies with the respondent. Similarly, the correct answer is not always in the same order in the list of answer choices.

The scale used for the TPLS was developed on the basis of a comparison with three recognized scales in Québec: the Échelle québécoise des niveaux de compétence en français des personnes immigrantes adultes, and the two scales produced at the request of the Ministère de l’Éducation, du Loisir et du Sport: the Quebec College ESL Benchmarks (provisional version), and the Échelle des niveaux de compétence en français langue seconde pour le collégial (provisional version).

These reference scales include 12 levels. For the proficiency test project, only the first 10 levels were taken into account, in order to comply with the reality of most CEGEP students. Also, only skills related to comprehension (oral and written) were included. The results of the comprehension test serve to guide decisions regarding the placement of candidates in the various courses offered. In the development of the linguistic scale, the design team first selected indicators whose content was common to that of at least two scales and that demonstrated potential for operationalization in an adaptive test. The team subsequently chose other indicators that were consistent with and complementary to the ones initially selected. It later added descriptions to the scale from the Canadian Language Benchmarks (CLB).

Importance of a Scale of Proficiency Levels

The interpretation of a proficiency level result based on a scale offers a number of advantages:

• The information obtained is standardized and can thus be compared with that derived from other tests and scales developed according to the same principles;
• The proficiency levels in the scale provide clear, practical information that can be used to analyze programs and develop their content with a view to standardizing second language services at the college level;
• These proficiency levels offer clear, practical information that can be used to formulate the learning objectives to be achieved: students can take the test at the beginning and end of their studies to measure their progression;
• Proficiency test results can be used by students in certain programs to evaluate their proficiency in preparation for language exams administered by the Office québécois de la langue française.

The Scale of Second Language Proficiency Levels for the College Student describes the development of second language proficiency in 10 levels. For each level, it defines what the learner is able to understand orally and in writing.

The proficiency test is made up of a number of items divided into two categories:

• Items that focus on audio recordings and measure the respondent’s degree of proficiency in oral comprehension;
• Items that focus on texts and measure the respondent’s degree of proficiency in written comprehension.