Charles Mifsud
Josephine Milton
Literacy Unit,
Faculty of Education, University of Malta
and
Greg Brooks
National Foundation for Educational Research, England
November 1998
Literacy Unit Monograph Series No. 1: University of Malta
CONTENTS
1. Background
2. Aims
3. Method
LIST OF TABLES
Table 1: Sample of Pupils by Gender
Table 2: Analysis of results by order of test presentation
Table 3: Mean scores for Maltese and English tests
by Gender
Table 4: Mean scores for the Maltese and English
tests by Pupilís Age
Table 5: Mean scores for the Maltese and English
tests by Preschool Education
Table 6: Mean scores for the Maltese and English
tests by Special Educational Needs
Table 7: Mean scores for tests according to Fatherís
Occupation
Table 8: Distribution of Parents' Levels of Education
Table 9: Mean scores for tests according to Motherís
Education
Table 10: Mean scores for the tests according to
Fatherís Education
Table 11: Mean scores according to the number of
Year 2 classes in School
LIST OF FIGURES
Figure 1: Correlation between Maltese and English
tests
Figure 2: Comparative distribution of frequencies
of Maltese and English Scores
The Maltese Islands in the Mediterranean Sea have two official languages. Maltese, a Semitic language, is the native tongue. English is used widely and is taught from early on at school. For Malta it is the Language of Wider Communication (LWC).
In primary schools oracy skills in English are introduced early on alongside Maltese at the kindergarten level and the beginning of Year 1 (five- and six-year-olds) of formal schooling. Literacy skills in English are introduced later on in Year 1. In Year 2 (six- to seven-year-olds) further oracy and literacy work is carried out in both Maltese and English. Increasingly, as children move from primary to secondary and then into tertiary education, English gains in importance.
Literacy in both languages is therefore essential; hence the decision of the Ministry of Education and the University of Malta in 1996 to carry out a national survey of literacy attainment, to take place in 1999. It was decided that this, the first ever national survey of educational performance undertaken in Malta, should focus on the reading attainment of Year 2 pupils, in both Maltese and English.
The survey needed to be preceded by a pilot study. The Ministry of Education commissioned this study from the Early Literacy Unit of the Faculty of Education of the University of Malta. The pilot study took place in early June 1998, and this is the report of that study. The report is also intended to serve as a consultation document for refining the planning of the main survey.
The objectives of this pilot study were:
One problem faced by those who would like to investigate literacy skills in the Maltese situation is the lack of availability of standardised tests. There have been some attempts to design and trial word reading tests in Maltese and English by Falzon (1972), and in Maltese by Bartolo (1988), and, for children in Year 1, there is a Maltese-language version of Clay's Concepts About Print Test (Martinelli, 1996). But these tests are either out of date, or too narrowly focused on the reading of single words, or too simple for the relevant age range.
English
After inspection of and consultation on the range of available English-language
tests, a standardised British test was chosen for assessing the pupils'
level of attainment in English. The test selected was the Literacy
Baseline within the Reading Progress Tests series (Vincent, Crumpler, and
de la Mare, 1996). This test is the first of a series of seven standardised
tests developed by the East London Assessment Group at the University of
East London. The Literacy Baseline is designed to measure pre-literacy
and initial literacy skills that are usually associated with later success
in learning to read. It covers children's early development and includes
the reading a child would encounter at home or at school.
The test is designed to be administered to small groups of pupils and includes different aspects of literacy. These are:
Phonological awareness: initial sounds (six items)
Phonological awareness: rhymes (two items)
Literacy concepts (seven items)
Letters: names (four items)
Letters: sounds (four items)
Reading: picture to word (five items)
Reading: word to picture (three items)
Reading: sentence to picture (two items)
Spelling (seven items)
The test contains 40 items in all, and is standardised (in Britain) for the age range 5 years 0 months to 6 years 2 months (5:00-6:02). The decision to use it with somewhat older Maltese children took account of the fact that for most Maltese children English is an additional language.
Maltese
A Maltese parallel version of the Literacy Baseline was devised by
a group of Maltese educators who are specialists in the Maltese language
and have extensive teaching experience. An important criterion was
that the words chosen should be high-frequency words that seven-year-old
children encounter at home or school. According to Grech (1997), the most
frequent themes treated in Maltese literature for children are their own
lives, nature and animals. His study provided guidelines for the test designers
as to which words would be most familiar to Year 2 students. It was
also important that items were highly imageable (that is, could easily
be depicted through pictures) as pictures are included in the test as memory
triggers.
The Literacy Baseline features mainly monosyllabic English nouns. Since the Maltese language, being Semitic, does not possess a vast repertoire of monosyllabic nouns, bisyllabic high-frequency and familiar nouns were used.
The Maltese test contained exactly the same number of items as the English version, and covered exactly the same skills. Also, the page layout, graphic style and typeface were identical to those in the English version.
Instruction Booklets
For the English version of the test, the original Teacher's Manual
was used as the Instruction Booklet for test administrators. However,
if the test administrators felt that the pupils were not understanding
all the instructions in English they were free to explain in Maltese as
well.
For the Maltese version of the test, the original Teacherís manual was translated into Maltese and adapted according to the Maltese test items. The test administrators were free to explain in English if there were pupils who were unable to follow the instructions in Maltese.
3.2 Other Research Instruments
The other research instruments used in this study were intended to gather information about the schools, Year 2 classes and students involved, and to obtain the test administrators' opinion of the tests after they were administered.
School Questionnaire
A School Questionnaire was designed (see Appendix I, p.1) which requested
data at school level: the geographical region of the school and the number
of male and female pupils in Year 2. The Head or Assistant Head of School
was requested to provide this information.
Pupil Data Form
A Pupil Data Form (see Appendix I, p. 2) was designed which requested
information about a variety of pupil- and family-level background variables:
age, gender, first language, father's occupation, mother's occupation,
father's level of education, mother's level of education, special educational
needs, and pre-school education. It was envisaged that the test administrators
would be able to obtain most of the information from the Cumulative Record
Card (CRC) that schools usually compile for every student. The CRC lists
the pupil's name, surname, date of birth, father or guardian's name, mother's
name, address, father's occupation, and the pupil's personality and behaviour,
physique and health, and school(s) attended. Space is also provided for
other relevant information and for final reports by the Heads of school.
Additional information which does not feature in the CRC, such as mother's
occupation, mother's level of education and father's level of education,
would need to be obtained through the class teacher or the pupils' parents
or primary carer(s).
Test Administrator's Questionnaire
A Questionnaire (see Appendix I, p. 3) was designed which the test
administrators were to be requested to complete after administering the
Maltese and English tests. This questionnaire dealt with whether the test
administrators felt that the tests were suitable and matched the students'
levels of literacy, and whether the tests were easy or difficult to administer.
The Questionnaire also invited the test administrators to provide other
comments about the tests.
Both tests, and the School Questionnaire and Pupil Data Form, were field tested in two state primary schools. Neither school was participating in the pilot study.
Field testing of the Maltese and English Literacy Baseline tests was carried out by members of the project team. Both were field tested with a group of five pupils in one of the schools. The Maltese version was further field tested with five pupils in the other school. In both schools, the testing took about half an hour, the pupils understood the instructions they were asked to follow, their performance was satisfactory as they obtained high scores, and no major problems were encountered. As a result of the experience gained, two of the stimuli in the Maltese version were modified.
The School Questionnaire and Pupil Data Form were field tested by two teachers, one in each of the schools.
The School Questionnaire was considered to be straightforward and easy to complete.
The Pupil Data Form, which had to be completed for all the students in the class, was considered time-consuming as the background details required were not always readily available. As expected, the items concerning mother's occupation and father's and mother's level of education proved to be problematic since this information was not readily available in the CRC. On the other hand, the father's occupation was usually listed in the CRC.
The Test Administrator's Questionnaire was reviewed by all 13 complementary teachers involved in the pilot study at one of the training seminars organised by the Early Literacy Unit of the University of Malta and they considered it to be very clear and easy to complete.
Sampling was conducted at two levels, schools and pupils.
At school level, the sample consisted of 12 state schools, two from each of the six geographical regions in Malta and Gozo (Inner Harbour, Outer Harbour, South Eastern, Western, Northern and Gozo), to ensure that the sample was geographically representative. One school in each region was deemed by educators to be high-achieving and the other to be low-achieving, according to the results obtained at the primary school-leaving national examinations.
To achieve the pupil-level sample, a Year 2 class from each of the 12 schools was selected randomly.
The test administrators were 13 complementary teachers based in the 12 schools participating in the study. Two of the teachers covered the same school (and returned one School Questionnaire, etc., between them, hence the overall number of 12 in section 4.2). Since these teachers are not responsible for a particular class they were able to attend the training sessions during school hours without disrupting the school schedule.
In Year 2, complementary teachers are usually attached to one or two primary schools in the state sector. Their role is to provide learning support services to children who are under-achieving in the basic subjects (Maltese, English and Mathematics) at Year 2 level. This support is provided either in class or in withdrawal groups.
A first training seminar was held on 5 May 1998 and included: an introduction to the aims and objectives of the study, their role as complementary teachers and test administrators, an introduction to the various research instruments they would be required to complete, and instructions for administering and marking the tests. There were opportunities for questions and feedback throughout.
During the second seminar on 29 May 1998, all the test administrators became familiar with all the research instruments and instructions since these were reviewed extensively. At the end of this seminar they were provided with a pack including the School Questionnaire, Test Administrator's Questionnaire, Pupil Data Forms, mark sheets, tests and instruction booklets.
A post-test seminar was held on 15 June 1998 so that the teachers could provide feedback regarding the tests. They also reported the pupils' reactions to the tests.
Each class was administered the English and Maltese tests. To reduce practice effects the teachers were asked to administer the Maltese test first to the pupils who were listed as odd numbers and the English test first to the pupils listed as even numbers in the class register. (For breaches of this instruction, and implications of that for the main survey, see sections 4.3 and 5.2.) Test administrators were to allow at least one day between the administration of the two versions of the test.
Pupils who were absent on one of the testing days were allowed to sit for the test if they returned to school in the same week in which the test was administered to the rest of the class. The tests were administered to groups of about 10 to 12 pupils at a time. All items were presented orally by the test administrator, except for the reading sections towards the end of the test. The pupils were given a test booklet and the test administrator took them through the test item by item, giving the pupils adequate time to complete each item before proceeding to the next. It was very important for the pupils to complete their answers without discussion, collaboration or copying. The test administrator encouraged the pupils to attempt all questions and explained that most choices should be indicated by underlining or circling a choice.
Each section of the tests included a practice item. The test administrator proceeded with the rest of the section after making sure that all the pupils had understood what should be done and giving feedback as to whether their responses were right or wrong. During the rest of the test, only feedback relating to the method of answering was allowed (if more than one choice had been marked or if the child was at the wrong item). Where there were target words, these were repeated twice and up to three times if requested. Suggested wording and detailed item-by-item instructions were provided in the Instructions Booklet for both the Maltese and English tests.
The test administrators were trained to score the tests according to the instructions in the original Teacher's Manual (English version) and the adapted Instructions Booklet (Maltese version). Each right answer was to be awarded one mark. Some tests from each Year 2 class were double checked at random before data entry to ensure that the marking was accurate.
This section first gives the size of the achieved sample, and then reports the administrators' opinions of the tests. Following some technical data on the reliability of the tests, the rest of the section is devoted to reporting statistical analyses of the test results, first overall, then against background variables. Within the background variables, those relevant to individual pupils (gender, first language, age, preschool education, Special Educational Needs) are considered first; then those relevant to their homes (parents' occupations and levels of education); finally those relevant to their schools (region, number of Year 2 classes in school).
It should be noted that many of the analyses reported here (and others which were carried out on the data but are not reported) were preliminary practice analyses. They were not necessarily intended to provide definitive information - they could not, because of the small sample size - but were intended to put in place the procedures that will be needed for the main survey, and to indicate how those procedures will be implemented.
The pilot study was successfully administered in all 12 selected classes in the 12 selected schools. Eight of the schools were Primary 'A' schools (containing Kindergarten and Years 1 to 3), and the remaining four schools were Primary 'C' schools (containing Kindergarten and Years 1 to 6).
In all, 255 pupils were tested, or an average of 21 per class. Almost all (250) took both versions of the test, but because of pupil absences the final numbers taking each version, broken down by gender, were as shown in Table 1.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4.2 Suitability and Manageability
All 12 Test Administrators' Questionnaires were completed.
Regarding the Maltese language test:
Regarding the English language test:
Overall, the test administrators' ratings showed a high level of
approval of the tests.
4.3 Reliability
The reliability of the tests was estimated in two ways: the correlation
between the two versions, and their internal reliability. Also, the
tests had been (mainly) administered in counter-balanced order (half Maltese
first, half English first); in order to be able to merge data across the
two orders of presentation for analysis against background variables it
had to be established that performance on each test did not differ according
to order of presentation.
The correlation between the two tests was very high: on Pearson's Product-moment coefficient of linear correlation, r=0.82. Figure 1 illustrates the correlation.
Cronbach's standardized alpha computed from correlations also indicated clearly that both the Maltese (alpha=0.76) and the English (alpha=0.87) versions of the tests had high internal reliability.
Table 2 presents the results by order of presentation
of the tests.
| Table 2 Analysis of results by order of presentation |
|
|
|
|||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Overall, there were no significant differences (Maltese test t=0.51, p>0.05; English test t=0.64, p>0.05) according to which test was taken first.
However, two classes are not included in Table 2 because their test administrators found it more convenient to give the whole class one test first and then the other. One test administrator administered first the English and then the Maltese version to the whole class. Here the pupils' performance was better for Maltese. The difference here is not surprising as all 10 of the schools represented in Table 2 obtained a higher score for Maltese.
Another test administrator administered the Maltese test first and the English test second to the whole class. This particular class obtained a higher score in English. This unusual result could be due to their having already been exposed to the Maltese test. This reinforces the need for administration of the tests to be counter-balanced in all schools in the main survey, and for administrators to be punctilious in doing this.
Because the results in these two schools went in opposite directions, and because the results for the other 10 schools were not significantly different according to order of presentation, the data for the two orders of presentation were merged, and the whole sample was treated as one on each test, not as two independent samples.
4.4 Main results
Table 3 presents the results for the two test
versions, overall and by gender.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The maximum score on both tests was 40. The overall average score on the Maltese version (33.5) was 84 per cent of the maximum, and the overall average score for English (30.2) was 76 per cent of the maximum. The skew in the results towards the top end of the range is clearly visible in Figure 2, which shows the distribution of scores for both tests.
This 'ceiling effect' suggests that the test was slightly easy for Maltese
Year 2 pupils.
It is consistent with this that no pupil scored less than 15 on the
Maltese test, and no pupil scored less than 11 on the English test. However,
there was a noticeable 'tail' in the distribution of scores, namely of
pupils who scored 25 or less. These pupils might well need extra
teaching for literacy, in both languages; and the tests might therefore
serve as screening devices for pupils who need this special attention.
There was a highly significant difference (p<0.001) between the average scores for the Maltese and English versions, both for the whole group and for male and female pupils separately, with the score for Maltese being higher. This result is not surprising, given that virtually all of the pupils were native speakers of Maltese and had English as an additional language.
What is more remarkable and important about these results is the narrowness of the difference between the two languages: 2.3 score points, or 6 per cent of the maximum score. This suggests that Maltese society and its education system are very successful in teaching children to be bilingual and biliterate at an early age.
Pupil variables
4.5 Differences in performance between
boys and girls
The relevant results have already been presented, in Table 3.
Girls had higher average scores than boys for both languages. The difference in average scores was not statistically significant for English (p>0.05), but a significant difference (t=0.0007, p<0.001) was found for Maltese.
4.6 First Language
Very few pupils (6 out of 255, or 2 per cent) had a first language
other than Maltese. Four pupils had English as their first language, while
only two pupils had a language other than Maltese or English as their first
language. All twelve schools participating were state schools. The picture
might have been different if Church or private schools had been included.
Thus, results against pupils' first language are not reported.
4.7 Age
The original intention was that the pupil sample should consist of
children who belonged officially to Year 2 according to their age (that
is, according to their date of birth), and not of children who happened
to be enrolled in Year 2. According to the official definition, Year 2
children in the school year 1997/98 were those born between 1st January
1991 and 31st December 1991 inclusive. Given that the pilot took place
in early June 1998, the childrenís ages at the date of testing should all
have been within the range 6:05 to 7:05, with an average of 6:11.
Most of the children tested in the pilot met this definition, but there were a few children (less than 10 for both the Maltese and the English test) who had been born in 1990, and even one child who was 9:02 (110 months) old at the date of testing and who had therefore been born in 1989. This child was excluded from all the analyses.
The children born in 1990 were retained in most of the analyses, but for the present purpose, the analysis by age, these childrenís results were split into two bands according to their dates of birth, as follows:
January to June 1991 (age-range on date of testing: 7:00-7:05)
July to December 1991 (age-range on date of testing: 6:05-6:11)
Table 4 presents the results according to the
pupils' ages.
| Maltese | English |
| Category | Mean | SD | N (246) | Mean | SD | N (247) |
| 1 | 33.4 | 5.7 | 79 | 29.5 | 6.4 | 81 |
| 2 | 33.7 | 5.5 | 167 | 30.3 | 6.2 | 166 |
The differences between the two bands were not statistically significant.
4.8 Preschool Education
Preschool education was measured according to the number of years of
preschool education: 1, 2 or 3. Table 5
presents the pupils' mean scores according to how many years of preschool
education they had had. Only one student in the sample happened to
have had three years of preschool education.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Pupils who had had two years of preschool education performed significantly better on both the Maltese (p<0.001) and English (p<0.001) versions of the test than those with only one year of preschool education. This result is consistent with those from many countries in the world, and reinforces the importance of preschool education.
4.9 Special Educational Needs
The test administrators were asked to identify pupils with special
educational needs (SEN). This was to be interpreted to include a wide range
of learning difficulties and disabilities. Table
6 presents the mean scores according to whether pupils had special educational
needs.
|
|
|
| Category | Mean | SD | N (253) | Mean | SD | N (255) |
| SEN | 32.9 | 6.6 | 32 | 29.0 | 6.7 | 32 |
| Others | 33.6 | 5.5 | 221 | 30.3 | 6.2 | 223 |
The differences in Table 6 were not statistically significant, but this may have been because of the small number in the SEN category.
Home variables
4.10 Parents' Occupations
The parents' occupations were originally classified into seven categories
as follows:
Professional
Managerial and administrative
Higher clerical, skilled craftsmen, technicians
Skilled manual workers, foremen
Semi-skilled, unskilled workers, labourers
Income provided by the state
Not known
The information was collected by the test administrators. However, for Mother's Occupation the information was available for only 39 pupils (15 per cent). Thus, results against this variable are not reported.
Information regarding Father's Occupation was available for 242 (95 per cent) of the children. However, since the numbers in each category were very uneven, the categories were collapsed for statistical purposes as follows:
1. Professional, managerial and administrative, higher clerical, skilled
craftsmen, technicians
2. Skilled manual workers, foremen
3. Semi-skilled, unskilled workers, labourers, income provided by the
state
Table 7 presents the results according to father's
occupation.
| Maltese | English | ||||
| Mean | SD | Mean | SD | N | |
| 1 | 34.8 | 4.6 | 32.1 | 5.8 | 55 |
| 2 | 33.7 | 5.3 | 29.9 | 5.9 | 97 |
| 3 | 32.8 | 6.3 | 29.3 | 6.7 | 90 |
| Total | 242 |
The differences on the Maltese test were not significant. However, a significant difference was found on the English test (one-way analysis of variance, F=3.69, p<0.05). From inspection of the mean scores it is clear that children whose fathers had middle-class jobs had a significantly higher average score than others. If replicated in the main survey, this finding might imply that working-class children's literacy in English was not progressing fast enough, and might need attention.
4.11 Parents' Levels of Education
Information on this variable was also collected by the test administrators.
Despite their reporting that this information was time-consuming to collect,
they were able to gather it for 232 mothers (93 per cent) and 238 fathers
(95 per cent).
The parents' levels of education were originally coded according to
the categories shown in Table 8, which also shows
the numbers of children according to parents' educational levels.
|
|
|
|
|
|
|
No formal schooling |
|
|
|
|
Primary School |
|
|
|
|
Secondary school |
|
|
|
|
Pre-university & Vocational |
|
|
|
|
Tertiary Education |
|
|
|
|
Not known |
|
|
There was a large difference between the mothers' and fathers' levels of education. Again, because of the uneven numbers, categories were collapsed for analysis as follows:
1. No formal schooling, Primary School
2. Secondary school, Pre-university & Vocational courses,
Tertiary Education
The results by mother's and father's levels of education are shown in Tables 9 and 10 respectively. In neither case were any of the differences statistically significant, though again this may have been because of the small numbers of mothers with more than primary education and of fathers with less.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
School variables
4.12 Geographical Region
The numbers of students in each region were too small to allow statistical
analyses to be conducted at this stage.
4.13 Number of Year 2 classes in school
The number of Year 2 classes in the school in 1997/98 was classified
into two bands:
(i) schools with 1 to 4 Year 2 classes
(ii) schools with 5 or more Year 2 classes.
There were seven schools in the first band and five in the second. The
first band ranged from 2 to 4 Year 2 classes while the second band ranged
from 5 to 7 Year 2 classes. Table 11 presents
the mean scores according to the number of Year 2 classes in the school.
The differences were not statistically significant. If replicated in the
main survey, this would seem to imply that both smaller and larger schools
are efficient in teaching literacy in both languages.
| Maltese | English |
| Category | Mean | SD | N (253) | Mean | SD | N (255) |
| 1 | 34.0 | 5.3 | 136 | 30.2 | 6.3 | 136 |
| 2 | 33.4 | 5.5 | 109 | 30.3 | 6.1 | 111 |
5. Summary, Implications and Recommendations
5.1 Summary
A pilot survey of the reading attainment in Maltese and English of
Maltese Year 2 pupils was carried out in June 1998.
The tests used were: for English, the Literacy Baseline within the Reading Progress Tests series published in Britain by Hodder & Stoughton; for Maltese, a specially-devised parallel version of that test.
Background data were collected on the pupils and schools involved.
The number of pupils tested in both languages was 250. They belonged to 12 Year 2 classes in 12 state primary schools, two in each geographical region of the Maltese islands.
The test administrators' ratings showed a high level of approval of the tests.
The technical reliability of both tests was satisfactory, though there was a ceiling effect on both.
The average score on both tests was high. The average score for the pupils' first language (Maltese) was higher than for their second language (English); but the difference was not large, and therefore most pupils were making good progress in literacy in both languages.
However, the tests also seemed capable of identifying pupils with literacy difficulties.
Girls' average score was higher than boys', in both languages. However, the difference was only statistically significant for Maltese (p<0.001).
Children who had had two years of preschool education had higher average scores than those who had had one year, in both languages.
Children whose fathers had middle-class jobs had a higher average score in English than other children (but not in Maltese).
First, and most important, the pilot study was largely successful.
In particular:
Also, as a pilot study should, this exercise identified a number
of problems to be solved for the proposed main survey:
1) Two out of the 13 test administrators adopted a non-standard
approach. Instead of administering the test in Maltese first to half their
class, and in English first to the other half, they gave it in one language
first to all their pupils. If this approach were adopted at all widely
in the main survey it would compromise the reliability of the results.
The necessity of uniform administration will need to be emphasised to test
administrators recruited to administer the survey.
2) Some background data proved difficult or even impossible to
obtain. The test administrators reported that information on parents' levels
of education was time-consuming to gather, because it was not routinely
recorded in pupils' Cumulative Record Cards. However, presumably by dint
of perseverance, this information was obtained for over 90 per cent of
parents. For the main survey it would seem worthwhile to continue
to gather this information.
On the other hand, it would not seem worthwhile for mother's occupation. Information on this was gathered for only a small number of mothers, and the effort required to gather it would not only be disproportionate to the result, but would also risk discovering that such a high proportion of mothers were 'looking after the home and family' that no statistical analysis would be possible for this variable anyway.
3) For several variables, the small numbers in some (or indeed all) categories made statistical analysis uninformative. This affected the following variables: age of pupils, Special Educational Needs, parents' levels of education, and geographical region. However, the exercise of carrying out the analyses was valuable in itself as preparation for the much larger task that the main survey will present - the number of pupils in the main survey will be at least 20 times as large as in this pilot. This will in turn mean that categories which were tiny here will become statistically feasible, and they have the potential to yield interesting results. Except for mother's occupation, therefore, all the background data categories used in this pilot should be carried forward.
5.3 Recommendations
First, and most important, the main survey should go ahead. This is
justified not only by the success of the pilot study, but also by the worthwhile
information that the survey would provide. The main survey should cover
all state primary 'A' and 'C' schools, and all Year 2 pupils (defined by
date of birth) in those schools. The number of pupils involved would be
about 5500.
This in turn implies all the practical steps that will have to be taken: printing of tests, instruction booklets and other instruments; recruitment and training of test administrators; test scoring; data checking, entry and analysis; feedback of individual pupils' results to their schools; publication of the national results; and the finance to support all of this.
Certain administrative recommendations were implied in the previous
section. In brief, these are that:
In all other respects the main survey should be organised in the
same way as the pilot survey, subject only to differences entailed by the
difference in scale. For instance, it might be necessary to recruit all
or most of the pilot test administrators as trainers of the much larger
number of administrators who would be needed.
Throughout this report, it has been made clear that the pilot study involved only state schools, and the main survey has so far also been discussed only in terms of such schools. However, the question of the possible involvement of private and Church schools should be raised. They could not be required to participate, but they might wish to, in order to check their pupils' attainment against the national picture. The cost of the survey for state schools would be met from public funds; if private and Church schools wished to participate, they should perhaps be charged for this. The charge would need to cover at least the direct costs, in terms of extra printing, training, and data entry, analysis and feedback, plus a contribution to the administrative costs.
Reference has been made (section 4.4) to the possible use of the tests as screening devices. That is, pupils whose scores are below a certain level could be identified as having weak literacy skills, and therefore as being at risk of literacy failure and as needing extra help. It is recommended that the results of the main survey should be used in this way; and in particular that children so identified in Spring 1999 should be given extra help during the Summer in order to boost their educational prospects in Year 3, and beyond.
Some essential advance measures for the main survey have already been
taken:
Finally, this report should be published and made widely available,
in order that consultation on the decisions needed for the main survey
can be as informed as possible.
Bartolo, P. (1988). Report on Maltese Word-Reading Test, Malta: Education Department.
Falzon, J. (1972). Maltese group Reading Test, Malta: Education Department.
Grech, D. (1997). Analizi u Klassifikazzjoni tat-Temi fit-Testi Maltin ghat-Tfal tal-Primarja. University of Malta: unpublished dissertation.
Martinelli, V. (1996). Maltese-language version of Clay's Concepts About Print Test Unpublished PhD Thesis. U.K.: University of Manchester.
Vincent, D., Crumpler, M. and de la Mare, M. (1996). Manual for Stage One of the Reading Progress Tests. Great Britain: Hodder & Stoughton Educational.
Reading Progress Tests Literacy Baseline. (1996). Great Britain: Hodder & Stoughton Educational.
Reading Progress Tests Literacy Baseline - Maltese version. (1997). Malta: Early Literacy Unit.
Dr Charles Mifsud, Project Leader, Faculty of Education, University of Malta
Ms Josephine Milton, Project Researcher, Faculty of Education, University of Malta
Dr Greg Brooks, Senior Research Officer at the National Foundation for Educational Research, Slough, England, and Consultant to the Project
The authors of this report gratefully acknowledge the good work of:
All the schools and pupils who took part in the study.
The test administrators:
Margaret Vella
Josephine Preca
Dawn Camilleri
Joe Buttigieg
Carmen Aquilina
Joe Vella Camilleri
Nicolina Cachia
Philip Grech
Anne Marie Ellul
Paul Borg
Silvia Sammut
Saviour Tabone
Felicia Saliba
Ms Mary Anne Spiteri, Education Officer, Education Division.
Mr Frank Fabri for help with creating the Maltese version of the test and translating the Teacher's Manual into Maltese.
Professors Saviour Chircop and Anton Buhagiar for their assistance with statistical analyses.
Funding Agencies:
National Commission for UNESCO (Malta)
Ministry of Education (Malta)
National Commission for Education (Malta)
Faculty of Education, University of Malta
APPENDICES
1. School Questionnaire
2. Pupil Data Form
3. Test Administrators' Questionnaire
