«MIST (Melbourne Initial Screening Test) Evaluation Studies February 1998 Part 1: Reliability and validity testing of the MIST Part 2: A negative ...»
All data from each child’s MIST and visual assessment was coded and entered into a computer statistical program (Statview) for analysis. The return referral form requested measurements of VA; a yes/no response for the diagnoses of amblyopia, refractive error, strabismus and other; a yes/no response for treatment, no treatment and review. The classification of a true or false referral was then determined by the Project Officer.
As VA of 6/9 is considered a pass for children of this age group,1 the criterion of 6/12 or worse was set as a true failure of the screening test. Any child diagnosed with a strabismus, or a VA difference of two lines or more21, 28, 30 was also considered to be a true failure. VA of 6/9 or better was classified as a false referral. Some children had VA recorded at a level between 6/9 and 6/12, usually due to such factors as the test optotypes available and the testing distance. These children were recorded as a pass result, as their acuity was better than 6/12.
There were a number of children for whom VA was not recorded, mostly because a reliable result was not achieved. The other information on the form was then used to classify the result for these This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) children, including the diagnosis according to the clinician and whether any treatment or review was recommended.
RESULTS Over a 3-year period, referral forms were returned for 3,854 children (2,015 males and 1,839 females). The mean age of the children was 44.3 months (SD 3.17), range 36 to 60 months. Of these children, 98% of parents or guardians gave signed consent for the clinical information to be returned. Eye health professionals returned 2,264 (59%) of these forms without any requests or reminders required.
Attempts were made to contact the remaining 1,590 children. Contact was made with 793 of these parents. Of the parents who were contacted, 535 were able to provide the specialist’s name.
Requests were then sent to the clinicians for information, of which 367 (69%) were returned.
Fifty-nine parents reported that their child had been seen, but were unable to recall the details of the clinician. It was found that 199 parents had not followed up the failed vision screening. This means that of all the children whose visual assessment results were known and reported by either the parent or the clinician, a total of 3,057 children, 199 (6.5%) were known not to have continued on for an eye examination. This is a measure of the known response of the number of children who were not taken for a full ocular assessment. The outcome is unknown for 797 children who were unable to be contacted, or who had not given their consent for further contact.
If the unlikely presumption was made that every one of these children did not follow up with an eye examination, then the maximum rate of non-attendance following vision screening failure would be 996 of the 3,854 children (26%).
The proportion of the population who attend the 3.5-year assessment varies around 49% to 52% each year. Over the 3-year period, the referral rate of children who failed the vision screening by the nurses was 6.5%.53 Reasons for referral after vision screening The vision screening protocol includes vision testing with the MIST and an observation of the ocular structures, including corneal reflections. Most of the children were referred because they had failed the MIST (3,395 children, 88%) but a number were referred for other reasons. A number of children actually passed the MIST, but were still referred (235 children, 6.1%).
Reasons were provided for some of these; behaviour whilst performing the test (45%), strabismus (11%), family history (10%), parent request (3.5%), pathology/observation (1.5%). The remaining 224 children (5.8%) were those not able to complete the MIST, with similar reasons given for referral.
The MIST referral procedure recommends an eye examination by an optometrist, ophthalmologist or orthoptist, either privately or through the public health system. The majority of children attended an optometrist (78%). Others attended an ophthalmologist (14%), an orthoptist (4%) or both (4%). The mean time lapse between the vision screening test and the eye examination was 6.3 weeks (SD 9.97), ranging from one to 135 weeks. Most children (88%) were assessed within three months.
Analysis of referral outcomes Collation of the returned referral forms allowed analysis of the level of true or false referrals arising from the MIST visual surveillance program. There were 2,623 forms returned from the specialists, providing the clinical information from the examination. Each result was graded into one of seven categories for classification as true or false referrals (see Table 9).
The results have been analysed in total, but are also considered in the light of whether the child actually failed the MIST or was referred for some other reason. It can be seen that the confirmed true positive rate was 42.3% of all the children referred (see Table 10). There was a further 2.8% who may be true referrals, those who received some form of treatment, but with a VA of 6/9 or with no VA obtained. Another finding of interest was the 3.2% of children for whom testing was a difficulty even with the clinician, those for whom a VA was not established. The incidence of children diagnosed with strabismus was 4.2%.
This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) Table 10 Referral outcomes for all children with an ocular report (N = 2,623)
For the 2,623 children with an ocular report, the positive predictive value (PPV) calculated for those children who had a confirmed reduced VA or strabismus was 42.3% (see Table 11). If the PPV was calculated only for those children who failed the MIST, a slightly higher value of 44.4% was found. If these children were combined with those who did not complete the test, which would mean the inclusion of all children who did not successfully complete the MIST, then the PPV was also a slightly higher value. It can be seen that the PPV calculated for those children who were referred even though passing the MIST was significantly lower than for any other children.
Visual acuity The MIST is designed as a pass-fail test only, not as a measure of threshold VA. In order to calculate the level of VA defects that have been detected by the screening test, the level of the worst eye for each child was noted. These results are shown in Table 12.
Treatment outcomes According to the returned clinicians’ forms, 30.8% of the children were prescribed some form of treatment, with another 47.9% receiving no treatment, but requested to return for review at a later stage.
DISCUSSION This study of the PPV of the MIST vision screening program is based on a 3-year period from April 1998 to March 2001, with 3,854 referral forms returned. This included approximately 65% of the children referred by the maternal and child health nurses according to Department of Human Services, figures.53 As this was an extra task requested it might be expected that there would not be a complete return rate, complicated by the large number of nurses, more than 550, who are distributed over more than 800 centres throughout 78 municipalities. The 6.5% referral rate by the nurses is within the expected range for the detection of amblyopia, which has an accepted incidence of less than 5%. Other nurse based screening programs have reported similar referral levels.34, 49 A number of children (6%) were referred even though they had passed the MIST, another 6% because they had not been able to complete the test. The major reason cited for referral in both these groups was the child’s behaviour while doing the test, with strabismus being suspected in 11% of the children who passed. The proportion of children who were untestable was consistent with other studies.13 It has been recommended that repeat testing reduces the rate of false referrals, and that all children should be retested prior to referral.11-13, 40, 54 This recommendation may have been confirmed by the reduced true positive rate in this group of children, however this must be balanced against the management and time constraints of the nurses in the screening situation. This group of children also had a higher rate of non-completion of the testing procedures reported by the eye health professionals.
Of interest were those parents who did not follow up the vision screening failure with a full ocular assessment for their child. The design of this study did not allow this figure to be truly determined, but could only be calculated for those parents who were contactable. The rate of known non-follow-up was 6.5%, but it may have been greater if all children could have been traced. This figure is lower than that reported by other studies, which varied from 35% to 9%.54, 55, The follow-up time for the children in this study was also good, with more than half the children being seen within one month and the majority (88%) within three months of the vision screening.
Other authors have outlined the major reasons for not following up on the screening results.57 The screening process issues have been stated as lack of confidence in the process, lack of information about the screening, and lack of communication. Other factors include confusion regarding the choice of eye professional, difficulty in scheduling appointments, and the cost of This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) assessment and glasses, if these are required. One criteria for a successful screening program includes effective, available treatment.9, 34, 58 It appears from this current study that most of these factors must be adequate in the maternal and child health program and in the availability of eye health professionals for assessment. The fact that the majority of children attended an optometrist is understandable given the health care system, where optometric assessments are bulk-billed and require no medical referral. Some parents who had not followed up the screening test said they would do so when the child was ready for school, or when the parent next had their appointment.
There may be some need to further emphasise the importance of the timing of assessment and treatment for amblyopia, though in general this appeared to have been done.
The Positive Predictive Value The PPV of the MIST as a vision screening tool for those children who failed the test was 44.4%, with 95% confidence limits of 42.4% to 46.4%. This value is similar to that of other studies which have nurses or other health professionals as the primary screening personnel, ranging from 25% to 69%,21, 24, 34, 44-49 with several being in the range of 40% to 50%.21, 44, 45, 47, 49 A comparison with other studies is difficult as not all were based on a defined standard of VA, acuity difference or strabismus, but were based on the clinicians’ reports of whether a problem was diagnosed.
This allows clinical bias to alter the criteria of definition of a ‘problem’ and so may vary the rate of true or false outcomes. However, it can be seen that even with the stricter criteria, the PPV of the MIST is within the range of that cited by other screening programs.
Previous studies have shown that experienced eye health professionals may provide more accurate referrals from vision screening,7, 8 but this must be balanced against what may be seen as an ineffective screening program that relies on specialists providing screening services. This would mean that screening personnel would cost more and would involve only one aspect of the child’s general development, so would require multiple screening programs. The implementation of the MIST as the vision screening tool by maternal and child health nurses means that the vision test is part of a general assessment and so the results can be viewed in the context of the child’s whole development. As part of the 3.5-year assessment the MIST does not add to the cost of the process, except for a slight increase in the time required.
A PPV of 44.4% means an over-referral rate of approximately one in two children being referred unnecessarily.
De Becker et al21 stated that this may be considered acceptable with a condition of such low prevalence as amblyopia. This study has shown the MIST to have an acceptable PPV, with general acceptance by the nurses and a good follow-up rate by parents, all contributing to the community awareness. In terms of the economic cost to the community, there is minimal cost involved for the nurses’ assessment, but the costs of over-referral will be distributed to the general health care cost. The emotional costs to parents brought about by a failure on the vision screening test and the necessity to attend for an ocular examination also needs to be considered.
Parents need to understand the importance of following through with the process, without raising unnecessary anxiety.
The level of VA documented by the clinical assessment was of interest. Of the 2,537 children with a recorded visual acuity, 6% had 6/36 vision or less in at least one eye, another 17% had less than 6/18, significantly reduced levels of VA, which were detected by the vision screening program.
This study has found that the PPV of the MIST vision screening test was 44.4%, with 95% confidence limits of 42.4% to 46.4%, comparable to other vision screening programs which have other non-eye care professionals as the primary screening personnel.21, 24, 34, 44-49 This value means that for each child referred correctly, there is another child who may have been referred unnecessarily. However, given the less than 5% prevalence of amblyopia in children of this age group, this over-referral rate would be considered acceptable.21 This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) The negative predictive value was found to be 97.5%. In any vision screening program, a balance must be achieved between the positive and negative predictive values, between the likelihood of a false referral and the importance of obtaining a full examination to determine whether there is a problem.