«MIST (Melbourne Initial Screening Test) Evaluation Studies February 1998 Part 1: Reliability and validity testing of the MIST Part 2: A negative ...»
With stereoacuity, six children did not achieve the pass level of 140 seconds of arc. If these children are included with those unable to do the test, then this meant that 5% of the children were not able to complete testing of stereoacuity, similar to that of previous studies.10, 38 Of these six children, five were orthophoric, with normal VA, prism vergences and refraction.
On reviewing the results to each test, five children were considered to have failed the gold standard assessment. The details of these children are shown in Table 8.
One child (#162) was found to have a small intermittent esotropia, evident only after continued dissociation and a stereoacuity of 400 seconds of arc, consequently failing the gold standard on two criteria. Results to other tests were normal, with good prism vergences, VA and only a small hypermetropic spherical refractive error, all indicating good control of the deviation, so this would not be considered a significant concern. On follow-up assessment at a later date that there was no sign of the esotropia.
With VA testing only one child failed the gold standard assessment. This child (#115) failed the VA level in the left eye with both the chart and single optotypes, had a VA difference of greater than two lines and an astigmatic refractive error of 1.50 dioptres in the left eye, so failed the gold standard on three criteria.
Three other children failed the gold standard for the criterion of astigmatic refractive error (#6, #19, #158). Each of these passed the VA standard, with only one line or less difference.
Negative predictive value Of the 201 children tested who had all passed the vision screening, five failed the gold standard assessment. This resulted in a negative predictive value of 97.5%, with 95% confidence limits of 95.4% to 99.7%, for the MIST vision screening program.
This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) DISCUSSION The NPV of the MIST vision screening program of 97.5%, with 95% confidence limits of 95.4% to 99.7%, is similar to previous values reported. De Becker at al21 with a gold standard assessment of 157 children, reported a value of 93% for the detection of any ocular problem, but a value of 98.7% (95% confidence limits of 95.4% to 99.85%) for amblyopia, strabismus and high refractive errors, which are the usual target conditions of a vision screening program. The Enhanced Vision Screening Program21 has referral criteria of VA less than 6/9 or stereoacuity less than 200 seconds of arc. Kennedy et al,23 with a gold standard assessment of 241 children, reported a value of 94% for their screening program, though it must be noted that the VA referral criteria was less than 6/12, which may be considered rather low for kindergarten children and so may assist in explaining the lower value. Enzenauer22 reported a NPV of 97% for public health screeners over a wider age group. Marsh-Tootle,24 using single optotype E-test and De Becker et al’s21 criteria, reported a NPV of 77.8%. The children in this study were younger than De Becker’s, which may explain the lower NPV. None of the false negatives were considered to require treatment, merely increased surveillance. Newman,25 in a retrospective study testing linear acuity two years after single optotype orthoptic preschool screening reported a NPV of 99.6% (95% CI, 98.7% to 99.9%) for the acuity test alone, but increased to 100% when other tests of ocular alignment were included.
De Becker et al21 commented upon the NPV for vision threatening conditions, calculating this at 97.6%, with four of their children classed as having potentially vision threatening disorders and the other seven with minor ocular problems. In the present study, one of the five children would be classed as having amblyopia on the first assessment, with a VA difference of more than two lines. Another had an intermittent strabismus, with normal visual function. The remaining three children with an astigmatic refractive error all had VA results within the normal range for either the chart or single letter test, with a VA difference of no more than one line. The astigmatic error was only 1.50 dioptres in each case, so failing the gold standard according to De Becker et al’s criteria.21 However one other study set this level at greater than 1.50 dioptres, in which case these three children would have passed, giving a NPV of 99%.
The level of VA achieved by the children in this study was comparable to that reported in the literature. All of the 201 children were able to perform the letter matching task, a result similar to other studies.13, 27 If the complexity of the task was considered, it was found that 80% of the children in this study could satisfactorily complete the linear test, similar to other reports.12, 13 Considering the level of VA, 3/3.8 was achieved by 60% of the children on the LogMAR letter matching chart, with 81% achieving 3/4.8, results similar to those reported using the SSAS linear test.28 Methodologically, one bias introduced by the use of volunteer participants is that many parents participated as there was a known family history of ocular problems and so wanted a vision assessment. This may have meant that the sample contained a higher incidence of children with a family history.
The examiners were aware that each of these children had passed the screening test, so some examiner bias towards expecting normal results may have been possible, as discussed by De Becker et al.21 As the aim, as emphasised in the assessment protocol, was to determine whether any visual problems had been undetected by the screening, and so perform a gold standard examination to determine this, it was felt that this was not a concern. The gold standard assessment, which involved a combination of objective and subjective tests and two different examiners with measurements of each aspect of ocular and visual function, would be expected to detect any problem, regardless of bias. The issue of the use of different testers for the assessment of each of these children must be considered. The importance of the performance of all tests according to standard procedures and the data sheet protocol was discussed prior to the commencement of testing in order to minimise any effects. The comparison of each child’s test This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) results against the gold standard criteria was done at a later stage by the principal researcher, not by the clinicians.
CONCLUSION This study has found that the negative predictive value of the MIST vision screening program was 97.5%, a similar value to those previously reported.21-23 This means that there is a risk that 2.5% of children who pass the MIST 3.5-year old vision screening program may actually have either amblyopia, strabismus or refractive error. Any screening program must maintain a balance between the positive and negative predictive values. It must be noted that any attempt to increase the negative predictive value of the MIST would mean a higher incidence of false referrals.
The importance of this finding must be part of any vision screening protocol and the information provided to parents. It must be emphasised that even though a child has passed the screening test, there is a still a very low risk that a potentially vision threatening problem may have been missed.
Both screeners and parents must be aware of this when vision screening is performed.
This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) Part 3: A positive predictive study of the MIST
INTRODUCTIONThe positive predictive value (PPV) is a measure of the proportion of the population which fails the screening test and is actually found to have the disorder confirmed on diagnostic testing,34 the true referrals. It must be noted that the prevalence of a condition has an effect on the false positive rate. A condition with low prevalence, even with high sensitivity and specificity, may still have a relatively large number of false positives.39 It has been suggested that screening programs conducted by orthoptists, or experienced eye health care professionals, have higher sensitivity and specificity than those by other personnel.40, Orthoptist-based screening programs have reported a PPV from 74% to 94%,40, 42, 43 with nurses or other health personnel from 25% to 69%, but generally around 40% to 50%.21, 22, 24, 34, 44-49 Some studies compare referral rates, reporting orthoptists as referring less children, 4.2% to 5.1%,40, 49 compared to 6.5% to 8.6% by nurses.34, 49 The total efficiency of a screening program needs to be considered. Though orthoptists may provide more accurate referrals from vision screening, this needs to be offset by the advantage of the more general developmental assessment that can be provided by a community based nurse or health worker.
The efficacy of a screening program is dependent upon the attendance rate.50 The uptake for preschool vision screening programs has been reported, varying from 20% to 99.5%.25, 40, 44, 46, 51, 52 The MIST program generally attracts around 50% of the children in this age group.53 The attendance rate at school entry screening would be a much higher level than this. However, the major concern with the detection of amblyopia is that vision screening at school entry may be too late for effective treatment to be implemented, particularly now that children may be turning six or seven years of age at the time of school screening.51 A further issue to consider with any vision screening program is the follow-up by parents when a child fails the screening test, the need to obtain a specialist ocular assessment. Follow-up rates have been reported from 61% to 65%.45, 54, 55 Over-referral is a problem for the acceptance of any screening program. If it is noted that there are too many children referred from a vision screening who do not have a problem, then this may be perceived by both the parents and the eye health professionals that the test is worthless.56 Over-referral results in a cost to either the individual or the community, though this must be balanced against the cost of undetected visual problems if there was no screening program. It must also be noted that parents are good observers of their children and ‘are rarely wrong when they think that there is a problem with their child’s eyes or vision’.56 In this case any child who passes a vision screening test, but their parents are concerned, is likely to be referred on for full assessment. This is compounded by the tendency to refer if there is any doubt about the child’s visual function, as a screening test is not able to exclude all visual problems.
The problem with determining the PPV of any program is that the diagnosis of true or false referral is generally not set against a gold standard but is reliant upon clinical reports. These may be received from any number of eye health professionals whom the children have attended after referral from the vision screening program. This is particularly noted by De Becker et al21 in their negative predictive study of a vision screening program. When a sample of children who had failed the screening were given a gold standard ocular examination, the positive predictive value was found to be much less than what was defined by clinical reports, reducing to 50% from an originally cited 72%.34 The criteria for classifying results into true or false positive outcomes needs to be set independently of the clinicians, as clinical bias will result in different interpretations of the screening outcome.
This document is managed by the Department of Education and Early Childhood Development, Victoria (as of 27 August 2007) METHOD This stage of the evaluation project aimed to analyse the results of the ocular examination of all children who were referred to an eye health professional following testing with the MIST, to assess the level of true or false positive results. Ethics approval was granted by the Human Ethics Committee, La Trobe University and the Department of Human Services Ethics Committee.
Procedures The MIST kits were supplied to all Maternal and Child Health Centres in Victoria. During the training sessions the evaluation projects were explained to the nurses and their assistance requested. At the time of implementation of the program, information was sent to ophthalmologists, optometrists and orthoptists advising them of the program and requesting assistance in the return of the forms. A triplicate copy pad of referral forms was provided with the MIST kits.
For any child who failed the screening test the nurse completed the referral form, including the child’s identifying details, the nurse’s details and the MIST score. A section was available for the addition of any further comments or observations of appearance or visual behaviour. One copy was returned by the nurse to the Project Officer. Two copies were then given to the parent. The referral process included the explanation to parents of the MIST results, and the need to obtain a full ocular examination to determine whether this was a true result and if so, the reasons for the reduced vision.
The next stage of the process for the parent and child was to obtain an appointment for an eye examination. Upon completion, the clinician was requested to complete Section 2 of the form and return one copy to the Project Officer. This included the clinician’s details, the assessment results, diagnosis and any action taken. Parents gave signed consent for this information to be passed on.
The researchers then matched the nurses’ referrals with the returned specialist forms. As copies of the referral form were expected to be received from nurses for all children who had failed the MIST during the 3-year time period, this was the basis of the sample of children. If no specialist form had been received, those parents who had given consent were contacted. For those parents who could be contacted, the project was explained, the parents were reminded about the screening test, and were asked whether they had yet attended a vision assessment. If the clinician’s details were known, then a form was sent requesting the results. If the clinician’s details were not recalled, then an attempt was made to assess the outcome of the examination by asking the parents about the result and this information was added to the form, annotated as parent information.