Data Issues

Waves 1 to 7
Data issues - Waves 1 to 7 – February 2019

9 Data issues in Wave 4

9.1 Instances of child but not parent participation

Ideally, it is expected that parents who provide consent to interview their children would agree to be interviewed themselves. However, this does not happen in 100% of cases. In Wave 3, there was only one case out of 8,718 home visits in which a parent was not interviewed but provided consent to interview a child. In Wave 4, there were five out of 8,405 cases where parents were not interviewed but agreed to their children being interviewed. The main reasons for parents to refuse a home interview but allow their child to be interviewed were lack of interest and time.

Parent 1's refusal to the home interview might lead to missing household information.4 If household information is not available this record is not included in the household file. As a result, in Wave 4, the household file hhgrk10 contains 4,164 records and main file lsacgrk10 contains 4,169 records. If data users intend to merge these datasets, they need to be aware that there is a mismatch between the datasets for five cases.

To help a data user identify cases with available household data the following variables were created: *hhresp for both cohorts. In variable names the asterisk refers to an age indicator (In Wave 4, d refers to the B cohort and f refers to the K cohort).

While in Waves 3 and 4 the discrepancy in child and parent participation is minor, it might increase in future waves due to: (1) changes to the interview procedure; or (2) children becoming more active participants/refusers in the study; or (3) increases in other activities of parents and children meaning fewer times when both are at home at the same time and available for interview.

Changes to the interview procedure

From Wave 4, interviewers were provided with two laptops and were able to conduct 'parallel interviewing'. The interview was split into two streams; all the Parent 1 (P1) questions were on laptop 1 and all the study child (SC) questions were on laptop 2. As a result, the interviewer had the flexibility to complete child and parent interviews either at the same time or at different times.

Children becoming active participants/refusers of the study

All cases where children were interviewed but parents were not belong to the K cohort in both Waves 3 and 4.

9.2 ACASI

The audio computer-assisted self-interview (ACASI) contains questions that are skipped if the study child has no mother and/or father figure in their life or does not attend school. As these circumstances were determined in the CATI component of the Parent 1 interview on laptop 1, they were not apparent in the interview on laptop 2 when the ACASI was conducted. Consequently, in order for the relevant questions to be skipped, prior to providing the laptop 2 to a study child to complete the ACASI, the interviewer was required to enter into the ACASI instrument whether there was 'Mum' and/or 'Dad' figures in the study child's life and whether the study child had been attending school. The interviewers were asked to use their knowledge of the family that they gained after completing the CATI component with the Parent 1.

When deciding if it would be appropriate to ask about a 'Mum' or 'Dad' the interviewers were asked to be sensitive towards the situation of the child as family structure can be complicated. The interviewers were instructed that if they were unsure whether there was a 'Mum' or 'Dad' in the child's life and/or whether a child had any contact with them to enter 'no father' or 'no mother' so as to not distress the child. For example, in the situation where the current Parent 1 and Parent 2 were not the biological parents, it was unclear as to who the child would be referring when asked about their mother or father. In just a few cases there was an interviewer's error and wrong information was rolled into the ACASI module.

In just a few cases there were inconsistencies between the household information and the interviewer's assessment of whether there was a mother in the child's life. As a result, there were 10 cases for which a mother was recorded in the household file but where all questions about the mother were skipped in the ACASI module. There were also 32 cases where all questions about the father in the ACASI module were skipped but a father was identified in the study child's life in the household file. To identify these problematic cases the following variables were created (asterisk refers to an age indicator):

  • *mumsk - a mother is identified in the household but questions about 'Mum' figure are skipped in ACASI module
  • *dadsk - a father is identified in the household but questions about 'Dad" figure are skipped in ACASI module
  • *schsk - child is in school but questions about school are skipped in ACASI module.

With regards to school attendance, 40 children who were in school (as identified in the education module) skipped all the questions about school in the ACASI module. This mismatch was mainly due to manual errors and one of the problems of the method used; that is, information about the child's education is provided by Parent 1 and stored on the laptop 1 and the study child completes ACASI separately on the laptop 2. This information being stored on different computers means that these instruments do not talk to each other.

9.3 Matrix Reasoning

Matrix Reasoning (MR) is a test from the Wechsler Intelligence Scale for Children, 4th edition (WISC-IV) (Wechsler, 2004) for ages 6-7, 8-9 and 10-11 years. This test of non-verbal intelligence presents a child with an incomplete set (later referred to as item) of pictures and requires the child to select the picture that completes the set from five different options. The instrument is comprised of 35 items of increasing difficulty.

Administration rules

According to the WISC-IV manual, the administration of Matrix Reasoning should follow a set of rules. We are not going to discuss all the rules in detail but rather focus on the rules crucial for our purposes.

Administration of the test should start at the age-specific start-point, which is indicated in the WISC-IV manual. Item 4 is the start-point for children aged 6-7 (B cohort) and Item 7 is the start-point for children aged 10-11 (K cohort).

Items prior to age-appropriate start-points are called reversal items. Reversal items are asked only if a child provides incorrect answers on the first or second start-point item. If a child answers incorrectly either of the first two items from the start-point, the interviewer asks the preceding items (reversal items) in reverse sequence until the child answers correctly two consecutive items and then goes back to the age-appropriate items and proceeds with the rest of the test. This is called reverse administration. For example, if a 6-year-old child answered correctly on Item 4 and incorrectly on Item 5, an interviewer should reverse to Item 3, then Item 2. If Item 3 or Item 2 is incorrect then Item 1 is administered. If Items 3 and 2 are correct, Item 1 is not administered. After administering reversal items, the interviewer goes back to Item 6 and proceeds with the rest of the test.

Scoring rules

The total raw score of MR is equal to the number of correct items starting from an age-appropriate start-point plus the total score on the reversal items. For items administered from the age-appropriate start-point a raw score of 1 is assigned for each correct answer.

For reversal items the following scoring rules are applied:

  • Rule 1 - Each reversal item gets a score of 1 if the reverse administration is not required (first two items from the start-point are answered correctly). For example, if a 6-year-old child answers correctly Items 4 and 5, the reversal Items 1, 2 and 3 are scored 1 each.
  • Rule 2 - Each reversal item gets a score of 1 if a child correctly answers two consecutive reversal items. For example, if a 6-year-old child answers correctly on reversal Items 3 and 2 and Item 1 is not administered or answers incorrectly on Item 3 and correctly on Items 2 and 1, the reversal Items 1, 2 and 3 are scored 1 each.
  • Rule 3 - Each correctly answered reversal item gets a score of 1 and each incorrectly answered reversal item gets a score of 0 if a child does not answer correctly on any two consecutive reversal items. For example, a 6-year-old child answers incorrectly on the reversal Items 3 and 1 but correctly on the reversal Item 2. Then, Items 3 and 1 are scored as 1 each and Item 2 is scored as 0.

Administration of MR in LSAC

Due to the technical difficulties in programming, the reverse administration was not implemented in the LSAC MR instrument; that is, if LSAC children answered either of the two items from the start-point incorrectly the reversal items were never administered. Table 14 shows a number of cases where the first two items from an age appropriate start-point were answered correctly and incorrectly for B and K cohorts.

Table 14: Frequencies of correct responses on the start-point items
  n %
B cohort
Item 4 and Item 5 are correct 3,964 95
Item 4 or Item 5 is incorrect 216 5
Total 4,180 100
K cohort
Item 7 and Item 8 are correct 3,908 95
Item 7 or Item 8 is incorrect 195 5
Total 4,103 100

It can be seen from Table 14 that 95% of children answered the first two items from the age-appropriate start-point correctly and did not require the reverse administration. The raw scoring for these children was based on Rule 1. The other 5% of children answered one of the first two administered items incorrectly and, therefore, required the reverse administration to identify which rule for scoring should be used - Rule 2 or Rule 3. Given that the reverse administration was not available, it was decided to assign all reversal items a raw score of 1 regardless of whether the first two administered items were answered correctly or not. As a consequence, some of the 5% of children might have had their MR scores overestimated. The following variable was created to identify these 216 cases in the B cohort and 195 cases in the K cohort:

A variable that was created to identify these 216 cases in the B cohort and 195 cases in the K cohort:

where * refers to appropriate age indicator.

The MR scores on Items 1-6 from previous waves are examined below.

K cohort

Out of 195 children from the K cohort who did not answer either one of two first items from the start-point at Wave 4, 185 children did MR at Wave 3. While the reverse administration was not implemented in Wave 3, all items were administered; that is, Item 1 was the first administered item. This allows us to cross-check how many children out of 185 gave two consecutive correct answers on Items 1, 2, 3, 4, 5 and 6. There were 179 who answered correctly either on all Items 1, 2, 3, 4, 5 and 6 or answered correctly on two consecutive items. In this instance, at Wave 3, they were assigned the maximum possible score. Assuming that cognitive ability of children remains stable over time, we would expect these children would obtain the maximum possible score for the first six items at Wave 4 too.

B cohort

In Wave 4, B cohort children were administered the MR test for the first time. However, in Wave 2, K cohort children did the MR test and they were the same age as the B cohort children in Wave 4. Therefore, the relative comparison could be made against the K cohort children of the same age. In Wave 2, there were 269 (6%) of the K cohort children who answered Items 4 or/and 5 incorrectly. Out of these 269 children, only 16 children did not answer correctly Items 3 and 2 or Items 2 and 1.

Therefore, based on the data from previous waves, we would expect only a very small number of children in either cohort to have their MR ability overestimated through the changes in administration and scoring.

4 If a Parent 1 completes CATI prior to the home interview, household information is not missing.