7 July 2010

Critique of a meta-analysis by Wax and colleagues which has claimed that

there is a three-times greater risk of neonatal death among babies without
congenital anomalies planned to be born at home

Gill Gyte, Trainer in Research Methodology, NCT; Mary Newburn, Head of Research
and Information, NCT; Alison Macfarlane, Professor of Perinatal Health, Department
of Midwifery and Child Health, City University London.

Summary

Along with other previous researchers, Wax and colleagues found no difference in
the safety of planned home birth and planned hospital birth for women without known
risk factors when comparing perinatal mortality rates, the usual measure of safety.
They found a two-fold increase risk for all planned home birth babies in what they
called neonatal mortality, rising to a three-fold risk for those without congenital
anomalies. Is there really evidence that there is a three-times greater risk of neonatal
death among ‘non-anomalous’ babies planned to be born at home, based on the
Wax et al meta-analysis (2010)? In summary, NCT has found that this study has
serious methodological limitations, including:

• Insufficient detail about the assessment of the quality of the primary research
papers identified by the authors and their specific reasons for inclusion or
exclusion of each study.

• Lack of information about the included studies, and the specific data contributed by
each one to the outcomes under investigation. Scrutiny of the primary research
papers has led to somewhat different numbers being identified. (A ‘forest plot’
should have been included.)

• Lack of clarity and consistency about the definition of neonatal mortality in each of
the included studies, including whether stillbirth data were included.

• The small size of the sub-group contributing to the calculation of comparative risk
of neonatal death for planned home birth and planned hospital birth. It has been
suggested that 200-400 adverse events are needed in order to be confident of
avoiding a systematic error of insufficient data, but there were just 64 neonatal
deaths reported by Wax for all neonatal mortality and 37 for non-analogous
neonatal mortality.

• A number of the non-randomised studies included in the meta-analysis were not

matched for confounding risk factors nor had adjustments been made to data
afterwards, and some included women at increased risk of complications.

On the basis of these limitations, it is completely unjustifiable to claim that ‘Less

medical intervention during planned home birth is associated with a tripling of the
neonatal mortality rate’. On the basis of the poor quality data of their data, the
authors should not have reached this conclusion. Furthermore, the editors of the
American Journal of Obstetrics and Gynecology should not have accepted the paper
for publication without major modifications.

Reporting

The press release from the American Journal of Obstetrics and Gynecology
emphasised the size of the review as including ‘a total of 342,056 planned home
births and 207,551 planned hospital births’. There were, however, around only 5% of
this number of births in the smaller group of studies for which a measure of ‘neonatal
mortality’ was identified (16,500 planned home births in total, and 15,633 ‘non-
anomalous’ planned home births). We assume that ‘non-anomalous’ refers to babies
without congenital malformations.

The reason that the neonatal mortality rate calculated by Wax and colleagues relates
to just 15,633 planned non-anomalous births is that most of the research studies
included in the meta-analysis had not used that as their measure of safety. Most had
used the usual, more relevant measure, the perinatal death rate, which includes
stillbirths as well as early neonatal deaths. The perinatal mortality rate was reported
by Wax for over 330,000 planned home births. Based on this measure of safety,
there was no difference between planned home and planned hospital births.

Critical appraisal of the methodology

Gill Gyte reviewed and critically appraised the meta-analysis on the safety of home
birth by Wax and colleagues and also went back to identify the design of the primary
studies and the deaths reported in them. Mary Newburn reviewed some of the
papers. Alison Macfarlane peer reviewed the deaths reported and the critical
appraisal.

There are a number of methodological weaknesses in the meta-analysis:

1. The authors’ inclusion criteria were: ‘studies in developed Western countries,

published in English language peer reviewed literature, maternal and newborn
outcomes were analysed by planned delivery location, and data were presented
in a 2x2 table. Manuscripts were evaluated for quality using a published
instrument (Zaza 2000)’. The authors retrieved 47 studies for detailed analysis
and included 12 in the final meta-analysis. However, they do not identify the 35
excluded studies or say why each one was excluded. As a result, we do not know
why they excluded studies like Johnson and Daviss 2007, Bastian 1996;
NRPMSCG 1996). Why studies are excluded is critical information that any well
conducted systematic review or meta-analysis would provide. A study from the
UK by Chamberlain in 1996 was excluded as it was not published in a peer
reviewed journal, yet it uses better methodology than some of the studies that
Wax included.

2. Wax et al do not say which of the 12 studies supplied data for any of the reported
outcomes in Table 3. Again this is very poor practice.

3. The authors should have provided a forest plot for the main outcomes they were
reporting of PNM and NNM so the specific data from individual studies is visible
to readers. Instead they focused on several pages of comments providing their
opinion of various aspects of this topic.
4. Wax and colleagues defined:

• Perinatal death as ‘stillbirth of at least 20 weeks or 500g or death of liveborn

within 28 days of birth’.

• Neonatal death as ‘death of a liveborn within 28 days of delivery’.

However, not all the included studies used the same definitions and some gave
no definition of perinatal or neonatal deaths. Normally, researchers would contact
the authors to be sure of the specific data they were including in their meta-
analyses. As a minimum, Wax et al should have explained to readers the
variations in the definitions of the data they included.

5. The authors say the following studies referenced 4, 7, 10, 13, 15 and 17 are
included in the neonatal mortality (NNM) rate. (This information can be found on
the third page of the meta-analysis, third half way down the left-hand column)
saying: “Importantly, these latter observations were consistent across all studies
examining neonatal mortality, regardless of the covered time period, 4,7,10,13,15,17. .
This lists six studies yet Table 3 states there were seven studies on NNM and six
on PNM. Careful scrutiny of the primary research papers that have been included
in the meta-analysis suggests that there are eight studies that contribute to PNM
and - with some overlap and some differences - eight studies that contribute to
NNM, as follows. However, missing data and absence of clear definitions in some
papers means that further work is needed to ascertain for sure which studies
contribute data for each of the two different outcomes measures, and which
included or excluded any babies with congenital anomalies.

6. Extracting data from the original studies, we have found somewhat different
numbers. These need to be checked with the authors of the original studies, to
minimise any uncertainty about coding of data. We found PNM excluding
congenital anomalies outcome should be based on 367 deaths out of 517,107
women NNM excluding congenital malformations based on 62 deaths out of
62,047 women.

7. It is important that there are a sufficient number of ‘events’ to avoid the risk of
systematic error; the GRADE assessment tool (2004) suggests this should be
200-400 events. The PNM data meets the criteria of needing more than 300
events to be confident of avoiding a systematic error due to insufficient data,
Several other outcomes, including the neonatal mortality rate, fall far short of this
threshold.

8. As well as insufficient data, all studies providing data on PNM and NNM were
non-randomised; many did not matched adequately for confounding risk factors
nor had adjustments made to data afterwards. One study included twins, preterm
births and post-term births in the home birth group but not in the planned hospital
births group (Lindgren 2008 7).

9. Pang 2002 adds most weight to the meta-analysis on overall NNM with 38 baby
deaths in 16,726 women and for NNM excluding congenital anomalies 19 deaths
in 16,726 women. This study was retrospective and was based on birth registry
data of home births. As ‘planned home births’ were not recorded only actual
home births, the authors defined planned home births as ‘those singleton
newborns of at least 34 weeks gestation who were delivered at home and who
 had a midwife, nurse, or physician listed as either the birth attendant or certifier
on the birth certificate’.1 The authors continued, ‘In addition, singleton newborns
with gestational age of at least 34 weeks who were born after transfer from home
to a medical facility were considered to be planned home births if their birth
certificate indicated that delivery was initially attempted at home by a health care
professional’. The potential for this study to misclassify unplanned home births as
planned home births seems considerable.

10. Wax et al provide no information on study quality, which is critical when

assessing non-randomised studies. They say they undertook quality assessment
and did a sensitivity analysis, but it is a significant weakness that they do not
report their quality assessment for each of the included studies nor which studies
they included in their sensitivity analysis.

11. The authors say that they assessed heterogeneity using the Breslow-Day test but
do not reference it. Wax et al claim their data on NNM are robust because they
are not heterogeneous but they fail to report in the written text that they are
based on small numbers, which are too small to be confident of the findings,
whereas PNM data are based on large enough numbers and show no significant
difference.

1
There is a further note which is difficult to interpret: ‘(if attendant is not listed on the
birth certificate, then the person listed as the certifier attended the delivery).’
Conclusions

In summary, there were a range of serious methodological limitations in this study.

These included:

• Insufficient detail about located primary research papers, the specific reasons for
inclusion or exclusion of each study and lack of the assessment of the quality of
the included studies. Lack of information on the specific studies that contributed to
the various outcomes, and lack of information on the data contributed by each
study to the outcomes under investigation. Scrutiny of the primary research papers
has led to somewhat different numbers being identified. (A ‘forest plot’ should have
been included.)

• Lack of clarity about the definition of neonatal mortality in each of the included
studies, including whether stillbirth data were included.

• The small size of the sub-group contributing to the calculation of comparative risk
of neonatal death for planned home birth and planned hospital birth. It has been
suggested that 200-400 events are required to be confident of avoiding a
systematic error of insufficient data (GRADE, 2004), but there were just 64
neonatal deaths reported by Wax for all neonatal mortality and 37 for non-
analogous neonatal mortality.

• A number of the non-randomised studies included in the meta-analysis were not

matched for confounding risk factors nor had adjustments made to data
afterwards.

On the basis of these limitations, it is completely unjustifiable to claim that “Less

medical intervention during planned home birth is associated with a tripling of the
neonatal mortality rate “. The authors should not have reached this conclusion based
on the quality of their data and the editors of the American Journal of Obstetrics and
Gynecology should not have accepted the paper for publication in its current form.

Actions

Gill Gyte will write to:

• Janssen 2009 12 – to ask for data. They provided information on OR but not
specific data. They did report there were no deaths between eight and 28
days.

• Pang 2002 15 to ask for their definition of neonatal death and to see if they
collected stillbirth data.

• De Jonge 2009 16 to ask if they collected data on neonatal deaths between

seven and 28 days.
References

GRADE Working Group. Grading quality of evidence and strength of

recommendations. BMJ 2004; 328: 1490-1494.

Mueller PS et al. Ethical issues in stopping randomized trials early because of

apparent benefit. Annals of Internal Medicine 2007; 146(12):878-82.

Wax JR et al. Maternal and newborn outcomes in planned home birth vs planned
hospital births: a metanalaysis. AJOG 2010;203:x.ex-x.ex.

Contact details

To discuss any aspect of this paper or to seek permission to quote or reprint, please
contact Mary Newburn, Head of Research and Information, NCT:

Telephone: 0208 752 2334

Email: m_newburn@nct.org.uk
About NCT

NCT is here to support parents. Becoming a parent is a life-changing experience,

and everyone wants to get it right. But what is right? These days, there is a
bewildering amount of advice and information out there, about everything to do with
pregnancy, birth and bringing up a child.

Helping parents decide

We’re an independent UK charity, funded by donations and course fees. We don’t
push a particular view; we believe in giving parents unbiased, accurate information
based on evidence, in a supportive, non-judgemental atmosphere, so that they can
decide what’s right for them and their family.

We offer practical and emotional support through our network of 300 local branches,
helplines, classes and counsellors. Our antenatal teachers, breastfeeding
counsellors and postnatal leaders are sympathetic experts. They set out the choices
and encourage questions so that parents can explore all the possibilities and make
their own decisions.

Sharing the experience

Becoming a parent can be daunting, especially if someone is the first in their peer
group. There’s nothing like hearing it from people who have done it before. NCT’s
local support groups are run by parents for parents. They can be a lifeline in the early
days, as well as a bit of fun through shared experiences, and continue to be a
valuable support as children get older.

A trusted voice for parents

Improvements to maternity care, better services, greater support – for over 50 years,
NCT, as the UK’s leading parenting charity, has campaigned on behalf of parents on
the issues that matter to them. We are independent experts, known for basing our
work on research evidence and parents’ views. Parents trust us to speak for them in
advising the government, academia and other bodies, and to influence the research
agenda on their behalf.

We believe that everyone can and should feel confident and supported in their role
as a parent. We’re here to help parents – and our society – to get there.
NCT

Alexandra House
Oldham Terrace
London W3 6NH

Pregnancy and birth line: 0300 330 0772

Postnatal line: 0300 330 0773
Breastfeeding line: 0300 330 0771

Enquiries: 0300 330 0770

Fax: 0844 243 6001
Email: advice@nct.org.uk

Registered Charity No: 801395

Registered Company No: 2370573
Registered in England and Wales
Registered address: Alexandra House, Oldham Terrace, Acton, London W3 6NH

© NCT maintains copyright on all content included in this paper. Reprint or

reproduction is not permitted without prior consent.