In a recent article by Hartley et al titled The Landscape of International Event-based Biosurveillance, the authors highlight a series of meetings held by the Global Health Security Action Group (GHSAG) of the Global Health Security Initiative that have culminated in the development of "a biosurveillance capability that integrates and leverages component systems from member nations". I presented Project Argus at the February 2008 Risk Management and Communications Working Group (RMCWG) meeting in Luxembourg that "focused on identifying, within the context of CBRN hazards and risks, the capacities and input needs of existing IT systems working currently in the early detection of public health threats." At that meeting, the efforts of BioCaster, GPHIN, HEDIS, HealthMap, MedISys, ProMED, PULS, and Argus were presented in with a discussion of comparative features.
The unfortunate outcome of those discussions was, as is often the case in biosurveillance, an overemphasis on information technology without an appreciation of the human element. I emphasized repeatedly throughout those discussions the need for a professional discipline of operators who conduct warning analysis. As is apparent in the published review, this critical point was lost.
The reason for this emphasis was the well-known warning failures during the emergence of SARS in 2002 (see below attachment). Both GPHIN and ProMED were functional at that time. GPHIN reported they caught a single media article with their web harvesting engine, however the article remained in the native Chinese language untranslated for some time before it was noticed. ProMED also claimed reporting an early warning signature on February 10th with the well known post by Stephen Cunion highlighting rumor of unusual respiratory disease was killing people. But reports of concern about unusual respiratory disease did appear in publicly available Chinese media for months before a critical threshold of international awareness was reached to declare a warning and enable response. De facto, public health engagement did not occur until atypical pneumonia was documented in Hong Kong in mid-March. Unfortunately, it was too late before translocation into the air traffic grid. This failure to warn of an international public health emergency was nearly identical in event sequence to the pandemics of 1957 and 1968.
While at Argus, we ourselves experienced a failure in warning of vaccine-drifted A/H3N2 influenza:
This past influenza season, the Argus team issued nearly 3,000 event reports across 128 countries and 27 languages, which included 181 Advisories, 58 Watches, and 38 Warnings. We identified hundreds of reports of an H3N2 influenza virus that had possibly drifted away from the current vaccine strain of H3N2 beginning eight months ago in a multitude of countries and collaboratively worked with CDC to track this important finding. The value of this information was validated when the World Health Organization and its
partners recommended a change in the southern hemisphere influenza vaccine to include an updated H3N2 strain.
In the 2007-2008 influenza season, we did not have a vaccine for this drifted strain of A/H3N2, which we reported in China and then conveyed by email and phone to CDC six weeks prior to the vaccine composition meeting in February 2007. Eighty-three children in thirty-three states died that season in the United States, 90% of whom were not vaccinated. It begs the question of whether these parents would have changed their decision to vaccinate if a public warning was issued months beforehand. It is thought an imprecisely matched influenza vaccine may still protect from severe clinical outcomes.
Failure of signal recognition and warning process again was noted during the emergence of pandemic H1N1 in 2009. HealthMap and ProMED claimed to have reported the signal, a Spanish language media article harvested from the Internet. Unfortunately this was amidst hundreds of other undifferentiated reports in their system at the time. It is also known that Argus similarly presented reports of unusual respiratory disease stemming from the same sources, however the analysts did not take steps to warn the appropriate authorities. The finer distinction of an automated engine catching a raw untranslated, unanalyzed report versus 1) a human being recognizing the significance of that report and 2) acting to communicate a high level warning was missed, as it was during the emergence of SARS. These failures in warning deserve scrutiny of human process inasmuch as other intelligence failures such as Pearl Harbor and 9/11. Although the outcomes of those failures are vastly different, the reasons for the failures are quite similar. It begs the question of whether timely warning will actually be elicited when a biological threat of greater magnitude is encountered. It was our experience during the warning process of the pandemic that an astute and experienced operational biosurveillance analyst trumps any automated approach, as has been long recognized by the syndromic surveillance community.
During my time with the National Biosurveillance Integration Center (NBIC) at the Department of Homeland Security, we were presented with the raw outputs of federally funded programs such as BioSense and BioWatch. One of the key problems we identified immediately was lack of a professional discipline within these detection subsystems whereby a human being was able to explain the meaning of the information. Without that in place, we found these outputs to be nearly useless operationally. Indeed, the most useful information we noted was during BioWatch inter-agency teleconferences with local authorities who well understood their environmental baselines and could express their experienced opinion as to the meaning of the signal. In the end, experienced humans were required at the detection subsystem and integration levels to properly assess and issue advisories that were operationally relevant.
Hartley et al also highlighted "the GHSAG event-based surveillance systems, which use the media as the primary source of information, form a unique part of the landscape of international biosurveillance". Although a very important source, media alone is insufficient in operational biosurveillance. Timely and operationally relevant ground truth is essential to validate media signals, as we found during the pandemic and certainly during our support of Haiti. This remains a critical missing component of nearly every one of the GHSAG systems and again highlighted during the warning failures of the 2009 pandemic. A global network of trust with deep lines of key expertise associated with critical infrastructures in public health, agriculture, medicine, and other civil infrastructures and communities is the platinum standard in operational biosurveillance.
Several months before the pandemic, we briefed HHS/ASPR about the need for ground truth, inclusion of domestic (US) biosurveillance, and a National Weather Service-like warning capability for the country. Aside from the skepticism encountered, they provided an honest, but telling comment: "Even if we had the warning beforehand, what would we do with it?" This highlights another missed opportunity for the GHSAG's examination of biosurveillance: the interface between warning and response. Knowing how warning information will be used by stakeholders is crucial to understanding the relative merit of the "system of systems".
The conclusion of the GHSAG after the February 2008 meeting was disappointing: “cooperation at all levels, between systems, between systems and users, and users amongst themselves. Such cooperation should be considered at the level of the collection of data, at the level of data analysis of the data available and the subsequent sharing of the relevant information through a common restricted platform.” Here the GHSAG again failed to understand the human element embodied by the sociology of warning. They propose a "system of systems" without understanding how crisis information management systems function. One of the key lessons we found during the pandemic was restrictions to information flow compromised our ability to properly assess and issue warning of key changes in the evolution of the crisis. CDC's lack of access to all available information, including that produced by integrating the input of multiple medical disciplines, resulted in delays in understanding and conveying understanding of issues such as adverse social and clinical outcomes observed in pregnant women and children and indigenous peoples. Delays were also observed in recognizing the eventual demand for pulmonary bypass (ECMO) as an important clinical care option in the intensive care unit setting.
Should the GHSAG create a "common restricted platform" and conduct operations using researchers instead of experienced operational biosurveillance analysts, the "system of systems" will fail to reach its desired operational potential. This is evident in the current status of both NBIC and Project Wildfire. Open communication of biosurveillance information is essential for timely recognition of patterns, as we have repeatedly observed in Haiti, where formal surveillance data is not being shared openly and informal surveillance has repeatedly highlighted the mounting problem of diarrheal disease.
Silos of information / lack of information sharing were a significant part of the problem associated with the 9/11 warning failure. Here we observe avocation by the GHSAG to build a restricted access "system of systems", where two of the GHSAG capabilities, GPHIN and Argus, are themselves restricted access. Current White House policy for countering biological threats does not advocate this kind of approach:
Building Global Capacity for Disease Surveillance, Detection, Diagnosis, and Reporting
- Permits them to detect, identify, and report promptly any public, animal, or plant health or agricultural emergencies of international concern;
- Focuses on major population centers, known locations of endemic and epidemic disease and their vectors, and any known associated local terrorist or criminal threats;
- Is integrated and interoperable with their existing logistical infrastructure and sensitive to their public and agricultural health priorities;
- Is sustainable within the often limited resources of the country/region, either unilaterally or with other partners;
- Improves coordination between human, plant, and veterinary disease reporting systems, especially in relation to zoonotic diseases; and
- Is transparent and enables the sharing of information with international human, plant, and
animal health agencies and the United States.
In my testimony as the chief architect and Principal Investigator of Project Argus, I reported the following:
In late 2004 and early 2005, Argus participated in the tsunami response by providing daily situational awareness reports to humanitarian responders. In commenting on our operations, the US Pacific Command wrote, “Information is power only when it's shared. The situational awareness that portions of ARGUS provided during tsunami relief efforts was an impressive attention step. We see some tremendous opportunities and value added for this capability within our area of operational responsibility, which literally covers half the globe. Thanks for keeping our situational awareness up during difficult times."
US Pacific Command's observation carries on to our present-day efforts in Haiti.
As a final point, I would highlight the evolution of tornado warning systems in the 1940s and 1950s, initially created as a classified military warning unit under national security interest (protection of bombers and munitions plants). Once Congress and the public were aware that successful warnings were provided of tornadoes that subsequently killed civilians, the demand for transparency was immediate.
Recent Comments