Operational Biosurveillance

Forecasting Event Signatures: "Partly Cloudy With A Chance Of Quarantine"

Our team's focus is on the recognition of infectious disease event signature patterns.  Note the emphasis on recognition, implying a human has come to realize the importance of a pattern of social stress related to an outbreak... a pattern that may highlight the unusual.  

The importance of being able to recognize the unusual pattern is revealed when attempting to provide early warning of influenza pandemics, the emergence of novel pathogens, laboratory accidents involving transmissible pathogens that have escaped into the surrounding community, or acts of biological terrorism.  This is a crucial toolset for monitoring Biological and Toxic Weapons Convention compliance, for example.  

We have pointed out it is crucial for an analyst to be adept at signature processing... otherwise, we will continue to see report of unexpected public health crises that were later found to have actually begun months prior to first public notice.

Depending on the type of pathogen involved, epidemics are associated with an incredible array of indicators of medical, public health, and general socio-economic stress.  These indicators exhibit different temporal patterns, depending on whether it is a "first contact" presentation or a routinely encountered disease in a given country.  Different pathogens produce different patterns that are recognizable to the astute analyst.  We have learned that a pathogen may be reliably recognized by an analyst without the need for traditional epidemiological data based on its associated indicator pattern.  

We have been forecasting the appearance of a multitude of pathogens for several years now in various operational environments.  Recently, we activated an automated capability able to reduce what took us millions of dollars in human labor and months of time down to a few thousand dollars and seconds.  One of the key features is an ability to forecast an entire event signature library for a pathogen.

For example, below is a forecast library for Ebola hemorrhagic fever in Uganda, which reached an IDIS Category 5 (i.e. disaster conditions) during that country's first contact / recognition of the pathogen in 2000.    These examples are but a small sample set of indicators, broken down by (a) proxy indicators of an Ebola outbreak, (b) medical countermeasures (e.g. isolation of patients), (c) public health countermeasures (e.g. quarantine), and (d) public response (e.g. avoidance of infected victims).  

As we have pointed out several times here in Operational Biosurveillance, it is to the analyst's advantage to anticipate the entire signature pattern of an infectious disease event before it happens.  Pattern anticipation is an important component of priming one to recognize an unusual or unexpected pattern.

(a)

(b)

(c)

(d)

 

Related articles

Infectious Disease Event Signatures: Another Tool in the Box of Pattern Recognition

Novel Coronavirus: Indication of Another Warning Failure?

Nested Warning Failure: Novel Coronavirus

"The Matrix Has You": H7N9 Signature Tracking

Review: The Novel Coronavirus Warning Sequence (KSA/UK)

The Global "Song" of Suffering Due to Infectious Disease

"The Matrix Has You": H7N9 Signature Tracking

Reminiscent of The Matrix, the astute analyst monitors the signature pattern in near-real time as information is reported on H7N9:  

This is the actual "rhythm strip" of H7N9, read from top to bottom, where the bottom includes the more recent reports.  Each dot is an indicator of specific interest to the analyst.

The state of the art in operational biosurveillance is 1) automation, 2) overlays of artificial intelligence-based assessments, and 3) signature pattern forecasting.  This is how the work is done when watching the global output of infectious disease event reporting.

 

 

Proof of Performance: Daily Forecasts of Routine Pediatric Diagnoses

Here we compare daily disease forecasts issued 60 days in advance during 2012.  These forecasts were based on four years of prior data (2008-2011).

We've been reaching daily forecast accuracies that are generally in the 85-90% range, meaning 85-90% of the time our daily forecasts for a given disease are falling with +1 / -1 standard deviation of the forecasted mean for the given day.  Forecast lines related to RSV and hMPV were off in 2012 due to the appearance of the worst winter respiratory season documented in the last five years of the hospital.  In this case, the forecast facilitated rapid recognition of unusual disease patterns.

Below are example proofs of forecast performance.  We expect these statistics to improve over time with access to more data.  What is being forecasted here is the Infectious Disease Medical Infrastructure Strain Index (IDMIS), which is a function of case count, by medical disposition (outpatient, emergency dept, inpatient, ICU).  The IDMIS is an operational "bottom line" that highlights expected stress placed on a local medical infrastructure.  Higher IDMIS values indicate greater strain.

For each of the example diseases below, the top graph reflects the daily forecasted mean, with +1 / -1 standard deviation values.  The majority of the diseases are associated with zero -1 standard deviation values.  In other words, we could either see X cases or zero cases for any given day.  The bottom graph reflects what was actually observed in 2012.

 

Gastroenteritis (82% daily forecast accuracy)

 

Strep Pharyngitis (98% daily forecast accuracy)

 

Viral Infection (100% daily forecast accuracy)

 

 

Acute Conjunctivitis (84% daily forecast accuracy)

 

 

Acute Otitis Media (89% daily forecast accuracy)

 

Acute Sinusitis (89% daily forecast accuracy)

 

Acute Pharyngitis (88% daily forecast accuracy)

 

Croup (57% daily forecast accuracy- due to unusually severe RSV season in 2012)

 

Acute Upper Respiratory Infection (94% daily forecast accuracy)

 

RSV Bronchiolitis (65% daily forecast accuracy due to non-routine, unusually severe season in 2012)

 

RSV-/Influenza-Negative Bronchiolitis (81% daily forecast accuracy due to non-routine, unusually severe season in 2012)

 

 

Pneumonia (86% daily forecast accuracy)

 

Reactive Airway Disease Exacerbation (86% daily forecast accuracy)

 

Cellulitis (86% daily forecast accuracy)

 

H7N9 Nowcast: IDIS Cat 3, With Shift In Probabilities For Cat 4

 

Disease Outbreak Nowcast: H7N9

Heightened Threat from H7N9 issued by Ascel Bio on 2013.04.17.    Ascel Bio's quantitative models are showing a heightened threat from H7N9 and a significantly increased probability of community crisis.     The probability of a high impact IDIS Category 4 or higher event has increased from 11% to 29% over the past fourteen days.  Further, the likelihood that the event will be a lower impact locally containable IDIS Category 1 or 2 event has dramatically dropped from 56% to 9%.   Ascel Bio's quantitative process draws from official disclosures and other reliable open source reports.  Ascel Bio is monitoring a national and international public health response incorporating poultry culling, vaccine development, and airport screenings. The broad measures being taken testify to an increasing uncertainty about the virus and its potential to induce an international pandemic.    The recognition of mild and asymptomatic cases in humans, which portend more widespread activity than currently recognized, and family clusters, which indicate potential person-to-person transmission, are also important.  There has also been an increase in the geographical spread of the virus. With the virus now affecting persons in Shanghai and Beijing, the threat of disease transmission and social disruption is heightened.  Analysis of Indicators Using the most up-to-date information, Ascel Bio's forecast model is indicating a high categorical impact for  H7N9 as clustering, international response measures, and geographic spread all indicate an increasingly disruptive disease event. Ascel Bio will continue to monitor the outbreak and provide analysis of vital information in order to assess the ongoing risk of H7N9.

The Global "Song" of Suffering Due to Infectious Disease

This is a real operational "tracking strip" of about two weeks' of monitoring over approximately ten countries.  Each row is an event report.  Each of the markers in the row is an indicator associated with a health event.  Viewed from this perpective, and if you scroll it through time across tens of thousands of reports, one notices patterns to social response to infectious disease worldwide... 

As the sun strikes the earth, the earth begins to sing of suffering due to disease.  It is a song familiar to every culture in the world...

 

We have often referred to these signature patterns as a "song", or the code to a social DNA of reaction to the presence of outbreaks and epidemics.  After years of seeing this "code", it continues to amaze us.

When an analyst understands this code / song, he understands meaning when the code has changed... and the potential need to warn.

Dialogue Between a Meteorologist and an Operational Biosurveillance Analyst

Dialogue Between a Meteorologist [M] and an Operational Biosurveillance Analyst [OBA]:

 

[M] A forecast is a prediction of an event before it begins.  If the event has begun (e.g. a hurricane) at a given location, updates are called "nowcasts." For example, the initial wind speed forecast for hurricane Sandy at Battery Park might have been 60 mph. If the storm has started and the forecast is raised to 70 mph, that would be a nowcast. 

 [OBA] There are many in public health who claim forecasting cannot be done.  This is simply not true for a wide variety of infectious disease conditions.  Indeed we are discovering there are a multitude of health outcomes only peripherally related to infectious disease that can be forecasted as well.  Among those who are practicing physicians, this has been long captured in the classic discourse between physician and patient.  For example, "don't smoke or you may develop lung cancer".  The meteorological notion of a "nowcast" has been done routinely now for several years at the National Infectious Disease Forecast Center as well as some of our other engagements overseas such as Haiti.  "Nowcasts" are also performed by public health, however at nowhere near the depth of coverage as we are able to provide now via direct linkages to clinical medical data (which is a major source of public health advisories).  Public health advisories are but the barest subset of what is observed on a daily basis in clinical medicine and that is relevant to the Public Interest.

[M] Climatology represents the long term averages (in meteorology, usually 30 years or more). Climatology tells us that, in NYC, January, on the whole, will be colder than July.   Persistence is the tendency of the weather of tomorrow to be like the weather of today.  A forecast is considered "skillful" if it beats both "climatology" and "persistence".  If it is sunny and 75° today and exactly the same thing occurs tomorrow, even a perfect forecast is not "skillful" because there was no change in the weather.

[OBA]  Excellent point.  However what is considered state of the art in meteorology is not so in operational biosurveillance.  Indeed, the vast majority of healthcare providers operate in reactive mode, simply addressing disease one patient at a time and deciding whether or not to treat with antibiotics.  With the exception of influenza, most physicians have no idea what their local baselines are for rhinovirus, metapneumovirus, Salmonella, etc etc etc.  In other words, a lack of comprehensive understanding of one's own local baseline.  This unfortunately extends to local antibiotic resistance patterns, where we are clearly losing several important battles with resistance in our communities.

So, current state of the art in operational biosurveillance is a few exceptions representative of "skilled" forecasts (e.g. influenza vaccine drift) and a greater majority of "unskilled" forecasts.  However what is considered "unskilled" from the strict definition of meteorology is considered utterly "skilled" by your average healthcare provider and the public because they are unaware of their baselines in the first place.

[M]  We have previously discussed the concept of adopting the Watch / Warning terminology used by meteorology in the event a forecastable major adverse health event presents itself. If I recall correctly, watches by the National Weather Service are issued at longer time periods (~ two days out) with a 30% threshold. In other words, if the forecasters believes there is a one-in-three chance of an occurrence, a Watch is issued. If, as the start of the event grows closer, a Warning is issued when the confidence reaches 70% at roughly 24 hours out. 

An issue we will have to consider as our forecasts are adopted on a wide basis is, to what extent did our forecast cause a different outcome than what would have occurred naturally? Triple-digit tornado death tolls were relatively common prior to the creation of the tornado warning program. They are exceedingly rare now (even with a large population) because of the warning system. The forecasts have changed the outcomes (fewer deaths than would exist without the warning system). 

Assume we issue an "Influenza Watch." Our watch then prompts a proactive and effective vaccination program. Our watch may have the effect of surpressing the number of cases needed to "verify" (another word would be "validate") our forecast.  In other words, if we forecast 10% of the population will contract influenza and there is a subsequent vaccination program, and the actual number is turns out to be 6% was it because we made an inaccurate forecast or was it because of the vaccination program?

[OBA]  This is a crucial point.  Anecdotally, we have observed patients flocking to our pediatric practice requesting influenza vaccination because 1) we issued advisories to them months in advance of the forecasted drift season; 2) the bulk of our patient families support influenza vaccination; and 3) the community is small enough where "everyone knows each other" and therefore share stories of children who were unvaccinated and later hospitalized with influenza.  So, the pre-event forecast (i.e. well ahead of the first confirmed influenza case in the county) began the community orientation process, followed by discussion after discussion family by family in the clinic as they asked about the forecast, followed by that single case of a hospitalized unvaccinated child to show the "proof of threat" as well as "proof of preventive action".  We then issued "nowcasts" updating the public as to what we were seeing in the hospital: that the hospitalized were almost universally unvaccinated.  We observed several of the "vaccine refusers" reconsider their position in the context of this powerful local community influence.

All of the above said, operational validation does not equate to academic validation.  And we readily acknowledge the question of validation to be a tremendous challenge when prognosticating disease. Conversely, we have often pointed out that academic studies and peer-review does not often transition into the reality of daily clinical medicine.   There is tremendous challenge here. 

Only by building years of operational experience may we have the necessary data to address this challenge.

 

 

Google Flu Trends: Missing The Mark

The New York Times recently called out an operational problem with Google Flu Trends that we have highlighted several times here on Operational Biosurveillance. 

The bottom line is nothing beats direct access to medical data or communications from experienced observers.  The current "bright light / shiny object" phenomenon associated with the fascination around social media has clouded some governments and academics, who believe it to be the global situational awareness Holy Grail.  The reality is, one cannot have a production line of scalpels without the expertise to know what to do with them.  Ideally, one has actually involved the surgeon before making the scalpel in the first place.

The lessons in America's sinking healthcare system and its botched electronic medical record implementation comes to mind.  Very little physician input went into the design of the information technology platforms now being touted as a "solution" to American medicine.  Now physicians are tethered to capabilities that have only questionably improved the business of medicine.  Some physicians claim it has made the situation even worse.

CDC was warned about marketing Google Flu Trends without an operational versus academic evaluation by our team in 2008.  That said, one could say that Flu Trends may be considered by CDC to be nothing more than a public awareness tool for influenza education.

Google Flu Trends: Google Flu Trends uses aggregated Google search data in a model created in collaboration with CDC to estimate influenza activity in the United States. For more information and activity estimates from the U.S. and worldwide, see http://www.google.org/flutrends/

We often find academic literature may be tremendously misleading in regards to true operational utility- a finding now being roundly discussed in the medical literature as the discovery of pharmaceutical industry-backed "academic" studies espouse the virtue of a given drug.  The most poignant example of this in the biosurveillance world has been the years-long foray into syndromic surveillance, which has yet to prove the operational value versus the expenditure invested thus far by taxpayers.  (That said, the author believes syndromic surveillance may yet have utility but for different objectives than originally intended or marketed.)

The missing link in the vast majority of social media-based surveillance is a true understanding of what drives the human discourse in the first place and how to capture the real "seed" of the conversation.  In the end, social media, like news media, is but a proxy for ground truth.  Here again, we see the information technology serving as a tail wagging the dog.

Our advice: beware of the hype when it comes to social media monitoring platforms. 

 

 

 

 

The Biosurveillance Watch: What We Are Monitoring

We are often asked the important question of "what is the priority?" when monitoring an entire world of biological activity.  As of today, here is the answer to that question, keeping in mind we are based in the United States with a bias towards those biological agents that are potentially capable of acutely disrupting US-based infrastructure.  We use different prioritized reporting requirements for different clients / countries as appropriate.

1. A/H3N2 variant influenza virus within the US
2. A/H3N2 seasonal influenza virus within the US
3. A/H1N1 seasonal influenza virus within the US
4. (other type A influenza virus variants within the US)
5. RSV/adenovirus/metapneumovirus/influenza virus combinatory effect on medical infrastructure as winter approaches

Note there is no mention of A/H5N1 or other HPAI strains, novel coronvirus, Ebola virus, or other attention / headline-grabbing pathogens or marketing campaigns by research programs.  We always keep an eye on A/H5N1 reporting, for example, but until indicators of sustained human-human transmission have been reported it remains a low priority.  One must understand the associated indicator pattern if one does not have access to the direct medical or laboratory diagnostic data, of course.

Prioritized tracking bullets 6, 7, 8, 9, 10, etc include issues such as pertussis, however these have not proven to be acutely disruptive to the infrastructure at this juncture.

This is operational reality from our point of view, which is obviously not "hot and sexy".  It is always more interesting and exciting to talk about the most dramatically deadly pathogens such as novel coronavirus, Ebola, or H5N1.  But therein is the bias in human communication behavior: death > high morbidity > mild illness > largely asymptomatic illness.

One of the biggest responsibilities for a operationally-minded biosurveillance analysts is to focus on realistic probabilities and ask the question, "so what?"

 

 

 

West Nile Virus: Loss of a Warning Signature

During the emergence of West Nile virus in the United States from 1999-2006 (which year depended on what part of the country you are in) provided biosurveillance operators with a couple of easy indicators for impending transmission: wild bird die-offs (i.e. corvids) and equine illness / mortality. 

This year, however, in the current surge of West Nile transmission (our local example may be found here), we did NOT see pre-event bird die-offs reported.  In our part of the country there are plenty of corvids to monitor, but presumably thanks to herd immunity we were not provided the benefit of a month or two "heads up".  The fallback position was the early and abrupt WNV-positive mosquito pools noted 6 weeks in advance of the earliest known date of positivity. 

Although our part of the country hosts a substantial equine population, the West Nile vaccine has mitigated much of the illness we would have normally seen.  Indeed, this year it was almost as if human cases were sentinels for equine cases.

This year has indeed been associated with a warmer than usual winter, spring, and summer.  Temperature is a crucial driver for mosquito transmission competency, dramatically shortening the time it takes for a mosquito to go from a viremic avian blood meal to being able to infect another host (less than 24 hrs if incubated from 26-30 degrees Centrigrade in the lab, for instance).  However, as noted above, the loss of the expected signature pattern may present a challenge now for those US locales that have not yet seen a resurgence of West Nile- especially those cities who have deactivated or toned down their mosquito surveillance programs thanks to the poor economy.

 

Progress in Operational Infectious Disease Forecasting in Haiti

In collaboration with several National Academy of Science members, we have completed an assessment of forecast signatures in Haiti for cholera.  The results are a promising beginning to anticipating endemic transmission patterns in Haiti to facilitate proactive preparedness and response.