Operational Biosurveillance

Forecasting Event Signatures: "Partly Cloudy With A Chance Of Quarantine"

Our team's focus is on the recognition of infectious disease event signature patterns.  Note the emphasis on recognition, implying a human has come to realize the importance of a pattern of social stress related to an outbreak... a pattern that may highlight the unusual.  

The importance of being able to recognize the unusual pattern is revealed when attempting to provide early warning of influenza pandemics, the emergence of novel pathogens, laboratory accidents involving transmissible pathogens that have escaped into the surrounding community, or acts of biological terrorism.  This is a crucial toolset for monitoring Biological and Toxic Weapons Convention compliance, for example.  

We have pointed out it is crucial for an analyst to be adept at signature processing... otherwise, we will continue to see report of unexpected public health crises that were later found to have actually begun months prior to first public notice.

Depending on the type of pathogen involved, epidemics are associated with an incredible array of indicators of medical, public health, and general socio-economic stress.  These indicators exhibit different temporal patterns, depending on whether it is a "first contact" presentation or a routinely encountered disease in a given country.  Different pathogens produce different patterns that are recognizable to the astute analyst.  We have learned that a pathogen may be reliably recognized by an analyst without the need for traditional epidemiological data based on its associated indicator pattern.  

We have been forecasting the appearance of a multitude of pathogens for several years now in various operational environments.  Recently, we activated an automated capability able to reduce what took us millions of dollars in human labor and months of time down to a few thousand dollars and seconds.  One of the key features is an ability to forecast an entire event signature library for a pathogen.

For example, below is a forecast library for Ebola hemorrhagic fever in Uganda, which reached an IDIS Category 5 (i.e. disaster conditions) during that country's first contact / recognition of the pathogen in 2000.    These examples are but a small sample set of indicators, broken down by (a) proxy indicators of an Ebola outbreak, (b) medical countermeasures (e.g. isolation of patients), (c) public health countermeasures (e.g. quarantine), and (d) public response (e.g. avoidance of infected victims).  

As we have pointed out several times here in Operational Biosurveillance, it is to the analyst's advantage to anticipate the entire signature pattern of an infectious disease event before it happens.  Pattern anticipation is an important component of priming one to recognize an unusual or unexpected pattern.

(a)

(b)

(c)

(d)

 

Related articles

Infectious Disease Event Signatures: Another Tool in the Box of Pattern Recognition

Novel Coronavirus: Indication of Another Warning Failure?

Nested Warning Failure: Novel Coronavirus

"The Matrix Has You": H7N9 Signature Tracking

Review: The Novel Coronavirus Warning Sequence (KSA/UK)

The Global "Song" of Suffering Due to Infectious Disease

"The Matrix Has You": H7N9 Signature Tracking

Reminiscent of The Matrix, the astute analyst monitors the signature pattern in near-real time as information is reported on H7N9:  

This is the actual "rhythm strip" of H7N9, read from top to bottom, where the bottom includes the more recent reports.  Each dot is an indicator of specific interest to the analyst.

The state of the art in operational biosurveillance is 1) automation, 2) overlays of artificial intelligence-based assessments, and 3) signature pattern forecasting.  This is how the work is done when watching the global output of infectious disease event reporting.

 

 

Proof of Performance: Daily Forecasts of Routine Pediatric Diagnoses

Here we compare daily disease forecasts issued 60 days in advance during 2012.  These forecasts were based on four years of prior data (2008-2011).

We've been reaching daily forecast accuracies that are generally in the 85-90% range, meaning 85-90% of the time our daily forecasts for a given disease are falling with +1 / -1 standard deviation of the forecasted mean for the given day.  Forecast lines related to RSV and hMPV were off in 2012 due to the appearance of the worst winter respiratory season documented in the last five years of the hospital.  In this case, the forecast facilitated rapid recognition of unusual disease patterns.

Below are example proofs of forecast performance.  We expect these statistics to improve over time with access to more data.  What is being forecasted here is the Infectious Disease Medical Infrastructure Strain Index (IDMIS), which is a function of case count, by medical disposition (outpatient, emergency dept, inpatient, ICU).  The IDMIS is an operational "bottom line" that highlights expected stress placed on a local medical infrastructure.  Higher IDMIS values indicate greater strain.

For each of the example diseases below, the top graph reflects the daily forecasted mean, with +1 / -1 standard deviation values.  The majority of the diseases are associated with zero -1 standard deviation values.  In other words, we could either see X cases or zero cases for any given day.  The bottom graph reflects what was actually observed in 2012.

 

Gastroenteritis (82% daily forecast accuracy)

 

Strep Pharyngitis (98% daily forecast accuracy)

 

Viral Infection (100% daily forecast accuracy)

 

 

Acute Conjunctivitis (84% daily forecast accuracy)

 

 

Acute Otitis Media (89% daily forecast accuracy)

 

Acute Sinusitis (89% daily forecast accuracy)

 

Acute Pharyngitis (88% daily forecast accuracy)

 

Croup (57% daily forecast accuracy- due to unusually severe RSV season in 2012)

 

Acute Upper Respiratory Infection (94% daily forecast accuracy)

 

RSV Bronchiolitis (65% daily forecast accuracy due to non-routine, unusually severe season in 2012)

 

RSV-/Influenza-Negative Bronchiolitis (81% daily forecast accuracy due to non-routine, unusually severe season in 2012)

 

 

Pneumonia (86% daily forecast accuracy)

 

Reactive Airway Disease Exacerbation (86% daily forecast accuracy)

 

Cellulitis (86% daily forecast accuracy)

 

One of the Futures of Biosurveillance

This is current reality in operational biosurveillance: the routine forecasting of routine infectious disease... where you are presented with a forecast months ahead of peaking disease activity.

Dialogue Between a Meteorologist and an Operational Biosurveillance Analyst

Dialogue Between a Meteorologist [M] and an Operational Biosurveillance Analyst [OBA]:

 

[M] A forecast is a prediction of an event before it begins.  If the event has begun (e.g. a hurricane) at a given location, updates are called "nowcasts." For example, the initial wind speed forecast for hurricane Sandy at Battery Park might have been 60 mph. If the storm has started and the forecast is raised to 70 mph, that would be a nowcast. 

 [OBA] There are many in public health who claim forecasting cannot be done.  This is simply not true for a wide variety of infectious disease conditions.  Indeed we are discovering there are a multitude of health outcomes only peripherally related to infectious disease that can be forecasted as well.  Among those who are practicing physicians, this has been long captured in the classic discourse between physician and patient.  For example, "don't smoke or you may develop lung cancer".  The meteorological notion of a "nowcast" has been done routinely now for several years at the National Infectious Disease Forecast Center as well as some of our other engagements overseas such as Haiti.  "Nowcasts" are also performed by public health, however at nowhere near the depth of coverage as we are able to provide now via direct linkages to clinical medical data (which is a major source of public health advisories).  Public health advisories are but the barest subset of what is observed on a daily basis in clinical medicine and that is relevant to the Public Interest.

[M] Climatology represents the long term averages (in meteorology, usually 30 years or more). Climatology tells us that, in NYC, January, on the whole, will be colder than July.   Persistence is the tendency of the weather of tomorrow to be like the weather of today.  A forecast is considered "skillful" if it beats both "climatology" and "persistence".  If it is sunny and 75° today and exactly the same thing occurs tomorrow, even a perfect forecast is not "skillful" because there was no change in the weather.

[OBA]  Excellent point.  However what is considered state of the art in meteorology is not so in operational biosurveillance.  Indeed, the vast majority of healthcare providers operate in reactive mode, simply addressing disease one patient at a time and deciding whether or not to treat with antibiotics.  With the exception of influenza, most physicians have no idea what their local baselines are for rhinovirus, metapneumovirus, Salmonella, etc etc etc.  In other words, a lack of comprehensive understanding of one's own local baseline.  This unfortunately extends to local antibiotic resistance patterns, where we are clearly losing several important battles with resistance in our communities.

So, current state of the art in operational biosurveillance is a few exceptions representative of "skilled" forecasts (e.g. influenza vaccine drift) and a greater majority of "unskilled" forecasts.  However what is considered "unskilled" from the strict definition of meteorology is considered utterly "skilled" by your average healthcare provider and the public because they are unaware of their baselines in the first place.

[M]  We have previously discussed the concept of adopting the Watch / Warning terminology used by meteorology in the event a forecastable major adverse health event presents itself. If I recall correctly, watches by the National Weather Service are issued at longer time periods (~ two days out) with a 30% threshold. In other words, if the forecasters believes there is a one-in-three chance of an occurrence, a Watch is issued. If, as the start of the event grows closer, a Warning is issued when the confidence reaches 70% at roughly 24 hours out. 

An issue we will have to consider as our forecasts are adopted on a wide basis is, to what extent did our forecast cause a different outcome than what would have occurred naturally? Triple-digit tornado death tolls were relatively common prior to the creation of the tornado warning program. They are exceedingly rare now (even with a large population) because of the warning system. The forecasts have changed the outcomes (fewer deaths than would exist without the warning system). 

Assume we issue an "Influenza Watch." Our watch then prompts a proactive and effective vaccination program. Our watch may have the effect of surpressing the number of cases needed to "verify" (another word would be "validate") our forecast.  In other words, if we forecast 10% of the population will contract influenza and there is a subsequent vaccination program, and the actual number is turns out to be 6% was it because we made an inaccurate forecast or was it because of the vaccination program?

[OBA]  This is a crucial point.  Anecdotally, we have observed patients flocking to our pediatric practice requesting influenza vaccination because 1) we issued advisories to them months in advance of the forecasted drift season; 2) the bulk of our patient families support influenza vaccination; and 3) the community is small enough where "everyone knows each other" and therefore share stories of children who were unvaccinated and later hospitalized with influenza.  So, the pre-event forecast (i.e. well ahead of the first confirmed influenza case in the county) began the community orientation process, followed by discussion after discussion family by family in the clinic as they asked about the forecast, followed by that single case of a hospitalized unvaccinated child to show the "proof of threat" as well as "proof of preventive action".  We then issued "nowcasts" updating the public as to what we were seeing in the hospital: that the hospitalized were almost universally unvaccinated.  We observed several of the "vaccine refusers" reconsider their position in the context of this powerful local community influence.

All of the above said, operational validation does not equate to academic validation.  And we readily acknowledge the question of validation to be a tremendous challenge when prognosticating disease. Conversely, we have often pointed out that academic studies and peer-review does not often transition into the reality of daily clinical medicine.   There is tremendous challenge here. 

Only by building years of operational experience may we have the necessary data to address this challenge.

 

 

Village Level Validation of Cholera Forecasting In Haiti

We have completed our assessment of cholera forecasts issued at the village level for the province of Artibonite, Haiti.  To our astonishment, several of the forecasts were quite accurate and useful from an operational perspective.  Astonishment because the time series we used were very short (~2 years of data).  We are very excited with these results.

Forecasting Infectious Disease For Literally Half The Planet

Today the Ascel Bio National Infectious Disease Forecast Center has made history with the issuance of national level forecasts for 15 countries and 100 infectious diseases.   

The breadth of these forecasts covers more than half the world's population.

 

Forecast Warning Sequence: 2012-2013 US Influenza Season

As the US national influenza season peaks out, it is important to review the importance of operational forecasting, "live" situational awareness, and proactive response in the local community setting.

Introduction

The Ascel Bio National Infectious Disease Forecast Center resides on the Western Slope of Colorado and issues monthly forecasts and "live" advisories covering approximately 100 infectious diseases.  The director of the center is a practicing pediatrician whose patients voluntarily sign up for access to the forecasts.  These forecasts are distrbuted electronically to local healthcare providers, school nurses, veterinarians, public health officials, pharmacists, EMS, and other members of the networked healthcare community.

The lay community is sharply polarized between those who vaccinate their children versus those who refuse the influenza vaccine only or the entire pediatric vaccination series (e.g., DTaP, HIB, IPV, Rota, etc).  This belief system does not fall squarely within one political bias or another such as liberal versus conservative.  However, the anti-vaccination belief system is firmly rooted in this rural community and is extremely difficult to counteract through clinical guidance.

Forecast Warning Sequence

The forecast warning sequence for the 2012-2013 influenza season has been previously described here.  This is the second time our team has successfully forecasted a vaccine drift season of influenza. 

 Forecasts were issued to the local community as such:

1. The community was pre-sensitized in the first week of September to the topic of influenza as a result of advisories issued by the Center regarding the local county agriculture fair (i.e. the concern of emerging A/H3N2 variant).
2. This was followed by the general forecast, which again emphasized concern for emerging A/H3N2 variant in late September.
3. In October, concern about a vaccine drift season of influenza was shared with patient families in the clinic, with emphasis on the need to vaccinate with the seasonal vaccine.  It was expressed the vaccine was expected to be well-matched to the drifted strains.  Limitations of vaccine efficacy overall were known to the clinician and expressed to patient families such as the vaccine is thought to prevent hospitalization, severe morbidity, and mortality.  This face to face dialog was maintained with every family that came to the clinic, regardless of chief complaint, throughout the season.
4. On October 25th, the November-December forecast was released to the community, which emphasized the same points described in bullet #3 above. 
5. On November 14th, an advisory was issued that described the first hospitalization of a combined RSV/influenza B patient. 
6. The December-January forecast was released on December 2nd, again emphasizing the need to vaccinate for influenza. 
7. Another advisory issued on December 12th with the second hospitalization of RSV/influenza B.
8. On January 7th, the January-February forecast was issued, again with emphasis on vaccination.

Results Thus Far

• All hospitalizations have been of children either incompletely or totally unvaccinated with this season's influenza.
• We have "converted" 4 families (each with between 1-3 children) who previously were staunch opponents to the influenza vaccine.
• Only 2 families (x2 children) refused influenza vaccination.  We estimate >99% compliance.
• The pediatric service has not been overwhelmed by influenza hospitalizations.  We have seen less than 5 children who presented at the outpatient level that were previously vaccinated.  None of these children had serious disease.

Preliminary Conclusions

• Development of trust with the receiver community was paramount to successful deployment of infectious disease forecasts.
• Issuance of operational forecasts coupled to updated "live" advisories drove compliance.
• Full transparency about vaccine efficacy directly relates to maintenance of trust with patient families.
• Issuance of operational forecasts coupled to updated "live" advisories that were trusted by the public ultimately resulted in mitigation of surge within a fragile pediatric healthcare infrastructure. 
• Prevention of hospitalization or transfer out of facility due to influenza is more cost-effective than lack of an aggressive vaccination education campaign facilitated by forecast and tactical warning.

 

 

 

CDC Validates The Ascel Bio US National Influenza Forecast

We previously highlighted initial indicators of validation for the US national influenza forecast.  The Centers for Disease Control and Prevention have now non-ambiguously affirmed what we already noted.   

We note this affirmation and validation of our forecast is more than 15 weeks after issuance of our first forecast.

Post-Sandy Influenza Forecast Validation

We had been concerned about the potential for influenza transmission post-Sandy among stressed, displaced citizens.  Particularly in emergency shelters. 

Reporting from the state of New York on Nov 17th revealed an increase in influenza activity (see attached).  The question that will likely go unanswered is to what degree, if any, population displacements due to Sandy influenced transmission.  We know that much of the country has been reporting early activity of influenza, likely mitigated by higher vaccination rates post-the 2009 H1N1 pandemic.  This was a validated forecast that applied to North America writ large.

Download Flu_report_current_week