Operational Biosurveillance

Round 2: The Black Canyon Forecasting Station Versus Google's FluTrends, ProMED, And SickWeather Combined

Round 2. 

The first such comparison generated quite a bit of Internet traffic.  Record-breaking for this blog, in fact, from multiple countries around the world.

Here again we perform a simple comparison between information produced and made available to the public by the Black Canyon Infectious Disease Forecasting Station (#1) and the combined output of Google's Flu Trends, ProMED, and SickWeather.  Screenshots from these websites were taken at approximately 2130 on 19 Feb 2012.

 

Black Canyon Infectious Disease Forecasting Station.

As previously noted, we forecasted in December a surge in patient volume and clinical severity due to a variety of viral respiratory pathogens, notably RSV, for our specific location in the US: Delta County, Colorado, which is near Grand Junction (geographic coordinate Lat/Long: 38° 44' 43.4394", -108° 2' 49.1994").

We indeed did see this, along with the forecast-facilitated detection of unusually severe, high volume RSV disease related to coinfection with other pathogens.  Here in Colorado, this has produce pediatric infrastructure strain at a grid level, meaning near-diversion status for several PICUs along the Front Range (our referral loci).  We were warned that beds may not be available the next time we needed to execute a transfer...

The entire staff complement of our office was infected, yet again, but no one took time off.  Much misery among staff working very long hours, with physicians pulling 48 hr call that afforded little sleep.

 

Google Flu Trends.

Again tells us we are at "moderate" levels of influenza. 

We have not documented a single positive specimen to-date in our clinic.  We are aware of a clinic 20 miles east of us that documented two positive cases in their kids about ten days ago- no activity since.

 

ProMED. 

Absolutely nothing on Colorado.

 

SickWeather.

Thirteen (13) different symptoms selected, as shown below.  One single report of "pneumonia" submitted.  Nothing else.  We were unable to obtain evidence of true pre-event operational forecasting in the SickWeather website.

Summary.

Again, the limitations of using open source information and social media is revealed.  We just experienced an unusual, non-routine infectious disease signature pattern in our county that resulted in significant medical infrastructure strain and near-diversion status at the grid level for the pediatric critical care infrastructure in our state.  There is no indication of this significant level of infectious disease activity in Google's Flu Trends, ProMED, and SickWeather combined.

The Black Canyon Forecasting Station in "War Time Operations"

This is a local healthcare community WARNING posted today by the Black Canyon Infectious Disease Forecasting Station.  Of note, this is a warning related to mostly non-notifiable infectious disease- thus reflective of the socialization process of routine infectious disease forecasting analogous to meteorological forecasting as a vital component to community health.  Public health is certainly a partner and involved, but until a notifiable disease is involved, they do not engage.  Meanwhile, we are dealing with near-maximum patient volumes, clinical severity, and awareness of an entire medical grid strained (which effects our decisions to transport to critical care facilities).

 

A new epidemiological situation in Delta County has been detected by the Black Canyon Infectious Disease Forecasting Station over the course of the last week.

The forecast for February (first issued in December) called for heightened activity levels of multiple pathogens, including RSV, human metapneumovirus, adenoviruses, coronaviruses, influenza A and B, and Moraxella catarrhalis, among other pathogens.  The forecast also indicated we would reach maximum medical infrastructure strain this month, and indeed we and the entire state have been experiencing this- multiple PICUs on the front range are at near-diversion status.  What is remarkable is this is NOT due to influenza virus, as we have had a remarkably mild season (again, as forecasted). 

The current situation represents expected high RSV transmission in the community.  What is not expected is the older age of children involved and the clinical severity- we are seeing an abrupt and unusually high volume of admissions, clinical severity, and age of our patients (2 to 2.5 year olds).  Of significant concern is the atypical pattern of these patients exhibiting apparent clinical recovery, then abruptly decompensating and requiring intubation and transport to facilities with more appropriate critical care facilities.  We have discovered there is an (apparent) high and unusual rate of RSV coinfection with other pathogens such as human metapneumovirus, adenovirus, Moraxella, and notably for Delta County, Haemophilus influenzae.  PICUs in our transport referral area are indicating a rare rate of coinfection with influenza A.  RSV coinfection, in general, is often associated with more severe and life-threatening complications.  Consultation with pediatric emergency departments and PICUs nationwide resulted in reports from several areas of the country indicating similar patterns.  We continue to receive updates...

When influenza is exhibiting its typical level of activity, transmission activity produced by these other pathogens are thought to be suppressed in the community by mechanisms still as-yet unclear.  While the forecast of a mild influenza season initially appeared to be "good news", in this case it revealed a disruptive phenomenon involving multiple pathogens (with RSV leading the way).

James M. Wilson V, M.D.
Chief of Station
Black Canyon Infectious Disease Forecasting Station
-and-
Department of Pediatrics
Delta County Memorial Hospital
Delta, Colorado

 

The other important phenomenon is the social network that has evolved around operational forecasting.  Physicians simply come forward quicker with information vital to confirm signals that would otherwise be missed.  This, of course, is a crucial sociological process when contemplating rapid recognition of non-routine or unusual infectious disease activity signatures.

 

 

Peaking Activity of hMPV Confirmed in Multiple US Cities

Cross-checking with other pediatric colleagues nationwide has revealed that yes indeed, from Rhode Island to Chicago to Dallas pediatricians are noting a substantial level of hMPV activity.  Technology is surely a factor here, where we are actually able to 'see' this pathogen via molecular diagnostics. 

Let it be said, we are seeing a sociological shift in our medical colleagues in even discussing this pathogen in the first place, which has largely been ignored because such recognition did not change clinical management.  Important because of a social shift in understanding our country's baselines of biological activity.  Such finer understanding and discourse about this pathogen feeds into the process of developing our core of "astute clinicians" so important to rapidly recognizing unusual biological activity signatures in our country.

 

 

"Frequency-Modulated Shielding" of a Hospital Using Operational Forecasts For Antibiotic Resistance

One of the greatest challenges we face in medicine is the judicious management of our armamentarium of antibiotics as we confront resistant microbes in our community.  There continues to be debate in the literature regarding the ultimate best strategy for deciding which antibiotic, by pathogen, one should use now versus a year from now when resistance profiles may have migrated. 

The Black Canyon Forecasting Station recently operationalized over 250 pathogen-antibiotic forecast lines for one of our client hospitals.  The major challenge of the hospital is how to manage its antibioitic armamentarium with an eye towards the future.  In other words, "when will we lose the ability to use a given antibiotic for a given pathogen, and based on this knowledge, should we continue to use it?"  Sometimes it's best to holster that "weapon" until a later time when it might be the only option left to you as a physician.

This is a key question from a variety of perspectives- pharmaceutical supply & demand, temporal and financial cost to develop new antibiotics, and patient treatment costs, among many others.  What was interesting to us was the disparity between guidance from The Sanford Guide To Antimicrobial Therapy 2011, Epocrates, and current reality in this particular hospital.  With antibiotic resistance forecasting now a part of the analysis, these disparities become even more poignant.  An example is Pseudomonas aeroginosa, which is horribly resistant to multiple antibiotics, and the forecasts are dismal.

Wiesch et al recently published a nice review of "Population Biological Principles of Drug-Resistance Evolution in Infectious Diseases" (see attached), which reminds us of

The emergence of resistant pathogens in response to selection pressure by drugs and their possible disappearance when drug use is discontinued are evolutionary processes common to many pathogens.

Constant forecasting and real-time monitoring of drug resistance may be our best defense given the research, development, and distribution of new antibiotics is not keeping up with the global problem of resistance. 

For those willing to take a leap from reality, look to the Star Trek-themed "frequency-modulated shielding" of a starship (i.e. hospital).  A bit of silliness, but a highly relevant analogy when considering the very real threat of encroaching drug-resistant bacteria and the reality of a too-slowly evolving antibiotic armamentarium. 

As the Black Canyon Forecasting Station brings this new forecasting capability on-line at the hospital, we note this may be the first time this has ever been attempted...

Download Population biological principles of drug-resistance evolution in infectious diseases

Pediatricians Testing All Admits For Human Metapneumovirus (hMPV)

We have become aware of word that some local pediatricians are testing all admits in Indiana for human metapneumovirus (hMPV).  In Colorado, an increase in activity has been observed, activity that was forecasted by the Black Canyon Station weeks ago, as previously posted. 

What is crucial for anyone in government observing this phenomenon is the first operational demonstration in the US of a National Weather Service-inspired infectious disease forecasting process that

1. presented a pre-event probablistic-based forecast that was deliberately used by clinicians to
2. ask important questions in the context of a subsequently detected emergent infectious disease phenomenon;
3. rapidly recognize the etiology of the phenomenon; and
4. report it out to others for net-centric awareness
5. at a sub- official public health notification threshold
   

Of course, this was routine procedure in the first 150 days post-quake in Haiti.  Operational infectious disease forecasting is a critical social primer to tighten the delta between anticipation-detection-recognition-warning-response. 

Note this process has further facilitated the 'digestion' and habituation of healthcare providers to ask specific questions about a virus that was previously unknown but endemic and routine.  Indeed most physicians have only care about the question "bacterial versus viral", ignoring the important microbiology behind the question that determines treatment and prognosis.  While knowing what the virus is does not change clinical management at this point in time, it does provide an important risk communication tool for our patients and their families: resolution of uncertainty.  This is such an important part of the physician-patient communication.

The key question to ask now is whether genomic sequences should be obtained, and an epidemiological workup should be entertained.  hMPV does exhibit genomic variability similar to RSV- therefore, we may be seeing a "bad" year of hMPV versus unmasking of activity due to low influenza and RSV activity. 

Either way, the question should be asked if pediatricians are now testing all admissions for hMPV...

 

The Black Canyon Forecast Station Versus Google's Flu Trends, HealthMap, and SickWeather

Here we conduct a simple comparison between the output of the Ascel Bio Black Canyon Infectious Disease Forecasting Station, Google's Flu Trends, HealthMap, and SickWeather.  We are going to compare the output of the enhanced forecast provided by the Black Canyon Station that includes integrated national and local health intelligence data. 

 

Black Canyon Infectious Disease Forecasting Station

Currently, we are dealing with a probable large-scale seasonal outbreak of human metapneumovirus (hMPV) here in Delta County, Colorado.  Large enough we saw two more families affected with several more RSV- and influenza-negative cases of pneumonia.  Disruptive enough to keep a large portion of our patients out of school for days as well as moms and dads.  Disruptive enough to infect 2/4 of our clinical staff, prompting time off to recuperate and infect a third who worked through her illness. 

This event represents a validated forecast specific for both the clinical syndrome and pathogen issued 30 days ago for the geographic coordinate Lat/Long: 38° 44' 43.4394", -108° 2' 49.1994" (Delta County, Colorado).

Pneumonia, as forecasted, has reached a peaking score on the Infectious Disease Medical Impact Scale (IDMIS), which indicates we are reaching an expected level of strain in terms of incidence and clinical severity. 

RSV bronchiolitis is creeping upward as well, in keeping with forecasted expectations.  Influenza activity has been zero in our pediatric population, meaning to-date we have not had a single positive test for influenza.  Acute URI, otitis media, and bronchitis have all been observed, consistent with our forecasts.  We are aware that Group A Streptococcus, rhinovirus, hMPV, adenovirus, and coronavirus are circulating at heightened levels as well- all of which were also forecasted.  ... And this is but a snapshot of the nearly 100 diseases the station is currently forecasting and tracking live.

All disease activity considered, we are approaching our peak clinical load in terms of combined disease incidence and clinical severity, with evidence of mild infrastructure strain due to healthcare worker illness and absenteeism.

 

Flu Trends

This is the most current Flu Trends view of the state of Colorado regarding influenza:

The team at Google provided the following statement to the journal Nature:

Because the relative frequency of certain queries is highly correlated with the percentage of physician visits in which a patient presents with influenza-like symptoms, we can accurately estimate the current level of weekly influenza activity in each region of the United States, with a reporting lag of about one day.

According to Google, we are now at moderate levels of influenza for the state of Colorado.  This is grossly incorrect.  We have not seen a single positive influenza sample in any of our patients in our county, which is located west of the Front Range.  To quote the latest influenza surveillance report from the State Department of Health, which utilized data up to Jan 28th, which is 5 days before the Flu Trends graph:

For the week ending January 28th, influenza activity remained low but increased somewhat based on sentinel hospital lab data and influenza-like illness (ILI) data. Influenza activity remained along the Front Range, as well as the northeast and southwest parts of the state.

According to the above Flu Trends graph for the state of Colorado, we were moving away from low levels of influenza activity in mid September. 

Had we used Flu Trends, we would have been told there was moderate activity when activity was clinically non-existent.  Further, there is no indication of whether influenza was at worse levels than prior years.  In essence, a useless source of information for the purposes of operational biosurveillance or clinical medicine. 

 

HealthMap

Below is a screenshot, taken on Friday, Feb 3, 2012 at 2000.

What is remarkable is the appearance of minimal infectious disease activity, not only for Delta County (which is located southeast of Grand Junction), but for the entire state of Colorado.  This is most certainly not the case. 

Clicking the only two pins shown for the state of Colorado reveals the following:

This is the limitation of solely relying on media reports and other internet-based reporting of infectious disease events.

 

SickWeather

SickWeather recently tweeted:

"#doctors #nurses be prepared for #patients - see what's going around this #cold & #flu season using http://t.co/9z59kOFA"

Had we been using SickWeather's map for our clinic, it too would have failed us.  Below are screenshots of all of the symptoms we are seeing.  There is no indication, based on the SickWeather mapping of the Western Slope, of the volume and clinical severity of the illnesses we are now seeing as physicians.

[The above figures were updated per SickWeather's comment- to be fair to them!]  The only symptom displayed by SickWeather is pneumonia, with no indication of the true pervasiveness and severity in our kids that we are seeing.  What would be interesting is to see how long that signal had been there and how closely it matched our clinical records.  Either way, we have been issuing pre-event forecasts for pneumonia in our area for weeks prior to its appearance.

SickWeather claims to forecast infectious disease activity,

however this is incorrect and misleading.  There is no pre-event (meaning an expression of what will happen before the event has actually occurred).  It is highly doubtful that a patient would tweet a symptom before our knowledge of what was already happening in our community.  Particularly regarding events of significance such as the outbreak of viral pneumonia we are seeing and associated medical infrastructure strain at the outpatient level.

 

Summary

The combined consideration of situational awareness provided by Flu Trends, HealthMap, and SickWeather failed to provide an accurate picture of current infectious disease activity in Delta County, or even the state of Colorado.  None of these capabilities provided true forecasting of infectious disease activity.

While flashy, web-based technologies capture the attention and fancy of the media, academic journals, TED talks, blogs, and the like, there remains the practical reality of the need for experienced professionals to choose the appropriate data, assess the significance of the information, and practically answer the "so what" question.  The current operational state of the art actually does not predominantly use the data provided by technologies such as Flu Trends, HealthMap, or SickWeather. 

To place the world of operational biosurveillance in context, we often use the Microscope Analogy:

We consider open source / web-based event or horizon scanning, as offered by Flu Trends, HealthMap, or SickWeather, to be analogous to the low power lens of the microscope.  The low power lens helps one become oriented to a specimen slide (i.e. a given location in the world), especially if one has not seen that slide before.  Syndromic surveillance data represents a medium power lens.  Clicking further as one zooms in for more comprehensive detail, one is using clinical laboratory diagnostic data. 

This is precisely how the Haiti Epidemic Advisory System functioned.  Initially open source information was used for orientation prior to deliberate social network engineering.  Eventually, open source information was discarded in favor of direct ground observations and clinical data.  One could not execute emergency medical response that included helo drops based solely on open source data.

It is important to view the media hype and the marketing around these technologies with a skeptical eye... because operational reality is quite a bit different: better accuracy, far faster, more complex, operates in the pre-event domain, and has been used clinically on a daily basis.

 

 

 

 

 

In The Newspaper: Your Infectious Disease Forecast Along With The Weather

We have now established the 1950s meteorological model of sponsored infectious disease "weather" forecasts that appear in the local newspaper just as weather forecasts do.  This is an important step in operational biosurveillance because it promotes social habituation to this kind of information.  By "social habituation" we infer the "routinization" of a community to see infectious disease forecasts in a manner that promotes deeper awareness of what is supposed to be normal and routine baseline. An example of this was yesterday's post about probable hMPV transmission in our community.

We observed in Haiti such a process creates an environment of enhanced discourse about perceived "unusual" disease activity.  This is important because without a forecast to drive engagement of participants to consider the forecasted "hypothesis" and compare that to what they are seeing, one finds fragmentation and individualism in the medical community.  We have seen examples where a physician does not even speak to his partner who is in the same office about unusual infectious disease activity indicative of a non-routine outbreak.  In this case, the partner was sending multiple samples of bloody stool to the lab for testing under suspicion of STEC and did not communicate within the office "to be on the lookout". 

We find it important to not only share forecasts with healthcare providers but the public as well, which creates a check and balance system- parents ask about forecasted diseases and whether what their child has is the forecasted disease in question or not.  ... So if our physicians are not paying attention, many of our families are.

 

Forecasting Your Own "Demise"

It is indeed ironic to be an accurate forecaster.  Ironic when that forecast affects you and your family. 

In the last 7-10 days, we experienced a surge in respiratory disease affecting our pediatric patients, including transmission involving entire families with pneumonia in multiple instances that included two of our staff's families... and my own.  Three of our staff were infected, and yours truly was forced to go half-day on Friday and take Monday off.  My partner had to absorb the patient load, then she herself caught it.

Testing has been negative for influenza, RSV, and pertussis.  Clinical findings range from URI to pneumonia with a viral presentation.  We have seen both croup and reactive airway / asthma exacerbation including hospital admissions. 

This clinical entity and severity was forecasted in late December.  Notably, non-RSV / non-influenza etiologies also forecasted included human metapneumovirus (HMPV), a pathogen described only in the last 7 years as an ubiquitous respiratory virus clinically indistinguishable from RSV.  This virus is a well known trigger of reactive airway / asthma exacerbation and does cause croup.  Pneumonia is more frequently diagnosed in children with hMPV infection than RSV... and we are certainly seeing quite a bit of pneumonia.

This morning we received report from a key diagnostic lab they have molecular diagnostic confirmation of a notable increase in hMPV activity.   Therefore, we believe it most likely the recent surge in non-RSV non-flu respiratory disease activity and clinical severity in our county is most likely due to hMPV, which is a validated forecast issued by the Black Canyon Forecasting Station in late December.

While a validated forecast should make us thrilled, we are reminded by one of our partners who is a meteorologist to "root for the humans, not the disease".  I'm thinking about this comment as I continue to cough up a lung full of phlegm...