Skip to main content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
PLoS One. 2018; 13(7): e0199616.
Published online 2018 Jul 12. doi: 10.1371/journal.pone.0199616
PMCID: PMC6042698
PMID: 30001345

Reducing healthcare-associated infections incidence by a probiotic-based sanitation system: A multicentre, prospective, intervention study

Elisabetta Caselli, Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing,1,2,* Silvio Brusaferro, Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing,3 Maddalena Coccagna, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Writing – review & editing,2 Luca Arnoldo, Data curation, Formal analysis, Investigation, Project administration, Resources, Software, Writing – review & editing,3 Filippo Berloco, Data curation, Investigation, Project administration, Resources, Writing – review & editing,4 Paola Antonioli, Conceptualization, Data curation, Investigation, Project administration, Resources, Writing – review & editing,5 Rosanna Tarricone, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Writing – original draft, Writing – review & editing,6 Gabriele Pelissero, Data curation, Investigation, Project administration, Resources, Writing – review & editing,7 Silvano Nola, Data curation, Investigation, Project administration, Resources, Writing – review & editing,8 Vincenza La Fauci, Data curation, Investigation, Project administration, Resources, Writing – review & editing,9 Alessandro Conte, Data curation, Investigation, Project administration, Resources, Writing – review & editing,3 Lorenzo Tognon, Data curation, Investigation, Project administration, Resources, Writing – review & editing,10 Giovanni Villone, Data curation, Investigation, Project administration, Resources, Writing – review & editing,11 Nelso Trua, Data curation, Investigation, Project administration, Resources, Writing – review & editing,12 Sante Mazzacane, Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing,2 and for the SAN-ICA Study Group1,2,3,4,5,6,7,8,9,10,11,12,
John Conly, Editor

Associated Data

Supplementary Materials
Data Availability Statement

Abstract

Healthcare Associated Infections (HAI) are a global concern, further threatened by the increasing drug resistance of HAI-associated pathogens. On the other hand, persistent contamination of hospital surfaces contributes to HAI transmission, and it is not efficiently controlled by conventional cleaning, which does not prevent recontamination, has a high environmental impact and can favour selection of drug-resistant microbial strains. In the search for effective approaches, an eco-sustainable probiotic-based cleaning system (Probiotic Cleaning Hygiene System, PCHS) was recently shown to stably abate surface pathogens, without selecting antibiotic-resistant species. The aim of this study was to determine whether PCHS application could impact on HAI incidence. A multicentre, pre-post interventional study was performed for 18 months in the Internal Medicine wards of six Italian public hospitals (January 1st 2016—June 30th 2017). The intervention consisted of the substitution of conventional sanitation with PCHS, maintaining unaltered any other procedure influencing HAI control. HAI incidence in the pre and post-intervention period was the main outcome measure. Surface bioburden was also analyzed in parallel. Globally, 11,842 patients and 24,875 environmental samples were surveyed. PCHS was associated with a significant decrease of HAI cumulative incidence from a global 4.8% (284 patients with HAI over 5,930 total patients) to 2.3% (128 patients with HAI over 5,531 total patients) (OR = 0.44, CI 95% 0.35–0.54) (P<0.0001). Concurrently, PCHS was associated with a stable decrease of surface pathogens, compared to conventional sanitation (mean decrease 83%, range 70–96.3%), accompanied by a concurrent up to 2 Log drop of surface microbiota drug-resistance genes (P<0.0001; Pc = 0.008). Our study provides findings which support the impact of a sanitation procedure on HAI incidence, showing that the use of a probiotic-based environmental intervention can be associated with a significant decrease of the risk to contract a HAI during hospitalization. Once confirmed in larger experiences and other target populations, this eco-sustainable approach might be considered as a part of infection control and prevention (IPC) strategies.

Trial registration—ISRCTN International Clinical Trials Registry, ISRCTN58986947.

Introduction

Healthcare-associated infections (HAIs) are a global concern impairing the clinical outcome of up to 15% of all hospitalized patients in the world [1]. In Europe, about 3.2 million patients acquire a HAI every year, and 37,000 die as a HAI direct consequence and also because of the increasing multi-drug resistance (MDR) of HAI-associated pathogens [1, 2]. Studies conducted in Italian hospitals show a 5–10% HAI incidence with a mortality rate up to 20–30% [35].

On the other hand, it is known that hospital surfaces are persistently contaminated by many microorganisms which can contribute to HAI transmission [611], as surfaces represent the reservoir of several pathogens spread by hospital inpatients and personnel [6, 8, 1015]. Control of surface contamination has been so far approached by conventional chemical-based sanitation, which has limitations, as it cannot prevent recontamination phenomena [1619], has an high environmental impact, and can contribute to selection of disinfectant-resistant and even antibiotic-resistant pathogens [20, 21], potentially contributing to a further increase of HAI-associated MDR pathogens [22, 23].

Recently, the ‘health’ of hospital surfaces has been re-thought as the health of the human body, considering that, rather than eradicating all pathogens, replacing them by beneficial microbes might be more effective in preventing infections [24, 25]. Toward this principle, a sanitation approach based on eco-sustainable detergents containing spores of Bacillus probiotics (Probiotic Cleaning Hygiene System, PCHS) was recently studied, showing that it is safe for hospitalized patients [26], it can stably decrease surface pathogens up to 90% more than conventional disinfectants [27, 28], and it does not select for resistant strains, rather reducing them [29], without increasing sanitation costs [27, 30].

Here we aimed to analyze the impact on HAI incidence by implementing an 18-month multicentre interventional study (from January 1st 2016 to June 30th 2017) in six Italian hospitals, to assess whether the unique use of an innovative eco-sustainable microbial-based cleaning procedure can influence HAI occurrence.

Methods

Study design and participants

A multicentre, prospective, pre-post interventional study simultaneously analyzing surface contamination and HAI incidence was conducted in six public medium to large Italian hospitals for 18 months (from January 1st 2016 to June 30th 2017). The trial protocol was approved by the Institutional Ethics Committees of each enrolled healthcare structure. The trial was registered in the ISRCTN Registry (ISRCTN58986947).

Eligibility criteria for enrolled hospitals included: i) approval of local ethical committee before entering the study, ii) presence of internal medicine/geriatrics and neurology wards (which were included in the study), iii) size larger than 100 in-patients beds, iv) presence of an established HAI surveillance program and infection control team, v) acceptance not to introduce any new intervention focused on Infection Control and Prevention (ICP), potentially affecting HAI incidence, except those already existing in the enrolled hospital wards and those necessary to manage possible outbreaks, throughout the whole study.

Enrolled hospitals represented different geographical Italian regions (North, Centre and South), and were randomly allocated in one of two Intervention groups (I1, I2). One further hospital, meeting all eligibility criteria, was included as an external contemporaneous control (extC), as it did not receive intervention and was only monitored for HAI incidence and environmental bioburden. I1-group included three hospitals entering the study on January 1st 2016 (Feltre, Roma, Foggia); I2-group included two hospitals entering 5-months later, on May 1st 2016 (Vigevano and Tolmezzo); the extC hospital was monitored starting from May 1st 2016, as for I2 group (Messina). Random allocation was performed by an independent investigator using computer-generated random numbers.

The intervention consisted uniquely of the introduction of PCHS sanitation (a patented system by Copma, Ferrara, Italy), replacing the conventional chemical-based (chlorine products) one. Enrolled hospitals agreed not to introduce any other intervention potentially affecting HAI incidence throughout the whole study, except those already existing at the enrolment time. Cleaning staff did not change during the study and were adequately trained for the appropriate PCHS application in all the hospitals receiving the intervention. More precisely, no general cleaning topics were covered, as the training was limited to the correct modalities to prepare and use the PCHS cleansers. No other differences were introduced, either in number and qualification of cleaning staff, nor in frequency of the procedures, resulting in no different perception by cleaning staff, healthcare personnel and patients about the change of the cleaning system. Healthcare personnel, data extractors and patients were not aware about the change of the cleaning system.

The study included two phases: a 6 month pre-intervention period (pre-PCHS), when hospitals maintained the conventional chemical-based sanitizing procedures, and a 6 month post-intervention period (PCHS), when PCHS was routinely applied, with a minimum 2 month interval between the two phases, when PCHS was introduced.

All new patients admitted at the enrolled wards in the pre-PCHS and PCHS periods were included in the study and surveyed for the development of HAIs, without distinction of age or gender and keeping their identity completely anonymous, so that informed consent was not needed. Surveillance of HAIs was already done in all enrolled hospitals, and not implemented for the study, but during the study HAIs were monitored daily, in continuum, in order to obtain their true incidence value. Patients already present at the beginning of pre-PCHS and PCHS periods, and in the window period between pre-PCHS and PCHS phases, were excluded. Observation of patients was stopped on the last day of pre-PCHS and PCHS periods.

HAI analyses

Each HAI occurring during the observation periods in the patients admitted to the enrolled hospital wards in the two observed periods was identified according to the criteria defined by the European Centre for Disease Prevention and Control (ECDC) [31]. All HAI types were included in the study, namely: urinary tract infections (UTI), bloodstream infections (BSI) including those central-vascular catheter (CVC)-related, systemic-clinical sepsis, gastrointestinal infections (GI), skin and soft tissue infections, pneumonia, lower respiratory tract infections (LRI), surgical site infections (SSI), reproductive tract infections, EENT (eye, ear, nose and throat or mouth) infections, bone and joint infections, intra-abdominal infections, and non-specified infections.

HAI etiological agents were identified by microbiology laboratories of each hospital, based on routine diagnostic tests. No changes were applied to the conventional diagnostic process of each hospital, except for Bacillus species for which routine searches were done in all clinical samples.

Environmental sampling and analyses

Hospital surface microbiota was analyzed monthly by a central team (CIAS centre, University of Ferrara). To this aim, three points/room (floor, bed footboard and sink) were sampled in duplicate as previously described [27, 29], in 3–6 randomized rooms/hospital (respectively in hospitals with less or more than 100 enrolled ward beds). Total bacteria, Staphylococcus spp., Enterobacteriaceae spp., Acinetobacter spp., Mycetes, Pseudomonas spp., and Clostridium difficile were quantified on specific Rodac contact plates (CFU/m2).

Quarterly (twice in the pre-PCHS and twice in the PCHS phases), the same points were also analyzed by molecular assays, as previously described [29]. Briefly, total bacterial amount, Bacillus count and microbiota resistome were respectively quantified by panB real time quantitative PCR (panB-qPCR), spo0A-qPCR and a qPCR microarray for 84 resistance genes (Qiagen Antibiotic Resistance Genes, BAID-1901ZRA, Hilden, Germany). Resistome was also analyzed in four PCHS-Bacillus isolates from each sampling campaign of the PCHS-phase.

Data collection and management

Dedicated healthcare professionals (recruited and trained in a standard way) collected in continuum data from patients’ clinical records in a standardised spreadsheet, per each hospital. Professionals collecting clinical data were only aware of an incidence study to be conducted during all the study period (18 months), and were blinded to the intervention time and hospital’s groups.

A first electronic clinical record was filled out for each admitted patient, and contained general data: gender, age, origin, admission date, admission diagnosis, presence of specific risk factors, antibiotic therapy in the two weeks preceding admission, presence of colonization by alert microorganisms, eventual presence of infection at admission and its etiological agent.

A second form, filled out in case of HAI onset, included information about HAI onset, location, etiological agent, drug therapy and infection resolution/outcome. All data were anonymized and submitted centrally via a secure, password-protected website. A central team was available during the whole study period to solve informatics problems, standardizing and validating completeness of data and their consistency. Data analyzers were blinded to the intervention time and hospital’s allocation. A quota corresponding to at least 10 recorded HAIs per hospital setting were validated by a blinded second expert, to minimize the risk of infection miscoding.

Bioburden data, collected monthly by the central team (University of Ferrara), were also uploaded in the same password-protected website.

Outcome measures

The primary outcome measure was the reduction of HAI incidence in the PCHS compared to the pre-PCHS phase. Variations in infection rates were analyzed both as cumulative incidence per 100 admitted patients, and as HAIs incidence rates per 1,000 patient days. Secondary outcome measures were qualitative and quantitative characterisation of hospital surface bioburden in the surveyed areas.

Statistical methods

The study power was estimated based on admissions and HAI incidence rates in Italian hospitals [35]. The sample size was calculated considering an 80% power to detect an infection incidence reduction of at least 25% starting from a hypothesized rate of 4%, assuming a two-sided test with an alpha level of 0.05, and corresponded to 10,476 patients.

Statistical analyses were performed using chi-square test, Kolmogorov-Smirnov test for evaluating normality, parametric (Student’s t test) and non-parametric (Mann-Whitney) tests, chi-square test of association, and multivariable analysis (logistic regression), assuming as statistically significant a P value at least <0.05. Multivariable model was developed including all the parameters which showed a statistically significant correlation with HAI occurrence by univariate analysis. Bonferroni correction for multiple comparisons was applied for analysis of microarray data (a Pc value <0.05 was considered significant). Analyses were performed using the software IBM® SPSS20® Statistics (IBM, Bologna, Italy).

Recruitment

All enrolled hospitals completed the study, guaranteeing a continuous monitoring for a 6-months period in the pre-intervention (pre-PCHS) phase and a 6-months period in the intervention (PCHS) phase (Fig 1). Overall, the study surveyed 11,842 patients, 11,461 from intervention I1-I2 hospitals and 381 from the external control hospital (Table 1). Globally 24,875 environmental samples were analyzed.

An external file that holds a picture, illustration, etc.
Object name is pone.0199616.g001.jpg
Study design graphic representation.

Six Italian hospitals from different geographical regions were enrolled in the study (North: Feltre, Tolmezzo, Vigevano; Centre: Rome; South: Foggia, Messina). Five hospitals were randomly allocated in two Intervention groups (I1, I2) and one further hospital represented an external control (extC): I1-group included Roma, Foggia and Feltre hospitals, entering the study on January 1st 2016; I2-group included Vigevano and Tolmezzo hospitals, entering 5-months later, on May 1st 2016; extC hospital was represented by Messina hospital, receiving no intervention and monitored from May 1st 2016. The phases of the study are indicated by colours: orange, 6-months pre-intervention period (pre-PCHS); light green, stabilization period, when PCHS was introduced; green, 6-months post-intervention period (PCHS), when PCHS was routinely applied. Sampling campaigns for microbiological analyses are indicated by circles: conventional microbiological analyses were performed monthly (black circles), and molecular analyses were performed quarterly (red circles) in all enrolled hospitals.

Table 1

Population characteristics of study participants in pre-PCHS and PCHS phases, stratified by enrolled hospitals.
 Patients
No.
Age
(mean±SD)
Length of stay
(mean±SD)
Patients with at least one HAI
No. (%)
GroupHealthcare StructureTotalPre-PCHSPCHSPre-PCHSPCHSPre-PCHSPCHSPre-PCHSPCHSStatistical significance
I1Feltre2,8121,5991,21373.1±16.474.9±15.48.7±5.710.0±6.177 (4.8%)30 (2.5%)P = 0.0013
OR, 0.50
95% CI, 0.33–0.77
Foggia1,95196698572.4±15.974.7±14.89.9±5.412.0±7.1106 (11.0%)36 (3.7%)P<0.0001
OR, 0.31
95% CI, 0.21–0.45
Roma3,1161,6111,50568.0±17.868.1±17.210.4±8.911.0±7.350 (3.1%)20 (1.3%)P = 0.0008
OR, 0.42
95% CI, 0.25–0.71
I2Tolmezzo2,4531,1861,26774.3±14.375.9±13.310.6±9.89.8±6.325 (2.1%)21 (1.7%)P = 0.4111
OR, 0.78
95% CI, 0.44–1.41
Vigevano1,12956856172.7±15.572.6±16.18.9±5.49.6±6.226 (4.6%)21 (3.7%)P = 0.4829
OR, 0.81
95% CI, 0.45–1.46
Tot. (I1+I2)11,4615,9305,53171.8±16.473.0±15.89.7±7.610.5±6.7284 (4.8%)128 (2.3%)P<0.0001
OR, 0.47
95% CI, 0.38–0.58
extCMessina38114623571.3±14.972.3±15.711.6±8.99.7±5.712 (8.2%)16 (6.8%)P = 0.6080
OR, 0.82
95% CI, 0.37–1.78

I1, Intervention group 1; I2, Intervention group 2; extC, external control hospital.

Results

Impact of sanitation on HAI incidence

Globally 11,842 patients were surveyed, all hospitalized in the Internal Medicine wards of the enrolled hospitals, this aimed to analyze the incidence of HAIs in a more homogeneous patients’ sample possible.

Table 1 summarizes the main characteristics of the studied population by a participating hospital. HAI cumulative incidence (patients with HAI/enrolled patients) decreased significantly in the PCHS period compared to the pre-PCHS phase of the I1-I2 hospitals, from 4.8% (283/5,930) to 2.3% (128/5,531) (range 1.3–3.7%) (P<0.0001), regardless of the geographical location and entering time in the study. HAI incidence rate per 1,000 patient-days decreased from 5.4 (314/57,742) to 2.4 (141/58,201), with an incidence rate ratio of 0.45 (95% CI, 0.36–0.54). The decrease was evident in each individual hospital, including the structure with a previous very low HAI incidence (Tolmezzo, from 2.1% to 1.7%). A slight not significant reduction was observed also in the external control hospital, where the total number of HAIs was 15 infections in 12 patients in the first 6-month period, and 16 infections in 16 patients in the second one, with a respective cumulative incidence rate of 8.2% and 6.8% (OR = 0.82; 95% CI, 0.37–1.78; P = 0.6), and a relative incidence rate per 1,000 patient-days corresponding to 9.4 (15/1,600) and 7.0 (16/2,279) respectively (OR = 0.75; 95% CI, 0.37–1.54).

The analysis of HAI incidence bimonthly rates in the pre-PCHS and PCHS phases, showed no tendency to decrease in the pre-PCHS period, prior to the intervention, whereas a stable reduction was observed following the introduction of PCHS sanitation (Fig 2).

An external file that holds a picture, illustration, etc.
Object name is pone.0199616.g002.jpg
HAI incidence rates in the I1-I2 intervention hospitals.

Results are expressed as bimonthly value of incidence rate per 1,000 patient-days, respectively in the pre-PCHS (red) and PCHS periods (blue). 95% CI intervals are also reported.

The main clinical features of the observed patients were very similar in the pre-PCHS and PCHS periods of the study, as shown in Table 2. The univariate analysis results confirmed as risk factors for HAI occurrence those already reported in the literature, indicating for example a positive correlation with the presence of urinary or central venous catheters and increasing age, whereas a protective effect emerged for being a male and self-sufficiency (S1 Table).

Table 2

Patient characteristics of the I1-I2 hospitals in the pre-PCHS and PCHS periods (11,461 patients).
Patients characteristicsPre-PCHSPCHS
Total patients
No. (%)
Total patients
No. (%)
Total5,9305,531
Gender: male2,977 (50.2%)2,928 (52.9%)
Age <651,518 (25.6%)1,265 (22.9%)
Age 65–741,261 (21.3%)1,177 (21.3%)
Age 75–841,821 (30.7%)1,753 (31.7%)
Age ≥851,330 (22.4%)1,336 (24.2%)
Incontinence1,448 (24.4%)1,369 (24.8%)
Disorientation804 (13.6%)747 (13.5%)
Self-sufficiency3,671 (61.9%)3,632 (65.7%)
Pressure sores393 (6.6%)237 (4.3%)
Surgery 30 day before122 (2.1%)80 (1.4%)
Ventilation215 (3.6%)161 (2.9%)
Parenteral nutrition200 (3.4%)141 (2.5%)
ATB 2 week before566 (9.5%)294 (5.3%)
MDRO at admission131 (2.2%)83 (1.5%)
Infection at admission1,216 (20.5%)1,089 (19.7%)
Urinary catheter (any type)1,368 (23.1%)1,166 (21.1%)
CVC264 (4.5%)260 (4.7%)

Self-sufficiency, ability to provide for themselves autonomously, measured by SSM (Self Sufficiency Matrix) scale; ATB, antibiotics; MDRO, multi drug resistant organism; CVC, central vascular catheter.

Among all the observed HAIs, urinary tract infections (UTI) represented the most prevalent infection type (Table 3), followed by bloodstream infections (BSI), systemic clinical sepsis, gastrointestinal infections (GI), skin and soft tissue infections, and respiratory infections. Following PCHS intervention, the cumulative incidence of the most frequent HAIs decreased: UTI, from 3% (179/5,930) to 1.2% (70/5,531); bloodstream infections-BSI, from 0.9% (54/5,930) to 0.6% (31/5,531); clinical sepsis, from 0.4% (22/5,930) to 0.1% (5/5,531); gastro-intestinal infections from 0.3% (17/5,930) to 0.1% (6/5,531); and skin/soft tissue infections from 0.3% (16/5,930) to 0.1% (6/5,531). Instead, the relative burden of each HAI type did not change significantly in the PCHS phase compared to the pre-PCHS one.

Table 3

HAIs in pre-PCHS and PCHS phases, stratified by type.
 Pre-PCHS
(I1 + I2)
No. (%)
PCHS
(I1 + I2)
No. (%)
extC
1st 6-months period
No. (%)
extC
2nd 6-months period
No. (%)
No. of HAIs314* (100%)141° (100%)15# (100%)16§ (100%)
TypeUrinary tract infections-UTI179 (57.0%)70 (49.6%)8 (53.3%)6 (37.5%)
Bloodstream infections-BSI [CVC related]54 [10] (17.2%)31 [7] (22.0%)-3 (18.8%)
Clinical sepsis22 (7.0%)5 (3.5%)1 (6.7%)-
Gastrointestinal-GI17 (5.4%)6 (4.3%)2 (13.3%)-
Skin and soft tissue15 (4.8%)6 (4.3%)3 (20.0%)1 (6.2%)
Pneumonia12 (3.8%)8 (5.7%)-2 (12.5%)
Lower respiratory tract10 (3.2%)6 (4.3%)3 (18.8%)
Reproductive tract1 (0.3%)---
Eye, ear, nose and throat or mouth EENT1 (0.3%)2 (1.4%)--
Bone and joint-1 (0.7%)--
Intra-abdominal-1 (0.7%)--
Surgical Site Infection---1 (6.2%)
Not specified3 (1.0%)5 (3.5%)1 (6.7%)-

* 256 patients with 1 HAI, 26 with 2 HAIs, and 2 with 3 HAIs

° 115 patients with 1 HAI, and 13 with 2 HAIs

# 9 patients with 1 HAI, and 3 with 2 HAIs

§ 16 patients with 1 HAI

Similarly, the number of HAI-associated microorganisms (identified in HAI patients) decreased significantly from 332 in the pre-PCHS phase to 137 in the PCHS phase (Table 4), whereas the relative percentages of isolated microorganisms remained unaltered: E. coli, E. faecalis, S. aureus, P. mirabilis and P. aeruginosa were in fact the most frequently isolates in both phases. Importantly, no infections sustained by PCHS-derived Bacilli were detected in any of the hospitalized patients in the enrolled structures, further supporting the absence of infectious risks associated with PCHS-Bacilli use indicated by previous studies.

Table 4

Microorganisms isolated from HAIs during pre-PCHS and PCHS phases in I1-I2 hospitals.
Pre-PCHSPCHS
Infections*301135
Exam not available or negative2719
Exam available274116
Isolated microorganismsSamples (n,%)Samples (n,%)PCHS vs pre-PCHS
S. aureus21 (6.3%)16 (11.6%)-23.8%
Staphylococcus spp.30 (9.0%)10 (7.2%)-66.6%
Enterococcus spp.57 (17.2%)24 (17.5%)-57.8%
Streptococcus spp.7 (2.1%)4 (2.9%)-42.8%
C. difficile9 (2.7%)3 (2.2%)-66.6%
E. coli93 (28%)27 (19.7%)-70.9%
Klebsiella spp.19 (5.7%)12 (8.7%)-36.8%
P. mirabilis15 (4.5%)6 (4.3%)-60.0%
P. aeruginosa15 (4.5%)10 (7.2%)-33.3%
Enterobacter spp.8 (2.4%)1 (0.7%)-87.5%
Citrobacter spp.3 (0.9%)0-100%
A. baumannii8 (2.4%)5 (3.6%)-37.5%
Morganella spp.3 (0.9%)0-100%
Other Enterobacteriaceae1 (0.3%)0-100%
Candida spp.26 (7.8%)11 (8.0%)-57.7%
Virus5 (1.5%)3 (2.1%)-40.0%
Others12 (3.6%)5 (3.6%)-58.3%
Total332 (100%)137 (100%)-

*During pre-PCHS phase, 301 HAIs included 13 co-infections; during PCHS phase, 135 HAIs included 6 co-infections.

The relative role of PCHS in the reduction of HAI onset was explored by a multivariable model including all the parameters emerged as variables positively associated with HAI occurrence by univariate analysis. The results (summarized in Table 5), while confirming as statistically significant risk factors the presence of urinary or central venous catheters (respectively OR = 2.68; 95% CI, 2.10–3.41 and OR = 1.99; 95% CI 1.40–2.82), showed PCHS use as a statistically significant independent protective effect (OR = 0.44; 95% CI, 0.35–0.54) (P<0.0001).

Table 5

Risk factors associated with HAI onset in patients of I1-I2 hospitals: Multivariable model*.
Population characteristicsPOR95% CI
Male0.018120.780.63–0.96
Age 65–74 vs Age <650.00471.711.18–2.48
Age 75–84 vs Age <650.00041.881.33–2.67
Age 85 or more vs Age <650.00261.781.22–2.58
Length of stayp<0.00011.081.07–1.09
Incontinence0.22530.850.66–1.10
Disorientation0.02261.371.05–1.76
Self-sufficiency0.56000.920.69–1.43
Pressure sores0.97570.990.69–1.44
Ventilation0.77021.070.68–1.67
ATB 2 week before0.84790.970.68–1.37
MDRO at admission0.62300.860.47–1.57
Urinary catheter (any type)p<0.00012.682.10–3.41
CVC0.00011.991.40–2.82
PCHSp<0.00010.440.35–0.54

* multivariable model included all the factors emerged as significantly associated with HAI onset by univariate analysis (11,461 patients).

Impact of sanitation on hospital surface microbiota

Surface bioburden analyses, including detection and quantification of Staphylococcus spp., Enterobacteriaceae spp., Pseudomonas spp., Acinetobacter, Clostridium difficile and Candida spp., showed a persistent contamination in all the enrolled wards in pre-PCHS period, with an overall pathogen load corresponding to 22,737 CFU/m2 (median value, range 17,053–60,632 CFU/m2), mostly attributable to Staphylococcal contamination (median load 21,895 CFU/m2, range 13,684–57,263 CFU/m2). Other microbial genera were less abundant: Enterobacteriaceae (median value 1,784 CFU/m2; range 444–3,015 CFU/m2), Acinetobacter (mean value 2,538 CFU/m2; range 214–3,836 CFU/m2), Pseudomonas spp. (mean value 361 CFU/m2; range 43–2,125 CFU/m2), C. difficile (mean value 286 CFU/m2; range 137–842 CFU/m2) and Candida spp. (mean value 1,480 CFU/m2; range 1,075–5,508 CFU/m2) (supporting information files in BioStudies repository, Accession No. S-BSST75).

The introduction of PCHS in the five intervention hospitals (I1 and I2 groups) induced a statistically significant decrease of pathogen contamination from 22,737 CFU/m2 to 4,632 CFU/m2 (median value; range 842–12,632 CFU/m2) (P<0.0001, corresponding to a mean 83% decrease of surface pathogen load (range 70–96.3%) (Fig 3A). By contrast, no variations were observed in the external control hospital between the two observation periods.

An external file that holds a picture, illustration, etc.
Object name is pone.0199616.g003.jpg
Surface contamination in the surveyed hospitals.

(A) Pathogen load on hospital surfaces, expressed as CFU/m2. Six pathogens were measured by direct CFU counting on specific Rodac plates, as described in Methods (Staphylococcal spp., Enterobacteriaceae spp., Acinetobacter spp., Candida spp., Pseudomonas spp., Clostridium spp.). Graphed results represent the sum of the median values obtained for each measured pathogen. Median values (lower part of the box) and Q3 values (upper part of the box, representing the 75% percentile values) are shown for each hospital, and for pre-intervention (pre-PCHS) and intervention (PCHS) phases. Values reported for the external control hospital (Messina), correspond to those detected in the 1st and 2nd 6-month periods of the study. (B) Total bacterial load and PCHS-Bacilli count, respectively measured by a pan-bacterial qPCR (panB) and a specific qPCR for Bacillus genus (spo0A). Results are expressed as genome copy number per 100 ng of tested DNA. The median values ± SD of pre-PCHS and PCHS phases are shown. Values reported for the external control hospital (Messina), correspond to those detected in the 1st and 2nd 6-month periods of the study.

Meanwhile, the quota of PCHS-Bacilli increased significantly on surfaces of intervention-hospitals from 0% (median value, range 0–30%) to 69.8% (median value, range 39.9–86.8%) of the total surface microbiota (P<0.0001) (Fig 3B). No increase in Bacillus counts was observed in the extC control hospital.

Microarray analysis of the microbiota resistome showed a significant global decrease of resistance genes in the I1-I2 hospitals during the PCHS-phase compared to what detected in the pre-PCHS period (P<0.0001; Pc = 0.008)(S1 Fig)(supporting information files in BioStudies repository, Accession No. S-BSST75). The prevalence of R genes was different in the individual hospitals, likely reflecting the selective pressure exerted in each setting, but the decrease of the R genes originally present during the pre-PCHS phase was observed in all hospitals. No decrease was instead observed in the external control hospital.

In parallel, resistome microarray analysis of PCHS-Bacilli isolates from surfaces of treated hospitals, showed no acquisition of R genes in all tested isolates during the whole study period (Fig 4), confirming previous studies supporting the genetic stability of the PCHS-Bacillus strains.

An external file that holds a picture, illustration, etc.
Object name is pone.0199616.g004.jpg
Resistome analysis of PCHS-Bacillus strains.

Antibiotic resistance genes were analyzed by microarray both in the PCHS detergent prior to application, containing a blend of three Bacillus species (Original) and in the Bacillus isolates (Isolates) collected from hospital surfaces in the PCHS phase of I1 and I2 hospital groups. For original PCHS-Bacilli, results are expressed as mean values ± SD of six replicates. For Isolates, results are expressed as the mean value ± SD of 120 Bacillus isolated from hospital surfaces. Both Original and Isolates values were compared to negative control values (NTC). Each Bacillus isolate was identified by PCR and sequencing prior to microarray analysis, as previously described [29].

Discussion

The role of persistent surface contamination in HAI transmission is recognized [6, 811], but the impact of environmental cleaning on HAI incidence lacks of robust data, since so far most studies correlating environmental bioburden with HAIs incidence considered a bundle of factors, or were limited to specific ward types (i.e. ICU)[32, 33].

Since we recently reported that a probiotic-based sanitation (PCHS) can modulate surface hospital microbiota [27, 29], the present pre-post interventional study was aimed to investigate directly the potential impact of this system on HAI incidence.

According to our results, in the absence of any other ICP intervention, PCHS was associated with a significant reduction (P<0.0001) of HAI incidence in the medical wards of acute hospitals. The reduction was observed in all HAIs commonly detected in internal medicine wards, some of which are associated with contact transmission, and this was associated with a concurrent decrease of the responsible isolated microorganisms, while their relative frequencies remained unaltered.

Multivariable analysis, while confirming the role of well known risk factors [34], such as the presence of catheters (OR = 2.68 and OR = 1.99 for urinary and CVC, respectively), revealed PCHS to be an independent protective factor (OR = 0.44; 95% CI, 0.35–0.54) (P<0.0001).

Furthermore, the bioburden data confirmed in a large sample that PCHS is able to reduce and remodulate the environmental contamination, inducing a significant decrease (-83%) of the overall surface pathogen load as well as of the resistance genes harboured by the surface microbiota (up to 2 Logs) [2729], suggesting that probiotic Bacilli can displace and replace pre-existing pathogens, limiting colonization and spreading of new potentially pathogenic and drug-resistant entries (contamination from care-givers, new patients, healthcare workers), a competitive mechanism well known in nature [3538].

In addition, microbiological and molecular systematic monitoring of the present study supports the safety of use of PCHS observed in previous trials [26, 27, 39], confirming the genetic stability of the PCHS-Bacilli and the absence of any infectious risk correlated to the use of PCHS probiotics in hospital settings.

Limitations

Although showing a strong protective effect of PCHS, this study has some potential limitations. A first possible one is related to the study design, which is a pre-post intervention run in the same hospitals. Nevertheless, the size of the sample and the magnitude of the resulting reduction seem to indicate a clear role of PCHS. Based on these results, further developments could include studies based on larger samples and different methodologies, such as stepped wedge trials and/or cluster randomized trials, including cost effectiveness. Also, the numbers of the external control hospital are small, and the non significant reduction observed might be due to this limitation. Since the explored settings are limited to internal medicine, geriatrics and neurology, further studies would benefit from exploring the impact in other healthcare settings, in order to better understand the generalizability of the obtained results.

Secondly, the sample size was calculated to detect differences in the global sample and not in the individual hospitals; nevertheless, the results showed a HAI decrease in all hospitals, although it was not statistically significant in those hospitals with a low HAI incidence at baseline.

Thirdly, a potential bias might be represented by the awareness of the healthcare personnel about the study itself, but healthcare professionals were aware only of an incidence study to be conducted during the whole study period (18 months). Furthermore, this point was addressed by limiting the information exclusively to hospital managers, recruiting external data collectors and data extractors, and including an external control hospital to monitor the potential impact related only to the presence of a study. At the same time it should be considered that the period of the study was very long (all together the span of time was 18 months), thus limiting the potential attention bias of the healthcare workers teams.

A further potential bias could be related to seasonality and geographical distribution of the enrolled hospitals. The 6 month follow-up period may not be sufficient to negate the effects of seasonal variability, and similarly the lack of enrolled sites from all three regions in both I1 and I2 groups may be a limitation. Further studies could therefore include a longer follow-up period and/or a more robust enrolment. Nevertheless, the size and the characteristics of the HAI reduction and the relative effect in term of displacement of microbiota seems to indicate that these potential confounders, if existing, could have a limited effect.

Lastly, although there was an agreement not to introduce measures to improve infection control in the enrolled hospitals, a potential for confounding is represented by the lack of measurement of hand hygiene over the study period.

Conclusions

This is the first study, to our knowledge, which shows an association between HAI incidence and environmental microbiota in such a large sample. Overall, collected results may contribute to emphasize the role of environmental microbiota modulation for cleaning in healthcare settings, introducing the possibility of an ecological approach in the area of environmental cleaning, which might be included among the effective tools available for infection prevention and control (IPC). This could support policies aimed at reducing the development of microbial resistance to disinfectants and antibiotics, leading to an effective reduction of costs related to HAI management. On another hand, our results might be useful to introduce methodologies to investigate environmental bioburden and circulation of resistome in healthcare settings, as its systematic analysis might open the possibility to explore new strategies in controlling its spread. Last, this study opens new issues to be explored: the applicability and the impact in different settings, the impact on different types of HAIs, the long-term effect of the routine use of PCHS, the dynamics between human pathogens population and probiotic Bacilli and the impact on costs related to management of HAIs. Of course, deep analyses about cost effectiveness will be needed, as well as future studies optimally designed to address information still lacking in this study.

In conclusion, these results might be important to better understand the role of environmental microbiota in healthcare settings, supporting the development of guidelines about environmental cleaning addressed to enhance IPC strategies.

Supporting information

S1 Table

HAI frequency in relation to patient characteristics in I1-I2 hospitals.

Univariate analysis results of risk factors for HAI occurrence in Pre-PCHS and PCHS patients.

(DOCX)

S1 Fig

Resistome analysis of the surface microbiota.

(A) Analysis of the antibiotic resistance genes in the whole bacterial surfaces population of the five hospitals subjected to intervention (I1-I2 hospitals), in the pre-PCHS and PCHS phases of the study. Results are expressed as mean ± SD fold changes, compared to negative control values (for the pre-PCHS phase) and to pre-PCHS values (for PCHS phase). (B) Analysis of the antibiotic resistance genes in the whole surface microbiota of the external control (extC) hospital, in the 1st and in the 2nd 6-month periods of the study. First-period results are expressed as mean fold changes ± SD compared to negative control values (NTC); 2nd-period results are expressed as mean values ± SD compared 1st-period values, similarly to what performed for pre-PCHS and PCHS phases in the hospitals subjected to intervention.

(TIF)

Acknowledgments

The SAN-ICA Study Group includes: Maria D’Accolti, Micol Piffanelli, Irene Soffritti, Luca Lanzoni, Matteo Bisi, Antonella Volta, Maria Teresa Camerada, Piergiorgio Balboni, Carla Rognoni, Antonella Mastretti, Francesca Campanella, Paolo Grotto, Francesca Trozzi, Ermelinda Del Buono. We acknowledge the staffs of all the enrolled hospitals for their technical assistance.

Funding Statement

The authors declare that they received unrestricted funding from Copma Scrl (via Veneziani 32, 44124 Ferrara, Italy). However, this does not influenced in any way the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Availability

All relevant data are within the paper and its Supporting Information files, including data held in the public repository BioStudies (https://www.ebi.ac.uk/biostudies/), Accession No. S-BSST75.

References

1. Allegranzi B, Bagheri Nejad S, Combescure C, Graafmans W, Attar H, Donaldson L, et al. Burden of endemic health-care-associated infection in developing countries: systematic review and meta-analysis. Lancet. 2011;377(9761):228–41. 10.1016/S0140-6736(10)61458-4 [PubMed] [CrossRef] [Google Scholar]
2. Suetens C, Hopkins S, Kolman J, Diaz Högberg L. Point prevalence survey of healthcare associated infections and antimicrobial use in European acute care hospitals Stockholm, Sweden: European Centre for Disease Prevention and Control; 2013. July 1 [cited 2018 May 29]. In: ECDC Surveillance Report [www.ecdc.europa.eu]. Available from: https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/healthcare-associated-infections-antimicrobial-use-PPS.pdf [Google Scholar]
3. Mancini A, Verdini D, La Vigna G, Recanatini C, Lombardi FE, Barocci S. Retrospective analysis of nosocomial infections in an Italian tertiary care hospital. New Microbiol. 2016;39(3). [PubMed] [Google Scholar]
4. Capozzi C, Capozzi A, Visconti G, Ignisti F, Panà A, Mastrobuono I. Le infezioni ospedaliere: elementi di epidemiologia e prevenzione. Organizzazione Sanitaria. 2004;3(4):3–26. [Google Scholar]
5. Messineo A, Marsella LT. Biological hazards and healthcare-associated infections in Italian healthcare facilities: some considerations on inspections and accountability. Ann Ig. 2015;27(6):799–807. 10.7416/ai.2015.2073 [PubMed] [CrossRef] [Google Scholar]
6. Boyce JM. Environmental contamination makes an important contribution to hospital infection. J Hosp Infect. 2007;65 Suppl 2:50–4. [PubMed] [Google Scholar]
7. Mitchell BG, Dancer SJ, Anderson M, Dehn E. Risk of organism acquisition from prior room occupants: a systematic review and meta-analysis. J Hosp Infect. 2015;91(3):211–7. 10.1016/j.jhin.2015.08.005 [PubMed] [CrossRef] [Google Scholar]
8. Hota B. Contamination, disinfection, and cross-colonization: are hospital surfaces reservoirs for nosocomial infection? Clin Infect Dis. 2004;39(8):1182–9. 10.1086/424667 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
9. Dancer SJ. Controlling hospital-acquired infection: focus on the role of the environment and new technologies for decontamination. Clin Microbiol Rev. 2014;27(4):665–90. 10.1128/CMR.00020-14 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
10. Otter JA, Yezli S, French GL. The role played by contaminated surfaces in the transmission of nosocomial pathogens. Infect Control Hosp Epidemiol. 2011;32(7):687–99. 10.1086/660363 [PubMed] [CrossRef] [Google Scholar]
11. Dancer SJ. The role of environmental cleaning in the control of hospital-acquired infection. J Hosp Infect. 2009;73(4):378–85. 10.1016/j.jhin.2009.03.030 [PubMed] [CrossRef] [Google Scholar]
12. Weber DJ, Anderson D, Rutala WA. The role of the surface environment in healthcare-associated infections. Curr Opin Infect Dis. 2013;26(4):338–44. 10.1097/QCO.0b013e3283630f04 [PubMed] [CrossRef] [Google Scholar]
13. Weber DJ, Rutala WA. Role of environmental contamination in the transmission of vancomycin-resistant enterococci. Infect Control Hosp Epidemiol. 1997;18(5):306–9. [PubMed] [Google Scholar]
14. Kramer A, Schwebke I, Kampf G. How long do nosocomial pathogens persist on inanimate surfaces? A systematic review. BMC Infect Dis. 2006;6:130 10.1186/1471-2334-6-130 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
15. Weber DJ, Rutala WA, Miller MB, Huslage K, Sickbert-Bennett E. Role of hospital surfaces in the transmission of emerging health care-associated pathogens: norovirus, Clostridium difficile, and Acinetobacter species. Am J Infect Control. 2010;38(5 Suppl 1):S25–33. [PubMed] [Google Scholar]
16. Goodman ER, Platt R, Bass R, Onderdonk AB, Yokoe DS, Huang SS. Impact of an environmental cleaning intervention on the presence of methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci on surfaces in intensive care unit rooms. Infect Control Hosp Epidemiol. 2008;29(7):593–9. 10.1086/588566 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
17. Carling PC, Parry MF, Bruno-Murtha LA, Dick B. Improving environmental hygiene in 27 intensive care units to decrease multidrug-resistant bacterial transmission. Crit Care Med. 2010;38(4):1054–9. 10.1097/CCM.0b013e3181cdf705 [PubMed] [CrossRef] [Google Scholar]
18. Rutala WA, Weber DJ. Selection of the ideal disinfectant. Infect Control Hosp Epidemiol. 2014;35(7):855–65. 10.1086/676877 [PubMed] [CrossRef] [Google Scholar]
19. Almatroudi A, Gosbell IB, Hu H, Jensen SO, Espedido BA, Tahir S, et al. Staphylococcus aureus dry-surface biofilms are not killed by sodium hypochlorite: implications for infection control. J Hosp Infect. 2016;93(3):263–70. 10.1016/j.jhin.2016.03.020 [PubMed] [CrossRef] [Google Scholar]
20. Bock LJ, Wand ME, Sutton JM. Varying activity of chlorhexidine-based disinfectants against Klebsiella pneumoniae clinical isolates and adapted strains. J Hosp Infect. 2016;93(1):42–8. 10.1016/j.jhin.2015.12.019 [PubMed] [CrossRef] [Google Scholar]
21. Wand ME, Bock LJ, Bonney LC, Sutton JM. Mechanisms of increased resistance to chlorhexidine and cross-resistance to colistin following exposure of Klebsiella pneumoniae clinical isolates to chlorhexidine. Antimicrob Agents Chemother. 2017;61(1). [PMC free article] [PubMed] [Google Scholar]
22. Cornejo-Juarez P, Vilar-Compte D, Perez-Jimenez C, Namendys-Silva SA, Sandoval-Hernandez S, Volkow-Fernandez P. The impact of hospital-acquired infections with multidrug-resistant bacteria in an oncology intensive care unit. Int J Infect Dis. 2015;31:31–4. 10.1016/j.ijid.2014.12.022 [PubMed] [CrossRef] [Google Scholar]
23. Caini S, Hajdu A, Kurcz A, Borocz K. Hospital-acquired infections due to multidrug-resistant organisms in Hungary, 2005–2010. Euro Surveill. 2013;18(2). [PubMed] [Google Scholar]
24. Pettigrew MM, Johnson JK, Harris AD. The human microbiota: novel targets for hospital-acquired infections and antibiotic resistance. Ann Epidemiol. 2016;26(5):342–7. 10.1016/j.annepidem.2016.02.007 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
25. Al-Ghalith GA, Knights D. Bygiene: the new paradigm of bidirectional hygiene. Yale J Biol Med. 2015;88(4):359–65. [PMC free article] [PubMed] [Google Scholar]
26. Caselli E, Antonioli P, Mazzacane S. Safety of probiotics used for hospital environmental sanitation. J Hosp Infect. 2016;94(2):193–4. 10.1016/j.jhin.2016.06.021 [PubMed] [CrossRef] [Google Scholar]
27. Vandini A, Temmerman R, Frabetti A, Caselli E, Antonioli P, Balboni PG, et al. Hard surface biocontrol in hospitals using microbial-based cleaning products. PLoS One. 2014;9(9):e108598 10.1371/journal.pone.0108598 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
28. La Fauci V, Costa GB, Anastasi F, Facciolà A, GO C., Squeri R. An innovative approach to hospital sanitization using probiotics: in vitro and field trials. Microbial & Biochemical Technology. 2015;7(3):5. [Google Scholar]
29. Caselli E, D'Accolti M, Vandini A, Lanzoni L, Camerada MT, Coccagna M, et al. Impact of a probiotic-based cleaning intervention on the microbiota ecosystem of the hospital surfaces: focus on the resistome remodulation. PLoS One. 2016;11(2):e0148857 10.1371/journal.pone.0148857 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
30. Caselli E. Hygiene: microbial strategies to reduce pathogens and drug resistance in clinical settings. Microb Biotechnol. 2017;10(5):1079–83. 10.1111/1751-7915.12755 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
31. ECDC. European surveillance of healthcare associated infections in intensive care units: HAI-Net ICU protocol, version 1.02 2015. March 18 [cited 2018 May 29]. In: European Centre for Disease Prevention and Control, Publication & data [www.ecdc.europa.eu]. Available from: https://ecdc.europa.eu/en/publications-data/european-surveillance-healthcare-associated-infections-intensive-care-units-hai. [Google Scholar]
32. Donskey CJ. Does improving surface cleaning and disinfection reduce health care-associated infections? Am J Infect Control. 2013;41(5 Suppl):S12–9. [PubMed] [Google Scholar]
33. Hall L, Farrington A, Mitchell BG, Barnett AG, Halton K, Allen M, et al. Researching effective approaches to cleaning in hospitals: protocol of the REACH study, a multi-site stepped-wedge randomised trial. Implement Sci. 2016;11:44 10.1186/s13012-016-0406-6 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
34. Rodriguez-Acelas AL, de Abreu Almeida M, Engelman B, Canon-Montanez W. Risk factors for health care-associated infection in hospitalized adults: systematic review and meta-analysis. Am J Infect Control. 2017. [PubMed] [Google Scholar]
35. Ilinskaya ON, Ulyanova VV, Yarullina DR, Gataullin IG. Secretome of intestinal Bacilli: a natural guard against pathologies. Front Microbiol. 2017;8:1666 10.3389/fmicb.2017.01666 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
36. Wu L, Wu HJ, Qiao J, Gao X, Borriss R. Novel routes for improving biocontrol activity of Bacillus based bioinoculants. Front Microbiol. 2015;6:1395 10.3389/fmicb.2015.01395 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
37. Elshaghabee FMF, Rokana N, Gulhane RD, Sharma C, Panwar H. Bacillus as potential probiotics: status, concerns, and future perspectives. Front Microbiol. 2017;8:1490 10.3389/fmicb.2017.01490 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
38. McFarland LV. Use of probiotics to correct dysbiosis of normal microbiota following disease or disruptive events: a systematic review. BMJ Open. 2014;4(8):e005047 10.1136/bmjopen-2014-005047 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
39. Caselli E, D'Accolti M, Soffritti I, Zatelli MC, Rossi R, Degli Uberti E, et al. HHV-6A in vitro infection of thyrocytes and T cells alters the expression of miRNA associated to autoimmune thyroiditis. Virol J. 2017;14(1):3 10.1186/s12985-016-0672-6 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Articles from PLOS ONE are provided here courtesy of PLOS

-