Skip to main content

Survival of highly related ESBL- and pAmpC- producing Escherichia coli in broiler farms identified before and after cleaning and disinfection using cgMLST



Broiler chickens are frequently colonized with Extended-Spectrum Beta-Lactamase- (ESBL-) and plasmid mediated AmpC Beta-Lactamase- (pAmpC-) producing Enterobacterales, and we are confronted with the potential spread of these resistant bacteria in the food chain, in the environment, and to humans. Research focused on identifying of transmission routes and investigating potential intervention measures against ESBL- and pAmpC- producing bacteria in the broiler production chain. However, few data are available on the effects of cleaning and disinfection (C&D) procedures in broiler stables on ESBL- and pAmpC- producing bacteria.


We systematically investigated five broiler stables before and after C&D and identified potential ESBL- and pAmpC- colonization sites after C&D in the broiler stables, including the anteroom and the nearby surrounding environment of the broiler stables. Phenotypically resistant E. coli isolates grown on MacConkey agar with cefotaxime were further analyzed for their beta-lactam resistance genes and phylogenetic groups, as well as the relation of isolates from the investigated stables before and after C&D by whole genome sequencing. Survival of ESBL- and pAmpC- producing E. coli is highly likely at sites where C&D was not performed or where insufficient cleaning was performed prior to disinfection. For the first time, we showed highly related ESBL-/pAmpC- producing E. coli isolates detected before and after C&D in four of five broiler stables examined with cgMLST. Survival of resistant isolates in investigated broiler stables as well as transmission of resistant isolates from broiler stables to the anteroom and surrounding environment and between broiler farms was shown. In addition, enterococci (frequently utilized to detect fecal contamination and for C&D control) can be used as an indicator bacterium for the detection of ESBL-/pAmpC- E. coli after C&D.


We conclude that C&D can reduce ESBL-/pAmpC- producing E. coli in conventional broiler stables, but complete ESBL- and pAmpC- elimination does not seem to be possible in practice as several factors influence the C&D outcome (e.g. broiler stable condition, ESBL-/pAmpC- status prior to C&D, C&D procedures used, and biosecurity measures on the farm). A multifactorial approach, combining various hygiene- and management measures, is needed to reduce ESBL-/pAmpC- E. coli in broiler farms.

Peer Review reports


Colonization of broiler chickens with Extended-Spectrum Beta-Lactamase- (ESBL-) and plasmid mediated AmpC Beta-Lactamase- (pAmpC-) producing Enterobacterales is frequently detected throughout the livestock production chain [1,2,3,4,5,6,7]. Since day-old broiler chickens are already colonized with these resistant bacteria and high prevalence are frequently detected in broiler chicken farms, transmission routes were thoroughly investigated [8]. Horizontal and vertical transmission routes in the hatchery, from the environment to the farm, and from farm to farm have been described, illustrating the diversity of possible introduction routes into broiler chicken farms [9]. In addition to the different colonization routes, it was shown that even 101 − 102 colony forming units (cfu) of ESBL- and pAmpC- producing Escherichia coli (E. coli) are sufficient for colonization of day-old broiler chickens [10, 11]. In broiler farms, ESBL- and pAmpC- prevalence in newly hatched day-old broiler chickens appears to be low, and prevalence often increases throughout the fattening period [12]. Therefore, intervention measures are being investigated to prevent initial colonization or reduce the prevalence of ESBL- and pAmpC- producing bacteria in broiler chickens. Various interventions affecting broiler management or broiler gut composition have been studied with variable results on broiler chicken colonization rates [13,14,15,16,17]. Other important interventions include biosecurity measures that target introduction (external biosecurity) or spread within the farm and between flocks (internal biosecurity) [18, 19]. Here, cleaning and disinfection (C&D) procedures are of utmost importance to reduce the load of ESBL- and pAmpC- producing bacteria, but few data are available for broiler farms. Inadequate C&D after a fattening period can result in survival of ESBL- and pAmpC- producing bacteria in the broiler stable. Even small amounts of these resistant bacteria that are accessible to broiler chickens can lead to colonization of newly housed day-old broiler chickens. We therefore investigated the hazard of cleaned and disinfected broiler stables serving as a reservoir for ESBL- and pAmpC- producing bacteria. Our aim was to identify locations of ESBL- and pAmpC- survival within broiler stables and their nearby environment having the potential to subsequently colonize newly housed day-old broiler chickens.

Materials and methods

Sampled stables

Conventional broiler stables of identical construction belonging to the same company were screened for ESBL-/pAmpC- producing E. coli at the end of a broiler fattening period (day 23 to 31 of production). Fifty stables were screened and out of 21 ESBL-/pAmpC- positive broiler stables, five were selected on the basis of detected ESBL-/pAmpC- genes (see ‘Laboratory methods’) and information about the service period (e.g. duration of service period and disinfectants used, Table 1) to evaluate the efficacy of C&D procedures to eliminate ESBL-/pAmpC- producing E. coli from these stables. Sampling was conducted between April and October 2016. The sampled stables belonged to three different farms located within two kilometers of each other. Stable A and stable D belonged to one farm as well as stable B and stable C. Stable E was located on a third farm. On each farm, stables were freestanding with a minimum distance of 20 m from other stables, and two roads were used to transport goods to the stables (road A; feed, litter, hatchlings) or from the stables (road B; manure, fattened broiler chickens). Each broiler stable had an effective area of 1000 m2 with an attached anteroom that served as a hygiene lock and for storage of stable-specific equipment. Both the inner stables and the anterooms were made of concrete, having inside riveted steel plates on the ceiling and the walls (above a level of 50 cm from the floor).

Table 1 Disinfection procedures of the five investigated broiler stables

Cleaning and disinfection procedure

C&D of broiler stables was performed between two fattening periods by staff members who did not change throughout the study, each person being responsible for one step. The applied C&D corresponds to the commonly applied procedure for C&D in broiler stables and was performed within 6 to 12 days in four steps: (i) dry cleaning, (ii) wet cleaning, (iii) first disinfection, and (iv) second disinfection. Dry cleaning was performed with a front-end loader after housing-out of broiler chickens. A high-pressure washer with cold water and an all-purpose detergent (Grundreiniger, Weber-Chemie GmbH, Gladbeck, Germany) was used for wet cleaning and carried out by a single person. After a minimum drying time of at least 12 h after wet cleaning, the first disinfection was carried out with a fogging device using formaldehyde (Formaldehydlösung BIOZID PA 3, methanolarm stab., Chemie-Vertrieb GmbH & Co. KG, Hannover, Germany) and an exposure time of 4 h at > 28 °C. The second disinfection was performed 24 h after the first disinfection with two different disinfectants used for the five investigated houses (Aldekol DES® 03 or calcium hydroxide, Table 1). The disinfectants were selected by the broiler producer and represent agents commonly applied in livestock farming. We sampled each empty broiler stable 24 h after the second disinfection, right before preparations for the subsequent broiler flock.

Samplings of the stables

Each stable was sampled twice. The first samples were taken close to the end of the fattening period to assess the ESBL-/pAmpC- status of the current broiler flock using boot swab (VWR, Darmstadt, Germany; equal to ‘boot socks’) and pooled feces samples. The boot swab was taken by walking up and down the entire stable, while ten fresh droppings were collected along the entire length of the barn for the pooled feces sample. A stable was considered positive if ESBL- or pAmpC- producing E. coli were detected in the boot swab and/or the pooled feces sample. The second sampling was performed after C&D of the ESBL-/pAmpC- positive broiler stable and before preparations for the subsequent broiler flock (dry and empty stable without litter and feed). Stables’ inside (where broiler chickens are kept), the attached anteroom, and the surrounding environment were sampled with twisted gauze swabs (30 × 30 cm; Henry Schein, Berlin, Germany), boot swabs, and rinse water was collected (Supplementary Table 1). Prior to each sampling, autoclaved gauze swabs or boot swabs were transferred to sterile bags (Carl Roth, Karlsruhe, Germany) and moistened under sterile conditions with 5 ml or 10 ml phosphate buffered saline (PBS; Phosphate Buffered Saline tablets, Thermo Fisher Diagnostics GmbH, Wesel, Germany). No neutralizer for inactivation of disinfectant residues was added. Depending on the location, single gauze swabs or two to three pooled gauze swabs were used for one sample, and each gauze swab was used to sample a 10 × 10 cm area. Thirty-seven different sampling locations inside the stable, 21 different sampling locations in the anteroom and four different sampling locations in the surrounding environment were sampled to investigate various locations from ceiling to floor in each stable (Supplementary Table 1). To ensure comparability of samples between the five stables investigated, a precise sampling scheme was used that systematically structured the stable into sections (A - F and 1–3; Fig. 1, Supplementary Table 1). In addition, boot swabs were taken from stable’s floor, anteroom’s floor and from each of the four surrounding sides of the broiler stable (by walking up and down both sides of stable’s and anteroom’s floor and walking the entire length of each surrounding side), and one rinse water sample (75 ml) was collected from the anterooms’ floor drain using a sterile syringe. In total, the 69 samples reflect all the different materials (e.g. concrete, metal, plastic and organic origin), objects (e.g. removable and non-removable) and locations (e.g. floor, wall, ceiling of stable and anteroom) in and around a stable to systematically investigate the risk of ESBL-/pAmpC- detection in broiler farms (Supplementary Table 1).

Fig. 1
figure 1

Schematic top view of investigated broiler stables. Stables were structured in length (A-F) and width (1–3) into 18 sections for a reproducible sampling scheme after cleaning and disinfection. The sampling location “A1, B2, C3” is highlighted in dark grey as an example. Together with anteroom and equipment, each of the five investigated broiler stables was sampled following the sampling scheme. Boot swabs outside the broiler stable were taken from supply airside and exhaust airside as well as from road A (transport of goods to stable) and road B (transport of goods off stable)

Laboratory methods

Sample preparation. On the collection day, each gauze swab or boot swab was mixed with 20 ml or 200 ml Luria Bertani broth (LB; Carl Roth, Karlsruhe, Germany), and 3 ml of rinse water or 20 g of pooled feces were mixed in a ratio of 1:10 with LB before homogenizing for 2 min at 200 rounds per minute in a stomacher (except rinse water). Enrichment was performed 18–24 h at 37 °C and 10 µl were streaked out on MacConkey agar no. 3 (Fisher Scientific GmbH, Schwerte, Germany) containing 1 µg/ml cefotaxime (AppliChem, Darmstadt, Germany) and on agar plates without antibiotic supplementation. Agar plates were incubated 24 h at 37 °C for qualitative analysis of ESBL- and pAmpC- producing and non-resistant E. coli. Two phenotypically suspected ESBL- and pAmpC- producing E. coli isolates per colony morphology and sample were stored at -80 °C for further analyses. Additionally, samples taken after C&D were quantitatively and qualitatively analyzed on bile aesculin azide agar (BAA; Fisher Scientific GmbH, Schwerte, Germany) to detect residual enterococci. Enrichment was performed as described above and streaked out on BAA. For enterococci quantification, non-enriched gauze swab and boot swab samples were 1:10 diluted in PBS, and appropriate dilutions were plated on BAA and 50 ml of rinse water was filtered in a bottle top filter (Nalgene, Fisher Scientific GmbH, Schwerte, Germany) using 0.22 μm nitrocellulose filter (Carl Roth, Karlsruhe, Germany) which was then transferred to BAA agar. BAA agar plates were incubated for 48 h at 37 °C and two dilution levels were counted for quantification.

Species identification. Isolates’ species was identified using matrix-assisted laser desorption time of flight (MALDI Microflex LT and Biotyper database; Bruker Daltonics, Bremen, Gemany).

Phylogenetic grouping and determination of ESBL- and pAmpC- genes. Multiplex PCR was performed to determine the phylogenetic group [20] with modifications according to Projahn et al. [5]. To identify ESBL-/pAmpC- genes, real-time PCR was performed [21] and identified ESBL-/pAmpC- genes (blaTEM, blaSHV, blaCTX−M, blaCMY) were verified by sequencing of at least one isolate per sample according to Projahn et al. [5]. DNA was extracted by boiling method.

Pulsed-field gel electrophoresis. 27 E. coli isolates taken before (first sampling) and after C&D (second sampling) of the five investigated broiler stables harboring identical ESBL-/pAmpC- genes and belonging to the same phylogenetic group were further analyzed by pulsed-field gel electrophoresis (PFGE) according to Projahn et al. [5]. XbaI-PFGE patterns were grouped by presence or absence of more than three bands, resulting in a new PFGE-number (I-X). Minor differences in one to three bands resulted in a new subgroup labelled by the letters a-d (Supplementary Fig. 1).

Whole genome sequencing (WGS) and bioinformatic analyses. 21 E. coli isolates taken before (first sampling) and after C&D (second sampling) from stables B – E harboring the same resistance gene and phylogenetic group which clustered together in PFGE were whole genome sequenced to further investigate their phylogenetic relationship (six isolates did not cluster together in PFGE and were excluded). No samples from stable A were whole genome sequenced as they did not cluster together with any other isolate in PFGE. DNA was extracted with Invitrogen PureLink Genomic DNA Mini Kit (Thermo Fisher Scientific, Wesel, Germany) and WGS was performed by Illumina NextSeq 300-bp paired-end with a coverage between 80x and 100x (LGC Genomics GmbH, Berlin, Germany). Raw reads were uploaded to EnteroBase for data pre-processing and de novo assembly. Genome sequence data of E. coli isolates are publicly available in the NCBI Sequence Read Archive under the BioProject Accession number PRJNA1044304 ( Phylogenetic analyses of E. coli strains were performed using single nucleotide polymorphism (SNP) and core-genome multi-locus sequence typing (cgMLST) based on 2513 loci using EnteroBase and visualized with GrapeTree and MS Tree V2 algorithm [22, 23]. MLST sequence types and phylogroups were assigned using EnteroBase [24, 25]. Serotypes were predicted with EnteroBase and ambiguous results were checked against the EcOH database [26]. Antibiotic resistance determination was done with the E. coli functional genotyping tool (version 1.2) implemented in BioNumerics 7.6.3 (Applied Math, Sint-Martens-Latem, Belgium).

Statistical analyses

Statistical analyses were performed using SPSS Statistics 25 (IBM, New York, USA). All 69 different sampling locations from inside of the stable, anteroom and surrounding environment were grouped to 14 categories to analyze the detection frequencies of ESBL-/pAmpC- E. coli and quantities of enterococci (‘stable floor’, ‘stable wall’, ‘stable ceiling’, ‘stable interior’, ‘stable ventilation system’, ‘anteroom floor’, ‘anteroom wall’, ‘anteroom ceiling’, ‘anteroom interior’, ‘anteroom rinse water’, ‘anteroom door’, ‘surrounding floor’, ‘surrounding tractor tires’, ‘surrounding ventilation system’; Supplementary Table 1). The grouping was based on the three different areas ‘stable’, ‘anteroom’ and ‘surrounding environment’, which have different requirements for C&D (i.e. no C&D of the surrounding environment and gentle C&D of the anteroom due to electrical devices and standard C&D of the stable) resulting in a different probability of ESBL-/pAmpC- detection. Within these three areas, grouping was based on locations (e.g. floor, wall, ceiling) as it was expected that sampling locations near the floor have a higher probability of ESBL-/pAmpC- detection due to fecal contamination. Quantitative data of enterococci were log10 transformed and grouped into the categories no detection or low detection rate (detection in enrichment to < 107 cfu/swab) of enterococci and high detection rate (≥ 107 cfu/swab) of enterococci for statistical analyses. Supplementary Table 3 gives an overview of enterococci negative sampling locations, detection via enrichment and swabs with enterococci quantities of < 107 cfu/swab and ≥ 107 cfu/swab.

Chi-square test and Fisher’s exact test (if 25% or more of the cells have expected values below 5) were used to compare the detection frequencies of ESBL- and pAmpC- producing E. coli and enterococci inside the stable, anteroom and environment as well as between the defined categories. Concerning the comparison between ESBL-/pAmpC- E. coli (present or not) and enterococci (no or low detection (< 107 cfu/swab) vs. high detection rate (≥ 107 cfu/swab)), odds ratios including 95% confidence intervals (CI) were calculated.

The significance level was set to 0.05.


ESBL- and pAmpC- producing E. coli were detected in four out of five investigated broiler stables after C&D (stables B, C, D and E). Each stable showed highly related isolates taken before and after C&D using cgMLST. One stable positive for ESBL-producing E. coli before C&D was tested negative after C&D (stable A). Detected ESBL- and pAmpC- resistance genes in the four positive broiler stables were blaCTX−M−1 (stables C, D and E) and blaCMY−2 (stable B) with phylogenetic groups A/C (stable B), E/D (stable B) and F (stable B, C, D and E) detected in PCR (Supplementary Table 2). Using WGS, further resistance determinants of the investigated E. coli isolates from stable B, C and E were identified (Supplementary Table 2). Isolates belonging to one stable (stable B, C or E) harbored identical resistance genes, most likely due to acquisition of resistance plasmids. Interestingly from stable C, only three of the six isolates harbored additional resistance determinants. The remaining three isolates from stable C (one before C&D, two after C&D: tractor tires and road B) and all isolates from stable D only carried the ESBL resistance gene. Additionally, WGS revealed the phylogenetic groups A (stable B) and G (stable B, C, D and E) and serotypes O68:12 (stable B), O132:H4 (stable B), Onovel12:H4 (stable C and D) and O143:H4 (stable E) (Supplementary Table 2).

After C&D the ESBL-/pAmpC E. coli detection rate of all investigated samples was 4.9% (n = 17 out of 345). Highest ESBL-/pAmpC- E. coli rate was detected in the surrounding environment of the broiler stables (20%, n = 8 out of 40) which was significantly higher compared to the stable (3.7%, n = 7 out of 190) and the anteroom (1.7%, n = 2 out of 115) (p < 0.001, Fisher’s exact test, Supplementary Table 1). The ESBL-/pAmpC- detection rate was highest in the categories ‘stable floor’ (20% detection rate, n = 7 out of 35; floor cracks, transition between floor and wall, wooden board at entry) and ‘surrounding floor’ (35% detection rate, n = 7 out of 20; boot swabs), with ‘stable floor’ positive in four out of five tested stables and ‘surrounding floor’ positive in three out of five tested stables. Further ESBL-/pAmpC- positive categories were ‘anteroom floor’ (floor cracks), ‘surrounding tractor tires’ (tractor used for litter supply) and ‘anteroom interior’ (sink’s drain). All other categories were negative for ESBL-/pAmpC- E. coli in the five investigated broiler stables after C&D. Noteworthy, from ESBL-/pAmpC- negative sampling locations of the investigated broiler stables, a proportion of 10.0% (n = 34 out of 345) of samples showed growth of E. coli on non-selective agar plates after C&D, indicating a survival of E. coli on these spots (stable floor, cable lines, nipple drinkers, feed trough, anteroom floor, anteroom door, sink’s drain, rubber boots, dismantled metal boxes from stable, trash bin lid, ventilation flap, tractor tires and surrounding floor; Supplementary Table 1). The overall detection rate of (non-selectively grown) E. coli in all investigated samples, including ESBL-/pAmpC- positive samples, was 14.8% (n = 51 out of 345; with 47.5% (n = 19 out of 40) of samples from the surrounding environment, 11.1% (n = 21 out of 190) from stable, 9.6% (n = 11 out of 115) from anteroom). Although this study is focused on ESBL-/ pAmpC- E. coli it is worth mentioning that we detected one TEM-52 producing Klebsiella pneumoniae on farm C what implies that multiple ESBL-resistance genes can be present on one broiler farm at the same time (data not shown).

cgMLST revealed highly related isolates taken before and after C&D for stables B, C, D and E differing in less than 5 alleles by hierarchical clustering between related isolates for stables B, C and D. For stable E, isolates clustered with less than 10 allele’s difference in cgMLST (Hierarchical Cluster (HC) 10; Fig. 2). SNP analyses for isolates of each cgMLST cluster confirmed phylogenetic relationships (Supplementary Fig. 2). In stable B, two distinct cgMLST clusters of pAmpC- E. coli were detected, differing in at least 2381 alleles, with both clusters harboring isolates from before and after C&D. The pAmpC- E. coli after disinfection of stable B were detected in the stable (cluster 1, transition floor/wall) and the close environment of the broiler stable (cluster 2, boot swabs). Samples from stable C harbored highly related isolates after C&D from inside the stable (transition floor/wall), anteroom (sink’s drain) and the surrounding environment (tractor tires and boot swab). All isolates of stable C clustered together with all isolates of stable D in the cgMLST analysis. SNP analysis of these isolates revealed no distinct separation of the stable C and stable D isolates indicating transmission events between the two stables (Supplementary Fig. 2). As the stables belong to different broiler farms and ESBL- positive samples were only detected inside stable D (transition floor/wall and floor cracks), a transmission from farm (stable C) to farm (stable D) via shared equipment or the presence of a common ESBL- source (e.g. tractor) is highly likely. The described transmission event is also supported by the additional resistance determinants of the isolates from stable C and stable D (see above), as the resistance profile of the tractor tires and road B isolates from stable C is identical to the profile of stable D isolates. Finally isolates from stable E clustered together with isolates from inside stable (transition floor/wall and floor cracks) and the close environment of the broiler stable (boot swabs).

The qualitative analysis of enterococci revealed significant differences between the locations with higher enterococci detection rates in the surrounding environment and lower detection rates in the anteroom (p < 0.001, chi-squared test, Supplementary Table 3). Comparing the enterococci quantities of the different categories, the highest detection rates were found in ‘stable floor’, ‘surrounding floor’ and ‘surrounding tractor tires’ (p < 0.001, chi-squared test), with more frequent detection of high enterococci quantities (107 cfu/ swab) in these categories. The probability to detect ESBL-/ pAmpC- E. coli was 73.7 times higher in samples where high enterococci quantities were detected compared to those where no or low quantities were detected (p < 0.001, chi-squared test; 95% CI 9.6–566.5).


The applied C&D procedures did not eliminate ESBL-/pAmpC- producing E. coli from the broiler stables studied, and highly related isolates were detected before and after C&D. Only one of five positive broiler stables before C&D was negative for ESBL-/pAmpC- producing E. coli after C&D (stable A). This result is in accordance with other studies in which elimination of E. coli was not achieved after C&D procedures [27,28,29]. Each of the four ESBL- and pAmpC- positive broiler stables after C&D had highly related resistant E. coli isolates before and after C&D according to cgMLST, emphasizing survival and/or transmission of bacteria on broiler farms (stable B - E). Carryover of resistant bacteria from positive broiler stables to the anteroom and surrounding environment or vice versa is possible. Especially when broilers are housed and the bacterial load in the stable is high or when broilers are housed-out and C&D procedures start (e.g. dry cleaning), carryover from the stable is possible as broiler chicken flocks still regularly tested positive for ESBL-/pAmpC- producing E. coli [30,31,32], as do environmental samples from the adjacent area of broiler stables, which can lead to the introduction of bacteria into the stable at any time. Environmental samples carrying ESBL-/pAmpC- E. coli include barn equipment, dust, air, rinse water, surface water, and soil [8, 12, 33,34,35,36]. We identified cracks in the stable’s floor, the transition between the stable’s floor and wall, the sink’s drain in the anteroom, and tractor tires used for litter supply, as well as the broiler stable’s surrounding floor as sources of ESBL-/ pAmpC- producing bacteria. The ESBL-/pAmpC- positive spots indicate that the floor and equipment in contact with the floor require special attention during C&D procedures. One possible approach is to repair defects in the stable’s floor and a thorough C&D of the equipment and surrounding floor whenever possible (e.g. roads for the transportation of goods). It was already shown that poor floor quality (e.g. cracks in concrete floor) is a significant risk factor for broilers’ mortality, as these spots cannot be properly cleaned and disinfected, resulting in the survival of pathogens that can then cause disease in subsequent flocks [37]. Since the required colonization dose for ESBL-/ pAmpC- producing bacteria in day-old broiler chickens is as low as 101 − 102 cfu, the likelihood of ESBL-/pAmpC- colonization of newly housed broiler flocks originating from such sites is very high [10]. A major reason for bacterial survival is contamination of stables with organic matter that has not been removed by cleaning procedures prior to disinfection, as organic matter reduces the effectiveness of disinfectants, resulting in bacterial survival [27, 38]. In agreement, non-selectively grown E. coli was mostly detected on spots with contact to the floor or close to the floor. A possible explanation for the detection of E. coli on ventilation flaps, trash can lids, and cable lines may be settled dust that was not removed by C&D, as E. coli can survive in dust for several years [39]. In general, we have to consider that each farm or integrated production has different biosecurity measures, different cleaning protocols with personnel performing C&D or external companies contracted for different work [40]. The C&D in this study was performed under field conditions and reflects the common procedure in broiler farming. The cleaning agents and disinfectants used were suitable for preventive measures, whereby Aldekol Des® 03 was only used at temperatures above 10 °C (not effective at lower temperatures) [41]. For all biosecurity measures, including C&D, compliance by staff members is of utmost importance to achieve satisfactory results. Once a biosecurity concept has been developed, it must be communicated to all employees and consistently monitored for compliance and improvement. As we detected highly related isolates in distinct stables (stable C and D), an improvement of the biosecurity measures seems possible. In addition, each broiler stable has different structural preconditions, such as the age of the buildings and the construction material, with all of the above factors affecting the initial bacterial contamination of the broiler stable and the effectiveness of C&D procedures [27, 42].

Isolates before and after C&D from stables B - E clustered together with less than 10 alleles difference in cgMLST, which displays short-term, single source transmissions of E. coli [43]. In stable B, two cgMLST clusters of pAmpC- E. coli were detected, both harboring isolates from before and after C&D. The results from stable B demonstrate that distinct ESBL-/pAmpC- E. coli strains can be detected together in a broiler stable [12] and that ESBL-/pAmpC- E. coli can survive C&D procedures within the stable (stable B, cgMLST cluster 1, HC10, Fig. 2) or be disseminated to the close surrounding environment of the stable or introduced into the stable from the close surrounding environment (stable B, cgMLST cluster 2, HC10, Fig. 2). Similarly, in stables D and E, survival of ESBL- E. coli within the stable and dissemination from or introduction into the stable was detected. Horizontal transmission routes at farm level were described extensively, but survival of ESBL-/pAmpC- E. coli after C&D of broiler stables with detection of highly related isolates in and around the stables was not shown previously [9, 12, 44, 45]. The high relation of isolates from stables C and D with two indistinguishable isolates using cgMLST and the fact that all samples from stables C and D cluster together in cgMLST and differ by less than 5 alleles underscores the risk of transmission of resistant E. coli between distinct broiler farms or the presence of a common source leading to ESBL- detection in both stable C and D. Another important factor in resistance transmission is plasmid-mediated resistance. Plasmids play a major role in E. coli with dissemination of these mobile genetic elements from resistant strains to other bacteria, as well as in plasmid acquisition [46]. In our study, isolates from three of four ESBL-/pAmpC- positive broiler stables harbored additional resistance determinants after C&D, most likely due to plasmid acquisition. The complex interactions of resistance gene transfer are well seen in isolates of stable C, as only three of six investigated isolates harbored additional resistance determinants. Since plasmid acquisition is not depicted in cgMLST or SNP analyses, isolates of stables C and D with distinct resistance profiles are undistinguishable in these analyses. Biosecurity measures were implemented in all broiler farms studied. Measures included restricted access of vehicles and visitors to the farm, with a hygiene lock to be passed before entering. Each broiler stable had a shoe disinfection bath, sink, stable-specific equipment (except heavy machines), and rodent control was applied. However, strict compliance to biosecurity measures is required [47]. One possible route of transmission from stable C to stable D or the presence of a common source is the contaminated tractor tires that tested positive for ESBL- E. coli in stable C after C&D procedures, as some equipment such as heavy machines was shared between the farms. Further biosecurity measures such as cleaning and disinfection baths for shared heavy machines before entering distinct farms could minimize the risk of inter-farm transmission if exposure times are met.

Fig. 2
figure 2

Core genome multilocus sequence typing (cgMLST) of ESBL- and pAmpC- producing E. coli isolates. Isolates are from four investigated broiler stables B – E before and after cleaning and disinfection which clustered together in PFGE. Isolates are colored according to the cgMLST hierarchical clustering (HC) 10, indicating that the four clusters include all strains with links no more than 10 alleles difference. Information on each sample is given in Supplementary Table 2. * Isolates detected before cleaning and disinfection of broiler stables

To date, only one study investigated the presence of ESBL-producing E. coli after C&D procedures in broiler stables, but did not investigate transmission routes during C&D procedures [29]. The study by Benameur et al. (2023) detected ESBL-producing E. coli in 3.8% of 104 stables investigated using 104 enriched swab samples after C&D, but did not use selective laboratory methods to detect ESBL-/pAmpC- producing E. coli because they investigated the presence of multidrug-resistant E. coli in cleaned and disinfected poultry stables. The overall detection rate of E. coli after C&D was very high in the study by Benameur et al. with 88.46%. One reason for the high detection rate could be the long vacancy period of three to six weeks between C&D and the sampling, which makes (re)contamination likely. We used moistened gauze and boot swabs to sample the cleaned and disinfected stables and selectively enriched the samples overnight because we expected low growth of ESBL-/pAmpC- E. coli immediately after C&D procedures. Moistened gauze swabs and enrichment can lower the detection limit compared to contact agar plates, which are regularly used to evaluate C&D procedures [28]. Another approach to lower the detection limit is the addition of neutralizers to sampling materials and agar plates, as these inactivate disinfectant residues. As expected, only 4.9% of all samples tested were positive for ESBL-/pAmpC- E. coli after enrichment, and the positive sampling locations were difficult to access. In a study by Luyckx et al. [38], C&D procedures reduced the E. coli detection rate of enriched samples in broiler stables from 93% before cleaning to 7% after disinfection. Since high E. coli detection rates are expected before C&D due to fecal contamination, the detection rate should decrease significantly due to the applied C&D measures. We did not investigate the quantitative occurrence of ESBL-/pAmpC- E. coli before C&D as we aimed to trace possible transmission routes from positive broiler flocks during C&D procedures. In our study, the ESBL-/pAmpC- detection rate after C&D in all samples tested was 4.9% with a total E. coli detection rate of 14.8%. The high detection rate is caused by the samples from the surrounding environment (20% ESBL-/pAmpC- E. coli; 47.5% total E. coli), as C&D procedures are only applied in the stable and anteroom and not in the surrounding environment. It is worth mentioning that we only included initially ESBL-/ pAmpC- positive broiler stables in our study, which might increase the detection rate after C&D compared to initially negative broiler stables [40].

We showed a significantly higher detection probability of ESBL-/pAmpC- E. coli on sampling locations with high enterococci quantities. In our analysis, due to power restrictions we did not consider that samples from different locations in and around a stable and farm might be related to each other, but we assume that the effect would be statistically significant even in a random factors model. Enterococcus spp. is commonly used as an indicator bacterium for fecal contamination and has been shown to detect organic matter after C&D procedures [28, 48]. Samples from the floor and from the immediate environment of broiler chickens, where organic matter can accumulate (e.g. cracks and crevices in the floor), showed higher enterococci quantities in our study (Supplementary Table 3). These spots contribute to residual contamination and biofilm formation and require spot treatment or double-strength disinfection for effective C&D [27]. It was shown that the effectiveness of disinfectants against biofilms is lower compared to planktonic cells and a longer exposure time of the disinfectants is recommended [49]. As the effectiveness of commercially available disinfectants is usually only tested on planktonic cells, the test methods must be adapted to practical scenarios and biofilms must also be examined. Enterococci quantities were also high on spots that were not regularly cleaned and disinfected (e.g. wooden board at the stable’s entrance, sink’s drain in the anteroom, tractor tires used for litter supply, and the broiler stables’ surrounding environment). C&D of these spots may be too laborious or there may be no knowledge of the need for C&D procedures at these spots, resulting in bacterial survival. Therefore, detection of high Enterococcus spp. quantities at cleaned and disinfected spots in broiler stables is eligible to estimate the ESBL- and pAmpC- E. coli presence at these spots.


Showing the survival of ESBL- and pAmpC- producing E. coli after C&D, we conclude that complete elimination of these resistant bacteria is unlikely in conventional broiler stables. Strict biosecurity measures and the implementation of improved disinfection procedures could enhance C&D results, but it seems likely that locations which are difficult of access remain positive. Prophylactic C&D procedures do not aim to “sterilize” broiler stables, but to reduce overall infection pressure and control infectious agents in livestock farming [28, 50]. C&D should be considered as one tool that, together with other hygiene and management measures, can reduce the presence of resistant bacteria in broiler farms.

Data availability

All data supporting this article are included within the article and the provided supplementary files. Genome sequence data of E. coli isolates are publicly available in the NCBI Sequence Read Archive under the BioProject Accession number PRJNA1044304 (


  1. van Hoek AHAM, Veenman C, Florijn A, Huijbers PMC, Graat EAM, de Greeff S et al. Longitudinal study of ESBL Escherichia coli carriage on an organic broiler farm. J Antimicrob Chemother. 2018; 73:3298–304. PMID: 30219829.

  2. Dame-Korevaar A, Fischer EAJ, Stegeman A, Mevius D, van Essen-Zandbergen A, Velkers F et al. Dynamics of CMY-2 producing E. coli in a broiler parent flock. Vet Microbiol. 2017; 203:211–4. Epub 2017/03/19. PMID: 28619146.

  3. Dierikx CM, van der Goot JA, Smith HE, Kant A, Mevius DJ. Presence of ESBL/AmpC-producing Escherichia coli in the broiler production pyramid: a descriptive study. PLoS ONE. 2013; 8:e79005. PMID: 24244401.

  4. Osman KM, Kappell AD, Elhadidy M, ElMougy F, El-Ghany WAA, Orabi A et al. Poultry hatcheries as potential reservoirs for antimicrobial-resistant Escherichia coli: A risk to public health and food safety. Sci Rep. 2018; 8:5859. Epub 2018/04/11. PMID: 29643424.

  5. Projahn M, Daehre K, Roesler U, Friese A. Extended-Spectrum-Beta-Lactamase- and Plasmid-Encoded Cephamycinase-Producing Enterobacteria in the Broiler Hatchery as a Potential Mode of Pseudo-Vertical Transmission. Appl Environ Microbiol. 2017; 83. PMID: 27795309.

  6. Cohen Stuart J, van den Munckhof T, Voets G, Scharringa J, Fluit A, Hall ML-V. Comparison of ESBL contamination in organic and conventional retail chicken meat. Int J Food Microbiol. 2012; 154:212–4. PMID: 22260927.

  7. von Tippelskirch P, Gölz G, Projahn M, Daehre K, Friese A, Roesler U et al. Prevalence and quantitative analysis of ESBL and AmpC beta-lactamase producing Enterobacteriaceae in broiler chicken during slaughter in Germany. Int J Food Microbiol. 2018; 281:82–9. PMID: 29890401.

  8. Laube H, Friese A, von Salviati C, Guerra B, Käsbohrer A, Kreienbrock L et al. Longitudinal monitoring of extended-spectrum-beta-lactamase/AmpC-producing Escherichia coli at German broiler chicken fattening farms. Appl Environ Microbiol. 2013; 79:4815–20. PMID: 23747697.

  9. Dame-Korevaar A, Fischer EAJ, van der Goot J, Stegeman A, Mevius D. Transmission routes of ESBL/pAmpC producing bacteria in the broiler production pyramid, a literature review. Prev Vet Med. 2019; 162:136–50. PMID: 30621893.

  10. Robé C, Blasse A, Merle R, Friese A, Roesler U, Guenther S. Low Dose Colonization of Broiler Chickens With ESBL-/AmpC- Producing Escherichia coli in a Seeder-Bird Model Independent of Antimicrobial Selection Pressure. Front Microbiol. 2019; 10:2124. PMID: 31572330.

  11. Dame-Korevaar A, Fischer EAJ, van der Goot J, Velkers F, van den Broek J, Veldman K et al. Effect of challenge dose of plasmid-mediated extended-spectrum β-lactamase and AmpC β-lactamase producing Escherichia coli on time-until-colonization and level of excretion in young broilers. Vet Microbiol. 2019; 239:108446. PMID: 31767080.

  12. Daehre K, Projahn M, Semmler T, Roesler U, Friese A. Extended-Spectrum Beta-Lactamase-/AmpC Beta-Lactamase-Producing Enterobacteriaceae in Broiler Farms: Transmission Dynamics at Farm Level. Microb Drug Resist. 2018; 24:511–8. PMID: 28981392.

  13. Robé C, Daehre K, Merle R, Friese A, Guenther S, Roesler U. Impact of different management measures on the colonization of broiler chickens with ESBL- and pAmpC- producing Escherichia coli in an experimental seeder-bird model. PLoS ONE. 2021; 16:e0245224. Epub 2021/01/07. PMID: 33411808.

  14. Persoons D, Haesebrouck F, Smet A, Herman L, Heyndrickx M, Martel A et al. Risk factors for ceftiofur resistance in Escherichia coli from Belgian broilers. Epidemiol Infect. 2011; 139:765–71. PMID: 20587122.

  15. Ceccarelli D, van Essen-Zandbergen A, Smid B, Veldman KT, Boender GJ, Fischer EAJ et al. Competitive Exclusion Reduces Transmission and Excretion of Extended-Spectrum-β-Lactamase-Producing Escherichia coli in Broilers. Appl Environ Microbiol. 2017; 83. PMID: 28314728.

  16. Saliu E-M, Ren H, Boroojeni FG, Zentek J, Vahjen W. The Impact of Direct-Fed Microbials and Phytogenic Feed Additives on Prevalence and Transfer of Extended-Spectrum Beta-Lactamase Genes in Broiler Chicken. Microorganisms. 2020; 8. PMID: 32110912.

  17. Roth N, Hofacre C, Zitz U, Mathis GF, Moder K, Doupovec B et al. Prevalence of antibiotic-resistant E. coli in broilers challenged with a multi-resistant E. coli strain and received ampicillin, an organic acid-based feed additive or a synbiotic preparation. Poult Sci. 2019; 98:2598–607. PMID: 30690607.

  18. Caekebeke N, Jonquiere FJ, Ringenier M, Tobias TJ, Postma M, van den Hoogen A et al. Comparing Farm Biosecurity and Antimicrobial Use in High-Antimicrobial-Consuming Broiler and Pig Farms in the Belgian-Dutch Border Region. Front Vet Sci. 2020; 7:558455. Epub 2020/10/30. PMID: 33330687.

  19. Becker E, Projahn M, Burow E, Käsbohrer A. Are There Effective Intervention Measures in Broiler Production against the ESBL/AmpC Producer Escherichia coli. Pathogens. 2021; 10. Epub 2021/05/15. PMID: 34063430.

  20. Clermont O, Christenson JK, Denamur E, Gordon DM. The Clermont Escherichia coli phylo-typing method revisited: improvement of specificity and detection of new phylo-groups. Environ Microbiol Rep. 2013; 5:58–65. PMID: 23757131.

  21. Roschanski N, Fischer J, Guerra B, Roesler U. Development of a multiplex real-time PCR for the rapid detection of the predominant beta-lactamase genes CTX-M, SHV, TEM and CIT-type AmpCs in Enterobacteriaceae. PLoS ONE. 2014; 9:e100956. PMID: 25033234.

  22. Zhou Z, Alikhan N-F, Sergeant MJ, Luhmann N, Vaz C, Francisco AP et al. GrapeTree: visualization of core genomic relationships among 100,000 bacterial pathogens. Genome Res. 2018; 28:1395–404. Epub 2018/07/26. PMID: 30049790.

  23. Zhou Z, Alikhan N-F, Mohamed K, Fan Y, Achtman M. The EnteroBase user’s guide, with case studies on Salmonella transmissions, Yersinia pestis phylogeny, and Escherichia core genomic diversity. Genome Res. 2020; 30:138–52. Epub 2019/12/06. PMID: 31809257.

  24. Beghain J, Bridier-Nahmias A, Le Nagard H, Denamur E, Clermont O. ClermonTyping: an easy-to-use and accurate in silico method for Escherichia genus strain phylotyping. Microb Genom. 2018; 4. Epub 2018/06/19. PMID: 29916797.

  25. Wirth T, Falush D, Lan R, Colles F, Mensa P, Wieler LH et al. Sex and virulence in Escherichia coli: an evolutionary perspective. Mol Microbiol. 2006; 60:1136–51. PMID: 16689791.

  26. Ingle DJ, Valcanis M, Kuzevski A, Tauschek M, Inouye M, Stinear T et al. In silico serotyping of E. coli from short read data identifies limited novel O-loci but extensive diversity of O:H serotype combinations within and between pathogenic lineages. Microb Genom. 2016; 2:e000064. Epub 2016/07/11. PMID: 28348859.

  27. Castañeda-Gulla K, Sattlegger E, Mutukumira AN. Persistent contamination of Salmonella, Campylobacter, Escherichia coli, and Staphylococcus aureus at a broiler farm in New Zealand. Can J Microbiol. 2020; 66:171–85. Epub 2019/11/13. PMID: 31721603.

  28. Luyckx K, Dewulf J, van Weyenberg S, Herman L, Zoons J, Vervaet E et al. Comparison of sampling procedures and microbiological and non-microbiological parameters to evaluate cleaning and disinfection in broiler houses. Poult Sci. 2015; 94:740–9. PMID: 25681611.

  29. Benameur Q, Gervasi T, Dahloum L, Rechidi-Sidhoum N, Boutaiba Benklaouz M, Yakubu A. Multidrug-resistant Escherichia coli isolated from cleaned and disinfected poultry houses prior to day-old chick placement. J Environ Qual. 2023. Epub 2023/02/04. PMID: 36738475.

  30. Apostolakos I, Mughini-Gras L, Fasolato L, Piccirillo A. Assessing the occurrence and transfer dynamics of ESBL/pAmpC-producing Escherichia coli across the broiler production pyramid. PLoS ONE. 2019; 14:e0217174. Epub 2019/05/17. PMID: 31100096.

  31. Päivärinta M, Latvio S, Fredriksson-Ahomaa M, Heikinheimo A. Whole genome sequence analysis of antimicrobial resistance genes, multilocus sequence types and plasmid sequences in ESBL/AmpC Escherichia coli isolated from broiler caecum and meat. Int J Food Microbiol. 2020; 315:108361. Epub 2019/10/29. PMID: 31734617.

  32. European Food Safety Authority (EFSA), European Centre for Disease Prevention and Control (ECDC). The European Union Summary Report on Antimicrobial Resistance in zoonotic and indicator bacteria from humans, animals and food in 2020/2021. EFS2. 2023; 21:e07867. Epub 2023/03/06. PMID: 36891283.

  33. Blaak H, van Hoek AHAM, Hamidjaja RA, van der Plaats RQJ, Kerkhof-de Heer L, de Roda Husman AM et al. Distribution, Numbers, and Diversity of ESBL-Producing E. coli in the Poultry Farm Environment. PLoS ONE. 2015; 10:e0135402. Epub 2015/08/13. PMID: 26270644.

  34. Hering J, Frömke C, von Münchhausen C, Hartmann M, Schneider B, Friese A et al. Cefotaxime-resistant Escherichia coli in broiler farms-A cross-sectional investigation in Germany. Prev Vet Med. 2016; 125:154–7. PMID: 26783199.

  35. Projahn M, Daehre K, Semmler T, Guenther S, Roesler U, Friese A. Environmental adaptation and vertical dissemination of ESBL-/pAmpC-producing Escherichia coli in an integrated broiler production chain in the absence of an antibiotic treatment. Microb Biotechnol. 2018. PMID: 29345080.

  36. Siller P, Daehre K, Thiel N, Nübel U, Roesler U. Impact of short-term storage on the quantity of extended-spectrum beta-lactamase-producing Escherichia coli in broiler litter under practical conditions. Poult Sci. 2020; 99:2125–35. PMID: 32241498.

  37. van Limbergen T, Sarrazin S, Chantziaras I, Dewulf J, Ducatelle R, Kyriazakis I et al. Risk factors for poor health and performance in European broiler production systems. BMC Vet Res. 2020; 16:287. Epub 2020/08/12. PMID: 32787841.

  38. Luyckx KY, van Weyenberg S, Dewulf J, Herman L, Zoons J, Vervaet E et al. On-farm comparisons of different cleaning protocols in broiler houses. Poult Sci. 2015; 94:1986–93. PMID: 26047671.

  39. Schulz J, Ruddat I, Hartung J, Hamscher G, Kemper N, Ewers C. Antimicrobial-Resistant Escherichia coli Survived in Dust Samples for More than 20 Years. Front Microbiol. 2016; 7:866. PMID: 27375587.

  40. Mo SS, Kristoffersen AB, Sunde M, Nødtvedt A, Norström M. Risk factors for occurrence of cephalosporin-resistant Escherichia coli in Norwegian broiler flocks. Prev Vet Med. 2016; 130:112–8. PMID: 27435654.

  41. Deutsche Veterinärmedizinische Gesellschaft. DVG-Desinfektionsmittelliste für den Tierhaltungsbereich [cited 3 Mar 2024].

  42. Hancox LR, Le Bon M, Dodd CER, Mellits KH. Inclusion of detergent in a cleaning regime and effect on microbial load in livestock housing. Vet Rec. 2013; 173:167. Epub 2013/07/09. PMID: 23839725.

  43. Achtman M, Zhou Z, Charlesworth J, Baxter L. EnteroBase: hierarchical clustering of 100 000s of bacterial genomes into species/subspecies and populations. Philos Trans R Soc Lond B Biol Sci. 2022; 377:20210240. Epub 2022/08/22. PMID: 35989609.

  44. van Hoek AHAM, Dierikx C, Bosch T, Schouls L, van Duijkeren E, Visser M. Transmission of ESBL-producing Escherichia coli between broilers and humans on broiler farms. J Antimicrob Chemother. 2020; 75:543–9. PMID: 31800052.

  45. Laube H, Friese A, von Salviati C, Guerra B, Rösler U. Transmission of ESBL/AmpC-producing Escherichia coli from broiler chicken farms to surrounding areas. Vet Microbiol. 2014;172:519–27.

    Article  CAS  PubMed  Google Scholar 

  46. Poirel L, Madec J-Y, Lupo A, Schink A-K, Kieffer N, Nordmann P et al. Antimicrobial Resistance in Escherichia coli. Microbiol Spectr. 2018; 6. PMID: 30003866.

  47. Gelaude P, Schlepers M, Verlinden M, Laanen M, Dewulf J. Biocheck.UGent: a quantitative tool to measure biosecurity at broiler farms and the relationship with technical performances and antimicrobial use. Poult Sci. 2014; 93:2740–51. Epub 2014/09/05. PMID: 25193257.

  48. Byappanahalli MN, Nevers MB, Korajkic A, Staley ZR, Harwood VJ. Enterococci in the environment. Microbiol Mol Biol Rev. 2012; 76:685–706. PMID: 23204362.

  49. Olja Todorić L, Pezo Ljubiša, Šarić V, Kolarov A, Varga. Ivana Čabarkapa and Sunčica Kocić-Tanackov. Comparison of the Efficiency of Selected Disinfectants against Planktonic and Biofilm Populations of Escherichia coli and Staphylococcus aureus. 2023.

  50. Wales AD, Gosling RJ, Bare HL, Davies RH. Disinfectant testing for veterinary and agricultural applications: A review. Zoonoses Public Health. 2021; 68:361–75. Epub 2021/05/03. PMID: 33939312.

Download references


Not applicable.


This study was funded by the Federal Ministry of Food and Agriculture ( and is part of the EsRAM research consortium. Grant number awarded to UR: 2817701014. We acknowledge support by the Open Access Publication Fund of the Freie Universität Berlin. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and Affiliations



AF and UR designed the study. CR, AB, MP and KB performed the samplings and laboratory work. RM, KB and CR performed the statistical analyses. CR, MP and AF performed the bioinformatics analyses. CR evaluated the final data and wrote the manuscript. All authors have read and approved the final draft of the article.

Corresponding author

Correspondence to Caroline Robé.

Ethics declarations

Ethics approval and consent to participate

The environmental samples were taken with the consent of the broiler farm owner. No institutional or national ethics committee approval was required for this study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Robé, C., Projahn, M., Boll, K. et al. Survival of highly related ESBL- and pAmpC- producing Escherichia coli in broiler farms identified before and after cleaning and disinfection using cgMLST. BMC Microbiol 24, 143 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: