Canine chronic inflammatory enteropathy implicates multifactorial pathogenesis where immunological dysregulation and gut microbiota changes have a central role. Most sequencing-based taxonomic studies have been focused on the fecal microbiota. However, the analysis of these samples does not provide complete information regarding the composition of the small intestine affected by this canine disease. Therefore, in this study, we aimed to characterize the intestinal bacterial microbiota in dogs with inflammatory bowel disease (IBD) (n = 34) by means of duodenal biopsies and fecal samples collected at the time of the diagnosis and to compare those to a group of healthy dogs (n = 12) using the 16S ribosomal RNA (16S rRNA) gene-targeted sequencing (Illumina MiSeq platform). Our study showed that IBD dogs presented differences in the fecal bacterial communities when compared with healthy dogs, with a lower relative abundance of Prevotellaceae (p = 0.005), Prevotella (p = 0.002), and Prevotellaceae Ga6A1 group (0.006); Erysipelotrichales (p = 0.019), Candidatus Stoquefichus (p < 0.001), Erysipelotrichaceae (p = 0.011), and Allobaculum (p = 0.003); Lachnospiraceae NK4A136 group (p = 0.015), Sellimonas (p = 0.042), Oscillospirales (p = 0.037), Oscillospiraceae UCG-005 (p < 0.001), Faecalibacterium (p = 0.028), and Fournierella (p = 0.034); Acidaminococcales, Acidaminococcaceae, and Phascolarctobacterium (p = 0.001); Aeromonadales (p = 0.026), Succinivibrionaceae (p = 0.037), and Succinivibrio (p = 0.031). On the other hand, a higher relative abundance of Enterococcaceae (Enterococcus; p = 0.003), Streptococcaceae (Streptococcus, p = 0.021), Enterobacterales (p = 0.027), Enterobacteriaceae (p = 0.008), and Escherichia-Shigella (p = 0.011) was detected. Moreover, when evaluating α-diversity, the dogs with IBD showed lower diversity in terms of richness and abundance of species (observed species [p = 0.031] and Shannon index [p = 0.039]). Furthermore, fecal microbiota in dogs with IBD was significantly different from healthy dogs (p = 0.006). However, only a few taxa relative abundance shifts (lower Rubrobacteria, Rubrobacterales, Rubrobacteriaceae, and Rubrobacter [p = 0.002]; Cyanobacteria [p = 0.010], Vampirivibrionia, Obscuribacterales, and Obscuribacteraceae [p = 0.005]; Neisseriaceae [p = 0.004] and Conchiformibius [p = 0.003]) were observed when assessing duodenal-associated microbiota of dogs with IBD. Thus, even if the bowel inflammation mainly affects the small intestine in the IBD-affected dogs of the study, fecal specimens may constitute a better sample due not only to their easy availability but also in terms of searching for bacterial taxa as biomarkers for canine IBD. The use of different diets in the study can also have a partial influence on the microbiota composition. Future studies encompassing multi-omics approaches should evaluate the functionality in both levels to unravel the pathophysiology of canine IBD.