Yogurt consumption shows a wide range of effects on the gut microbial composition, and correlation of components in yogurt with the changes of gut microbia remains largely uncharacterized. We aimed to determine the effect of brown yogurt (SSN) on the composition of the gut microbiota and to explore the effects of the major components. We performed a randomized study of 70 healthy adults to compare the effects of SSN and standard probiotic-containing yogurt (YJD) during a 28-day intervention and a 10-day follow-up period. The results showed that the SSN group showed significant increases in the butyrate-producer Akkermansia muciniphila, Ruminococcus, and Veillonella (p < 0.05), whereas the YJD group showed increases in the butyrate-producer Megasphaera, Anaerostipes, and Eubacterium. There were reductions in the potential pathogens Haemophilus parainfluenzae and Gemmiger formicilis in both groups (p < 0.05). The SSN group had more Faecalibacterium prausnitzii, Prevotella copri, Bifidobacterium and B. longum than the YJD group (p < 0.001), but fewer Bacteroides, unspecified Clostridiales and Coprococcus eutactus (p < 0.01). These differences might be at least in part explained by the higher concentrations of monosaccharide, palmitoleic acid, and glutamine synthetase adenyltransferase in the SSN product (p < 0.05), which were positively associated with F. prausnitzii (p ≤ 0.001) and B. longum (p < 0.05), and negatively associated with C. eutactus (p < 0.01). The single strain of starter culture and lower content of polyunsaturated fatty acids (PUFA) in the SSN product were also related to the different changes of gut microbia, and the taxa F. prausnitzii, Bifidobacterium and B. longum were negatively associated with starter culture and PUFA (p < 0.01). These findings suggested that SSN is rich in prebiotic components and might be beneficial for healthy adults. Furthermore, bacterial taxa with potential health benefits could be encouraged through improving the formulation and technology used to produce the dairy products.