https://orcid.org/0000-0002-4488-4042 
Figures
Abstract
Molecular-bacterial vaginosis (BV) is characterized by low levels of vaginal Lactobacillus species and is associated with higher risk of sexually transmitted infections (STI). Perceived psychosocial stress is associated with increased severity and persistence of infections, including STIs. American Indians have the highest rates of stress and high rates of STIs. The prevalence of molecular-BV among American Indian women is unknown. We sought to evaluate measures of psychosocial stress, such as historic loss (a multigenerational factor involving slavery, forced removal from one’s land, legally ratified race-based segregation, and contemporary discrimination) and their association with the vaginal microbiota and specific metabolites associated with BV, in 70 Northwestern Plains American Indian women. Demographics, perceived psychosocial stressors, sexual practices, and known BV risk factors were assessed using a modified version of the American Indian Service Utilization, Psychiatric Epidemiology, Risk and Protective Factors Project survey. Self-collected mid-vaginal swabs were profiled for bacterial composition by 16S rRNA gene amplicon sequencing and metabolites quantified by targeted liquid-chromatography mass spectrometry. Sixty-six percent of the participants were classified as having molecular-BV, with the rest being either dominated by L. crispatus (10%) or L. iners (24%). High levels of lifetime trauma were associated with higher odds of having molecular-BV (adjusted Odds Ratio (aOR): 2.5, 95% Credible Interval (CrI): 1.1–5.3). Measures of psychosocial stress, including historic loss and historic loss associated symptoms, were significantly associated with lifestyle and behavioral practices. Higher scores of lifetime trauma were associated with increased concentrations of spermine (aFC: 3.3, 95% CrI: 1.2–9.2). Historic loss associated symptoms and biogenic amines were the major correlates of molecular-BV. Historical loss associated symptoms and lifetime trauma are potentially important underlying factors associated with BV.
Citation: Borgogna J-LC, Anastario M, Firemoon P, Rink E, Ricker A, Ravel J, et al. (2021) Vaginal microbiota of American Indian women and associations with measures of psychosocial stress. PLoS ONE 16(12): e0260813. https://doi.org/10.1371/journal.pone.0260813
Editor: Andrew R. Dalby, University of Westminster, UNITED KINGDOM
Received: February 5, 2021; Accepted: November 17, 2021; Published: December 10, 2021
Copyright: © 2021 Borgogna et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: In the United States, American Indian tribes have the legal right to regulate all affairs in their tribal community, including research and the legal right of sovereignty and governance over data conducted from community members. The tribal community from which these data come from have asserted sovereignty over the data collected from their community members, and therefore have ownership over their data. We invite interested researchers with reasonable requests to reach out to the contacts described herein (Carl Yeoman, Elizabeth Rink, or the Montana IRB (irb@montana.edu) wherein we would confidentially point them towards the Tribal IRB for a request.
Funding: This research was supported by the National Institutes of Health's National Institutes of General Medical Sciences (NIGMS; https://www.nigms.nih.gov/) and National Institute for Allergy & Infectious Disease (NIAID; https://www.niaid.nih.gov/) under award numbers U54GM115371 (CY), P20GM103474 (CY), and R21AI111145 (CY) and by the Montana Agricultural Experiment Station (https://agresearch.montana.edu/maes.html) (CY). The sponsors played no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: JR is the co‐ founder of LUCA Biologics, a biotechnology company focusing on translating microbiome research into live biotherapeutic drugs for women’s health. All other authors have declared that no competing interests exist.
Introduction
Racial disparities in the burden of sexually transmitted infections (STIs) continue to persist at unacceptable levels in the US [1–4]. American Indian (AI) women are disproportionately affected, having rates of Neisseria gonorrhoeae (GC), Chlamydia trachomatis (CT), Treponema pallidum (the causative agent of syphilis), Trichomonas vaginalis (TV) and HIV infection 1.5–6 times higher than non-Hispanic white women [1–4]. Racial misidentification, insufficient levels of surveillance, and under-reporting of urogenital diseases affect AI populations, and it is estimated that the prevalence of these STIs are at least 30–57% higher than documented [5–10]. Population-based surveys clearly indicate that these inequities cannot be attributed to behavioral practices, including sexual activity, age of first intercourse, condom use, and drug use [11–13], and the mechanisms affecting racial disparities in susceptibilities to female reproductive tract infections remain unclear.
We hypothesized that the vulnerability of AI women to high rates of STIs may be partially mediated by psychosocial stress [14]. Psychosocial stress occurs when an individual perceives an event as taxing, threatening, or otherwise harmful, and their coping resources as inadequate [15–18]. Thus, a stressful experience is individualized, and experiencing an event may be perceived as stressful for some but not others [15, 17, 19]. Stressors can take on many forms, including short-term demands such as occupational burdens and deadlines [15, 17–19] and chronic demands such as those that persist over an extended duration of time (e.g., caring for a partner with dementia), traumatic life events (e.g., experiencing a sexual assault, early childhood trauma, or death of a loved one), and demands derived from poverty or perceived discrimination [15, 17–20].
AIs have the highest rate of stress exposure in the US [19, 21–25], and this has been tied to historical loss (HLS) associated trauma, a multigenerational factor involving slavery, forced removal from one’s land, legally ratified race-based segregation, and contemporary discrimination [26–30]. Factors that may exacerbate the high rates of stress include the lowest national employment rates [31, 32], highest rates of poverty [33, 34], exposure to violent acts and repeated loss [14, 33], with death rates being as much as 46% higher than that of non-Hispanic whites [35]. AI women are particularly imperiled, reporting the most extreme rates of interpersonal (emotional, physical, or sexual abuse, and emotional, or physical neglect) trauma within the US [14, 26, 36–39], with over 84% reporting having had experienced some form of violence during their lifetime [40, 41].
Exposure to psychosocial stress, including community-level stressors, can elicit physiological responses leading to suppressed immune function, and increased susceptibility, severity, and persistence of infections, including STIs [16, 22, 42–51]. Psychosocial stress has also been linked to bacterial vaginosis (BV) [22, 45–47, 52], the most common gynecological morbidity in the US [53]. Molecular features of BV include a paucity of protective lactobacilli [54–57], an abundance of diverse anaerobes [58–61], a high vaginal pH (>4.5) [55], and high concentrations of metabolites called biogenic amines (BAs) [62–65]. The low-lactobacilli state of BV (recently termed molecular-BV [58]) is associated with adverse gynecological outcomes, including an elevated risk of acquisition of HIV [57, 58, 66, 67] and other STIs [68–71]. Conversely, vaginal microbiota dominated by protective lactobacilli are typically associated with positive health outcomes [62, 72–75].
Most BAs commonly associated with BV are microbially produced via specific amino acid decarboxylation reactions, a common acid- and oxidative-stress resistance mechanism [76]. BAs negatively affect growth and lactic acid production of lactobacilli [77], increase odds of BV [77], are associated with smoking [78], CT [79], and HPV [80], and improve the growth and virulence of several pathogens, including GC [81–84]. Further, physiological and emotional stress are known to elicit increases in levels of BAs in some tissues, including the brain and liver—a phenomenon described as the polyamine-stress response [85–88]. To date, it has not been examined if stress can affect BA concentrations in the vagina.
We hypothesized that high psychosocial stress may be associated with both vaginal microbiota and vaginal BAs, creating a vaginal environment susceptible to STIs. While the role of the vaginal microbiota in sexual and reproductive health is widely established, we are unaware of any study describing the vaginal microbiota of AI women. Therefore, in this study we utilized 16S rRNA gene amplicon sequencing to characterize the vaginal microbiota of AI women in a small cross-sectional cohort. We also performed targeted metabolomics to quantify the vaginal BAs and their amino acid precursors, and evaluated their relationship to HLS trauma, psychosocial stress, and behavioral practices of AI women.
Results
Participant characteristics and demographics
Participant characteristics and demographics are reported in Table 1. This study did not assess gender identity. The average age of participants was 30 years (SD: 7 years, range: 18–45 years). All participants reported being enrolled in a federally recognized Northern Plains Tribe, with four participants (5.7%) identifying as additionally having associate tribal membership. Most participants reported having had some college education (50%) compared to those that reported either having some or no high school (31%) or having received a high-school diploma or GED (19%) (p-values estimated using pairwise two-sample proportion z-test<0.04). Most participants identified homemaker as their sole (40%) or primary occupation (13%), self-reported as recipients of public assistance (63%) and indicated that they cohabitated with either a relative (40%) or their partner (30%).
Participants were predominately characterized by molecular-BV
The vaginal microbiota of the seventy participants was characterized into three major vaginal community state types (CSTs), two of which were dominated by either Lactobacillus crispatus (CST I) or L. iners (CST III), and one comprised an abundance of anaerobes, including Gardnerella, Prevotella, Atopobium, and Sneathia species consistent with molecular-BV (CST IV). The frequency of CST varied significantly among the participants (Fig 1). Molecular-BV was most common with 66% of participants having CST IV microbiota, 24% having CST III, and 10% having CST I (Poisson regression, p-values <0.0001). None of the demographic variables listed in Table 1 were significantly associated with CST.
Participants with molecular-BV had higher concentrations of BAs
We then quantified the concentration of common vaginal biogenic amines and their amino acid precursors (Table 2). All amino acids measured (arginine, lysine, ornithine, and tryptophan) were significantly higher in participants with CST I microbiota when compared to participants with either CST III or IV microbiota (all q-values <0.05). Participants with molecular-BV had higher concentrations of cadaverine, putrescine, trimethylamine, and tyramine compared to those with either CST I or III microbiota (q-values <0.05). Spermine and spermidine were higher in participants with CST I compared to participants with CST IV microbiota (q-values <0.05), but there was no significant difference between the concentrations observed in participants with CST III microbiota compared to those with either CST I or IV (q-values <0.05).
Association between CST and participant behavioral and sexual practices
We next assessed the association between participant behavioral and sexual health practices with CST (Table 3). Of the 54 (79%) participants that identified as smokers, most (66%) had molecular-BV compared to 25% with CST III microbiota. Of the 18 (26%) participants that reported a history of douching, 78% had molecular-BV compared to those with CST III (17%) or CST I microbiota (5%). However, these proportions were not significantly different.
Most participants (97%) reported that they only, or primarily, engaged in intercourse with men, with zero participants reporting that they engaged in sex with both men and women. All but six participants reported that that they were currently sexually active (defined as engaging in vaginal, oral, or anal intercourse). Most participants (66%) reported engaging in intercourse with a partner (not a spouse), and having had one sexual partner in the last month (76%). On average, participants reported having had intercourse 9 times (range 0–60 times) within the last month. Twenty-six participants (37%) reported the use of either alcohol or drugs (or a combination) either by (7%) or with (65%) their partner during their last sexual encounter. Eighteen participants (26%) reported that their partner was suspected or known to use needles and 29 participants (41%) reported that their partner was suspected or known to be non-monogamous. The frequency of CST varied among those that reported alcohol and/or drugs being involved in the last sexual encounter, and among those that reported their partner was suspected or known to either use needles or be non-monogamous, with molecular-BV being most frequent compared to either CST I or CST III (q-values <0.03). In models unadjusted and adjusted for CST, none of the variables listed in Table 3 were significantly associated with the measured vaginal metabolites.
Association between psychosocial stress, vaginal microbiota, and vaginal metabolites
We next interrogated the relationship between measures of psychosocial stress (adult attachment, social support, self-worth, depression, lifetime trauma, historical loss, and historical loss associated symptoms) with the vaginal metabolites and vaginal microbiota. An exploratory PCA of vaginal metabolites and measures of psychosocial stress indicated that participants with molecular-BV primarily clustered together (Fig 2). The first component largely corresponded to the collective measures of all amino acids and the non-BV associated BAs (spermidine and spermine), which were corelated with each other. The second component largely consisted of the collected measure of the BV-associated BAs (agmatine, cadaverine, putrescine, tyramine, trimethylamine, and trimethylamine oxide), as well as social support, historic loss, historic loss associated anxiety, the adult attachment measures of security and avoidance, and depression.
Circles represent individuals color coded by CST and black arrows indicate the two principal component loading vectors.
Bayesian multinomial logistic regression modeling adjusted and unadjusted for smoking status indicated that higher levels in BV-associated BAs and lifetime trauma were associated with increased odds of having molecular-BV compared to a grouped category of participants with either CST I or III microbiota (Fig 3) (BV-associated BAs adjusted odds ratio (aOR): 1.36, 95% credible interval (CrI): 1.1–1.8; lifetime trauma aOR: 2.3, 95% CrI: 1.1–5.25). Higher levels of amino acids and non-BV-associated BAs (spermine and spermidine) were associated with decreased odds of having molecular-BV compared to participants with either CST I or III microbiota (non-BV associated BAs aOR: 0.8, 95% CrI: 0.6–0.97; Amino Acid aOR: 0.56, 95% CrI: 0.39–0.78).
Circles represent point estimates while bars indicate 95% credible intervals. Odds ratios were adjusted for smoking status.
Measures of psychosocial stress are associated with lifestyle and sexual practices
We next evaluated the associations between measures of psychosocial stress with lifestyle and sexual practices (Fig 4). Unadjusted Bayesian regression analysis indicated that participants with greater social support scores were less likely to report having been with a partner known or suspected to use needles (aOR: 0.11, 95% CrI: 0.02–0.55) or known or suspected to be non-monogamous (aOR: 0.13, 95% CrI: 0.02–0.61). Both greater social support and self-worth scores were significantly associated with decreased odds of reporting as a smoker. Participants with high historic loss associated symptoms (HLAS) and adult attachment-avoidance scores were significantly associated with greater odds of being a smoker (HLAS aOR: 3.37, 95% CrI: 1.1–11.2; adult attachment-avoidance aOR: 5.1, 95% CrI:1.1–27.9). High scores of adult attachment-avoidance were also significantly associated with reporting having had intercourse with a non-monogamous partner (aOR: 15.3, 95% CrI: 2.8–114.3) within the past year, while participants with higher scores of lifetime trauma and HLAS were more likely to report drugs and/or alcohol being used in last sexual encounter (Lifetime trauma aOR: 2.2, 95% CrI: 1.1–5.1; HLAS aOR: 4.2, 95% CrI: 1.4–13.3). Notably, high scores of lifetime trauma were associated with douching (aOR: 2.23, 95% cRI: 1.1–5.57).
Circles represent point estimates while bars indicate 95% credible intervals.
Association between measures of psychosocial stress and vaginal metabolites
The association between measures of psychosocial stress and select vaginal metabolites were evaluated using Bayesian regression and adjusting for CST and smoking status given their independent associations with these metabolites (S1 Table). For every one-unit increase in social support scores, participants had higher concentrations of lysine (adjusted fold change (aFC): 4.5, 95% CrI: 1.3–15.9). Similarly, every one-unit increase in adult attachment-security scores was associated with higher concentrations of tryptophan (aFC: 4.5, 95% CrI: 1.5–14.1). Finally, every one-unit increase in self-worth score was associated with lower concentrations of trimethylamine oxide (aFC: 0.1, 95% CrI: 0.02–0.69). High scores of historic loss were also significantly associated with cadaverine (aFC: 3.1, 95% CrI: 0.97–9.7). Notably, higher scores of lifetime trauma were associated with higher concentrations of spermine (aFC: 3.3, 95% CrI: 1.2–9.2).
Measurement model
Finally, we evaluated the relationship between the three identified latent variables, historic loss (HLS) associated anxiety/depression, interpersonal reliance, and vaginal BAs with the dependent variable (molecular-BV vs. a grouped category of participants with either CST I or III microbiota). The measurement model (S1 Fig) had an adequate fit to the data, with a Tucker-Lewis Index of 0.967, a comparative fit index of 0.977, a root-mean-square error of approximation of 0.057, and an insignificant chi-squared test of model fit (χ2 = 39.3, df = 32, p = 0.174). Standardized factor loadings for all three latent variables were statistically significant and averaged >0.7 for each construct. The largest correlation was between interpersonal dependency and the historical loss associated anxiety/depression construct (-0.299, p = 0.017). The correlations between the vaginal BA construct, the historical loss associated anxiety/depression construct, and the interpersonal reliance construct, were positive but relatively smaller (<0.2).
Path diagram and direct effect estimates of the final SEM
The pathways from the HLS associated anxiety/depression construct, the interpersonal reliance construct, the vaginal biogenic amines construct, and CST-IV status are shown in Fig 5. The overall fit of the model was χ2 = 32.97, df = 41, p = 0.81 with a comparative fit index of 1.0, a Tucker-Lewis Index of 1.2, and a root-mean-square error of approximation of 0.00. The extreme fit values reflect the small sample size (n = 70). An increase in the vaginal biogenic amine construct score was appreciably related to an increase in the probability of having molecular-BV (effect (E) = 0.83, standard error (SE) = 0.16). The interpersonal dependency construct did not appear to have any notable effect on the vaginal biogenic amines construct but the HLS associated anxiety/depression construct showed an effect on both the interpersonal dependency construct (E = -0.36, SE = 0.12) and the vaginal biogenic amines construct (E = 0.23, SE = 0.12). The total indirect effect of HLS associated anxiety on CST-IV status was 0.16 (SE = 0.095).
Pathway to molecular-BV (CST IV) through interpersonal reliance and vaginal biogenic amines.
Discussion
It is well-established that the vaginal microbiota play an important role in mediating female reproductive and sexual health [89–91]. In 2011, Ravel et al. observed that the vaginal microbiota of U.S. reproductive-aged women clustered broadly into five community state types (CSTs) and demonstrated that the prevalence of each CST varied across race and ethnicities [91]. In a study of nearly 400 U.S women, CST IV/molecular-BV microbiota was most prevalent within Hispanic (38%) and non-Hispanic Black (40%) women compared to non-Hispanic white (10%) and Asian (20%) women [91]. Consistently, this study reveals that the majority of AI participants were also dominated by CST IV/molecular-BV (66%)—the highest reported prevalence within a given population, to date. This high rate of molecular-BV corresponds to the higher observed rates of STIs among AI populations [1], and is consistent with findings that BV is associated with increased risk for STIs [68, 92–94].
The reasons for the racial disparities in molecular-BV prevalence are not clear, with differences in sexual and hygienic behaviors having previously been proposed [95–98]. Hygienic practices such as douching and individual behaviors such as the involvement of drugs and alcohol during the last sexual encounter, and condom use have been shown to vary by race and socioeconomic status and to increase odds of BV [99–101]. Consistently, we observed a history of douching, the involvement of drugs and/or alcohol during the last sexual encounter, partner suspected or known to use needles, and partner concurrency to be significantly associated with molecular-BV (CST-IV microbiota). These data support emerging evidence that points towards a need to investigate the association between sexual networks, of which partner concurrency is a marker, and STI and BV prevalence in AI populations [13, 102]. It is critical to acknowledge that previous studies have found that both individual-level sexual practices and sociodemographic factors do not fully account for observed disparities in STIs among other minority populations [1, 11–13, 103–105] and thus may not fully account for the discrepancies in molecular-BV observed herein.
Our data and others suggest that differences in perceived stress may play a mediating role in development of BV [22, 46, 106, 107]. The effect of stress may enrich for this non-optimal CST by impairing the host immune response [106, 107], inhibiting the deposition of glycogen [106], or by increasing sexual practices [107], such as having sex while influenced by alcohol or drugs [108]. Consistently, we observed that measures of psychosocial stress were associated with behavioral practices such as smoking, douching, and having sex with partners suspected or known to use needles. A large percentage of our participants reported the involvement of drugs or alcohol during their last sexual encounter (37.1%), of which 70% had CST IV microbiota. Further, most of these participants also had high scores of historic loss associated symptoms and a history of smoking. Smoking itself has been shown to have a dose-dependent association with both CST-IV and BV [109] and among adolescent and reproductive-age women, ‘coping with stress’ and ‘stress relief’ are the most commonly cited motives for smoking [110–113]. Among African American women, smoking has been associated with the frequency and perceptions of overall race-related, individual, and cultural race-related stress [23].
One measure of psychosocial stress, lifetime trauma was associated with molecular-BV. Further, several measures of psychosocial stress, including HLAS and lifetime trauma were associated with sexual and lifestyle behaviors, including smoking and partner concurrency. Further, the HLS anxiety construct had a positive effect on the vaginal biogenic amine construct comprised of putrescine, cadaverine and trimethylamine [55, 61, 114, 115]; and importantly, we observed associations between psychosocial stress and BV-associated BAs or their precursors. This is important as it identifies specific risk factors directly related to BAs, and indirectly to molecular-BV, and along with the observed high concentrations of these metabolites in women with molecular-BV compared to CST I or CST III women, is consistent with previously reported literature [62–64, 78, 80, 116].
Conventionally thought of as biomarkers of BV [63, 117], these biogenic amines are generally produced from bacterial amino acid decarboxylation reactions in the presence of acid stress [62, 63, 76, 118]. These reactions are known to decrease environmental acidity, and are hypothesized to promote the colonization and outgrowth of BV-associated bacteria [63, 76]. Further, biogenic amines have been shown to increase pathogen virulence [76, 118], protect pathogens, including Neisseria gonorrhoeae, against innate immune defenses [84, 119], and have been associated with infections of the most prevalent bacterial and viral STIs [79, 80]. Several studies have demonstrated increases in levels of BAs in response to stressful stimuli [85, 87, 88, 120]. Conversely, spermidine and spermine tend to be associated with ameliorative effects against stressful stimuli [88], and some research indicates that diets rich in spermidine and spermine may enhance mitochondrial function and autophagy within the brain thereby stave-off age-related cognitive decline and may calm neuroinflammatory responses [121–123]. This may explain the positive association we observed between lifetime trauma and spermine. While we hypothesize psychosocial stress impacts vaginal metabolites, it is likely multi-faceted as behavioral practices such as smoking have been previously associated with BAs [78]. It is possible that, in addition to measures of psychosocial stress, other behavioral and sexual practices affect the BAs leading to a vaginal environment less optimal for vaginal lactobacilli and at increased susceptibility to STIs. However, the hypothesized biological mechanism is currently unknown, and mechanistic studies designed to establish the relationship between psychosocial stress, vaginal metabolites, and vaginal bacteria are merited.
There are several limitations to our study. First, while this is the first study characterizing the vaginal microbiota of AI women, our sample size is relatively small, limiting our power to detect differences among some variables. This is particularly evident in our unadjusted analysis of participant measures, as we visually observe differences in distributions across variables, but while approaching statistical significance, they did not meet the criteria of α = 0.05. This also hindered our ability to have adequate representation across factors (such as CSTs I and III). Secondly, we are limited by the cross-sectional and self-reported nature of our data; therefore, we cannot make causal inferences regarding the impact on stress upon the vaginal microbiota, nor can we capture short term fluctuations of the metabolites or microbiota. Further, the data described herein may not be representative of other tribal communities or sovereign nations and must be interpreted in context of Northwestern Plains American Indian Populations. It is critical to note that although we identified important variables within our model, this does not preclude the value of other variables. Finally, we did not measure biological stress variables, such as cortisol, and it is possible that individuals may have under-reported or underestimated their stress.
In summary, we were able to describe the frequency of molecular-BV among reproductive-aged AI women, identify the association between measures of psychosocial stress and lifestyle and sexual practices upon the vaginal microbiota and select vaginal metabolites.
Methods
Positionality statement
This research was conducted by Indigenous and non-Indigenous scholars from universities throughout the United States and the tribal college on the reservation where the study took place.
Study setting
This study took place on a reservation in a western state in the Northern Plains region of the United States. There are nearly as estimated 6,000 enrolled tribal members living on the reservation who are descendants of the Nakoda, Nakota, Nakona, Lakota, and Dakota Nations.
Ethics statement
This study, including the recruitment of participants and research presented herein, builds upon a 14-year partnership between tribal members and researchers affiliated with Montana State University (MSU), as previously described [124–132]. Briefly, this collaborative partnership utilizes a community based participatory research framework to combine Indigenous expertise in traditional knowledge, contemporary reservation culture, and local tribal resources with Westernized research skills in sexual and reproductive health research among tribal members. In the United States, American Indian tribes have the legal right to regulate all affairs in their tribal community, including research and the legal right of sovereignty and governance over data conducted from community members. In the sovereign nation described herein, the tribal Institutional Review Board (IRB) oversees and makes decisions about research and data as directed by the Tribal Executive Board. The Tribal IRB approved the undertaking of this research and its publication, as indicated, with the requirement that neither the reservation nor tribes be explicitly named. A community advisory board (CAB) made up of five tribal members provided oversight and guidance on the study, including development of the survey and data collection instruments, study design, and interpretation of the qualitative and quantitative data. The CAB and the IRB in the tribal community where the study took place provided ethical approval for the study, reviewed, and approved the language within the manuscript for publication prior to submitting (FWA00019355, approved 08/11/16). The MSU IRB also provided ethical oversight, guidance, and approval for this study (FWA00000165, approved 09/08/16). All participants provided written informed consent. All research was conducted in compliance with relevant guidelines, and regulations.
Population and sampling information
This is a cross-sectional study that primarily used word-of-mouth to recruit Northwestern Plains American Indian women in 2016. Networking through agencies on the reservation was also used. For confidentiality purposes, we use the general descriptors of Northern Plains rather than specific tribal names. Eligibility for participation was restricted to include enrolled tribal members who were 18–45 years of age and having a menstrual cycle. Participants were ineligible be involved if they were pregnant or reported use of antibiotics in the prior three months, a total of 70 participants were recruited. This study did not assess gender or sexual identity. Participants were asked to provide detailed information regarding sexual preferences, habits, and practices, including the gender that they usually engage in sex with, and to disclose patterns and concerns experienced with their partner (e.g., whether a partner was jealous a lot of the time); however, the gender of the partner was left ambiguous. Following recruitment, participants were surveyed in a one-on-one interview with a trained social scientist who was assigned female at birth (survey described below). Prior to administration of the survey, participants were informed that this could elicit physiological stress and were advised that they do not have to participate and could rescind consent at any time. At the completion of the interview, participants were provided instructions and a private stall to self-collect mid-vaginal swabs that were immediately frozen at -20°C before being moved to -80°C storage within 6 hours. Samples remained stored at -80°C until use. All participants received $100 VISA gift card on completion of the study.
Survey measures
The interview followed a modified version of the American Indian Service Utilization, Psychiatric Epidemiology, Risk and Protective Factors Project (AI-SUPERPFP) survey [133] (S1 File), an in-depth quantitative and qualitative survey covering demographics, perceived psychosocial stress and stressors, gynecological health, sexual behavior practices, and known STI and BV risk factors. Measures obtained from the survey and used in this analysis are described below:
Self-worth.
Self-worth was measured using a series of six questions adapted from the Rosenberg Self-Esteem Scale [134] to assess global self-esteem. Responses were provided on a 5-point Likert Scale ranging from (1) disagree to (5) agree. An overall Self-worth score was calculated for each participant by reverse-coding negative Likert questions and summing responses across all questions, where higher scores indicate high self-worth.
Adult attachment.
Adult attachment was measured using the Measure of Attachment Qualities (MAQ) Instrument Scale [135] which consisted of 14 questions and included subscales to evaluate 1) secure attachment and 2) avoidance. Responses were provided on a 4-point scale ranging from (1) disagree to (4) agree. Scores for each subscale were calculated by reverse-coding negative responses and summing the responses per participant, where higher scores indicate secure attachment tendencies or relationship avoidance tendencies.
Social support.
Social support was measuring using the Multidimensional Scale of Perceived Social Support (MSPSS) [136]. The MSPSS is a validated scale that asks participants 12 questions about their perceived social support (i.e., emotional support, comfort, and assistance in making decisions) from family, friends, or significant others. To evaluate whether there was an association between social support upon the vaginal microbiota and vaginal BAs, we reverse-coded negative Likert questions, and calculated a total Social Support Score per participant, where higher scores indicate more social support. For the CFA and SEM, a latent factor, interpersonal reliance, was identified during preliminary psychometric tests, and was parceled to include four items: 1) I have friends with whom I can share my joys and sorrows; 2) I can talk about my problems with my friends; 3) if someone wants to fight me, my family will stand by me; and 4) there is someone I can borrow money from in an emergency. This latent factor was included in the CFA with the assumption that interpersonal reliance could potentially mitigate effects of anxiety and depression.
Depression.
Depression was assessed by an adapted Center for Epidemiological Studies Depression Scale [137, 138]. This scale assessed nine symptoms with responses ranging from 1) never or rarely to 4) most of the time, or all of the time. A total depression score was calculated by reverse-coding negative responses and summing the responses per participant; higher scores indicate higher scores of depression.
Lifetime trauma.
Adverse life events were measured using a 28-question construct which assesses various types of exposure to stressors including verbal and emotional abuse, physical injuries, and suicide. Each response was scored dichotomously (0 = unexposed, 1 = exposed). An overall summary score, representing the total of number of adverse life events, was calculated where higher scores indicating greater exposure to lifetime trauma.
Historical loss and historical loss associated symptoms.
In addition to the measures described above, the frequency with which participants think about historical loss (loss of culture, land, and people as a result of colonization) was measured using the Historical Loss Scale (HLS) [139], which consists of 12 questions with responses provided on a 6-point frequency-based Likert scale, ranging from (1) never to (6) several times a day. The frequency by which participants experience particular symptoms associated with historical loss was measured using the Historical Loss Associated Symptoms (HLAS) scale [139]. The HLAS scale comprises 12 questions with responses provided on a 5-point scale ranging from (1) never to (5) always. Higher scores of both scales indicate a participant self-reported thinking about HLS or being affected by HLAS often.
Taxonomic assignment and community state type (CST) profiling
DNA extraction, PCR amplification and sequencing of the 16S rRNA gene V3-V4 and bioinformatics processing using the dada2 pipeline [140] followed the methods described by Holm et al. 2018 [141]. Taxonomic assignments were performed using the RDP Classifier trained with SILVA (version 128) and speciation of specific vaginal bacteria with SpeciateIT (ravel-lab.org/speciateit). For each sample, CST classifications were assigned as previously described, using Jensen-Shannon divergence and Ward linkage hierarchical clustering [90, 91].
Sample preparation for metabolomics
Samples were eluted from Copan Nylon Flocked dry swabs using 70% methanol and analyzed with the 5500 QTRAP LC/MS/MS system (Sciex, Framingham, MA) in Metabolomics Lab of Roy J. Carver Biotechnology Center, University of Illinois at Urbana-Champaign. Software Analyst 1.6.2 was used for data acquisition and analysis. The 1200 series HPLC system (Agilent Technologies, Santa Clara, CA) includes a degasser, an autosampler, and a binary pump. The LC separation was performed on an Agilent Eclipse XDB-C18 (4.6 x 150mm, 5μm) with mobile phase A (0.1% formic acid in water) and mobile phase B (0.1% formic acid in acetonitrile). The flow rate was 0.4 mL/min. The linear gradient was as follows: 0-3min, 95%A; 9-13min, 5%A; 13.5-18min, 95%A. The autosampler was set at 10°C. The injection volume was 5 μL. Mass spectra were acquired under positive electrospray ionization (ESI) with the ion spray voltage of +5000 V. The source temperature was 500°C. The curtain gas, ion source gas 1, and ion source gas 2 were 33, 65, and 55 psi, respectively. Multiple reaction monitoring (MRM) was used for quantitation: cadaverine (m/z 103.0 → m/z 69.0); putrescine (m/z 189.0 → m/z 30.0); spermine (m/z 199.9 → m/z 91.1); spermidine (m/z 146.0 → m/z 30.0); and tyramine (m/z 138.1 → m/z 77.1). The levels of detection for each metabolite were as follows: cadaverine (100nM), putrescine (100nM), spermidine (20nM), spermine (20nM) and tyramine (5nM). For the purpose of plotting, “missing values” were imputed with one-half the minimum value obtained for a given metabolite [142–145].
Statistical analysis
We sought to characterize the vaginal bacterial community composition of an American Indian population and to evaluate associations between behavioral, demographic, and measures of psychosocial stress with the vaginal microbiota and vaginal biogenic amines. Unadjusted associations were assessed using a two-sample test for equality of proportions with continuity correction in R Statistical Software (Tables 1 and 3) or a two-sample t-test (Table 2). An exploratory biplot showing the first two components of a principal component analysis (PCA) was constructed on scaled variables to visualize samples and the influence of variables. The PCA was conducted using the prcomp function and visualized using the ggbiplot package in R Statistical Software.
Bayesian multinomial logistic regression was utilized for modeling the association between measures of stress, high-risk behavior and vaginal CSTs as the data were too sparse for large-sample frequentist inference [146, 147]. Due to the small number of participants with CST I microbiota (n = 7), we created a combined reference category using participants with both CST I and CST III microbiota. When modeling the associations between the vaginal CSTs (outcome) and measures of psychosocial stress and vaginal metabolites (Fig 3), we performed unadjusted and adjusted analysis with only smoking was included as a covariate, and this is because smoking is known to be associated with the vaginal microbiota and because a large number of participants self-reported as smokers [109]. When modeling the association between sexual practices (outcome) and measures of psychosocial stress (Fig 4), we did not include any covariates. We also used Bayesian regression to model the association between measures of stress and select vaginal metabolites (response) while adjusting for CST and smoking status, and these covariates were included given their established relationship [78, 109].
For all models, estimation was carried out using Hamiltonian Monte Carlo [148–150] and its extension, the No-U-Turn (NUTS) Sampler [151]. We used weakly informative priors for the fixed and random effects (i.e., we did not impose any prior information on the estimates). Bayesian multinomial logistic regression was performed in the brms package [152] in R which runs RStan (Stan Development Team, 2015) in the background. Statistical significance was defined as Bayesian credible intervals for odds ratios excluding 1.
Structural equation modeling (SEM)
Three latent variables were identified: HLAS anxiety and depression, interpersonal reliance, and the standardized values of three biogenic amines, trimethylamine, cadaverine, and putrescine. HLAS anxiety and depression was developed from a three-item solution with symptoms including: loss of concentration, feeling isolated or distant from other people when thinking about losses, and a loss of sleep. Spearman correlation analyses were performed to examine the associations between independent variables in the CFA and SEM. In the measurement model (S1 Fig), latent constructs were permitted to intercorrelate. SEM was used to explore relationships between HLAS of Anxiety/Depression, interpersonal dependency, and vaginal biogenic amines with the dependent variable (molecular-BV status versus non-molecular-BV status). We estimated parameters using linear regression coefficients for indicators with Gaussian distributions and probit regression to model effects on CST-IV status. We examined model fit using χ2, the comparative fit index, the Tucker-Lewis Index, and the root-mean-square error of approximation. To accept the model, the χ2 should be non-significant, the comparative fit index and Tucker-Lewis Index>0.95, and the root-mean-square error of approximation <0.08 [153, 154]. Given that this was a preliminary analysis conducted in a small sample, we focused on relative effect sizes and standard errors of the effects. Spearman’s correlation analysis was conducted in R Statistical Software. CFA and SEM modeling were conducted in Mplus v. 8, STATA 14 for data management, cleaning, and recoding. Results from the pathway analysis are shown in Fig 5.
Supporting information
S1 Table. Associations between measures of psychosocial stress and vaginal metabolites.
https://doi.org/10.1371/journal.pone.0260813.s002
(XLSX)
References
- 1. Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance 2018. Atlanta: US Department of Health and Human Services. 2019. https://doi.org/10.15620/cdc.79370
- 2. Hogben M, Leichliter JS. Social determinants and sexually transmitted disease disparities. Sex Transm Dis. 2008 Dec;35(12 Suppl):S13–8. pmid:18936725.
- 3. Newman LM, Berman SM. Epidemiology of STD disparities in African American communities. Sex Transm Dis. 2008 Dec;35(12 Suppl):S4–12. pmid:18971796.
- 4. Harling G, Subramanian S, Bärnighausen T, Kawachi I. Socioeconomic disparities in sexually transmitted infections among young adults in the United States: examining the interaction between income and race/ethnicity. Sex Transm Dis. 2013 Jul;40(7):575–81. pmid:23965773; PMCID: PMC3752095.
- 5. Kaufman CE, Shelby L, Mosure DJ, Marrazzo J, Wong D, de Ravello L, et al. Within the hidden epidemic: sexually transmitted diseases and HIV/AIDS among American Indians and Alaska Natives. Sex Transm Dis. 2007 Oct;34(10):767–77. pmid:17538516.
- 6. Toomey KE, Oberschelp AG, Greenspan JR. Sexually transmitted diseases and native Americans: trends in reported gonorrhea and syphilis morbidity, 1984–88. Public Health Rep. 1989 Nov-Dec;104(6):566–72. pmid:2511589; PMCID: PMC1580156.
- 7. Bertolli J, Lee LM, Sullivan PS; AI/AN Race /Ethnicity Data Validation Workgroup. Racial misidentification of American Indians/Alaska Natives in the HIV/AIDS Reporting Systems of five states and one urban health jurisdiction, U.S., 1984–2002. Public Health Rep. 2007 May-Jun;122(3):382–92. pmid:17518310; PMCID: PMC1847482.
- 8. Walker FJ, Llata E, Doshani M, Taylor MM, Bertolli J, Weinstock HS, et al. HIV, Chlamydia, Gonorrhea, and Primary and Secondary Syphilis among American Indians and Alaska Natives Within Indian Health Service Areas in the United States, 2007–2010. J Community Health. 2015 Jun;40(3):484–92. pmid:25371109; PMCID: PMC6785743.
- 9. Lieb LE, Conway GA, Hedderman M, Yao J, Kerndt PR. Racial misclassification of American Indians with AIDS in Los Angeles County. J Acquir Immune Defic Syndr (1988). 1992;5(11):1137–41. pmid:1403645.
- 10. Thoroughman DA, Frederickson D, Cameron HD, Shelby LK, Cheek JE. Racial misclassification of American Indians in Oklahoma State surveillance data for sexually transmitted diseases. Am J Epidemiol. 2002 Jun 15;155(12):1137–41. pmid:12048228.
- 11. Rutman S, Park A, Castor M, Taualii M, Forquera R. Urban American Indian and Alaska Native youth: youth risk behavior survey 1997–2003. Matern Child Health J. 2008 Jul;12 Suppl 1:76–81. Epub 2008 May 16. pmid:18483839.
- 12. de Ravello L, Everett Jones S, Tulloch S, Taylor M, Doshi S. Substance use and sexual risk behaviors among american Indian and alaska native high school students. J Sch Health. 2014 Jan;84(1):25–32. pmid:24320149; PMCID: PMC4311718.
- 13. Eitle D, Greene K, Eitle TM. American Indians, substance use, and sexual behavior: do predictors of sexually transmitted infections explain the race gap among young adults? Sex Transm Dis. 2015 Feb;42(2):64–7. pmid:25585062; PMCID: PMC4295642.
- 14. Manson SM, Beals J, Klein SA, Croy CD; AI-SUPERPFP Team. Social epidemiology of trauma among 2 American Indian reservation populations. Am J Public Health. 2005 May;95(5):851–9. pmid:15855465; PMCID: PMC1449268.
- 15. Cohen S KR, Gordon L. Measuring Stress: A Guide for Health and Social Scientists: Oxford University Press; 1998.
- 16. Cohen S, Tyrrell DA, Smith AP. Psychological stress and susceptibility to the common cold. N Engl J Med. 1991 Aug 29;325(9):606–12. pmid:1713648.
- 17. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983 Dec;24(4):385–96. pmid:6668417.
- 18. Cohen S, Janicki-Deverts D, Miller GE. Psychological stress and disease. JAMA. 2007 Oct 10;298(14):1685–7. pmid:17925521.
- 19. Cohen S, Williamson GM. Perceived stress in a probability sample of the United States. In: Spacapan S & Oskamp S (editors). The Social Psychology of Health. 4th ed. Newbury Park, CA: Sage; 1988. p. 31–67.
- 20. Cohen S, Doyle WJ, Turner RB, Alper CM, Skoner DP. Childhood socioeconomic status and host resistance to infectious illness in adulthood. Psychosom Med. 2004 Jul-Aug;66(4):553–8. pmid:15272102.
- 21. Office of the Surgeon General (US); Center for Mental Health Services (US); National Institute of Mental Health (US). Mental Health: Culture, Race, and Ethnicity: A Supplement to Mental Health: A Report of the Surgeon General. Rockville (MD) Substance Abuse and Mental Health Services Administration (US)2001 Aug.
- 22. Culhane JF, Rauh V, McCollum KF, Elo IT, Hogan V. Exposure to chronic stress and ethnic differences in rates of bacterial vaginosis among pregnant women. Am J Obstet Gynecol. 2002 Nov;187(5):1272–6. pmid:12439519.
- 23. Fernander A, Moorman G, Azuoru M. Race-related stress and smoking among pregnant African-American women. Acta Obstet Gynecol Scand. 2010;89(4):558–564. pmid:20078254.
- 24. Stress in America™ 2020: A National Mental Health Crisis. American Psychological Association, 2020. Available at: apa.org/news/press/releases/stress/2020/sia-mental-health-crisis.pdf
- 25. Centers for Disease Control and Prevention, National Center for Injury Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS). Available at: cdc.govinjury/wisqars/index.html.
- 26. Walters KL, Simoni JM. Trauma, substance use, and HIV risk among urban American Indian women. Cultural Diversity and Ethnic Minority Psychology. 1999;5(3):236–48.
- 27. Walters KL, Mohammed SA, Evans-Campbell T, Beltrán RE, Chae DH, Duran B. Bodies don’t just tell stories, they tell histories: Embodiment of Historical Trauma among American Indians and Alaska Natives. Du Bois Rev. 2011 Apr;8(1):179–189. pmid:29805469; PMCID: PMC5967849.
- 28. Simoni JM, Sehgal S, Walters KL. Triangle of Risk: Urban American Indian Women’s Sexual Trauma, Injection Drug Use, and HIV Sexual Risk Behaviors. AIDS and Behavior. 2004;8(1):33–45. pmid:15146132
- 29. Williams-Washington KN. Historical trauma. Handbook of African American health. New York, NY, US: Guilford Press; 2010. p. 31–50.
- 30. Williams-Washington KN, Mills CP. African American Historical Trauma: Creating an Inclusive Measure. Journal of Multicultural Counseling and Development. 2018;46(4):246–63. https://doi.org/10.1002/jmcd.12113.
- 31. Sandefur GD, Liebler CA. The demography of American Indian families. Population Research and Policy Review. 1997;16(1):95–114.
- 32. Adamsen C, Schroeder S, LeMire S, Carter P. Education, Income, and Employment and Prevalence of Chronic Disease Among American Indian/Alaska Native Elders. Prev Chronic Dis. 2018 Mar 22;15:E37. pmid:29565785; PMCID: PMC5871354.
- 33. Sarche M, Spicer P. Poverty and health disparities for American Indian and Alaska Native children: current knowledge and future prospects. Ann N Y Acad Sci. 2008;1136:126–36. pmid:18579879; PMCID: PMC2567901.
- 34. Castor ML, Smyser MS, Taualii MM, Park AN, Lawson SA, Forquera RA. A nationwide population-based study identifying health disparities between American Indians/Alaska Natives and the general populations living in select urban counties. Am J Public Health. 2006 Aug;96(8):1478–84. Epub 2006 Mar 29. pmid:16571711; PMCID: PMC1522100.
- 35. Espey DK, Jim MA, Cobb N, Bartholomew M, Becker T, Haverkamp D, et al. Leading causes of death and all-cause mortality in American Indians and Alaska Natives. Am J Public Health. 2014 Jun;104 Suppl 3(Suppl 3):S303–11. Epub 2014 Apr 22. pmid:24754554; PMCID: PMC4035872.
- 36. Beals J, Novins DK, Whitesell NR, Spicer P, Mitchell CM, Manson SM. Prevalence of mental disorders and utilization of mental health services in two American Indian reservation populations: mental health disparities in a national context. Am J Psychiatry. 2005 Sep;162(9):1723–32. pmid:16135633.
- 37. Evans-Campbell T. Historical trauma in American Indian/Native Alaska communities: a multilevel framework for exploring impacts on individuals, families, and communities. J Interpers Violence. 2008 Mar;23(3):316–38. pmid:18245571.
- 38. Bohn DK. Lifetime physical and sexual abuse, substance abuse, depression, and suicide attempts among Native American women. Issues Ment Health Nurs. 2003 Apr-May;24(3):333–52. pmid:12623689.
- 39. Kunitz SJ, Levy JE, McCloskey J, Gabriel KR. Alcohol dependence and domestic violence as sequelae of abuse and conduct disorder in childhood. Child Abuse Negl. 1998 Nov;22(11):1079–91. pmid:9827313
- 40. Tjaden PT N. Full report of the prevalence, incidence, and consequences of violence against women: Findings from the National Violence Against Women Survey Centers for Disease Control and Prevention, US Department of Justice; National Institute of Justice & the US Department of Health and Human Service 2000.
- 41. Black MC, Basile K.C., Breiding M.J., Smith S.G., Walters M.L., Merrick M.T. et al., The National Intimate Partner and Sexual Violence Survey (NISVS): 2010 Summary Report. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, 2011.
- 42. Biondi M, Zannino LG. Psychological Stress, Neuroimmunomodulation, and Susceptibility to Infectious Diseases in Animals and Man: A Review. Psychotherapy and Psychosomatics. 1997;66(1):3–26. pmid:8996711
- 43. Glaser R, Rabin B, Chesney M, Cohen S, Natelson B. Stress-induced immunomodulation: implications for infectious diseases? JAMA. 1999;281(24):2268–70. Epub 1999/07/01. pmid:10386538.
- 44. Cohen S, Janicki-Deverts D, Doyle WJ, Miller GE, Frank E, Rabin BS, et al. Chronic stress, glucocorticoid receptor resistance, inflammation, and disease risk. Proc Natl Acad Sci U S A. 2012 Apr 17;109(16):5995–9. Epub 2012 Apr 2. pmid:22474371; PMCID: PMC3341031.
- 45. Culhane JF, Rauh V, McCollum KF, Hogan VK, Agnew K, Wadhwa PD. Maternal stress is associated with bacterial vaginosis in human pregnancy. Matern Child Health J. 2001 Jun;5(2):127–34. pmid:11573838.
- 46. Culhane JF, Rauh VA, Goldenberg RL. Stress, bacterial vaginosis, and the role of immune processes. Curr Infect Dis Rep. 2006 Nov;8(6):459–64. pmid:17064639.
- 47. Nansel TR, Riggs MA, Yu KF, Andrews WW, Schwebke JR, Klebanoff MA. The association of psychosocial stress and bacterial vaginosis in a longitudinal cohort. Am J Obstet Gynecol. 2006 Feb;194(2):381–6. pmid:16458633; PMCID: PMC2367104.
- 48. McEwen B, Stellar E. Stress and the individual: Mechanisms leading to disease. Archives of Internal Medicine. 1993;153(18):2093–101. pmid:8379800
- 49. McEwen BS. Physiology and neurobiology of stress and adaptation: Central role of the brain. Physiological Reviews. 2007;87(3):873–904. ISI:000248378900003. pmid:17615391
- 50. McEwen BS. Protective and damaging effects of stress mediators. N Engl J Med. 1998;338(3):171–9. Epub 1998/01/15. pmid:9428819.
- 51. McEwen BS. Central effects of stress hormones in health and disease: Understanding the protective and damaging effects of stress and stress mediators. Eur J Pharmacol. 2008 Apr 7;583(2–3):174–85. Epub 2008 Jan 30. pmid:18282566; PMCID: PMC2474765.
- 52. Turpin R, Slopen N, Borgogna JC, Yeoman CJ, He X, Miller RS, et al. Perceived Stress and Molecular-BV in the NIH Longitudinal Study of Vaginal Flora. Am J Epidemiol. 2021. Epub 2021/05/20. pmid:34008013.
- 53. Peebles K, Velloza J, Balkus JE, McClelland RS, Barnabas RV. High Global Burden and Costs of Bacterial Vaginosis: A Systematic Review and Meta-Analysis. Sex Transm Dis. 2019 May;46(5):304–311. pmid:30624309.
- 54. Spiegel CA, Amsel R, Eschenbach D, Schoenknecht F, Holmes KK. Anaerobic bacteria in nonspecific vaginitis. N Engl J Med. 1980;303(11):601–7. Epub 1980/09/11. pmid:6967562.
- 55. Amsel R, Totten PA, Spiegel CA, Chen KC, Eschenbach D, Holmes KK. Nonspecific vaginitis. Diagnostic criteria and microbial and epidemiologic associations. Am J Med. 1983 Jan;74(1):14–22. pmid:6600371.
- 56. Martin DH. The microbiota of the vagina and its influence on women’s health and disease. Am J Med Sci. 2012 Jan;343(1):2–9. pmid:22143133; PMCID: PMC3248621.
- 57. Martin HL, Richardson BA, Nyange PM, Lavreys L, Hillier SL, Chohan B, et al. Vaginal lactobacilli, microbial flora, and risk of human immunodeficiency virus type 1 and sexually transmitted disease acquisition. J Infect Dis. 1999 Dec;180(6):1863–8. pmid:10558942.
- 58. McKinnon LR, Achilles SL, Bradshaw CS, Burgener A, Crucitti T, Fredricks DN, et al. The Evolving Facets of Bacterial Vaginosis: Implications for HIV Transmission. AIDS Res Hum Retroviruses. 2019;35(3):219–28. pmid:30638028; PubMed Central PMCID: PMC6434601.
- 59. Hillier SL. Diagnostic microbiology of bacterial vaginosis. Am J Obstet Gynecol. 1993 Aug;169(2 Pt 2):455–9. pmid:8357044.
- 60. Hillier SL, Krohn MA, Rabe LK, Klebanoff SJ, Eschenbach DA. The normal vaginal flora, H2O2-producing lactobacilli, and bacterial vaginosis in pregnant women. Clin Infect Dis. 1993 Jun;16 Suppl 4:S273–81. pmid:8324131.
- 61. Hillier SL, Holmes K Bacterial Vaginosis. 3 ed. Holmes PS K. K., & Marsh P editor. New York: McGraw-Hill; 1999.
- 62. Yeoman CJ, Thomas SM, Miller ME, Ulanov AV, Torralba M, Lucas S, et al. A multi-omic systems-based approach reveals metabolic markers of bacterial vaginosis and insight into the disease. PLoS One. 2013;8(2):e56111. Epub 2013 Feb 6. pmid:23405259; PMCID: PMC3566083.
- 63. Nelson TM, Borgogna JL, Brotman RM, Ravel J, Walk ST, Yeoman CJ. Vaginal biogenic amines: biomarkers of bacterial vaginosis or precursors to vaginal dysbiosis? Front Physiol. 2015;6:253. pmid:26483694; PubMed Central PMCID: PMC4586437.
- 64. Srinivasan S, Morgan MT, Fiedler TL, Djukovic D, Hoffman NG, Raftery D, et al. Metabolic signatures of bacterial vaginosis. mBio. 2015 Apr 14;6(2):e00204–15. pmid:25873373; PMCID: PMC4453549.
- 65. Wolrath H, Boren H, Hallen A, Forsum U. Trimethylamine content in vaginal secretion and its relation to bacterial vaginosis. APMIS. 2002;110(11):819–24. pmid:12588422.
- 66. Atashili J, Poole C, Ndumbe PM, Adimora AA, Smith JS. Bacterial vaginosis and HIV acquisition: a meta-analysis of published studies. AIDS (London, England). 2008;22(12):1493–501. pmid:18614873
- 67. Gosmann C, Anahtar MN, Handley SA, Farcasanu M, Abu-Ali G, Bowman BA, et al. Lactobacillus-Deficient Cervicovaginal Bacterial Communities Are Associated with Increased HIV Acquisition in Young South African Women. Immunity. 2017;46(1):29–37. Epub 2017/01/15. pmid:28087240; PubMed Central PMCID: PMC5270628.
- 68. Brotman RM, Klebanoff MA, Nansel TR, Yu KF, Andrews WW, Zhang J, et al. Bacterial vaginosis assessed by gram stain and diminished colonization resistance to incident gonococcal, chlamydial, and trichomonal genital infection. J Infect Dis. 2010 Dec 15;202(12):1907–15. Epub 2010 Nov 10. pmid:21067371; PMCID: PMC3053135.
- 69. Brotman RM. Vaginal microbiome and sexually transmitted infections: an epidemiologic perspective. J Clin Invest. 2011 Dec;121(12):4610–7. Epub 2011 Dec 1. pmid:22133886; PMCID: PMC3225992.
- 70. van der Veer C, Bruisten SM, van der Helm JJ, de Vries HJ, van Houdt R. The Cervicovaginal Microbiota in Women Notified for Chlamydia trachomatis Infection: A Case-Control Study at the Sexually Transmitted Infection Outpatient Clinic in Amsterdam, The Netherlands. Clin Infect Dis. 2017 Jan 1;64(1):24–31. Epub 2016 Aug 27. pmid:27567124.
- 71. Tamarelle J, Thiebaut ACM, de Barbeyrac B, Bebear C, Ravel J, Delarocque-Astagneau E. The vaginal microbiota and its association with human papillomavirus, Chlamydia trachomatis, Neisseria gonorrhoeae and Mycoplasma genitalium infections: a systematic review and meta-analysis. Clin Microbiol Infect. 2019;25(1):35–47. Epub 2018/05/08. pmid:29729331.
- 72. Srinivasan S, Liu C, Mitchell CM, Fiedler TL, Thomas KK, Agnew KJ, et al. Temporal variability of human vaginal bacteria and relationship with bacterial vaginosis. PLoS One. 2010 Apr 15;5(4):e10197. pmid:20419168; PMCID: PMC2855365.
- 73. DiGiulio DB, Callahan BJ, McMurdie PJ, Costello EK, Lyell DJ, Robaczewska A, et al. Temporal and spatial variation of the human microbiota during pregnancy. Proc Natl Acad Sci U S A. 2015;112(35):11060–5. Epub 2015/08/19. pmid:26283357; PubMed Central PMCID: PMC4568272.
- 74. Jespers V, Menten J, Smet H, Poradosu S, Abdellati S, Verhelst R, et al. Quantification of bacterial species of the vaginal microbiome in different groups of women, using nucleic acid amplification tests. BMC Microbiol. 2012;12:83. Epub 2012/06/01. pmid:22647069; PubMed Central PMCID: PMC3418157.
- 75. Srinivasan S, Hoffman NG, Morgan MT, Matsen FA, Fiedler TL, Hall RW, et al. Bacterial communities in women with bacterial vaginosis: high resolution phylogenetic analyses reveal relationships of microbiota to clinical criteria. PLoS One. 2012;7(6):e37818. Epub 2012 Jun 18. pmid:22719852; PMCID: PMC3377712.
- 76. Kanjee U, Houry WA. Mechanisms of acid resistance in Escherichia coli. Annu Rev Microbiol. 2013;67:65–81. Epub 2013 May 20. pmid:23701194.
- 77. Borgogna JC, Shardell MD, Grace SG, Santori EK, Americus B, Li Z, et al. Biogenic Amines Increase the Odds of Bacterial Vaginosis and Affect the Growth and Lactic Acid Production by Vaginal Lactobacillus spp. Appl Environ Microbiol. 2021. Epub 2021/03/07. pmid:33674429.
- 78. Nelson TM, Borgogna JC, Michalek RD, Roberts DW, Rath JM, Glover ED, et al. Cigarette smoking is associated with an altered vaginal tract metabolomic profile. Sci Rep. 2018;8(1):852. Epub 2018/01/18. pmid:29339821; PubMed Central PMCID: PMC5770521.
- 79. Borgogna JC, Shardell MD, Yeoman CJ, Ghanem KG, Kadriu H, Ulanov AV, et al. The association of Chlamydia trachomatis and Mycoplasma genitalium infection with the vaginal metabolome. Sci Rep. 2020;10(1):3420. Epub 2020/02/27. pmid:32098988; PubMed Central PMCID: PMC7042340.
- 80. Borgogna JC, Shardell MD, Santori EK, Nelson TM, Rath JM, Glover ED et al. The vaginal metabolome and microbiota of cervical HPV-positive and HPV-negative women: a cross-sectional analysis. BJOG. 2020 Jan;127(2):182–192. Epub 2019 Nov 20. pmid:31749298; PMCID: PMC6982399.
- 81. Gong Z, Luna Y, Yu P, Fan H. Lactobacilli inactivate Chlamydia trachomatis through lactic acid but not H2O2. PLoS One. 2014 Sep 12;9(9):e107758. pmid:25215504; PMCID: PMC4162611.
- 82. Gong Z, Tang MM, Wu X, Phillips N, Galkowski D, Jarvis GA, et al. Arginine- and Polyamine-Induced Lactic Acid Resistance in Neisseria gonorrhoeae. PLoS One. 2016 Jan 25;11(1):e0147637. pmid:26808268; PMCID: PMC4726613.
- 83. Goytia M, Dhulipala VL, Shafer WM. Spermine impairs biofilm formation by Neisseria gonorrhoeae. FEMS Microbiol Lett. 2013 Jun;343(1):64–9. Epub 2013 Apr 4. pmid:23506248; PMCID: PMC3651792.
- 84. Goytia M, Shafer WM. Polyamines can increase resistance of Neisseria gonorrhoeae to mediators of the innate human host defense. Infect Immun. 2010;78(7):3187–95. pmid:20439477; PubMed Central PMCID: PMC2897401.
- 85. Gilad VH, Rabey JM, Kimiagar Y, Gilad GM. The polyamine stress response: tissue-, endocrine-, and developmental-dependent regulation. Biochem Pharmacol. 2001;61(2):207–13. Epub 2001/02/13. pmid:11163335.
- 86. Fiori LM, Turecki G. Implication of the polyamine system in mental disorders. J Psychiatry Neurosci. 2008;33(2):102–10. pmid:18330456.
- 87. Zahedi K, Huttinger F, Morrison R, Murray-Stewart T, Casero RA, Strauss KI. Polyamine catabolism is enhanced after traumatic brain injury. J Neurotrauma. 2010 Mar;27(3):515–25. pmid:19968558; PMCID: PMC2867553.
- 88. Sandusky-Beltran LA, Kovalenko A, Ma C, Calahatian JIT, Placides DS, Watler MD, et al. Spermidine/spermine-N1-acetyltransferase ablation impacts tauopathy-induced polyamine stress response. Alzheimers Res Ther. 2019 Jun 29;11(1):58. pmid:31253191; PMCID: PMC6599347.
- 89. Ravel J, Brotman RM, Gajer P, Ma B, Nandy M, Fadrosh DW et al. Daily temporal dynamics of vaginal microbiota before, during and after episodes of bacterial vaginosis. Microbiome. 2013 Dec 2;1(1):29. pmid:24451163; PMCID: PMC3968321.
- 90. Gajer P, Brotman RM, Bai G, Sakamoto J, Schütte UM, Zhong X, et al. Temporal dynamics of the human vaginal microbiota. Sci Transl Med. 2012 May 2;4(132):132ra52. pmid:22553250; PMCID: PMC3722878.
- 91. Ravel J, Gajer P, Abdo Z, Schneider GM, Koenig SS, McCulle SL, et al. Vaginal microbiome of reproductive-age women. Proc Natl Acad Sci U S A. 2011 Mar 15;108 Suppl 1(Suppl 1):4680–7. Epub 2010 Jun 3. pmid:20534435; PMCID: PMC3063603.
- 92. Wiesenfeld HC, Hillier SL, Krohn MA, Landers DV, Sweet RL. Bacterial vaginosis is a strong predictor of Neisseria gonorrhoeae and Chlamydia trachomatis infection. Clin Infect Dis. 2003 Mar 1;36(5):663–8. Epub 2003 Feb 7. pmid:12594649.
- 93. Nardis C, Mosca L, Mastromarino P. Vaginal microbiota and viral sexually transmitted diseases. Ann Ig. 2013 Sep-Oct;25(5):443–56. pmid:24048183.
- 94. van de Wijgert JH, Morrison CS, Brown J, Kwok C, Van Der Pol B, Chipato T, et al. Disentangling contributions of reproductive tract infections to HIV acquisition in African Women. Sex Transm Dis. 2009 Jun;36(6):357–64. pmid:19434010.
- 95. Brotman RM, Klebanoff MA, Nansel TR, Andrews WW, Schwebke JR, Zhang J, et al. A longitudinal study of vaginal douching and bacterial vaginosis—a marginal structural modeling analysis. Am J Epidemiol. 2008;168(2):188–96. Epub 2008/05/27. pmid:18503038; PubMed Central PMCID: PMC2574994.
- 96. Brotman RM, Ghanem KG, Klebanoff MA, Taha TE, Scharfstein DO, Zenilman JM. The effect of vaginal douching cessation on bacterial vaginosis: a pilot study. Am J Obstet Gynecol. 2008 Jun;198(6):628.e1–7. Epub 2008 Mar 4. pmid:18295180; PMCID: PMC2494605.
- 97. Vodstrcil LA, Twin J, Garland SM, Fairley CK, Hocking JS, Law MG, et al. The influence of sexual activity on the vaginal microbiota and Gardnerella vaginalis clade diversity in young women. PLoS One. 2017 Feb 24;12(2):e0171856. pmid:28234976; PMCID: PMC5325229.
- 98. Plummer EL, Vodstrcil LA, Fairley CK, Tabrizi SN, Garland SM, Law MG, et al. Sexual practices have a significant impact on the vaginal microbiota of women who have sex with women. Scientific reports. 2019;9(1):19749–. pmid:31874964.
- 99. Arbour M, Corwin EJ, Salsberry P. Douching patterns in women related to socioeconomic and racial/ethnic characteristics. J Obstet Gynecol Neonatal Nurs. 2009 Sep-Oct;38(5):577–85. pmid:19883479.
- 100. Adimora AA, Schoenbach VJ, Taylor EM, Khan MR, Schwartz RJ. Concurrent partnerships, nonmonogamous partners, and substance use among women in the United States. Am J Public Health. 2011;101(1):128–36. Epub 2010/08/19. pmid:20724694.
- 101. Copen C. Condom use during sexual intercourse among women and men aged 15–44 in the United States: 2011–2015 National Survey of Family Growth. Hyattsville, MD: National Center for Health Statistics, 2017 Contract No.: National health statistics reports; no. 105.
- 102. Kenyon C, Buyze J, Klebanoff M, Brotman RM. The role of sexual networks in studies of how BV and STIs increase the risk of subsequent reinfection. Epidemiology and infection. 2018;146(15):2003–9. Epub 2018/09/05. pmid:30182860.
- 103. Tanfer K, Cubbins LA, Billy JO. Gender, race, class and self-reported sexually transmitted disease incidence. Fam Plann Perspect. 1995 Sep-Oct;27(5):196–202. pmid:9104606.
- 104. Miller HG, Cain VS, Rogers SM, Gribble JN, Turner CF. Correlates of sexually transmitted bacterial infections among U.S. women in 1995. Fam Plann Perspect. 1999 Jan-Feb;31(1):4–9, 23. pmid:10029926.
- 105. Ellen JM, Aral SO, Madger LS. Do differences in sexual behaviors account for the racial/ethnic differences in adolescents’ self-reported history of a sexually transmitted disease? Sex Transm Dis. 1998 Mar;25(3):125–9. pmid:9524987.
- 106. Amabebe E, Anumba DOC. Psychosocial Stress, Cortisol Levels, and Maintenance of Vaginal Health. Frontiers in endocrinology. 2018;9:568–. pmid:30319548.
- 107. Turpin R, Brotman RM, Miller RS, Klebanoff MA, He X, Slopen N. Perceived stress and incident sexually transmitted infections in a prospective cohort. Ann Epidemiol. 2019 Apr;32:20–27. Epub 2019 Jan 26. pmid:30799204; PMCID: PMC6446572.
- 108. Seth P, Patel SN, Sales JM, DiClemente RJ, Wingood GM, Rose ES. The impact of depressive symptomatology on risky sexual behavior and sexual communication among African American female adolescents. Psychol Health Med. 2011 May;16(3):346–56. pmid:21491342; PMCID: PMC3086508.
- 109. Brotman RM, He X, Gajer P, Fadrosh D, Sharma E, Mongodin EF, et al. Association between cigarette smoking and the vaginal microbiota: a pilot study. BMC Infect Dis. 2014;14:471–. pmid:25169082.
- 110. Fidler JA, West R. Self-perceived smoking motives and their correlates in a general population sample. Nicotine Tob Res. 2009 Oct;11(10):1182–8. Epub 2009 Jul 28. pmid:19640835.
- 111. Elkind AK. Do nurses smoke because of stress? J Adv Nurs. 1988 Nov;13(6):733–45. pmid:3230215.
- 112. Scales MB, Monahan JL, Rhodes N, Roskos-Ewoldsen D, Johnson-Turbes A. Adolescents’ perceptions of smoking and stress reduction. Health Educ Behav. 2009 Aug;36(4):746–58. Epub 2008 May 13. pmid:18477670.
- 113. McEwen A, West R, McRobbie H. Motives for smoking and their correlates in clients attending Stop Smoking treatment services. Nicotine Tob Res. 2008 May;10(5):843–50. pmid:18569758.
- 114. Hillier SL. Diagnostic microbiology of bacterial vaginosis. Am J Obstet Gynecol. 1993 Aug;169(2 Pt 2):455–9. pmid:8357044.
- 115. Sobel JD, Karpas Z, Lorber A. Diagnosing vaginal infections through measurement of biogenic amines by ion mobility spectrometry. Eur J Obstet Gynecol Reprod Biol. 2012 Jul;163(1):81–4. Epub 2012 Apr 19. pmid:22520996.
- 116. Vitali B, Cruciani F, Picone G, Parolin C, Donders G, Laghi L. Vaginal microbiome and metabolome highlight specific signatures of bacterial vaginosis. Eur J Clin Microbiol Infect Dis. 2015;34(12):2367–76. Epub 2015/09/20. pmid:26385347.
- 117. Blankenstein T, Lytton SD, Leidl B, Atweh E, Friese K, Mylonas I. Point-of-care (POC) diagnosis of bacterial vaginosis (BV) using VGTest ion mobility spectrometry (IMS) in a routine ambulatory care gynecology clinic. Arch Gynecol Obstet. 2015;292(2):355–62. Epub 2015/02/02. pmid:25638448.
- 118. Shah P, Swiatlo E. A multifaceted role for polyamines in bacterial pathogens. Mol Microbiol. 2008 Apr;68(1):4–16. Epub 2008 Mar 5. pmid:18405343.
- 119. Chattopadhyay MK, Tabor CW, Tabor H. Polyamines protect Escherichia coli cells from the toxic effect of oxygen. Proc Natl Acad Sci U S A. 2003 Mar 4;100(5):2261–5. Epub 2003 Feb 18. pmid:12591940; PMCID: PMC151328.
- 120. Gilad GM, Gilad VH. Stress-induced dynamic changes in mouse brain polyamines. Role in behavioral reactivity. Brain Res. 2002 Jul 5;943(1):23–9. pmid:12088835.
- 121. Yang Q, Zheng C, Cao J, Cao G, Shou P, Lin L et al. Spermidine alleviates experimental autoimmune encephalomyelitis through inducing inhibitory macrophages. Cell Death Differ. 2016 Nov 1;23(11):1850–1861. Epub 2016 Jul 22. pmid:27447115; PMCID: PMC5071574.
- 122. Wirth M, Schwarz C, Benson G, Horn N, Buchert R, Lange C et al. Effects of spermidine supplementation on cognition and biomarkers in older adults with subjective cognitive decline (SmartAge)-study protocol for a randomized controlled trial. Alzheimers Res Ther. 2019 May 1;11(1):36. pmid:31039826; PMCID: PMC6492385.
- 123. Atiya Ali M, Poortvliet E, Strömberg R, Yngve A. Polyamines in foods: development of a food database. Food Nutr Res. 2011 Jan 14;55. pmid:21249159; PMCID: PMC3022763.
- 124. Cox GR, Anastario M, FireMoon P, Ricker A, Rink E. Narrative frames as choice over structure of American Indian sexual and reproductive health consequences of historical trauma. Sociol Health Illn. 2021. Epub 2021/07/23. pmid:34293204.
- 125. Rink E, Anastario M, Johnson O, GrowingThunder R, Ricker A, Firemoon P, et al. The Development and Testing of a Multi-Level, Multi-Component Pilot Intervention To Reduce Sexual and Reproductive Health Disparities in a Tribal Community. J Ethn Cult Divers Soc Work. 2021;30(1):138–48. Epub 2021/03/19. pmid:33732097; PubMed Central PMCID: PMC7959407.
- 126. Anastario M, FireMoon P, Rink E. Sexual risk behaviors and the legacy of colonial violence among Northern plains American Indian youth: A mixed methods exploratory study. Soc Sci Med. 2020;258:113120. Epub 2020/06/24. pmid:32574888; PubMed Central PMCID: PMC7971236.
- 127. Anastario M, FireMoon P, Ricker A, Holder S, Rink E. Self-reported Exposure to Sexual and Reproductive Health Information among American Indian Youth: Implications for Technology Based Intervention. J Health Commun. 2020 May 3;25(5):412–420. Epub 2020 Jun 25. pmid:32584646; PMCID: PMC8018870.
- 128. Anastario MP, FourStar K, Rink E. Sexual risk behavior and symptoms of historical loss in American Indian men. J Community Health. 2013 Oct;38(5):894–9. pmid:23624772.
- 129. Rink E, Anastario MP, FourStar K. Perceived Level of Relationship Commitment, Sexual Risk Taking and Condom Use Among American Indian Men. J Immigr Minor Health. 2015 Aug;17(4):1078–85. pmid:24952907.
- 130. Rink E, Montgomery-Andersen R, Anastario M. The effectiveness of an education intervention to prevent chlamydia infection among Greenlandic youth. Int J STD AIDS. 2015 Feb;26(2):98–106. Epub 2014 Apr 8. pmid:24713230.
- 131. Anastario M, FourStar K, Ricker A, Dick R, Skewes MC, Rink E. A preliminary needs assessment of American Indians who inject drugs in northeastern Montana. Harm Reduct J. 2017;14(1):22. Epub 2017/05/10. pmid:28482846; PubMed Central PMCID: PMC5422938.
- 132. Rink E, FourStar K, Anastario MP. The Relationship Between Pregnancy Prevention and STI/HIV Prevention and Sexual Risk Behavior Among American Indian Men. J Rural Health. 2017 Jan;33(1):50–61. Epub 2015 Dec 22. pmid:26696246.
- 133. Beals J, Manson SM, Mitchell CM, Spicer P, Team A-S. Cultural specificity and comparison in psychiatric epidemiology: walking the tightrope in American Indian research. Cult Med Psychiatry. 2003;27(3):259–89. Epub 2003/09/27. pmid:14510095.
- 134. Rosenberg M. Conceiving the self. New York: Basic Books, 1979.
- 135. Carver CS. Adult attachment and personality: Converging evidence and a new measure. Personality and Social Psychology Bulletin. 1997;23(8):865–83.
- 136. Zimet GD, Powell SS, Farley GK, Werkman S, Berkoff KA. Psychometric characteristics of the Multidimensional Scale of Perceived Social Support. J Pers Assess. 1990 Winter;55(3–4):610–7. pmid:2280326.
- 137. Carleton RN, Thibodeau MA, Teale MJN, Welch PG, Abrams MP, Robinson T, et al. The center for epidemiologic studies depression scale: a review with a theoretical and empirical examination of item content and factor structure. PloS one. 2013;8(3):e58067–e. Epub 2013/03/01. pmid:23469262.
- 138. Radloff LS. The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Applied Psychological Measurement. 1977;1(3):385–401. pmid:26918431
- 139. Whitbeck LB, Adams GW, Hoyt DR, Chen X. Conceptualizing and measuring historical trauma among American Indian people. Am J Community Psychol. 2004 Jun;33(3–4):119–30. pmid:15212173.
- 140. Callahan BJ, McMurdie PJ, Rosen MJ, Han AW, Johnson AJ, Holmes SP. DADA2: High-resolution sample inference from Illumina amplicon data. Nat Methods. 2016 Jul;13(7):581–3. Epub 2016 May 23. pmid:27214047; PMCID: PMC4927377.
- 141. Holm JB, Humphrys MS, Robinson CK, Settles ML, Ott S, Fu L et al. Ultrahigh-Throughput Multiplexing and Sequencing of >500-Base-Pair Amplicon Regions on the Illumina HiSeq 2500 Platform. mSystems. 2019 Feb 19;4(1):e00029–19. pmid:30801027; PMCID: PMC6381223.
- 142. Yang J, Zhao X, Lu X, Lin X, Xu G. A data preprocessing strategy for metabolomics to reduce the mask effect in data analysis. Front Mol Biosci. 2015 Feb 2;2:4. pmid:25988172; PMCID: PMC4428451.
- 143. Wei R, Wang J, Su M, Jia E, Chen S, Chen T, et al. Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data. Sci Rep. 2018;8(1):663. pmid:29330539; PubMed Central PMCID: PMC5766532.
- 144. Smilde AK, van der Werf MJ, Bijlsma S, van der Werff-van der Vat BJ, Jellema RH. Fusion of mass spectrometry-based metabolomics data. Anal Chem. 2005;77(20):6729–36. pmid:16223263.
- 145. Xia J, Wishart DS. Web-based inference of biological patterns, functions and pathways from metabolomic data using MetaboAnalyst. Nat Protoc. 2011;6(6):743–60. pmid:21637195.
- 146. Gelman A HJ. Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press; 2007.
- 147. Imai K, King G, Lau O. Toward a Common Framework for Statistical Analysis and Development. Journal of Computational and Graphical Statistics. 2008;17(4):892–913.
- 148. Duane S, Kennedy AD, Pendleton BJ, Roweth D. Hybrid Monte Carlo. Physics Letters B. 1987;195(2):216–22. https://doi.org/10.1016/0370-2693(87)91197-X.
- 149. Neal R. Handbook of Markov Chain Monte Carlo. In: Press C, editor. 22011.
- 150. Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, et al. Stan: A Probabilistic Programming Language. Journal of Statistical Software; Vol 1, Issue 1 (2017). 2017.
- 151. Hoffman MD GA. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. The Journal of Machine Learning Research. 2014;15:1593–623.
- 152. Bürkner P-C. brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software. 2017;80(1).
- 153. Schreiber JB. Update to core reporting practices in structural equation modeling. Res Social Adm Pharm. 2017 May-Jun;13(3):634–643. Epub 2016 Jul 21. pmid:27567146.
- 154. Schreiber JB. Core reporting practices in structural equation modeling. Res Social Adm Pharm. 2008 Jun;4(2):83–97. pmid:18555963.