Introduction

Might the genetical contribution to fluctuating asymmetry (FA), or nondirectional variation in the subtle differences between left and right sides of bilateral characters, be primarily epistatic in nature? Although we now know that little or no additive genetic variation exists for FA in most characters (see the review by Fuller and Houle, 2003), the possibility remains that nonadditive (epistatic) genetic variation may be important in determining the overall level of FA (Leamy, 1984, 2003; Clarke, 1993; Leamy et al, 2002). One reason for assuming this is that significant differences in FA levels often have been found between heterozygotes and homozygotes or between hybrid species and their parents (Thoday, 1958; Leamy, 1984; Livshits and Kobyliansky, 1991; Clarke, 1993; Livshits and Smouse, 1994; Alibert and Auffray, 2003), suggesting that epistatic coadaptations are present in parents, which tend to be broken down in their hybrid offspring (Clarke, 1993). Also, Klingenberg and Nijhout (1999) generated epistatic effects in a theoretical model that simulated character variation and developmental noise, so epistasis for FA may well be a normal outcome of the developmental processes responsible for bilateral characters.

An understanding of the contribution of epistasis to FA levels would seem to be important given that FA is widely used as an indicator of developmental instability (DI). DI occurs when development of structures along their normal developmental pathway in a specific environment is disturbed by any of a number of genetic or environmental stressors (Zakharov, 1989). Individuals in populations subjected to some stressor therefore are expected to exhibit greater DI than those in nonstressed populations, and this in turn should be reflected in increased levels of FA (Clarke, 1992; Graham, 1992; Møller and Pominakowski, 1993; Zakharov, 1994; Markow, 1995; Polak, 2003). If it turns out that the genetic basis for FA is primarily epistatic, therefore, this would suggest that we will need to look more carefully at interactions of genes to explain at least part of the differences we sometimes see in FA levels between stressed and nonstressed populations.

The evidence regarding a possible epistatic genetic basis for FA has been mixed. Previous studies using traditional line cross analyses, for example, did not detect epistasis for FA in bristle number in Drosophila (Blows and Sokolowski, 1995; Polak and Starmer, 2001). However, these kinds of analyses depend on the contribution of epistasis to the interaction genetic variance only and thus may lack sufficient statistical power to detect epistasis even when present (Cheverud and Routman, 1995). In contrast, Leamy et al (2002) recently demonstrated an abundance of epistasis for FA in mandible size in an F2 population of mice generated from an original intercross of the Large (LG/J) and Small (SM/J) inbred strains. Their test for ‘physiological’ epistasis was a sensitive one since it accounted for the contribution of epistatic genotypic values to the additive, dominance, and interaction genetic variances (Cheverud and Routman, 1995; Routman and Cheverud, 1997; Cheverud, 2000).

Given the paucity of evidence for an epistatic basis for FA, we wanted to know if this discovery by Leamy et al (2002) was specific for mandible size, or whether it might hold true for FA in other characters as well. And if epistasis could be demonstrated for FA in other characters, it also was of interest to know whether the loci involved in these epistatic interactions would be common for both characters, or whether they might be character specific. We therefore made use of these same mice to test for epistasis for FA in molar size and shape, and if present, to compare the loci involved with those previously found to epistatically affect FA in mandible size. The genetic basis of FA in molar shape was of special interest since shape FA only recently has received much attention but may have the potential to be a sensitive indicator of environmental stress (Hoffman and Woods, 2003).

Materials and methods

The population and measurements

All mice used in this study were of 70 days of age, and were the F2 progeny generated from an intercross of the Large (LG/J) and Small (SM/J) inbred strains (Cheverud et al, 1996). DNA extracted from each mouse was used to genotype a total of 76 polymorphic microsatellite loci on the 19 autosomes using PCR techniques that have previously been described (Routman and Cheverud, 1994, 1995). We did not use one marker (D2Mit1), however, because recombination percentages between this marker and the next marker on chromosome 2 (D2Mit17) approached 50% (Cheverud et al, 1996). The positions of the other 75 microsatellite loci were determined using Mapmaker 3.0b program (Lander et al, 1987; Lincoln et al, 1992), with all distances between markers calculated by this program having previously been given (see Cheverud et al, 1996).

All mice were skeletonized and the left and right sides of the mandibles separated at the mandibular symphysis to allow digitizing of the tooth rows on each side. A total of 10 landmarks on the left and right mandibular molar rows were digitized twice (in two separate measurement sessions) in a total of 502 mice (Workman et al, 2002). These points chosen ensured an adequate representation for the first (M1), second (M2), and third mandibular molar (M3) that comprise the molar row (see Figure 1).

Figure 1
figure 1

Locations for the 10 landmarks on the first (M1), second (M2), and third (M3) mandibular molars that comprise the molar row.

Full size image

Size and shape asymmetry

The 10 landmark points on each side of the tooth row were subjected to a Procrustes superimposition technique that has been especially adapted for the study of FA of shape (Klingenberg and McIntyre, 1998). This procedure produced a new set of shape variables from the original x,y values by a series of steps that eliminated effects of size, location, orientation and reflection (see details in Klingenberg and McIntyre, 1998). It also produced an overall measure of size of the tooth row, centroid size, by calculating the square root of the sum of the squared distances between each landmark and the mean x and y values for the entire configuration of points.

Measurement error was assessed via a conventional mixed model two-way ANOVA for molar centroid size (Palmer, 1994) and by a two-way Procrustes ANOVA for the shape variables (Klingenberg and McIntyre, 1998). In these analyses, the main factors were individuals and sides, with the error variances reflecting measurement error and the sides by individuals interaction assessing the extent of FA (Palmer, 1994). Measurement error accounted for 8.9% of the variance for molar centroid size and nearly 22% of the total variance for molar shape (Workman et al, 2002). Although this amount of measurement error was substantial, especially for molar shape, the interaction mean square was significantly greater than the error mean square for both molar size and shape, suggesting significant FA (Palmer, 1994). Once this preliminary assessment of measurement error was completed, we used the mean of the two replicates of left and right sides for both centroid size and for each of the 20 shape variables in all subsequent analyses. This effectively reduced the measurement error contribution (which is distinct from that contributed by single- or two-locus genetic variation) to the total phenotypic variation by one-half.

While FA is regarded as a property of populations rather than individuals (Palmer and Strobeck, 2003), it was necessary to calculate individual FA values for the genetic analysis. To calculate measures of molar centroid size FA, the distribution of signed differences of left and right sides of centroid size was first examined and found to be normal, suggesting that there was no apparent antisymmetry, a kind of asymmetry characterized by significant platykurtosis (Palmer and Strobeck, 1992). The mean of the right minus left differences of centroid size then was subtracted from the signed difference between the sides to statistically adjust for the directional asymmetry present (Palmer, 1994). The absolute values of these adjusted differences provided measures of FA of centroid side. Although the distribution of these absolute differences between sides was half-normal as expected (Palmer, 1994), we did not transform these values to achieve normality since these values estimate variances of left minus right signed differences, the conventional measure of FA (Palmer and Strobeck, 2003). These unsigned differences were adjusted for potential effects of sex, dam, experimental block, and litter size (see Cheverud et al, 1996) in the manner previously explained (Workman et al, 2002). They also were regressed on centroid size left–right means, but the nonsignificant result suggested that there was no scaling problem and therefore no further adjustment of these molar centroid size asymmetry measures was necessary prior to the epistasis analysis.

For molar shape, an individual score for FA was needed for the genetic analysis, but Procrustes distance is problematic because the variation is not isotropic. An appropriate measure of individual shape asymmetry therefore should be relative to the amount of asymmetry variation in each direction. We calculated a single measure of molar shape FA with a new procedure suggested by Klingenberg and Monteiro (unpublished manuscript). Their solution produces a measure analogous to Mahalanobis distance instead of Procrustes distance.

To obtain the shape FA measure, signed differences of left and right sides for all 20 shape variables were calculated and adjusted for any directional asymmetry in the same manner as described for centroid size. A multivariate regression procedure of these signed differences on the signed differences for molar centroid size was significant (P<0.01), so we adjusted each variable for this allometric association by obtaining the residuals from this regression. A principal components analysis was run on the covariance matrix of these residuals, and standardized component scores were calculated for the first 16 components (the last four components had zero eigenvalues) by dividing each score by the square root of its appropriate eigenvalue. The square root of the sums of squares of these standardized PC scores was adjusted for effects of sex, dam, experimental block, and litter size, and provided our measure of FA of molar shape.

Single-locus quantitative trait loci (QTL) analyses

Prior to the epistatis analysis (see below), we ran separate single-locus QTL analyses for FA of molar size and FA of molar shape via the approach to interval mapping described by Haley and Knott (1992) and implemented previously for molar size and shape in these same mice (Workman et al, 2002). For each of these two FA characters, this approach consisted of multiple regression of the FA values on genotypic index values. Additive (+1, 0, −1) and dominance genotypic index values (0, 1, 0) were assigned for the LG/LG, LG/SM and SM/SM genotypes at each marker location and also estimated for all locations 2 cM apart between flanking microsatellite markers on each chromosome from the recombination percentages between markers (see Cheverud et al, 1996) and the equations in Haley and Knott (1992). We also used microsatellite markers located on chromosomes other than the one being analyzed as conditioning variables in each analysis to account for the effect of background QTLs (Zeng, 1994). The markers chosen for conditioning for the analysis of FA for both molar size and shape were those reaching significance in preliminary stepwise multiple regression analyses.

For each 2 cM interval, the regression analyses provided F statistics with their associated probabilities that were converted to linkage odds (LOD) scores used to test the null hypothesis that no QTL was present. If the highest LOD score on each chromosome exceeded a threshold value appropriate for that chromosome, it was assumed a QTL was present at the position of the LOD score. In that case, the regression coefficients were used as estimates of the additive (a) and dominance (d) genotypic values of the QTL (Falconer and Mackay, 1996).

Threshold values were obtained from permutation tests (Churchill and Doerge, 1994) conducted for FA of molar size (and shape) for each individual chromosome. Each permutation test consisted of 1000 iterations where the measures of FA for tooth size/shape for each individual mouse were randomly permuted, merged with the genotypic index scores and appropriate conditioning markers, and then run through the canonical correlation analysis. These permutation runs provided 5% threshold values for each chromosome as well as a 5% experimentwise threshold value applicable across all chromosomes (see Workman et al, 2002). LOD scores exceeding the chromosomewise threshold values are suggestive of QTLs, whereas LOD scores exceeding the experimentwise level provide significant evidence for QTLs (Lander and Kruglyak, 1995).

Epistasis analyses

We conducted separate two-way genome scans to test for physiological epistasis (Cheverud and Routman, 1995) for FA in molar size and in molar shape in the manner previously described for mandible centroid size and FA in mandible centroid size (Leamy et al, 2002). To initiate these tests, all additive and dominance genotypic index values first were merged with centroid size and shape asymmetry values and subjected to canonical correlation analysis via the CANCORR procedure in SAS (SAS Institute Inc., 1989). Canonical correlation runs were made using every location 2 cM apart on each of the 171 pairs of 19 autosomes. Each such analysis used molar centroid size FA (or shape FA) as a single character in one group, and all four interactions of the additive and dominance genotypic scores from the two chromosomes (XaijXakl, XaijXdkl, XdijXakl, XdijXdkl) as the characters in the other group while partialing the main effects associated with the additive and dominance genotypic scores (Xaij, Xdij, Xakl, Xdkl).

These analyses generated probabilities from F approximations to Rao's statistic for each pair of locations on each pair of chromosomes. These pairs of positions exhibiting the lowest probability within valleys of probabilities of 10% or less were assumed to be likely combinations for potential epistasis. Two-chromosome plots of probabilities often yielded two or more valleys clearly separated by ridges (areas with probabilities greater than 10%), and in these cases, each valley was accepted as a potential separate instance of epistasis.

We next conducted multiple regression analyses of the centroid size and shape FA values on the genotypic index values (and all four of their pairwise products) at each of the combinations of sites previously identified. The regression coefficients from these analyses estimated genetic parameters that were converted into additive (aA, aB) and dominance genotypic values (dA, dB) for each locus and their epistatic interactions (aAaB, aAdB, dAaB, and dAdB) using equations given in Routman and Cheverud (1997). The regression analyses also provided tests of the overall significance of epistasis from F statistics calculated by the mean of the squared t-values associated with the four individual genotypic epistasis terms (Routman and Cheverud, 1997). If significant overall epistasis was detected, the significance for each of the four genotypic epistasis terms (aAaB, aAdB, dAaB, dAdB) was tested with t-tests (Routman and Cheverud, 1997).

We adjusted for the multiple comparisons problem inherent in the many tests for epistasis by first calculating the effective number of independent tests for each chromosome (Cheverud, 2000, 2001). This number was calculated as Me=M(1−[Vλ(M−1)/M2)]), where M is the number of markers scored (75) and Vλ is the variance of the eigenvalues of the correlation matrix of markers. The total number of independent epistasis tests then was estimated by the sum of the crossproducts of the effective number of markers for all 171 pairs of chromosomes. This calculation yielded a sum of 1850, suggesting that we might expect about 92 tests to be significant at the 5% level (19 at the 1% level and 2 at the 0.1% level) because of chance alone. Epistasis was indicated if the number of F values reaching conventional significance for epistasis significantly exceeded 92 (tested with a χ2 statistic associated with 1 d.f.).

Although the procedure above allowed us to test for overall epistasis, we corrected our threshold level of significance via the Bonferroni procedure in order to test for individual instances of epistasis. Specifically, epistasis was considered to be significant at the experimentwise level when a given probability from the F test reached the 0.1 Bonferroni threshold level of significance, 5.41 × 10−5 (0.1/1850). Following Peripato et al (2002), we also considered epistasis to be significant if any of the four epistatic components reached significance at the 1.35 × 10−5 (5.41 × 10−5/4) level even if the F test for overall epistasis was not significant.

Results

FA of molar centroid size

For FA of tooth centroid size, the highest LOD scores achieved in the QTL analyses for each of the 19 autosomes varied from 0.31 (chromosome 19) to 1.99 (chromosome 5) and averaged 0.93. None of these 19 values exceeded the 5% chromosomewise threshold values (range 1.52–2.17, mean=1.94), and thus there is no evidence for any single-locus QTLs that affect centroid size FA.

The results of the two-way genome scan for FA of tooth centroid size produced 173 F values with associated probabilities less than 5%, this being significantly greater than the number (92) expected at this level by chance alone (χ2=75.0, d.f.=1, P<0.0001). Among these 173 cases, the numbers of the aa, ad, da, and dd epistatic components significant at the 5% levels are 43, 62, 60, and 66. Thus aa types are the lowest in frequency, but the frequencies of all four types do not differ significantly from a theoretical 1:1:1:1 ratio (χ2=5.35, d.f.=3, P>0.05). At the 1% significance level, there were 68 cases of overall epistasis, again more than the number (19) expected by chance alone at this level (χ2=127.7, d.f.=1, P<0.0001). At the 0.1% level, there are still 16 cases of significant epistasis, where only two cases are expected due to chance alone (χ2=98.1, d.f.=1, P<0.0001). Thus, epistasis for FA in tooth centroid size is abundant.

Five pairwise combinations of QTLs showed epistasis for tooth centroid size FA at the 0.1 experimentwise level of significance (see Table 1). As may be seen, these combinations involve chromosomes 1 (two instances), 4, 7 (two instances), 9, 10 (two instances), 11, and 12. A total of seven epistatic components, four positive and three negative in sign, are statistically significant. Four of the seven are dd types, although the numbers of epistatic components do not deviate significantly from a theoretical 1:1:1:1 distribution (χ2=5.0, df=3, P>0.05). Dominance-by-dominance epistasis (dd) implies that the single-locus dominance genotypic value at a given locus changes depending on the genotype at another locus and vice versa. An example of this type of epistasis for the QTLs on chromosome 7 at D7Mit21+36 and on chromosome 10 at D10Mit20+14 affecting molar size FA is illustrated in Figure 2a. Note the apparent underdominance (where heterozygotes are less asymmetric than the homozygotes) for all rows and columns that involve homozygotes at the interacting QTLs, but overdominance (where heterozygotes are more asymmetric than the homozygotes) when either the QTL at D7Mit21+36 or the QTL at D10Mit20+14 is heterozygous.

Table 1 Epistatic combinations of QTLs affecting FA of tooth size that reached significance at the 0.1 experimentwise level of significance
Full size table
Figure 2
figure 2

Dominance-by-dominance epistatic interactions between QTLs on chromosomes 7 and 10 affecting molar size FA (a) and additive-by-dominance interactions between QTLs on chromosomes 3 and 5 affecting molar shape FA (b).

Full size image

To assess the relative effect of epistasis on the total variability of centroid size FA, we ran a series of multiple regressions. A multiple regression of centroid size FA on the additive and dominance direct effects of the loci involved in the significant epistatic interactions generated a multiple coefficient of determination of 6.5% (or when adjusted for the number of parameters, 2.4%), but the F value did not quite reach significance (P=0.056). Adding the seven significant epistatic components to this model accounts for 21.8% (17.0% when adjusted) of the variance and is significant (P<0.0001), suggesting that epistasis from just these seven combinations of loci accounts for nearly 22% of the total variability of tooth size FA.

FA of molar shape

The interval mapping analysis for FA of tooth shape suggested two potential QTLs, one on chromosome 9 and one on chromosome 11 (Table 2). The LOD scores in each case exceeded their appropriate chromosomewise threshold values (1.81 and 2.07, respectively), although neither reached the 5% experimentwise threshold value of 3.42. The putative QTL on chromosome 9 acts in an overdominant manner, while that on chromosome 11 is primarily additive in its action and its negative additive genotypic value suggests that the LG/LG genotype has the smallest tooth shape FA. Both QTLs contribute less than 3% of the total variation for tooth shape FA.

Table 2 LOD scores, locations, confidence intervals (CIs), and unstandardized additive (a) and dominance effects (d), and their standardized values (2a/s, d/s) for the two QTLs significantly affecting FA of tooth shape
Full size table

The two-way genome scan for FA of tooth shape yielded 193 combinations of loci that generated F values with associated probabilities less than 5%, 74 combinations at the 1% significance level, and 23 combinations at the 0.1% significance level. As was the case for FA of tooth centroid size, all these represent far more than the number of combinations expected by chance alone (P<0.0001) and thus there is ample evidence for epistasis for FA of tooth shape as well. Among the 193 combinations significant at the 5% level, the numbers of significant (P<0.05) aa, ad, da, and dd epistatic components, respectively, are 48, 78, 65, and 88. These components deviate from a theoretical 1:1:1:1 ratio (=12.8, P<0.01) chiefly because aa types are under-represented and dd types are over-represented.

Five pairwise combinations of QTLs showed epistasis for tooth shape at the 0.1 experimentwise level of significance (Table 3). These combinations involve QTLs on chromosomes 3 and 4 (two instances each), 5, 8, 13, 14, 15, and 16. All QTLs except those on chromosome 4 are on different chromosomes than the QTLs found to epistatically affect FA of molar centroid size (Table 1). A total of six epistatic components reach significance, four of which are ad types. Additive-by-dominance epistasis implies that the single-locus dominance genotype value at a given locus (say A) changes depending on what genotype is at another locus (B), whereas the single-locus additive genotypic value at the B locus differ depending on the locus A genotype. This is illustrated in Figure 2b for the two QTLs on chromosome 3 at D3Mit13+2 and on chromosome 5 at D5Mit6+12 that affect molar shape FA. Note that D5Mit6+12 shows overdominance for the D3Mit3+2 LL homozygotes, but underdominance for the the D3Mit3+2 SS homozygotes. Further, D3Mit3+2 LL homozygotes are smaller than SS homozygotes for both D5Mit6+12 SS and LL homozygotes, but the pattern is reversed for D5Mit6+12 heterozygotes.

Table 3 Epistatic combinations of QTLs affecting FA of tooth shape that reached significance at the 0.1 experimentwise level of significance
Full size table

The two single-locus QTLs affecting tooth shape FA jointly account for 4.5% (or 3.6% when adjusted for the number of parameters) of its total variation. The main effects of these two single-locus QTLs plus the 10 loci involved in the five epistatic interactions account for 9.9% (adjusted multiple r2=4.5%) of the total variation in tooth shape FA, but this is not a significant improvement in fit from the single-locus model (χ2=21.1, P>0.05). When the six significant epistatic effects are added to the multiple regression model, however, there is a significant increase (χ2=56.5, P<0.01) to 21.4 (adjusted multiple r2=15.5%). Thus, two-locus epistasis significantly influences tooth shape FA by contributing a 17% (or 12% when adjusted) increase to its total variation over that due to single-locus genetic effects.

Mandible size FA epistatic comparisons

For ease in comparison, Table 4 shows the locations of pairs of QTLs exhibiting epistasis at the 0.1 experimentwise significance level for FA of molar size and molar shape, as well as FA of mandible centroid size previously discovered in these same mice by Leamy et al (2002). Locations of the QTLs on each of the two chromosomes (C1 and C2) involved in these interactions are given in cM from the most proximal marker. For FA in both molar size and shape, none of the epistatic chromosomal combinations match those exhibited by mandible size FA. Thus, the epistatic interactions of QTLs affecting FA in the molar characters are entirely independent from those affecting FA in mandible size. Further, only two individual QTLs on chromosome 7 (7–110 for molar size FA, 7–98 for mandible size FA) are on the same chromosome at a location within 15 cM from each other. Therefore few, if any, of the individual genes involved in epistatically affecting FA in the three characters appear to be the same.

Table 4 Locations (in cM from the most proximal marker) of QTLs on each pair of chromosomes (C1 and C2) exhibiting significant epistasis at the 0.1 experimentwise significance level for FA of molar size, FA of molar shape, and FA of mandible size
Full size table

Discussion

This study was conducted primarily to test whether there was an epistatic genetic basis to FA in molar size and shape in the F2 intercross population of mice, and in fact there was overwhelming evidence for this. Many more instances of epistasis were found than were predicted by chance alone for FA in both size and shape of the mandibular molar row, especially at the 0.1% significance level. Although it is impossible to distinguish which of these results might have been false positives, we reduced their expected frequency to one in 10 across the entire genome scan by reporting only those results significant at the 0.1 experimentwise level of significance. This significance level seemed particularly appropriate to balance the risks both of detecting false positive results and of not detecting many true instances of epistasis. This also was the level of significance used by Peripato et al (2002), (2004) in their analysis of epistasis for maternal effects and litter size in LG/J by SM/J intercross mice, making it possible to compare our results directly to theirs.

Genetics of molar size FA

Use of this experimentwise level of significance revealed five pairwise interactions of putative QTLs on seven different chromosomes, including two each on chromosomes 1, 7, and 10, that affected FA of molar centroid size. Confidence intervals (CIs) for the two QTLs on chromosome 1 and on chromosome 10 (but not on chromosome 7) overlap, so eight distinct regions among the autosomes are involved in these epistatic interactions. These are the regions showing the most highly significant epistatic effects, however, and no doubt many more regions also are involved in epistatic interactions that affect molar size FA. Since 173 combinations of QTLs showed significance epistasis at the 5% level where 92 are expected by chance alone, for example, this suggests that there may be as many as 81 true cases of epistasis for molar size FA. If so, we would expect the total impact of epistasis on the variance of molar size FA to be even higher than that of 22% contributed from just the five interactions significant at the experimentwise level.

Four of the seven forms of epistasis exhibited among these five interactions of QTLs affecting molar size FA were of the dominance-by-dominance (dd) type. The two positive dd values (one generated by the interaction of D7Mit21+36 and D10Mit20+14 previously illustrated in Figure 2a) imply that the single heterozygotes have less FA than either the double homozygotes or the double heterozygotes, whereas the reverse is true for the two negative dd values. FA has often been thought to decrease with increasing heterozygosity (reviews in Polak, 2003), in which case we might expect predominantly positive dd epistatic types. However, this expectation did not hold for dd values calculated from epistasis significant at the 0.1% (5+, 4−), 1% (12+, 19−), or 5% level (29+, 39−). For the 16 epistatic interactions reaching significance at the 0.1% level, the mean molar size FA of the four heterozygotes (0.0414) is lower than that for the four homozygotes (0.0444), but their difference does not reach significance (P=0.22). For mandible size FA in this same population of mice, however, the mean of the four heterozygotes of 0.406 was significantly less (P<0.01) than that of 0.433 for the four homozygotes (Leamy et al, 1997).

Although epistatic effects obviously were important for molar size FA in our intercross mice, no additive genetic variance was detectable for this character, at least as evidenced from the absence of significant single-locus QTL effects. A low or nonsignificant heritability is typical for FA in most characters (see review by Fuller and Houle, 2003), however, and therefore was expected here as well. On the other hand, inclusion of all single-locus effects from the four pairs of QTLs exhibiting significant epistasis generated a near-significant result in a multiple regression model. And a model that included all single-locus effects from the QTLs exhibiting significant epistasis at the 0.1% level generated a significant result (P=0.009) and a multiple r2 of 45.7% (adjusted r2=18.5%). Thus, it would appear that even though no individual QTL could be identified as significantly affecting molar size FA, single-locus effects at a number of loci collectively may influence this character.

Genetics of molar shape FA

FA of molar shape was significantly affected by two individual QTLs, one on chromosome 9 and one on chromosome 11. The LOD scores in both cases reached significance at the chromosomewise level only, however, and since one false positive QTL is expected at this level by chance alone, these QTLs are suggestive only and would need verification from additional studies. Thus, evidence for the presence of individual loci that independently affect FA of molar shape in our population of mice must be considered rather weak. This result is in sharp contrast to the 18 QTLs discovered for molar shape itself in these mice, all of which had LOD scores that reached significance at the 5% experimentwise level (Workman et al, 2002). Peripato et al (2004) also detected only two suggestive QTLs affecting litter size in mice, although their joint contribution to the variance of litter size of 12.5% was considerably greater than that of 4.5% for the two QTLs detected here that affected molar shape FA.

As was the case with FA of molar size, the two-way genome scan uncovered many instances of significant epistasis for FA of molar shape. A total of 23 pairwise combinations reached significance at the 0.1% level, for example, this being 11 times greater than the number (2) expected by chance alone. Use of the more stringent experimentwise level of significance reduced the number of significant epistatic pairs to 5 that involved putative QTLs in distinct regions on eight different chromosomes. All regions except one on chromosome 4 were different from those of QTLs epistatically affecting molar size FA, so the epistatic combinations affecting FA in molar size versus FA in molar shape appear rather distinct. Comparisons of epistatic combinations of QTLs reaching significance at the 0.1% level also showed much the same trend: only one combination (involving chromosomes 13 and 15) among the 23 for molar shape FA and the 16 for molar size FA involved QTLs that both were in the same regions.

Significant additive-by-dominance epistatic types were most conspicuous among the five combinations of QTLs significantly affecting molar shape FA, although overall (at the 5% significance level) dd types were the most prevalent. The numbers of positive and negative dd values were about the same at the 5% level (42+, 44−), but positive values predominated at the more stringent 1% (15+, 8−) and 0.1% (10+, 3−) significance levels. Additionally, the one dd value significant at the experimentwise level also was positive in sign (Table 3). This suggests that heterozygotes should show less FA in molar shape than homozygotes, and in fact there is a significant difference (P=0.004) between the mean molar shape FA values of 3.83 and 3.99 for the four heterozygotes and four homozygotes, respectively, at each of the two loci for all 23 epistatic interactions reaching significance at the 0.1% level. Epistatic dd values for other characters such as body weight in LG/J by SM/J intercross mice are predominantly negative in sign (Cheverud, 2000), so body weight would be expected to show positive heterosis in crosses of inbred lines.

Genetic architecture of molar FA

Our results suggest that, as was the case for FA of mandible size (Leamy et al, 1997, 2002), the genetic basis of FA in mandibular molar size and shape in this population of mice is almost exclusively epistatic in nature. No individual QTLs were found that significantly affected FA of molar size, and only two QTLs affected FA of molar shape, both significant only at the chromosomewise level and jointly accounting for only about 4% of the total variance. However, many QTLs were found whose epistatic interactions affected FA in molar size/shape, the most significant five pairs of which contributed an average of about 20% of the total variation in these characters. Thus, there appears to be at least a moderate amount of genetic variance for FA in these molar characters, but it is nonheritable rather than heritable in origin.

This general pattern for FA in the tooth characters is strikingly similar to that discovered for two fitness characters, litter size and maternal performance, in LG/J by SM/J intercross mice (Peripato et al, 2002, 2004). Both of these characters were significantly affected by only two individual QTLs but by a number of epistatic combinations of QTLs that contributed substantial nonadditive genetic variance (Peripato et al, 2002, 2004). They also exhibit low or nonsignificant heritabilities in outbred populations (Peripato et al, 2004). This is a genetic profile that is much different than nonfitness characters such as body size and skeletal size and shape that are affected by many single-locus QTLs each with generally small effects (Cheverud et al, 1996, 1997, 2001; Leamy et al, 1999; Klingenberg et al, 2001; Workman et al, 2002).

While the genetic architecture for FA in both molar size and in shape suggests that both may somehow be related to fitness, the genes involved in the epistatic interactions reaching significance at the experimentwise level appear to be different for these two characters. However, because of the limited statistical power inherent in this QTL experiment, there may be other undetected epistatic interactions that commonly affect FA in both molar characters. Nonetheless, the relative independence of the genes involved in the epistatic interactions reported here for FA in the two characters is not particularly surprising since only one of three QTLs affecting molar centroid size mapped in about the same location as any of the 18 QTLs found to affect molar shape in these same mice (Workman et al, 2002). Workman et al (2002) actually only reported QTLs reaching significance at the experimentwise level, and six additional QTLs affecting molar size reached significance at the chromosomewise level, two of which also were at locations similar to those found for QTLs affecting molar shape. Altogether, then, only three of the 18 QTLs affecting molar shape may be commonly affecting molar size as well.

The nature of the genes involved in the epistatic interactions affecting FA in molar size/shape is unknown, but it seems reasonable to hypothesize that some or most of these genes are among those affecting molar size/shape. Klingenberg and Nijhout (1999) showed that epistasis for FA in a given bilateral character can be produced as a normal outcome of the developmental processes controlled by genes responsible for the bilateral character itself. Of the QTLs involved in epistatic interactions, six of the 10 affecting molar size FA and seven of the 10 affecting molar shape FA, respectively, map to locations within the CIs of individual QTLs for molar size and shape (Workman et al, 2002). This suggests that some of the genes directly controlling size or shape of the mandibular molar row may be involved in epistatic interactions as well. Other genes epistatically affecting FA in molar size or shape may be more indirectly involved by controlling overall tooth/bone mineral deposition or hormones and their receptors. For example, a potential candidate for the QTL near the distal end of chromosome 1 that epistatically affects molar size FA (see Table 4) is Tbbmd1 (location 110), a gene that controls total body bone mineral density (Masinde et al, 2002).

Implications

If some of the genes involved in the epistatic interactions described above become fixed because of forces such as selection or inbreeding, this could lead to the generation of additive genetic variance for FA in the molar characters. For the dd epistatic type most prevalent for molar size FA, for example, the maximum additive genetic variance is produced when an allele is fixed at one locus and the frequency of alleles at the other epistatic locus is either 0.15 or 0.85 (Cheverud and Routman, 1996). Thus, heritability levels for FA in various characters in natural populations might be expected to change as epistatic interactions vary in response to evolutionary pressures. However, the fact that most heritability estimates for FA have been consistently quite low in magnitude (average about 0.04; Fuller and Houle, 2003) suggests that the generation of additive genetic variation is probably transitory and generally is followed by the fixation of alleles at most individual loci affecting FA.

Given the difference in the genes involved in epistatic genetic interactions affecting FA in molar size versus molar shape, we might expect different characters to respond differently to various stressors such as inbreeding or selection. Thus, if this LG/J by SM/J intercross population of mice experienced inbreeding that increased the overall level of homozygosity, changes in the level of FA in molar size might be very different from that for molar shape. Fixation of the LG allele at the two loci shown in Figure 2a would increase molar size FA, for example, whereas if this occurred for the two loci shown in Figure 2b, molar shape FA would decrease. It has been increasingly recognized that FA responses to a given stressor generally are character specific (Woods et al, 1999; Indrasamy et al, 2000), and perhaps diversity in the epistatic combinations affecting FA levels in different characters contributes to this specificity.

To the extent that epistatic genetic interactions are responsible for maintaining certain levels of developmental stability in populations, meaningful comparisons of FA levels between wild or laboratory outbred populations stressed/not stressed may be quite difficult. While it has been typically assumed that the heritability of FA in most populations is negligible and that there is little genetic variation confounding FA comparisons among populations, the potential confounding role for epistasis has been much less appreciated. The use of isogenic individuals (clones) for comparing FA levels in populations subjected/not subjected to a given stressor would circumvent this problem. This should be feasible in laboratory populations and in some cases in wild populations as well (see Kristensen et al, 2004).