The popularity of the pedometer as a tool for objectively assessing physical activity is increasing, in part due to its unique combination of validity (1,3), simplicity, and affordability (12). A simple search of the PubMed database from the years 1993 to 2003 using the keyword “pedometer” elicits 123 responses, 44 (36%) of which were published between 2000 and 2003. The availability of new commercial pedometer brands is also expanding quickly, making between-study comparisons of pedometer-determined physical activity questionable. Recent reviews of published studies have catalogued that the pedometers most frequently used in research are manufactured by Yamax, Omron, and Sportline (12).
Of the three, the most data has been collected using Yamax pedometers, sparked in large part by a 1996 study by Bassett and colleagues (1). That study documented the superior performance of the Yamax DW-500 brand compared to four others under controlled bouts of treadmill exercise (although all pedometers had increased error during slow walking). Since that study by Bassett et al., the specific Yamax model tested has been discontinued, and new models from Yamax, as well as several other manufacturers, have appeared on the market. Two recent studies have examined the validity of 10 electronic pedometers under controlled conditions (3) and during a 400-m walk (8). Under controlled conditions, most pedometers were fairly accurate at walking speeds of 80 m·min−1 and above. Six pedometers were accurate to within 1% of actual steps taken at speeds of 80 m·min−1 and above, whereas only four pedometers showed acceptable accuracy at the slowest speed of 54 m·min−1. During a 400-m walk at self-selected speeds where the average walking speed was 96.5 m·min−1, the Omron significantly overestimated the actual number of steps, the SL underestimated the actual number of steps, and the Yamax was exceptionally accurate (8).
Instrument performance can be evaluated against observed steps taken using the established treadmill protocol designed by Bassett and colleagues. However, researchers and practitioners are also interested in how instruments compare under free-living conditions (10). Furthermore, instrument performance has consistently been presented as mean percent error based on the difference between steps detected and observed steps, computed as: ((steps detected × observed steps)/observed steps × 100) (1,3,5,8). Because pedometer error can vary in direction (over- and undercounting of steps), studies that have described pedometer error without accounting for the direction of error may have overestimated motion sensor accuracy by calculating the mean of positive and negative percent error values. For example, if a pedometer over counts by 5% and another un-dercounts by 10% (−10%), the resultant mean percent error derived by summing the error terms and dividing by two yields a mean percent error of −2.5%. Using the absolute value of mean percent error ((5% + 10%)/ 2) produces a mean percent error of 7.5%. This example clearly illustrates how calculating the mean without adjusting for the direction of error compresses the mean percent error and underestimates motion sensor error. Thus, reporting accuracy in terms of mean percent error without accounting for the direction of error will misrepresent the magnitude of error attributed to motion sensors.
Therefore, we undertook two studies to examine the concurrent accuracy of the Yamax SW-200 (YAM), Omron HJ-105 (OM), and Sportline 330 (SL) pedometers, as well as a CSA accelerometer, under controlled conditions. Study 1 examined motion sensor accuracy during treadmill walking at five different speeds, and study 2 examined motion sensor accuracy under a 24-h free-living condition using the CSA accelerometer as the criterion standard.
METHODS
Participants.
A convenience sample of male and female participants between the ages of 20 and 55 yr of age participated in both studies (N = 12; 6 males, 6 females). The sample size of the present study is consistent with previously published research addressing motion sensor accuracy (3). Procedures were reviewed and approved by the Institutional Review Board at Arizona State University. Written informed consent was obtained from all subjects before participation. Height and weight were assessed in light street clothing (without shoes) to the nearest 0.5 cm and 0.1 kg, respectively. Because BMI > 30 kg·m−2 has been implicated as a source of error when using motion sensors (9), all participants were specifically recruited to be lower than this cut point. Characteristics of the participants are presented in Table 1.
;)
TABLE 1:
Participant characteristics.
Instruments.
The Yamax SW-200 (Yamax Corporation, Tokyo, Japan), Omron HJ-105 (Vernon Hills, IL), Sportline 330 (Campbell, CA), and a dual-mode CSA accelerometer (CSA; model 7164 version 2.2, MTI Health Services, Fort Walton Beach, FL) were used in both studies. The YAM pedometer and the SL digitally display step counts. The OM pedometer displays step counts, distance in miles, kilocalorie counts, and time of day. For purposes of this study, only the step count feature was used. The OM also has a variable sensitivity switch that remained in the middle (neutral) position at all times. Five new pedometers of each brand were checked for calibration. Pedometers were checked using a shake test (13), and a brief walking test (11). If the error exceeded ±5% (e.g., greater than one step error in 20 total steps) in either calibration test, the pedometer was not used for study purposes. All instruments passed the shake test. All five YAM pedometers passed the walking test, whereas only three of five (60%) of both the OM and SL pedometers met the calibration criteria.
Accelerometers were checked for calibration using manufacturer-recommended hardware and software, and calibrated if necessary. The CSAs were initialized to detect steps taken in 30-s epochs (study 1) or 1-min epochs (study 2) and synchronized to the investigator’s timing device. All motion sensors were worn concurrently (on an elastic waistband) using bilateral attachment on the right and left hips according to the manufacturer’s recommendations (e.g., SL: “for best results, keep the unit in line with the crease line of your trousers”). The order of bilateral attachment was counterbalanced between participants so that each monitor was worn in a variety of positions. Recent research has demonstrated that positioning of the pedometer (right or left leg) is not an important threat to accuracy (3). Although no studies have examined the effect of positioning (right and left hip) on the interinstrument reliability of the CSA when detecting steps, two studies (2,7) have demonstrated that the CSA has moderate to high interinstrument reliability at a range of walking and running speeds when detecting activity counts.
The final outputs for each motion sensor in both studies were steps detected. At the end of each study, CSA data were downloaded using manufacturer recommended hardware and software. CSA data represented accumulated accelerometer steps detected for each epoch between washout periods (defined below), verified with synchronized time records.
Study 1: accuracy of motion sensors under controlled conditions.
The purpose of this study was to compare the accuracy of the three pedometers and the CSA relative to observed steps taken while walking on a motor-driven treadmill (Quinton model Q55, Seattle, WA) using an established protocol (1,5). Specifically, participants walked for 5-min bouts at five different speeds (54, 67, 80, 94, and 107 m·min−1). Before testing, treadmill speed was determined by measuring the belt length (3.2 m) and the time it took to complete 25 revolutions of the treadmill belt. A carpenter’s level was used to calibrate the treadmill to a 0% grade according to the manufacturer’s instructions. The accuracy of the carpenter’s level was checked by turning it horizontally 180° and observing that the bubble was still centered, as previously described (1).
Before each bout participants stood still (straddling the treadmill belt) for a 2-min washout period. This was performed to ensure that any steps detected by the accelerometer before the official bout were not considered in the analysis. The 2-min washout period was repeated between each bout and after the last one. At the end of each bout, the number of steps detected by each pedometer was recorded and the unit was reset to zero before a subsequent bout. Observed steps taken were counted and later verified by a video recording aimed at the participant’s lower extremities.
Study 2: accuracy of pedometers under free-living conditions.
The purpose of study 2 was to compare the accuracy of the YAM, OM, and SL pedometers relative to the CSA accelerometer during 24-h of free-living (excluding sleep and water activities). The attachment of the motion sensors was identical to the treadmill condition for all participants. To correct for CSA overcounting error associated with motor vehicle travel, participants were asked to report the odometer-recorded mileage they traveled in any vehicle over the 24-h period.
The CSA was used to estimate the duration that the motion sensors were worn in the 24-h free-living condition. This was accomplished by: 1) synchronizing the computer’s clock (hence the CSA’s clock) to the investigator’s digital timing device, 2) collecting the CSA and pedometers from the participants after a 24-h period, and 3) using the CSA output to determine the amount of time the motion sensors were removed (e.g., for sleep and bathing).
Study 1: data treatment and statistical analysis.
Statistical analyses were carried out using SPSS version 11.0.01 for windows (SPSS Inc., Chicago, IL). For study 1, a two-way repeated measures ANOVA (instrument × speed) was used to compare mean difference scores (motion sensor detected steps − actual steps). When significant main effects were found, a one-way ANOVA with a Bonferroni post hoc procedure was used to determine where the differences existed. This statistical treatment is similar to that used in previous research (3,5,8).
To illustrate the direction and magnitude of motion sensor error under controlled conditions, instrument error at each treadmill speed was computed as percent error similar to previous studies (1,3,5,8) ((steps detected − observed steps)/ observed steps × 100) retaining the instrument effects of over-and undercounting (i.e., + or − in direction). Direction of percent error was categorized as under- (≤1%), exact (within ±1%), or over (>1%) counting of observed steps taken and illustrated as frequencies. The rationale for this strict definition of percent error category was based on research by Bassett et al. (1) that demonstrated the Yamax pedometer exhibited mean scores that were accurate to within 1% of actual steps taken. Computing the mean for percent error by averaging positive and negative values misrepresents the true magnitude of the error incurred. Therefore, the absolute value of percent error (regardless of direction) was calculated and presented as the absolute value of percent error. The absolute value of percent error more accurately describes the magnitude of motion sensor error.
Study 2: data treatment and statistical analysis.
Le Masurier and Tudor-Locke (5) recently showed that the dual-mode CSA accelerometer is more sensitive to slow walking and therefore performs better (with respect to detecting steps taken) than the Yamax pedometer. Therefore, the CSA could be used as a criterion instrument to evaluate pedometer performance under free-living conditions. The single known caveat is that the CSA is also more sensitive to nonstep agitation, including motor vehicle travel, and should be corrected for such error (5). Therefore, based on our previous study of CSA error during motor vehicle travel (5), a more conservative estimate of steps taken detected by the CSA accelerometer was computed, correcting by 12.5 steps for every mile traveled.
A one-way ANOVA was used to compare mean difference scores (pedometer detected steps − CSA criterion steps). A Bonferroni post hoc test was used to determine where significant differences existed. Significance was set a priori at the 0.05 level.
In order to illustrate the magnitude of motion sensor error under free-living conditions, pedometer accuracy during free-living was computed as percent error ((steps detected − CSA criterion steps)/CSA criterion steps × 100) using the CSA criterion corrected for vehicle travel and expressed as the absolute value of percent error. Similar to study 1, the direction of pedometer error was categorized as under (≤1%), exact (within ±1%), or over- (>1%) counting of CSA corrected steps taken and illustrated as frequencies.
RESULTS
Study 1: accuracy of motion sensors under controlled conditions.
The SL pedometer significantly (P < 0.05) underestimated actual steps taken at all treadmill speeds. The number of steps detected by the CSA, YAM, and OM at the five different treadmill speeds was not significantly different from actual steps taken. Although not the focus of the study, we found no significant differences from actual steps taken by the motion sensors between overweight (N = 4) and normal weight participants (N = 8). Figure 1 shows the absolute value of percent error for each motion sensor at the five treadmill speeds. The absolute value of percent error exhibited by the SL was greatest at all treadmill speeds. The absolute value of percent error exhibited by the YAM and OM was large at 54 m·min−1 but decreased at subsequent speeds. The absolute value of percent error was relatively small (<1.5%) for the CSA at all speeds.
FIGURE 1:
Absolute value of percent error scores of four motion sensors at five different speeds.
Figure 2 presents the frequency of under-, exact, and overcounting of steps taken detected by the motion sensors tested at each treadmill speed. In terms of direction of error, the SL underestimated steps taken at all speeds. The YAM tended to underestimate steps at 54 and 67 m·min−1. At 80, 94, and 107 m·min−1, YAM and OM pedometers exactly counted (within ±1%) observed steps taken most of the time. The CSA exactly counted observed steps taken most of the time at all treadmill speeds.
FIGURE 2:
Frequency of under- (≤1%), exact (within 0.99%), and over- (>1%) detection of steps, expressed as percent error of actual steps taken, for four motion sensors at treadmill speeds of a) 50 m·min−1, b) 67 m·min−1, c) 80 m·min−1, d) 94 m·min−1, and e) 107 m·min−1 (N = 12 cases for each motion sensor at each speed).
Study 2: accuracy of pedometers under free-living conditions.
Participants wore the motion sensors an average of 14.3 ± 1.5 h during the free-living condition, similar to previous studies (6,10). Mean steps per day for participants in the free-living condition are presented in Table 2. The mean reported vehicle travel was 32.2 miles·d−1 (SD 25.0 miles). Using the correction factor of 12.5 steps for every mile (see Data Treatment), the mean correction factor for the CSA criterion was 403 steps (SD 249 steps). This correction factor amounted to 3% of the steps detected by the CSA (see Table 2). Compared with the CSA criterion, only the SL pedometer significantly (P < 0.05) underestimated steps taken. Figure 3 illustrates the absolute value of percent error for the three pedometers. The absolute value of percent error was lowest for the YAM during the free-living condition, larger for the OM, and largest for the SL. In terms of the direction of error (over-, exact, and undercounting), the YAM and SL tended to undercount steps, whereas the OM over- and undercounted with similar frequency (Fig. 4).
TABLE 2:
Mean steps per day recorded by four motion sensors under free-living conditions.
FIGURE 3:
Absolute value of percent error for three pedometers in free-living conditions (using CSA as criterion).
FIGURE 4:
Frequency of under- (≤1%), exact (within 0.99%), and over- (>1%) detection of steps, expressed as percent error based on CSA criterion, for three pedometers in free-living conditions (N = 12 cases for each pedometer).
DISCUSSION
Although previous studies have evaluated pedometer and accelerometer performance under controlled (1,5,8) and free-living conditions (10), this is the first study to compare the performance of multiple pedometer brands under free-living conditions using the CSA accelerometer as the criterion standard. Such lines of inquiry can advance our understanding of how objective motion sensors translate and depict physical activity under these contrasting conditions. In addition, previous research (1,3,5,8) presented motion sensor error as mean percent error without accounting for the direction of error. For example, a recent study by Crouter et al. (3) found that only four pedometers (Walk4Life LS 2525, New Lifestyles NL-2000, Yamax SW-701, and Kenz Lifecorder) of the 10 tested showed acceptable validity (within 12% mean percent error, regardless of error direction, of actual steps) at 54 m·min−1. The present study extends this research line by considering the magnitude of motion sensor error by calculating the absolute value of percent error.
Another unique aspect of this study was the presentation of the data in terms of percent error categories, which provided an illustration of the direction of error for each motion sensor (see Figs. 1 and 3). By categorizing the unadjusted percent error in terms of over-, exact, and undercounting of observed steps taken, we were able to identify the error patterns in step measurement for the motion sensors under both controlled and free-living conditions. Under controlled conditions the CSA demonstrated little percent error, that is, it exactly counted observed steps taken the majority of the time at all treadmill speeds. The YAM tended to undercount observed steps at slow speeds, whereas the OM was less consistent in direction of error at the slowest speed, and the SL tended to undercount steps at all speeds.
In terms of the magnitude of error, expressed in the present study as absolute value of percent error, the greatest magnitude of error was evident with the SL at all treadmill speeds (due to consistent undercounting; see Figs. 1 and 2). The absolute value of percent error exhibited by the YAM was relatively large at the slowest speed (54 m·min−1), and was due to consistent undercounting, which is in agreement with previous findings (1,3,5). Hendelman et al. (4) suggested that such slow speeds of walking are much slower than typical normal walking and therefore should not be an important source of error in studies of free-living activity in ambulatory populations. The absolute value of percent error of the OM at 54 m·min−1 was also relatively large, but this cannot be attributed to consistent undercounting or over counting because the direction of error for the OM was inconsistent (see Fig. 2). When mean difference scores were used to statistically analyze the performance of the motion sensors, as was done in previous research (3,8), the SL underestimated actual steps at all speeds. In the study by Crouter et al. (3), the SL only significantly underestimated actual steps taken at the slowest speed (54 m·min−1). However in a recent study by Schneider et al. (8), the SL significantly underestimated steps taken when the average walking speed was 96.5 m·min−1. The accumulating evidence suggests that the SL pedometer is not a good choice for researchers. In contrast, YAM pedometers have consistently performed well regardless of brand (1,3,5,8). In the present study, the YAM SW-200 showed no significant differences from actual steps taken at all treadmill speeds. This finding provides additional support for the accuracy and consistency of YAM pedometers. The OM did not show significant differences from actual steps at any treadmill speeds in the present study but did overestimate actual steps in the studies by Crouter et al. (3) and Schneider et al. (8). This result suggests that OM performs more consistently under controlled conditions than the SL but less consistently than the YAM.
Considering magnitude of error as more important than direction of error, both the YAM and OM were superior to the SL. This finding supports previous research (3). Whereas both the OM and YAM had relatively large magnitudes of error at 54 m·min−1, the OM had less consistent patterns of error at that speed. In terms of error direction, consistent patterns of error (e.g., always undercounting) are desirable; if error pattern is erratic we are unable to adjust it. The accuracy of the CSA at all speeds and the accuracy of the OM and the YAM at speeds of 80 m·min−1 and above were in agreement with previous research (3,5).
Accuracy can be assessed under both controlled and free-living conditions and can be classified into two categories: sensitivity (ability to detect actual steps taken within an acceptable margin of error) and specificity (ability to discriminate against nonstep movements, e.g., from external agitation such as driving in a car) (5). In the present study, we only considered instrument sensitivity. Theoretically, increasing motion sensor sensitivity (i.e., by lowering the force threshold necessary to register a step) will improve a pedometer’s ability to detect steps taken at slower walking speeds but compromises specificity under free-living conditions.
The manufacturers of the YAM pedometer report that their instrument has a 0.35-g threshold (10); forces below that threshold go undetected as observed in the slowest treadmill speed in study 1 herein. The force thresholds of the OM and SL pedometers are not reported by their manufacturers, although extrapolating from the results of study 1 we would expect that the force threshold for the OM would be lower and the SL would be higher than the force threshold of the YAM. The inconsistency of the direction of error pattern of the OM suggests that the force threshold is likewise not held consistently. It should be noted that the OM pedometer has a switch that allows the user to adjust (increase or decrease) the instrument’s sensitivity, which could be useful for detecting steps in populations that have slow or shuffling gates. Caution should be exercised when using this feature for the purposes of research because of the sensitivity/specificity trade-off (i.e., an increase in one leads to a decrease in the other). In addition, inconsistent use of force thresholds would make it difficult to compare results between individuals and/or studies. Knowledge of the OM and SL force thresholds may provide further insight as to their sensitivity and specificity under controlled and free-living conditions.
Study 1 confirmed our choice of the CSA accelerometer for the criterion standard in study 2. This finding supports previous research (5). Compared to the CSA in the free-living condition, the YAM had the smallest magnitude of error (12.8%) followed by the OM (25.8%) and the SL (37.0%; see Fig. 3). Similar to the controlled conditions, the YAM and SL consistently undercounted steps taken, whereas the OM under- and overcounted with similar frequency (Fig. 4). There was not an expectation that any pedometer would be within 1% of the CSA criterion during free-living. In fact, this would require a pedometer to be within 138 steps of the steps detected by the CSA, which itself required a 403-step correction. This minor correction was deemed necessary because the relatively lower sensitivity threshold of the CSA (0.30 g) results in detection of nonstep movements as steps taken (i.e., false positives). Detecting steps within 1% of the CSA criterion (corrected for vehicle travel) is not a realistic expectation and, more importantly, is not a meaningful expectation. This value was used to illustrate the differences in step detection and error direction between the different devices. Because this is the first study to examine pedometer accuracy under free-living conditions using the CSA criterion, we are reluctant to make a judgment regarding an acceptable level of pedometer accuracy. Only one other study (10) has compared pedometer and CSA step counts in free-living conditions. Using the calculation ((YAM steps·d−1 − CSA steps·d−1)/CSA steps·d−1 × 100), the data from previous research (9) shows that the YAM undercounted (relative to the CSA) by 16%. Using the same calculations on the data from Table 2, the YAM undercounted by 10%. The 3% correction factor for vehicle travel, not used in the previous study, improved the agreement in the present study. However, with only two studies to date focused on a single pedometer brand, we are unable to comment on what magnitude of pedometer error is acceptable relative to a CSA criterion under free-living conditions.
Based on the results of magnitude and direction of error, the YAM again appears to be a better choice for researchers examining steps taken in free-living conditions. The YAM pedometer also has some important advantages over the more sensitive CSA for measuring steps in free-living conditions, particularly for field-based research, even though the CSA performs better than the YAM under controlled conditions. The YAM is an accurate, inexpensive ($20–30), and easy to use device that provides immediate feedback to participants and researchers. The CSA is an accurate, expensive ($450 plus the cost of hardware and software to capture data), more difficult device to use that does not provide immediate feedback. The cost of motion sensors is important to consider, because it is not uncommon for participants to lose these small devices. YAM pedometers are also effective for physical activity interventions because participants can monitor their progress throughout the day, record their own step data very easily, and combine these two practices to set and meet step goals.
In summary, this investigation studied motion sensor performance under controlled and free-living conditions. Its strengths include the use of the same participants and the same attachment protocols under both conditions. The results of study 1 confirmed the use of the CSA accelerometer as a criterion standard in study 2. It is clear that different brands of motion sensors detect steps differently. Taking into consideration the results of both studies and the initial walking test used for instrument screening purposes it appears that, of the three pedometers tested, the YAM pedometer is most consistently accurate under both controlled and free-living conditions.
Caution must be used when comparing step counts between studies that have used different brands of motion sensors. To more accurately characterize motion sensor error, future research assessing the accuracy of motion sensors should consider expressing both direction and magnitude of error.
We would like to acknowledge Dr. Pamela Swan’s financial support of this study.
The authors do not have a professional relationship with companies or manufacturers who may benefit from the results of the present study. The results of the present study do not constitute endorsement of the products by the authors or the ACSM.
REFERENCES
1. B, D. R., B. E. Ainsworth, S. R. Leggett, et al. Accuracy of five electronic pedometers for measuring distance walked.
Med. Sci. Sports Exerc. 28:1071–1077, 1996.
2. Brage, S., N. Wedderkopp, P. W. Franks, L. B. Andersen, and K. Froberg. Reexamination of validity and reliability of the CSA monitor in walking and running.
Med. Sci. Sports Exerc. 35:1447–1454, 2003.
3. Crouter, S. E., P. L. Schneider, M. Karabulut, and D. R. Bassett. Validity of ten electronic pedometers for measuring distance walked.
Med. Sci. Sports Exerc. 35:S283, 2003.
4. Hendelman, D., K. Miller, C. Baggett, E. Debold, and P. Freed-son. Validity of accelerometry for the assessment of moderate intensity physical activity in the field.
Med. Sci. Sports Exerc. 32:S442–S450, 2000.
5. Le Masurier, G. C., and C. Tudor-Locke. Comparison of pedometer and accelerometer accuracy under controlled conditions.
Med. Sci. Sports Exerc. 35:867–871, 2003.
6. Leenders, N. Y., W. M. Sherman, and H. N. Nagaraja. Comparisons of four methods of estimating physical activity in adult women.
Med. Sci. Sports Exerc. 32:1320–1326, 2000.
7. Nichols, J. F., C. G. Morgan, L. E. Chabot, J. F. Sallis, and K. J. Calfas. Assessment of physical activity with the Computer Science and Applications, Inc., accelerometer: laboratory versus field validation.
Res. Q. Exerc. Sport 71:36–43, 2000.
8. Schneider, P. L., S. E. Crouter, O. Lukajic, and D. R. Bassett, Jr. Accuracy and reliability of 10 pedometers for measuring steps over a 400-m walk.
Med. Sci. Sports Exerc. 35:1779–1784, 2003.
9. Shepherd, E. F., E. Toloza, C. D. McClung, and T. P. Schmal-zried. Step activity monitor: increased accuracy in quantifying ambulatory activity.
J. Orthop. Res. 17:703–708, 1999.
10. Tudor-Locke, C., B. E. Ainsworth, R. W. Thompson, and C. E. Matthews. Comparison of pedometer and accelerometer measures of free-living physical activity.
Med. Sci. Sports Exerc. 34:2045–2051, 2002.
11. Tudor-Locke, C., and A. M. Myers. Challenges and opportunities for measuring physical activity in sedentary adults.
Sports Med. 31:91–100, 2001.
12. Tudor-Locke, C., and A. M. MYERS. Methodological considerations for researchers and practitioners using pedometers to measure physical (ambulatory) activity.
Res. Q. Exerc. Sport 72:1–12, 2001.
13. VINCENT, S. D. and C. L. SIDMAN. Determining measurement error in digital pedometers.
Meas. Phys. Educ. Exerc. Sci. 7:19–24, 2003.
