Comparing the Validity of a GPS Monitor and a Smartphone Application to Measure Physical Activity

Comparing the Validity of a GPS Monitor and a Smartphone Application to Measure Physical Activity

Manolis Adamakis, PhD¹

¹Department of Physical Education and Sport Science, National and Kapodistrian University of Athens, Greece

Journal MTM 6:2:28–38, 2017

doi:10.7309/jmtm.6.2.4

---

---

Introduction

Every year approximately 3.2 million deaths and 32.1 million illnesses are associated with physical inactivity.¹ In most western societies people between 18 and 35 years belong in the high-risk group of becoming either overweight or obese, due to lack of physical activity (PA).² In order to reverse these statistics, individuals are recommended to accumulate at least 30 min/day, 5 days/week of moderate intensity PA or 15 min/day, 5 days/week of vigorous intensity PA (75 min/week), or a combination of both.³

A recent approach to increasing PA levels, managing weight and improving health has been via technological advances, such as the Web 2.0 technologies, activity trackers and smartphones.⁴ Griffiths and colleagues⁵ proposed that these approaches might be effective due to reduced cost, user-friendly environment, real-time information and feedback provided. Sufficient evidence exists to support the positive effect of these programs with the use of new technologies, especially when these are combined with other research approaches, such as face to face interventions. For example, King and colleagues⁶ used three mobile applications to change the physical (in)activity and sedentary behavior of 95 underactive adults during an 8 week intervention program, and the results provided initial support for promoting PA and reducing sedentary behavior. Also Laurson, Welk and Eisenmann⁷ asked 111 children to wear pedometers over a seven day period and record their steps. Children who wore the pedometers longer appeared more active, with a significant increase of steps per minute.

Global Positioning System (GPS) technology, even though it is quite new in PA and exercise, is a low cost, objective and discreet way to track individuals’ movement.⁸ Most previous studies have concluded that it is a valid way to measure distance travelled. For example, Specht and Szot⁹ tested six GPS receivers and found that logging receivers, such as Garmin Forerunner 310XT, demonstrated the highest accuracy in determining positions. In another similar research, four wearable GPS monitors were compared while four adults walked a distance of 1.24 km and the most accurate device was Garmin Forerunner 205.¹⁰ Using a similar methodological approach, Lee et al.¹¹ chose four low cost GPS minotors and found that all devices were valid for distance estimation, with Garmin 60 to be the most accurate.

Even though GPS monitors are accurate for distance estimation, commercially available applications that use the GPS sensor of the smartphones are sparsely tested for their accuracy. Bauer¹² for example compared ten GPS applications over a 1 km distance, with the Adidas miCoach been the most accurate and Runkeeper having a divergence of 20 m.

Nowadays these monitors and applications have the ability to estimate, using anthropometric and GPS data, the energy expenditure (EE) of individuals during exercise. However, no published research, only a Master’s dissertation¹³ and a conference paper¹⁴ have validated these outcomes. Mallula¹³ compared a Garmin Forerunner 405CX with Nike+ application for iPod and found that the application was more accurate in EE, while the device had a more valid estimation of distance and speed. Furthermore, Adamakis and Zounhia¹⁴ in a preliminary study compared a Garmin Forerunner 310XT with the Runkeeper application for Android smartphones, with the Garmin monitor having the best results in distance, speed and EE, while Runkeeper overestimated all exercise parameters. In conclusion, Bort-Roig and colleagues¹⁵ suggested that well designed studies are needed that comprehensively assess physical activity measurement accuracy, however till these days few researches have validated GPS monitors and smartphone GPS-enabled applications.

The purpose of this study was to compare the criterion validity for distance travelled and total energy expenditure (TEE) between a commercially available GPS monitor, Garmin Forerunner 310XT, and a freeware GPS application for Android smartphones, Runkeeper, under semi-structured activity settings.

Method

Research design

An experimental research design was used. The independent variable was the type of GPS monitor: Garmin Forerunner 310XT and freeware Runkeeper Android application. The primary outcome was distance and the secondary outcome was total energy expenditure (TEE), during walking and running.

Participant

One single, healthy and physically active individual, who could run at least 5 km continuously, took part in all trials. The same protocol was repeated on 40 occasions (20 walking and 20 running sessions) during 40 consecutive days. Relevant characteristics such as height (190 cm), weight (80 kg), gender (male) and age (33 years) were entered into each monitor separately. The anthropometric measures were obtained at the beginning of the data collection session. Standing height was measured to the nearest 0.1 cm with the use of a wall mounted Harpenden Stadiometer (Harpenden, London, UK), using standard procedures. Body mass was measured by participants in light clothes and bare feet on an electronic platform scale (Tanita Corp., Tokyo, Japan) to the nearest 0.1 kg. Following anthropometric measurements, the participant was asked to lay down in bed for 10 minutes and then fitted with the portable indirect calorimeter (Cosmed K4b2) to measure resting energy expenditure (REE). REE was measured for 15 minutes with the subject quiet, but awake. The first 5 min as well as the last minute of measurement were eliminated and the REE was obtained from the average of 9 min. REE measurement was performed after a 10-hour fast, following previously published guidelines.¹⁶ The REE was expressed as kilocalories (kcal) per minute by dividing the TEE by 9 and the estimation was 1.05 kcal × min⁻¹.

Instruments

Smartphone – Samsung Galaxy S4 mini (Samsung Electronics Co., Ltd., Suwon, South Korea): The Samsung Galaxy S4 mini uses Google’s Android Jelly Beam 4.2.2 mobile operating system equipped with a built-in GPS receiver, accelerometer and gyroscope. This phone is small, 125 by 61 by 9 mm, and lightweight, 107 grams.

Runkeeper Android application: Runkeeper, software version 4.4.3 (FitnessKeeper, Inc.; http://Runkeeper.com/) is one of the most commercial applications for Android and iOS smartphones.¹⁷ Runkeeper is a free of charge application, which calculates average pace and speed, laps’ speed, route distance, elevation and estimated calorie burn for a variety of fitness activities, in high accuracy and real time, using the in-built GPS sensor of the Android phone. EE is calculated taking into account distance computed, speed, time, elevation, activity type, body weight measurements and REE. Body measurements are entered manually in the application settings.

It has been downloaded 10 – 50 million times and has a total of 4.5 out of 5 rate out of 421,124 users in the Google Play store (as accessed on 21/9/2016). The program has full and efficient functionality after downloading, with no need for additional software download being necessary. Furthermore it can record and upload exercise data to a computer database after been registered to an online system. It has the ability to track running and walking sessions separately and calculate energy expenditure based on these activities. Additionally, we made sure from its description that the application relied solely on GPS for tracking the user. Finally, Runkeeper has been identified as the application with the most applied behavior changing techniques (eight in number) in the market.¹⁸

Garmin Forerunner 310XT: Garmin, software version 4.50 (Garmin Ltd., USA; https://buy.garmin.com/en-US/US/into-sports/running/forerunner-310XT/prod27335.html), is a GPS-enabled training and heart rate monitor for multisport athletes. Its physical dimensions are 54×56×19 mm, with display size 33×20 mm and lightweight, 72 grams. It is water resistant up to 50 meters, with a 20-hour battery life and a memory history of 1000 laps. Forerunner tracks time, distance, average and lap speed and pace, heart rate with a premium heart rate monitor, on land and estimated calorie burn.

The main method for EE estimation on Garmin fitness monitors uses the Firstbeat algorithm.¹⁹ The calculation takes into account the user’s inputted variables including gender, height, weight and fitness class. It then combines the data with heart rate information from the heart rate strap. More specifically, it evaluates the time between heartbeats (beat to beat) to determine estimated Metabolic Equivalent (MET), which in turn is used to determine actual work expenditure. This method is inexpensive and with relatively high accuracy, with a marginal error of 7-10%.¹⁹ It is considered a reliable method for estimating the rate of oxygen consumption (VO₂) and EE, however it may underestimate these parameters by 6% to 13% respectively.²⁰ Furthermore, this prediction method may be considered sufficiently accurate to determine the average VO₂ in field use, but it does not allow precise estimation of VO₂.²¹

Objective-criterion measurements

Distance: The criterion distance was measured by: a. Leica DISTO™ D810 laser range finder, which provides typical measuring accuracy ±1 mm and has a range up to 200 m,²² b. Calibrated measuring wheel, and c. Google Maps Distance Calculator (http://www.daftlogic.com/projects-google-maps-distance-calculator.htm). The three-way estimated distance course was 3.58 km.

Energy Expenditure: Indirect calorimetry was utilized in this study as the criterion method for TEE.²³ The accuracy of estimated TEE by the two monitors was compared to that measured breath by breath with Cosmed K4b2 (Cosmed S.r.I., Rome, Italy) portable metabolic analyzer. Cosmed allows measurement of oxygen consumption under free-living conditions providing valid and reliable results in the general population.²⁴ Previous research, in which Cosmed was compared with the Douglas airbag method^25,26 and the traditional metabolic analyzer Medgraphics D-Series,²⁷ showed that Cosmed provided comparable energy expenditure estimates in stable, submaximal exercise intensities and the results of oxygen consumed, carbon dioxide eliminated and respiratory quotient ranged within acceptable limits of agreement. It comprises an analyzer unit and a facemask. Volume and gas calibrations (4% CO₂, 16% O₂) were performed before each trial by following manufacturer recommendations.

Experimental procedures

The study was conducted at an urban outdoor environment, on a wooded paved trail with no grade (0%) around an Athens central park (Figure 1). This is an area of 49 acres, which formerly was hosting the military camp ‘Goudi’ and currently is used as a recreational and sports park. Before the start of every session, both GPS monitors were activated and the GPS signal was ensured to be acceptable for adequate measurement. The participant’s personal data were entered manually in the smartphone application and the GPS monitor prior to the start of the test. The chest strap for measuring heart rate for the GPS unit as well as the calorimeter was placed around the chest of the participant, while Garmin was place on the left wrist and the smartphone on the left arm. The calorimeter was placed on a harness over the subject’s shoulders and strapped around the subject’s chest and back. The face mask was placed covering the subject’s mouth and nose so as to not allow air to escape.

Figure 1: The 3.58 km park route.

The 3.58 km test began at the quarter mile mark (point A, depicted on Figure 1) and was run or walked on an out and back course ending at the measured ending line (point B, depicted on Figure 1). The calorimeter needed a brief calibration before being started. This was conducted on the trail prior to the start of the 3.58 km course. When the calibration was complete, the 3 devices (Garmin, Runkeeper and calorimeter) were started simultaneously (± 2 sec) and the participant began to walk or run. One researcher started the calorimeter and the GPS unit, and the participant started the Runkeeper. The participant walked, ran at a comfortable pace, and self-selected speed. At the end of the 3.58 km course all measuring devices were stopped simultaneously (± 2 sec) by the same researcher who started them. The data from all devices were collected and recorded on a data sheet and, afterwards, data were transferred to the online websites of each monitor.

In total, the participant conducted 20 running and 20 walking sessions. The average time needed in order to complete the 3.58 km course running was 21 min 5 sec±55 sec and walking 38 min 22 sec±52 sec. The average heart rate, as measured by Garmin’s heart rate strap, was 155.65±3.95 bpm for running and 99.60±7.91 bpm for walking.

Each monitor uses different outcomes measures to summarize EE data. The Runkeeper provides estimates of TEE, while the Garmin reports estimates of activity EE. In order to provide comparable estimates in the EE results, it was necessary to add REE to the activity EE values for the Garmin, adding the measured REE obtained prior to the activity protocol to the estimated EE. This ensured that we had comparable outcome measures of TEE for both monitors. This procedure was implemented in previous studies²⁸ and ensured that we had comparable outcome measures of TEE for both monitors.

Statistical analysis

The statistical analysis was conducted with the use of the statistical package SPSS version 21.0 (IBM SPSS Corp., Armonk, NY, USA) and the significance level was set at p<0.05. Data analysis was based on Bruton, Conway and Holgate²⁹ recommendations. Before the main procedures, variables were screened for accuracy of data entry, missing values, potential outliers and distribution (skewness and kurtosis) for running and walking separately. No missing values were observed and the box plots, skewness and kurtosis analysis indicated that no extreme values existed and data were approximately normally distributed.

Data were analysed using descriptive (mean, standard deviation) and inferential statistics. A two-way repeated measures ANOVA (measurement method×type of activity) was performed to detect differences in the dependent variables between methods at two intensity levels. Post-hoc analyses using paired t-tests with the Bonferroni correction were conducted to examine specific differences between the two GPS monitors and the criterion methods. Mean absolute percent errors (MAPE) were also calculated to provide an indicator of overall measurement error (MAPE=[(monitor measurement – actual measurement) / actual measurement]×100) and was used as an outcome measure. A smaller MAPE represents better accuracy, and less than 3% is considered acceptable for distance³⁰ and 10% for TEE.²⁸ This method is a more conservative estimate of error that reflects the true error in estimation and provides the most appropriate indicator of overall error. Further analyses included Bland-Altman plots, which were calculated to examine the level of agreement between each monitor and the criterion methods across all dependent variables.³¹ Limits of agreement were calculated as ±2 SD from the overall mean bias between the dependent variables and each GPS monitor.

Results

Distance

Table 1 provides descriptive statistics (means, SD) for all of the different monitors compared with the measured values for distance in the two conditions, walking and running. Separate 3 (2 GPS monitors+criterion)×2 (walking or running) repeated measures ANOVA were conducted to examine differences in distance obtained from Garmin and Runkeeper. There was no significant interaction effect between the 3 measurement methods and the 2 types of activities [F(1,21)=1.58, p=0.23, η²=0.08] and no significant main effect between walking and running [F(1,19)=2.94, p=0.10, η²=0.13]. However there was a significant main effect between the 3 measurement methods [F(1,21)=64.44, p<0.001, η²=0.77]. Post-hoc within-subjects contrasts with Bonferroni correction revealed that Runkeeper significantly overestimated distance [F(1,19)=73.75, p<0.001, η²=0.80], while Garmin provided almost the same results as the measured distance [F(1,19)=0.57, p=0.46, η²=0.03].

Table 1: Descriptive statistics for Distance (km)

The MAPE (computed as the average absolute value of the errors relative to the measured distance) observed for Garmin during the running condition was 0.30% and for the walking condition was 0.74%. A larger MAPE was observed for Runkeeper during running (3.28%) and walking (4.43%) (Figure 2).

Figure 2: Mean absolute percent error (±SD) for distance estimation.

Bland-Altman plot analysis showed the distribution of error and assisted with testing for proportional systematic bias in the estimates. The plots show the residuals of the various estimates on the y-axis (measured distance – monitor) relative to the mean of two methods (x-axis). The plots (see Figure 3) revealed the narrowest 95% limits of agreement for Garmin during running (mean bias=0.00; 95% CI=−0.01 to 0.01 km) and Garmin during walking (mean bias=-0.01 km; 95% CI=−0.02 to 0.01 km), while higher values for Runkeeper during running (mean bias=−0.12 km; 95% CI=−0.17 to −0.07 km) and walking (mean bias=−0.16 km; 95% CI=−0.20 to -0.12 km) were observed. The slopes for the fitted lines were significant for all measurements; Runkeeper walking (slope=−1.93, p<0.001), Runkeeper running (slope=−1.96, p<0.001), Garmin walking (slope=−1.78, p<0.001) and Garmin running (slope=−1.05, p<0.001), which suggests significant patterns of proportional systematic bias for these monitors.

Figure 3: Bland-Altman plots for distance estimation.

TEE

Table 2 provides descriptive statistics (means, SD) for all of the different monitors compared with the measured values from the Cosmed in the two conditions, walking and running. Separate 3 (two GPS monitors+criterion)×2 (walking or running) repeated measures ANOVA were conducted to examine differences in TEE obtained from Garmin and Runkeeper application. There was a significant interaction effect between the 3 measurement methods and the 2 types of activities [F(2,38)=16.10, p<0.001, η²=0.46], a significant main effect between walking and running [F(1,19)=267.47, p<0.001, η²=0.93] and a significant main effect between the 3 measurement methods [F(2,38)=128.74, p<0.001, η²=0.87]. Post-hoc within-subjects contrasts with Bonferroni correction revealed that Runkeeper significantly overestimated TEE for both conditions [F(1,19)=44.72, p<0.001, η²=0.70], while Garmin significantly underestimated TEE for both conditions [F(1,19)=15.14, p=0.001, η²=0.44].

Table 2: Descriptive statistics for estimated TEE (kcal), with added measured REE for Garmin

The MAPE (computed as the average absolute value of the errors relative to the measured distance) observed for Garmin during the running condition was 17.39% and for the walking condition was 17.32%. A smaller MAPE was found for Runkeeper during running (6.29%) and larger for walking (35.52%) (Figure 4).

Figure 4: Mean absolute percent error (±SD) for TEE estimation.

Bland-Altman plot analysis showed the distribution of error and assisted with testing for proportional systematic bias in the estimates. The plots (see Figure 5) revealed the narrowest 95% limits of agreement for Garmin during walking (mean bias=13.95 kcal; 95% CI=−4.45 to 32.35 kcal) and for Runkeeper during running (mean bias=−19.65 kcal; 95% CI=−31.10 to −8.20 kcal), while higher values for Garmin during running (mean bias=54.75 kcal; 95% CI=43.78 to 65.72 kcal) and Runkeeper during walking (mean bias=−70.25 kcal; 95% CI=−81.77 to −58.73 kcal) were observed. The slopes for all fitted lines were significant; Runkeeper walking (slope=−1.73, p<0.001), Runkeeper running (slope=−1.75, p<0.001), Garmin walking (slope=−1.94, p<0.001) and Garmin running (slope=−2.03, p<0.001), which suggests significant patterns of proportional systematic bias with these monitors.

Figure 5: Bland-Altman plots for TEE estimation (adding REE for Garmin).

Discussion

The present study aimed to examine the criterion validity of Garmin Forerunner 310XT GPS monitor and freeware Android application Runkeeper for distance and TEE estimation, during walking and running. To our knowledge, this is the first study that has tried to compare the accuracy of a commercially available GPS and a freeware smartphone application under semi-structured settings. Even though Garmin Forerunner 310XT and Runkeeper have been validated for distance estimation, the algorithms that they use for EE prediction have never been validated previously. Furthermore, studies, which compare wearable activity monitors and smartphone PA applications are scarce in the international literature and only recently some similar attempts have been conducted.^32,33

A unique feature of this study was the naturalistic design that sought to replicate free-living overground movement. In contrast with traditional validation studies,³⁴ the participants were given the option to select the intensity of walking and running they preferred during exercise. The results of the present study support the accuracy of these methods for distance recording, however TEE estimates had large errors.

A further advancement, as indicated by Bai et al.,³⁵ was that both individual-level and group-level accuracy in distance and TEE estimation were evaluated. MAPE values and 95% limits of agreement provided a useful indicator of individual-level validity and reflected the error that individuals could expect if they were tracking their personal activity estimates. Overall distance error estimates were similar to results from previous evaluations of wearable monitors and smartphone applications,^12,14 with overall MAPE values typically ranging from 0.30% to 4.43% for both Garmin and Runkeeper. Garmin had the smallest MAPE in both conditions and was the most precise estimate based on the 95% limits of agreement. Runkeeper had higher errors, overestimating distance; however, these errors of 3% to 4% are acceptable. Taking into account the small difference between the lower and upper limit of agreement of 0.10 km, this application may also be considered valid for distance counting.

On the other hand, TEE error estimates were larger, ranging from 6.29% (Runkeeper during running) to 35.52% (Runkeeper during walking). Garmin underestimated TEE, whereas Runkeeper overestimated it. The slopes for the fitted lines in the Bland-Altman plots were all significant due to at least two outliers in each graph, which might have significantly biased the outcomes of these analyses. Since no previous study has validated the EE of GPS devices and applications, no direct comparable results could be made. Most previous studies, which examined the validity of accelerometer-based wearable activity trackers (FitbBit, Jawbone, Nike+, etc.) came to similar conclusions.^28,35,36 Runkeeper during running gave a MAPE less than 10%, making it accurate in TEE under this condition, while MAPE during walking was significantly higher. Garmin was more stable during the two conditions, with an overall medium to large MAPE of 17%.

Mean bias and repeated measure ANOVA provided alternative indicators of group-level accuracy. The mean bias and repeated measure ANOVA results favored only Garmin for distance estimation, which provided group-level validity. Both the monitor and the application had low group-level accuracy for TEE estimation, differing significantly from the criterion measurement.

A novel finding is that Garmin’s TEE estimation was less accurate than expected. Garmin, in order to estimate TEE, uses the Firstbeat algorithm,¹⁹ which combines the data obtained from the heart rate strap. Previous research showed that this might underestimate EE by 6% to 13%.²⁰ In this study, the error was 17%. The accuracy of EE predictions was not improved with the addition of heart rate measures to traditional GPS device, in accordance with previous research findings in accelerometer-based monitors.³⁷ A possible explanation for the limited accuracy of EE prediction could be that the algorithm used could not accurately compute EE based on distance and heart rate, even though these initial raw data were very accurate. Technical assistance was sought from the company to ascertain specific information regarding the algorithm used to determine EE, however this information was not disclosed.

The present study had some limitations. Only one healthy participant performed all activities and thus we did not account for different potential confounding factors such as BMI, monitor placement, gender and age, which could potentially influence accuracy. In addition, we examined the accuracy during walking and running in outdoor terrain, so the results cannot be generalized in other settings. Lastly, these estimates were obtained for a specific distance and may not reflect accuracy for longer distances, in example for a marathon race.

Conclusion

In conclusion, the present study offers initial evidence for the validity of GPS technology of PA monitors and smartphone applications for measuring distance travelled during walking and running. However, estimates of EE were low, except for Runkeeper during running which provided acceptable error. This limits their use for monitoring energy balance, and therefore as a weight management tool. These results can assist consumers, researchers and health care providers to make evidence-based choice for a GPS PA monitor to measure distance during exercise. People who own Android smartphones have a valid alternative for distance estimation during walking and running with this freeware GPS application. Caution should be made when mixing the results from different activities into one estimated measure, due to probable cancelling of overestimation and underestimation from these activities, which may lead to an illusion of improved accuracy.

No competing interests

All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

References

1. World Health Organization. Prevalence of insufficient physical activity: Situation and trends. 2014. Availiable at http://www.who.int/gho/ncd/risk_factors/physical_activity_text/en/ (accessed 25 June 2016).

2. Reither EN, Hauser RM, Yang Y (2009). Do birth cohorts matter? Age-period-cohort analyses of the obesity epidemic in the United States. Soc Sci Med 2009;69(10):1439–48.

3. Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL, et al. European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice. Eur Heart J 2016;37:2315–81.

4. Bardus M, Smith JR, Samaha L, Abraham, C. Mobile phone and web 2.0 technologies for weight management: A systematic scoping review. J Med Internet Res 2015;17(11):e259.

5. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res 2006;8(2):e10.

6. King AC, Hekler EB, Grieco LA, Winter SJ, Sheats JL Buman MP, et al. Effects of three motivationally targeted mobile device applications on initial physical activity and sedentary behavior change in midlife and older adults: A randomized trial. PLoS ONE 2016;11(6):e0156370.

7. Laurson KR, Welk GJ, Eisenmann JC. Estimating physical activity in children: Impact of pedometer wear time and metric. J Phys Act Health 2015;12:124–31. http://dx.doi.org/10.1123/jpah.2013-0111.

8. Duncan MJ, Badland HM, Mummery WK. Applying GPS to enhance understanding of transport-related physical activity. J Sci Med Sport 2009;12:549–56.

9. Specht M, Szot T. Accuracy analysis of GPS sport receivers in dynamic measurements. An Nav 2012;19:165–76.

10. Wieters KM, Kim JH, Lee C. Assessment of wearable global positioning system units for physical activity research. J Phys Act Health 2012;9:913–23.

11. Lee K, Kim JY, Putti K, Bennett DH, Cassady D, Hertz-Picciotto I. Use of portable Global Positioning System (GPS) devices in exposure analysis for time-location measurement. Kor J Envir Health Sc 2009;35(6):461–67.

12. Bauer C. On the (in-)accuracy of GPS measures of smartphones: A study of running tracking applications. In: Proceedings of MoMM ’13 International Conference on Advances in Mobile Computing & Multimedia. New York, NY, USA: ACM 2013:335–40.

13. Mallula CL. Comparing Garmin Forerunner 405CX GPS and Nike+ iPod to accurately measure energy expenditure, distance and speed of overground running. Master’s Thesis, Department of Health, Physical Education, Recreation and Dance, Cleveland State University. 2010. Availiable at http://rave.ohiolink.edu/etdc/view?acc_num=csu1273708884 (accessed 3 September 2016).

14. Adamakis M, Zounhia K. Comparing the validity and output of the Garmin Forerunner 310XT and Runkeeper android application in an urban environment. In: Dasheva D, Antala B, Djobova S, Kuleva M, (Eds.), Physical Education and sport – Competences for life: Book of abstracts of the 9^th FIEP European congress. Sofia, Bulgaria: FIEP 2014:59–60.

15. Bort-Roig J, Gilson ND, Puig-Ribera A, Contreras RS, Trost SG. Measuring and influencing physical activity with smartphone technology: A systematic review. Sports Med 2014;44(5):671–86.

16. Compher C, Frankenfield D, Keim N, Roth-Yousey L. Best practice methods to apply to measurement of resting metabolic rate in adults: a systematic review. J Am Diet Assoc 2006;106(6):881–903.

17. Duarte L, Ribeiro P, Guerreiro T, Carrico L. Defining a design space for persuasive cooperative interactions in mobile exertion applications. Collab Tech 2014;8658:105–12.

18. Middelweerd A, Mollee JS, van der Wal CN, Brug J, te Velde SJ. Apps to promote physical activity among adults: A review and content analysis. Int J Behav Nutr Phys Act 2014;11(97):1–9.

19. Firstbeat Technologies. An energy expenditure estimation method based on heart rate measurement. Firstbeat Technologies Ltd, 2012. Availiable at http://www.firstbeat.com/userData/firstbeat/Energy_Expenditure_Estimation.pdf (accessed 16 August 2016).

20. Montgomery PG, Green DJ, Etxebarria N, Pyne DB, Saunders PU, Minahan CL. Validation of heart rate monitor-based predictions of oxygen uptake and energy expenditure. J Strength Cond 2009;23(5):1489–95.

21. Smolander J, Ajoviita M, Juuti T, Nummela A, Rusko H. Estimating oxygen consumption from heart rate and heart rate variability without individual calibration. Clin Physiol Funct Imaging 2011;31(4):266–71.

22. Leica Geosystems (2015). Leica DISTO™ D810 touch: The smartest solution to measuring and documenting. Available at http://www.leica-geosystems.com/en/Leica-DISTO-D810-touch_104560.htm (accessed 15 August 2016).

23. Hills AP, Mokhtar N, Byrne NM. Assessment of physical activity and energy expenditure: an overview of objective measures. Front Nutr 2014;1(5):1–16.

24. McLaughlin JE, King GA, Howley ET, Bassett DRJr, Ainsworth BE. Validation of the COSMED K4b2 Portable Metabolic System. Int J Sports Med 2001;22:280–84.

25. Parr BB, Strath SJ, Bassett DR, Howley ET. Validation of the Cosmed K4b2 portable metabolic measurement system. Med Sci Sports Exerc 2001;333(Suppl. 5):S300.

26. Eisenmann JC, Brisko N, Shadrick D, Welsh S. Comparative analysis of the Cosmed Quark b2 and K4b2 gas analysis systems during submaximal exercise. J Sports Med Phys Fitness 2003;43(2):150–55.

27. Schrack JA, Simonsick EM. Ferrucci L. Comparison of the Cosmed K4b2 portable metabolic system in measuring steady-state walking energy expenditure. PLoS ONE 2010;5(2):e9292.

28. Lee JM, Kim Y, Welk GJ. Validity of consumer-based physical activity monitors. Med Sci Sports Exerc 2014;46(9):1840–48.

29. Bruton A, Conway JH, Holgate ST. Reliability: What is it, and how is it measured? Physiotherapy 2000;86(2):94–9.

30. Crouter SE, Schneider PL, Karabulut M, Bassett DRJr. Validity of 10 electronic pedometers for measuring steps, distance, and energy cost. Med Sci Sports Exerc 2003;35(8):1455–60.

31. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;8:307–10.

32. Adamakis M. Preliminary study of consumer-level activity monitors and mobile applications for step counting under free living conditions. J Mob Technol Med 2016;5(3):1–8.

33. Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA 2015;313(6):625–26.

34. Boyce G, Padmasekara G, Blum M. Accuracy of mobile phone pedometer technology. J Mob Technol Med 2012;1(2): 16–22.

35. Bai Y, Welk GJ, Nam YH, Lee JA, Lee JM, Kim Y, et al. Comparison of consumer and research monitors under semistructured settings. Med Sci Sports Exerc 2015;48(1):151–158.

36. Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: A cross-sectional study. Int J Behav Nutr Phys Act 2015;12:42.

37. Wallen MP, Gomersall SR, Keating SE, Wisloff U, Coombes JS. Accuracy of heart rate watches: Implications for weight management. PLoS ONE 2016;11(5):e0154420.