Saturday, May 16, 2015

Activity Tracker Analytics - Part 1: Fun with Fit-Bits

I've been experimenting with fitness and activity tracking devices the last few months. Somehow, I ended up testing three of them (let's call them blue, red, and green), each from a different company, one of them being a 'fit bit'. Blue and red showed similar daily readings, which also seemed to tally with small manual walking samples I took. On the other hand, the cheap, low-end green device was way off and severely under-predicting. Rather than simply junk the green tracker, can we 'rescue' it by noting its measurements and then optimally re-calibrate it using a more accurate tracker? This is the simple idea of this blog.

Figure 1 shows a one-month sample of normalized daily step-count readings obtained from the three trackers.
Figure 1.


The blue and red represent the 'good trackers', and the green one, which we wanted to junk, is the green line. This plot shows that although trackers from different manufacturers can differ significantly in terms of the absolute number of steps they count, their relative readings between days is likely to be more consistent. If you walk more, it is quite likely that the tracker will count more steps. Comforting.

This February 2015 JAMA article discusses the quality of readings from a variety of activity trackers.  Here is a snapshot of a paragraph from the preview page of this journal article.


(Picture source: preview page from http://jama.jamanetwork.com)

It appears that this paper also mentions good 'intra-device' repeatability. This consistency of response (something that is incredibly important to ORMS decision support systems) offers hope for our green tracker.  However, there is also plenty of scope for noise. The counting is done based on an accelerometer/sensor, and therefore, acceleration is what it detects and translates into step counts. If you are not a smooth driver during your morning and evening commute, you will have 'walked' a lot. I've taken a lot of steps driving my lawn mower tractor for 45 minutes. Different trackers respond differently to specific types of activity. Exceptions aside, these devices are generally quite useful. Moreover, they do really tell us if our exercise levels have increased or dropped over time, which is a very useful and healthy thing to know. This second figure below shows the ratio of step counts between two successive days.

Figure 2

The green-tracker looks pretty reasonable compared to the other two when it comes to reporting the relative response over time. During peak days, it closely matches the blue tracker, while on other days, it sticks reasonably close to the blue and red lines. Therefore, the green tracker is not a total write-off. We can rescue it by re-calibrating it upward. For example, let us assume the average of blue and red as the 'true' reading and employ linear regression (minimizing sum of squared 'error') to identify the green correction, as shown in Figure 3 below.

Figure 3.

For this specific usage profile, scaling the green reading up by 12% and adding 0.32 gets us close to the red and green numbers. However, the presence of peak-days may have skewed our calibration, so this may not be an 'optimal' idea. Let us remove the peak-days, and also swap the X and Y axes to get a 'normal day' recalibration. The result is shown in Figure 4 below.

Figure 4.

Aha! This regression equation is interesting. The R-squared value drops a bit, but more importantly, it suggests that we may do not need to do scaling. We can do something as simple as adding 0.36 to the green reading. Of course, the relative response plot in Figure 2 should have given us that clue. Did the green-tracker company engineers miss a trick here: Do these missing '0.36 steps' represent a default 'idle' activity level that they forgot to account for? If the green engineers spread the 0.36 over 16 waking hours, their final readout would be in good shape. Let us compare the output of this (0.36 + green) with the blue-red plot in our last and final figure.

Figure 5.

By simply adding 0.36 to the count of the cheap tracker, we have obtained a good daily step count that compares well with the more expensive 'blue-red' trackers. Go Green!