Scaling wearable foundation models
We strive to create an environment conducive to many different types of research across many different time scales and levels of risk.
Inspired by empirical successes in generative modeling, where large neural networks learn powerful representations from unstructured and noisy data, we demonstrate the scaling properties of this model for consumer health data, highlighting how these models enable sample-efficient downstream learning for tasks like exercise and activity recognition.
Wearable devices that measure physiological and behavioral signals have become commonplace. There is growing evidence that these devices can have a meaningful impact promoting healthy behaviors, detecting diseases, and improving the design and implementation of treatments. These devices generate vast amounts of continuous, longitudinal, and multimodal data. However, raw data from signals like electrodermal activity or accelerometer values are difficult for consumers and experts to interpret. To address this challenge, algorithms have been developed to convert sensor outputs into more meaningful representations.
Historically, algorithms for wearable sensors have relied on supervised, discriminative models (i.e., a class of models often used for classification) designed to detect specific events or activities (e.g., recognizing whether a user is running). This approach, however, faces several significant limitations. First, the limited volume and severe class imbalance of the labeled events means that there are large amounts of potentially valuable unlabeled data left unused. Second, supervised models are trained to do only one task (e.g., classification) and thus create representations that may not generalize to other tasks. Third, there can be limited heterogeneity in the training data since it is frequently collected from small study populations (usually tens or hundreds of participants).
.webp)