Non-speech body sounds collected from the human body have many medical uses: several respiratory and cardiovascular diseases are diagnosed using audio data. In recent years, wearable devices for fitness and health tracking have become popular, but no commercially available device tracks audio data collected from the body, as there remain many technical challenges. This dissertation explores the viability of building a chest-mounted wearable device that can be used for continuous health monitoring. A custom device was designed and manufactured, and was subsequently used to collect a data set from 9 individuals. The user study focuses on exploring the noise tolerance of a wearable device; unlike related work, this study explicitly considered robustness to ambient noise and user motion. Two algorithms were proposed for continuous heart monitoring: a autocorrelation-based technique that yielded an estimate of the heart rate, and a more complicated technique which segments the collected audio into the different phases of the cardiac cycle. Both techniques yielded accurate heart rate estimates when the user was resting: 1.86% and 0.26 ± 0.02% median percentage errors were found for the two algorithms respectively, even under challenging ambient noise conditions. The segmentation algorithm yielded good estimates even when the user was walking, and could also be used to obtain an accurate measure of heart rate variability. Both algorithms were also evaluated by considering the viability of running them on-device; it was shown that the cheaper algorithm could run continuously for over a week on a typical battery found in a wearable. Finally, the viability of using a device for continuous asthma symptom detection was considered. An approach employing convolutional neural networks running on-device was assessed. It was found that the model could obtain near-human level performance at detecting wheezing from audio continuously, with a battery life of over 3 days.