We began by analysing the performance of a market leading wrist wearable device, against our own HeartKey product, with the goal of comparing signal conditioning and peak detect algorithm performance of the two systems via ECG-derived heart rate analysis.
We passed medical grade ECGs of known heart rate into both devices to derive mean heart rate values.
ECGs of various amplitudes were used to replicate real life test cases and incremental, simulated noise was applied to the ECG signals prior to the ECG being passed into both systems.
A result was obtained for both HeartKey and the competitor.
Heart rate algorithm performance was analysed at these varying noise levels and compared to the known true heart rate value.
We were delighted to see that HeartKey strongly outperformed the competitor device.
The competitor device had an average mean absolute percentage error (MAPE) of 5.9% whereas HeartKey had an average MAPE of 0.8%.
HeartKey also had a significantly smaller HR range than the competitor.
“The accuracy of wrist health wearables is massively important due to their global popularity. They are often relied on by the user as a health tracker in regards to weight loss, as a performance measurement tool for athletes training for an event or for general cardiac health and wellbeing monitoring.
“The results of this study demonstrate the large variability of results found in a market leading device.
“These inaccuracies must be considered when trusting such devices for reliable medical standard information.”
Medical Device Benchmarking
We next performed a review of the performance of the HeartKey signal conditioning and peak detect algorithms when compared to an FDA 510(k) approved medical device and a market leading ECG chest strap.
The investigation assessed R-R interval data for low amplitude ECGs (<0.5mV) as this enabled heart rate variability (HRV) metrics to be determined for comparative analysis.
The protocol involved was as follows:
- 1 minute standing baseline
- 1 minute light walk
- 2 minute quick walk
- 1 minute stair exercise
- 1 minute standing recovery
HeartKey outperformed the commercial ECG chest monitor and matched the ECG medical device in each HRV related metric - particularly in SDNN (standard deviation of R-R intervals), mean R-R and RMSSD (root mean square of successive differences of the R-R intervals).
This indicates the leading commercial ECG monitoring devices are not up to a medical grade standard. While relatively accurate for general use and training purposes, they are not yet suitable for medical monitoring use cases.