Multimodal CV Project (KUMC/WSU Collaboration)

Background

Existing wearable devices provide partial coverage of important cardiovascular biometrics like ECG, but provide limited cardiovascular information, leaving significant gaps in achieving a comprehensive ability to detect and diagnose various cardiovascular diseases.

Objective

The objective of this project was to design a proof-of-concept multimodal cardiovascular monitoring system, combining multiple sensors: Electrocardiogram (ECG), Phonocardiogram (PCG), Seismocardiography (SCG), and Photoplethysmography (PPG). Synchronized capture of these modalities allows for improved reliability of cardiovascular health monitoring and disease diagnosis.

The goal is to take this prototype and convert it into a modular wearable product that may be used in hospitals, at home, or for advanced uses, such as monitoring pilots and astronauts.

---

System Overview

Role and Responsibilities

The project was conceived by Dr. Amit Noheria and Dr. Yongkuk Lee.

My responsibilities were as follows:

• Selection of circuit components
• Circuit diagrams and PCB layouts (KiCAD)
• Writing firmware (C99) for sensor drivers and system for synchronized collection and transfer of data over USB
• Solving system integration and synchronization problems (See Technical Notes)
• Writing software for Windows (C++11 & Jai) which collects the data from multiple USB devices and synchronizes them in real-time (See Technical Notes)

As you can see, I pretty much did all the technical work solo. This entire project spanned 9 months and required a total of ~400 hours of work.

Technical Notes

• The primary challenge was figuring out how to keep all the data from multiple USB devices in sync
• How I resolved this issue is a secret, so you'll have to ask me about it.
• Just kidding: what I did was connect a GPIO pin together on all the devices, where one of the devices would be the "Master" device and toggle the pin on a timer. This would be detected by the other devices and all devices would replace the samples in their datastreams during the time the pin was active. It's kind of an ugly solution, so next time I will try to figure out a way to achieve the same effect using an EM impulse instead, so I don't have this awkward wire connecting four independent devices.
• As a result, this gave us a synchronization error of 1 to 8 ms depending on the sampling rate of each sensor, at the cost of only losing a few seconds of data every few minutes.

Data Sample: Synchronized Patient Data

Data Sample: 12-Lead ECG

Contacts / References:

• Dr. Amit Noheria (Kansas University Medical Center)
  email: anoheria[ at ]kumc[ dot ]edu
• Dr. Yongkuk Lee (Wichita State Univerity)
  email: yongkuk.lee[ at ]wichita[ dot ]edu

Addendum: Software Video Demo

This demo shows multiple devices streaming in a synchronized fashion, with the synchronization blip coming in at ~25 seconds.
This method can be used with any number of independent devices, in our case we demonstrated this with 4 USB devices.

This program is my personal development platform that I use for prototyping hardware connectivity with Bluetooth, Wifi and USB. It is proprietary and the code available upon request.