Information Fusion in Medical Systems

Richard F. Zarr, MTS Technologist, National Semiconductor Corporation by Texas Instruments


Providing higher quality data to augment information

Click image to enlarge

Figure 1

As medical instrumentation acquires increased processing power, the amount of information that is gathered from a patient is growing exponentially. Whenever large amounts of data are presented to operators (in this case medical professionals), it is possible to overwhelm them with the information sometimes preventing them from identifying root causes or other underlying conditions. In this article we will examine the concept of data fusion and the additional requirements of medical equipment to allow real-time analysis from the many sources. Sensing and imaging technologies have revolutionised the medical industry allowing much faster diagnosis and treatment of disease. What is most interesting to note is that many different types of sensors and imaging systems are employed today depending on what a physician or team is looking for. In effect, they already need to have some idea of the problem. However, imaging and sensing technologies have different problems and by incorporating data fusion, the quality of the data can be greatly improved. Some experimental imaging systems are already combining Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI) scanners which show both function and anatomy. From a hardware point of view, this usually means collecting large amounts of data and by computational methods fuse this data in post processing thus a new problem of rapid, high speed data acquisition appears as well as Time of Acquisition (ToA) accuracy. The Benefits and Drawbacks of Data Fusion The concept of sensor fusion is to overcome individual sensor technology shortcomings by combining the best features of each. For example, it is very common today to find hybrid PET/CT scanners. These machines provide two modalities of imaging - Positron Emission Tomography and x-ray Computed Tomography - in a single gantry. This helps eliminate the error with doing the scan separately and attempting to combine the images. The benefit of both is that the resulting image provides both function and anatomy. Imaging is a clear area that can benefit from data fusion, however it can increase the patient cost since hybrid machines must incorporate both sensor technologies as well as additional computational power. However, the advantage of this configuration is that both sensors are co-located. The question then becomes - what happens when sensors are distributed between different systems? Time coordination is extremely important when gathering data since the algorithms used must know the exact time the data was sampled in order to properly combine sources. For instance, there may be benefit from combining an Electrocardiogram (EKG) with ultrasound imaging allowing the electrical information to augment the video of the moving heart. This allows for what is often termed "augmented reality" which is a fusion of multiple data sources to the actual event being observed - usually in real time. In the case of the EKG / ultrasound hybrid, separate devices must provide data to the computational engine doing the data fusion. If Pulse Oximetry or CO-oximetry is added, then multiple devices must all coordinate their data with accurate time stamps to ensure the delivery of the information for proper fusion (see figure 1). If the data is not accurately time-stamped, then fusion of the data becomes error prone and more difficult making real-time augmentation complicated.

Sensor Fusion Many times sensors have advantages and disadvantages. Different sensor types provide good quality information on different aspects of the physical property being sensed. That is, some provide good long term drift but poor dynamic response. Others provide good dynamic response, but poor drift or accuracy. If both sensor types are employed, then the outputs can be combined (or fused) to provide the best of both worlds. Both worlds in the example above would be good dynamic response and long term drift or accuracy. Usually this combining function is accomplished through a method called Kalman Filtering. This method is very useful in noisy environments, but can be used in sensor fusion as well. Kalman filtering uses calculations to predict estimates of what the output should be. The uncertainty of the prediction is estimated and a weighted average of this value and the actual measurement (or input) is calculated. The most influence is given to the average with the least uncertainty. This method provides a much closer value to the actual value since the weighted average has a lower uncertainty to either the estimated or measured value. For a medical example, certain medical sensors do an excellent job at specific tasks, but do not always provide enough information. In the case discussed earlier, pulse oximetry provides the percentage blood oxygen saturation information, however in the case of certain blood poisoning (e.g. CO2 or Cyanide) these devices do not provide enough information, so supplemental equipment is used to measure the exchange rate of oxygen with CO2. By using sensor fusion between these two devices, a more complete picture of the patient's blood oxygen levels is provided. Standard Time Coordination Protocols Medical data fusion is only one area that suffers from the problem of sample time accuracy. Many other distributed data acquisition and communications applications such as cellular radio require extremely controlled timing information. Much research has gone into providing highly accurate clocks locally, however in a distributed environment, it becomes much more difficult. There are standard time protocols provided within certain frameworks like Ethernet such as Network Time Protocol (NTP), but this lacks the precision required to build high performance distributed systems with any accuracy. Knowing the limitations of NTP and other time protocols, the IEEE proposed the IEEE1588 standard in 2002. This proposal later was adopted into a standard (IEEE1588-2008) which is now called Precision Time Protocol (PTP). This standard can easily provide sub-microsecond synchronisation between Ethernet nodes that support it. PTP works by exchanging time information in a deterministic way from a master clock which is chosen via the Best Master Clock (BMC) algorithm. Once the master clock in known within the domain, nodes calculate the offset from the master by a series of exchanges with the master clock (see figure 2). There is a great deal of hidden complexity in this protocol to assure accurate time keeping. The closer the logic of this protocol is to the physical layer, the more tightly controllable is the time. If the protocol is performed in software above the MAC layer, then the accuracy may only be within 100's of microseconds. This be may OK for some alternate reality fusion applications, but for higher resolution, it needs to closer to the hardware. The best solution is to place the 1588 protocol engine directly into the PHY which removes much of the non-deterministic jitter caused by random processing events in the layers above the physical and MAC layers. Some semiconductor vendors have built this logic directly into an Ethernet physical layer device such as the National Semiconductor DP83640 which when connected to other DP83640 devices in the system can provide sub 10 nanosecond accuracy in synchronisation. Conclusion Sensor fusion can overcome the short-comings of individual sensors providing higher quality data along with the ability to augment information being provided to physicians. When considering systems that provide augmented reality in distributed architectures, time synchronisation is extremely important. Without it, greater amounts of processing power are required to "re-align" the data to the point where real-time imaging or displays become difficult. In the future, scanners that show a patient's condition will use many different inputs to provide physicians the ability to "see" combined information in a way that will make diagnosis faster and more accurate.