This revised two-day course introduces the student to sensor and data fusion methods that improve the probability of correct target detection, classification, identification, and state estimation. These techniques combine information from collocated or dispersed sensors that utilize either similar or different technologies to generate target signatures or imagery. The course begins by describing the effects of the atmosphere and countermeasures on millimeter-wave and infrared sensors to illustrate how the use of different phenomenology-based sensors enhances the effectiveness of a data fusion system.
This class introduces the Data Fusion Information Group (DFIG) enhancements to the JDL data fusion processing model, several methods for describing sensor and data fusion architectures, and the taxonomies for the data fusion algorithms used for detection, classification, identification, and state estimation and tracking. This is followed by descriptions of the higher-level data fusion processes of situation and threat assessment that are considered part of situation awareness. Process refinement, now deemed part of resource management, and user refinement dealing with human-computer interactions and human decision making are treated next. Subsequent sections of this course more fully develop the Bayesian and Dempster–Shafer algorithms, radar tracking system design concerns, multiple sensor registration issues, track initiation in clutter, Kalman filtering and the alpha-beta filter, interacting multiple models, data fusion maturity, and several of the topics that drive the need for continued data fusion research.
Examples demonstrate the advantages of multisensor data fusion in systems that use microwave and millimeter-wave detection and tracking radars, laser radars (imagery and range data), and forward-looking IR sensors (imagery data). You can also apply many of the data fusion techniques when combining information from almost any grouping of sensors as long as they are conditionally independent of each other and can supply the input data required by the fusion algorithm.
Course Benefits
Participants enhance their understanding of the:
- Advantages of multisensor data fusion for object discrimination and state estimation
- JDL-DFIG model for data fusion processing levels
- Taxonomies for target detection, classification, identification, and state estimation algorithms
- Appreciation of skills needed to develop and apply data fusion algorithms to complex situations
- Bayesian and Dempster–Shafer approaches to object and event identification
- Sequential probability ratio test to initiate target tracks
- Kalman filter operation for updating the state estimate of a target
- Need for sufficient process noise when a target maneuver is suspected
- Alternatives to the Kalman filter for nonlinear systems
- Procedure for implementing the Interacting Multiple Model process
The course benefits:
- Engineers, scientists, managers, designers, military operations personnel, and other users of multisensor data fusion for target detection, classification, identification, and tracking
- Those interested in selecting appropriate sensors for specific applications and applying data fusion techniques to advanced dynamic systems, such as classification of airborne targets, ground-based targets, and underwater targets
- Developers and users of real-time algorithms and multiple sensor technologies for non-cooperative target recognition and tracking
Recommended Prerequisite
There are no specific course prerequisites; however, a general background in electrical engineering, electro-optics, mathematics, or statistics is recommended for a better understanding of the concepts presented in the course.
Course Materials
Participants receive the text Sensor and Data Fusion: A Tool for Information Assessment and Decision Making, 2nd Edition, Lawrence A. Klein (SPIE, PM 222, 2012) and lecture notes on the first day of the course. These notes are for participants only and are not for sale or unauthorized distribution.
Coordinator and Lecturer
Lawrence A. Klein, PhD, is a consultant specializing in developing multiple sensor concepts for tactical and reconnaissance military applications, millimeter-wave and infrared sensors for homeland defense, and sensor and data fusion concepts for intelligent transportation systems. While at Hughes Aircraft Company, Dr. Klein developed missile deployment strategies and sensors used in missile guidance. As a systems manager at Aerojet ElectroSystems, he was responsible for the conceptual design and execution of programs that integrated active and passive millimeter-wave and infrared multispectral sensors in satellites and smart “fire-and-forget” weapons. He was the program manager of three Manufacturing Methods and Techniques projects that lowered the cost of millimeter-wave integrated circuits. At Honeywell, he developed passive millimeter-wave midcourse missile guidance systems and millimeter-wave sensors for mine applications.
In addition to the course text, Dr. Klein is the author of Millimeter-Wave and Infrared Multisensor Design and Signal Processing (Artech House, 1997), which describes multisensor applications and design; Sensor Technologies and Data Requirements for ITS (Artech House, 2001), which discusses sensor applications to traffic and transportation management systems; and ITS Sensors and Architectures for Traffic Management and Connected Vehicles (Taylor and Francis, 2018), which examines traffic management centers, data requirements, sensor technologies and sensor installation, concerns related to the introduction of automated and connected vehicles, and national ITS architectures. He collaborated with colleagues to prepare a review of data fusion methods that emphasize fusion of image features that aid object tracking using particle filtering. It appears as “Sensor and Data Fusion: Taxonomy, Challenges, and Applications” in the Handbook on Soft Computing for Video Surveillance (Taylor and Francis, 2011). His latest book, Sensor and Data Fusion for Intelligent Transportation Systems, PM305 (SPIE, 2019) emphasizes applications of data fusion to traffic management. Dr. Klein received his PhD in electrical engineering from New York University in 1973. He is a past reviewer for the IEEE Transactions on Antennas and Propagation, IEEE Transactions on Geoscience and Remote Sensing, and IEEE Transactions on Aerospace and Electronic Systems.
Defense Applications of Multisensor Systems and Data Fusion
- Need for smart sensors
- Defining detection, classification, and identification
Sensor Systems
- Multiple sensor phenomenology
- Military data fusion architectures
- Benefits of multiple sensor systems
- Atmospheric and obscurant effects on IR and MMW sensors
- Influence of sensor application on choice of MMW frequency and IR wavelength band
Sensor and Data Fusion—What is It?
- JDL and DFIG data fusion models
- Levels 0, 1, 2, 3, 4, 5, and 6 processing
- Duality of data fusion and resource management
- Data fusion architectures
Taxonomy of Detection, Classification, and Identification Data Fusion Algorithms
- Physical models
- Feature-based inference: parametric (classical inference, Bayesian, Dempster-Shafer, and others); information-theoretic models (templates, artificial neural networks, clusters, voting, figures of merit, pattern recognition, and others)
- Cognitive-based (knowledge-based expert systems, fuzzy set theory, and others)
Taxonomy of State Estimation and Tracking Data Fusion Algorithms
- Search direction
- Correlation and association of data and tracks:
- Data alignment
- Data and track association (prediction gates, correlation metrics, data, and track-to-track association)
- Position, kinematic, and attribute estimation
Fusion Levels 2, 3, 4, 5, and 6
- Situation Assessment (Level 2)
- Threat Assessment (Level 3)
- Refinement of the data fusion process (Level 4)
- User Refinement (Level 5)
- Mission Management (Level 6)
- Multiple-level data fusion architectures—examples
Bayesian Inference
- Conditional probabilities
- Bayes’ rule with illustrative examples
- Comparison of Bayesian and classical inference
- Bayesian inference fusion process with multiple sensor data
- Recursive updating of posterior probabilities
- Multispectral sensor example
- Mine detection example
- Freeway incident detection example
Dempster-Shafer Evidential Reasoning
- Dempster-Shafer fusion process
- Probability mass
- Uncertainty interval
- Dempster’s rule
- Comparison with Bayesian inference
- Methods to generate probability mass functions
- Probability mass function examples
– Using known characteristics of data gathered by the sensors
– Using confusion matrices - Modifications of D-S theory with examples
– Pignistic transferable belief, plausibility transform, plausible and paradoxical reasoning, and other modifications of the original theory
Radar Tracking System Design Considerations
- Radar characteristics
- Radar surveillance system functional block diagram
- Critical factors that determine performance
- Measurements versus tracks
- Measures of quality for tracking
- State space and coordinate conversion
Multiple Sensor Registration
- A requirement for multisensor tracking
- Functional requirements
- Sources of bias error
- Impact of bias errors on tracking
- Bias error budget
Track Initiation in Clutter
- What is the initiation problem?
- The Sequential Probability Ratio Test (SPRT)
– Definition of the decision criteria
– Example application of the SPRT
– Properties of the SPRT - Recommendations
Introduction to Kalman Filtering
- What does it do?
- Kalman filter equations
– State transition model
– Measurement model
– Discrete-time Kalman filter algorithm - Kalman gain
- Determining process noise
- The alpha-beta filter for tracking a constant velocity object
Interacting Multiple Models (IMM)
- When are these used?
- IMM process
- Components of a model
Future Directions for Data Fusion Research
- Data fusion maturity
- Continuing challenges in fusion system performance assessment
For more information contact the Short Course Program Office:
shortcourses@uclaextension.edu | (310) 825-3858 office
