New software aims to enable real-time decision-making for structural health.

Machine learning has significantly increased in the last 15 years, particularly in object detection and self-driving cars. But machine learning can also support and analyze the sensing awareness of time, space and frequency for structures operating in high-rate dynamic environments. This includes civilian applications such as adaptive automobile crash systems and military usages like supersonic and hypersonic vehicles that need to make decisions on a millisecond timescale.

Mechanical Engineering Assistant Professor Austin Downey and Computer Science and Engineering Professor Jason Bakos are researching how machine learning systems can grow the fields of high-rate structural health monitoring, damage detection and reactions. Their goal is to enable real-time decision-making and control schemes that minimize risk.

“While some research has been done with algorithms and hardware, no previous research has looked at high-rate structures with timescales to make fast and correct decisions,” Downey says. “Our research is like an emergency room operation where you have to make fast decisions with life-threatening injuries.”

Downey’s research aims to achieve sensing and entity awareness as well as system development by working with Bakos to create a software package for defense and commercial use. The research is funded by the Air Force Office of Scientific Research (AFOSR) to work in collaboration with the Air Force Research Lab. Downey previously received grants from the AFSOR for his research on real-time edge computing and real-time model updating for structures experiencing shock.

High-rate dynamics are responses from a high-amplitude event, such as a blast or impact. Systems affected by high-rate dynamics may experience rapid changes.

“A system subject to this environment often experiences a sudden plastic deformation and damage may extend to the structure, electronics and sensors. Rapid state estimation for these systems is used within a feedback loop to prevent further damage and a complete failure,” Downey says. “For example, rapid detection is required in the deployment of a blast mitigation system or in adapting control decisions for a hypersonic vehicle following a ballistic impact.”

Structural health monitoring includes static and highly dynamic engineering systems. With the advent of real-time sensing, edge-computing, and high-bandwidth computer memory, high-rate structural health monitoring timescales can be enabled. Downey’s goal for the high-rate timescale is one millisecond for acquiring data, assessment and decision-making.

“This will be used to address constraints to allow for high-rate implementation that is robust, adaptable and beneficial. Technologies such as machine learning can be further developed to enable structural and functional prognostics for high-rate dynamic systems,” Downey says. “Time scales this short have never been investigated [for decision-making], but the Air Force has many applications that require fast decisions."

Downey is working with Eglin Air Force Base in Florida on the application of hard target penetrating weapons. The Air Force is interested in creating smart fuzes that are safer and more robust than those currently in use.

“This work can also be extended to a variety of high-speed applications. For example, if a bullet hits the back of a helicopter, there are only milliseconds to shut down battery power before it bursts into flames,” Downey says. 

Downey and his team are developing the programming architecture for real-time machine learning control. They will create a software infrastructure for engineers and computer scientists developing high-rate control systems. Modules designed for this special class of high-rate challenges could be simply dragged and dropped. Downey’s long-term goal is for his program to be used by commercial companies that are developing controllers for applications that experience high-rate dynamics.

“We don’t make systems. Our vision is to make software tools and validate them,” Downey says. “If a software designer wants their system to be autonomous and have high-rate decision-making capabilities, our goal is to create a software package. The software would be deployed to certain parts of custom computer architectures that would be co-designed for microsecond responses.”

While Downey will utilize his expertise in modeling, sensors and structures, Bakos will deploy the developed custom computing hardware and specialized real-time systems. Bakos will use Field Programmable Gate Arrays (FPGAs) to enable microsecond machine learning applications. FPGAs are logic chips with an integrated circuit that can be programmed to a required functionality or application. While FPGAs can be slower than customized chips, they can be reprogrammed in only seconds, rather than in months or years.

“The advantage with FPGAs is that you're basically implementing an algorithm or calculation in the hardware, and it offers very low latency for something like incoming sensor readings,” Bakos says. “You can connect these sensors directly to the pins on the chip and connect the outputs directly to the FPGA pins. The logic on the FPGA is a straight hardware implementation of what needs to happen in between the inputs and the outputs, so it's much faster than a traditional CPU that must contend with system wires.”

Downey has set up tests at the College of Engineering and Computing that were previously performed by the Air Force. This includes the testing of electronics under impact that can experience 5,000 times the force of gravity.

“We had to create our own data that's controlled and simplified to validate these algorithms. You want to know the condition of the system in real-time for making appropriate decisions,” Downey says. “Everything is constrained by time. While the tools used are not drastically new, no one has looked at this on the microsecond timescale. We have very stringent numbers and metrics on what we consider to be real-time.” 

Downey’s current research and other small projects are helping fund his overall vision of real time control for active structures. With Bakos' help, he hopes that their software and research can help machines make important decisions in mere microseconds.