International Test & Evaluation Association (ITEA)

A heartfelt thank you to our incredible sponsors! Your support empowers our Warfighters through your exceptional products and services, and strengthens ITEA through your generous contributions. We’re truly grateful for your commitment. Raven Defense Corporation, QinetiQ, Dell Technologies, NI (National Instruments), Georgia Tech Research Institute, Acquired Data Solutions, Inc.

It's time to think about nominating a T&E professional or T&E team for a job well done. https://itea.org/awards

Exhibition Space and Sponsorships still available. https://lnkd.in/dRW7g-8n

Conversations with Experts Testing Without Being a Tester: A Conversation with Dr. Bill D’Amico Dr. D’Amico is a retired U.S. Army civilian and a retired Johns Hopkins University Applied Physics Laboratory researcher. He is currently consulting on autonomous systems testing. Read the full article: https://lnkd.in/djhwjzUt Bill D'Amico

Technical Article #5 An AI Model Performance Benchmarking Harness for Reproducible Performance Evaluation Abstract AI models are complex and are often designed to solve domain-specific tasks on resource-constrained platforms. The resource constraints on edge devices, such as available memory, disk space, and processing power, require optimization before deployment. Optimizations, such as quantization and pruning, can effectively reduce model size or latency, but often at the cost of accuracy. A well-designed, adaptive, and scalable AI benchmark harness is needed to test the models before and after optimizations are applied to establish if the models maintain acceptable performance. In this paper, we design and develop a comprehensive and generalized benchmark harness and test its functionality against optimized artificial intelligence (AI) models, measuring several performance metrics. Read the full article:https://lnkd.in/dkYQWF54 Jakob Adams, Dr. Venkat R. Dasari, and Dr. Manuel M. Vindiola

Technical Article #4 Real-Time Inference for Unmanned Ground Vehicles Using Lossy Compression and Deep Learning Abstract Autonomous vehicles rely on on-board perception systems for safe terrain navigation which becomes exceedingly important in rural areas. The aim of this study is to explore the effect compressed training images have on the performance of deep learning segmentation architectures and determine if lossy compression is a practical solution for providing real-time transfer speed for autonomous vehicle perception systems. To test the performance of compression on deep learning we apply ZFP, JPEG, and SZ3 to EfficientViT and UNet and rank test accuracy. As a result, this study found JPEG to achieve the highest compression ratio of 144.49× at JPEG quality level 0; while also achieving the fastest transfer speed of the compressors used on the Nvidia Xavier Edge Device. Furthermore, JPEG achieved the highest mIoU accuracy for both architectures tested in comparison to SZ3 and ZFP. Of the two deep learning architectures tested, EfficientViT outperforms U-Net for all lossy compressors at all levels of compression. EfficientViT achieves a peak mIoU of 95.5% at a JPEG quality level of 70. While U-Net peaks with an mIoU of 90.683% at a JPEG quality of 40. This study advances autonomous vehicle development in two ways. First, it demonstrates that JPEG compression outperforms specialized scientific compressors (SZ3/ZFP) for off-road RGB perception systems. Second, it validates EfficeintViT’s effectiveness for resource-constrained autonomous navigation. These findings benefit autonomous vehicle engineers implementing perception systems, computer vision researchers working on embedded applications, and industry teams deploying off-road autonomous navigation solutions.  Read the full article: https://lnkd.in/dW73J9_9 Ethan Marquez, Adam Niemczura, Cooper Taylor, Max Faykus, III

Technical Article #3 SCOUT – Pushing High Performance Computing to the Data SCOUT (SuperComputing OUTpost) is a first-of-a-kind mobile, deployable, turnkey high performance computing (HPC) system. It contains the same processors as one of the world’s fastest computers, SUMMIT, and is available to all DoD services and agencies. It is housed in a 53’ trailer with compute, memory, storage, network, cooling, backup power, power conditioning, and fire suppression. All it needs for operation is external power, networking, and a ½” water line` for humidification. Computing requirements in remote locations and at the tactical edge continue to grow, especially for artificial intelligence, machine learning, and autonomous systems. The DoD High Performance Computing Modernization Program provides this supercomputing capability in support of the DoD science & technology, test & evaluation, and acquisition engineering communities. SCOUT details are presented and three use cases are described. Lessons learned are enumerated in consideration for future deployable systems.  Read the full article: https://lnkd.in/dypKu5hR Michael Barton, Thomas Kendall, Jamie Knapil Stack

Technical Article #2 Defect Characterisation in ICT Scanned Energetic Materials using Machine Learning Abstract Solid-propellant rocket motors (SRMs) are commonly used in spaceflight and in the military for missile defence systems. Reliability and durability are critical traits for an SRM, however, both the ageing process and environmental exposure can lead to the formation of defects. Following non-destructive testing on the SRM, Industrial Computed Tomography (ICT) imagery is used to support manual defect analysis. This paper proposes a two-step machine learning solution for the automatic detection and characterisation of selected defects within an SRM using ICT imagery. From the output of the machine learning algorithm, an interactive three-dimensional visualisation of defects within an SRM is generated. Initial results reveal that a fine-tuned Faster R-CNN model was able to achieve an accuracy of 84% in identifying and detecting defects. Following this, binary thresholding was able to calculate exact defect area with 87% accuracy. Results indicate that a machine learning process can provide significant speed-up for SRM defect analysis, with processing completed within 30 minutes for an example SRM ICT scan volume. A digital map of the defects within an SRM will provide a framework to support simulation and predictive modelling regarding defect propagation throughout the lifetime of an SRM.  Read the full article: https://lnkd.in/d8Dpcha7 Christophe Harvey

Technical Article #1 INNOVATION Independent Automated Verification and Validation Testbed for Test and Evaluation Abstract The Johns Hopkins University Applied Physics Laboratory (JHU/APL) in collaboration with the Department of Homeland Security (DHS) Continuous Diagnostics and Mitigation (CDM) program office has developed an automated testing infrastructure and testbed to verify the CDM solutions deployed at United States federal departments and agencies. This testing capability is secure, hosted with trusted infrastructure using Amazon Web Services (AWS), and fully customizable to adapt to a variety of use cases. It utilizes open-source tools like Selenium and Jenkins to automate CDM testing, while synthetic data is generated to independently validate test requirements. It effectively replaces manual testing events, resulting in an increase in efficiency of testing and risk reduction throughout the CDM program. This allows for testing to keep pace with Agile development. This experience paper details the process of creating this capability and adapting it to the unique use case of complex government system test and evaluation. Read the full article: https://lnkd.in/dCPT6U65 Emily Pozniak, David Warren, and Christopher Rouff

Thank you to Our Sponsors! QinetiQ, Raven Defense Corporation, NI (National Instruments) and Georgia Tech Research Institute.