State of play on the Big Things
An honest conversation on Replicator, CCA, and responsible AI
Welcome back to the Nexus Newsletter. In today’s edition, we share our thoughts on some of the highest-profile programs, initiatives, and policies related to defense autonomy: Replicator, Collaborative Combat Aircraft (CCA), and recent updates and comments on responsible AI.
Let’s talk Replicator
Deputy Secretary of Defense Hicks recently framed the Replicator initiative as follows:
Replicator is removing the kinks in the hose that is innovation in the DoD. … There are a multitude of programs that exist that need help in getting from where they are to delivery at scale,” she said. Furthermore, she outlined the criteria for programs and systems to be included in Replicator: programs that are “on the delivery pathway, but facing some challenges delivering at scale.
Replicator is an initiative designed to help existing autonomy programs summit roadblocks that are preventing them from delivering autonomy at scale in the short term. The specific methods by which the DoD intends to accelerate existing programs remains unclear, and will likely depend on which specific programs are incorporated into the initiative.
The goal is admirable. Replicator is an important focusing effort to align the might of the entire defense industrial base, DoD components, and other stakeholders behind a single goal: delivering autonomy at scale in the near term. The goals of the initiative are ambitious. Still, the timeline and goals are achievable, at least if the effort were focused exclusively on the commercial autonomy industry. Why the distinction? The commercial autonomy industry consistently employs enterprise-level development and testing infrastructure that enables an iterative, ongoing approach to developing and sustaining autonomy software.
By and large, the vast majority of the discussion around Replicator focuses on the ultimate effects that these large numbers of A2AD systems will have on the battlefield and on our adversaries. The discussion is centered around the vehicles themselves. Only a small part of the conversation is focused on what is actually the most difficult and important part of an autonomous system: the software that enables autonomous operations and capabilities. If Replicator’s goal is to deliver systems that are “small, smart, cheap, and many,” the “smart” part of that equation deserves at least equal consideration to the other hardware-focused elements of the end systems.
The software side is the most challenging part of developing, deploying, fielding, and sustaining autonomous systems at scale. There are only a few current defense autonomy programs that are set up in a way that will enable the rapid development, as well as ongoing testing, improvement, and sustainment, of autonomy software. The software pathway used for the Army’s RCV program, which separates the hardware from software to enable iterative, ongoing improvements to autonomy software over the entire lifecycle of the program, is one example. Unfortunately, however, the vast majority of autonomy programs are not currently structured this way, presenting a significant challenge for Replicator.
Sustainment is a particularly important part of the puzzle. The ability for the Department to acquire systems at scale to employ “mass” does not negate the need for the Department to improve the performance or adapt those systems to new battlefield requirements. As the Department has said before, software is never done. It requires constant improvement to stay relevant in a rapidly changing battlefield. Consider, for example, an automated target recognition algorithm designed to recognize a specific type of adversary system. Adversaries adapt rapidly: Russian forces in Ukraine have done everything from camouflaging aircraft with tires to adding cages to armored vehicles to protect against ATGM and drone attacks. Target recognition algorithms would need to adapt to recognize adversary systems through new types of camouflage and occlusions. The software development pipeline needed to update that perception system on an ongoing basis based on changing battlefield conditions is what we mean by sustainment. Similar challenges apply to elements of autonomy stack that come into play post perception (planning, controls, etc.).
There are still many questions that remain unanswered about Replicator. The specific roles of each DoD component, which existing programs will be involved, how money will be shifted around and allocated, etc. No matter how all of that shakes out, one thing is clear: a plan must be in place to accelerate the delivery and ongoing improvement of the software associated with each of the programs incorporated into the initiative.
Joe Larson, Deputy CDAO for Algorithmic Warfare, recently offered his perspective on the need for software-tooling-based testing infrastructure for autonomy:
What we really need are - in advance of getting to the operational testing, because it will be simply impossible to test autonomy against every potential combat scenario - we need a system that is enabled by tools to test autonomous solutions at every level where the autonomy will exist within the platform. That means we'll need environments to do that. We need simulation environments, we need digital twins, we'll need a way to bring autonomous solutions together in software and run them against CONOPs to see how they perform. This is not a question of standards, this is a question of the Department actually investing in software tooling and infrastructure that makes it feasible … It's not just the infrastructure to build the autonomy, it's the infrastructure to test the autonomy.
We’re glad to hear that CDAO understands the value proposition and importance of the infrastructure for autonomy software development, testing, and sustainment. Replicator’s goals and timelines are ambitious. They are also achievable, but only if the Department invests in the foundational tooling needed to accelerate development, enhance testing, and enable ongoing software sustainment.
State of play: CCA
“Air Force ‘crystal clear’ on drone wingmen urgency,” according to a recent article. It’s true: the “urgency” is crystal clear. But that clarity ends there.
CCA is the Air Force’s biggest bet for gaining a competitive advantage in a near-peer competition. In fact, the race is now on with other countries pursuing the development of similar technology. Still, despite its high profile, actual details on the program are slim and, at times, seemingly in conflict. First: what will each platform cost and how will that impact whether or not the platforms are attritable?
The FY24 NDAA called for three cost tranches of CCAs: $3M for an “expendable” version, $10M for an “attritable” version, and $25M for an “exquisite” model. In response, however, Secretary Kendall stated that the Air Force is “very opposed to those targets.” He recently reiterated those comments, saying that the expected cost of a CCA will be “on the order of a quarter or a third” of an F-35, in the $20M+ range per unit.
There is clearly a disconnect between Congressional and Air Force leadership on cost. CCA is still a “new start” program, requiring the full passage of a defense budget to begin the program in earnest. That disconnect on cost is a fundamental problem with significant impacts on program timelines and end capabilities.
With that in mind, let’s talk about timelines. In March, Service Acquisition Chief Andrew Hunter stated that the Air Force is “very much focused on speed-to-ramp, so we are looking to field that capability as rapidly as possible,” targeting deployments alongside crewed aircraft in “the later 2020s.” Recently, Secretary Kendall re-emphasized that timeline, saying “We intend to have the uncrewed collaborative combat aircraft in production by the end of the five year plan, if we can get started.”
That’s a big if, particularly if that disconnect between Congressional and Air Force leadership persists. That timeline is also liable to shift because the hardware and warfighting requirements are still being refined: “the Air Force is still in the ‘early stages’ of establishing key definitions regarding what it wants from its CCAs and working out the ‘right balance’ in terms of requirements.” A vast range of capabilities and goals have been outlined for CCAs over the last year or so, including which crewed platforms will be expected to integrate with CCAs, whether they’re expected to have dedicated roles and systems or "the full complement of systems that are on a fighter," range and refueling capability, etc. Making final decisions about what capabilities each CCA will have, rather than simply throwing CCAs at every program, and mapping those capabilities to requirements will be a major undertaking with significant implications for both the hardware and software elements of the program.
Furthermore, that 2028 timeline for “production” is likely in reference to LRIP - it will take another 2-3 years to reach full rate production and make real progress against the initial goal of fielding 1,000 CCAs. The current five year spending plan shows gradual increases over FY24 and FY25 (likely covering the initial RDT&E, requirements definition, and evaluation of different operating concepts for CCA) followed by a decrease in 2026 (perhaps for a bakeoff among the various competitors), and then $1.64B in FY27 and $3.03B in FY28 (for initial LRIP production). That is a far cry from the estimated initial program cost of over $20B, given the per-unit cost of $20M and initial order of 1,000 airframes.
The development of truly deterministic, and even lower level, autonomy is typically very expensive and requires a significant investment in both real world and simulated testing. Given the relatively short timeline for CCA to reach Initial Operational Capability (IOC), particularly in the context of a rapidly evolving threat environment, we expect there to be a greater emphasis on the development of tangible near-term capabilities that can be integrated into existing platforms and weapons systems.
Despite the high costs and significant ambitions for the effort, there has been little to no discussion of the software development and sustainment approach for CCA. As we have said repeatedly, the core challenge with autonomous systems is the software. The software that manages perception (allowing teaming and collision avoidance beyond just an intermittent Link 16-based data link), planning (enabling adaptive mission planning based on changing battlefield conditions), and airframe controls management (actually flying the aircraft autonomously) is the most difficult part of the equation. What is the Department’s approach to autonomy? Will there be a single stack developed or multiple stacks developed based on mission profile?
It is imperative that the Air Force prioritize establishing a robust approach to software development, testing, and sustainment alongside its efforts to define operating concepts and requirements. This approach and infrastructure should not be unique just to CCA, but should enable the whole of Air Force autonomy programs to together benefit from lessons learned and best practices.
In short, we’re incredibly excited about the prospect of the CCA program, but there is a tremendous amount of work to be done over the next five years. As discussions about budgeting, timelines, capabilities, requirements, and end use cases for CCAs continue to evolve, it is imperative that discussions about the associated software infrastructure that will enable all of those capabilities are not relegated to the background.
On a more lighthearted note, we will leave you with something that resonates with us, given our roots in the automotive industry, and speaks to the way in which CCAs could enhance survivability, lethality, and capability:
Beyond the headlines: Responsible AI
The last few weeks have been packed with responsible AI related news. The White House’s executive order, DoD’s updated AI strategy, and a supposed AI “risk and safety” cooperation coming out of the Biden-Xi meeting led the headlines, but there are a few other important articles and developments worth highlighting:
Lieber Institute at West Point | The Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy
Shawn Steene, one of the authors of this article, is the man in charge of managing DoDD 3000.09, making his interpretations outlined in this piece particularly important to anyone working on the national security applications of autonomy or AI.
The Declaration goes beyond political endorsement and involves that signing nations “work to ensure that their military and defense organizations adopt and implement in practice certain measures.”
Paragraph C of the Declaration calls on senior officials to go beyond typical oversight when it comes to AI capabilities, such as the senior reviews required under 3000.09.
“The less well-defined the use case for the given military AI capability, the greater the chance that the military AI capability in question will have functions or will generate outcomes that are not those that were intended.
“The more consequential that failures of a given military AI capability might be, the more rigorous the testing and assurance that should be applied to that military AI capability.”
DAF Scientific Advisory Board | Responsible AI for Supporting Combat Engagements
The study echoes several of the recommendations outlined in another study commissioned by the DAF that focused on T&E for AI-enabled and autonomous systems. For more on that study, including our assessment of its recommendation to use a DevSecOps process borne out of the autonomous vehicles industry, read the last edition of our newsletter.
“The study found that there are commercial practices and frameworks that the DAF can immediately leverage to accelerate implementation of RAI principles across its relevant programs.”
“Invest in the physical and digital infrastructure required to train AI models and incorporate RAI principles, including support for data curation and provenance. In addition, continued use and enhancement of the department’s data fabric structure, to assure accessibility to developers and operators as these capabilities evolve. The study’s framework facilitates continuous V&V and red teaming to increase model robustness and resilience, capable of countering potential adversarial actions targeting undesired behaviors.”
“Expand human-machine teaming and integration into a DevSecOps flow that include multidisciplinary teams over the full lifecycle. As a part of this process, identify data simulation needs and limitations from rare or high-consequence, and corner-case events.”
“Establish system-engineering boundaries (technical, risk, and performance) for AI agent delegation, grounded in the test and training contexts. Model processing products and derived results should be recorded for later review, with functions similar to an enhanced black box, to ensure auditability.”
News we’re reading
Autonomy is everywhere in defense these days. Make sense of the latest headlines by reading key quotes from recent articles of interest, plus brief commentary from Applied Intuition’s government team, below:
National Defense Magazine | Passing Ships - Navy Developing Simulations to Test Autonomous Vessels
Key quote: When determining what testing data is useful, cases that clearly fail or succeed are far less interesting than what Papelis called border cases: uncertainty that creates scenarios compelling enough to put a real boat in the water “and seeing what would happen under that condition.”
While every condition in an unlimited set can never be fully accounted for, the limitations removed in a simulation environment can bring it closer than it has ever been before.
“So the goal of this project is to build a simulation framework, a simulation environment, in which you can plug in different autonomy models … and build scenarios that are … rich enough to increase your confidence that this autonomy will actually be safe to use and it will meet its mission requirements,” he added.
Our take: Validating the performance of a maritime system’s perception and controls stacks is critical, but recreating a wide range of weather, sea states, and scenarios in real-world testing is costly, non-reproducible, and potentially dangerous. Simulation enables robust testing and validation of system performance across a much broader spectrum of scenarios.
As outlined in this article, however, simulation can also make real world testing significantly more targeted. Testing performance in simulation first helps development teams narrow down the number of scenarios that need further testing in the real world, saving money and reducing development times.
C4ISRNET | Is the Army moving too fast on drone adoption?
Key quote: As the service’s aviation branch chief put it, “There’s a path to autonomy. It’s not that we’re not autonomous and then [suddenly] we are autonomous — it’s going to be a graduated scale, or I kind of envision it as stair steps.” This is exactly the right approach and those “stair steps” should be data collection, driver assistance technology, and mapping. Given the multiple lines of effort that the Army is pursuing towards autonomy, ensuring a common set of steps for the Army to practice is critical. [...]
To collect the data that will eventually feed the algorithms powering the drones of the coming decades, the Army should take a page from what the private sector is doing in San Francisco: testing and collecting data from ground vehicles at scale. While it’s encouraging that the Army appears to be doing this to a limited extent, it should do more and instrument and collect data off of every ground combat vehicle. [...]
Thus, the Army must gain its own competency in mapping terrain by outfitting its fleet of wheeled and tracked vehicles with 3-D mapping technology. The instant updating of maps and algorithms’ ability to retrain during combat will become ever more important in ground operations.
Finally, the service should implement driver-assistance technology on its existing platforms where possible, allowing soldiers to benefit from technology that is now reaching maturity and, for the most part, comes standard on new commercial automobiles. Implementing this technology also serves the dual-purpose of upgrading current systems, helping to avoid costly multi-year and multi-billion dollar acquisition programs for new systems.
Our take: The recommendations here are on-point. Autonomy cannot be thought of as a black box capability that can be purchased off-the-shelf. Instead, every autonomy program needs to think about the pathway required to achieve an autonomous capability, and to update that algorithm based on changing battlefield conditions. Data collection and management, mapping, and retraining are important steps in that pathway, but it doesn’t end there.
Upgrading existing “legacy” platforms with sensors and capabilities that are common in the commercial automotive space is an easy starting point that could have significant impact on both soldier safety - by enhancing situational awareness and avoiding tragic accidents and mishaps, for example - and survivability and lethality. Army-wide modernization should involve both upgrading the vast numbers of legacy platforms with new features and developing entirely new platforms with higher degrees of autonomy.
Breaking Defense | With Middle East ‘deterrence’ in mind, US 5th Fleet conducts first unmanned live fire test
Key quote: “At-sea exercises have primarily focused on maritime domain awareness (MDA) duties by conducting high-speed maneuvers in formation or testing the intelligence, surveillance, and reconnaissance (ISR) capabilities of the USVs in real-world operational scenarios,” Mazzucco said. “While an absolute first for Washington’s USVs in Middle Eastern waters, the testing of lethal munitions from a MARTAC T38 Devil Ray USV represents the natural development of Task Force 59’s original mandate.”
He added that NAVCENT’s decision to step up the game on USVs and to expand the contribution of unmanned assets to the fleet’s daily tasks from non-kinetic to kinetic naval operations speaks volumes about two main elements: “first, NAVCENT’s growing confidence in the mission execution capabilities of unmanned platforms; second, a regional insecurity-driven pressing need to hasten a comprehensive integration of USVs in the 5th Fleet’s full-spectrum operations.”
Our take: This is an important step that speaks to the growing trust in uncrewed maritime systems. Though the NAVCENT statement highlights how a human operator “made the engagement decisions” and references how crewed and uncrewed ships collaborated to “identify and target” simulated hostile forces, the degree of actual autonomy involved in the test still remains unclear.