AI can make logistics data as valuable as intelligence or operational data for mission success

Having insight into a broad array of data is critical for multi-domain operations. Now, military logisticians and industry can marshal data outside of its intended use to provide valuable, all-source intelligence for the all-domain warfighter.

This novel capability can be found in a U.S. Marine Corps logistics IT modernization effort known as Technical Data Management (TDM)-Catalyst. The underlying technology is from Progress MarkLogic, and it also has applications across the other military services and commercial industry.

TDM-Catalyst is the first-ever cloud-based application for the USMC and achieved full accreditation and DoD Authority to Operate (ATO) on AWS in 2021. It replaces four legacy systems to more efficiently track, catalog, and search material in the Marine Corps inventory. That’s everything from vehicles to radios and tools, literally every repair part and end-item configuration.

So what sort of business intelligence does Progress MarkLogic offer under TDM-Catalyst? To begin, equipment specialists define a full top-down breakout of every weapon system for the Marine Corps; think of an IKEA catalog and bill of material. They not only get a table of all the parts but an exploded view of them, which helps to better manage and define equipment variants, repairable items, spares, out year requirements and maintenance activities. TDM also implements automated generation of inventory compliance forms and supplies real-time contract planning information like source of supply rollup and supplier breakdown.

The data that’s being created in TDM-Catalyst can be useful at every level – from initial requirements, provisioning and cataloging, to procurement, acquisition, inventory, tracking, planning, trending and parts-heavy depot-level maintenance.

“Through this program, we’re showing that logistics data has huge importance on the intelligence side,” said Troy Gilbert, director of Public Sector Partnerships for Progress MarkLogic. “We tend to think that we should only focus on intelligence data or operational data, but this data is foundational to mission success. Organizations are now able to take data that’s used for logistics and use it to help determine business management decisions or if a supply shortfall is pending.”

And because the logistics data needs to be protected at different security levels, from confidential to top secret, organizations can use the Progress MarkLogic security controls to apply their field-level security policies – roles and permissions – to help determine who and what can and can’t view the information based on their security privileges.

In addition to logistics data, gleaning business intelligence insights can pay major dividends in commercial aerospace. For example, Airbus must go through a significant amount of flight testing in order to keep certifications current for their aircraft. Certifications can take hundreds if not thousands of flight hours, costing a fortune.

Data that they collected during flight testing was often unusable for additional certifications because it was very difficult to find it recorded for the specific parameters or certification needed. That meant every time a new certification or additional data was needed, additional, expensive tests would have to be flown.

“Progress MarkLogic gives aerospace companies the ability to search through significantly large amounts of data from multiple flight tests, whose volume can reach multiple petabytes,” said Gilbert. “Using technology like the MarkLogic Data Hub, companies don’t have to conduct additional flight tests to answer new and complex questions. They’re able to leverage the wealth of historical data that Progress MarkLogic can store and easily serve up through its powerful search engine.”

Putting a lens atop the data

It’s not hyperbole to say that the data in any system, in any business, is the core foundation of all business processes. If data quality and data security are not precise then business processes are going to fail.

That goes for both complex data and simple data, the quantity of which is exploding exponentially. This is seen across federal agencies, defense organizations, the intelligence community, and industry. Data can be text, structured or unstructured, binary, JSON (JavaScript Object Notation), and XML, to name several.

Lots of systems can handle simple data well from tables, rows, columns, and fields. Complex data like emails, reports, and log files is another matter. Harmonizing the two is even harder, but if you know how to look, that’s where the intelligence value is found and where Progress MarkLogic excels.

“Number one, it is complicated to get all your business data together,” said Gilbert. “Then it’s even more complicated to get a lens on top of all the business and operational data. When we talk about multi-model databases, what that means is whatever data format you have — from tabular to documents to geospatial — Progress MarkLogic is going to help you consolidate the information and make sense of it. It’s effectively that easy.

“There is different commercially available data management software. MarkLogic’s unique value lies in its ability to centralize heterogeneous data, creating a golden record that is accessible across the organization, while also giving you all the various lenses on top of your data to help you make sense of all the relationships and connections you have within the data.”

The ability to quickly ingest raw data, search, make sense, and then understand how to use it based on requirements has become a defining feature in the concept of information dominance – especially as requirements mature and evolve based on threat scenarios. Now add in the new capability of making sense of data outside of its intended use to provide intelligence, and users have a powerful new tool to squeeze every bit of value out of data.

Progress MarkLogic TDM-Catalyst does all that quickly, efficiently, and in repetitive cycles. It’s much quicker to design and implement the data when you can ingest in a raw format and can change the data on the fly in an agile fashion as you discover your needs, versus having to build out your full relational schema first and then figure out all the queries that need to be answered.

We’ve discussed how this results in new insights from data leading to business intelligence. The same is true of all-source intelligence and operational intelligence.

All-source intelligence, for example, can be leveraged by artificial intelligence and machine learning algorithms to inform new Concepts of Operation like Joint All Domain Command and Control. It can also be employed by organizations like the Committee on Foreign Investment in the United States (CFIUS), which reviews transactions involving foreign investment in the U.S. and real estate transactions by foreigners in order to determine the effect of those transactions on national security.

For the CFIUS mission, Progress MarkLogic can mark the criticality of individual parts. It doesn’t have to be the whole system; it might be just one part. Feeding that forward, you can identify all the critical parts that are essential to American military preparedness to ensure that the manufacturers of those vital parts aren’t surreptitiously purchased by a country like China or Russia.

An operational intelligence aspect of this capability could be the use of 3D additive manufacturing for certain parts. The Progress MarkLogic system can facilitate that through APIs to pull in the necessary data like design files, for example.

Conclusion

Using agile development practices and the Progress MarkLogic Data Hub technology, organizations like the Marine Corps have been able to overcome the distinct limitations of relational databases – namely the difficulty to ingest disparate data schemas and formats, and the challenge of creating usable data from undefined sources.

What Progress MarkLogic accomplishes is an ability to speak in different languages in terms of data, enabling rapid synchronization of business processes and data structures. Add in artificial intelligence and machine learning, and functions like military strategic planning that took weeks if not months can be accomplished in days.

Data, maps, schematics, images, documents, news clippings are ingested and understood to the point where users are proactively informed of areas that need close attention. Machines can be taught to engage certain functions in particular applications by turning a switch on or off, for instance.

In the context of JADC2 and the Air Force’s Advanced Battle Management System, for example, where the military is trying to create or connect disparate data sources for multi-domain operations, the implications for Progress MarkLogic at the user level, the tactical level, and even the senior-leader level to be able to quickly grab the information needed to create situational awareness and speed decision making is already game changing.

“We need to open the aperture of business data that is stored in silos and make them available to others via governance models and access controls. This ensures all data is unlocked and available for intelligence, artificial intelligence, machine learning or business decision,” said Gilbert.