OPC UA architectures for AI and Big Data

The big vision of the Josef Ressel Center for Intelligent and Secure Industrial Automation (JRC ISIA) is the creation of a digital assistant for industrial control systems of discrete automation (e.g., production systems) that shall assist humans and intelligently control machines. A conceptual overview is shown in the following figure:

Such a digital assistant acts to a certain extent as a digital twin of the machines in the sense of Kritzinger¹ on the one hand and as an intelligent agent in the sense of Russel & Norvig² on the other hand.

Aside from control, AI and security research the following research lines concerning architectural aspects originated from the above vision:

1. Aspects of model-based system architectures and digital twins for industrial control systems.

2. Aspects of OPC UA architectures that enable the integration of AI and Big Data into industrial control systems.

3. Aspects of OT security architectures and threat modeling for industrial control systems.

Here we present the second of those research lines. This research line is based on the following architectural outline of the digital assistant, which is heavily based on OPC UA: The digital assistant collects data of the shop floor machines via OPC UA, as well as from superordinate systems, like a SCADA system. This time varying data is stored in an information engine and provided to digital twins of the respective machines and intelligent atents that provide assistance and control capabilities, while acting on those digital twins.

From this conceptual picture, various research questions arise concerning OPC UA-based architectures that enable the integration of AI and Big Data into industrial control systems. Many of these questions are concerned with the “information engine” depicted above. In the following, we present a selection of our research papers that address these questions.

Big Data architecture with and for OPC UA

In [HHH23] we describe a Big Data architecture for industrial control systems. It targets OPC UA for both information collection but also configuration, i.e., the query model itself is based on an OPC UA information model. The Big Data architecture is based on the well-known Lambda architecture, which is a hybrid architecture that combines batch and stream processing.

The proposed system buffers and preprocesses the information for the purpose of harmonizing and providing a holistic state space of a factory, as well as mappings to the current state of a production site. That information can be made available to multiple processing sinks, decoupled from the data sources, which enables them to work with the information without interfering with devices of the production, disturbing the network devices they are working in, or influencing the production process negatively.

Time series databases with OPC UA semantics

Time-varying data is the prevalent data type in industrial control systems and this data is made persistent in time series databases, such as InfluxDB. With the rise of Industry 4.0, OPC UA became the prevalent standard for data communication. Most notably, however, OPC UA also brought semantic modeling of information with a whole ecosystem of information models.

Interestingly, the combination of both concepts, time series databases with OPC UA semantics, has not been established so far. In [LUH26] we present an concept that brings OPC UA semantics to time series databases. This allows to reconstruct the original OPC UA address space from the time series stored, enabling a whole range of applications, e.g., for digital twins, for virtual commissioning, for unit testing, for cybersecurity honeypots, and so on.

Reinforcement Learning with OPC UA

Reinforcement Learning (RL) is the machine learning paradigm that targets sequential decision making problems, such as optimal control of industrial processes. In the recent decade, RL achieved state-of-the-art results in various fields, such as robotics, natural language processing, autonomous driving, and or for scheduling tasks of industrial systems.

Given that OPC UA is the de-facto standard for industrial communication and information modeling, the natural question arises how RL can be integrated into industrial control systems via OPC UA. In the following, we present two of our research papers that address this question.

In [SUH23] we present a mini review on the utilization of RL with OPC UA. We provide a brief technical overview of both technologies and carry out a semi-exhaustive literature review to gain insights on how RL and OPC UA are applied in combination. With this survey, three main research topics have been identified, following the intersection of RL with OPC UA:

1. Industrial applications of RL using OPC UA for communication (10 papers)

2. Architectures for the integration of RL into industrial environments utilizing OPC UA (5 papers)

3. RL applied for information inference from OPC UA information models (2 papers)

The results of the literature review show that RL is a promising technology for the control and optimization of industrial processes, but does not yet have the necessary standardized interfaces to be deployed in real-world scenarios with reasonably low effort.

In [SKWH23] we present an architecture for deploying RL in industrial environments. We present an OPC UA based Operational Technology (OT)-aware RL architecture, which extends the standard RL setting, combining it with the setting of digital twins. Moreover, we define an OPC UA information model allowing for a generalized plug-and-play like approach for exchanging the RL agent used. In conclusion, we demonstrate and evaluate the architecture, by creating a proof of concept. By means of solving a toy example, we show that this architecture can be used to determine the optimal policy using a real control system.