When Twins Go Bad — Model Poisoning Attacks on Digital Twins

The manufacturing industry already appreciates the cost savings and efficiency improvements that digital twins offer. With information about the design, operation, and maintenance of devices all in one place, the digital twin model facilitates everything from better quality control to advanced analytics to investment predictions. But what if the integrity of the digital twin’s information is poisoned?

The Benefits of Digital Twins

By a digital twin, we are referring to a unified interface for accessing the lifecycle data of a device. Information is collected during the design, construction, commission, operation and maintenance of the device, and collated in the digital twin. This supports such use cases as:

  • Advanced analytics and machine learning
  • Collaboration
  • Quality control
  • Preventative maintenance
  • Remote support

Digital twins have even been used by manufacturers to predict whether building a new factory was necessary to increase productivity, or if current installations could be optimized.

In fact, digital twins often support these use cases better than the actual devices. In some cases, digital twins have become so important in daily operations that downtime can’t be tolerated. Rather than maintaining the twin in the cloud, companies are moving these within their own networks, and sometimes directly on the factory floor.

If you are interested in reading more about digital twin implementations, here is the link to the IIC November 2019 edition of the Journal of Innovation in which my colleagues Mark Hearn and Simon Rix published a paper.

Data Integrity and Model Poisoning

All the use cases above critically rely on the data in the digital twin being correct. Imagine trying to remotely support a device based on incorrect construction information. Consider the consequences of decisions based on analysis of incorrect data.

By model poisoning, we are referring to the deliberate tampering (modification, deletion or insertion) of data in the digital twin, with the goal of compromising one or more use cases based on that data. The result might be a part that is manufactured incorrectly because the design specs were changed, or a costly business decision based on incorrect analytics, or unscheduled downtime because of maintenance based on incorrect information. On top of direct financial cost, model poisoning may damage a company’s reputation, affect employee morale, and in extreme cases, even cause physical injury.

There are several places where the model could be poisoned:

  • Input data in transit to the digital twin
  • Data stored in the digital twin
  • Data in use for analysis
  • Output data in transit to other systems (possibly including other digital twins)

The risk of model poisoning is higher when the digital twin is running on an open platform, and in a potentially hostile environment such as the factory floor.

Solution

How can model poisoning be prevented? Data integrity is a well-studied problem, especially in the cryptographic community. Data integrity can be safeguarded by carrying information alongside the data that can be used to confirm its origin and its contents, and where this supporting information cannot be easily modified. Cryptographic techniques such as message authentication codes and digital signatures were designed for exactly this purpose.

A solution for model poisoning must consider all the ways in which data tampering could occur and must be able to handle the situation where tampering is attempted by someone with legitimate access to the digital twin. This suggests a need for software protection, because an attack on the software working with the digital twin data could be just as damaging as an attack on the data itself.

Conclusion

Digital twins are used to improve efficiency and make critical business decisions. It is critical that the integrity of the digital twin be maintained to prevent model poisoning.

Have you been thinking about model poisoning prevention? Tell us about it!

Leave a Reply