Recent advances in Deep Learning have enabled the development of powerful AI systems. While such systems are increasingly being used in safety-critical domains, important aspects of such systems, such as their robustness, transparency, and decision fairness, are not yet fully understood. Security issues arise in particular when multiple parties share and exchange resources such as machine learning models or data sets. In this study, BSI examines the current state of research regarding attack vectors relevant to these scenarios and discusses the effectiveness of existing mitigations.

Based on the literature and practical investigations of specific use cases, open challenges in the research area are identified. In addition, this study provides a set of recommendations to improve the security of machine learning systems and related development processes.