Counter AI: What Is It and What Can You Do About It

This session will describe the technologies, counter-AI attacks, and threat models that occur within the Machine Learning Operations (MLOps) lifecycle. As the strategic importance of AI increases, so too does the importance of defending those AI systems.

To understand AI defense, it is necessary to understand AI offense—that is, counter AI. This session describes counter AI.

First, we describe the technologies that compose AI systems (the AI Stack) and how those systems are built in a MLOps lifecycle. Second, we describe three kinds of counter-AI attacks across the AI Stack and five threat models detailing when those attacks occur within the MLOps lifecycle.

Finally, based on Carnegie Mellon University Software Engineering Institute research and practice in counter AI, we give two recommendations. In the long term, the field should invest in AI engineering research that fosters processes, procedures, and mechanisms that prevent vulnerabilities being introduced into AI systems. In the near term, the field should develop the processes necessary to efficiently respond to and mitigate counter-AI attacks, such as building an AI Security Incident Response Team and extending existing cybersecurity processes like the Computer Security Incident Response Team Services Framework.

What Participants Will Learn:

• The importance of defending AI systems
• The difference between AI defense and AI offense
• The technologies and counter-AI attacks within the MLOps lifecycle
• Recommendations for near-and-long-term actions to take