Sponge Attacks: Service Denial

Imagine that your company is particularly concerned about its sustainability and measures its use of electricity and intends to keep its consumption at current levels. We are aware that AI systems can be very intensive users of computation and electricity. A sponge attack could involve attackers who intentionally misuse the system, sucking up computing power and energy, and overloading, destabilizing or physically damaging the system.

While these may not directly impact user privacy, there may be indirect impacts. Errors or failures may lead to data leaks, provide a smokescreen to draw attention from other cyberattacks attempting to exploit other vulnerabilities to steal data or introduce data malignancies, or to disrupt privacy-preserving AI system controls.

While security and engineering resources are more likely to take the lead protecting against these attacks, here are some questions a Privacy Engineer should be asking during the AI product lifecycle:

Plan/Design

• What is the baseline and the expected resource requirements of the AI system under normal operation?
• Is the prompt design going to gracefully handle unexpected or complex inputs, such as endless loops, recursive requests, or other natural language requests that would lead to significant processing?
• Is the system designed to abort requests that are leading to high energy consumption?
• What are the potential privacy implications of system failures caused by a sponge attack?

Implement/Test

• Are the designed input validation and sanitization techniques functioning and protecting against malicious or unexpected inputs?
• Are mechanisms built to monitor resource utilization and detect anomalies that might indicate a sponge attack in progress?
• What is the plan for responding to a potential attack?
• Did the tabletop tests consider an incident leading to a suspected privacy breach?

Maintain

• Are continuous monitoring or other monitoring in place to detect sponge attacks?
• How will the system be kept up-to-date, especially to prevent the successful repeat of sponge attacks that occur to the production system?
• Are the incident response plans functioning and consulting with the privacy team appropriately?