AI Systems Implementation Issues and Risks

Gone are the days when?we feared artificial intelligence (IA) taking our jobs. Now, it's no surprise that we anticipate AI functionalities, tools, and systems to integrate seamlessly into every software and product, enhancing human capabilities in ways that were once unimaginable. It is no longer a fearmongering trend; rather, it has become an indispensable part of our daily lives.

As businesses continue to leverage AI to improve their systems and operations in various fields, in sectors such as healthcare, transportation, education, e-governance and so on, AI systems and tools play an important role in streamlining operations, providing personalised services, enabling data-driven decision making, and most importantly offering innovative solutions to businesses on a daily basis.

AI Contracts ?

Depending on the nature and scope of the AI system, AI contracts may be entered into for the purchase of either an existing system from a third party, or for the development of a customised AI system tailored to fit the needs of a business. It may also be for an existing AI system however, with specific adjustments negotiated to further tailor it to customer needs.

Unlike ordinary tech contracts, AI contracts need to address various AI-specific risks, the changing nature of AI solutions, and the ever-evolving regulatory environment, including how AI vendors are adapting to the changing AI landscape.

Considerations When Negotiating AI Contracts?

Development and Assessment

At the outset, it is essential for businesses to clearly outline the problems they wish to resolve through AI and communicate them to the AI vendor/developer.? This essentially promotes innovation and allows the developer to provide bespoke solutions that fit the specific needs of the business.

When entering into AI agreements, businesses are encouraged to understand from the AI vendors the security risks and vulnerabilities of the AI system whilst working towards implementing appropriate safeguards and measures to mitigate them.

To ensure risk and vulnerabilities are addressed, the AI contract may set out obligations to be adopted by the AI vendor, such as ensuring mandatory risk assessment and testing procedures are routinely done.

By conducting risk assessments of the AI software, businesses can ensure that potential vulnerabilities and attacks are exposed and implement an action plan to mitigate these risks. This is notable to maintain system integrity and resilience in order to build trust with its users.

As the nature of AI is constantly developing and improving, businesses must ensure that performance is monitored, tested and recalibrated on an ongoing basis.

AI Security and Resilience ??

Security and resilience are essential for maintaining the confidentiality, integrity and availability of the data and the AI system, as well as preventing the exploitation of cyberattacks, data breaches, fraud, or sabotage.?

As security risks may result from various sources, such as malicious attacks, human errors, technical glitches, or external threats. During the negotiation stage of the AI contract, businesses must consider the importance of AI-related security risks, including where they can originate, and how the vendor addresses them in practice. ??

Data Manipulation

One major security risk includes the manipulation of training data, which allow attacks to exploit potential vulnerabilities in AI algorithms. Manipulation of the data or attempts to corrupt the learning process of the AI tool could lead to inaccurate or biased results.

When an attacker subtly alters AI models during training, causing unintended behaviour under certain triggers. Such ‘backdoor’ attacks are increasingly relevant in a world where users often rely on third-party models rather than building models from scratch. Thus, businesses should ensure that they understand how AI vendors mitigate such risks.

Another possible risk is when AI's internal architecture is directly modified through the insertion of malicious code or alter the model's structure. One of the ways to mitigate the risks of data manipulation is to ensure that safeguards are in place to prevent such manipulation of training datasets.

It is also vital to pay close attention to location of the data as it may be transferred to a different location for training purposes. Businesses must be aware of whether the protections in relation to the new location for data training are robust, as well as ensuring technical and operational safeguards are in place, including contractually, the allocation of responsibility and risk to protect against threats to the AI systems and appropriate considerations to liability in case of an attack.

Additionally, the implementation of controls to override, reverse or halt output or operation of AI systems in the case of an incident. AI contracts should clearly define notification requirements, response time and cooperation steps related to any incident.

Data Management

Businesses should understand how developers programme the AI systems and ensure that any failures or inadequacies can be communicated, traced, and addressed efficiently. In prioritising traceability, this can mitigate major issues, leading to any potential failures being addressed efficiently. With that, businesses can ensure a mechanism to investigate and correct any issues caused by AI systems. Business should also ensure that developers record details such as training data source, handling procedures, model design, and codebase changes.

Data training

The accuracy, dependability and performance of the AI software centres around the quality of data training by the AI vendor. As AI software are developed by relying on sets of data and intelligent algorithms that learn from patterns and features within the data, thus emphasising the importance of the quality in the development stage. Businesses should be aware of the significance of acquiring, refining, and managing high-quality data. If the quality of the data isn't prioritised, it can lead to AI systems making poor decisions, which may include bias and discrimination. Moreover, it's crucial to ensure that the training environment closely resembles the real-world context where the AI will be applied.

Businesses must also evaluate the legality of data acquisition, including potential risks of intellectual property infringement. It is crucial to assess whether the collection, utilisation, or creation of training data and outputs might violate privacy laws, intellectual property rights, competition regulations, or lead to discriminatory practices. Additionally, consider contractual obligations and prohibitions that could affect data handling. When dealing with third-party data, it is critical to also determine if it is confidential and whether contractual or equitable duties of confidence restrict its use for development or sharing with collaborators.

Data governance

One of the primary challenges in AI procurement lies in data governance. This encompasses ensuring the high quality, accuracy, security, and ethical handling of data collected, processed, and analysed by AI systems. The risk here is evident in how incomplete or poor-quality data can introduce performance and ethical concerns. For example, it may lead to biased supplier selections, misinterpretation of market trends, or incorrect assumptions about supplier performance. An illustrative case is AI hallucinations, where erroneous or misleading information—such as biases in training data or unintended interpretations of input data—can distort outcomes.? It is essential to recognise that AI is not infallible; it can be prone to errors and inaccuracies.?

Compliance

Businesses should adopt a structured approach which entails incorporating clauses that anticipate potential regulatory changes in the region. Additionally, contracts should clearly define responsibilities and risk allocation in scenarios where non-compliance results in financial penalties, transaction reversals and obligations to cooperate during regulatory scrutiny. Moreover, businesses must ensure compliance with a spectrum of laws including data protection, privacy, intellectual property, and cybersecurity regulations.

Conclusion

AI is transforming industries by boosting efficiency, personalising services, and enabling data-driven decisions. However, deploying AI requires careful consideration especially in contracts. AI contracts need to address risks in the development and implementation of AI and by applying clear compliance frameworks and strong governance practices, businesses can manage challenges effectively. ?As AI continues to advance, businesses must stay alert, regularly reviewing and adjusting their strategies to harness AI's full potential responsibly and sustainably.