Singapore: First-of-its-kind Generative AI Evaluation Sandbox

Singapore has launched an experimental platform which Generative AI developers may use in building responsible AI solutions.

Introduction

Earlier this week (on Tuesday 31 October 2023), the Singapore Infocomm Media Development Authority (“IMDA”) and the AI Verify Foundation¹ unveiled the first-of-its-kind Generative AI (“Gen AI”) Evaluation Sandbox (the “Sandbox”).

The Sandbox is part of a collective effort to provide a common standard approach to assess Gen AI, bringing global ecosystem players together through concrete use cases, and enabling the evaluation of trusted AI products.

Sandbox approach

The Sandbox will make use of a new evaluation catalogue (link here) as a shared resource, which lays out common baseline methods and recommendations for Large Language Models (LLM).

As a start, key model developers (such as Google, Microsoft, and Amazon Web Services etc); app developers with concrete use cases (such as Singtel, OCBC etc); and external testers (such as Deloitte and EY) have confirmed their participation in the Sandbox. Presently, there is a total of 16 participants within the sandbox, and the participants will assist in developing a more robust testing environment.

In addition, relevant regulators like the Singapore Personal Data Protection Commission (PDPC) will be assisting in the experimentation and development of the Sandbox, allowing all parties along the supply chain to be transparent about their needs.

Further, it is anticipated that the Sandbox use-cases will reveal gaps in the present landscape of Gen AI evaluations, particularly in domain-specific (e.g. in HR or Security) and cultural-specific areas which are currently under-developed. The Sandbox will also develop benchmarks for evaluating model performance in specific areas that are important for use cases, and for jurisdictions like Singapore and its unique cultural and language specificities.

How Clyde & Co can help

We welcome the introduction of the Sandbox since it will keep Singapore’s regulatory approach shoulder to shoulder with Gen AI developments in other jurisdictions. Additionally, we believe that the Sandbox will provide the space needed for players in the AI ecosystem to experiment, innovate and improve AI solutions including Gen AI solutions and overall work towards building safe and trustworthy AI solutions for all.

Clyde & Co has a cross-practice global AI working group focused on supporting clients in managing the risks and compliance in their adoption and deployment of AI. We are dedicated to providing comprehensive support to navigate these complexities and ensure you leverage AI's potential whilst mitigating its challenges effectively.

Should you like to discuss the Sandbox, the digital regulatory landscape or require any legal assistance with your AI solutions in Singapore (or other jurisdictions), please feel free to get in touch with our specialists listed below. 

1  AI Verify Foundation is a not-for-profit foundation which is a wholly owned subsidiary of the IMDA, and is a global open-source community that convenes AI owners, solution providers, users, and policymakers, to build trustworthy AI.