NIST Releases Dioptra a Tool for Testing AI Model Risk and Safety

The National Institute of Standards and Technology (NIST), a U.S. Commerce Department agency, has reintroduced a tool designed to evaluate how malicious attacks, particularly those that ‘poison’ AI model training data, can impact AI system performance. This tool, named Dioptra, is modular, open-source, and web-based.

Table of Contents

Dioptra’s Objectives and Capabilities

Dioptra, initially launched in 2022, assists companies and individuals in assessing, analyzing, and tracking AI risks. It allows for benchmarking and researching models, providing a platform to expose them to simulated threats in a ‘red-teaming’ environment.

‘Testing the effects of adversarial attacks on machine learning models is one of Dioptra’s goals,’ NIST stated in a press release. The software is open-source and freely available, intended to help government agencies and small to medium-sized businesses evaluate AI developers’ performance claims.

Enhancing AI Safety and Security

Released with documents from NIST and the new AI Safety Institute, Dioptra provides guidance on mitigating AI dangers, like generating nonconsensual pornography. This follows the U.K. AI Safety Institute’s Inspect toolset, aimed at assessing model capabilities and safety. The U.S. and U.K. are collaborating on advanced AI model testing, as announced at the AI Safety Summit in Bletchley Park last November.

Executive Order and AI Standards

Dioptra is part of President Joe Biden’s executive order on AI, which mandates NIST to assist with AI system testing. The order establishes standards for AI safety and security, requiring companies, including Apple, to notify the federal government and share safety test results before public deployment.

Challenges in AI Benchmarking

AI benchmarking is challenging due to the complexity and proprietary nature of sophisticated AI models. A report from the Ada Lovelace Institute found that current policies allow AI vendors to selectively choose evaluations, making it difficult to determine real-world safety.

Limitations of Dioptra

NIST acknowledges that Dioptra cannot fully de-risk models but suggests it can highlight which attacks might degrade AI system performance and quantify the impact. However, Dioptra is currently limited to models that can be downloaded and used locally, like Meta’s Llama family. Models accessible only through an API, like OpenAI’s GPT-4, are not supported at this time.

Benefits of Using Dioptra

The benefits of using Dioptra include:

Improved AI safety: By testing models against simulated threats, developers can identify vulnerabilities and improve the overall security of their systems.
Enhanced transparency: Dioptra provides a platform for researchers and developers to share information about model performance and vulnerabilities, promoting transparency in AI development.
Better decision-making: With Dioptra’s results, stakeholders can make informed decisions about AI system deployment and use, reducing the risk of unintended consequences.

Future Developments

Dioptra is an ongoing project, with future developments planned to address the limitations mentioned earlier. Some potential areas for improvement include:

Supporting more models: Expanding Dioptra’s capabilities to support a wider range of models, including those accessible only through APIs.
Enhancing user experience: Improving the user interface and workflow to make it easier for developers to use and interpret the results of Dioptra.

Collaboration and Future Directions

The development and testing of AI systems require collaboration across industries and organizations. The U.S. and U.K. are collaborating on advanced AI model testing, as announced at the AI Safety Summit in Bletchley Park last November. This partnership aims to:

Promote international standards: Establishing common standards for AI safety and security across countries.
Foster innovation: Encouraging collaboration and knowledge-sharing among researchers, developers, and industry leaders.

Conclusion

Dioptra is an important tool in the development of AI systems, providing a platform for testing and evaluating model performance against simulated threats. By using Dioptra, stakeholders can improve AI safety, enhance transparency, and make better decisions about AI system deployment and use. The ongoing development and improvement of Dioptra will be essential in addressing the challenges and limitations mentioned earlier.

Recommendations

Based on the benefits and limitations of Dioptra, we recommend:

Using Dioptra for testing and evaluation: Developers should consider using Dioptra to test and evaluate their models against simulated threats.
Collaborating with other stakeholders: Researchers, developers, and industry leaders should collaborate to improve Dioptra’s capabilities and address its limitations.