The National Institute of Standards and Technology (NIST), a U.S. Commerce Department agency, has re-released a testbed named Dioptra, which is designed to assess the impact of malicious attacks on AI systems, particularly those that poison AI model training data.
Dioptra is a modular, open-source web-based tool that helps companies and users measure, analyze, and track AI risks. It can benchmark and research AI models and simulate threats in a “red-teaming” environment to evaluate the robustness of these models against adversarial attacks.
Dioptra was first released in 2022 and aims to support companies, government agencies, and small to medium-sized businesses in testing AI models’ performance claims.
The tool allows users to conduct evaluations to understand better how adversarial attacks can affect machine learning models. The NIST emphasizes that Dioptra’s open-source nature enables widespread accessibility and use for AI risk assessment.
The release of Dioptra coincides with documents from NIST and the AI Safety Institute, which provide guidelines for mitigating AI risks, such as the misuse of AI to create nonconsensual pornography. The tool complements similar efforts by the U.K.’s AI Safety Institute, which has developed a toolset called Inspect for assessing AI model capabilities and safety.
This collaboration is part of an ongoing U.S.-U.K. partnership to advance AI model testing, initiated at the AI Safety Summit in Bletchley Park.
Dioptra is also a result of President Joe Biden’s executive order on AI, which mandates NIST’s involvement in AI system testing and the establishment of AI safety and security standards.
This order requires companies developing AI models to notify the federal government and share safety test results before public deployment. This regulatory framework aims to ensure that AI systems are comprehensively tested for safety and security.
Despite its capabilities, Dioptra has limitations. It is designed to work with models that can be downloaded and used locally, such as Meta’s Llama family. However, it cannot currently evaluate models that are accessible only via APIs, such as OpenAI’s GPT-4.
While NIST does not claim that Dioptra can completely eliminate AI risks, it believes the tool can identify specific attacks that degrade AI performance and quantify their impact, contributing to a better understanding and mitigation of AI vulnerabilities.
