OpenAI has released the GPT-4o System Card, a document detailing the safety protocols and risk assessments conducted prior to the model’s public launch in May. The card reveals how external security experts, known as red teamers, were tasked with identifying potential risks in the model.
These risks included issues like unauthorized voice cloning, generation of harmful content, and reproduction of copyrighted material. The findings from these evaluations are now available to the public.
The risk assessment framework used by OpenAI categorized GPT-4o as a “medium” risk model. This classification was based on four main categories: cybersecurity, biological threats, persuasion, and model autonomy.
While most categories were considered low risk, the “persuasion” category raised concerns due to the model’s potential to produce content that could be more persuasive than human-written material, which contributed to the overall medium-risk rating.
The system card also includes evaluations from OpenAI’s internal teams, as well as from external groups like Model Evaluation and Threat Research (METR) and Apollo Research. These evaluations are part of OpenAI’s ongoing efforts to ensure the safety of its models, a process that has been similarly applied to earlier models such as GPT-4, GPT-4 with vision, and DALL-E 3.
This release comes at a time when OpenAI is under increasing scrutiny for its safety standards, facing criticism from various stakeholders including its own employees and government officials.
Just before the system card was made public, a report highlighted a letter from Senator Elizabeth Warren and Representative Lori Trahan, raising concerns about OpenAI’s handling of safety issues and whistleblowers. This context underscores the significance of the system card’s release.
Given the proximity to the upcoming US presidential election, the release of the GPT-4o System Card is particularly relevant. The model’s potential to spread misinformation or be exploited by malicious actors adds to the concerns about its safety.
There are also growing calls for greater transparency from OpenAI regarding the model’s training data and safety testing. Legislative efforts, such as a proposed bill in California, could further impact how OpenAI and similar companies are required to evaluate and deploy their models. The system card ultimately highlights the heavy reliance on OpenAI’s own evaluations, even with the involvement of external experts.
