The White House finalized a framework in August for testing closed and advanced artificial intelligence (AI) models to assess safety and cybersecurity risks. This framework was implemented amid a resignation by a researcher from Anthropic due to concerns about controlling the capabilities of AI models and warnings from industry leaders about slowing competition in this area.
Challenges of Transparency and Effectiveness
Although this framework only refers to closed and advanced models, its precise criteria have not yet been publicly released and have only been provided to AI companies. This lack of transparency has led to significant criticism and raised questions about the government's ability to provide adequate oversight of advanced AI models. The framework operates voluntarily, and models can be submitted for evaluation before release, but the government has only thirty days to assess these models.
Read more: Steve Jones and the Challenges of AI Data Centers
Existing Gaps in Regulatory Capacities
There are many questions regarding the safety and cybersecurity criteria that models must pass. While various agencies such as the National Security Agency and the Office of Science and Technology Policy will participate in these evaluations, no information has yet been released about the experts and offices responsible for these assessments. Given the technical challenges and validation limitations in this area, the credibility of these evaluations depends on transparency and the presence of a strong technical workforce.
The White House's regulatory framework, with a combination of direct government assessment and a voluntary system, is at risk of inefficiency and lack of transparency. Due to three main gaps in its technical capacities, including a shortage of technical talent and an inability to attract and retain technical experts, the government struggles to conduct independent assessments. This issue could lead to conflicts of interest if reliance on industry expertise is used to assist in model evaluations.
Ultimately, for federal oversight of artificial intelligence to succeed, there is a need to design a framework that promotes transparency and independence. Establishing a regulatory board to oversee the market for external audits could be proposed as a suitable solution. This board could help improve independence and create a competitive market for AI auditing.
Read more: Researchers Warn: Risk of Human Extinction by 2030! · Regulatory Challenges of the U.S. AI Industry with Competition from China
