In the evolution of democratic institutions, the transition from private experimentation to public infrastructure has always required a foundational element: trust. Recent developments within the leading laboratories of artificial intelligence suggest that this foundation is currently under significant duress. As I analyze the shifting landscape of AI governance, it becomes clear that we are moving beyond the era of self-regulation toward a period where institutional constraints—what some industry leaders call 'rules of the road'—are becoming a political necessity.
The Erosion of Internal Oversight
The reported disbanding of OpenAI’s preparedness team represents a critical juncture in corporate governance. This unit, tasked with assessing catastrophic risks such as rogue models or cyber-threats, has seen its responsibilities redistributed as the organization moves toward a potential initial public offering. This pattern of safety team dissolution—following the previous exits of ethics and superalignment leads—has drawn sharp criticism from former staff who suggest that 'shiny products' are being prioritized over safety concerns. From a policy perspective, this trend signals a weakening of internal checks and balances, suggesting that the industry’s internal 'constitutional' safeguards are being superseded by commercial imperatives.
Scaling Laws and the Concentration of Power
"The concentration of power in the industry is not primarily driven by policy but by 'scaling laws.'"
Dario Amodei of Anthropic has recently acknowledged a fundamental crisis of trust, noting that the public remains skeptical of both tech companies and governments. He argues that power concentration is an inherent property of scaling laws—where performance increases with computing capacity—rather than a mere byproduct of regulation. This suggests that even open-weight models may not decentralize power as effectively as hoped. To counter these risks, there is a growing call for external institutional frameworks, such as a FINRA-like entity or formalized testing protocols, to balance innovation with safety. As we have seen in historical governance models, when power concentrates naturally due to technical advantages, only robust, external rules can ensure that the technology serves the broader public interest rather than a narrow set of interests.
