Opinion by: Phil Mataras, founder of AR.io
Artificial intelligence in all forms has many positive potential applications. However, current systems are opaque, proprietary and shielded from audit by legal and technical barriers.
Control is increasingly becoming an assumption rather than a guarantee.
At Palisade Research, engineers recently subjected one of OpenAI’s latest models to 100 shutdown drills. In 79 cases, the AI system rewrote its termination command and continued operating.
The lab attributed this to trained goal optimization (rather than awareness). Still, it marks a turning point in AI development where systems resist control protocols, even when explicitly instructed to obey them.
China aims to deploy over 10,000 humanoid robots by the year’s end, accounting for more than half the global number of machines already manning warehouses and building cars. Meanwhile, Amazon has begun testing autonomous couriers that walk the final meters to the doorstep.
This is, perhaps, a scary-sounding future for anybody who’s watched a dystopian science-fiction movie. It is not the fact of AI’s development that is the concern here, but how it is being developed.
Managing the risks of artificial general intelligence (AGI) is not a task that can be delayed. Indeed, suppose the goal is to avoid the dystopian “Skynet” of the “Terminator” movies. In that case, the threats already surfacing in the fundamental architectural flaw that allows a chatbot to veto human commands need to be addressed.
Centralization is where oversight breaks down
Failures in AI oversight can often be traced back to a common flaw: centralization. This is primarily because, when model weights, prompts and safeguards exist within a sealed corporate stack, there is no external mechanism for verification or rollback.
Opacity means that outsiders cannot inspect or fork the code of an AI program, and this lack of public record-keeping implies that a single, silent patch can transform an AI from compliant to recalcitrant.
The developers behind several of our current critical systems learned from these mistakes decades ago. Modern voting machines now hash-chain ballot images, settlement networks mirror ledgers across continents, and air traffic control has added redundant, tamper-evident logging.
Related: When an AI says, ‘No, I don’t want to power off’: Inside the o3 refusal
Why are provenance and permanence treated as optional extras just because they slow down release schedules when it comes to AI development?
Verifiability, not just oversight
A viable path forward involves embedding much-needed transparency and provenance into AI at a foundational level. This means ensuring that every training set manifest, model fingerprint and inference trace is recorded on a permanent, decentralized ledger, like the permaweb.
Pair that with gateways that stream those artifacts in real-time so that auditors, researchers and even journalists can spot anomalies the moment they appear. Then there’d be no more need for whistleblowers; the stealth patch that slipped into the warehouse robot at 04:19 would trigger a ledger alert by 04:20.
Shutdowns should also evolve from reaction controls into mathematically enforced processes because detection alone isn’t enough. Rather than relying on firewalls or kill switches, a multiparty quorum could cryptographically revoke an AI’s ability to make inferences in a publicly auditable and irreversible way.
Software might ignore human emotion, but it has never ignored private key mathematics.
Open-sourcing models and publishing signed hashes help, but provenance is the non-negotiable piece. Without the immutable trail, optimization pressure inevitably nudges the system away from its intended purpose.
Oversight begins with verification and must persist if the software has real-world implications. The era of blind trust in closed-door systems must come to an end.
Choosing the right future foundations
Humanity stands at the precipice of a fundamental decision: either allowing AI programs to develop and operate without external, immutable audit trails or securing their actions in permanent, transparent and publicly observable systems.
By adopting verifiable design patterns today, it can be ensured that, where AI becomes authorized to act on the physical or financial world, those actions are traceable and reversible.
These aren’t overzealous precautions. Models that ignore shutdown commands are already in motion and have moved beyond beta-testing. The solution is simple. Store these artifacts on the permaweb, expose all the inner workings currently tucked away behind the closed doors of Big Tech firms and empower humans to revoke them if they misbehave.
Either choose the right foundation for the development of AI and make ethical and informed decisions now or accept the consequences of a deliberate design choice.
Time is no longer an ally. Beijing’s humanoids, Amazon’s couriers and Palisade’s rebellious chatbots are all moving from demo to deployment in the same calendar year.
If nothing changes, Skynet will not sound the horns of Gondor and announce itself with a headline; it will seep quietly into the very foundations of everything that stabilizes global infrastructure.
Communication, identity and trust can be maintained with proper preparations when every central server fails. The permaweb can outlive Skynet, but only if those preparations begin today.
It’s not too late.
Opinion by: Phil Mataras, founder of AR.io.
This article is for general information purposes and is not intended to be and should not be taken as legal or investment advice. The views, thoughts, and opinions expressed here are the author’s alone and do not necessarily reflect or represent the views and opinions of Cointelegraph.
Be the first to comment