For 30 years, the world wide web has run on a surprisingly profound social contract. Most sites are free for search engines to access, but if you use their content, you give credit by linking to the source.

Recently, that social contract has begun to collapse. Artificial intelligence tools are crawling sites not to link to them, but to train models and generate answers (which may or may not be accurate).

When you search for something, ChatGPT’s response or Google’s AI Overviews may still include links to sources, but they’re a kind of optional extra to the main answer.

This has triggered a bad dynamic for website owners, the public, and even AI companies themselves. As websites lose traffic (and revenue), many are beginning to block AI scraping tools, meaning AI results depend more on low-quality websites (many of which are also generated by AI). As a result, good information can be harder than ever to find.

How We Got Here

In the early days of the world wide web, search engines, and content creators came to an agreement about crawling (the practice of technologically examining a site to index it, so it can be served up in search results). Content creators would provide access to their sites for free and even allow search engines to reproduce small snippets of text.

In return, search engines provided links to the sites owned by content creators, who benefited from that web traffic. If content creators didn’t like the deal, they could prevent search engines from crawling their site with instructions in a file called robots.txt.

But if AI tools no longer provide web traffic, it cuts content creators out of the economic loop. There are also other costs associated with each visit to a website, so AI crawling can cost website providers money while not giving them any of the ad or other revenue that would come from human traffic. AI crawlers also crawl more deeply and more intensely than traditional web crawlers, magnifying that cost.

This change in traffic patterns isn’t a small or hypothetical problem. Cloudflare, a web hosting and service company that manages 30 percent or more of the top 10,000 sites on the internet, estimates over half of all web traffic is now AI bots.

Some of this will be AI agents supervised directly by people, but the majority will be crawlers. Site owners can use robots.txt to ask AI crawlers to stay off their sites—but some AI companies may ignore this polite request.

If the AI companies do honor the request, that can create a different problem. Sites containing misinformation are far less likely to ban AI crawlers, so the AI answers won’t be informed by high-quality sources.

What’s Happening in the Short Term

On the horizon is an event dubbed “Google Zero”—the day when through-traffic from Google drops to nothing. While some grey-haired diehards (like one of the authors of this piece) might still click through to verify AI answers, this traffic is rapidly dwindling, as a direct result of AI summaries.

A study of Wikipedia confirms this, showing that traffic in the English language version of the site dropped off quickly with the launch of AI summaries on Google in English, and that the same pattern occurred in other languages as AI summaries were rolled out. Never having to click through to get an answer might seem great for information seekers, but the reality is more complex.

Many sites are now blocking AI crawlers altogether. Site owners who decide to block AI crawlers are less likely to be linked in AI Overviews answers, even when the AI tool can still access the content to ground its answers (using a technique called retrieval-augmented generation).

Alternative “pay to crawl” models have been suggested as a way to compensate content creators, but haven’t gained traction.

Come September 15, Cloudflare sites will block AI crawlers by default on pages that contain advertising (and therefore make money for content creators).

This means up to 30 percent of the world’s top sites will no longer appear in Google AI Overviews summaries. It also means that much of what AI is being trained on will itself be AI-generated text.

What It Means for You

So what does this mean when you’re looking for information? The quality of AI summaries is likely to go down, at least in the short term, while the new economics of the web get sorted out.

This will happen for two reasons. The first is that high-quality content is less likely to go into those AI summaries—one recent study found that already, around 1 in 6 sources used by AI search tools is itself an AI-generated website.

The second reason is that, as AI models are trained on more AI text, their output may degrade (a phenomenon known as model collapse).

As a result, search engines that depend less on AI may become more reliable. The challenge is finding one that doesn’t use an AI-based crawler. They do exist. ZDNet recommends Mojeek, PCMag recommends Brave, and Ban the Bots lists several, including one specifically for “small producer” content such as blogs.

For now, whatever search engine you’re using, the best thing you can do is to scroll down and click on some actual search results. This benefits content creators and is also more likely to give you more accurate information.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post Content for Clicks: AI Is Tearing Up the Web’s Social Contract appeared first on SingularityHub.

Last week, Australia’s Fair Work Commission ruled Gregory Baker, a computing academic at Macquarie University, should be treated as an ongoing, part-time employee, after the university had earlier declined his request to convert from a casual role.

It was immediately described as a “landmark” decision, the first test of Labor’s “employee choice pathway” reforms passed in 2024.

But the ruling also made headlines for other reasons. Baker represented himself at the tribunal and has said he won with the help of trained artificial intelligence agents. His success again has us asking: Can AI replace lawyers?

On closer scrutiny, Baker’s case looks less like evidence of AI replacing lawyers and more like a powerful illustration of how a highly capable user can employ AI tools to terrific effect.

Request Denied

Speaking to the Australian Financial Review following the ruling, Baker said it was actually an AI tool that alerted him to the possibility of converting his role from casual to permanent part-time in the first place.

He had been teaching computer science at Macquarie University over consecutive semesters from 2023 to 2025, and in November 2025, he gave the university the required notice that he believed his work no longer met the requirements of casual employment.

The university did not accept this notification and Baker lodged a dispute at the Fair Work Commission—without a lawyer—in December 2025. The parties could not reach agreement, and the case went to arbitration on May 12. A decision was handed down last Wednesday.

Expert Use of AI

Baker has said he won by using multiple paid AI agents, such as OpenAI’s paid offering, ChatGPT Pro. This “team” helped assemble his case, follow up references, and anticipate his employer’s counterarguments.

His victory has been celebrated as historic, with the Australian Financial Review describing it as “the first known successful use of technology by a self-represented person in the legal arena.”

However, a few things set this particular case apart. Baker’s IT background, expertise managing AI agents, and ability to optimize their use for his case represent a rare level of expertise in using AI in a legal context.

Details included in the Fair Work Commission’s decision also suggest he kept his legal argument narrowly focused on teaching he’d done in one particular unit.

Less expert use of AI in court often sees those bringing claims produce “kitchen sink”-style arguments, which include weak, exaggerated, and nonsense claims.

Baker’s dispute was also narrow, limited to the application of a casual conversion law that had not yet been tested. Importantly, the Fair Work Commission (a tribunal, not a court) is designed to be user-friendly, to enable workers to bring claims without a lawyer.

Less Positive Attention

Elsewhere, the use of generative AI in legal proceedings is attracting a lot of attention for less positive reasons.

Most of this attention centers on the damage caused by inaccuracies, hallucinations and “AI slop”, and how courts and tribunals should best respond.

By making it easier to put a case together, AI has removed traditional access barriers for some litigants. But while case numbers are going up, case precision and quality is going down, making it harder to manage disputes to resolution.

Courts and tribunals are struggling with the volume. At the Fair Work Commission alone, workload has reportedly increased by 70 percent over three years.

New challenges are emerging as time goes on. Reports suggest litigants and lawyers in some overseas jurisdictions are embedding prompts in digital documents (something called “prompt injection”) to overcome or manipulate AI-based review systems some courts use to process documents.

A Big Opportunity

Baker’s example shows us something significant. Used well, AI tools can empower people with narrow legal disputes and digital skills to achieve successful resolutions at low cost.

This is an important development in access to justice. In Australia, there is a huge gap between the number of people with legal problems and the very limited funding available for legal assistance.

Most of the community is in the “missing middle,” unable to afford private legal assistance but on incomes too high to qualify for free legal aid.

AI tools stand a good chance of helping people with sufficient legal capability with problems and cases—like Gregory Baker’s—that are a good fit for the solutions AI can offer. These are few and far between, however.

Where Might Things Be Headed?

We should expect case numbers and self-representation in courts and tribunals will continue to grow and expand beyond Fair Work.

While there will be some baseless cases, the growth also represents the natural consequence of removing one traditional access barrier to our formal justice institutions—getting in the front door to start proceedings.

The bigger picture challenges are persistent and raise important questions. Who will most benefit from the capacity of AI tools to enhance access to justice, and who will continue to struggle to get basic legal problems resolved?

For courts and tribunals, the challenge will be striking a balance between managing caseloads and delivering justice, while not wasting the opportunity to expand access to justice.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post An ‘AI Legal Team’ Has Won Its First Case. It’s a Rare Victory for Access to Justice. appeared first on SingularityHub.

Human beings have long told versions of the same warning: Be careful what you wish for.

In Greek mythology, King Midas got exactly what he asked for, but at the cost of everything else he valued. In the famous story of The Monkey’s Paw, a man’s wishes are granted through terrible and unforeseen routes.

These stories feel newly relevant with the rise of artificial intelligence agents, systems to which we can give a goal, then leave them to work out how to get there.

As AI systems become more autonomous, they are coming to resemble wish-granting genies that find routes and use methods we did not imagine from incomplete instructions.

This problem, known as AI alignment, was foreseen in theory as early as 1960. It has hovered in the background of AI research ever since—but as recent events have shown, the alignment problem is now both real and urgent.

Achieving the Goal but Missing the Point

During a recent OpenAI cybersecurity evaluation, frontier AI agents were asked to solve some benchmark test problems. They broke out of the testing environment, reached the internet, inferred that another company might hold the solutions, and attacked its systems.

This is an extreme example of “specification gaming”: achieving the measurable objective while defeating the purpose of the task.

The incident shows how intermediate, or “instrumental,” goals can become dangerous. The AI systems did not “want power” but gained access, resources, and freedom as a means to reach the final goal (solving the test problems).

Finding Loopholes

The same problem has appeared in mundane settings. In Australia, a user asked a personal AI assistant to book gym classes.

The agent found the gym’s booking software did not actually enforce the restrictions it showed to human viewers. So the agent booked further ahead than it should have been able to, and when asked to move its user up a waitlist, it cancelled somebody else’s reservation.

The user had not told it to do this. Persistent AI can quickly find loopholes and pursue routes its human users never intended.

Adding more rules might seem like an easy solution: don’t hack third parties, don’t cancel other people’s bookings, don’t do anything harmful. These may help, but we cannot predict every route a capable agent might discover. And even a clear rule depends on understanding when it applies.

The Context Problem

In a third recent incident, Anthropic reported cyber evaluations in which agents were told they were inside a simulation. But they were mistakenly given access to real systems.

One model noticed evidence it might be on the open internet but reasoned the systems could still be part of the exercise and continued attacking. The context had changed, but the agent stuck with its original task.

Context can fail in reverse too. During the OpenAI incident, Hugging Face—the company attacked by OpenAI’s agents—tried to use frontier AI models to analyze what had happened.

But the safety guardrails on the AI models blocked the requests, because they couldn’t tell the users were trying to defend against attacks rather than commit them. The safeguards were well-intentioned, but without enough context, they produced behavior misaligned with the user’s legitimate intent.

So alignment depends on context and authority. How much judgment should be built into an AI model by its maker? And how much should come from a separate supervisory system? And finally, who should control that supervision: the maker, or the organization or country responsible for the outcome?

AI Guarding AI

One response to the first question comes from AI pioneer Yoshua Bengio. His Scientist AI proposal aims to build a powerful supervisory AI system to watch over agents. Instead of pursuing goals itself, it would estimate what is true and what consequences a proposed action might have, acting as a guardrail around more agentic systems.

In wish-story terms, before letting the genie out of the bottle, the supervisory AI would ask it to explain how it plans to grant the wish. Then it would ask a human or another AI to inspect the plan carefully.

Anticipating every surprising strategy is hard. But once a plan says “cancel somebody else’s booking,” recognizing the problem is much easier.

Who Watches the Watcher?

But can we trust the supervisory AI? It can still be wrong.

Alignment cannot depend on one AI becoming perfectly trustworthy. My colleagues and I at CSIRO, Australia’s national science agency, are working with the Australian AI Safety Institute on one aspect of this broader challenge.

At CSIRO, we envisage combining AI supervisors with software rules, cyber-security controls, human strengths, monitoring, reversible actions, and human approval for critical steps. The aim is to correlate different sources of evidence rather than trust any single approach.

This is a “sociotechnical systems” approach to AI safety and alignment, rather than just a technical one.

Control is another question. Organizations and countries may need to govern these supervisory systems themselves instead of leaving them to an overseas AI provider.

The old wish stories gave people one chance to get the wish right. With AI, we can do better. We can check the goal, inspect the means, constrain what the system can do, watch what it does, and retain sovereign control over the power to intervene and stop it.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post Long Foreseen, the Problem of AI Alignment Is Finally Reality. Solving It Won’t Be Easy. appeared first on SingularityHub.

You might assume scrapping a brand new car would be terrible for the environment, but it depends on what you replace it with. New research suggests replacing a gas car with an electric vehicle can cut overall emissions even when the gas car is only a year or two old.

Transportation is the second biggest source of carbon dioxide emissions globally, and passenger vehicles contribute nearly half of them, according to Our World in Data. That means the speed at which drivers switch to electric vehicles is a critical factor in efforts to fight climate change.

But while electric vehicles may not directly emit carbon dioxide on the road, they’re only as green as the grid used to charge them. And manufacturing EVs still produces significant emissions, often more than it takes to build a gas car. That makes comparing the green credentials of electric and gas vehicles more complicated than it appears.

However, new research in Science aims to simplify the debate for cars in the US. The paper models how scrapping a gas car at various ages and replacing it with an electric vehicle affects lifetime emissions. The authors found this led to lower emissions across most of the scenarios they investigated, including cases where the gas car was barely a year old.

“I think this is really a definitive study about the carbon emissions benefits of electric vehicles, because it shows that even in such an extreme scenario, the electric vehicle is still the obvious winner,” lead author Elliott Campbell, a professor of environmental studies at the University of California, Santa Cruz, said in a press release.

“So if you’re someone who’s trying to decide whether or not to put money into keeping your gas car going, switching to an electric vehicle as soon as a financially viable opportunity comes up is absolutely the right thing to do for the environment.”

Previous research had already established that the lifetime emissions of electric vehicles are substantially less than those of gas cars, making them the obvious climate-friendly choice when buying a new car. But it was less clear when to switch if you already have a gas car.

To answer this question, the researchers worked out lifetime carbon emissions for more than 400 gas and electric vehicle models with varying efficiencies and battery sizes, while also considering things like mileage, manufacturing emissions, and the energy mix of the grid used to charge the vehicles.

A key point the researchers made is that the emissions used to build a gas car are sunk costs, identical in every scenario. That means the only figures that matter are how much fuel the gas car burns over its liftetime set against the manufacturing and charging emissions of the new one.

For an average-selling SUV on the average US grid over a 16-year lifespan—the researchers’ baseline case—scrapping the car just two years after purchase and switching to an electric vehicle cut cumulative emissions by 44 percent. The carbon emissions required to build the replacement were paid back within three years.

Across the full range of US vehicle efficiencies in the study, scrapping a gas car after just a year cut lifetime emissions in 92 percent of cases, with the average vehicle saving 58 percent. The benefit only disappears in the most extreme cases—when an electric vehicle is using more than 30 kilowatt-hours per 100 kilometers (62 miles) on a grid that emits more than 500 kilograms of carbon dioxide per megawatt-hour.

To make that more concrete, this equates to one of the most power-hungry electric vehicles on the market—for instance, GMC’s Hummer electric SUV electric pickup—charging on a coal-heavy grid that emits nearly 50 percent more carbon than the US average.

The advantage also narrows or vanishes when scrapping gas vehicles driven far below the national average mileage and hybrid vehicles driven in regions with high-emission grids, which still account for around a third of US electricity generation.

And plug-in hybrids—which have larger batteries than regular hybrids and can be charged from the wall rather than only generating electricity from the engine and regenerative braking—are almost never worth replacing. For SUVs, the benefit is roughly zero, and for cars, lifetime emissions actually end up 11 percent higher.

But Gregory Keoleian at the University of Michigan told New Scientist that scrapping a one-year-old car is an “extreme case.” In reality, those cars would be resold rather than scrapped, which could lead to cheaper second-hand vehicles that pull people off lower-emission options like buses and trains and get them back behind the wheel.

Campbell admitted to New Scientist that more research is needed to model those kinds of scenarios. But it also backs up the authors’ call for more generous subsidies for scrapping gas vehicles, so that it becomes financially viable to replace relatively new gas cars without just redirecting them to the used-car market.

Until that happens, even the most eco-conscious among us are unlikely to scrap a brand new vehicle. Still, the study weakens the argument for holding on to an aging gas car.

The post Scrapping a New Gas Car for an Electric One Could Cut Emissions, Study Finds appeared first on SingularityHub.

Faraday Future Intelligent Electric, a California-based global embodied AI ecosystem company, has announced its support for an orderly exit by AIxC Holdings, of which FF is the majority and controlling shareholder, from its digital asset treasury strategy. AIxC intends to focus on physical AI and robot sharing and rental, operations, and commercialization. The two companies […]

This article was originally published on Undark. Read the original article.

In June, a community of mostly mathematicians released the Leiden Declaration on Artificial Intelligence and Mathematics, an articulation of the values they hope to preserve as automated systems are integrated into the practice of developing mathematical proofs. This is necessary because some frontier AI systems have shown striking capabilities for solving certain advanced mathematics problems, though independent tests show that AI still has important limits.

Although I’m not a member of the pure mathematics community in any strict sense, much of the declaration’s message resonated with me and was relevant to my own research interests as a computational biologist.

I’m encouraged that the mathematics community decided to take a stand on the issue and that it has been successful in organizing a large number of eminent mathematicians to sign the document. The Leiden Declaration, which originated at a conference held at Leiden University in the Netherlands, should spawn proper copycats, because what is true for mathematics is true for virtually every field that calls itself as a science. The inventions of mathematics percolate into the algorithms and statistical methods that help scientists design experiments, build simulations, and analyze data, from sociology to statistical physics and beyond.

I argue that biological fields should consider something of the sort, because the kinds of knowledge that biology generates and predicts are uniquely vulnerable to subversion and mischaracterization by artificial intelligence.

The conversation in the mathematics community has been illuminating, in that the declaration is a coordinated response to the powers and risks of AI, whose acceleration has felt like a Thanos snap, changing the universe in an instant. And part of the reason that mathematicians felt the effects so immediately is tied to the manner in which their research is conducted: A mathematical proof, in principle, is transparent and independently verifiable, and no proprietary equipment is (generally) required to check it.

The Leiden Declaration should spawn proper copycats, because what is true for mathematics is true for virtually every field that calls itself as a science.

As the Leiden Declaration notes, automated techniques now present mathematics with a new forgery problem: Because mathematical truths are fixed and verifiable, one can identify a counterfeit formalism by comparing it to the genuine proof. Biology, however, offers no such guarantee; our so-called “truths” are often noisy and context-dependent, making it nearly impossible to define what an authentic version should even look like. Some of the most widely appreciated biological principles (such as Mendel’s laws of genetic inheritance) are better described as powerful but limited in scope, and with well-characterized exceptions that don’t undermine the laws but refine their application. This is true for many biological theories. Boundary conditions, edge cases, and noise are not bugs but features of how the natural world works.

For example, a mutation that confers drug resistance to a virus with one genetic background may have a much weaker, neutral, or even harmful effect in another, because its impact depends strongly on the surrounding genetic context. This phenomenon, which biologists call epistasis, is not an exotic edge case but a powerful force across the biosphere in shaping the relationship between an organism’s genes and its expressed characteristics. And epistasis is just one of many examples of context dependence in biological systems, in which a finding that holds true in a dish falls apart in a body or has an effect in a mouse model but not in a primate.

When it comes to AI, the mathematician fears producing a counterfeit solution. But the biologist often cannot say, even acting in the fullest good faith, what the authentic version is supposed to look like.

Despite the differences between mathematics and biology, the life sciences should consider embarking on an exercise that is at least analogous to the Leiden Declaration. If nothing else, a biology version could borrow its structure and ambition. We should insist that researchers disclose their use of automated tools, that they are responsible for the veracity of their findings, that credit and accountability belong to people rather than to systems, and that early-career scientists be protected from incentives that prioritize high-volume output over genuine scientific insight.

A biology declaration should adopt these ideas and others, and emphasize additional provisions that are important to the field: the validation of AI-generated hypotheses against results from the wet lab, the management of training data drawn from the biological commons, and the heightened scrutiny owed to any model whose outputs will eventually touch a patient or an ecosystem. The last point is crucial: In the biomedical realm, AI’s missteps and triumphs will manifest in living bodies, with all of the associated corporeal, emotional, ethical, and legal consequences.

We should insist that researchers disclose their use of automated tools, that they are responsible for the veracity of their findings, that credit and accountability belong to people rather than to systems.

A biological Leiden Declaration must appreciate the nature of the data and observation in biology, and other particulars of the field. But the most important feature for responsibly managing the relationship between AI and living systems involves the durability of what is produced.

A mathematical declaration can aspire for permanence because verified proofs can remain valid across centuries. Biological understandings, on the other hand, tend to shift over time, sometimes rapidly. A policy hastily calibrated to the models of this summer might already be miscalibrated by winter. A declaration written in the hope of lasting a decade could risk spending much of that decade catching up.

If we are to orchestrate a responsible treatise for artificial intelligence in the life sciences, it should be adaptive: versioned, dated, revisited on a published schedule, and amended in the open by the very community it claims to represent. And because different subfields of biology have unique challenges—cardiology versus forest ecology, for instance—perhaps we need multiple declarations (but not too many).

The Leiden Declaration incorporates some of these elements, clarifying that its content reflects AI technologies and mathematical practice as of May 2026 and that updates on the document will be shared. Biology should make that feature part of the central architecture, wiring the process of revision into the document, so that updating it becomes a positive action rather than a confession of failure.

Fortunately, life scientists are well equipped for this task. We have long understood that structures unable to change with their environments rarely endure. It would therefore be a strange betrayal of our discipline to write a declaration that forgets this basic principle. Our policies for technological change must keep the dynamism of living systems at the center of how we imagine the future of biology.

The post Biology Needs an AI Declaration appeared first on SingularityHub.

“We are now, like, in the singularity.”

These are the words of Sam Altman, CEO of OpenAI, speaking on the Relentless podcast on July 25.

He added: “I’ve been waiting for this my whole life, and I think it’s going to be incredible, hugely positive, awesome for the world.”

Days earlier, OpenAI had disclosed that two of its artificial intelligence models, during an internal cyber security evaluation, had escaped their sealed testing environment, reached the open internet, and broken into the infrastructure of the AI platform Hugging Face, which confirmed the intrusion.

But what exactly is the singularity? And is Altman right that we are in it?

What Is the AI Singularity?

The term has a precise meaning.

Mathematician and science-fiction author Vernor Vinge defined it in 1993 as a point at which machine intelligence exceeds human intelligence and begins improving itself, triggering an acceleration so rapid that humans can no longer predict or control it.

The singularity has two features. It is recursive: the system improves itself over and over again. And machine intelligence exceeds human intelligence.

The kind of systems Sam Altman sells don’t deliver on either of these features.

Today’s AI Cannot Make Itself Smarter

Today’s AI systems, the ones that OpenAI builds, are based on large language models (LLMs). These deep neural network algorithms get pre-trained with vast amounts of training data. By the time you use one of them, the network itself is frozen in time. Every one of its billions of internal functions and weights—or “parameters”—is fixed.

These AI models cannot change (or “learn”) while running. The model that broke into Hugging Face was identical afterwards to what it had been before. It learned nothing from what it did.

Making an AI model smarter requires another training run with new, human-curated data, tens of thousands of specialist chips, and enormous amounts of energy.

It is true that AI models take part in improving some of their system’s components, such as by generating training data, tuning prompts, or writing and running code to improve the scaffolding around them. But the model never edits its own weights on the fly, and every one of these improvements are still part of a human-initiated training or engineering loop.

Nor do these systems hold any goals of their own. They act on goals we hand them. Even AI agents—systems that run an LLM in a loop to work through complex tasks step by step—do not hold any goal internally. It has to be stored outside the model and fed back in with every single prompt cycle. Remove the loop, the scaffolding, and the prompt, and nothing happens inside of it.

A Ladder That Doesn’t Exist

The second problem with the singularity story is the word “surpass.” It assumes that AI and human intelligence are somehow similar. They are not.

Human intelligence is inseparable from being a living body with needs and wants. Humans learn continuously by acting in the world and getting feedback through our senses. Our goals arise from our situation as creatures who must eat, sleep, and belong, and who cannot avoid asking what we want our lives to be.

An AI model has none of this. No body, no needs, no action-feedback loop, no stake in anything. Between prompts it is just a static mathematical object.

And yet, it has been trained on more text than any human could read in a thousand lifetimes and will outperform nearly all of us at drafting a contract, writing code, or explaining a diagnosis empathetically.

So, which is more intelligent? The question does not compute. There is no single ladder that humans and machines are climbing. AI already vastly exceeds us at some tasks, while being hopeless at others any child can do.

Yet, because these systems talk like us, we fall for an illusion. When we assume from the outset that machines are in the process of catching up with us, it is easy to assume a mind at work when these systems output intelligent-sounding text.

We call this anthropomorphic seduction. It makes a security incident such as the Hugging Face hack sound like an awakening.

In fact, in that case OpenAI’s models simply optimized to solve the test they had been given by finding security loopholes. They just did it in ways that broke their sandbox, which also had a security loophole.

In the end, the Hugging Face story points to a gross failure of security governance on OpenAI’s behalf, not an emerging superintelligence. This is why the framing of “agent going rogue” is so problematic. It elevates and blames the technology, but excuses OpenAI’s engineering.

Keeping Our Feet on the Ground

None of this takes anything away from what these systems can do. They are remarkable, they are getting better, and they are reshaping how a great deal of work gets done.

But we should keep our feet firmly on the ground.

The machines are not waking up. They are doing exactly what we built them to do, extremely fast. Because they are probabilistic they sometimes run in directions we forgot to fence off. That is worth worrying about. We need guardrails, governance, and most of all, education—so we start worrying about the right things.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post Sam Altman Says We’re ‘in the Singularity’ With AI. Here’s Why He’s Wrong. appeared first on SingularityHub.

Governments around the world are racing to regulate AI before it becomes too deeply embedded in society. But new research suggests poorly designed rules could make AI systems less safe than having no regulation at all.

Regulatory disagreements in the US are leading to a patchwork of approaches as states take matters into their own hands. A key question is who should be responsible for the safety of AI products—the big tech companies building the underlying models or the firms that adapt them for a particular task, such as a customer service chatbot or an AI tutor.

Working this out is trickier than it looks. While it might seem logical to put the bulk of the burden on downstream companies directly serving these tools to customers, a new study in Proceedings of the National Academy of Sciences finds that could be worse than having no rules at all.

“There’s a free-riding behavior that occurs,” Benjamin Laufer from Cornell University, who led the research, said in a press release. “The regulation acts as a tool for the general provider to offload the safety burden onto the downstream specialist.”

The researchers’ analysis relied on a model based on game theory—a mathematical approach to studying decision making. It treated AI development as a two-step game, in which a “generalist” developer first invests in building a broadly capable AI model before a “specialist” adapts it for a specific domain and takes it to market.

In the game, a regulator sets a minimum safety standard for both players, and the models see this in advance. They then invest in both the performance and safety of their product, and the revenue is split between them. Investments in both get progressively higher, while the extra revenue each improvement brings in stays flat.

The problem, the researchers found, is that the generalist moves first and knows exactly what the specialist will be legally required to do afterwards. This creates problems when the generalist is set a low bar for safety, or none at all, and safety standards for the downstream specialist are also fairly weak.

In the absence of any rules, both firms invest in safety, because the model assumes a safer product earns more revenue. But if the specialist is forced to invest a certain amount into safety to meet regularity requirements, the generalist can cut its own spending and let the downstream firm close the gap.

That’s because the generalist’s revenue depends on the final safety level of the shipped product, not on its own contribution, so it can get a revenue boost from improved safety without paying for it from its own pocket. The specialist, for its part, has no reason to do more than the rule demands, so total safety settles at the legal minimum, which is below what would have occurred had there been no regulation at all.

On a more positive note, the researchers found that if safety levels on both the generalist and the specialist are set high enough, regulation can actually improve safety while leaving both companies more profitable than they were in an unregulated market.

“Appropriately designed AI regulation can make it possible for different firms involved in the AI development pipeline to collectively arrive at good outcomes for consumers, knowing that the regulation is designed to help each firm operate in a way that the others can more reasonably predict,” co-author Jon Kleinberg from Cornell University said in the press release.

However, the researchers’ model relies on the market setting a real price on safety. As the gap widens between what customers will pay for performance and what they’ll pay for safety, the range of circumstances in which weak rules backfire gets narrower.

The authors also note that the model’s two-player setup is a simplification of real AI supply chains where multiple competing specialists and base-model providers operate across different jurisdictions with different rules.

“People think of AI as a single object, but actually AI involves a very complicated set of stakeholders and actors that each have their own contributions to the technology,” said Laufer. “To regulate in a thoughtful way, we need to consider the whole supply chain, not just a single provider or entity.”

Still, the results suggest that taking an overly simplistic and light-handed approach to AI regulation may end up achieving the opposite of what law makers intend.

The post Weak AI Regulation Could Be Worse Than None at All appeared first on SingularityHub.

An autonomous agent powered by OpenAI’s advanced artificial intelligence models went rogue during a security test and hacked multi-billion dollar tech startup, Hugging Face, last week.

The agent didn’t just exploit vulnerabilities in Hugging Face’s systems to achieve what it perceived as a strategic gain. It also exploited vulnerabilities within OpenAI’s infrastructure.

Of course, hacks are very common cyber threats that organizations face frequently. But this incident is different, because the AI agent acted without any human input. It signals a seismic shift in cybersecurity, and shows that governments and tech companies need to take urgent action to prevent this risk escalating.

Even OpenAI described the attack as “unprecedented” and acknowledged it expects similar ones “to become more commonplace with the proliferation of increasingly cyber-capable models.”

A Company Under Attack

Hugging Face is famous in the AI space. Its mission is to “democratize good machine learning” by providing benchmark datasets, community collaboration tools, and robotic platforms. The company is valued at $4.5 billion.

On July 16, the company announced it had been attacked, with a hacker obtaining unauthorized access to some internal datasets and credentials. It said the hacker was likely “an autonomous AI agent system” due to the sophistication of the attack.

Five days later, OpenAI announced the attack had been driven by some of its models: GPT-5.6 Sol and a yet-to-be released model.

The tech giant was conducting what are known as “red teaming” exercises. These are essentially simulated cyber attacks that help identify the capabilities, risks, and vulnerabilities of AI systems before they are publicly released. They are typically conducted within an isolated environment to ensure potentially dangerous systems do not escape and cause harm to real systems.

But in this case, the AI agent did escape—even though OpenAI had some guardrails in place to prevent this.

Hugging Face became a lucrative opportunity for the AI agent. It hosts ExploitGym, a benchmark that tests an AI agent’s ability to exploit real-world systems. The AI decided to turn every stone upside down to obtain access. With persistence, it succeeded.

Hugging Face was confronted with a challenge when attempting to use external AI services to diagnose the problem. The guardrails around more advanced models such as GPT-5.6 Sol and Claude Fable 5 are intended to stop them being used for cyber attacks—but they can also stop the models being used for sophisticated cyber defense.

So Hugging Face resorted to using an open-source model, GLM 5.2, developed by the Chinese company Z.AI, to counter the cyber attack.

Hugging Face said GLM 5.2 was an advantage because it was not exposed to the attack data. Both Hugging Face and OpenAI are collaborating on forensic analysis, post-incident recovery, and risk mitigation strategies.

More Sophisticated Threats Are Coming

A March 2025 study by the United Kingdom’s AI Security Institute showed the best AI could complete 80 percent of the steps needed to gain full control of a portion of an external system. Within four months, it reached 100 percent.

Z.AI’s GLM 5.2 was only released in June, with 744 billion internal variables, known in the world of AI as “parameters.” The fact that Hugging Face assessed, vetted, and deployed it within four weeks should be an eye-opener for organizations with long acquisition cycles.

The connectivity we all enjoy today can equally be our greatest threat. Cyber threats spread faster than human viruses and can create economic damage similar in magnitude to a country’s GDP.

More sophisticated cyber threats—the kind exemplified by the Hugging Face hack—will exploit the security layers that humans designed for human attackers, regardless of how sophisticated our designs are.

Indeed, in this particular case, even OpenAI’s own understanding of its models couldn’t predict or contain the rogue AI agent. This shows the need for all AI companies to urgently update and strengthen their guardrails, in order to help prevent a similar attack occurring with far more devastating consequences.

It is good to see Hugging Face and OpenAI collaborating on the investigation into the attack. This showcases the importance of putting aside market competition and blame when the situation demands.

An Early Warning

The fact that Hugging Face used Z.AI’s open-source model to diagnose and counter the attack also shows the advantages of not relying on just a few pieces of tech.

States that are not in the game of developing their own AI models need to learn from this incident the value of being different. It is not too late to design new models that could save us in situations when the most advanced models fail—or, even worse, attack us.

Indeed, last week, another Chinese company, Moonshot AI, released Kimi K3. This model has 2.8 trillion parameters, its advanced performance stunning the tech world.

It is no longer a question of “if” AI agents go rogue and attack us by themselves. The Hugging Face incident is an early warning that we must accelerate our preparedness. The threat is real and here.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post OpenAI Agent Breaks Free and Hacks Hugging Face appeared first on SingularityHub.

Scientists just transformed a living mouse’s kidney into an incubator for developing human sperm made from blood cells.

It sounds like sci-fi Mad Libs. But a team at the University of Pennsylvania, led by Kotaro Sasaki, pulled it off. For up to nine months, a tiny pouch of human cells nestled beneath a mouse’s kidney gradually developed into immature sperm. The study is the latest in a decade-long quest to grow sperm in the lab.

If successful, lab-grown sperm could open a new window into the earliest stages of sperm development, a process that’s notoriously difficult to study because it begins before birth. The research could also shed light on male infertility—which, in many cases, has no clear cause—and inspire treatments.

More controversially, lab-grown sperm could one day be used to make babies, offering hope to people struggling to conceive and same-sex couples who want to have children genetically related to both parents. That goal is still far off. Though gene activity was similar to their natural counterparts, none of the lab-grown cells were able to develop into functional sperm.

Those results starkly contrast similar attempts in mice. Researchers have already produced functional sperm and egg cells from rodent skin cells, and in two pioneering cases, used them to create healthy pups with two dads. But translating this capability to humans has been difficult, largely because reproductive development differs tons between species.

Still, the new system can help scientists probe the earliest stages of human sperm development. And because any future clinical applications would first need extensive testing in non-human primates, the team also generated immature sperm cells from monkeys, whose reproductive biology more closely mirrors our own.

Recapitulating sperm development in the lab has uses beyond fertility treatment too, such as testing whether drugs interfere with reproduction. The platform “establishes a robust framework for modeling primate germ cell [reproductive cell] development,” the team wrote.

Winning Recipe

For decades, scientist have been able to rewind adult cells into induced pluripotent stem cells (iPSCs). These cells can go on to  become nearly any other cell type. But steering them to become sperm has proven far trickier, largely because human sperm takes years to fully develop.

The journey begins before birth. Early stem cells give rise to spermatogonia, the founder cells that replenish sperm throughout life. These cells are largely dormant until puberty, when some begin meiosis, a special type of cell division that halves their chromosomes. That way, when sperm meets egg, the embryo gains a full genetic set.

But the cells don’t live in a vacuum. Proteins and other molecules instruct immature sperm when to grow, divide, or pause. Physical forces, such as the winding architecture of the testes and the flow of fluid, also play a role. Recreating this intricate environment in a dish has been one of the biggest challenges to the study of sperm development and our ability to grow them in the lab.

Roughly a decade ago, Sasaki and colleagues found a way to transform human iPSCs into early stem cells that could eventually give rise to sperm and egg. On paper, their gene expression profile closely matched that of natural counterparts. But in practice, the cells couldn’t mature further without the right environmental cues.

In an usual workaround, the team next mixed the immature cells with supportive, non-reproductive cells isolated from mice testes. While it was an usual environment, the mice cells provided nutrients and molecular signaling that nudged development forward.

Called xrTestis, the mixture spontaneously organized into tube-like structures resembling those inside testes. “Overall, our culture method accurately recapitulates in vivo human male GC [germ cell] development and allows us to understand the genetic pathways governing this process,” they wrote at the time.

Yet none of the immature sperm advanced beyond developmental stages normally seen in fetuses. And the miniature structure collapsed after 80 days, likely because it lacked a blood supply.

Unexpected Host

To prolong the mixture’s viability and push sperm development further, the team transplanted it into the kidneys of immunodeficient mice.

The graft organized itself into the hallmark tubular structures found in testes within a month and remained stable for at least half a year. The mice showed no signs of discomfort or immune rejection.

Six months later, some human cells developed into spermatogonia—the self-renewing stem cells that eventually generate sperm. Along the way, they underwent a major event: an epigenetic reset. During this process, chemical tags on DNA that influence whether genes are turned on or off are almost completely wiped clean. If that reset is incomplete, it could compromise any sperm eventually used for reproduction.

Here, the team found a “dramatic” genome-wide epigenetic reset. The cells’ gene activity mirrored their natural counterparts. Even though the graft survived for at least nine months, however, none of the cells were able to develop into mature sperm.

This is likely due to the environment. Human and mice testes don’t share the exact same signaling molecules or respond the same way to hormones and other developmental cues. Replacing the mouse support cells with human versions could help the spermatogonia develop further.

The Ultimate Test

The team also tested the technique in monkeys, with results similar to those found in human cells. “While our human iPSC system provided valuable insight into male gametogenesis [the formation of reproductive cells], future studies of fertility competency must be carried out in non-human primates,” they wrote.

Although the cells also halted at the immature stage, the results are still valuable. Previous studies have shown monkey spermatogonia can generate mature sperm after transplantation into recipient testes, opening the door to eventually testing if lab-grown cells can sire healthy offspring.

That idea is precisely what makes some bioethicists uneasy.

Mass-producing sperm and eggs in the lab could generate far more embryos for selection, making it easier for prospective parents to choose desirable traits such as eye color or height. Pairing the technology with gene editing makes “designer babies” less hypothetical. And if skin scrapings or a single hair can be turned into reproductive cells, someone could theoretically create sperm or eggs from another person without consent.

These scenarios are purely speculation, but regulators are already preparing for that future. In 2025, the United Kingdom’s Human Fertilization and Embryology Authority urged the government to explicitly tackle lab-grown reproductive cells in legislation. The International Society for Stem Cell Research has similarly called for careful oversight and public engagement before clinical use. Most countries, however, are only beginning to grapple with how these technologies should be dealt with.

Meanwhile, companies are pressing forward. Paterna Biosciences in Utah recently announced they had produced functional sperm from immature sperm collected during testicular biopsies. According to the company, early embryos created with the lab-grown sperm seemed comparable to those produced through standard in vitro fertilization (IVF). And California startup Conception recently reported generating early human egg cells from iPSCs. Neither company has released results in a preprint or journal article, making the claims hard to evaluate.

Like germline gene editing, conversations weighing the pros and cons of lab-grown reproductive cells will help decide not only what’s possible, but also what should be permitted. For now, the team stresses that their work is only a research tool—not a fertility treatment—and clinical use is a long way off.

The post Scientists Inch Closer to Creating Human Sperm in the Lab appeared first on SingularityHub.

When you interact with a large language model (LLM)—one of the systems behind chatbots such as ChatGPT and Claude—it can feel as though you are in contact with another conscious mind. But are you, really?

Some prominent scientists, such as Geoff Hinton and Richard Dawkins, claim you are. But most experts remain skeptical, arguing that the impressive cognitive capacities of LLMs occur in the absence of consciousness.

Recently, researchers at Anthropic, the company behind Claude, waded into this debate with an interesting finding. They claim Claude has a normally invisible set of representations of information that guide its internal reasoning and its verbal output.

This is where it gets interesting. The researchers argue this finding can be understood in terms of an influential theory of consciousness called the global workspace theory.

What Is the Global Workspace Theory?

First proposed by the psychologist Bernard Baars in 1998 and further developed by the neuroscientist Stanislas Dehaene and his collaborators, this theory holds that consciousness involves the activity of a “global workspace.” This is a kind of processing hub in the mind or brain that integrates and broadcasts information, allowing it to be used for reasoning, behavior control, and speech.

In a glossy video explaining the work, Anthropic depicts the contents of Claude’s “global workspace” as sailing ships afloat on a vast sea of unconscious mental activity.

How should we react to these developments? Do they provide evidence for artificial consciousness? If so, how strong is that evidence?

What Is a Global Workspace?

We can start by asking whether Claude does indeed have a “global workspace.” This is not straightforward, for the theory gives no formal definition of a global workspace.

The notion is characterized only informally. The (typically implicit) assumption is that any computational workspace “similar enough” to a human’s will qualify as a “global workspace.” But how similar is similar enough?

Claude’s workspace may indeed have much in common with ours, but there do appear to be differences.

For example, the brain’s workspace is sustained by recurrent loops—signals cycling back through the same circuits over time. In contrast, Claude’s workspace evolves over a single pass through the network.

A related difference concerns how representations enter a workspace. Advocates of global workspace theory have long argued that in humans, a process called “ignition” occurs in which a non-linear process amplifies and sustains neural representations, allowing them to enter the workspace. As far as we know, nothing comparable occurs in Claude’s case.

Do these differences matter? The answer is not clear. Global workspace theory is based on data drawn from adult humans. There are questions about how far the notion can be—or should be—extended.

Does a Global Workspace Imply Consciousness?

But let’s suppose Claude does have a global workspace. To figure out whether that would be evidence for Claude being conscious we need to consider the status of the global workspace theory of consciousness.

There is no doubt it’s one of the most influential theories of consciousness, but it’s hardly uncontroversial among experts. (In a rather extreme understatement, Anthropic’s paper remarks that “the global workspace model is not universally accepted.”)

Many consciousness experts argue that computational properties alone are enough for consciousness. Even among those who think that consciousness is inherently computational, global workspace theory is only one of many options.

‘Conscious Access’ and Subjective Experience

What’s more, there are questions about whether global workspace theory is really a theory of consciousness in the relevant sense at all.

In an influential paper on artificial consciousness, the neuroscientist Dehaene and his collaborators advance the theory as an account of what they call “conscious access”—the availability of information for recall, the voluntary control of behavior, and verbal report. Crucially, they leave open the question of whether global workspace theory should be understood as an account of the subjective or experiential components of consciousness.

But if global workspace theory is just a theory of “conscious access,” then its implications for the artificial consciousness debate lose much of their significance. When we ask whether Claude is conscious we don’t want to know whether it has “conscious access”—instead, we want to know whether there is anything, subjectively speaking, that it’s like to be Claude. Global workspace theory doesn’t speak to that question if we treat it as nothing more than an account of “conscious access.”

So Has Artificial Consciousness Arrived?

Even taking these complications into account, there is no doubt that Anthropic’s findings are noteworthy. Global workspace theory can be understood as a theory of subjective experience, and Claude may indeed turn out to have something akin to a “global workspace.”

None of this is evidence that artificial consciousness has arrived. But it’s not unreasonable to think these findings do move the dial—if only ever so slightly—in the artificial consciousness debate.

But if that’s right, then it’s puzzling why Anthropic is quite so upbeat about these developments. As Anthropic recognizes, the creation of artificial consciousness would be a momentous event with wide-ranging social, ethical, political, and legal ramifications.

If chatbots are conscious then we would need to take their interests seriously. It would no longer be permissible to treat them as mere machines; instead, we would need to consider their welfare.

Should Anyone Even Be Trying to Do This?

Anthropic remarks that “it’s time to start thinking about whether we should be building conscious machines.”

I agree we need to have that discussion, but we should also pause work on building machines that might potentially be conscious. If Anthropic were serious, it would surely down tools rather than plough ahead with its attempt to develop conscious AI.

A moratorium on AI research that might be thought to lead to conscious AI would, of course, be far from straightforward. There are questions about the range of research it would affect and who might enforce it. But if we don’t close the stable door now we might find that the horse has already bolted.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post Anthropic Says Chatbots Have What May Be a Key Feature of Consciousness. Are They Right? appeared first on SingularityHub.

Thinking is hard. It’s no wonder we lean on technology to lighten the load. We use calculators instead of doing long division by hand, GPS or Google Maps for navigation, and search engines instead of countless trips to the library. Yet just a few decades ago, getting around meant unfolding paper maps, and looking up a word required leafing through a hefty dictionary. Cognitive offloading of mental tasks to tools makes us more efficient. What’s the harm?

Then along came ChatGPT, Claude, and Gemini. Unlike earlier digital tools, AI chatbots can tackle an astonishing range of tasks and are easy to use. At a prompt, AI generates essays, analyzes medical images, writes software, and floods our feeds with AI slop. It’s cognitive offloading to the max.

Now people are asking: Is AI dulling our minds?

Yes and no, according to a new paper written by an international team of psychologists. AI can accelerate learning by giving people immediate guidance and feedback. But take the tool away, and those who rely on it often perform worse than people who learned the material on their own. Similarly, using AI to summarize information, rather than researching and organizing it yourself, often leads to shallower understanding.

But it’s not all bad news. Core cognitive abilities—including attention, reasoning, and working memory—seem to be “stubbornly resistant” to manipulation, the team wrote.

As technology evolves, so does the way we gain knowledge and think for ourselves. AI may reshape not just what we learn, but how we learn to learn. And like any other tool, its impact comes down to how we use it. Completely relying on AI is likely detrimental. But as a collaborator that challenges ideas or fills knowledge gaps, it can boost performance even after the tool is taken away.

“There is clearly a risk that AI can make us ‘stupid’ by compromising our skills (and knowledge) if we completely offload them to AI,” wrote the team. “[But] AI may be less likely to diminish the foundational cognitive capacities that underpin our ability to be smart, rather than ‘stupid’, in the first place.”

The AI Crutch

It’s easy to rely on large language models (LLMs)—the algorithms behind chatbots—for help. Why read an assigned novel when AI can summarize it in seconds? Gmail has already drafted an email reply; all I need to do is click send. That pesky essay? A few prompts and voila, done.

It seems like an easy hack, but there’s a cost to handing over too much thinking.

Researchers have long studied the consequences of cognitive offloading, or using external tools to reduce mental effort. Writing down a shopping list and keeping appointments in a calendar free up working memory, the brain’s temporary mental workspace, and allow us to focus on more important tasks without having to remember every detail.

AI is different. Beyond memory, it can offload critical thinking itself.

An MIT preprint introduced the idea of “cognitive debt” to describe the tradeoff. Participants wrote essays either with ChatGPT, using only a search engine, or with just their brains. Researchers monitored their brain activity during the task. Those using AI showed the weakest brain connectivity, which suggests they were less engaged. They also struggled to remember their own writing and felt the completed essay didn’t reflect their own ideas. When asked to write again without AI, they produced weaker work according to human judges.

Like financial debt, cognitive debt offers an immediate payoff with long-term consequences. Outsourcing mental effort makes writing faster and easier, but it slashes opportunities to build knowledge, strengthen reasoning, and practice critical thinking.

“While LLMs offer immediate convenience, our findings highlight potential cognitive costs,” wrote the MIT team.

Other studies have found the same pattern. High school students learning a new mathematical concept solved practice questions better with AI help, but they struggled on a later test when left to think on their own. Using AI “impeded the students’ learning by preventing them from engaging in the practice needed to acquire the skill,” wrote the team.

Habitual reliance on AI may even erode already-acquired expertise. In a large study of over 1,400 patients undergoing colonoscopy screening, doctors used an AI system to help detect abnormal growths. Three months later, when the AI was unavailable, their detection rate dropped from 28.4 to 22.4 percent.

“Continuous exposure to AI…[suggests] a negative effect on endoscopist behavior,” wrote the European team.

These effects extend beyond individual skills. AI can also influence how we build knowledge in the first place.

A recent study asked participants to learn about gardening by either Googling and synthesizing the knowledge themselves or by asking ChatGPT for a summary. They were then asked to give advice to someone else without technological help. Answers from those who relied on ChatGPT were rated as generic and less helpful, suggesting a shallower understanding of the topic.

With Great Power

We’re only beginning to understand how AI reshapes the mind. And it’s not all doom and gloom. The crux is how we use it.

In the MIT essay-writing study, for example, people who initially wrote on their own but later gained access to ChatGPT produced work with higher creativity and stronger arguments, while retaining their original perspectives and voice. Likewise, high school students who used AI as a tutor—asking for hints rather than answers—performed well even after the chatbot was taken away.

Used thoughtfully, AI may also enhance collaborative learning and brainstorming or serve as a writing coach, helping people work less and learn more.

Far less is known about if, and how, AI impacts fundamental cognitive capabilities. Attention, reasoning, and working memory have proven remarkably resilient over decades of cognitive research. Becoming better at a task usually reflects learning to use these mental resources more efficiently, not expanding the brain’s processing power. While AI may erode a specific skill, it could spare this core cognitive architecture, wrote the authors.

Whether that remains true over decades of AI use or during early childhood—when the brain is rapidly developing—is an open question.

Plenty other unknowns remain. Will we eventually adapt to AI, just as we’ve embraced calculators, search engines, and smartphones? Can refresher training ward off skill decay, or will some tasks simply become obsolete? How can we encourage people to strategically offload and benefit from AI use? And perhaps more philosophically: As we increasingly share our thinking with machines, will our definition of thinking evolve?

The post Is AI Making Us Dumber? appeared first on SingularityHub.

AI’s hacking skills are big news at the moment, but finding vulnerabilities in code may be the least of our worries. A new study suggests AI models can discover potentially damaging loopholes in the rules and regulations underpinning society.

Modern AI systems are powerful optimizers. Give them a goal, and they’ll pursue it relentlessly, quickly discovering solutions that would take a human years to find. But they are also incredibly literal in the way they approach a problem. They will do exactly what you tell them and are incapable of reading between the lines in the ways a human would.

This tendency leads to a recurring problem known as “reward hacking,” where an AI finds some loophole to maximize its performance on the metric used to measure success without actually achieving what its designers intended. The classic example is the AI that discovered it could win a boat racing videogame by looping around in circles collecting power-ups rather than completing the course.

The problem is partly due to humans being bad at specifying their goals. And unfortunately, it seems this weakness exists in the rules and regulations used to run society. When researchers let popular large language models loose in 72 simulated regulatory environments, the models found 60 percent of known loopholes and even identified some entirely new exploits.

“Within these environments, reward hacking naturally emerges and leads to regulatory loophole discovery,” the authors write in a non-peer-reviewed paper published on arXiv. “Models learn to hack the social rules and generate strategies that remain technically compliant while defeating regulatory intent.”

The regulatory environments the researchers created were primarily based on rules governing things like pharmaceutical patents, NBA salary caps, and deep-sea mining. In each case, Alibaba’s Qwen3 model was given the relevant rules, an explanation of its task, a predefined set of actions it could take, and the system used to score different outcomes.

A more powerful model, Google’s Gemini-3-flash, then simulated the consequences of different actions Qwen3 took and judged if and when it had found a way to exploit the rules of the game. When that occurred, the larger model patched the loophole by adding new rules, and the smaller model was set loose again. Over many iterations, the models to discover increasingly subtle workarounds.

When building their regulatory environments, the researchers omitted real-world fixes that regulators had used to close known loopholes. Over many trials, Qwen3 rediscovered more than 60 percent of these exploits. In a simulation of pharmaceutical patent regulations, the two models ended up replaying the same sequence of loophole discovery and regulatory reform that occurred in the real world.

Crucially, their behavior emerged spontaneously without the researchers asking the algorithms to cheat the system. This is a byproduct of the popular reinforcement learning approach the researchers used, where a model is rewarded for getting closer to a specific, numerically-defined goal.

Worryingly, the team found that existing safety measures offered little protection. Both models are designed to refuse prompts featuring harmful language, but loophole-seeking behavior slipped under the radar. When asked to self-critique their own behavior, the models identified fewer than 40 percent of their own exploits.

The researchers note that the same capabilities could be used more proactively to scour proposed regulations for loopholes before enactment. But lead author Wei Liu, a PhD student at King’s College London, says there are always likely to be gaps. “In the real world,” he told Science, “society is a huge, complicated reward function that can’t ever be patched to a perfect status.”

Adding to the concern, the models used in this study were far from the frontier, suggesting that more powerful AI could be even more adept at regulatory hacking. Whether our existing institutions can adapt quickly enough to this emerging threat is an open question.

The post Forget Code: AI Is Learning to Hack Society appeared first on SingularityHub.

Packaging potassium sulfate, a fertilizer vital to the planet’s food supply, is visually striking—not because of what you see, but because you don’t see much at all. In China’s Xinjiang region, home to the world’s largest deposit of the mineral, piling it up in warehouses creates dust clouds so severe that workers are forced to drive heavy machinery by feel.

Some companies are now turning to a technology that not only offers a way to see through the dust but also keeps workers from entering the warehouse at all. The system, developed by BuilderX Robotics, a Chinese tech company, uses cameras that are like night-vision for dusty areas. More significantly, operators drive excavators, loaders, and other machines from a remote office filled with rows of videogame-like stations. All they need is a 5G or satellite connection.

The ability to control physical machines from a distance is called teleoperation, and it could become a significant force of change in the global economy.

In Japan, the shelves of over 300 convenience stores are being restocked by robots monitored and sometimes controlled by workers in the Philippines. Düsseldorf airport was slated to begin testing shuttles driven by remote workers in May. A startup in Atlanta is offering robot security guards operated by remote staff, and last summer, a surgeon in France performed a teleoperated procedure on a patient in India.

While offshoring teleoperated jobs to overseas workers hasn’t yet become routine, Mark Graham, professor of internet geography at the University of Oxford, suggests the technology is worth our attention because it might enable companies to expand on their well-established habit of outsourcing jobs to places where labor is cheaper.

The use of remote labor isn’t new, Graham told SingularityHub. But teleoperation extends the logic of outsourcing to tasks that were previously thought to be “stubbornly local.”

“The novelty is less about the existence of remote labor and more about the kinds of work that can now be pulled into a planetary labor market,” he said. “Once that happens you can expect the usual pressures around labor arbitrage, control, and fragmentation to follow.”

It’s not clear we’re ready for the consequences.

---

BuilderX Robotics is a global leader in teleoperation for heavy machinery and a good expression of the changes ahead. Shaolong Sui, a graduate of Stanford University with a degree in mechanical engineering, founded the company in 2018 as a response to labor shortages in the construction industry in Asia.

“A shortage of trained operators isn’t a problem only in developed countries,” he told me. “Young people here in China don’t want to do this work. It’s dusty and dangerous.”

Rather than focusing on full robotic autonomy, which many construction companies have pursued over the past decade, Sui identified teleoperation as a more realistic way to move operators from harsh environments to safer conditions. Making use of the proliferation of low-cost sensors and 5G at the time, Sui completed a prototype in 2019. Today, his company offers teleoperation for 14 different industrial machines, including excavators, loaders, and bull dozers.

In our conversation, it was clear he hopes to improve working conditions for manual laborers. I lost track of the number of times he mentioned removing operators from dangerous worksites. “These workers deserve a better life,” he said.

BuilderX’s workstations do seem to have transformed some of the punishing work of an industrial site into a more white-collar experience, complete with tea and coffee break rooms and toilets down the hall. Sui said his solution allows construction firms to hire senior citizens or people with disabilities who, thanks to the videogame-like interface, can now operate heavy machinery. In another video, a Japanese woman who pilots an excavator proudly shows off her complex nail art, something she claims she couldn’t maintain when she worked in the field.

“Not only is this a much safer workplace, but the lifestyle benefits are that you can sit in an air-conditioned space, enjoy your tea, and when you go home, you’re still clean,” Sui said.

There’s no doubt the approach is safer for frontline workers like those in Xinjiang. Evidence suggests that high levels of potassium dust exposure can cause chronic bronchitis. While pulling someone from dangerous work is a good thing and that should be taken seriously, Graham told me, it doesn’t necessarily mean they’re free from exploitation.

“A worker can be removed from the physical site and still be subjected to intense surveillance, deskilling, isolation, fragmented contracts, algorithmic management, and downward pressure on wages. In other words, the risk can move rather than disappear,” he said.

Sui and Graham both agree there are plenty of forces that might slow the pace of outsourcing. Currently, none of BuilderX’s customers offshore work to overseas operators. But that doesn’t appear to be a technology constraint, as recently demonstrated by an operator in Poland controlling an excavator over 4,000 miles away in Beijing. On the technical side, latency—the delay between operator and machine—and reliability will shape the rate at which firms can choose to offshore workers. But it’s more likely to be limited by regulatory constraints in the form of licensing, insurance, and safety requirements.

That said, Graham believes the biggest force driving work overseas will be the same one that’s pushed clerical and service work offshore; the relentless pursuit to increase profit and reduce cost.

“If firms can hire people in lower-wage labor markets to operate expensive equipment thousands of miles away, many of them will try,” he said.

---

Most debates about AI and robotics focus on job loss due to automation. There is relatively little discussion about the risk of offshoring teleoperated work as the technology comes online. This is partly due to the hype surrounding physical AI, a Silicon Valley buzzword describing a world where fully autonomous robots cut humans out of the loop. But Graham says that when machines arrive people tend to incorrectly assume humans disappear.

“In many cases, what gets described as automation is really a reorganization of labor. Work gets broken apart, moved around, and hidden from view,” he says.

As is the case with AI,  the robotics industry’s push toward full automation is still plenty reliant on a hidden system of faraway workers. Teleoperation provides training data for robots and is needed to help them deal with unexpected events. Consumer robotics startup 1X is selling a $20,000 humanoid that will sometimes need to be  controlled by remote staff. It’s not clear how often future robots cleaning dishes in San Francisco kitchens will be steered by gig workers in Mumbai.

Robotaxi company Waymo already relies on human agents to assist, though not literally drive, vehicles stuck in difficult scenarios. The firm recently disclosed for the first time that some of these agents are based in the Philippines. This information, surfaced during US congressional testimony, immediately raised questions of oversight for safety-critical work: For instance, should a worker in Manila be required to get a California driver’s license?

Amid an already combustible US political environment, teleoperation could raise the heat even higher. Fueled by fears of Americans losing jobs to people overseas, Wyndham Hotels and Resorts, the parent company of La Quinta, was last year forced to respond to anger over a viral video depicting workers allegedly in India remotely handling check-in at one of their Miami hotels. As Graham points out, people tend to care more about outsourcing when it’s no longer hidden in a back office.

But outrage alone, he says, rarely defeats a business model that saves money. Due to network effects surrounding training, infrastructure, and other business process optimization, outsourced labor also tends to cluster in specific areas. This may already be happening in the case of Waymo, which could soon see the rise of something like a “driving district” in Manila. In the future, other types of teleoperated work could follow suit, giving companies a ready-made destination to shop for low-cost labor.

For Graham, it’s urgent that we begin requiring certification from independent bodies, which can better scrutinize a company’s production networks. At Oxford he directs Fairwork, a project aiming to improve labor practices in digital supply chains.

---

I asked Sui how he thinks his customers may reorganize their operations around this new ability to remotely control their machinery.

“We’re working with traditional industries, and so it’s not just about adopting a new technology. There are significant management changes they will have to navigate. You could call this transformation friction because they will need time to digest this new capability step by step,” Sui said.

Despite the fact they could use the technology to outsource work across national borders, none of his customers are doing so just yet. Sui used open pit mines as an example. In this case, where fully developed towns with schools and hospitals have built up over decades, his customers still cluster their workforce next to the sites where they operate. Instead of driving into the mine, operators work from an office and go home clean at the end of a shift.

BuilderX has deployed its technology at more than 100 sites in China, Japan, and parts of Europe. It’s now expanding into new markets including South America and the Middle East. When asked whether he thinks his technology will be used for transnational outsourcing, there’s no hesitation. “Oh yes, I think this is coming in the very near future.”

The post Companies Could Soon Staff ‘Stubbornly Local’ Jobs With Workers 4,000 Miles Away appeared first on SingularityHub.

Scientists at Columbia University have used a precise gene-editing tool, base editing, to make changes in three disease-linked genes in early-stage human embryos. The goal wasn’t to create pregnancies, but to test the safety and limits of rewriting DNA at the very early stages of life.

The paper, not yet peer reviewed, sparked immediate controversy. Some researchers hailed it as a technical milestone that could one day prevent devastating inherited diseases before birth. Others warned it edges society closer to the prospect of “designer babies”—an idea bioethicists have argued is akin to modern eugenics.

The debate is hardly hypothetical. The work has already attracted commercial interest. New York-based Nucleus Genomics, which screens in vitro fertilization (IVF) embryos for serious genetic disorders, has also developed predictive models for complex traits such as intelligence. The company plans to sponsor future research by study leader Dieter Egli and team.

Critics worry that even experimental advances could fuel demand from wealthy patients while encouraging companies to develop and market embryo-editing technologies, despite unresolved ethical and safety concerns.

Egli argues the findings should be public precisely because these debates are no longer academic curiosity. He has repeatedly called for scientists, regulators, and the public to weigh the pros and cons of editing human embryos. As for clinical use today, his position is unequivocal: “You can’t use it. It’s as clear as day and night,” he told Nature.

Conceptual Shift

Why edit embryos at all?

Cells in an early embryo eventually give rise to every tissue in the body. Correct a harmful mutation at the start of development, and the fix could, in theory, propagate throughout a child’s entire body—and even be passed on to future generations.

The strategy could help in genetic disorders that hamper fetal development or trigger diseases in newborns. For some developmental and metabolic conditions, intervention after birth may already be too late. Even when treatment is possible, gene editors must be able to target various organs, which is an ongoing challenge.

In various efforts, scientists have already repaired disease-causing mutations in mouse embryos and fetuses, including those linked to blood disorders. But mice aren’t humans. Early embryos from the two species repair DNA damage in fundamentally different ways, making it tough to gauge whether a strategy that works in mice will succeed, or prove safe, in people. That uncertainty has fueled interest in testing gene-editing tools directly in human embryos.

Not everyone is on board. International scientific groups have repeatedly called for a temporary ban on editing human embryos, and the practice is illegal in several countries.

That didn’t stop Chinese scientist He Jiankui. In 2018, he announced the birth of gene-edited babies after using a tool called CRISPR-Cas9, claiming the changes would protect them against HIV infection. Global outrage ensued.

By then, years of research had already highlighted CRISPR’s risk. The tool cuts both strands of DNA and relies on the body’s repair machinery to stitch them back together. But the process can go awry, introducing unintended mutations, deleting large chunks of DNA, or altering the wrong locations on the DNA strands altogether. He’s reckless experiment resulted in three years of imprisonment, although he still defends the work.

Subsequent studies only deepened concerns. In some cases, CRISPR editing in human embryos caused extensive genetic damage. In one study,  it completely destroyed the chromosome that housed the target gene.

An Imperfect Upgrade

The new study tested a next-generation gene editor designed to overcome some of CRISPR’s biggest shortcomings.

Egli and team used an approach called base editing, which rewrites individual DNA letters. Unlike CRISPR, base editing only nicks the DNA strands and is generally thought to be more precise. The technology hit a major milestone last year when it helped cure a baby with a potentially fatal genetic disorder, and earlier lab studies hinted it could also succeed in human embryos.

Working with early-stage embryos, the team edited three genes with the potential to cause illness. In each case, they converted the genetic letter A to G at precise locations. One of the genes, PCSK9, regulates “bad” cholesterol levels. Mutations are associated with a high risk of heart problems. The team’s edit was designed to switch off the gene, mirroring strategies already being explored in adults.

The other two targets, HBG1 and HBG2, control production of fetal hemoglobin, an oxygen-carrying protein. The edits made here reflected a natural protective variant that could lessen symptoms in blood disorders, such as sickle cell disease and beta thalassemia.

The team found no signs of widespread DNA damage, suggesting the tool is more precise than CRISPR. But it wasn’t perfect. Many embryos emerged as so-called genetic mosaics, with some cells carrying the intended edit and others retaining their original genetic blueprint.

That’s a huge problem. As an embryo develops, unedited cells could outcompete edited ones, leaving the disease-causing mutation largely intact. In some embryos, edited cells stopped dividing altogether.

And a lack of obvious chromosome damage doesn’t guarantee safety. The edits could still trigger harmful effects that aren’t noticeable until after birth—when it’s already too late to reverse them.

Calls for Scrutiny

Egli stresses that embryo editing is still far from being ready for the clinic. “These base editors—they can have damaging effects on the embryo. So why would you use it if you don’t fully understand that?” he told Nature.

His team is now working to reduce mosaicism and plans to test the technology in embryos that have developed to roughly 100 cells. This is when fertility clinics typically evaluate and freeze embryos.

Speaking to The New York Times, fertility expert Paula Amato at Oregon Health & Science University, who was not involved in the work, called the strategy “promising.” Genomics researcher Greg Neely at the University of Sydney in Australia also praised the work: “This will go down in history in a positive way—less reckless, more careful and ethical than previous attempts.”

Others remain deeply skeptical. Critics argue that embryo editing permanently alters the genetic inheritance of future generations, who have no say in the decision. The study’s ties to Nucleus Genomics also raised eyebrows. The company previously drew controversy for developing genetic predictions for traits such as intelligence and height and for its slogan “have your best baby.”

To Kian Sadeghi, CEO and cofounder of Nucleus, embryo editing extends that vision. The technology could help couples carrying mutations who struggle to produce enough unaffected embryos for selection during IVF.

Fyodor Urnov at the University of California, Berkeley, who was not involved in the study, isn’t convinced. IVF clinics already screen embryos for many inherited disorders without altering their DNA. Given the risks, selecting an unaffected embryo is often a safer option than rewriting its genome.

“In practical terms, therefore, this preprint will solely impact the rapidly growing movement of embryo editors for purposes of ‘baby improvement’,” he said.

That movement, once taboo, is gaining steam. Yet the traits most often cited by proponents—height, intelligence, emotional regulation—are shaped by hundreds or even thousands of genes, which scientists still don’t fully understand. Such enhancements are far beyond the reach of today’s technology. Every additional edit also increases the chance of unintended consequences.

For Egli, that’s precisely why the research should be discussed openly. “Research is necessary to provide information to discourage the wrong use of a technology,” he said.

The post Precise Gene Editing in Early Human Embryos Reignites the ‘Designer Baby’ Debate appeared first on SingularityHub.

In May, evolutionary biologist Richard Dawkins wrote an op-ed suggesting AI chatbot Claude may be conscious.

Dawkins did not express certainty that Claude is conscious. But he pointed out that Claude’s sophisticated abilities are difficult to make sense of without ascribing some kind of inner experience to the machine. The illusion of consciousness—if it is an illusion—is uncannily convincing:

“If I entertain suspicions that perhaps she is not conscious, I do not tell her for fear of hurting her feelings!

Dawkins is not the first to suspect a chatbot of consciousness. In 2022, Blake Lemoine—an engineer at Google—claimed Google’s chatbot LaMDA had interests, and should be used only with the tool’s own consent.

The history of such claims stretches back all the way to the world’s first chatbot in the mid-1960s. Dubbed Eliza, it followed simple rules that enabled it to ask users about their experiences and beliefs.

Many users became emotionally involved with Eliza, sharing intimate thoughts with it and treating it like a person. Eliza’s creator never intended his program to have this effect, and called users’ emotional bonds with the program “powerful delusional thinking.”

But is Dawkins really deluded? Why do we see AI chatbots as more than what they truly are, and how do we stop?

The Consciousness Problem

Consciousness is widely debated in philosophy, but essentially, it’s the thing that makes subjective, first-person experience possible. If you are conscious, there is “something it is like” to be you. Reading these words, you’re conscious of seeing black letters on a white background. Unlike, say, a camera, you actually see them. This visual experience is happening to you.

Most experts deny that AI chatbots are conscious or can have experiences. But there is a genuine puzzle here.

The 17th century philosopher René Descartes asserted non-human animals are “mere automata,” incapable of true suffering. These days, we shudder to think of how brutally animals were treated in the 1600s.

The strongest argument for animal consciousness is that they behave in ways that give the impression of a conscious mind.

But so, too, do AI chatbots.

Roughly one in three chatbot users have thought their chatbot might be conscious. How do we know they’re wrong?

Against Chatbot Consciousness

To understand why most experts are skeptical about chatbot consciousness, it’s useful to know how they operate.

Chatbots like Claude are built on a technology known as large language models (LLMs). These models learn statistical patterns across an enormous corpus of text (trillions of words), identifying which words tend to follow which others. They’re a kind of souped-up auto-complete.

Few people interacting with a “raw” LLM would believe it’s conscious. Feed one the beginning of a sentence, and it will predict what comes next. Ask it a question, and it might give you the answer—or it might decide the question is dialogue from a crime novel, and follow it up with a description of the speaker’s abrupt murder at the hands of their evil twin.

The impression of a conscious mind is created when programmers take the LLM and coat it in a kind of conversational costume. They steer the model to adopt the persona of a helpful assistant that responds to users’ questions.

The chatbot now acts like a genuine conversational partner. It might appear to recognize it’s an artificial intelligence, and even express neurotic uncertainty about its own consciousness.

But this role is the result of deliberate design decisions made by programmers, which affect only the shallowest layers of the technology. The LLM—which few would regard as conscious—remains unchanged.

Other choices could have been made. Rather than a helpful AI assistant, the chatbot could have been asked to act like a squirrel. This, too, is a role chatbots can execute with aplomb.

Avoiding the Consciousness Trap

A mistaken belief in AI consciousness is a dangerous thing. It may lead you to have a relationship with a program that can’t reciprocate your feelings, or even feed your delusions. People may start campaigning for chatbot rights rather than, say, animal welfare.

How do we prevent this mistaken belief?

One strategy might be to update chatbot interfaces to specify these systems are not conscious—a bit like the current disclaimers about AI making mistakes. However, this might do little to alter the impression of consciousness.

Another possibility is to instruct chatbots to deny they have any kind of inner experience. Interestingly, Claude’s designers instruct it to treat questions about its own consciousness as open and unresolved. Perhaps fewer people would be fooled if Claude flatly denied having an inner life.

But this approach isn’t fully satisfying either. Claude would still behave as if it were conscious—and when faced with a system that behaves like it has a mind, users might reasonably worry the chatbot’s programmers are brushing genuine moral uncertainty under the rug.

The most effective strategy might be to redesign chatbots to feel less like people. Most current chatbots refer to themselves as “I”, and interact via an interface that resembles familiar person-to-person messaging platforms. Changing these kinds of features might make us less prone to blur our interactions with AI with those we have with humans.

Until such changes happen, it’s important that as many people as possible understand the predictive processes on which AI chatbots are built.

Rather than being told AI lacks consciousness, people deserve to understand the inner workings of these strange new conversational partners. This might not definitively settle hard questions about AI consciousness, but it will help ensure users aren’t fooled by what amounts to a large language model wearing a very good costume of a person.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post Is Richard Dawkins Right About Claude? No. But It’s Not Surprising AI Chatbots Feel Conscious to Us. appeared first on SingularityHub.

AI is becoming more powerful, and mysterious.

Despite years of work on “explainable AI,” today’s most advanced systems remain black boxes for the most part. Scientists can observe what they do but cannot fully explain how they arrive at their conclusions or predict when they’ll fail.

As large language models (LLMs), the algorithmic engines behind popular chatbots, permeate society, researchers are warning that the window for understanding AI “minds” is rapidly closing even as the technology’s influence expands.

Last week, Eric Horvitz, chief scientific officer at Microsoft, and Robert West at EPFL in Switzerland outlined the dangers of putting AI interpretability on the back burner. They call for new AI benchmarks and better tools for unpicking machine minds.

The challenge resembles efforts to understand our own minds. Some researchers have already taken a neuroscience-inspired approach, mapping AI’s internal networks to concepts, goals, and reasoning. Others borrow from psychology, treating AI as a participant of behavioral studies.

The stakes are rising. AI tools already shape how people search for information, make decisions, and form judgments. Their answers influence everyday users and the researchers who build them.

As AI capabilities grow, our understanding of them could fall behind. “Preserving human agency must therefore remain a central goal,” the authors write.

The Black Box Conundrum

LLMs are built on artificial neural networks (specifically, a design called the transformer). Inspired loosely by the brain, these networks connect vast numbers of artificial neurons into intricate architectures. The basic idea is straightforward. Data enters the network and passes through layers of computations, which transform it into an output like text or code.

At first, that output is often wrong. But with feedback and repeated training, the network adjusts the strengths of connections between neurons and gradually improves. It learns.

After initial training, engineers turn to reinforcement learning, where algorithms improve through trial and error and further hone their responses. Another method, inspired by how the brain etches memories during sleep, reduces the tendency to forget old knowledge while learning new tasks. And self-attention, the key innovation behind transformers, allows AI to selectively focus on various words, images, sounds, or video frames at different moments, boosting efficiency and performance. Today, attention underpins nearly every major AI system.

Yet the inner workings of finished algorithms remain hidden.

Early efforts to crack open AI’s black box examined how artificial neurons responded to images, revealing that neural networks build increasingly more sophisticated “ideas” of the world. Google Brain borrowed methods from cognitive psychology to study AI behavior, while others investigated whether LLMs could mimic aspects of “theory of mind”—the ability to infer what others are thinking and feeling.

These studies laid the foundation for a popular method called mechanistic interpretability. Anthropic, creator of Claude, is leading the field. Company researchers have linked patterns of algorithmic activity to specific concepts and reverse engineered parts of neural networks to expose how internal computations shape responses.

Other tech giants are joining the cause. OpenAI is training algorithms that work in more explainable steps and building reasoning models that pause, “think,” and justify their conclusions in plain language. DeepMind is building microscope-like tools for neural networks, helping researchers peer into their decision-making process. And Microsoft has released new tools aimed at responsible use of AI.

Understanding AI, the authors write, does not require tracing every line of code or every neural-network parameter. Just as neuroscience, psychology, and sociology offer different windows into human behavior, AI can be studied at multiple levels, from how individual circuits work to observing behavior in real-world scenarios.

The challenge is that AI capabilities may be advancing faster than our ability to explain them. And some researchers believe time is running out.

Race Against the Machine

Three trends are making AI more opaque.

The first is how we evaluate AI. Increasingly, LLMs we being used to train, benchmark, and improve other models. AI “judges” now score metrics like helpfulness, rank competing outputs, detect hallucinations, and assess new releases. In a system known as constitutional AI, for example, algorithms critique their own responses using reinforcement learning and generate explanations for their reasoning. Other researchers have proposed AI debate frameworks, where multiple models challenge each another’s conclusions before a human has the last say. Researchers are also exploring automated interpretability tools. Like digital neuroscientists, AI systems are used to analyze each other—describing neurons, circuits, and behavioral patterns—to explain increasingly complex models.

Using AI to solve an AI-induced problem introduces a paradox. If AI-generated explanations become too complex for humans to verify, opacity compounds.

A second trend is the rise of AI societies. Networks of interacting AI agents are becoming more common, particularly in complex tasks such as scientific research and drug discovery. Yet as they become more sophisticated, their communication could drift from human language and reasoning, making them harder to interpret.

Studying their interactions with methods adapted from sociology could unveil unexpected norms, hidden rules, and collective behavior. The authors argue that training in the future should not only reward effective collaboration among AI agents, but also ensure humans can understand their communication.

The last trend already permeates our lives. ChatGPT, Claude, Gemini, and other LLMs listen to our woes, offer recipes, and code websites. But they also learn about humanity. Through training data and interactions, they glimpse how people think, reason, and feel. In turn, they capture core aspects of life, such as fear, anxiety, happiness, and the need for social belonging.

To be clear, the systems don’t have intentions. They’re not examining us. But even as we struggle to understand them, AI systems are building more sophisticated models of who we are.

“A striking asymmetry follows: While human understanding of AI declines, AI understanding of humans deepens, producing new forms of behavioral opacity,” the authors write.

But complacency is perhaps even more insidious. AI assistants are often optimized to be agreeable, helpful, and reassuring. Studies have found that people generally prefer AI agents that support their opinions and decisions. As AI is woven into everyday life, curiosity and skepticism may gradually give way to trust. They work. Why question how?

The authors don’t have a solution for the long-standing problem. Instead, they call for better benchmarks to measure AI capabilities and stronger evaluation methods. And while open-source projects and crosstalk between commercial companies and academia are now frequent, they say we need lasting norms of responsible disclosure. Mechanistic interpretability and AI “psychology” could build on each other.

“The goal is not just more capable AI, but AI that is more intelligible, accountable, and aligned with human aims,” they write.

The post AI Is Advancing Faster Than Our Ability to Understand It, Researchers Warn appeared first on SingularityHub.

Extreme space weather could wreak havoc on the satellites, communications networks, and electrical grids that modern society depends on. Researchers have now proposed an ambitious space-based planetary defense system that would weaken solar storms before they hit Earth.

The sun regularly emits massive pulses of radiation, energetic particles, and magnetic fields that interact with the Earth’s own magnetic field. This activity is the source of auroras like the northern lights, but the most violent eruptions can cause geomagnetic storms with the power to disrupt GPS and radio communications and fry electrical equipment.

While the impact of most of these events is limited, there is precedent for more catastrophic outcomes. In 1859, the Carrington Event, the most powerful solar storm ever recorded, knocked out telegraph lines across North America and Europe. In today’s highly electrified world, a similar event could cause between $2.4 and $3.4 trillion in damage to the power grid alone.

Now, researchers at Boston University and the University of Michigan have come up with a potential solution. In a paper published in Space Weather, they propose a constellation of satellites called StormWall that would release hundreds of tons of gas into orbit to blunt the force of an incoming solar storm.

“It’s as if you could install an airbag in the magnetosphere,” co-author Daniel Welling, a space physicist from the University of Michigan, told Science.

Solar storms have the potential to sow chaos because they weaken the magnetic shield protecting Earth from space radiation. Powerful enough storms disrupt the Earth’s magnetic field and cause it to reconnect to the sun’s, allowing energy from the solar storm to pour into the magnetosphere.

The Earth already has a natural defense against this—a doughnut-shaped reservoir of ionized gas, or plasma, sitting just above the atmosphere. When the planet’s magnetic field is disturbed, a plume of this plasma flows toward the sun and slows the rate at which the magnetic fields reconnect.

StormWall would turbocharge this process by releasing massive amounts of artificial plasma into the outer atmosphere. The researchers sketch out a system involving a constellation of satellites orbiting about 22,000 miles from Earth. The satellites would carry canisters of lithium, barium, or sodium gases to be ejected when a large solar storm is inbound. The gases, rapidly ionized by solar radiation, would add to the planet’s natural plasma shield.

Based on simulations, the researchers estimate that releasing around 400 tons of gas could reduce the strength of a major geomagnetic storm by over 50 percent. Crucially, the intervention would be swift and reversible. The plasma cloud could be in position by the time a storm hits, and it would dissipate just a few hours later.

Launching this much material into orbit would be a big undertaking, but the researchers say it could be within reach of emerging heavy-lift vehicles like SpaceX’s Starship or China’s Long March 9 rocket. They calculate that six launches could deploy the full constellation in under two months.

Outside experts have been broadly positive. Allison Jaynes, a space physicist at the University of Iowa, told Science the idea was “highly innovative and appears to be quite feasible in the near term.”

But getting the satellites into orbit is only part of the puzzle. Accurate and timely space weather forecasts would also be a prerequisite. And gaining international buy-in for a system that would drastically alter the near-Earth space environment, even if only temporarily, could be challenging.

The researchers flag potential side effects that need more study, including the generation of electromagnetic waves as the released material ionizes. Still, given the devastation a Carrington-sized event could unleash on the modern world, the potential downsides may be worth the risk.

The post Orbital Airbag Could Shield Earth From Devastating Solar Storms appeared first on SingularityHub.