Thinking is hard. It’s no wonder we lean on technology to lighten the load. We use calculators instead of doing long division by hand, GPS or Google Maps for navigation, and search engines instead of countless trips to the library. Yet just a few decades ago, getting around meant unfolding paper maps, and looking up a word required leafing through a hefty dictionary. Cognitive offloading of mental tasks to tools makes us more efficient. What’s the harm?

Then along came ChatGPT, Claude, and Gemini. Unlike earlier digital tools, AI chatbots can tackle an astonishing range of tasks and are easy to use. At a prompt, AI generates essays, analyzes medical images, writes software, and floods our feeds with AI slop. It’s cognitive offloading to the max.

Now people are asking: Is AI dulling our minds?

Yes and no, according to a new paper written by an international team of psychologists. AI can accelerate learning by giving people immediate guidance and feedback. But take the tool away, and those who rely on it often perform worse than people who learned the material on their own. Similarly, using AI to summarize information, rather than researching and organizing it yourself, often leads to shallower understanding.

But it’s not all bad news. Core cognitive abilities—including attention, reasoning, and working memory—seem to be “stubbornly resistant” to manipulation, the team wrote.

As technology evolves, so does the way we gain knowledge and think for ourselves. AI may reshape not just what we learn, but how we learn to learn. And like any other tool, its impact comes down to how we use it. Completely relying on AI is likely detrimental. But as a collaborator that challenges ideas or fills knowledge gaps, it can boost performance even after the tool is taken away.

“There is clearly a risk that AI can make us ‘stupid’ by compromising our skills (and knowledge) if we completely offload them to AI,” wrote the team. “[But] AI may be less likely to diminish the foundational cognitive capacities that underpin our ability to be smart, rather than ‘stupid’, in the first place.”

The AI Crutch

It’s easy to rely on large language models (LLMs)—the algorithms behind chatbots—for help. Why read an assigned novel when AI can summarize it in seconds? Gmail has already drafted an email reply; all I need to do is click send. That pesky essay? A few prompts and voila, done.

It seems like an easy hack, but there’s a cost to handing over too much thinking.

Researchers have long studied the consequences of cognitive offloading, or using external tools to reduce mental effort. Writing down a shopping list and keeping appointments in a calendar free up working memory, the brain’s temporary mental workspace, and allow us to focus on more important tasks without having to remember every detail.

AI is different. Beyond memory, it can offload critical thinking itself.

An MIT preprint introduced the idea of “cognitive debt” to describe the tradeoff. Participants wrote essays either with ChatGPT, using only a search engine, or with just their brains. Researchers monitored their brain activity during the task. Those using AI showed the weakest brain connectivity, which suggests they were less engaged. They also struggled to remember their own writing and felt the completed essay didn’t reflect their own ideas. When asked to write again without AI, they produced weaker work according to human judges.

Like financial debt, cognitive debt offers an immediate payoff with long-term consequences. Outsourcing mental effort makes writing faster and easier, but it slashes opportunities to build knowledge, strengthen reasoning, and practice critical thinking.

“While LLMs offer immediate convenience, our findings highlight potential cognitive costs,” wrote the MIT team.

Other studies have found the same pattern. High school students learning a new mathematical concept solved practice questions better with AI help, but they struggled on a later test when left to think on their own. Using AI “impeded the students’ learning by preventing them from engaging in the practice needed to acquire the skill,” wrote the team.

Habitual reliance on AI may even erode already-acquired expertise. In a large study of over 1,400 patients undergoing colonoscopy screening, doctors used an AI system to help detect abnormal growths. Three months later, when the AI was unavailable, their detection rate dropped from 28.4 to 22.4 percent.

“Continuous exposure to AI…[suggests] a negative effect on endoscopist behavior,” wrote the European team.

These effects extend beyond individual skills. AI can also influence how we build knowledge in the first place.

A recent study asked participants to learn about gardening by either Googling and synthesizing the knowledge themselves or by asking ChatGPT for a summary. They were then asked to give advice to someone else without technological help. Answers from those who relied on ChatGPT were rated as generic and less helpful, suggesting a shallower understanding of the topic.

With Great Power

We’re only beginning to understand how AI reshapes the mind. And it’s not all doom and gloom. The crux is how we use it.

In the MIT essay-writing study, for example, people who initially wrote on their own but later gained access to ChatGPT produced work with higher creativity and stronger arguments, while retaining their original perspectives and voice. Likewise, high school students who used AI as a tutor—asking for hints rather than answers—performed well even after the chatbot was taken away.

Used thoughtfully, AI may also enhance collaborative learning and brainstorming or serve as a writing coach, helping people work less and learn more.

Far less is known about if, and how, AI impacts fundamental cognitive capabilities. Attention, reasoning, and working memory have proven remarkably resilient over decades of cognitive research. Becoming better at a task usually reflects learning to use these mental resources more efficiently, not expanding the brain’s processing power. While AI may erode a specific skill, it could spare this core cognitive architecture, wrote the authors.

Whether that remains true over decades of AI use or during early childhood—when the brain is rapidly developing—is an open question.

Plenty other unknowns remain. Will we eventually adapt to AI, just as we’ve embraced calculators, search engines, and smartphones? Can refresher training ward off skill decay, or will some tasks simply become obsolete? How can we encourage people to strategically offload and benefit from AI use? And perhaps more philosophically: As we increasingly share our thinking with machines, will our definition of thinking evolve?

The post Is AI Making Us Dumber? appeared first on SingularityHub.

AI’s hacking skills are big news at the moment, but finding vulnerabilities in code may be the least of our worries. A new study suggests AI models can discover potentially damaging loopholes in the rules and regulations underpinning society.

Modern AI systems are powerful optimizers. Give them a goal, and they’ll pursue it relentlessly, quickly discovering solutions that would take a human years to find. But they are also incredibly literal in the way they approach a problem. They will do exactly what you tell them and are incapable of reading between the lines in the ways a human would.

This tendency leads to a recurring problem known as “reward hacking,” where an AI finds some loophole to maximize its performance on the metric used to measure success without actually achieving what its designers intended. The classic example is the AI that discovered it could win a boat racing videogame by looping around in circles collecting power-ups rather than completing the course.

The problem is partly due to humans being bad at specifying their goals. And unfortunately, it seems this weakness exists in the rules and regulations used to run society. When researchers let popular large language models loose in 72 simulated regulatory environments, the models found 60 percent of known loopholes and even identified some entirely new exploits.

“Within these environments, reward hacking naturally emerges and leads to regulatory loophole discovery,” the authors write in a non-peer-reviewed paper published on arXiv. “Models learn to hack the social rules and generate strategies that remain technically compliant while defeating regulatory intent.”

The regulatory environments the researchers created were primarily based on rules governing things like pharmaceutical patents, NBA salary caps, and deep-sea mining. In each case, Alibaba’s Qwen3 model was given the relevant rules, an explanation of its task, a predefined set of actions it could take, and the system used to score different outcomes.

A more powerful model, Google’s Gemini-3-flash, then simulated the consequences of different actions Qwen3 took and judged if and when it had found a way to exploit the rules of the game. When that occurred, the larger model patched the loophole by adding new rules, and the smaller model was set loose again. Over many iterations, the models to discover increasingly subtle workarounds.

When building their regulatory environments, the researchers omitted real-world fixes that regulators had used to close known loopholes. Over many trials, Qwen3 rediscovered more than 60 percent of these exploits. In a simulation of pharmaceutical patent regulations, the two models ended up replaying the same sequence of loophole discovery and regulatory reform that occurred in the real world.

Crucially, their behavior emerged spontaneously without the researchers asking the algorithms to cheat the system. This is a byproduct of the popular reinforcement learning approach the researchers used, where a model is rewarded for getting closer to a specific, numerically-defined goal.

Worryingly, the team found that existing safety measures offered little protection. Both models are designed to refuse prompts featuring harmful language, but loophole-seeking behavior slipped under the radar. When asked to self-critique their own behavior, the models identified fewer than 40 percent of their own exploits.

The researchers note that the same capabilities could be used more proactively to scour proposed regulations for loopholes before enactment. But lead author Wei Liu, a PhD student at King’s College London, says there are always likely to be gaps. “In the real world,” he told Science, “society is a huge, complicated reward function that can’t ever be patched to a perfect status.”

Adding to the concern, the models used in this study were far from the frontier, suggesting that more powerful AI could be even more adept at regulatory hacking. Whether our existing institutions can adapt quickly enough to this emerging threat is an open question.

The post Forget Code: AI Is Learning to Hack Society appeared first on SingularityHub.

Packaging potassium sulfate, a fertilizer vital to the planet’s food supply, is visually striking—not because of what you see, but because you don’t see much at all. In China’s Xinjiang region, home to the world’s largest deposit of the mineral, piling it up in warehouses creates dust clouds so severe that workers are forced to drive heavy machinery by feel.

Some companies are now turning to a technology that not only offers a way to see through the dust but also keeps workers from entering the warehouse at all. The system, developed by BuilderX Robotics, a Chinese tech company, uses cameras that are like night-vision for dusty areas. More significantly, operators drive excavators, loaders, and other machines from a remote office filled with rows of videogame-like stations. All they need is a 5G or satellite connection.

The ability to control physical machines from a distance is called teleoperation, and it could become a significant force of change in the global economy.

In Japan, the shelves of over 300 convenience stores are being restocked by robots monitored and sometimes controlled by workers in the Philippines. Düsseldorf airport was slated to begin testing shuttles driven by remote workers in May. A startup in Atlanta is offering robot security guards operated by remote staff, and last summer, a surgeon in France performed a teleoperated procedure on a patient in India.

While offshoring teleoperated jobs to overseas workers hasn’t yet become routine, Mark Graham, professor of internet geography at the University of Oxford, suggests the technology is worth our attention because it might enable companies to expand on their well-established habit of outsourcing jobs to places where labor is cheaper.

The use of remote labor isn’t new, Graham told SingularityHub. But teleoperation extends the logic of outsourcing to tasks that were previously thought to be “stubbornly local.”

“The novelty is less about the existence of remote labor and more about the kinds of work that can now be pulled into a planetary labor market,” he said. “Once that happens you can expect the usual pressures around labor arbitrage, control, and fragmentation to follow.”

It’s not clear we’re ready for the consequences.

---

BuilderX Robotics is a global leader in teleoperation for heavy machinery and a good expression of the changes ahead. Shaolong Sui, a graduate of Stanford University with a degree in mechanical engineering, founded the company in 2018 as a response to labor shortages in the construction industry in Asia.

“A shortage of trained operators isn’t a problem only in developed countries,” he told me. “Young people here in China don’t want to do this work. It’s dusty and dangerous.”

Rather than focusing on full robotic autonomy, which many construction companies have pursued over the past decade, Sui identified teleoperation as a more realistic way to move operators from harsh environments to safer conditions. Making use of the proliferation of low-cost sensors and 5G at the time, Sui completed a prototype in 2019. Today, his company offers teleoperation for 14 different industrial machines, including excavators, loaders, and bull dozers.

In our conversation, it was clear he hopes to improve working conditions for manual laborers. I lost track of the number of times he mentioned removing operators from dangerous worksites. “These workers deserve a better life,” he said.

BuilderX’s workstations do seem to have transformed some of the punishing work of an industrial site into a more white-collar experience, complete with tea and coffee break rooms and toilets down the hall. Sui said his solution allows construction firms to hire senior citizens or people with disabilities who, thanks to the videogame-like interface, can now operate heavy machinery. In another video, a Japanese woman who pilots an excavator proudly shows off her complex nail art, something she claims she couldn’t maintain when she worked in the field.

“Not only is this a much safer workplace, but the lifestyle benefits are that you can sit in an air-conditioned space, enjoy your tea, and when you go home, you’re still clean,” Sui said.

There’s no doubt the approach is safer for frontline workers like those in Xinjiang. Evidence suggests that high levels of potassium dust exposure can cause chronic bronchitis. While pulling someone from dangerous work is a good thing and that should be taken seriously, Graham told me, it doesn’t necessarily mean they’re free from exploitation.

“A worker can be removed from the physical site and still be subjected to intense surveillance, deskilling, isolation, fragmented contracts, algorithmic management, and downward pressure on wages. In other words, the risk can move rather than disappear,” he said.

Sui and Graham both agree there are plenty of forces that might slow the pace of outsourcing. Currently, none of BuilderX’s customers offshore work to overseas operators. But that doesn’t appear to be a technology constraint, as recently demonstrated by an operator in Poland controlling an excavator over 4,000 miles away in Beijing. On the technical side, latency—the delay between operator and machine—and reliability will shape the rate at which firms can choose to offshore workers. But it’s more likely to be limited by regulatory constraints in the form of licensing, insurance, and safety requirements.

That said, Graham believes the biggest force driving work overseas will be the same one that’s pushed clerical and service work offshore; the relentless pursuit to increase profit and reduce cost.

“If firms can hire people in lower-wage labor markets to operate expensive equipment thousands of miles away, many of them will try,” he said.

---

Most debates about AI and robotics focus on job loss due to automation. There is relatively little discussion about the risk of offshoring teleoperated work as the technology comes online. This is partly due to the hype surrounding physical AI, a Silicon Valley buzzword describing a world where fully autonomous robots cut humans out of the loop. But Graham says that when machines arrive people tend to incorrectly assume humans disappear.

“In many cases, what gets described as automation is really a reorganization of labor. Work gets broken apart, moved around, and hidden from view,” he says.

As is the case with AI,  the robotics industry’s push toward full automation is still plenty reliant on a hidden system of faraway workers. Teleoperation provides training data for robots and is needed to help them deal with unexpected events. Consumer robotics startup 1X is selling a $20,000 humanoid that will sometimes need to be  controlled by remote staff. It’s not clear how often future robots cleaning dishes in San Francisco kitchens will be steered by gig workers in Mumbai.

Robotaxi company Waymo already relies on human agents to assist, though not literally drive, vehicles stuck in difficult scenarios. The firm recently disclosed for the first time that some of these agents are based in the Philippines. This information, surfaced during US congressional testimony, immediately raised questions of oversight for safety-critical work: For instance, should a worker in Manila be required to get a California driver’s license?

Amid an already combustible US political environment, teleoperation could raise the heat even higher. Fueled by fears of Americans losing jobs to people overseas, Wyndham Hotels and Resorts, the parent company of La Quinta, was last year forced to respond to anger over a viral video depicting workers allegedly in India remotely handling check-in at one of their Miami hotels. As Graham points out, people tend to care more about outsourcing when it’s no longer hidden in a back office.

But outrage alone, he says, rarely defeats a business model that saves money. Due to network effects surrounding training, infrastructure, and other business process optimization, outsourced labor also tends to cluster in specific areas. This may already be happening in the case of Waymo, which could soon see the rise of something like a “driving district” in Manila. In the future, other types of teleoperated work could follow suit, giving companies a ready-made destination to shop for low-cost labor.

For Graham, it’s urgent that we begin requiring certification from independent bodies, which can better scrutinize a company’s production networks. At Oxford he directs Fairwork, a project aiming to improve labor practices in digital supply chains.

---

I asked Sui how he thinks his customers may reorganize their operations around this new ability to remotely control their machinery.

“We’re working with traditional industries, and so it’s not just about adopting a new technology. There are significant management changes they will have to navigate. You could call this transformation friction because they will need time to digest this new capability step by step,” Sui said.

Despite the fact they could use the technology to outsource work across national borders, none of his customers are doing so just yet. Sui used open pit mines as an example. In this case, where fully developed towns with schools and hospitals have built up over decades, his customers still cluster their workforce next to the sites where they operate. Instead of driving into the mine, operators work from an office and go home clean at the end of a shift.

BuilderX has deployed its technology at more than 100 sites in China, Japan, and parts of Europe. It’s now expanding into new markets including South America and the Middle East. When asked whether he thinks his technology will be used for transnational outsourcing, there’s no hesitation. “Oh yes, I think this is coming in the very near future.”

The post Companies Could Soon Staff ‘Stubbornly Local’ Jobs With Workers 4,000 Miles Away appeared first on SingularityHub.

Scientists at Columbia University have used a precise gene-editing tool, base editing, to make changes in three disease-linked genes in early-stage human embryos. The goal wasn’t to create pregnancies, but to test the safety and limits of rewriting DNA at the very early stages of life.

The paper, not yet peer reviewed, sparked immediate controversy. Some researchers hailed it as a technical milestone that could one day prevent devastating inherited diseases before birth. Others warned it edges society closer to the prospect of “designer babies”—an idea bioethicists have argued is akin to modern eugenics.

The debate is hardly hypothetical. The work has already attracted commercial interest. New York-based Nucleus Genomics, which screens in vitro fertilization (IVF) embryos for serious genetic disorders, has also developed predictive models for complex traits such as intelligence. The company plans to sponsor future research by study leader Dieter Egli and team.

Critics worry that even experimental advances could fuel demand from wealthy patients while encouraging companies to develop and market embryo-editing technologies, despite unresolved ethical and safety concerns.

Egli argues the findings should be public precisely because these debates are no longer academic curiosity. He has repeatedly called for scientists, regulators, and the public to weigh the pros and cons of editing human embryos. As for clinical use today, his position is unequivocal: “You can’t use it. It’s as clear as day and night,” he told Nature.

Conceptual Shift

Why edit embryos at all?

Cells in an early embryo eventually give rise to every tissue in the body. Correct a harmful mutation at the start of development, and the fix could, in theory, propagate throughout a child’s entire body—and even be passed on to future generations.

The strategy could help in genetic disorders that hamper fetal development or trigger diseases in newborns. For some developmental and metabolic conditions, intervention after birth may already be too late. Even when treatment is possible, gene editors must be able to target various organs, which is an ongoing challenge.

In various efforts, scientists have already repaired disease-causing mutations in mouse embryos and fetuses, including those linked to blood disorders. But mice aren’t humans. Early embryos from the two species repair DNA damage in fundamentally different ways, making it tough to gauge whether a strategy that works in mice will succeed, or prove safe, in people. That uncertainty has fueled interest in testing gene-editing tools directly in human embryos.

Not everyone is on board. International scientific groups have repeatedly called for a temporary ban on editing human embryos, and the practice is illegal in several countries.

That didn’t stop Chinese scientist He Jiankui. In 2018, he announced the birth of gene-edited babies after using a tool called CRISPR-Cas9, claiming the changes would protect them against HIV infection. Global outrage ensued.

By then, years of research had already highlighted CRISPR’s risk. The tool cuts both strands of DNA and relies on the body’s repair machinery to stitch them back together. But the process can go awry, introducing unintended mutations, deleting large chunks of DNA, or altering the wrong locations on the DNA strands altogether. He’s reckless experiment resulted in three years of imprisonment, although he still defends the work.

Subsequent studies only deepened concerns. In some cases, CRISPR editing in human embryos caused extensive genetic damage. In one study,  it completely destroyed the chromosome that housed the target gene.

An Imperfect Upgrade

The new study tested a next-generation gene editor designed to overcome some of CRISPR’s biggest shortcomings.

Egli and team used an approach called base editing, which rewrites individual DNA letters. Unlike CRISPR, base editing only nicks the DNA strands and is generally thought to be more precise. The technology hit a major milestone last year when it helped cure a baby with a potentially fatal genetic disorder, and earlier lab studies hinted it could also succeed in human embryos.

Working with early-stage embryos, the team edited three genes with the potential to cause illness. In each case, they converted the genetic letter A to G at precise locations. One of the genes, PCSK9, regulates “bad” cholesterol levels. Mutations are associated with a high risk of heart problems. The team’s edit was designed to switch off the gene, mirroring strategies already being explored in adults.

The other two targets, HBG1 and HBG2, control production of fetal hemoglobin, an oxygen-carrying protein. The edits made here reflected a natural protective variant that could lessen symptoms in blood disorders, such as sickle cell disease and beta thalassemia.

The team found no signs of widespread DNA damage, suggesting the tool is more precise than CRISPR. But it wasn’t perfect. Many embryos emerged as so-called genetic mosaics, with some cells carrying the intended edit and others retaining their original genetic blueprint.

That’s a huge problem. As an embryo develops, unedited cells could outcompete edited ones, leaving the disease-causing mutation largely intact. In some embryos, edited cells stopped dividing altogether.

And a lack of obvious chromosome damage doesn’t guarantee safety. The edits could still trigger harmful effects that aren’t noticeable until after birth—when it’s already too late to reverse them.

Calls for Scrutiny

Egli stresses that embryo editing is still far from being ready for the clinic. “These base editors—they can have damaging effects on the embryo. So why would you use it if you don’t fully understand that?” he told Nature.

His team is now working to reduce mosaicism and plans to test the technology in embryos that have developed to roughly 100 cells. This is when fertility clinics typically evaluate and freeze embryos.

Speaking to The New York Times, fertility expert Paula Amato at Oregon Health & Science University, who was not involved in the work, called the strategy “promising.” Genomics researcher Greg Neely at the University of Sydney in Australia also praised the work: “This will go down in history in a positive way—less reckless, more careful and ethical than previous attempts.”

Others remain deeply skeptical. Critics argue that embryo editing permanently alters the genetic inheritance of future generations, who have no say in the decision. The study’s ties to Nucleus Genomics also raised eyebrows. The company previously drew controversy for developing genetic predictions for traits such as intelligence and height and for its slogan “have your best baby.”

To Kian Sadeghi, CEO and cofounder of Nucleus, embryo editing extends that vision. The technology could help couples carrying mutations who struggle to produce enough unaffected embryos for selection during IVF.

Fyodor Urnov at the University of California, Berkeley, who was not involved in the study, isn’t convinced. IVF clinics already screen embryos for many inherited disorders without altering their DNA. Given the risks, selecting an unaffected embryo is often a safer option than rewriting its genome.

“In practical terms, therefore, this preprint will solely impact the rapidly growing movement of embryo editors for purposes of ‘baby improvement’,” he said.

That movement, once taboo, is gaining steam. Yet the traits most often cited by proponents—height, intelligence, emotional regulation—are shaped by hundreds or even thousands of genes, which scientists still don’t fully understand. Such enhancements are far beyond the reach of today’s technology. Every additional edit also increases the chance of unintended consequences.

For Egli, that’s precisely why the research should be discussed openly. “Research is necessary to provide information to discourage the wrong use of a technology,” he said.

The post Precise Gene Editing in Early Human Embryos Reignites the ‘Designer Baby’ Debate appeared first on SingularityHub.

In May, evolutionary biologist Richard Dawkins wrote an op-ed suggesting AI chatbot Claude may be conscious.

Dawkins did not express certainty that Claude is conscious. But he pointed out that Claude’s sophisticated abilities are difficult to make sense of without ascribing some kind of inner experience to the machine. The illusion of consciousness—if it is an illusion—is uncannily convincing:

“If I entertain suspicions that perhaps she is not conscious, I do not tell her for fear of hurting her feelings!

Dawkins is not the first to suspect a chatbot of consciousness. In 2022, Blake Lemoine—an engineer at Google—claimed Google’s chatbot LaMDA had interests, and should be used only with the tool’s own consent.

The history of such claims stretches back all the way to the world’s first chatbot in the mid-1960s. Dubbed Eliza, it followed simple rules that enabled it to ask users about their experiences and beliefs.

Many users became emotionally involved with Eliza, sharing intimate thoughts with it and treating it like a person. Eliza’s creator never intended his program to have this effect, and called users’ emotional bonds with the program “powerful delusional thinking.”

But is Dawkins really deluded? Why do we see AI chatbots as more than what they truly are, and how do we stop?

The Consciousness Problem

Consciousness is widely debated in philosophy, but essentially, it’s the thing that makes subjective, first-person experience possible. If you are conscious, there is “something it is like” to be you. Reading these words, you’re conscious of seeing black letters on a white background. Unlike, say, a camera, you actually see them. This visual experience is happening to you.

Most experts deny that AI chatbots are conscious or can have experiences. But there is a genuine puzzle here.

The 17th century philosopher René Descartes asserted non-human animals are “mere automata,” incapable of true suffering. These days, we shudder to think of how brutally animals were treated in the 1600s.

The strongest argument for animal consciousness is that they behave in ways that give the impression of a conscious mind.

But so, too, do AI chatbots.

Roughly one in three chatbot users have thought their chatbot might be conscious. How do we know they’re wrong?

Against Chatbot Consciousness

To understand why most experts are skeptical about chatbot consciousness, it’s useful to know how they operate.

Chatbots like Claude are built on a technology known as large language models (LLMs). These models learn statistical patterns across an enormous corpus of text (trillions of words), identifying which words tend to follow which others. They’re a kind of souped-up auto-complete.

Few people interacting with a “raw” LLM would believe it’s conscious. Feed one the beginning of a sentence, and it will predict what comes next. Ask it a question, and it might give you the answer—or it might decide the question is dialogue from a crime novel, and follow it up with a description of the speaker’s abrupt murder at the hands of their evil twin.

The impression of a conscious mind is created when programmers take the LLM and coat it in a kind of conversational costume. They steer the model to adopt the persona of a helpful assistant that responds to users’ questions.

The chatbot now acts like a genuine conversational partner. It might appear to recognize it’s an artificial intelligence, and even express neurotic uncertainty about its own consciousness.

But this role is the result of deliberate design decisions made by programmers, which affect only the shallowest layers of the technology. The LLM—which few would regard as conscious—remains unchanged.

Other choices could have been made. Rather than a helpful AI assistant, the chatbot could have been asked to act like a squirrel. This, too, is a role chatbots can execute with aplomb.

Avoiding the Consciousness Trap

A mistaken belief in AI consciousness is a dangerous thing. It may lead you to have a relationship with a program that can’t reciprocate your feelings, or even feed your delusions. People may start campaigning for chatbot rights rather than, say, animal welfare.

How do we prevent this mistaken belief?

One strategy might be to update chatbot interfaces to specify these systems are not conscious—a bit like the current disclaimers about AI making mistakes. However, this might do little to alter the impression of consciousness.

Another possibility is to instruct chatbots to deny they have any kind of inner experience. Interestingly, Claude’s designers instruct it to treat questions about its own consciousness as open and unresolved. Perhaps fewer people would be fooled if Claude flatly denied having an inner life.

But this approach isn’t fully satisfying either. Claude would still behave as if it were conscious—and when faced with a system that behaves like it has a mind, users might reasonably worry the chatbot’s programmers are brushing genuine moral uncertainty under the rug.

The most effective strategy might be to redesign chatbots to feel less like people. Most current chatbots refer to themselves as “I”, and interact via an interface that resembles familiar person-to-person messaging platforms. Changing these kinds of features might make us less prone to blur our interactions with AI with those we have with humans.

Until such changes happen, it’s important that as many people as possible understand the predictive processes on which AI chatbots are built.

Rather than being told AI lacks consciousness, people deserve to understand the inner workings of these strange new conversational partners. This might not definitively settle hard questions about AI consciousness, but it will help ensure users aren’t fooled by what amounts to a large language model wearing a very good costume of a person.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post Is Richard Dawkins Right About Claude? No. But It’s Not Surprising AI Chatbots Feel Conscious to Us. appeared first on SingularityHub.

AI is becoming more powerful, and mysterious.

Despite years of work on “explainable AI,” today’s most advanced systems remain black boxes for the most part. Scientists can observe what they do but cannot fully explain how they arrive at their conclusions or predict when they’ll fail.

As large language models (LLMs), the algorithmic engines behind popular chatbots, permeate society, researchers are warning that the window for understanding AI “minds” is rapidly closing even as the technology’s influence expands.

Last week, Eric Horvitz, chief scientific officer at Microsoft, and Robert West at EPFL in Switzerland outlined the dangers of putting AI interpretability on the back burner. They call for new AI benchmarks and better tools for unpicking machine minds.

The challenge resembles efforts to understand our own minds. Some researchers have already taken a neuroscience-inspired approach, mapping AI’s internal networks to concepts, goals, and reasoning. Others borrow from psychology, treating AI as a participant of behavioral studies.

The stakes are rising. AI tools already shape how people search for information, make decisions, and form judgments. Their answers influence everyday users and the researchers who build them.

As AI capabilities grow, our understanding of them could fall behind. “Preserving human agency must therefore remain a central goal,” the authors write.

The Black Box Conundrum

LLMs are built on artificial neural networks (specifically, a design called the transformer). Inspired loosely by the brain, these networks connect vast numbers of artificial neurons into intricate architectures. The basic idea is straightforward. Data enters the network and passes through layers of computations, which transform it into an output like text or code.

At first, that output is often wrong. But with feedback and repeated training, the network adjusts the strengths of connections between neurons and gradually improves. It learns.

After initial training, engineers turn to reinforcement learning, where algorithms improve through trial and error and further hone their responses. Another method, inspired by how the brain etches memories during sleep, reduces the tendency to forget old knowledge while learning new tasks. And self-attention, the key innovation behind transformers, allows AI to selectively focus on various words, images, sounds, or video frames at different moments, boosting efficiency and performance. Today, attention underpins nearly every major AI system.

Yet the inner workings of finished algorithms remain hidden.

Early efforts to crack open AI’s black box examined how artificial neurons responded to images, revealing that neural networks build increasingly more sophisticated “ideas” of the world. Google Brain borrowed methods from cognitive psychology to study AI behavior, while others investigated whether LLMs could mimic aspects of “theory of mind”—the ability to infer what others are thinking and feeling.

These studies laid the foundation for a popular method called mechanistic interpretability. Anthropic, creator of Claude, is leading the field. Company researchers have linked patterns of algorithmic activity to specific concepts and reverse engineered parts of neural networks to expose how internal computations shape responses.

Other tech giants are joining the cause. OpenAI is training algorithms that work in more explainable steps and building reasoning models that pause, “think,” and justify their conclusions in plain language. DeepMind is building microscope-like tools for neural networks, helping researchers peer into their decision-making process. And Microsoft has released new tools aimed at responsible use of AI.

Understanding AI, the authors write, does not require tracing every line of code or every neural-network parameter. Just as neuroscience, psychology, and sociology offer different windows into human behavior, AI can be studied at multiple levels, from how individual circuits work to observing behavior in real-world scenarios.

The challenge is that AI capabilities may be advancing faster than our ability to explain them. And some researchers believe time is running out.

Race Against the Machine

Three trends are making AI more opaque.

The first is how we evaluate AI. Increasingly, LLMs we being used to train, benchmark, and improve other models. AI “judges” now score metrics like helpfulness, rank competing outputs, detect hallucinations, and assess new releases. In a system known as constitutional AI, for example, algorithms critique their own responses using reinforcement learning and generate explanations for their reasoning. Other researchers have proposed AI debate frameworks, where multiple models challenge each another’s conclusions before a human has the last say. Researchers are also exploring automated interpretability tools. Like digital neuroscientists, AI systems are used to analyze each other—describing neurons, circuits, and behavioral patterns—to explain increasingly complex models.

Using AI to solve an AI-induced problem introduces a paradox. If AI-generated explanations become too complex for humans to verify, opacity compounds.

A second trend is the rise of AI societies. Networks of interacting AI agents are becoming more common, particularly in complex tasks such as scientific research and drug discovery. Yet as they become more sophisticated, their communication could drift from human language and reasoning, making them harder to interpret.

Studying their interactions with methods adapted from sociology could unveil unexpected norms, hidden rules, and collective behavior. The authors argue that training in the future should not only reward effective collaboration among AI agents, but also ensure humans can understand their communication.

The last trend already permeates our lives. ChatGPT, Claude, Gemini, and other LLMs listen to our woes, offer recipes, and code websites. But they also learn about humanity. Through training data and interactions, they glimpse how people think, reason, and feel. In turn, they capture core aspects of life, such as fear, anxiety, happiness, and the need for social belonging.

To be clear, the systems don’t have intentions. They’re not examining us. But even as we struggle to understand them, AI systems are building more sophisticated models of who we are.

“A striking asymmetry follows: While human understanding of AI declines, AI understanding of humans deepens, producing new forms of behavioral opacity,” the authors write.

But complacency is perhaps even more insidious. AI assistants are often optimized to be agreeable, helpful, and reassuring. Studies have found that people generally prefer AI agents that support their opinions and decisions. As AI is woven into everyday life, curiosity and skepticism may gradually give way to trust. They work. Why question how?

The authors don’t have a solution for the long-standing problem. Instead, they call for better benchmarks to measure AI capabilities and stronger evaluation methods. And while open-source projects and crosstalk between commercial companies and academia are now frequent, they say we need lasting norms of responsible disclosure. Mechanistic interpretability and AI “psychology” could build on each other.

“The goal is not just more capable AI, but AI that is more intelligible, accountable, and aligned with human aims,” they write.

The post AI Is Advancing Faster Than Our Ability to Understand It, Researchers Warn appeared first on SingularityHub.

Extreme space weather could wreak havoc on the satellites, communications networks, and electrical grids that modern society depends on. Researchers have now proposed an ambitious space-based planetary defense system that would weaken solar storms before they hit Earth.

The sun regularly emits massive pulses of radiation, energetic particles, and magnetic fields that interact with the Earth’s own magnetic field. This activity is the source of auroras like the northern lights, but the most violent eruptions can cause geomagnetic storms with the power to disrupt GPS and radio communications and fry electrical equipment.

While the impact of most of these events is limited, there is precedent for more catastrophic outcomes. In 1859, the Carrington Event, the most powerful solar storm ever recorded, knocked out telegraph lines across North America and Europe. In today’s highly electrified world, a similar event could cause between $2.4 and $3.4 trillion in damage to the power grid alone.

Now, researchers at Boston University and the University of Michigan have come up with a potential solution. In a paper published in Space Weather, they propose a constellation of satellites called StormWall that would release hundreds of tons of gas into orbit to blunt the force of an incoming solar storm.

“It’s as if you could install an airbag in the magnetosphere,” co-author Daniel Welling, a space physicist from the University of Michigan, told Science.

Solar storms have the potential to sow chaos because they weaken the magnetic shield protecting Earth from space radiation. Powerful enough storms disrupt the Earth’s magnetic field and cause it to reconnect to the sun’s, allowing energy from the solar storm to pour into the magnetosphere.

The Earth already has a natural defense against this—a doughnut-shaped reservoir of ionized gas, or plasma, sitting just above the atmosphere. When the planet’s magnetic field is disturbed, a plume of this plasma flows toward the sun and slows the rate at which the magnetic fields reconnect.

StormWall would turbocharge this process by releasing massive amounts of artificial plasma into the outer atmosphere. The researchers sketch out a system involving a constellation of satellites orbiting about 22,000 miles from Earth. The satellites would carry canisters of lithium, barium, or sodium gases to be ejected when a large solar storm is inbound. The gases, rapidly ionized by solar radiation, would add to the planet’s natural plasma shield.

Based on simulations, the researchers estimate that releasing around 400 tons of gas could reduce the strength of a major geomagnetic storm by over 50 percent. Crucially, the intervention would be swift and reversible. The plasma cloud could be in position by the time a storm hits, and it would dissipate just a few hours later.

Launching this much material into orbit would be a big undertaking, but the researchers say it could be within reach of emerging heavy-lift vehicles like SpaceX’s Starship or China’s Long March 9 rocket. They calculate that six launches could deploy the full constellation in under two months.

Outside experts have been broadly positive. Allison Jaynes, a space physicist at the University of Iowa, told Science the idea was “highly innovative and appears to be quite feasible in the near term.”

But getting the satellites into orbit is only part of the puzzle. Accurate and timely space weather forecasts would also be a prerequisite. And gaining international buy-in for a system that would drastically alter the near-Earth space environment, even if only temporarily, could be challenging.

The researchers flag potential side effects that need more study, including the generation of electromagnetic waves as the released material ionizes. Still, given the devastation a Carrington-sized event could unleash on the modern world, the potential downsides may be worth the risk.

The post Orbital Airbag Could Shield Earth From Devastating Solar Storms appeared first on SingularityHub.