This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

When I landed in Seoul after a grueling 12-hour flight from San Francisco, I walked through an unmanned immigration checkpoint, where a machine scanned my face and passport. On the subway home, people were glued to their phones (powered by flawless 5G even underground), as we raced past platforms lined with LED screens of ads celebrating K-pop idols’ birthdays. When I got off the station in Gangnam, a cartoon-eyed robot on wheels was waiting patiently at a crosswalk to deliver someone’s dinner. Internet cafés dotted the sidewalks, crammed with teenagers playing computer games, maybe hoping to become the next legendary pro gamer.

I stood at a bus stop with interactive touch screens showing real-time bus schedule updates. It will soon become an “AI bus stop,” the Gangnam district announced in June, with a kiosk that answers riders’ questions in multiple languages. The news didn’t surprise me. Having grown up in the city, I’ve watched Seoul transform from a scrappy boomtown into the gleaming tech capital it is today.

South Korea loves AI.

While a public backlash against AI is brewing across the US, South Koreans are optimistic. Only 16% say they are more concerned than excited about AI—the lowest of any of the 25 countries surveyed by the Pew Research Center—while 50% of Americans were more worried than excited. A majority of Koreans use AI every day, either as a sort of personal assistant or to do tasks at work, according to surveys by the Ministry of Culture, Sports, and Tourism and Korea Chamber of Commerce and Industry.

One of the most wired countries in the world, South Korea loves to street-test every new technology on the block—AI webcomics, virtual K-pop idols, and humanoid monks. And the appetite for experimentation doesn’t stop with ordinary citizens. Government agencies are early adopters too, deploying AI textbooks in schools and AI eldercare robots in welfare centers. South Koreans share a deep conviction that embracing technology is integral to modernizing the country and cementing its place in the global order. Their fascination with AI is just the latest incarnation of that ethos—and it’s making them anxious to stay ahead.

Engineered enthusiasm

All this techno-optimism has largely been engineered by South Korea’s national agenda to make AI a motor of economic growth. “The South Korean government has designated an AI-powered Fourth Industrial Revolution as the country’s path forward and aggressively promoted and invested in it,” says Chihyung Jeon, a professor of science and technology policy at the Korea Advanced Institute of Science and Technology. “South Koreans have consistently and relentlessly been told by the government about AI’s potential to create a better future.”

As South Korea rose from the ashes of the Korean War, technology lifted the nation from poverty into an economic powerhouse. In the 1970s, South Korea manufactured steel and ships, then semiconductors in the 1980s, broadband in the 1990s, and smartphones in the 2000s. Today, Samsung and SK Hynix supply most of the world’s high-bandwidth memory chips, which power the cutting-edge Nvidia hardware used to train AI models. South Korea’s economy now orbits these two semiconductor giants: The country’s main equity index, Kospi, surged to record highs in 2026, powered by the soaring share prices of both companies, each valued above $1 trillion.

Lee Jae-myung, president of South Korea, has pledged to vault the country into the ranks of the “top three AI powers” alongside the US and China. After taking office in 2025, he launched the Presidential Council on National AI Strategy to help buy massive amounts of computing power and a sovereign AI foundation model project that funds Korean companies to develop homegrown AI models. The government has also supported semiconductor titans, including Samsung and SK Hynix, through generous tax credits and low-interest financing. 

South Korea’s policy posture also prioritizes accelerating AI development over safety considerations. In 2024, South Korea’s legislature passed the AI Basic Act, one of the world’s first comprehensive AI laws, to promote AI development and establish light-touch regulatory guardrails. Seventy percent of South Koreans say advancing science and medicine through AI innovation is a bigger priority than protecting industries through regulation, according to the 2026 Stanford AI Index.

All of that effort might be paying off. The same index ranked South Korea as having the third largest number of notable AI models in the world, based on criteria such as state-of-the-art advancements or high citation rates. For many small countries like South Korea, AI is a chance to punch above their weight.

The blind spots

But that single-mindedness can crowd out critical reflection on AI’s broader societal impacts. “Because the national agenda on AI prioritizes economic development,” says Jeon, the professor of science and technology policy, “there isn’t much reflection on the social, political, ethical dimensions of the technology.” In 2025, the South Korean government faced a fierce backlash for rolling out AI textbooks riddled with factual inaccuracies and data privacy risks without testing them first in a pilot program to evaluate how they affect student learning.

And despite their optimism, South Koreans are still worried that AI could displace them from their jobs. After Hyundai announced in January that it will deploy Atlas humanoid robots across its car factories, the Hyundai Motor Group union protested vehemently. “Without labor-management agreement, not a single robot using new technology will be allowed to enter the workplace,” the union said. Sixty-four percent of South Koreans fear AI could displace human labor and exacerbate inequality, although 52% believe it could also increase productivity. 

On a recent Friday night in the Seoul Central Market, I went out with my cousins to a pocha, a late-night restaurant that serves fish cakes stacked in neat pyramids. As we clinked our cups of soju cut with beer—the scrappy staple cocktail of every Korean night out—one cousin asked me if I’d asked ChatGPT about my saju, a traditional Korean fortune-telling practice.

A 29-year-old insurance agent in Seoul praying for a new job and a boyfriend, she said asking ChatGPT about work and dating was her favorite pastime. She pulled up her phone and punched my birth date into the chatbot. 

Addicted to their screens, trapped between unemployment and dead-end jobs, and priced out of marriage and homeownership, 46% of South Koreans in their 20s have used a chatbot to read their fortunes, according to a survey by Korea Gallup. 

My cousin said she also asks ChatGPT for tips on trading stocks, dreaming big about making bank on her investment accounts into which she’s been pouring her salary. ChatGPT, she believes, is her portal out of reality into a better future.

Despite how fond she is of the chatbot as her shaman and financial advisor, she fears losing her job to AI. She still uses ChatGPT feverishly at work, as all her coworkers do, afraid of falling behind. 

“I sometimes fear AI, but for now, it’s just so useful,” she said.

Google DeepMind is funding research into the potential dangers of situations where millions of different AI agents interact with each other online.

According to Rohin Shah, who directs the company’s AGI safety and alignment research, the mass-market arrival of agents that can carry out tasks without human oversight and follow instructions given to them by other agents creates a whole new class of risk.

In an effort to address this, Google DeepMind—which made agent-based tools a centerpiece of Google I/O last month—has teamed up with several other organizations to announce a $10 million funding pot for researchers to study the behavior of multi-agent systems and come up with ways to prevent unsafe scenarios. Joining Google DeepMind are Schmidt Sciences, a philanthropic foundation set up by Eric and Wendy Schmidt; ARIA, the UK government’s moonshot agency; the Cooperative AI foundation, a UK-based nonprofit research outfit; and Google’s charitable arm, Google.org.

I asked Shah and James Fox, who leads the Science of Trustworthy AI program at Schmidt Sciences, what they hope to achieve with that $10 million. It’s no small sum, but it’s dwarfed by the budgets commanded by Google DeepMind’s own research teams.

The aim is to kick-start research outside tech companies, says Shah: “The strength of academia is that it can look really quite far into the future and do the kind of work that isn’t top of mind at industry labs.”

“The main issue is that there just isn’t really a field of research for multi-agent safety yet,” he adds. “And we would like there to be.”

The concern is that as more and more AI agents get deployed and begin working together, we could hit a tipping point where imagined scenarios become real. “We see this with humanity, too,” says Shah. “Our institutions can accomplish things that no individual human can.”

Shah thinks we have a few more months to go before agents are deployed throughout the economy in numbers that make potential risks a real concern. He wants to get ahead of that moment.

Risky business

What risks are we talking about, exactly? The possibilities that Shah and Fox have in mind mostly boil down to supercharged versions of bad things that happen on the internet already: scams, prompt injections (where an AI agent is fed malicious instructions, turning it into a self-guiding piece of malware), other forms of cyberattack. We look at what humans do now and ask what the agent version of that would be, says Shah.  

“We’ve got this digital commons that is integral to how society works, and you really want to ensure that this doesn’t descend into just absolute anarchy,” says Fox.

(I asked Shah if they were considering any worst-case scenarios more on the doomer end of the spectrum, such as widespread economic collapse. “Certainly not if we’re talking by the end of the year,” he said. That’s only six months away! He laughed. “Okay, a while after that.”)

Shah and Fox both think that the only way to understand what might happen when large numbers of multi-agent systems interact with each other is to run realistic simulations. They want researchers to drop AI agents into sandboxes and study what they do.

You can’t predict what’s going to happen by studying single agents, or even small groups of agents, in isolation. You can’t assume that AI agents underpinned by LLMs will always act rationally, says Fox. And the complexity comes from having huge numbers of interactions at once.

Some researchers, including a team at Google DeepMind, have argued that artificial general intelligence (if possible at all) could come not from a single super-smart model but from a kind of agent hive mind, where the capabilities of the whole add up to more than the sum of its parts.  

Lack of trust

Google DeepMind is not the only top AI firm warning about the risks of the technology it is building. A couple of weeks ago, Anthropic published guidelines for deploying AI agents based on an approach to cybersecurity known as zero trust, which starts with the assumption that a computer system is vulnerable, an agent is an attacker, and a breach will happen.

Refael Angel, cofounder and CTO of Akeyless, a cybersecurity firm based in Tel Aviv, agrees that understanding the new risks introduced by agent-based systems is crucial.  

Every approach to security in the past has assumed that the machine in question was software written by a human, doing fixed things on fixed paths, says Angel: “An agent breaks all of those assumptions. It reasons, it improvises, and it can be hijacked by a single sentence buried in a document it was asked to read.”

Angel welcomes this new funding. “No single lab should author the safety standards everyone else has to trust,” he says. But he cautions that safety researchers can overlook boring problems that are already here in favor of more exotic hypothetical ones.

And yet, Fox notes, risks that were hypothetical a few years ago are now very real: “The future’s come more quickly than perhaps expected.”

As adoption of AI agents looks set to surge by as much as 300% in the next two years, leadership teams are carefully considering the implications of a hybrid human-AI workforce. 

Unlike existing enterprise-level automation that relies on manual input, AI agents are capable of autonomously coordinating complex tasks, interacting with multiple tools and environments across an organization. In early applications that center on customer service, HR, and sales, adoption of agentic AI has led to productivity gains of 30-50%. 

Their autonomy positions agents more as collaborators than tools, working side-by-side with human employees in blended teams that look poised to upend traditional workplace dynamics. 

More than three-quarters of HR leaders believe that the deployment of AI agents will transform existing workplace norms, driving a complete reappraisal of how roles and responsibilities are distributed, how skills are prioritized, and how workplace culture is shaped.

Though many admit they’re in the early or preparatory phase of this shift, 86% of chief HR officers predict that navigating digital labor shaped by agentic AI will be a central component of their role in the years ahead.

Fluency in the change management aspect of agentic AI adoption will be a crucial differentiator when it comes to unlocking the full potential of the technology going forward, believes Ateet Jayaswal, chief culture and employee experience officer at Wipro, a leading technology services and consulting company. This moment is one that he says, “calls for a mindset shift in how HR leaders would enable their organizations.”

Redeploying roles to enable higher-value work

As AI agents assume ownership of more complex and integral tasks, the distribution of roles and responsibilities within an organization will undergo significant change. It’s estimated that three-quarters of current roles will require redesign, reskilling, or redeployment by 2030 as a result of agentic AI. 

For leadership, this shift should be about reskilling employees toward higher-value work in order to optimize the potential of an agent-human hybrid workforce, says Jayaswal. 

For example, Wipro is a complex organization of 240,000 employees across 65 countries. It previously had multiple policies, documents, and knowledge fragmented across different systems, which delayed response to employee queries. 

But the company has recently integrated a custom agentic AI assistant—an agent co-created in partnership with enterprise agentic AI platform Ema Unlimited—that can swiftly navigate this complex system, assuming responsibility for 50 HR tasks that had previously fallen to human employees. With the help of an AI agent, average response time to queries has lowered from 48 hours to five seconds. 

Human employees have more time to focus on work “that requires a creative and imaginative mind and cross-functional collaboration, leveraging diverse ideas and thoughts to problem-solve,” says Jayaswal. The AI agent, meanwhile, handles rote administrative tasks like sorting timesheets or helping employees navigate policies and take actions in the flow of work. 

When reallocating employee responsibilities, though, it is imperative that humans remain in the loop, Jayaswal caveats. When agentic AI is incorporated into enterprise technology, it must work with sensitive and personal data and therefore needs even more stringent guardrails and constraints than consumer applications. “When you expose an AI agent to organizational data, when you integrate it into multiple enterprise systems, then pathways around the AI agent become extremely important,” he says. “It’s an evolving space that leadership needs to have front-of-mind.” Governance should include robust data privacy rules and the establishment of governance layers, such as an AI council, he suggests.  

At a fundamental level, the adoption of AI agents will force a re-evaluation of human roles, believes Jayaswal. Rather than employees primarily performing repetitive tasks or troubleshooting, a significant proportion of their time will shift to designing, teaching, and optimizing an AI agent that can do this work for them with far greater speed and predictability and without the agent getting bored. 

“The nature of your job changes from being the hero who comes in to solve the problem to designing the hero who can solve the problem,” he summarizes. “The individuals who I have seen thrive in this environment are the ones who make this shift.”

An evolving employee skillset

Just as roles and responsibilities will be reconfigured to reflect the input of AI agents, the core skills of human employees will be reprioritized. More than four in five HR leaders say they’re planning to reskill workers to become more competitive in a market shaped by AI agents. 

Technical skills will be increasingly important. Leading employers such as Salesforce, Danone, and Walmart are already rolling out dedicated AI and digital skills programs that aim to equip everyone from frontline workers to C-suite executives with a baseline level of AI literacy in response to the pervasiveness of the technology. 

But desirable soft skills will also evolve, Jayaswal points out. Employees who assign tasks to an AI agent need to plainly articulate what modular steps may be needed to accomplish a task, what the desired outcome should be, and what parameters or guardrails need to be in place to ensure the agent doesn’t access or share confidential data. 

As HR executives adapt to a blended workforce, three skills are emerging as top priorities during recruitment, according to a recent survey: relationship building, like forging constructive partnerships and account management; collaboration; and adaptability. 

Maintaining a healthy workplace culture

In freeing up human employees to focus on higher-value tasks, the hope is that AI agents can elevate the employee experience, deepening fulfilment and satisfaction in the workplace. 

“At Wipro, our vision is to improve the life of Wiproites,” says Jayaswal. “We are taking away non-value added work by embracing modern ways of collaborating, engaging, and transacting, leaving associates with higher order work content.” 

But leadership teams embracing agentic AI will also need to plan for the new pressures and stressors that the technology can place on a workforce. 

There is already confusion and knowledge gaps, with 73% of HR leaders reporting their employees don’t yet understand how digital labor will impact their work. Many organizations have opted to define AI agents as teammates or colleagues on org charts, but new research says this could erode trust and a sense of professional identity. It also raises new questions around accountability and ownership. 

The role of management in addressing these concerns is critical, says Jayaswal. To maintain healthy dynamics, managers need to become skilled at orchestrating blended systems, splitting their focus between supervising AI agents and motivating human employees as they also build and supervise AI agents.

Upgrading employee well-being programs will be a core part of maintaining a robust workplace culture. “As there are more interactions with AI agents, you are losing some of the human touch that was provided by service delivery partners or leaders, or often even by colleagues and peers,” Jayaswal says. Employee services that encourage social connection and empathetic communication may help teams navigate this. 

A breakneck transformation

Agentic AI looks set to scale at breakneck speed across many enterprises, and it will significantly transform how these organizations operate. 

Carefully considering and deciding how to adapt to this newly blended workforce is now a top priority for leadership teams. Reviewing and refining organizational strategies is essential for optimizing both technological gains and the employee experience.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. It was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.

At SXSW London last week I gave a talk called “Five things you need to know about AI,” in which I shared what I think are the biggest themes in AI right now.

I pulled a few things from our first AI10 list, an annual guide to the most important trends in this buzzy world, but I also veered off on a number of tangents. In my half-hour slot, I tried to cover the key talking points that I think help to make sense of what’s going on in tech—and thus the economy—today.  

(I gave a talk with the same title at SXSW London last year with five different things you needed to know. A lot has happened since then!)

So: This is how I’m thinking about AI midway through 2026. Let me know if you would pick different points!

1. Strictly speaking, I didn’t need to show up to give this talk.

Tongue in cheek? Maybe. But generative AI tools have already become mundane, used by millions to automate everyday office tasks (including producing and delivering talks). It’s no surprise that one of the biggest questions out there right now is what this all means for jobs. People are confused and scared.

The frustrating answer is that despite the hype coming from the top about the potential for AI to join the workforce soon—and viral social media posts yelling that something big is happening—there is almost no data to say either way what kind of effect this technology will have on employment and the economy overall. That’s not to say it won’t have an impact, even a huge one, but it’s just too soon to tell.

In theory, teams of agents working together toward common goals could become assembly lines for white-collar work, doing to offices this century what Henry Ford’s innovations did to factories in the 20th century.

In theory. Because in order to know what will happen to jobs, we need to know what will happen inside the companies that create those jobs. But most companies are still figuring that out.

 2. AI is getting scary (for real this time).

There have been scary stories about AI for years—claims that it will kill us all or bring about the end of civilization. There’s still a loud crowd of doomers, but those scenarios remain dystopian science fiction.

What’s happened instead is that many of the worst near-term, real-world fears have come true.

Take deepfakes, AI-generated images or videos of people doing things they didn’t actually do. Deepfakes have been used to incite violence, swing votes, and sow distrust. Trump’s White House is among those creating and publishing fake images.

Many deepfakes are also used to abuse women and girls. One study found that 98% of deepfakes are pornographic and 99% involve women.

Another concern is the rise of dangerous and delusional relationships with chatbots. Many people turn to chatbots to seek private advice and to feel heard. But there are now multiple lawsuits against AI companies alleging that the technology encouraged or aided suicides and other forms of self-harm.

AI is also being used in warfare in new and worrying ways. LLMs are now giving advice, not just being used for analysis. One US defense official told my colleague James O’Donnell that you could now give a military chatbot a list of targets and ask which one to hit first. Anyone who uses AI knows that its output needs to be reviewed carefully. In fact-paced, high-stress active conflict, the risk that corners get cut is high.

3. A lot of people really hate AI.

I checked out an anti-AI protest in London earlier this year and found a very broad mix of complaints. Banners proclaiming the end times bounced along to chants of “Stop the slop! Stop the slop!” Protests are getting more organized and drawing larger crowds.

There’s pushback from fans of films and video games, who object to the use of generative AI in their favorite titles. In one notable case, the acclaimed 2025 game Clair Obscur was stripped of an award when the developers admitted to using AI in just one small, specific part of its production.

And there’s the data center backlash. The US has more than 5,400 data centers and counting. With the energy demands of AI growing, people are unhappy about the environmental impact and their rising electricity bills. Activists are managing to stall development in a number of places.

Regulation is becoming politically popular. Grassroots movements like QuitGPT have gained momentum. A small number have turned to violence; a few weeks ago somebody threw a Molotov cocktail at Sam Altman’s house. It’s not clear where all this leads. But the apocalyptic hype from tech leaders is not helping people stay calm.

4. AI for science is a very big deal.

It’s early days yet, but the potential for AI to help make a genuine and important scientific discovery is greater than ever.

Google DeepMind has developed Co-Scientist, a multipurpose tool that can help researchers dig up and compare previous results, generate hypotheses, and devise experiments to test them. OpenAI told me this year that its North Star is the goal of building a fully automated researcher by 2028.

Mathematicians are excited too. Fundamental math underpins many everyday technologies, from internet security to video streaming. The last few months have seen a string of claims that AI has cracked unsolved math problems. And software that can solve really hard math problems will be able—so the argument goes—to solve more general-purpose real-world problems too.

What are the downsides? Some scientists are warning that an overreliance on AI tools could narrow the scope of research because scientists may choose problems that are most suited to AI assistance. There are also concerns that AI-assisted research will lead to a flood of inaccurate or fake results: science slop.

5. AI is everywhere all at once.

So where does that leave us? There are a lot of exciting things, a lot of worrying things, and a lot of hot air. It can be exhausting to keep up, and yet it all feels inescapable. Some people will tell you we’re in a race to the top; some will tell you we’re in a race to the bottom. But it’s really not clear where we’re headed.

AI companies want us to march to their tune and buy into the propaganda about artificial general intelligence, whatever that means. They are selling a vision that feels inevitable, but it isn’t.

We’ve built a technology that can do humanlike things, and I think that makes it hard to get our heads around the fact that it is still just a technology.

Something is happening. Maybe even something comparable to the invention of electricity or the internet. But technologies like that take time to settle and bring lasting change.

Get ready for a marathon, not a sprint.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

On June 5, 404 Media reported that attackers had been using Meta’s AI customer support agent to steal Instagram accounts. Their approach was simple: They asked the agent to link the accounts to email addresses that they controlled, and the agent complied. One attacker broke into the dormant Obama White House account and made pro-Iran posts; others took over accounts with valuable, single-word handles, possibly in order to sell them.

AI cybersecurity concerns are nothing new. Since Anthropic announced in April that its Mythos model was too good at hacking to be released to the general public, commentators, researchers, and federal officials alike have fixated on the idea that superpowered AI systems could lay waste to our computer infrastructure. That’s not quite what this Instagram hack was: There, AI was the target rather than the attacker, and the method was far simpler than anything Mythos would cook up. But as companies offload more work to AI, these comparatively unsophisticated attacks could wreak their own havoc.

“As AI becomes more and more widely used—especially when AI is more and more widely used to automate our work flows, like account recovery—I think attackers are going to be more and more motivated to attack AI itself,” says Neil Gong, a professor of electrical and computer engineering at Duke University.

Gong and other scholars have been issuing warnings about the security vulnerabilities of AI agents for a while. They publish papers and blog posts detailing exploits such as indirect prompt injection, which involves hijacking agents using commands hidden in websites, emails, or other seemingly anodyne data sources. Compared with these techniques, the Meta hack was practically mindless. The only complication that hackers had to overcome was using a VPN that matched the true account owner’s location; then they directly asked the support agent to change the account’s email address, and it complied.

Meta has not commented publicly on how this vulnerability slipped through the cracks. But given the simplicity of the exploit, Gong says, it should have been uncovered easily, before the agent was deployed. “It’s really surprising,” he says. “I don’t understand why they didn’t find this simple problem.”

Jessica Ji, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, agrees. “It raises questions like: Were there even guardrails in place?” she says. “Did anyone think to test for this kind of scenario?” She notes that the oversight is particularly striking coming from a company like Meta, which has extensive expertise in both AI and cybersecurity. Meta did not respond to a request for comment for this article, but on Monday a Meta spokesperson said on X that the vulnerability had been resolved.

As embarrassing a moment as this might be for Meta in particular, it also highlights some core vulnerabilities shared by all AI agents. Unlike traditional software, agents can respond in flexible—and unexpected—ways to new circumstances, which is why they might be able to substitute for human customer support agents. But AI agents can also be tricked in ways that humans wouldn’t be, and because they can take real-world actions, those mistakes have consequences. “A human would say, ‘Okay, why do you want to change the email address?’ and maybe respond with a security question,” says Somesh Jha, a professor of computer science at the University of Wisconsin–Madison. “What is going on with these agents is they’re very eager to finish the task. It’s almost like some elementary school student who just wants to please the teacher.”

But there are also countervailing forces. Companies want to deploy capable agents, and the more power an agent has—and the fewer guardrails it is subject to—the more work it can potentially take on. “Security and utility always have a trade-off,” says Bo Li, a professor of computer science at the  University of Illinois Urbana-Champaign. And adequate red-teaming can be expensive. Defenders have to expend more resources than attackers do, because attackers only need to discover a single exploit, while defenders try to discover and patch as many as they can. When attackers are working toward something as valuable as a single-word Instagram handle, they’ll pour resources into finding exploits, so defenders have to spend even more money to protect that prize. 

As AI models continue to improve, hardening their defenses might actually get easier. Though the probabilistic nature of large language models means that LLM agents will always be vulnerable to some forms of attack, a more sophisticated model might have identified an attempt to change the email associated with the Obama White House account as suspicious. And AI systems can be used for agent red-teaming, much as participants in Anthropic’s Project Glasswing use Mythos to identify vulnerabilities in their software. 

Still, experts expect that the problem of securing AI agents will only become more pressing in the future. As agents grow more capable, companies that adopt them may want to give them more power, both to provide more services with fewer humans and to avoid being left behind by their competitors. In the fast-moving world of AI, the time needed to carefully secure risky agentic systems might seem like an unconscionable delay.

“Everybody wants to be the first to do something and just push things out without careful scrutiny and red-teaming,” Jha says. “I think it’s a very dangerous thing.”

Most days in her chambers, Judge Maritza Braswell, a federal magistrate judge in Colorado, sifts through stacks of documents written by people without a lawyer. Many of them can’t afford to hire a lawyer, and others have cases too weak or too small to interest one. She reads each one carefully, mindful of how daunting it is to walk into the courtroom alone. 

Lately, like many judges across the US, she has seen a noticeable uptick in such filings. According to a new study that examined 4.5 million federal civil cases from 2005 to 2026, the share of lawsuits brought by self-represented people increased from 11% in 2022 to 16.8% in 2025. Within those cases, the number of filings made more than doubled from pre-2023 levels. 

Judge Braswell puts that jump down to AI. 

“I do correlate that to AI in part because I see AI use,” she says. As a tech-savvy judge who uses AI to vet court documents, she’s learned to recognize how large language models write. She can tell from the prose and at times, hallucinated cases and fabricated quotes. 

“I’m also actually seeing better-drafted pleadings,” she says. 

But while AI appears to be expanding access to justice, it doesn’t seem to be improving people’s chances of winning. Judges are also starting to question what kinds of rights and responsibilities large language models should bear as they step into lawyers’ shoes. For example, they ask whether a chatbot has a duty to provide good advice, as a human lawyer does. And a growing number of lawmakers across the US are starting to grapple with who should pay the price when chatbots dish out bad legal advice. 

AI supercharges lawsuits

To test whether AI was driving the increase in lawsuits filed by people without a lawyer, the authors of the study, Anand Shah at MIT and Joshua Levy at the University of Southern California, ran 1,600 randomly sampled court documents through Pangram, a commercial AI-text detector. The share flagged as containing AI-generated writing rose from 1% in 2023 to 18% in 2026. 

To Judge Braswell, that’s not necessarily a cause for concern. While the surge of AI-assisted filings might be adding to their workloads, she and many other judges find the cases easier to rule on because AI is helping people without legal training better articulate their arguments. 

Court documents written by people without lawyers are notoriously hard to decipher. Some arrive as handwritten scrawls bordering on gibberish that judges take a while to decode. However cryptic, judges are required to read them charitably.

These days, Judge Braswell has been churning through motions drafted by AI faster than the ones written by the litigants. “I have to be really careful because some of them contain hallucinations and errors, but I can generally understand what they’re arguing better with AI assistance from them than without it,” she says.

The clearer filings let Judge Braswell hear them better. “If I understand an argument a little bit better, I’m probably going to be able to help a little bit more,” she says.

Online communities are springing up to trade self-help guides on using AI to sue. In December 2024, a viral Reddit post walked immigration applicants through suing the United States Citizenship and Immigration Services over delayed review of their applications: draft a writ of mandamus with Microsoft Copilot, pay a lawyer $150 to polish it, and file in the expedient District of Vermont. Cases filed by people without lawyers in Vermont rose from about 45 a year before 2022 to more than 1,100 in 2024. 

Even so, people without lawyers are far more likely to lose their case than people with lawyers, and that’s not changing even with the addition of AI, the study found. 

“It turns out that mounting a lawsuit is a complex, multifaceted task. Not all of it is just drafting text,” says Levy. 

Chatbot-client privilege

Judge William Garfinkel, a federal magistrate judge in Connecticut, has served on the bench for three decades, pondering all sorts of questions about lawyers’ relationship with their clients. Lately, he has been wondering whether people’s conversations with chatbots dispensing legal advice should be privileged, the way their conversations with lawyers are. 

“You can make a good argument that … conversations with large language models like Claude or ChatGPT or Grok should deserve some protection,” he says.

Courts are starting to grapple with this question. In February, a federal court in Michigan ruled that a self-represented person’s conversations with ChatGPT to prepare her case were work product—legal work that is shielded from the opposing side.

The decision came on the same day a federal court in New York held that documents a criminal defendant had generated using Claude were not privileged attorney-client conversations or work product. The court argued that Claude is not an attorney and that a user has no “reasonable expectation of confidentiality in his communication” with it because AI companies can disclose user data to third parties. 

In March, Judge Braswell ruled that a self-represented person’s use of a chatbot should stay off limits. “It is true that AI systems like ChatGPT, Claude, Gemini, and others … collect user data for training and other purposes. But … that does not eliminate all expectations of privacy,” she wrote. Courts have since remained split on the issue.

Malpractice without a pulse

Some judges are also wondering whether a chatbot, like a lawyer, has a duty to provide good legal advice. Judge Allison Goddard, a federal magistrate judge in California, has noticed that people without lawyers often get the wrong advice from ChatGPT when trying to assess the value of their case during settlement negotiations. In one case, a plaintiff who slipped and fell in a store asked for $700,000 from the store, which was wildly more than the case was worth.

“Where are you getting the idea that you’re getting $700,000? Did you go to ChatGPT?” Judge Goddard asked. “Well …” the plaintiff mumbled. She then walked the person through the law to explain why ChatGPT was wrong and suggested a lower amount. “It’s like Dr. Google went to law school,” she says.

Then there’s the question of who’s liable when a chatbot makes such mistakes. In March, Nippon Life Insurance Company sued OpenAI alleging that ChatGPT practiced law without a license and helped a woman reopen a lawsuit that was already settled, flooding the court with frivolous filings. “ChatGPT is not an attorney,” the lawsuit said. 

In May, OpenAI asked the court to dismiss the case, arguing that ChatGPT does not practice law. “ChatGPT is not a person and neither has nor uses any degree of legal ​knowledge or skill,” OpenAI said in its filing. The case is still pending before the court.

States have started to weigh legislation that would hold AI companies liable when their chatbots offer bad legal advice. New York introduced a bill in March that would bar chatbots from impersonating lawyers, even if they notify ​users that they are interacting with chatbots. In Congress, a series of bills have been proposed to ban chatbots from posing as lawyers, doctors, and other licensed professionals. The bills have yet to gain traction.

For now, people will continue turning to AI to be their lawyer. For many of them, the rewards outweigh the risks. Not long ago, when Judge Braswell asked self-represented litigants why they wanted a particular piece of evidence, they mumbled timidly. Now, they answer her questions confidently, having rehearsed with a chatbot. 

“This is a really tough system to navigate. With AI, though, it gets a little less complex,” she says.

The global health care sector is under increasing strain. 

Decades of chronic underinvestment and constraints in recruitment have coincided with a surge in demand for services for aging populations. Gaps in provision are already taking a toll, with fragmented access to care and high rates of stress and burnout among staff. And it’s getting worse. The World Health Organization has warned that current shortfalls will increase to 11 million workers by 2030. 

In their urgent hunt for a solution, many health-care providers are now pinning their hopes on agentic AI, with more than two-thirds (68%) having already adopted AI agents into their workforce, according to KPMG. 

The technology is being deployed to automate complex back-office processes, collaborate with medical teams, and even triage patients, all in a bid to reduce the cognitive load on clinicians and improve quality of care for patients as the supply of human health-care workers dwindles.

A different type of digitalization 

Until now, the benefits of digitalization within health care have been limited. 

Many staff have blamed slow or outdated technology for adding to the administrative burden rather than alleviating it. For example, U.S. patient data was migrated to electronic health records (EHRs) in the early 2000s, but this data remains fragmented and reliant on manual inputs. 

New telehealth services and digital care tools, like remote monitors, have had similar shortcomings, says Ashis Barad, MD, chief digital and technology officer at Hospital for Special Surgery (HSS), an academic medical center in New York that focuses on musculoskeletal health. Both technologies have helped improve access to health care by removing geographical barriers, he says, but they’ve failed to replicate the quality of in-person care or win trust from patients. 

Agentic AI is different from these existing technologies, he insists. 

Rather than relying on manual inputs or defaulting to human workers for any case that sits slightly outside a rigid framework, AI agents can handle nuanced, complex scenarios. They can make autonomous decisions, retrieve information from expert clinical sources, and iterate over time, freeing clinicians to focus on higher-level patient care. As Dr. Barad puts it: “Agentic AI takes your workflow and collapses it, augments it, supercharges it, and makes it more performant.” 

At HSS, AI agents have already been deployed in multiple areas. They handle complex backend processes, such as insurance claims that previously took several weeks to complete and involved both HSS staff and a third-party contractor to handle the volume. Now, says Dr. Barad, AI agents complete 1,100 claims per month. They’ve reduced the appeals stage from 45 minutes to five and improved the success rate of those appeals from 65% to 100% in the nine months since implementation. HSS now handles all claims in-house. 

Building on that success, HSS is now deploying AI agents in non-clinical patient-facing settings with an AI scheduling and triage service, as part of a collaboration with enterprise agentic AI developer Ema Unlimited. The service is accessible 24/7 via web, text, or phone. It uses conversational AI to ask patients clarifying questions about their condition and then books appointments with the most appropriate clinician, factoring in location, insurance coverage, and physician availability. “It completes the whole loop,” says Dr. Barad. The AI agent is trained on “all of our context, all of our rules, and all of our knowledge base,” he adds, providing patients with streamlined access to highly specialist knowledge from world-leading surgeons.

Given the high-stakes decisions delegated to AI agents, the triage service has built-in safeguards—sensitive, complex, or uncertain scenarios are escalated to human specialists. Every decision made by the AI agent is auditable and human staff can step in at any point. Patient data is kept secure and the system is trained on all HSS protocols, policies, and care pathways. By keeping humans in the loop, Ema says its technology strikes the balance between efficient automation, patient-first safety, and human-informed decision making. 

As the technology becomes more prolific, it will be incumbent on providers to ensure they have these sorts of guardrails embedded into systems, says Dr. Barad. At HSS all decisions around the technology are filtered through an AI subcommittee that Dr. Barad co-chairs alongside a senior nursing executive. AI agents that may touch on patient care will be scrutinized with far more rigor than, say, backend processes, he explains.

AI agents prompt systems-level change

For example, Dr. Barad has plans to create a dedicated AI lab at the HSS main campus in New York City—a move that aims to democratize access to the technology across the organization. It will be open to all staff looking to understand or build AI agents, he explains, with informative classes and one-on-one training. “We’re getting agentic AI into everybody’s hands,” he says. This echoes research by Deloitte, which found that leading agentic AI adopters in health care were far more likely to have opted for multiagent solutions, redesigning end-to-end workflows rather than sticking to narrow solutions or individual use cases.

The key, it appears, is to integrate AI agents across the entire enterprise, treating them as a general-purpose technology. As Dr. Barad puts it: “It’s wrong to think of agentic AI in use cases… It’s a general-purpose technology, analogous to electricity.”

In practice, this means health-care providers need to set the right foundation to achieve value with agentic AI. This includes creating a unified data strategy, one that integrates fragmented data sources across an organization to create a single, comprehensive source of truth. In health care, data is often split across multiple departments and providers, each with their own legacy IT system.

In systems that rely on fragmented data sources, metrics often lack standardized definitions too. For example, Dr. Barad says that each hospital he’s worked in has had a slightly different definition for “time to start surgery,” a metric commonly used to gauge operating room efficiency. This level of fragmentation impedes AI agents from retrieving information from different sources or applications and assimilating the tacit knowledge that differentiates them from other technologies.

By creating greater interoperability of data at HSS, patient-facing AI agents can draw from a patient’s clinical care history and existing recommendations from their clinician, combine this information with current symptoms, and decide whether a situation requires escalation before notifying the correct specialist and informing the patient. 

Building better outcomes

For Dr. Barad, the potential for AI agents to overhaul health care and alleviate the current pressures on resources, access, and patient care is huge. 

He envisions a future in which 90% of non-clinical health-care tasks could be administered by AI agents, freeing clinicians up for what he calls white-glove work, meaning the most complex, specialized, and sensitive cases.

Most health-care providers seem equally optimistic. According to research by KPMG, 84% of providers are already comfortable handing decision making about specific processes over to AI agents.

“We’re spending so much time on keyboards and computers right now that we’re actually not doing what we should be doing,” says Dr. Barad. “This is going to rehumanize health care.”

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. It was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.

This article is from Making AI Work, MIT Technology Review’s limited-run newsletter examining how to apply LLMs across industries. To receive it in your inbox,sign up here.

From accounting to design to market research and product development, there’s a staggering breadth of skills needed to run a business. A large company can hire experts to handle these tasks, but small businesses don’t always have this luxury.

That’s where AI comes in. Today’s AI models do a decent job at these tasks. The trick for small businesses is to understand where AI is good enough and where it’s not.

One place where a “good enough” AI can already be quite valuable to small business owners is in providing secretarial skills and handling basic administrative matters. Let’s take a look at how one private tutor is using it to improve his recordkeeping and free up his time.

Case study

Sam Finnegan-Dehn works in fundraising for a charity, but he moonlights as a math and philosophy tutor for university students from his home in London. Through this part-time business, he can leverage his degrees in philosophy and share his love of the subject with clients.

But meeting with students is only a fraction of the work it takes to be a good tutor. He also plans lessons and finds fresh reading materials, creates assignments, sends invoices, and keeps up with new research—all on top of his regular job. Given these demands, Finnegan-Dehn doesn’t have as much time as he’d like to grow his tutoring roster.

So he’s turned to AI for some help in managing the day-to-day aspects of his business. He says AI has taken on a secretarial role across all of his digital notebooks, where he jots down reminders about his clients’ progress and new readings to keep himself up-to-date. He describes using AI as kind of like having a second memory that helps him connect ideas he’s written down in various places.

While he has experimented with different tools like Claude and ChatGPT, he’s now landed on Notion AI because it integrates better with his tutoring notes, which live across his notebook tabs in the Notion app. Finnegan-Dehn doesn’t use AI to create teaching materials, but he does let Notion AI record meetings with his clients (after getting their consent), and then uses its automated summaries to refine his teaching strategy. For example, if he notices from the AI’s summary that it seems like a certain technique was not helping a student, he may change how he approaches the subject next time.

Beyond this, Notion AI also helps him with goal-setting, drafting lesson notes, invoicing, and generating and syncing social media posts. For goal-setting, for example, Finnegan-Dehn says he understands his long-term goals for his business but not always the concrete steps to build to them. He uses AI to help fill in these gaps. He starts by writing down a “North Star” goal—say, to have a certain number of clients by the end of the year. Next, he asks his AI to generate the steps that he needs to take to get there, given the profile he has built up in the app. Then, he can reflect on the results and choose which tasks to tackle first.

The tool

Notion has been a big player in note-taking software for many years. Its AI add-on, released in late 2023, now has tools that enable it to interact with many other online productivity platforms. There’s an email client, calendar integrations, and a newly released agent. And while this level of access has raised privacy concerns, it can also make for a pretty powerful virtual assistant.

Many of the tasks targeted by Notion AI are less creative and more rote: syncing information across documents or searching through old scribbles, for example. This makes the tool especially appealing to small business owners, who have limited bandwidth, particularly for menial work.

Other companies are developing tools targeted at specific industries. For example, Grandma’s Quilt Shop in Yuma, Arizona, uses Rain, which has a software suite tailored to craft companies, to generate inventory descriptions and pricing for its stock of fabric designs. The owners claim this AI tool cuts the time it takes to list items by 60 to 80%.

There are drawbacks, though, as Finnegan-Dehn described some of Notion AI’s idiosyncrasies as “clunky” at times. And the AI add-on for Notion costs $20 per month. As with all new tools, small business owners should carefully assess how the potential gains and headaches measure up against the cost of just doing the job themselves.

User tips

Consider these points when thinking about whether AI might be able to help you run a business, or make any part of your work life just a little bit easier. 

• Look before you leap. Since LLMs feed on the data you input to answer your queries or complete tasks, you want to give them information in a way that’s convenient for you and for the model. For many of these notebook AI services, this means, for example, using their platform for notetaking so you don’t have to input or upload notes later. Because of this, it’s a good idea to weigh your options carefully before committing to an AI-powered ecosystem.
• Work to your strengths. Think about what skills you lack in-house, and see if AI can either help with training or take these tasks on for you. Just be aware: AI hallucinates and makes mistakes, so think about where accuracy is needed and keep humans in charge there.
  
• AI isn’t always the best tool. It’s okay to use something off the shelf when that’s the better choice. It’s going to be safer, for example, to use existing payment processing platforms like Shopify or Square than to vibe-code one using AI.
  
• Consider using local models for any sensitive information. Our reporting has covered the risks that online AI models have in leaking sensitive data, and there have been many reports about how AI companies collect your data when you ask their chatbots questions. Even if your business doesn’t handle personal information, there can still be some things you’d prefer not to share publicly. In these cases, using an open-source model that makes inferences on your prompts locally can be a great option, instead of ChatGPT or Claude or other proprietary models. Thankfully, some LLMs can now be run off of laptops and small desktops. Here’s how to set one up and start using it.

Sign up for Making AI Work, MIT Technology Review’s limited-run newsletter examining how to apply LLMs across healthcare, climate tech, education, and more.