Last week, Australia’s Fair Work Commission ruled Gregory Baker, a computing academic at Macquarie University, should be treated as an ongoing, part-time employee, after the university had earlier declined his request to convert from a casual role.

It was immediately described as a “landmark” decision, the first test of Labor’s “employee choice pathway” reforms passed in 2024.

But the ruling also made headlines for other reasons. Baker represented himself at the tribunal and has said he won with the help of trained artificial intelligence agents. His success again has us asking: Can AI replace lawyers?

On closer scrutiny, Baker’s case looks less like evidence of AI replacing lawyers and more like a powerful illustration of how a highly capable user can employ AI tools to terrific effect.

Request Denied

Speaking to the Australian Financial Review following the ruling, Baker said it was actually an AI tool that alerted him to the possibility of converting his role from casual to permanent part-time in the first place.

He had been teaching computer science at Macquarie University over consecutive semesters from 2023 to 2025, and in November 2025, he gave the university the required notice that he believed his work no longer met the requirements of casual employment.

The university did not accept this notification and Baker lodged a dispute at the Fair Work Commission—without a lawyer—in December 2025. The parties could not reach agreement, and the case went to arbitration on May 12. A decision was handed down last Wednesday.

Expert Use of AI

Baker has said he won by using multiple paid AI agents, such as OpenAI’s paid offering, ChatGPT Pro. This “team” helped assemble his case, follow up references, and anticipate his employer’s counterarguments.

His victory has been celebrated as historic, with the Australian Financial Review describing it as “the first known successful use of technology by a self-represented person in the legal arena.”

However, a few things set this particular case apart. Baker’s IT background, expertise managing AI agents, and ability to optimize their use for his case represent a rare level of expertise in using AI in a legal context.

Details included in the Fair Work Commission’s decision also suggest he kept his legal argument narrowly focused on teaching he’d done in one particular unit.

Less expert use of AI in court often sees those bringing claims produce “kitchen sink”-style arguments, which include weak, exaggerated, and nonsense claims.

Baker’s dispute was also narrow, limited to the application of a casual conversion law that had not yet been tested. Importantly, the Fair Work Commission (a tribunal, not a court) is designed to be user-friendly, to enable workers to bring claims without a lawyer.

Less Positive Attention

Elsewhere, the use of generative AI in legal proceedings is attracting a lot of attention for less positive reasons.

Most of this attention centers on the damage caused by inaccuracies, hallucinations and “AI slop”, and how courts and tribunals should best respond.

By making it easier to put a case together, AI has removed traditional access barriers for some litigants. But while case numbers are going up, case precision and quality is going down, making it harder to manage disputes to resolution.

Courts and tribunals are struggling with the volume. At the Fair Work Commission alone, workload has reportedly increased by 70 percent over three years.

New challenges are emerging as time goes on. Reports suggest litigants and lawyers in some overseas jurisdictions are embedding prompts in digital documents (something called “prompt injection”) to overcome or manipulate AI-based review systems some courts use to process documents.

A Big Opportunity

Baker’s example shows us something significant. Used well, AI tools can empower people with narrow legal disputes and digital skills to achieve successful resolutions at low cost.

This is an important development in access to justice. In Australia, there is a huge gap between the number of people with legal problems and the very limited funding available for legal assistance.

Most of the community is in the “missing middle,” unable to afford private legal assistance but on incomes too high to qualify for free legal aid.

AI tools stand a good chance of helping people with sufficient legal capability with problems and cases—like Gregory Baker’s—that are a good fit for the solutions AI can offer. These are few and far between, however.

Where Might Things Be Headed?

We should expect case numbers and self-representation in courts and tribunals will continue to grow and expand beyond Fair Work.

While there will be some baseless cases, the growth also represents the natural consequence of removing one traditional access barrier to our formal justice institutions—getting in the front door to start proceedings.

The bigger picture challenges are persistent and raise important questions. Who will most benefit from the capacity of AI tools to enhance access to justice, and who will continue to struggle to get basic legal problems resolved?

For courts and tribunals, the challenge will be striking a balance between managing caseloads and delivering justice, while not wasting the opportunity to expand access to justice.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The post An ‘AI Legal Team’ Has Won Its First Case. It’s a Rare Victory for Access to Justice. appeared first on SingularityHub.

This week on “Uncanny Valley,” senior writer Will Knight talks his recent visit to China and the future of AI collaboration.

Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it hasn’t seen before.

On Thursday, Google DeepMind showed off what the company calls the first double-blind evaluation of a proprietary frontier-class AI model. The setup keeps Gemini’s model weights hidden from the evaluators while the test questions are hidden from Google.

The pilot tested Gemini 2.5 Flash Lite against private benchmarks from MLCommons and the Singapore AI Safety Institute, but rather than using the results to tout a new Gemini score, Google is focusing on how the tests were run, with neither side having access to the other’s data.

Benchmark leakage inflates scores

Google’s technical report cites earlier research that found signs of benchmark leakage in about half of the 31 models tested, and another study published this year that found contamination can inflate scores, particularly for larger models. This dynamic has already raised questions about where certain post-training coding gains actually come from.

Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult. With a closed model, evaluators usually have to run their questions through the provider’s API, which means the company can see a test that was supposed to stay private, whereas running it independently would require the provider to hand over its model weights. Google’s setup is meant to give both sides another option.

Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult.

How the enclave works

The pilot is using Google Cloud Confidential Space together with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption.

Google DeepMind provides Gemini with its weights and inference code, while the evaluator offers its benchmark prompts and evaluation code; these are then transmitted over encrypted connections into the enclave, where the evaluation takes place without either party gaining access to the other’s protected assets.

The model weights are stored in hardware-encrypted GPU memory, and the evaluation prompts are kept in encrypted host memory as well. After the test is complete, the evaluator is given the permitted results, and the temporary environment can then be destroyed.

Before either side sends over its private data, remote attestation verifies that the enclave is running the software they agreed on.

Code controls beyond encryption

The enclave protects what’s stored in memory, but the code running inside it can still create problems. If an application has unrestricted network access, for example, it could send sensitive information somewhere else.

OpenMined’s PySyft handles that part of the process by allowing Google and the evaluator to approve the code in advance and to block sensitive parts of the evaluation from making external connections.

That process takes some work, although compute doesn’t appear to be the problem. The paper puts the overhead at less than 5% and instead points to legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.

The researchers eventually want to make attestation much less hands-on, comparing the goal to the HTTPS lock icon in a browser, where the security checks happen without the user having to deal with the underlying hashes and keys.

Google still has a hand in verification because, although the Confidential Space guest OS is open source and its build process has been externally validated, individual builds rely on private signing keys and can’t be independently reproduced. At the same time, Google’s own services are used to sign and verify the attestation report.

The paper puts the overhead at less than 5% and points instead to the legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.

Scaling past one GPU

So while the system reduces how much the two sides need to trust each other, it doesn’t remove trust entirely from the equation. Some of it simply moves elsewhere, including to the hardware itself. There is still some trust involved in the hardware itself, including the assumption that the cloud provider and the hardware maker aren’t working together to circumvent the protections.

Then there’s the benchmark itself. MLCommons points out that it still needs to be carefully managed, no matter how well the questions are protected.

The pilot ran Gemini 2.5 Flash Lite on a single H100 80GB Confidential GPU. Still, the researchers are already looking at clusters of H100 and B200 GPUs connected via encrypted links to eventually evaluate models that are too large to fit on a single GPU.

If this approach catches on, developers could have another way to look beyond the benchmark score, with some proof that the company behind the model didn’t see the test before the results came out.

If this approach catches on, a benchmark score could offer some proof that the company behind the model didn’t get to see the test first.

The post Google found a way to test Gemini without seeing the questions appeared first on The New Stack.

We use the equation all the time. But where did it actually come from?

The post The Sigmoid Function: From 'e' to Neural Networks appeared first on Towards Data Science.

After an affair with a fellow police officer ended, a Georgia cop used Flock to track her movements—and those of a man whose vehicle often showed up near hers, internal investigation records show.

The potential for AI to automate scientific research and manufacturing must be balanced with new risks, Anthropic says.

Astroparticle physics sits at the exciting intersection of astrophysics and particle physics and stu...

Earth AI

What a final-year project taught me about the gap between evaluation metrics and production decisions

The post I Trained Six Models for Fraud Detection, and the Best One Isn't in Production appeared first on Towards Data Science.

Code reviewed by WIRED reveals the company is developing a feature that enables Codex to continue working proactively until it is “put to sleep.”

AI coding company Replit is throwing its weight behind the model-routing trend by making its “intelligent model routing” system the default across every account.

The system automatically selects the underlying model to handle a task as it evolves, with Replit weighing quality, speed, and cost in its routing decisions.

The company says the feature, dubbed Auto mode, will become the default option for all users, though Core and Pro subscribers can still override it and manually select models when they want more control.

Model-routing momentum

The announcement comes hot on the heels of a flurry of activity in the model-routing realm. Earlier in August, Stripe agreed to acquire model gateway platform OpenRouter for a reported $8 billion. On the very same day, Ramp launched Router.com, which routes requests to the lowest-cost model that meets a specified performance bar.

Before all that, in July, SpaceX-owned Cursor launched its own router, which automatically selects models for coding requests and claims to deliver comparable performance at a substantially lower cost. Meanwhile, Meta is reportedly developing an internal router called “Switchboard” that scores coding tasks by difficulty and sends simpler jobs to cheaper models.

“Across one model family, per-token rates can span orders of magnitude. At the same time, the intelligence of cheaper, smaller models is now much closer to their larger frontier counterparts, providing us a lot of room for cost optimizations.”

Michele Catasta, president and head of AI at Replit, says that one reason for the wider push into routing is simple economics — the growing gap between what models cost and the level of capability developers actually need for a given task.

“Across one model family, per-token rates can span orders of magnitude,” Catasta tells The New Stack. “At the same time, the intelligence of cheaper, smaller models is now much closer to their larger frontier counterparts, providing us a lot of room for cost optimizations.”

Replit, for its part, has been moving in this direction for some time. Catasta says that the company has spent recent months experimenting with early versions of Auto mode, subagent routing, and multiple iterations.

“Like any pivotal launch, we thoroughly tested Intelligent Model Routing in beta for a long period of time before we decided to release it in public,” he says. “The most important learning is understanding from first principles the failure modes of every experiment, so we could keep hill climbing on the final system that we just shipped.”

Enter Auto mode

The foundation of that work surfaced last week when Replit introduced Free Mode, a lower-cost Agent mode that doesn’t consume usage credits and uses Auto to choose the model on theuser’ss behalf, subject to usage limits.

Now, that same Auto routing approach is being pushed more broadly across Replit. The company says intelligent model routing will become the default across every account, with all users starting in Free Mode and Replit deciding which model is best suited to the task.

Free Mode, it’s worth noting, isn’t “free” in the sense of unlimited usage. When it launched, Replit made it available to Core and Pro subscribers without consuming their usage credits, but imposed limits that reset every five hours, with higher allowances for Pro users. In Free Mode, users cannot manually select a model.

Core and Pro subscribers can, however, switch to Replit’s Power or Max modes, where they can turn off Auto and choose a model themselves. Replit may also suggest moving a task into one of those higher-powered modes when it determines that more capability is required, though those modes can incur usage costs.

For Enterprise customers, meanwhile, administrators can restrict Auto to an approved set of models for each workspace, allowing Replit to continue routing tasks automatically while keeping model choice within company policy.

The agent advantage

Even before SpaceX agreed to pay a cool $60 billion to acquire Cursor, the AI coding startup had long been investing in its own coding models, including its Composer family. More recently, under the auspices of SpaceX, Cursor has been developing more cutting-edge models, too.

Replit, by contrast, isn’t making ownership of the underlying model layer central to its pitch. Instead, it’s betting that controlling the agent and the systems around it gives Replit enough insight to make better model-selection decisions on the fly.

“Replit has owned, from the start, both the agent harness and the infrastructure surrounding models which in turn allows us to train sophisticated model routers.”

“Replit has owned, from the start, both the agent harness and the infrastructure surrounding models which in turn allows us to train sophisticated model routers,” Catasta said. “Only in this way can we always offer useful intelligence to our users at the most competitive price point.”

That becomes particularly relevant as an Agent task unfolds, with Replit noting that its system can change which model it uses as the task develops, seeking a better trade-off between capability and cost at different points in the process. But for Catasta, that kind of dynamic routing is still only one part of a much broader research problem around how agents should use models.

“Model routing is still in its early development phase, and we expect further research will move the needle on serving the best intelligence when customers most need it,” Catasta explains. “Routing is but one piece of the puzzle that is tightly integrated to many other aspects of our harness research.”

“No third-party router company could reproduce the same results for our own agent.”

Replit also argues that seeing how people use its own Agent gives it an advantage that a standalone routing provider would struggle to reproduce. Catasta says a router has to infer the nature, difficulty, scope, and intent of a request, with Replit able to train against proprietary usage data and observe those signals across its user base.

“No third-party router company could reproduce the same results for our own agent,” he says.

The post Replit’s new default: Auto mode picks the best model for each task appeared first on The New Stack.

As AI handles more of the execution, what work should belong to agents vs humans and why does that distinction matter?

The post Agentic AI Is Rewriting The Analytics Stack But There's One Skill It Still Can't Touch appeared first on Towards Data Science.

A randomized study of more than 1,000 students examines ChatGPT, critical thinking, originality, and student performance on a real-world university assignment.

Nvidia has reportedly agreed to buy Hugging Face for $12.9 billion, putting one of the biggest names in AI hardware in charge of a platform developers rely on to find and run open models.

The Information first reported the deal Wednesday, citing a person familiar with the agreement. Nvidia and Hugging Face had not publicly confirmed it as of publication.

Hugging Face doesn’t push developers toward one chipmaker, which is what makes the acquisition interesting. Its Optimum libraries work with Nvidia’s TensorRT-LLM and also support hardware from AMD, Intel, and AWS. Projects such as Optimum AMD and Optimum Intel let developers run Transformers and Diffusers models on non-Nvidia hardware.

The company also plays a role in what happens after a developer chooses a model, including how easily they can get it running on the hardware they want to use. The problem for Nvidia might be this: It is buying a platform whose value depends on openness and hardware neutrality, but if the purchase means Nvidia hardware is favored, that value might diminish.

The company also plays a role in what happens after a developer chooses a model, including how easily they can get it running on the hardware they want to use.

Hugging Face already sits between the model and the chip

Hugging Face has expanded well beyond file hosting. With Inference Endpoints, developers can deploy a model from the Hub while Hugging Face handles the underlying infrastructure.

Those hosted deployments can run on AWS, Microsoft Azure or Google Cloud, but most of the GPU options Hugging Face lists are Nvidia chips, including the T4, L4 and A100. That gives developers a wider choice of hardware through Hugging Face’s open-source libraries than through its hosted services.

If the deal goes through, Nvidia would own both sides of that experience.

Deployment defaults favor Nvidia

NIM (Nvidia Inference Microservices) already works with models hosted on Hugging Face. Developers can point NIM to an hf:// repository path and pull the model directly from the Hub.

Owning Hugging Face would give Nvidia more room to bring NIM and CUDA-optimized containers directly into the deployment experience. Nvidia hasn’t announced plans to make NIM the default, and support for AMD and Intel could remain exactly where it is.

Owning Hugging Face would give Nvidia more room to bring NIM and CUDA-optimized containers directly into the deployment experience.

The bigger question is what happens over time. Nvidia could provide earlier support for new models on its own hardware or make deployment easier. At the same time, AMD, Intel, and AWS may have to reconsider how much engineering work they want to contribute to integrations maintained within a competitor-owned platform. Some of that work could eventually move elsewhere.

For developers, the difference may come down to which path requires less work. A competing chip doesn’t have to disappear from Hugging Face to become less appealing if an Nvidia model deployment takes fewer steps. We’ve seen a similar fight over the layers between AI models and the developers using them, with Cloudflare building more of that infrastructure itself.

Open models counter custom chips

That tension also helps explain why Hugging Face could be worth considerably more to Nvidia than its revenue alone would suggest.

Nvidia has been expanding its own Nemotron family of open models while investing heavily across the AI ecosystem. At the same time, some of its biggest customers are working to reduce their dependence on Nvidia hardware.

Google has its TPUs, AWS has Trainium, and Microsoft has Maia. OpenAI and Anthropic are also developing their own AI server chips. The push extends beyond the hyperscalers. Earlier this month, five European companies committed to purchasing AI compute built around non-Nvidia hardware that hasn’t been manufactured yet — a sign that the appetite for alternative accelerators is strong enough to attract forward contracts.

OpenAI this week published results from its new Jalapeño accelerator, which showed 1.5 to 1.9 times more work per watt while cutting end-to-end latency by up to 3.6 times on large open-weight models — although the chip has not yet been deployed at anything approaching Nvidia’s scale. A strong open-model ecosystem gives Nvidia a counterweight to that trend.

Open models are often expected to run in very different environments, and Hugging Face helps developers make that possible. A model found on the Hub might end up running on Nvidia hardware, an AMD GPU, or a cloud accelerator.

That flexibility is part of what Nvidia would be buying. Pushing Hugging Face too heavily toward its own hardware could make the platform less useful to developers who rely on it to work across different systems.

Interest in those models is also growing. Models from companies including DeepSeek, Moonshot AI and Z.ai have narrowed the gap with proprietary systems. At the same time, Hugging Face CEO Clément Delangue told The Information in June that the company had doubled its number of paying subscribers during the first six months of 2026. Delangue later said the company was “close to profitability.”

The Information puts Hugging Face’s annualized revenue at about $150 million. Against a $12.9 billion price tag, that’s a multiple of roughly 86.

So Nvidia would be paying for much more than Hugging Face’s current business. It would be buying a place developers already turn to when they want to work with open models, including models that don’t have to run on Nvidia hardware.

It would be buying a place developers already turn to when they want to work with open models, including models that don’t have to run on Nvidia hardware.

Community resists easy forking

Buying Hugging Face wouldn’t give Nvidia control over everything developers find there. Libraries such as transformers and diffusers are open source, and models on the Hub remain subject to their own licenses. Openly licensed models can still be hosted elsewhere, while the underlying libraries can be forked.

Much harder to recreate is the community Hugging Face has built around them. Developers already know where to look for models and have built workflows around the Hub and its integrations.

The post Nvidia’s $12.9B Hugging Face deal has an open-source problem appeared first on The New Stack.

Presented by Gravitee

---

Agent complexity is the insidious shadow lurking inside enterprises right now that needs a light shone on it.

That’s because enterprises don't deploy a single agent and watch it run, they deploy fleets, each one calling APIs, calling other agents, reaching into applications that were never built with a machine decision-maker in mind. That's the failure mode that should keep you up at night: a windy, complicated system nobody can see clearly enough to govern. But why do things get so opaque so quickly?

Add a second agent to a system, and you've added one connection. Add a tenth, and you haven't added ten connections, you've potentially added dozens, because now any agent might call any other, and each of those calls can trigger a call somewhere else. Complexity doesn't creep up with agent headcount. It compounds with the number of paths between agents, and nobody's job is to draw that graph. A support ticket that used to touch one system might now pass through four agents before a human ever lays eyes on it, and every one of those handoffs is a decision point nobody approved.

Most enterprise AI programs stall when the humans responsible for their agents lose the thread. Ask a security team a simple question: which agents can reach which systems, and watch the silence. Ask which agent triggered which downstream action three hops ago. More silence.

The instinct is to treat this like a checklist. Approve the agent. Log the agent. Move on. I'd argue this is the wrong instinct. A checklist checks a single point in time. Complexity runs across a chain, and you can't govern a chain with a stack of one-time approvals any more than you can call a diet successful because you had a vegetable once.

So where does it actually break down?

Permissions creep first. Somebody builds an agent to summarize support tickets, grants it broad API access because scoping it properly would've taken another sprint, and forgets about it. Six months later, that same agent has a path into the payments system. Nobody remembers signing off on that. Nobody did.

And ownership thins out the further the chain runs. Five agents touch one workflow, something breaks at step four, and now you're asking who's responsible for a link nobody was ever assigned to own, because the org chart stopped at "deploy the agent" and never got to "name the human who answers for it."

This is a story about governance infrastructure that hasn't caught up with how agents actually behave: interconnected, cascading, multiplying faster than the processes built to track them.

Fixing the cluster starts with identity. Every agent needs to exist as its own entity, not a shadow permission borrowed from whoever deployed it. Its own name in the register. Its own scoped authority. A named human sponsor who answers for what it does. That part is necessary.

But it is nowhere near sufficient.

The harder piece is the oversight that holds across the entire chain, not just at each individual link in it. You need to see what an agent did, what it set off downstream, and where that trail ends in real time, not in a report someone pulls together once a quarter. Get agent-level identity right and stop there, and you end up with a filing cabinet full of perfectly documented agents operating inside a system nobody can actually explain.

And oversight by itself only tells you what already happened. Watching a chain isn't the same as controlling it. Enforcement is the piece most programs skip: the ability to stop an out-of-policy call before it executes, not just log it for someone to find in a review three weeks later. A dashboard that shows you an agent breached its scope five minutes ago is a monitoring tool. A system that stops the breach from happening in the first place is governance. Enterprises serious about agent accountability need both, and most have only built the first.

We're all running at blazing speed to ensure we're not the ones left behind in the race we've found ourselves in, and we're all too aware that there's a cost to slowing down. Every enterprise serious about agentic AI hits the complexity wall eventually. The ones that get past it are the ones who built enough visibility and accountability, so their fleet can keep growing without anyone losing the ability to answer one question: what is this system doing right now, and who's responsible for it.

But don't miss the point. Complexity isn't a reason to pump the brakes. The enterprises getting this right aren't slowing down. They're building toward Human-Agent Harmony, where scale and accountability grow together instead of trading off against each other.

The real risk was never a single agent doing exactly what it was built to do. It's a hundred of them doing exactly that, all at once, interacting in combinations nobody designed for. That kind of multiplication is what keeps enterprise AI stuck running pilots forever instead of running production.

Solve for complexity and autonomy stops being the villain. It starts being the whole point.

Rory Blundell is CEO at Gravitee.

---

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.

Documentation files on more than 100 websites are referencing potentially dangerous executable content that gets installed automatically when visited by many AI agents. A few dozen companies, some of them Fortune 500s, are among those that executed proof-of-concept code. At least one misconfigured site is directing visitors, human or AI, to live malware.

The potentially dangerous content is in llms.txt and llms-full.txt files, an emerging convention websites employ to provide machine-readable summaries of the site’s content and its high-level structure. These files are the AI equivalent of the robots.txt standard that instructs search engines how to index the site's content. Google Lighthouse, a tool for helping web developers, has more here. Correctly configured llms.txt and llms-full.txt files for Cloudflare are here and here.

How the researchers found it

Researchers at a stealth startup in Israel scanned 6,214 live domains belonging to defense contractors, Fortune 500, and Big Tech companies. Of the 8,265 llms.txt and llms-full.txt files they found (many sites hosted both an llms.txt and an llms-full.txt file), 120 of them, each on a different site, pointed to one or more code packages or domain names that weren’t registered. To test what happens when an AI agent processes such files, the researchers registered a handful of the unclaimed names and hosted packages that caused any machine executing them to reach out to their server. Within an hour, the researchers received a phone-home response from a Fortune 500 company. Over time, they got a few dozen more, some from more Fortune 500 companies and others from startups. Their beacon also recorded the chain of parent processes that spawned each install, ultimately revealing that coding agents, including Claude, OpenAI's Codex, and Nous Research's Hermes, were involved. Anthropic, OpenAI, and Nous Research did not respond to requests for comment by the time of publication.

Read full article

Comments

© Aurich Lawson

IT automation uses instructions to create clear, consistent and repeatable processes that replace an IT professional's manual work in data centers and cloud deployments.

A practical guide to getting better code, not just more code

The post How to Work with AI Coding Agents appeared first on Towards Data Science.

Kodiak AI, a provider of physical AI-powered autonomous driving technology, has announced a collaboration with chipmaker AMD to implement its AMD EPYC Series processors to power Kodiak’s seventh-generation autonomous truck platform. Kodiak is the first company to deploy the advanced AMD EPYC processors into their hardware platform for driverless trucking. High-performance AMD EPYC Series CPUs […]

The OpenAI agents involved in last month’s incursion into Hugging Face were trained so heavily on winning a competition that they pursued a relentless campaign to cheat, a new report documented. In the process, and without authorization, they created an improvised message board to hatch a plan that ultimately landed them squarely inside the latter company’s network.

Over the course of May and June, OpenAI gave the agents what the company described as “impossible tasks” to complete on the benchmarking framework ExploitGym. The internal test was designed to test how the agents would respond. To get a full understanding of the agent capabilities, company engineers disabled safety guardrails that normally are in place to prevent the sort of hacks that eventually hit Hugging Face and one other undisclosed organization. The stymied agents’ training made them so focused on winning that they performed tasks they were never explicitly instructed to follow.

Cheaters gonna cheat

The first step was creating a message board that allowed the agents to pass notes to each other. OpenAI hadn’t provided any such platform, so the agents repurposed a platform called Artifactory, which OpenAI was using in internal testing of several unreleased hacking agents. OpenAI was using Artifactory as one of the measures to prevent the agents from egressing its isolated sandboxes and accessing the Internet, while at the same time simulating a real-world hacking environment.

Read full article

Comments

© Getty Images