Enterprise Document Intelligence [Vol.1 #B1] - Three sources of one problem. User typos, fast-typing transcription noise, OCR character errors. Classical spell-check handles one of them. Embeddings carry the rest
The post Noisy Text in RAG: Typos, OCR, and the Gap Classical Spell-Check Leaves appeared first on Towards Data Science.
Enterprise Document Intelligence [Vol.1 #B00] - Retrieval answers one kind of question. Classifying a request, matching free text to a reference list, reading a table, cleaning OCR noise: each has a cheaper method that works, and the engineering is knowing which one to reach for
The post RAG Is Not the Whole Toolkit: The NLP Techniques Real Problems Still Need appeared first on Towards Data Science.
Alibaba this week unveiled Qwen3.8-Flash, an open-weight, multimodal Mixture-of-Experts (MoE) model.
Hot on the heels of Qwen 3.8 Max, which arrived at the start of the month, this 125-billion-parameter AI model is offered as a prelude to Qwen 4. It is positioned as both a performance and value-for-money play. As such, it is claimed to have “superior capabilities in coding and office tasks” and an optimal balance among capability, latency, and cost.
“In this release, we are opening the weights of Qwen3.8-Flash-Next, a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4,” confirmed Alibaba in its release blog.
By releasing the architectural changes in Qwen3.8-Flash early, the organization hopes the community will examine the mechanics, constructs, and components within and start road-testing them before the full Qwen4 model family is built on top of them.
By stating that Qwen3.8-Flash paves the way for Qwen4, Alibaba is showcasing (and, importantly, openly sharing) design forms that it will carry into its subsequent models, so that developers can start building (or at least planning) their next codebases early.
Specifically then, Alibaba has stated that Qwen3.8-Flash “plays the same role” that Qwen3-Next played for Qwen3.5 i.e. by which the company means that Qwen3.8-Flash introduced developers to the company’s hybrid Gated DeltaNet + Gated Attention design (computational components that enable AI models to balance long-context efficiency with contextual focus and attention during inference and training), which was then used across the Qwen3.5, Qwen3.6, Qwen3.7 and Qwen3.8 series.
“Qwen3.8-Flash-Next upgrades the model systematically along four aspects – attention, residual, embedding and optimization – improving model capability while further optimizing computational efficiency, model capacity and training stability,” stated Alibaba.
When benchmarked on agentic coding, long-horizon agent tasks and multimodal intelligence, Qwen3.8-Flash appears to perform respectably against models including DeepSeek-V4-Flash and Claude-Opus-4.6 across SWE-bench Pro (agentic coding), CoWorkBench (long-horizon office work), Toolathlon Verified (real-world tool use), MathVision (visual maths problem-solving), AndroidWorld (agentic mobile use) and ERQA (embodied intelligence).
“Qwen3.8-Flash-Next upgrades the model systematically along four aspects – attention, residual, embedding and optimization – improving model capability while further optimizing computational efficiency, model capacity and training stability.”
Tested on agentic coding using SWE-bench Pro, Qwen3.8-Flash-Next scores 62.5, compared to 61.7 on Qwen3.8-27B, 55 on Qwen3.7-Plus, 56.0 on DeepSeek-V4-Flash-0731, and 53.4 on Claude-Opus-4.6 (Max).
Importantly, Qwen3.8-Flash requires only what Alibaba details as “around one-ninth of the training resources,” while delivering superior performance. The company says that this means Qwen3.8-Flash “significantly reduces” both training and inference costs compared with Qwen3.7-Plus, a model three times its size.
For the sake of nomenclature, Qwen3.8-Flash-Next is the open-weight research-frontier model available to developers on both the Hugging Face AI developer hub and Alibaba’s ModelScope community portal. Built on the same underlying architecture, Qwen3.8-Flash is the production version of the model, offered via the QwenCloud API with 1 million tokens by default and official built-in tools.
Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. It natively supports 262,144 tokens of context and can be extended to 1,000,000 tokens with YaRN.
As noted, Qwen3.8-Flash introduces architectural extensions across attention mechanisms, residual connections, embeddings, and optimization. Its hybrid attention architecture combines Gated DeltaNet (GDN), which compresses historical information, with Qwen Sparse Attention (QSA), a novel design that uses a lightweight compressed indexer to select relevant context, substantially reducing attention costs for long sequences.
Alibaba has explained that the Gated Residual (GR) mechanism expands data pathways between layers while strengthening cross-layer information flow and training stability. The N-gram Embedding technique scales model capacity with minimal additional computation, while the Muon Optimizer enhances the efficiency of large-scale model training.
“I’d love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either…it’s just not there (yet) for local professional software development.”
In terms of developer reaction, it’s been a mixed bag so far.
Mechatronics engineer Alok posts on X, saying he thinks the Video RAM barrier (i.e., the need for physical, high-speed GPU-based memory for LLM memory management in the face of model quantization that aims to enable better long-context inference) is now officially dead.
“I just ran Qwen3.8-Flash-Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090 – 21 tokens/sec decode. 364 t/s prefill – no mtp. No dflash. No KV cache quantization! We are running datacenter models on consumer hardware,” enthused Alok.
Multi-disciplined software developer Embedding Shapes is less happy.
They post on Hacker News using some colorful language to describe how models keep [insert expletive]-ing up very basic things before saying, “I’d love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either. But I too got hopeful reading the sentiment on the Internet about Qwen 3.8, but it’s just not there (yet) for local professional software development.”
Qwen3.8-Flash can be accessed via API Model Studio and Qwen Cloud, Alibaba’s AI-native cloud platform. Pricing per 1 million tokens is US$0.16 for input and US$0.47 (or 3 RMB for developers inside China) for output.
The model is also available on QwenWork, Alibaba’s workplace AI agent platform, where it runs a redesigned “standard mode” that cuts token consumption per task by 75% and “roughly doubles generation speed” compared with the current mode.
Alibaba has claimed that this brings flagship-level capabilities within reach of everyday workloads and that developers can run this model on hardware that they likely already own.
Alibaba has not confirmed a firm launch date for Qwen4. Still, a casual web search on the topic yields a range of commentators and market watchers who broadly agree it will arrive before the end of the year, possibly as early as September.
Conjecture in this space explores whether the next model family could be optimized for complex 3D coding and design tasks in advanced spatial modeling. Others think that the Mixture-of-Experts (MoE) architecture will be extended and that deeper native multimodal processing capabilities might be featured.
Alibaba was contacted for broader comment on this story but declined to engage.
Qwen, in traditional Chinese, 通義千問 (pronounced Tōngyì qiān wèn), translates to “a thousand questions on general meaning” in English. Use that in your local pub quiz this weekend.
The post Alibaba just released Qwen3.8-Flash: “An early preview of the architecture in Qwen4” appeared first on The New Stack.
A hand-written CUDA inference runtime for Vision-Language-Action robots that decides what to remember, what to forget, and when it's simply too late to think.
The post Can an LLM Forget the Right Things? appeared first on Towards Data Science.
How DFlash trades spare compute for saved memory bandwidth, and why its gains shrink as concurrency rises
The post Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash appeared first on Towards Data Science.
The Pew Research Center analyzed nearly half a million English-language web pages for AI-generated content. More than a third of pages published since ChatGPT's launch show signs of machine-written text, and commercial .com sites are ten times more likely to contain AI content than .edu or .gov domains.
The article Pew study confirms sharp rise of AI-written text on the web since ChatGPT's launch appeared first on The Decoder.
Most LLM inference runtimes have no idea a physical deadline exists. This one refuses admission rather than miss a 33ms robot control cycle, evicts KV cache by meaning instead of age, and is written entirely in hand-written CUDA — no cuBLAS, no libtorch.
The post Can an LLM Forget the Right Things? appeared first on Towards Data Science.
Speculative decoding can turn underused CPU compute into faster token generation, without changing the model's output. In our vLLM tests, DFlash delivered 3.92x the autoregressive throughput with Qwen3.5-9B on Intel Xeon 6 at concurrency 1. We break down where the speedup comes from, explain the acceptance metrics, and show what determines whether speculation pays off.
The post Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash appeared first on Towards Data Science.
A hands-on guide to fine-tuning LLMs for the real world
The post How to Fine-Tune an LLM: An End-to-End Guide appeared first on Towards Data Science.
A controlled comparison of a top-5 RAG pipeline and a full 127,000 token prompt on the same 12 questions, same system prompt and same model. Graded blind on correctness, completeness and grounding.
The post Kimi K3’s 1M Token Context Window vs. RAG: Cost, Latency and Answer Quality appeared first on Towards Data Science.
The number that fooled every hallucination detector
The post Ten Is Not a Hundred appeared first on Towards Data Science.
Here's how to be the Data Scientist who thrives in a world where coding is a commodity.
The post How to Shine as a Data Scientist in the Vibe Coding Era appeared first on Towards Data Science.
Can a language model do live adversarial level design? Yes, emphasis on the adversarial part
The post I Made an LLM Lay Siege to My Minecraft House appeared first on Towards Data Science.
Google's Open Knowledge Format (OKF) is a Markdown+YAML skeleton for sharing knowledge between humans and AI agents. This post reuses that skeleton for a very specific job — an agent-to-agent hand-off of pre-tokenized integer arrays between three Qwen2.5-Coder models (7B, 3B, 1.5B) — and shows the 28–37% TTFT reduction plus the one full-vocabulary equivalence check that keeps the whole thing safe.
The post How to Utilize OKF Efficiently to Enable Knowledge Exchange Among LLMs appeared first on Towards Data Science.
An open, 2.8-trillion-parameter model shipped with 47 pages of its own recipe. Reading it tells you what building a frontier model now involves, and how little of it is the model.
The post How a Frontier Model Gets Built, Read from the Kimi K3 Report appeared first on Towards Data Science.
Research-backed cues to detect LLM-generated text along with the mathematical intuition as to 'why'
The post Is This Slop? Detecting AI-Generated Content Without a Model appeared first on Towards Data Science.
How to use Claude to craft an outstanding resume that lands offers
The post How Claude Help Me Build My $200k+ ML Resume appeared first on Towards Data Science.
The seemingly unquenchable thirst of the AI data ingestion pipeline spans language, numerical, and tabular data in the first instance, while other tangential platforms have been building large audio, image, and video models at the same time.
Straddling potentially all of these domains are the file structures where complex documents and forms of unstructured data reside; this is the road less traveled in terms of the source DNA modern AI draws from.
In a bid to bridge connections to the schema-less, freeform, uncurated information that all organizations naturally harbor, enterprise visual intelligence company Valantor announced its acquisition of unstructured information RAG specialist EyeLevel on Tuesday. The acquisition formally launches Valantor’s Enterprise Visual Intelligence platform, combining EyeLevel’s document intelligence with its own operational expertise.
Benjamin Fletcher, CEO and co-founder of EyeLevel, tells The New Stack that where organizations fail to adopt visual intelligence, human-only processing breaks down pretty quickly in the age of AI.
“About 80% of corporate knowledge is in millions of pages of visually complex PDFs, PPTX, and DOCX files,” Fletcher says. “This information is far beyond the capacity of any LLM context window and is effectively inaccessible to LLMs and agents.”
“We’ve found the golden datasets that teams build by hand routinely carry 10 to 25 percent error rates. Ironically, those same teams often hold AI to a far higher standard than their own people.”
He explains that transactional workflows (such as invoice and claims processing) typically involve documents “so visually complex and diverse” that enterprises still rely on humans to process them, who can be slow, expensive, and error-prone.
“We’ve found the golden datasets that teams build by hand routinely carry 10 to 25 percent error rates,” Fletcher says. “Ironically, those same teams often hold AI to a far higher standard than their own people. If data sovereignty matters to a business, everything gets harder now: solving these problems with AI while your documents stay inside your own infrastructure is the hard mode version of the job, and very few tools can do it.”
Valantor has noted that while most AI companies concentrate on models, the company itself is “focused on the information those models can’t see” today. The suggestion is that this unseen morass of valuable data is locked inside documents, claims files, contracts, engineering drawings, reports, forms, presentations, and other visually complex content.
Valantor’s flagship platform product, GroundX, operates where data resides, including private cloud, sovereign infrastructure, on-premises deployments, and fully air-gapped environments.
“GroundX is the ingestion and retrieval layer for unstructured documents,” explains Fletcher. “It is one tightly tuned system where retrieval consumes exactly what ingestion produces. Everything is exposed through REST APIs, SDKs, and MCP. It ships as REST APIs, SDKs, and MCP, and the Helm chart drops straight into a team’s existing deploy pipeline, and our agent harness gives coding agents like Claude and Codex the skills to build the integration themselves.
As part of the acquisition announcement, Valantor is introducing GroundX Studio. The harness capabilities within GroundX Studio integrate with modern AI development environments, enabling developers to build secure AI applications that operate on enterprise knowledge while remaining within existing infrastructure.
GroundX Studio also extends capabilities to business users, allowing organizations to create AI-powered workflows and applications without extensive custom development.
“Each agent does one small task, so cheaper models are often good enough, and teams that want direct control over cost can run the whole stack on their own hardware with Helm.”
If it feels like this new data ingestion stream is going to place a new burden on cloud workloads, application execution latency, database retrieval times, and (of course) overall token usage, then Valantor and EyeLevel say that this consideration has been taken into account by dint of their own platform’s orchestration layers.
“We never send a whole schematic to a language model; our vision model splits each page into its elements first,” Fletcher confirms. “Processing runs in multiple passes at different levels of the document, and everything inside a pass runs in parallel, so there’s a minimum processing time, but it does not scale linearly with page count. Each agent does one small task, so cheaper models are often good enough, and teams that want direct control over cost can run the whole stack on their own hardware with Helm.”
sWhile we already know that AI and handwriting do mix in the same cocktail glass — the ViWoods AiPaper digital e-ink handwriting tablets have a useful set of AI functions on board, and similar products are available from manufacturers including reMarkable — it’s not a widely deployed use case yet. Valantor claims that its underlying data models and custom heuristics bridge the “data comprehension gap” when processing handwritten annotations.
“Our proprietary vision model, fine-tuned on more than a million pages of enterprise documents, sees the page the way a human does: tables, paragraphs, and figures,” underlines Fletcher.
He says that handwritten marks are captured as page elements with their layout context intact. Narrow agents then distill each element into a contextual object tuned for both search and LLM completion.
“Smaller pieces, less cognitive load — that’s how we close the gap, with better accuracy at lower cost, driving better performance and significant cost advantages,” he adds.
Working examples of this technology include Air France-KLM, which used GroundX to develop an AI-powered customer service assistant trained on thousands of policy documents, achieving 96+% accuracy on complex policy-related questions. AskVet used the platform to operationalize more than a decade of proprietary veterinary data, enabling autonomous resolution of up to 85% of customer inquiries while significantly improving operational efficiency.
Taking all of this on board, are we at the point where we can ask whether document management has just become interesting, compelling, and sexy?
No, of course it didn’t; it will arguably always suffer from a degree of stigmatized disdain. That may change in the future as we interact more directly with AI tools that begin analyzing the unstructured information we know organizations have been sitting on for so long. For now, it may still remain the corporate equivalent of eating your vegetables — pass the Brussels sprouts and steamed turnips, please.
The post AI can finally read your handwriting — here’s why enterprises care appeared first on The New Stack.
While the household-name frontier models race forward with version numbers and context windows of at least a million tokens, a new breed of upstart data science specialists is pushing the context window into double figures.
Subquadratic debuted a 12-million-token window in May of this year, and Silicon Valley and South Africa-based Refiant launched its 10-million-token context-window model, Protea, on Wednesday. It’s a move that may signal the long-context AI race is now on.
But more context windows alone are not enough. This is because even the most capable models have a few hundred thousand tokens in working memory, which can force workarounds to compensate for what the model can’t access.
Refiant co-founder Dr. Viroshan Naicker tells The New Stack that he believes modern LLMs “fail to be organically efficient at an elemental level” and that his organization’s approach mimics how systems in nature, from ant colonies to beehives, find efficient solutions to complex problems.
“This is no case of pseudoscientific puff; nature is the most computationally efficient system that we know, and many algorithms used in science are nature-inspired,” Naicker says. “This is a road well-traveled in science. There are multiple teams globally working in this particular (nature-inspired) direction, trying to bridge the gap between AI inference as we know it and the energy efficiency of natural systems.”
“Fish and birds coordinate their movements to converge on the mathematically shortest, most efficient routes — honeybees, fireflies and bacteria are also programmed to use degrees of swarm-style optimization.”
Did Naicker just mention ant colonies and honeybees?
Yes, because Refiant uses swarm-style optimization. It’s seen in ant colonies, which initially move randomly until a food source is detected, after which they leave a pheromone trail for other ants to optimize their journeys. Fish and birds also coordinate their movements to converge on the mathematically shortest, most efficient routes. Honeybees, fireflies, and bacteria are also programmed to use degrees of swarm-style optimization.
Naiker, along with his co-founders, Siddharth Gutta and Mathew Haswell, form a team with experience spanning quantum mathematics, traditional finance, and commercial scaling. Applying swarm-style optimization to data in Protea means inference is performed through a combination of compression and context management.
“From our perspective, we are also advancing a technology which provides context-specific inference models grounded in data,” Naicker clarifies. “We think this has value for reducing model hallucinations, replacing RAG, and constructing better, more reliable, agentic workflows. This adds a layer of trust in sensitive application scenarios, rather like an added insurance, rather than taking it away.”
The Refiant team describes 10 million tokens as equivalent to 7.5 million words in a single conversation (and we know from Anthropic’s own benchmarks this year that Claude has a 1-million context window), or five years of a user’s emails, 83 novels, or 830 podcast episodes, all held in active memory at the same time.
The team claims Protea is capable of working on entire enterprise codebases or decades of clinical trial data — datasets that previously had to be broken apart and fed to models in fragments — so they can be processed in a single pass with full fidelity. Engineers on Protea also submit that they can successfully tackle the “lost in the middle” problem — a limitation of million-plus-token windows, where models stay accurate at the start and end of the context but lose the thread of everything buried in between.
Refiant first applied these techniques to model compression, shrinking OpenAI’s GPT-OSS-120B so it could run on a MacBook Pro with 18GB of RAM.
“Rather than publishing benchmarks, we’re inviting users to run the models and try them out.”
The Protea series is open and live, and Refiant is inviting teams to stress-test the context window across different industries and use cases. But should we trust sensitive hould enterprise data archives to a completely unproven startup founded only one year ago?
“We adhere to data management best practices, processes and compliance requirements,” Naiker confirms. “This is reasonable for a startup at our particular stage. Privacy and data sovereignty are important values for us, and we are actively exploring edge, self-hosted, and bring-your-own-cloud data models.”
But a 10 million-token context window is big. Won’t that fall short when Protea starts to suffer from massive latency spikes when processing a full dataset? Naiker agrees that “latency is a core issue with long-context inference models,” but in the tests his company has run, it has delivered inference at a reasonable latency, even with large token windows.
“We have internal reports and tests that validate the technology, including Ruler, MRCR and Babilong, but we aren’t asking anyone to take our word on this. Rather than publishing benchmarks, we’re inviting users to run the models and try them out,” adds Naiker.
Although the technology industry is littered with apocryphal statements and Bill Gates almost certainly never said “64K ought to be enough for anyone” in real life, we have to ask ourselves today whether we’ll be laughing about those “silly little” 10 million token context windows by the end of the decade.
It may not take that long. Internally, Refiant maintains that it has already demonstrated a working prototype with a 100-million-context window and is exploring how best to benchmark and productionize it at that scale in the future.
Coming next, then, as Dr. Evil from Austin Powers would say, the one-hundred-billion-context window, right?
The post “Nature is the most computationally efficient system we know”: How Refiant used swarm optimization to build a 10-million-token AI model appeared first on The New Stack.
About a year ago, Google demoed a diffusion model at its I/O developer conference, but went quiet about the technology soon after.
On Wednesday, however, Google broke that silence with the launch of DiffusionGemma, an experimental 26B mixture-of-experts model that uses diffusion to generate text 4x faster than its existing Gemma models.
Diffusion has long been the standard for generating images (think Stable Diffusion). Instead of generating one word at a time, models like DiffusionGemma or Inception’s Mercury 2 generate words in parallel.
At first, those blocks of text don’t make sense and seem random. But then, with each new step, the model refines the text and reduces the noise until it becomes the answer you were looking for. If you’ve ever looked at a diffusion image model generate images in real-time, that’s essentially the same process, but for text.
With each step, the model denoises 256 tokens in parallel, which is why it can be much faster than a traditional autoregressive large language model. It basically iterates on the text with each step until it.
All of these tokens attend to all others, which Google says is especially helpful for use cases such as inline editing, code infilling, working with amino acid sequences, and mathematical graphs.
Google says DiffusionGemma can produce more than 1,000 tokens per second on a single Nvidia H100. And since the model uses the mixture-of-experts technique, it doesn’t have to keep the full 26 billion parameters in memory; instead, it activates only 3.8 billion during inference. This means it can easily run on a GPU with 18GB of VRAM.
There are some tradeoffs, though. On all benchmarks, the DiffusionGemma model underperforms when compared to Gemma 4 26B A4B. That’s something Google itself acknowledges. There’s no technical reason why a diffusion model couldn’t perform just as well as a more traditional large language model, but the focus here is on speed.
“For applications that demand maximum quality, we recommend deploying standard Gemma 4,” Google says in its announcement.
The model is now available on HuggingFace, with Unsloth and other quantizations available for those who want to run it locally using llama.cpp and (soon) similar local inference tools.
Google also worked with Nvidia to optimize the model for its hardware, including high-end GPUs like the GeForce RTX 5090 and 4090, as well as the Nvidia DGX Spark and DGX Station (for those who can afford them). Nvidia NIMs are also available for the model.
The post Google’s DiffusionGemma is 4x faster than its other Gemma models appeared first on The New Stack.
Login