DeepSeek V4 Flash Appears on Nvidia DGX Spark, Flashing Local Multi-Model Switching Speed

Overview

Recently, the Athena inference engine for the Nvidia DGX Spark device has attracted industry attention. The engine supports the simultaneous deployment of two large models, DeepSeek V4 Flash and Qwen3.8 Flash-Next, on the same device and achieves ultra-fast multi-model switching through efficient context management technology. This breakthrough makes it possible to deploy local multi-model agent servers, significantly lowering the barrier to entry for enterprises and individual developers.

Technical Architecture and Performance

The Athena engine is specifically designed for the Nvidia GB10 system, fully leveraging the 128GB memory resources of the DGX Spark. Since a single card cannot load two massive models simultaneously, Athena employs a unique checkpoint mechanism that preserves the agent's session state during the process of unloading the current model and loading a new one, enabling seamless switching. Benchmark data shows that model switching takes only about 46 seconds, while maintaining constant OpenAI/Anthropic-compatible API endpoints throughout, so clients require no modifications to use the service.

In terms of performance, running DeepSeek V4 Flash on a single card yields a prefill speed of 1,126 tps with 8K context and 948 tps with 256K context, while the decoding speed (@256K) reaches 19.4 tps. Qwen3.8 Flash-Next also delivers impressive results, achieving a decoding speed of 32.1 tps at 256K context, surpassing the former. Notably, there is virtually no performance loss in prefilling when scaling from 8K to 256K context.

Tool Calling Capabilities and Cache Optimization

The built-in evaluation system of Athena covers 69 scenarios and 52 available tools, focusing on testing capabilities such as chained calling, error recovery, and schema-valid JSON output. Both quantized models scored 91/100 in tool usage evaluations, demonstrating powerful complex task processing abilities. Although challenges exist when handling specific tool output prompt injection scenarios, the overall performance already meets the vast majority of practical application needs.

The engine also introduces a disk cache mechanism, significantly improving response speeds for long conversations. A conversation log of 141,519 tokens can be restored via cache in just 2.1 seconds, whereas reprocessing takes approximately 2 minutes and 20 seconds—an efficiency improvement of over 60 times. Additionally, Athena supports speculative decoding, Qwen image/document input, and persistent agent context, and can be easily installed via Docker.

Application Prospects

Currently, Athena is open-sourced for personal, research, and educational purposes, but its core engine remains a closed-source commercial product. Developer David Hendrickson pointed out that this solution perfectly aligns with the vision of "running giant models, ultra-long context windows, and practical speeds on a single desktop device." As Athena gains adoption, local multi-model collaboration will become an important trend in edge computing and private deployments.