Nas的天空
点亮思维火花，畅享数字世界
用户数
1
文章数
504
评论数
21
阅读量
184945
Latest articles
双卡4090折腾记：基于vLLM+LiteLLM搭建高并发本地大模型聚合网关实战
拒绝单并发玩具！本文基于双卡RTX 4090服务器，手把手记录如何使用vLLM充分释放Qwen2.5与DeepSeek模型的批处理潜能，并结合LiteLLM构建兼容OpenAI协议的企业级高并发、多租户流式网关。包含NCCL多卡互联暗坑、KV Cache显存精细化调优与真实压测数据。
单卡 RTX 4090 极限压榨 DeepSeek-R1 蒸馏版：vLLM 显存调优与 LiteLLM 流式代理实战
记录在家用工作站与私有算力节点上，基于单张 RTX 4090 部署 DeepSeek-R1 蒸馏模型的硬核调优实战。深度拆解 vLLM PagedAttention 显存分配机制、KV-Cache FP8 压缩技巧与 LiteLLM 高并发流式反代避坑方案，附压测数据与生产级 Docker 配置。
单卡4090跑满32B！vLLM + LiteLLM 搭建高并发流式大模型网关实战
告别Ollama在多并发场景下的卡顿短板，本文手把手拆解基于单张RTX 4090通过vLLM部署DeepSeek-R1-Distill-32B（AWQ量化）的完整方案。结合LiteLLM构建支持OpenAI标准协议、流式传输、多租户鉴权与故障降级的高性能推理网关，详解KV Cache分配与防OOM排障实录。
单卡4090跑满极限：vLLM高并发流式推理架构调优与避坑实战
很多团队把开源大模型拉起来跑Demo很顺，但在多并发流式请求下，TTFT骤增、显存直接CUDA OOM。本文基于单张RTX 4090（24G）实测，详解vLLM+FastAPI异步架构落地细节，剖析PagedAttention显存分配、KV Cache压缩及连续批处理参数调优，助你将单卡吞吐拉满到工业级并发水准。
实战打造高并发 LLM 流式网关：FastAPI + HTTPX 异步代理、背压与降本避坑指南
在自建大模型服务或对接 DeepSeek/vLLM 等上游推理引擎时，中间网关的流式传输与并发性能至关重要。本文记录如何用 Python (FastAPI + httpx) 打造低延迟、高并发的 LLM 流式转发网关，深入剖析连接池调优、客户端断连取消推理（背压控流）以及流式 Token 计费的硬核实操经验。
Latest Comments
macau18 link alternatif
Thanks for any other fantastic article. Where else may anyone get that type of information in such an ideal method of writing?
I have a presentation subsequent week, and I am at the look for such information.
all about casino comped cruises
Hi, i read your blog from time to time and i own a similar one
and i was just curious if you get a lot of spam comments?
If so how do you reduce it, any plugin or anything you can recommend?
I get so much lately it's driving me insane so any help is very much appreciated.
medical boric powder
I'm not sure why but this site is loading
very slow for me. Is anyone else having this issue or is it a problem on my end?
I'll check back later and see if the problem still exists.
best leather conditioner
I have been surfing on-line more than three hours nowadays,
yet I never found any attention-grabbing article like yours.
It is lovely price sufficient for me. In my view, if
all web owners and bloggers made excellent content material
as you did, the net can be much more useful than ever
before.
projector screen hyderabad
Ꭺt Nissi Offife Systems, we specialize іn providing top-of-thе-line projectors,
interactive panels, аnd comprehensive hone theater solutions tߋ transform ｙouг vierwing
experience.
using gain in audio mix
Your method of explaining the whole thing in this article is really good,
all be capable of effortlessly understand it, Thanks a lot.
artistas femeninas de música techno
Simply wish to say your article is as astonishing. The clarity in your post
is just spectacular and i could assume you are an expert
on this subject. Well with your permission let me to grab
your RSS feed to keep updated with forthcoming
post. Thanks a million and please keep up
the enjoyable work.
阿滨
在安装Qb所欲依赖时报错:pkg install qt5-base qt5-qmake libtorrent-rasterbar
Python 基础入门 Day12 - Nas 的天空
[…] Day08: Python 数据分析 […]
Python 基础入门 Day12 - Nas 的天空
[…] Day01: Python 基础入门 […]
热评文章
实测LiteLLM高并发流式网关落地：负载均衡、语义缓存与降级容灾全攻略
面对多模型API成本飙升、厂商网络偶发抖动以及流式传输（SSE）连接泄露痛点，本文详解如何基于LiteLLM构建高可用统一LLM API网关。涵盖Docker生产级部署、Redis语义缓存降本、自适应重试与降级路由配置，附真实压测数据与排障避坑记录。
告别OOM与高延迟：基于vLLM与Prefix Caching的高并发流式Agent代理实战
面对多Agent协同与长上下文对话场景，自建LLM网关经常遭遇显存溢出与首字延迟（TTFT）飙升问题。本文记录了一套基于vLLM深度调优、结合Prompt Cache与FastAPI流式代理的高并发落地架构，涵盖关键启动参数拆解、SSE背压控制及单卡实测数据，帮你在有限显存内榨干算力。
NAS玩转本地私有知识库：Docker轻量部署Ollama+Qdrant+Dify实战调优
本文手把手记录在家庭/工作室NAS环境（群晖/Unraid/TrueNAS/自建Debian）下，利用Docker Compose全容器化部署Ollama、Qdrant向量数据库与Dify构建私有本地知识库的完整流程。深入剖析内存溢出排查、CPU/GPU量化选型、混合检索调优及多节点内网穿透实战，提供生产级配置模板，打造安全、低延迟且零订阅成本的个人AI生产力中枢。