Nas的天空
点亮思维火花，畅享数字世界
用户数
1
文章数
524
评论数
21
阅读量
192610
Latest articles
双卡3090榨干指南：vLLM + DeepSeek-R1-32B 高并发流式推理私有化实操
记录在双路 RTX 3090（48GB 显存）服务器上，基于 vLLM 部署 DeepSeek-R1-Distill-32B 的硬核实操。实测解决多卡 NCCL 互联抖动、FP8 KV Cache 显存压缩、Nginx 反代 SSE 流式截断三大痛点，附压测数据与生产级 Docker Compose 配置。
Python + FastAPI 搭建生产级 LLM 流式代理：连接池、背压与客户端断连排坑实战
直面 LLM 流式网关生产环境痛点：手把手用 Python 和 FastAPI 实现高性能 SSE 代理。深度拆解 httpx 异步连接池泄漏、客户端中途断连导致的算力空耗与 Token 浪费，配合 Nginx 零缓冲配置与 asyncio 任务生命周期管理，打造单机稳定承载数千并发流式响应的网关服务。
榨干单卡24G显存：vLLM + Qwen2.5-32B 生产级高并发API搭建与压测排坑
放弃开箱即用但并发吞吐受限的Ollama，本文手把手教你基于RTX 3090/4090搭建基于vLLM与Qwen2.5-32B-AWQ的生产级LLM接口。涵盖Docker编排、Prefix Caching开启、Nginx流式代理断流避坑以及真实压测数据对比。
告别 OOM 与断流：vLLM + Caddy 搭建生产级高并发流式推理网关实战
针对本地部署开源大模型在高并发场景下频发的 KV Cache 显存溢出、SSE 流式输出截断与首字延迟（TTFT）飙高问题，本文深入解析 vLLM 关键底层参数调优与 Caddy 反代配置实战，提供双卡 RTX 4090/A100 环境下的即插即用生产级部署架构方案。
Mac Studio/MacBook 榨干 Apple Silicon：破解统一内存限制与 MLX 搭建本地大模型服务
针对 Apple Silicon Mac 本地运行 32B/70B 级别大模型频繁 OOM 的痛点，本文实战演示如何突破 macOS 默认的 75% 显存上限，并结合 Apple 原生 MLX 框架与 launchd 守护进程，搭建高吞吐、低延迟的 OpenAI 兼容本地推理服务。
Latest Comments
macau18 link alternatif
Thanks for any other fantastic article. Where else may anyone get that type of information in such an ideal method of writing?
I have a presentation subsequent week, and I am at the look for such information.
all about casino comped cruises
Hi, i read your blog from time to time and i own a similar one
and i was just curious if you get a lot of spam comments?
If so how do you reduce it, any plugin or anything you can recommend?
I get so much lately it's driving me insane so any help is very much appreciated.
medical boric powder
I'm not sure why but this site is loading
very slow for me. Is anyone else having this issue or is it a problem on my end?
I'll check back later and see if the problem still exists.
best leather conditioner
I have been surfing on-line more than three hours nowadays,
yet I never found any attention-grabbing article like yours.
It is lovely price sufficient for me. In my view, if
all web owners and bloggers made excellent content material
as you did, the net can be much more useful than ever
before.
projector screen hyderabad
Ꭺt Nissi Offife Systems, we specialize іn providing top-of-thе-line projectors,
interactive panels, аnd comprehensive hone theater solutions tߋ transform ｙouг vierwing
experience.
using gain in audio mix
Your method of explaining the whole thing in this article is really good,
all be capable of effortlessly understand it, Thanks a lot.
artistas femeninas de música techno
Simply wish to say your article is as astonishing. The clarity in your post
is just spectacular and i could assume you are an expert
on this subject. Well with your permission let me to grab
your RSS feed to keep updated with forthcoming
post. Thanks a million and please keep up
the enjoyable work.
阿滨
在安装Qb所欲依赖时报错:pkg install qt5-base qt5-qmake libtorrent-rasterbar
Python 基础入门 Day12 - Nas 的天空
[…] Day08: Python 数据分析 […]
Python 基础入门 Day12 - Nas 的天空
[…] Day01: Python 基础入门 […]
热评文章
Mac 本地部署 DeepSeek-R1 最佳实践：MLX vs llama.cpp 性能压测与显存优化
在 Apple Silicon（M系列）芯片上本地跑大模型，还在无脑开 Ollama 默认配置？本文基于 M3 Max 与 M2 真实环境，深入对比 llama.cpp 与 Apple 原生 MLX 框架在量化精度、显存占用及推理吞吐量（Token/s）上的实测差异，附带统一内存上限解锁、Metal 编译调优与守护进程自动化配置指南。
单卡4090跑满吞吐：vLLM多并发流式代理架构与KV Cache避坑实战
单张消费级显卡部署高并发开源大模型时，首字延迟突增与显存碎片化是致命痛点。本文基于vLLM结合FastAPI与Nginx构建高并发流式代理架构，拆解PagedAttention显存管理机制，详解真实生产压测数据、Chunked Prefill与多流转发配置，并附赠显存OOM与TCP连接悬挂的硬核排障指南。
NAS玩转本地私有知识库：Docker轻量部署vLLM+Qwen2.5与Dify实战调优
嫌公有云大模型API按量扣费太肉疼？本文分享我在自建TrueNAS/Debian宿主机上，利用Docker容器化低延迟跑通vLLM、Qwen2.5与Dify全功能私有化知识库的实操全流程。深度解析NVIDIA-Container-Toolkit透传、显存与上下文优化、GPU P2P排障及混合检索调优实测。