Nas的天空
点亮思维火花，畅享数字世界
用户数
1
文章数
514
评论数
21
阅读量
186910
Latest articles
NAS玩转本地私有AI：从Ollama+Docker部署到GPU直通与内存分配优化指南
本文记录了在自建TrueNAS/Unraid家庭NAS服务器上，利用Docker与Nvidia GPU直通部署本地大模型（Ollama + Open-WebUI）的实战全流程。涵盖Container Toolkit踩坑记录、上下文窗口显存挤占优化及真实推理Tokens测速。
告别 Ollama 并发瓶颈：vLLM + LiteLLM 搭建生产级本地大模型高并发网关实战
在面对多 Agent 协同与批量推理场景时，Ollama 极易遭遇显存阻塞与并发队列停滞。本文分享如何基于 vLLM 的 PagedAttention 显存优化技术，搭配 LiteLLM 构建高吞吐、低延迟且全兼容 OpenAI 协议的私有大模型 API 网关，内含生产级 Docker Compose 配置、核心参数调优与压测对比。
榨干双卡算力：基于 vLLM + DeepSeek 蒸馏模型的私有化高吞吐部署与显存压榨实战
记录在双卡工作站上利用 vLLM 部署 DeepSeek-R1-Distill 模型的全流程。深入分析 PCIe 带宽瓶颈、NCCL 通信踩坑、KV Cache 内存碎片与 Chunked Prefill 调优，附带生产级 Docker Compose 配置与真实压测吞吐对比。
榨干 Apple Silicon：Mac 本地大模型高并发推理与统一内存调优实战
针对 Apple Silicon (M系列) 统一内存架构，深度评测与调优 llama.cpp (Metal) 与 Apple 原生 MLX 运行 Qwen2.5/DeepSeek 的实战表现。解析突破 macOS 默认 75% 显存配额限制、编译优化、高并发连续批处理（Continuous Batching）流式部署及排坑指南。
本地部署 DeepSeek-R1 32B 踩坑实录：双卡 RTX 3090 + vLLM 高并发推理调优指南
针对个人实验室与中小团队部署推理大模型的显存瓶颈，本文记录了在双卡 RTX 3090（48G 显存）环境下，使用 vLLM 部署 DeepSeek-R1-Distill-Qwen-32B AWQ 量化版的完整实战过程。涵盖 Docker 容器化配置、KV Cache 显存分配调优、Nginx 流式反代超时断流排障及 Ollama 与 vLLM 的并发压测数据对比。
Latest Comments
macau18 link alternatif
Thanks for any other fantastic article. Where else may anyone get that type of information in such an ideal method of writing?
I have a presentation subsequent week, and I am at the look for such information.
all about casino comped cruises
Hi, i read your blog from time to time and i own a similar one
and i was just curious if you get a lot of spam comments?
If so how do you reduce it, any plugin or anything you can recommend?
I get so much lately it's driving me insane so any help is very much appreciated.
medical boric powder
I'm not sure why but this site is loading
very slow for me. Is anyone else having this issue or is it a problem on my end?
I'll check back later and see if the problem still exists.
best leather conditioner
I have been surfing on-line more than three hours nowadays,
yet I never found any attention-grabbing article like yours.
It is lovely price sufficient for me. In my view, if
all web owners and bloggers made excellent content material
as you did, the net can be much more useful than ever
before.
projector screen hyderabad
Ꭺt Nissi Offife Systems, we specialize іn providing top-of-thе-line projectors,
interactive panels, аnd comprehensive hone theater solutions tߋ transform ｙouг vierwing
experience.
using gain in audio mix
Your method of explaining the whole thing in this article is really good,
all be capable of effortlessly understand it, Thanks a lot.
artistas femeninas de música techno
Simply wish to say your article is as astonishing. The clarity in your post
is just spectacular and i could assume you are an expert
on this subject. Well with your permission let me to grab
your RSS feed to keep updated with forthcoming
post. Thanks a million and please keep up
the enjoyable work.
阿滨
在安装Qb所欲依赖时报错:pkg install qt5-base qt5-qmake libtorrent-rasterbar
Python 基础入门 Day12 - Nas 的天空
[…] Day08: Python 数据分析 […]
Python 基础入门 Day12 - Nas 的天空
[…] Day01: Python 基础入门 […]
热评文章
实战踩坑：用 Python + vLLM 搭建生产级大模型流式网关（高并发+背压处理）
本文手把手带你基于 Python、FastAPI 与 vLLM 搭建生产级大模型流式代理网关。深度解析 SSE 长连接内存泄漏、客户端断开连接导致的算力白嫖（背压与取消传播）、流式分词解码乱码等真实生产踩坑经验，并附完整的异步流式反向代理架构与压测对比。
本地vLLM+Nginx流式网关高并发调优：突破TTFT延迟与显存瓶颈实战
针对本地自建大模型API服务在高并发流式响应（SSE）下的卡顿、显存OOM与TTFT首字延迟飙升问题，本文基于vLLM与Nginx构建高并发流式代理生产架构。结合PagedAttention参数调优、流式反向代理缓冲穿透、动态Batching配置与压测实测数据，详解极客运维与工程落地的避坑指南。
单卡4090跑满吞吐：vLLM + LiteLLM 构建私有高并发流式LLM网关实战
面对多业务线直连本地大模型的并发阻塞与显存碎片痛点，本文记录了在一台24G单卡RTX 4090服务器上，利用vLLM PagedAttention配合LiteLLM搭建企业级高并发流式API网关的完整实战。涵盖显存KV Cache压榨配置、SSE流式响应卡顿排查、动态负载均衡与真实压测数据对比。