Nas的天空
点亮思维火花，畅享数字世界
用户数
1
文章数
428
评论数
28
阅读量
168021
Latest articles
Apple Silicon压榨指南：MLX生态下本地满血微调与量化实战
别让你的M系列芯片只跑Ollama当聊天玩具。本文基于Apple Silicon统一内存架构与MLX框架，详解如何在macOS上满血运行Llama-3/Qwen2.5的LoRA微调与4-bit量化转换。包含虚拟内存限制解除、Metal统一内存显存分配调优、全流程量化脚本及避坑实测，单机榨干Mac的本地AI生产力。
单卡4090跑满吞吐：vLLM与FastAPI流式网关高并发工程化改造实战
单张RTX 4090部署大模型做线上服务，并发一上来就OOM、首字延迟（TTFT）飙升到秒级？本文分享我将vLLM底层推理引擎与自研FastAPI流式代理网关整合的实战方案。包含PagedAttention核心显存调优、SSE断流重试容错、动态批处理参数压测以及完整的Docker Compose生产级配置，手把手带你榨干单卡性能。
实战踩坑：基于 vLLM + One-API 搭建自建与聚合的大模型高并发生产级网关
本文分享如何使用 vLLM 部署本地开源大模型（以 Qwen2.5 为例），结合 One-API 搭建兼具高吞吐并发、动态负载均衡与 Fallback 降级能力的统一 API 网关。包含显存显卡分配陷阱（CUDA OOM）、流式 SSE 代理超时调优、多卡并行配置以及真实压测数据。
单机双卡打造高可用AI网关：vLLM与LiteLLM流式代理、Prefix Caching与动态降级实战
记录在自建双卡工作站上基于 vLLM 与 LiteLLM 构建私有化 OpenAI 兼容网关的实操全流程。深度解析 AWQ 量化吞吐调优、Prefix Caching 提速 300% 实测、Nginx 流式 SSE 缓冲截断排查，以及 GPU 满载或异常时秒级熔断降级至公网模型的工程化落地指南。
实战踩坑：基于 FastAPI 与 Asyncio 搭建高并发 LLM SSE 流式代理服务
本文记录使用 Python FastAPI 与 Asyncio 重构 LLM 流式代理网关的实操经验。针对高并发场景下大模型 SSE（Server-Sent Events）长连接导致的内存泄漏、协程雪崩、连接池耗尽以及客户端断开后的显存空转问题，提供完整的背压控制、连接生命周期管理与带压测对比的生产级工程化代码。
Latest Comments
hello world
hello world
hello world
web site
It's hard to find knowledgeable people on this topic, however,
you seem like you know what you're talking about!
Thanks
site
You made some really good points there. I looked on the internet to
learn more about the issue and found most
individuals will go along with your views on this site.
1xbet clone script
Hello friends, good paragraph and pleasant arguments commented at this
place, I am in fact enjoying by these.
buy casino script
An intriguing discussion is worth comment. I do believe that you should write
more on this topic, it may not be a taboo matter but typically
people do not speak about such topics. To the next!
Many thanks!!
casino api
I'm excited to find this website. I wanted to
thank you for your time for this fantastic read!! I definitely
liked every bit of it and i also have you book marked to check
out new information on your site.
gudanggacor
It's in reality a great and helpful piece of info. I am happy that
you shared this useful information with us. Please stay us informed like this.
Thank you for sharing.
macau18 link alternatif
Thanks for any other fantastic article. Where else may anyone get that type of information in such an ideal method of writing?
I have a presentation subsequent week, and I am at the look for such information.
all about casino comped cruises
Hi, i read your blog from time to time and i own a similar one
and i was just curious if you get a lot of spam comments?
If so how do you reduce it, any plugin or anything you can recommend?
I get so much lately it's driving me insane so any help is very much appreciated.
medical boric powder
I'm not sure why but this site is loading
very slow for me. Is anyone else having this issue or is it a problem on my end?
I'll check back later and see if the problem still exists.
热评文章
Mac 本地部署 DeepSeek-R1 最佳实践：MLX vs llama.cpp 性能压测与显存优化
在 Apple Silicon（M系列）芯片上本地跑大模型，还在无脑开 Ollama 默认配置？本文基于 M3 Max 与 M2 真实环境，深入对比 llama.cpp 与 Apple 原生 MLX 框架在量化精度、显存占用及推理吞吐量（Token/s）上的实测差异，附带统一内存上限解锁、Metal 编译调优与守护进程自动化配置指南。
实战踩坑：用 FastAPI+SSE 打造低延迟 AI Agent 流式代理与高并发背压治理
在给生产环境 LLM Agent 搭建流式转发中间件时，很多开发者直接套用异步生成器，结果在客户端断联、多轮工具调用并发时频频遭遇连接泄漏与内存暴涨。本文基于 FastAPI、httpx 与 SSE（Server-Sent Events），拆解高并发流式代理中的背压机制、连接清理陷阱与断点续传设计，提供经受过压测检验的生产级工程化代码方案。
实战踩坑：用 Python + vLLM 搭建生产级大模型流式网关（高并发+背压处理）
本文手把手带你基于 Python、FastAPI 与 vLLM 搭建生产级大模型流式代理网关。深度解析 SSE 长连接内存泄漏、客户端断开连接导致的算力白嫖（背压与取消传播）、流式分词解码乱码等真实生产踩坑经验，并附完整的异步流式反向代理架构与压测对比。