The Pitfalls of '1 Million Token Support'—Why AI Performance Dips as Conversations Grow, and How We Moved Memory 'Outside'

Prologue: Continuing the Series and Greetings from the FieldThe first article (Billing Troubles Edition) covered the ...

Real-time monitoring that strips bare the memory, power, and running processes of an iPhone

In my previous article, I wrote about my struggle to investigate why the apps on my beloved iPhone 13 mini kept crashing, ...
blog.csdn

Python异步Web框架Starlette：轻量高性能ASGI实践指南

1. 项目概述：为什么我会盯上这个冷门好用的ASGI框架 聊到Python Web框架，绝大多数人第一反应是Flask或Django，近期可能还会提到FastAPI。但有件事很少人注意：FastAPI底下真正干重活的，其实是starlette——一个体量不大、文档漂亮、设计极简的ASGI框架。我第一次注意到它，是因为某次在高并发的实时推 ...
blog.csdn

vLLM AsyncLLM深度解析：用Python协程突破高并发推理瓶颈

本内容遵循CC 4.0 BY-SA版权协议 把 vLLM 部署上线之后，我第一次压测就吃了大亏：单条请求的时延看着挺漂亮，一旦并发拉到 20 以上，请求就在排队，GPU 利用率却只有 60% 出头。后来排查发现 ...