InferSim is a lightweight simulator for LLM inference, written in pure Python without any 3rd-party depenencies. It calculates the TTFT, TPOT and throughput TGS (tokens/GPU/second) based on ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results
Feedback