GitHub - SJTU-IPADS/PowerInfer: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs - GitHub - SJTU-IPADS/PowerInfer: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

Related Keywords

Shanghai ,China ,Falcon Re ,Powerinfer Relulla ,Zeyu Mi ,Haotong Xie ,Haibo Chen ,Shanghai Jiao Tong University , ,Fast Large Language Model Serving ,Large Language Model ,Deployment Ease ,Hugging Face ,Original Model Weights ,Yixin Song ,Distributed Systems ,Shanghai Jiao Tong ,