Training and Serving System of Foundation Models: A Comprehensive Survey
Foundation models (e.g., ChatGPT, DALL-E, PengCheng Mind, PanGu-$Sigma$) have demonstrated extraordinary performance in key technological areas, such as natural language processing and visual recognition, and have become the mainstream trend of artificial general intelligence.This has led more and more major technology giants to dedicate significan