You must log in or register to comment.
340B is fucking huge, holy shit. How big is GPT-4?
The rumor is 1.76 trillion, or 8x220B (mixture of experts) to be specific: https://wandb.ai/byyoung3/ml-news/reports/AI-Expert-Speculates-on-GPT-4-Architecture---Vmlldzo0NzA0Nzg4