Gavin Baker, an xAI investor and hedge fund manager, explains the main problems that OpenAI and the others has was they believed they could not make more than 32000 h100s share the same memory and work coherently using current networking. Elon and then xAI team went to first principles and figured out scaling to 100k chips ...

Read more

---

Link :
https://www.nextbigfuture.com/2024/12/xai-scaling-beyond-33000-h100s-was-thought-impossible.html