Nvidia is building a trillion-parameter open model, and it would still be smaller than China’s

Nvidia released Nemotron 3.5 Lightning on Tuesday. It is a 30 billion parameter mixture-of-experts model with three billion active at any moment. The architecture is a hybrid of Mamba-2, MoE and attention layers, with a one million token context wind...