10/17 2024 528
According to Fast Tech on October 17, today, NVIDIA quietly open-sourced an AI model named Nemotron-70B.
Upon its release, this model surpassed over 140 open and closed-source models, including GPT-4 from OpenAI and Claude 3.5 Sonnet from Anthropic, in multiple benchmarks, ranking second only to OpenAI's latest model, o1.
The AI community was stunned, asking if a new open-source king had arrived? Industry insiders also commented that training a small model with Llama 3.1 to outperform GPT-4o was nothing short of genius.
As the name suggests, Nemotron-70B is developed based on Llama-3.1-70B. Without specific prompts or additional reasoning tokens, Nemotron-70B can still answer complex reasoning questions, such as the classic conundrum, "How many 'r's are in 'strawberry'?"
Industry insiders praised NVIDIA's relatively small model, trained on the basis of Llama 3.1, for surpassing GPT-4o and Claude 3.5 Sonnet, calling it a technological leap.
Currently, Llama-3.1-Nemotron-70B-Instruct is available for online experience.
Furthermore, NVIDIA has also open-sourced Nemotron's training dataset, HelpSteer2, which includes the following:
Constructed 21,362 prompt responses to make the model more aligned with human preferences, helpful, factual, and coherent, with customization options based on complexity and detail;
Constructed 20,324 prompt responses for training and 1,038 for validation.