Solar 10.7B

Solar 10.7B is a 10.7-billion-parameter language model released in 2023 by Upstage AI, a South Korean AI company. It achieved top benchmark scores among models in its parameter size class when released.

Depth up-scaling is Upstage's technique of taking Meta's Llama 2 architecture and extending its depth (number of layers) to create a more capable model without training entirely from scratch. This approach allowed efficient scaling from an existing open foundation.

Upstage AI, a South Korean AI startup, developed Solar 10.7B. The company focuses on practical large language model applications and has ties to the Korean AI research community.

Yes, Solar 10.7B's model weights were released publicly, making it available for research and commercial use. Its strong benchmark performance at the 10-billion-parameter scale attracted significant attention in the open-source LLM community.