Heaptalk, Jakarta — Sakana AI unleashed a suite of AI models, developed through a distinct method called evolutionary model merge, inspired by evolution and collective intelligence.
The evolutionary model merge automatically discovers several ways to merge models from various domains, for instance, non-English language and Math, or non-English language and Vision, that might be difficult for humans to discover themselves. The most successful models from each generation were then identified as parents for the next generation.
“To test our approach, we initially tested our method to automatically evolve for us a Japanese Large Language Model (LLM) capable of Math reasoning, and a Japanese Vision-Language Model (VLM),” Sakana AI stated on its official page (03/21).
As a result, LLM and VLM achieved the top performance on several benchmarks. The evolved Japanese Math LLM, a 7B parameter model, achieved the top performance on other Japanese LLM benchmarks. Meanwhile, the Japanese VLM can handle culturally specific content and achieve top results when tested on a Japan-sourced dataset of Japanese image-description pairs.
Therefore, the startup released three Japanese language models called EvoLLM-JP, EvoVLM-JP, and EvoSDXL-JP. The two of them, EvoLLM-JP and EvoVLM-JP, are being open-sourced in Hugging Face and GitHub to accelerate the AI model development in Japan. The startup said, “Our goal isn’t to just train any particular individual model. We want to create the machinery to automatically generate foundation models for us.”
Based in Tokyo, Sakana AI was founded in 2023 by former Google researchers David Ha and Llion Jones. In January 2024, the startup secured $30 million in funding led by Lux Capital to develop nature-inspired AI in Japan. Khosla Ventures also injected some investments into this startup at the seed stage. The Japanese tech ecosystem supports this startup as well, including NTT Group, KDDI CVC, and Sony Group.
