Sora has developed an early version of our Luna AI software named Luna AI LDIE 1. Here, "LDIE" stands for "Live Data Instruction Engine," a core feature designed to act as an intermediary between our modified Large Language Model (LLM) and the internet. The LDIE scrapes websites for real-time data, processes it, and feeds it to our customized version of Meta's Llama-3.2-3B-Instruct LLM. The LDIE allows the Luna AI LDIE 1 LLM to dynamically update its knowledge base, offering up-to-date, user-friendly responses.
Unlike many LLMs, such as Anthropic's Claude Sonnet 3.5, which are constrained to their static training data, Luna AI LDIE 1 can access and utilize real-time information from the world wide web, and is able to fetch new information from the web with every new prompt. Many LLMs require human intervention or retraining to update, often leading to outdated responses on current events or recent discoveries. Luna AI LDIE 1, however, continuously refreshes its knowledge, avoiding such limitations.
We're well aware that many AI companies (such as OpenAI and xAI) are integrating a web search feature into their products. Luna AI LDIE 1 was created solely for experimentation and learning.
Luna AI LDIE 1 also incorporates an OpenAI o1-like system, in which it the LLM refines its responses through a process similar to chain-of-thought reasoning. Complex problems are broken down into smaller, logical, simple steps, enhancing the accuracy and precision of the LLM's responses. It's also worth pointing out that this feature works in tandem with the LDIE to ensure that not only is the information current, but it's also presented in a coherent, conversational format. However, even with the LDIE aside, the o1-like "thinking" feature allows the LLM to respond with a correct answer by thinking carefully and sequentially from the ground up, rather than using pre-existing answers found on the internet or in training data.
Luna AI LDIE 1 is designed solely for internal use at Sora, not for public release. It serves as an experimental AI model solely for internal learning purposes.
Initial internal results are very promising, as Luna AI LDIE 1 demonstrated far superior performance over the base Llama-3.2-3B LLM it builds upon, as well as the majority of other similarly-sized LLMs available today. While still somewhat unrefined due to its experimental status, it, in my opinion, represents one of the best personal 3 billion parameter AI LLMs developed.
This project also shows that creating advanced LLMs with capabilities for internet interaction and thoughtful response generation isn't exclusive to already-existing tech giants like OpenAI. Anyone can achieve what they have with enough effort, motivation, and perhaps a team.
Using the knowledge gained from this project, we hope to develop more advanced, meaningful, and useful artificial intelligence models and software in the future to benefit humanity in every way we can.