Startup Martian Introduces Cost-Reducing Tool for AI Models
Shriyash Upadhyay and Etan Ginsberg, AI researchers from the University of Pennsylvania, have noticed a concerning trend in the AI industry. They believe that many large AI companies are sacrificing basic research in order to focus on developing competitive and powerful AI models. The duo attributes this to market dynamics, where the majority of funds raised by companies are allocated towards staying ahead of rivals rather than studying fundamentals.
Martian Emerges with a Cost-Reducing Tool for AI Models
Seeing the need for change, Upadhyay and Ginsberg decided to found their own company, Martian, with a mission to further interpretability research rather than capabilities research. Martian has recently emerged from stealth with $9 million in funding from investors including NEA, Prosus Ventures, Carya Venture Partners, and General Catalyst. The company aims to address the challenge of making AI research profitable.
A Solution for Expensive Language Models
Martian’s first product is a “model router,” which is a tool designed to optimize the usage of large language models (LLMs) such as GPT-4. Instead of relying on a single LLM for each endpoint, Martian’s model router automatically routes prompts to the “best” LLM based on factors such as uptime, skillset, and cost-to-performance ratio. By utilizing a team of models in an application, companies can achieve higher performance and lower costs compared to relying solely on a high-end LLM.
Maximizing Performance and Minimizing Costs
Using a high-end LLM like GPT-4 can be prohibitively expensive for many companies. For example, Permutable.ai revealed that it costs them over $1 million per year to process 2 million articles per day using OpenAI’s high-end models. Martian addresses this issue by estimating how a model performs without actually running it, allowing the model router to intelligently switch to cheaper models when possible. This enables companies to achieve similar performance to expensive models while minimizing costs.
Pioneering Breakthrough in Model Routing
Martian claims that their model router stands out from the competition due to their deep understanding of how these models fundamentally work. This understanding is crucial in building an effective model router. Although other startups, such as Credal, offer automatic model-switching tools, Martian’s ability to estimate model performance and seamlessly incorporate new models into applications sets them apart.
Early Uptake and Future Prospects
Martian has already gained traction among “multibillion-dollar” companies, indicating the potential value of their cost-reducing tool. The success of Martian will depend on the competitiveness of their pricing and their ability to deliver in high-stakes commercial scenarios. By pioneering a breakthrough in model routing, Martian aims to revolutionize the AI industry and make AI research more profitable.