What happened to open weights models?

Open weights models haven't disappeared but have evolved significantly from their early days of widespread adoption. Initially fueled by community demand and hardware democratization (e.g., models like Llama 2, Mistral 7B, and Stable Diffusion), they've since faced challenges that reshaped their role. The landscape now reflects a shift toward 'open weights' (releasing model parameters) rather than full open source (which includes training data and pipelines), driven by strategic and economic factors.

Key developments include:

1. ·Compute and Cost Barriers: Training state-of-the-art models now requires millions in compute (e.g., Llama 3 405B at $10M+), limiting access to well-funded players like Meta, Mistral, and Alibaba.
2. ·Performance Convergence: Open models (e.g., Llama 3, Mistral 8x7B) have narrowed the gap with closed counterparts (GPT-4, Claude 3) in many benchmarks, but closed models still lead in multimodal, long-context, and agentic tasks.
3. ·Licensing and Legal Risks: Lawsuits (e.g., NYT vs. OpenAI) and restrictive licenses (e.g., Llama 2’s custom terms) have made companies cautious about releasing models trained on scraped data.
4. ·Business Model Shifts: Many open-weight providers now monetize inference (e.g., Mistral’s La Plateforme) or hybrid approaches (open weights + closed APIs), while closed APIs dominate for ease of use.
5. ·Niche Strengths: Open weights excel in privacy-sensitive applications (healthcare, finance), fine-tuning (via LoRA/QLoRA), and edge deployment (smaller models like Phi-3-mini).

The future is bifurcated: open weights will underpin infrastructure (customization, privacy, cost efficiency), while closed models dominate frontier capabilities and integrated services. Regulatory pressures (EU AI Act, U.S. executive orders) may further shape this divide, with open models potentially facing stricter scrutiny for high-capability releases.