Hugging Face's chief policy officer Irene Solaiman has stated that AI model benchmarking methods are complicated and still maturing, challenging the notion that evaluations provide neutral and accurate comparisons between AI products. The discussion highlights concerns about the reliability of the evaluation systems that AI labs use to demonstrate model capabilities.
Source: TechCrunch
New York City Mayor Zohran Mamdani called on residents to resist inevitability narratives around AI and demand protective regulations. Mamdani criticized federal authorities for abdicating responsibility on AI governance. Source: TechCrunch
Nscale and Anthropic are reportedly in discussions for an additional $45 billion compute deal, following their August announcement of a similar arrangement. The potential deal underscores significant investment in computational infrastructure to support AI model development and deployment. Source: TechCrunch
OpenAI has developed new evaluation methods to improve ChatGPT's performance across low-resource programming languages, including software engineering tasks, SQLite, and COBOL migration. The work addresses capability gaps in languages that have limited training data and real-world usage examples. Source: OpenAI
Microsoft has issued guidance warning that AI agents should be monitored similarly to insider threats, cautioning that frontier models from outside controlled environments cannot be fully trusted. The company recommends enterprises carefully oversee which AI agents their employees deploy and use. Source: TechCrunch