Unlurker|Replay|About
Artificial Analysis Intelligence Index v5 (artificialanalysis.ai)
•
Honestly it does somewhat match what I’ve felt playing a lot with all OpenAI / Claude / deepseek models One thing I’m still surprised / unsure on is haiku 5.5 max besting latest deepseek - but also I think the last haiku was so incredibly bad I still hold some bias whilst using the new one.
•
Glancing over the methodology[1] made me curious about three things: 1. Is there a benchmark measuring how much better an LLM performs when you tell it "you are an X"? 2. Does telling an AI agent "you are an AI agent" actually help? Is it the LLM equivalent of reminding a senior developer they're a senior developer, making them feel guilty about being lazy and finally getting them to do their job? 3. What happens when the persona conflicts with the task? If you tell it "you are a PM" and then ask it to write code, does it struggle? Produce worse code? Spawn a sub-agent and tell it they're a developer? [1] https://artificialanalysis.ai/methodology/intelligence-benchmarking
•
Comparing the scoring rubric for AAv5.0 vs AAv4.3: AAv5.0: 25% Knowledge work: AA-Briefcase v1.1 - 15% AutomationBench-AA - 5% GDP.pdf - 5% (-5%) 25% Coding: FrontierCode 2.0 - 15% (new) Terminal-Bench4.0 - 10% 25% Science: Terminal-Bench Science .1- 15% (new) CritPt v2 (hard subset) - 10% (new version) 25% General: AA-Omniscience v1.1 - 15% HLE-Diamond - 10% AAv4.3: 30% Agents: AA-Briefcase v1.1 - 15% GDPval-AA v2 - 10% AutomationBench-AA - 5% 20% Coding: Terminal-Bench4.0 - 10% SciCode - 10% 30% General: AA-Omniscience (Accuracy) - 10% AA-Omniscience (Non-hallucination) - 5% GDP.pdf - 10% AA-LCR v1.1 - 5% 20% Scientific Reasoning: HLE - 10% CritPt 10% tldr: gained Terminal-Bench Science and FrontierCode, dropped SciCode, GDPval-AA and AA-LCR. Lowered weight of GDP.pdf.
I keep seeing that MiMo is in the frontier in terms of performance and cost but I see very few talking about it and the only ones that do talk about it are doing it in context of AA
•
I used MiMo for a couple of days waiting for my Codex sub to reset. I really liked it, its communication style was a breath of fresh air compared to how stale gpt-sol is.
•
It's also dog slow at AA index tasks compared to other models in its class. Makes me think they optimised for getting on the intelligence/cost frontier at the cost of real world usability.
Git for the US Code (lexifina.com)
•
I'm doing this as open source for all amendment-driven law for the world. Finnish/Estonian law basically solved now but US/UK are a lot more complex to do correctly for some reason. Now major version coming within 1 month probably with many architectural changes. The thesis is that it's basically all doable as a compiler but there may be certain cases which are irreducibly ambiguous and need human/AI to adjudicate them. The rest must be deterministic. https://github.com/eliask/lawvm
•
Given that chicken scratch in the margins gets into bills that are voted upon, it seems unlikely that this would be accepted by Congress. Love it though.
•
German Constitution "Grundgesetz" on Github with proper time stamps https://github.com/c3e/grundgesetz
The system would be gamed by the politicians. Insert language you know will never get enough votes Don’t strike out but add language to make it useless Plus the bill sponsor often makes changes others ask for, so the true author isn’t noted It would be a mess and ignored by the public after a while
•
But there would be a papertrail that can be revisited. And journalists can access to keep people informed of systemic or major issues.
•
Every system gets gamed, and so is the current one. The goal is that the people can more easily stay informed. Even with mixed authors problem you highlight would be still an improvement.
We don’t need version control for legislative bills. We need journalists that track down who tried to insert what, summarize it, and present it to the public with context. That would satisfy the oversight purpose without getting lost in technicalities. Our problem is that the press isn’t doing their job, not that legislation doesn’t have a commit graph. The latter, even if it existed, would not fix the former.
•
> Our problem is that the press isn’t doing their job The problem is, barely anyone is willing to pay the press for doing this job. Outrage farming and clickbait make money, honest reporting does not.
 Top