Unlurker

↙ time adjusted for second-chance

IPv6 traffic crosses the 50% mark (google.com)

Can someone reconcile for me the constant chatter about how IPv6 isn't getting impemented, versus this result that more than half of all traffic (as measured by google) is now IPv6? It sounds to me like its a tool which is available to be used when needed and when no better workarounds exist, and it is slowly but surely being adopted as needed.

Spain: 9.9% What's going on in Spain?

As a French national, I am surprised to discover we are topping the charts according to this analysis. Does anybody know why that might be the case? What's the story of IPv6 deployment in France?

I worked with the internet society to mobitor ipv6 adoption for the top million sites ipv6matrix.org it's broken down by country so might answer some of your curiosity

The regulatory body, ARCEP, has been very proactive since 2002 (!) on IPv6. The recent uptick is due to IPv6 obligations bundled in the 5G spectrum licences. https://www.arcep.fr/la-regulation/grands-dossiers-internet-et-numerique/lipv6.html

I'm wondering the same thing for India. Not the top but looks surprisingly surprisingly high. Perhaps I'm reading the data wrong.

Technical literacy, hacker culture, and culture of well-considered infrastructure, have been French characteristics - at least, historically. Has something changed for the worse?

I'm surprised it's reporting is listed <5% - I thought it was pretty much ipv6 first?

Zoom in on that graph using the controls at the bottom, and you'll see a repeating pattern of crests and troughs, weekly. There's about a 5% difference between the crests and the troughs: the crests are hitting the 50% line or just below it, and the troughs are down around 45%. The real question is, why are the crests so predictable? They're always on Saturdays; Sunday dips down a little below the crest, then Monday-Friday is down in the 45% range before the next Saturday jumps up to 50% again. (Fridays usually have a small rise, up to the 46-47% area). My theory: mobile access rises on weekends. People are more often accessing Google services from their work computers Monday-Friday, but on Saturdays and Sundays most (not all) people are away from the office. Many of them will end up using smartphones rather than laptops for Internet access, for various reasons such as being outdoors. And since smartphones are nearly all using IPv6 these days, that means an uptick in IPv6 usage over the weekends.

It's not just mobile networking but residential ISPs in general have better IPv6 support. In the US, Comcast was one of the first big IPv6 deployments, in Europe CGNAT+IPv6 is common in many places. Meanwhile corporate IT for business and education networks have less incentive to upgrade and typically lag behind in adoption in general.

If GitHub flipped a switch and enabled IPv6 it would instantly break many of their customers who have configured IP based access controls [1]. If the customer's network supports IPv6, the traffic would switch, and if they haven't added their IPv6 addresses to the policy ... boom everything breaks. This is a tricky problem; providers don't have an easy way to correlate addresses or update policies pro-actively. And customers hate it when things suddenly break no matter how well you go about it. [1] https://docs.github.com/en/enterprise-cloud@latest/organizations/keeping-your-organization-secure/managing-security-settings-for-your-organization/managing-allowed-ip-addresses-for-your-organization

Anyone who relies on IP filtering for security deserves to have it broken. Change my mind.

Having being messing around personally with getting my own blocks of IP addresses and routing[1] - I've become terrified at the idea of implementing access control based on IP address. Unless your own organisation in the RR has the IP addresses assigned to you as Provider Independent resources, there just seems to be so many places where 'your' IP address could, albeit most likely accidentally, become not yours any more. And even then, just like domain names, stop renewing the registration and someone else will get them - I was that someone else recently... [1] AS202858

Meanwhile: one of the major mobile network in my country announced cisco collab/ipv6 ~5 years ago, but still doesn't provide v6, just v4 CGNAT. Personal web server running dual stack since early 2010s currently sees 18-20% v6 traffic. When split by type, counting only mobile users it reaches 30% at peak. Bot/crawler traffic is ironically 100% v4. Meanwhile: enabled h3 in september last year for the fun of it, instantly at >40% traffic by request count, passing 50% since the beginning of the year, h2 accounting almost all the remaining traffic and plain ssl/http requests <1% being just bots.

Maybe the best anti-scraper/LLM protection is going IPv6 only. I'd do that on my website, but I'm afraid some clients might not connect.

NB: this is not "IPv6 traffic crosses the 50% mark" but "availability of IPv6 connectivity among Google users", which is a very important difference. This means roughly half of Google users have IPv6 capability , which does not 1:1 correspond how much traffic is actually transferred over IPv6, which is what this submission says in the title.

It also means you're excluding China, who has has it as a long-term priority to deploy IPv6 and have made huge strides.

One of the foremost obstacles to wide adoption is that IPv4 still works great and it's ubiquitous. There is no advantage or up-side to deprecating or abandoning IPv4 support at all. The only result of disabling IPv4 is a denial of service to a certain sector of customers or clients. The only way this will change is by increasing pressure on the resource of IPv4 networks. It was a few years ago that AWS broke the news to me that I'd be paying for IPv4 addresses but IPv6 would remain free. If enough services are forced, financially, to abandon an IPv4 presence, then their clients would be likewise forced to adopt IPv6 in order to retain connectivity. But with the ubiquity of CGNAT and other technologies, it seems unrealistic that IPv4 will become so rare that it becomes prohibitively expensive, or must be widely abandoned. So that availability of the legacy protocol will inhibit widespread adoption and transitions to IPv6.

[delayed]

IPv4-with-more-bytes is not backwards compatible with IPv4. So you'd have to replace/upgrade every existing network stack, both hardware and software. To get, basically, the same effect as moving to IPv6.

> IPv4-with-more-bytes is not backwards compatible with IPv4 Neither is IPv6 > To get, basically, the same effect as moving to IPv6 The only thing that IPv6 solves which is of interest to 99.99% of the users is having more adressable space. The rest of IPv6 features are either things that nobody asked for, or things which are genuinely worst compared to IPv4. I consider the mere fact of enabling IPv6 an unacceptable security risk, as I would now have to make sure my IPv4 and IPv6 firewall stack are perfectly mirroring each other. That would be trivial with IPv4-with-more-bytes, it's a nightmare with IPv6.

I wonder why Germany has a relative high adoption rate with 77%? They are normally behind when it comes to new technology. Is it because they have more carrier NAT? In Denmark I can get cheap 1 / 1 Gbit/s fiber, but still no ipv6 :(

We have enough IPv4 addresses (combined with CGNAT) in Denmark so the providers have no business incentive to spend money on supporting IPv6 :/

Carrier-grade NAT for home connections is pretty rare in Germany. I only know of Deutsche Glasfaser - a fairly new ISP that isn't doing too well.

Vodafone cable's cgnat struggles. I went v6 for home so that at least the v6 sites and my own connections avoid the congestion.

Dell, HP and Lenovo have had laptops with cellular modems for maybe 15 years at this point.

I can confirm this. I work at an e-waste recycling company, and the vast majority of my inventory is corporate IT decommissioned gear. About 1 out of 10 laptops I tear down has a cellular modem, going back to about Intel Core 5th gen.

Perhaps a little tin foil hatty and definitely not the only reason but Microsoft owns Github and also makes a boatload of money off of Azure. Incumbent cloud providers like Azure have a major advantage in terms of having plenty of IPv4 addressing available whereas a new entrant to that market would have to buy or lease that space at a premium. Thus, these companies have an incentive to keep IPv4 a necessity.

IPv4 is going to be a necessity for many many decades no matter what Microsoft do. Even when IPv6 is at 99%, people aren't going to want 1 in every 100 people to not be able to access their site at all. It'll need to be like 99.9% before we start seeing serious IPv6-only services.

Or the most likely more expensive rate limiting (computational wise)

I mean, given how the site performs on average I don't think they've optimized so much that the extra cpu cycles of ANDing with the fixed constant of 2^64-1 and then looking up or hashing a 16 byte integer - whatever they do - rather than a 4 byte one would increase the load significantly. Let's be pessimistic and say it's 20 extra cpu cycles, that's not gonna be much of a problem if their load balancers were made in the past 20 years.

Outdated beliefs probably. When I talk about v6 support in our b2b saas, PM laughs and says nobody uses that shit. Big tech are massive laggards on this funnily enough.

Definitely not for the biggest ones. Google and Meta have so many machines in their data centers that IPv6 addressing becomes a technical necessity due to the risk of exhausting the RFC 1918 address space. Naturally, they were early adopters of IPv6.

IPv6 is a recursive WTF. It might _look_ like a conservative expansion of IPv4, but it's really not. A lot of operational experience and practices from IPv4 don't apply to IPv6. For example, in IPv4 each host has one local net address, and the gateway uses NAT to let it speak with the Internet. Simple and clean. In IPv6 each host has multiple global addresses. But if your global connection goes down, these addresses are supposed to be withdrawn. So your hosts can end up with _no_ addresses. ULA was invented to solve this, but the source selection rules are STILL being debated: https://www.ietf.org/archive/id/draft-ietf-6man-rfc6724-update-23.html Then there's DHCP. With IPv4 the almost-universal DHCP serves as an easy way to do network inspection. With IPv6 there's literally _nothing_ similar. Stateful DHCPv6 is not supported on Android (because its engineers are hell-bent on preventing IPv6). And even when it's supported, the protocol doesn't require clients to identify themselves with a human-readable hostname. Then there's IP fragmentation and PMTU that are a burning trash fire. Or the IPv6 extension headers. Or.... In short, there are VERY good reasons why IPv6 has been floundering.

How do the working IPv6 deployments cope with these issues?

The reason: Skill issue.

The big reason is that domestic ISPs don't want to switch (not just in the US, but everywhere really.) Data centers and most physical devices made the jump pretty early (I don't recall a time where the VPS providers I used didn't allow for IPv6 and every device I've used has allowed IPv6 in the last 2 decades besides some retro handhelds), but domestic ISPs have been lagging behind. Mobile networks are switching en masse because of them just running into internal limits of IPv4. Domestic ISPs don't have that pressure; unlike mobile networks (where 1 connection needing an IP = 1 device), they have an extra layer in place (1 connection needing an IP = 1 router and intranet), which significantly reduces that pressure. The lifespan of domestic ISP provided hardware is also completely unbound by anything resembling a security patch cycle, cost amortization or value depreciation. If an ISP supplies a device, unless it fundamentally breaks to a point where it quite literally doesn't work anymore (basically hardware failure), it's going to be in place forever. It took over 10 years to kill WEP in favor of WPA on consumer grade hardware. To support IPv6, domestic ISP providers need to do a mass product recall for all their ancient tech and they don't want to do that, because there's no real pressure to do it. IPv6 exists concurrently with IPv4, so it's easier for ISPs to make anyone wanting to host things pay extra for an IPv4 address (externalizing an ever increasing cost on sysadmins as the IP space runs out of addresses) rather than upgrade the underlying tech. The internet default for user facing stuff is still IPv4, not IPv6. If you want to force IPv6 adoption, major sites basically need to stop routing over IPv4. Let's say Google becomes inaccessible over IPv4 - I guarantee you that within a year, ISPs will suddenly see a much greater shift towards IPv6.

Except that is completely wrong. Consumer/residential networks have significantly higher ipv6 adoption rates that corporate/enterprise networks. That is why you see such clear patterns (weekend vs weekday) in the adoption graphs.

Sure, the data plane supports it - but what about the management plane? I wouldn't be surprised if ISPs did all the management tasks through a 30-year-old homebrew pile of technical debt, with lots of things relying on basic assumptions like "every connection has exactly one ip address, which is 32 bits long". Porting all of that to support ipv6 can easily be a multi-year project.

This is true, I worked for an old ISP/mobile carrier that started in the 80s about 10-15 years ago. They had basically any system you could think of still running, from decently modern vmware with windows and linux to hp-ux, openvms, sunos, AIX, etc. Could walk around and see hardware 30 years old still going, I think one console router had an uptime of 14 years or so. One time I opened a cabinet and found a pentium 1 desktop pc on the floor still running and connected, served some webpage. The old SMSC from the 80s on DEC hardware was still in its racks though not operational, they didn't need the space as the room couldn't provide enough power or cooling for more than a few modern racks. The planning program for fiber, transmission, racks, etc, required such an old java that new security bugs didn't apply to it, and looked and worked like an old mainframe program. The core team supported ipv6 for a long time, but its rather easy to do that part. The hard part is the customer edge and CPE and the stack to manage it, it may have a lifetime of 2 decades.

That's extremely common unless on "active" fiber (vs GPON, DOCSIS3, DSL, most fixed wireless, satellite, mobile, etc.) Your wifi isn't symmetrical either.

Those are designed to have static asymmetrical bandwidth though, *dm split gives ISP side more of possible shared bandwidth. Wifi bandwidth is shared and dynamic so client can use all of it.

Germany suspends military approval for long stays abroad for men under 45 (bbc.com)

The law is still there though and thus its application can be resumed at any time without need of parliament approval.

The whole handling of it is utterly incompetent. To reinstate a law and then have it informally and temporarily deactivated. https://youtu.be/dZUu6OkTHlY

I cannot understand how German men can be expected to fight while women are exempt. It's pure sexism against men and also very insulting to women. I'm opposed to conscription in general, but I live in Sweden with gender-neutral conscription laws, and I would do my best to defend my country if it became necessary. If I were a man in an alternate version of Sweden with male-only conscription, I would feel so disrespected and devalued by the state that I couldn't imagine myself defending it, so I would either join a non-state-affiliated resistance group or flee the country. If I lived in Germany right now (even as a woman, but especially as a man), I would seriously consider emigrating to a more egalitarian country as soon as possible.

I think you ask the question wrong. There can be endless debates about whether woman should fight or not. The _real_ question is why only German men are restricted by the law. Even if women do not fight they should be subjected to the same restrictions as they'd have country-bound functions in a war scenario as well - be it fighting or not. And I'd even go a step further and argue that the rule should apply to each and everybody in Germany. It's kind of ridiculous that German men have their movement restricted because of a hypothetical defense situation while Ukrainian men are not just invited by Germany to avoid being drafted and they can come and go as they like. Having said that. The real problem with that law is not even the law itself but how it came to being, which unveils a completely messed up and incompetent legislative procedure in the German government and parliament.

The US military does not do combat, look at real engaged armies like the Ukranian/Russian one which are the closest examples to modern warfare between nation states.

How do you mean? I see daily videos of Russian men being killed by drones.

I agree, I meant that armies engaged in conflicts are male like all armies have been in history, save the Soviets who had female battalions for propaganda purposes

We already disgrace ourselves by having some of the core ideologies of our society being blatant lies. We have been repeatedly told that "equality" is primordial to our values. That men had to forego their privileges in the name of it. The hypocrisy of the defenders of those ideas suddenly being so complacent when we look at the other side of the coin is revolting.

> That men had to forego their privileges in the name of it. Care to name a few?

https://www.tagesschau.de/inland/innenpolitik/wehrdienst-auslandsreise-genehmigung-100.html "Die Regelung habe bereits in den Zeiten des Kalten Krieges gegolten "und hatte keine praktische Relevanz", teilte das Ministerium mit. Sie sei auch nicht sanktioniert. Im Gegensatz zur alten Fassung gilt die Genehmigungspflicht nun auch außerhalb des Spannungs- und Verteidigungsfalls." The rule existed, but apparently they broadened the scope. In any case, even if the rule is ignored nothing happens - so the question is of course why that rule exists in the first place of course.

When there's a rule with a condition that meant the rule hasn't applied for decades, and then the condition is removed so that the rule always applies, it's no longer the same rule.

If NATO still exists by then.

If Poland is attacked then Germany will deploy troops there. Whether NATO exists or not.

It's ridiculous that the German government now has to officially state "Please ignore the letter of the law, we didn't mean it that way."

That is the main problem with it. It shows the incompetence of our legislative procedures.

↙ time adjusted for second-chance

Too much discussion of the XOR swap trick (heather.cafe)

Used to do this in assembly back when registers actually mattered. Today it mostly hurts readability more than anything.

> Are there other XOR tricks? Yes, error correction. You have some packets of data a, b, c. Add one additional packet z that is computed as z = a ^ b ^ c. Now whenever one of a, b or c gets corrupted or lost, it can be reconstructed by computing the XOR of all the others. So if b is lost: b = a ^ c ^ z. This works for any packet, but only one. If multiple are lost, this will fail. There are way better error correction algorithms, but I like the simplicity of this one.

https://en.wikipedia.org/wiki/XOR_linked_list

Whether people are liable is a question for the courts, and I suspect they simply look through the tech and ask "do you end up with a copy of the work?" (unless you're an AI company, in which case you can copy the whole internet just fine)

But that's an unsolvable question. Just like when A is caught on camera stealing a diamond, but A turns out to have an identical twin B. So the prosecutor can't do anything.

That's called encryption using a one time pad.

That's one way of looking at it. Note that you can easily extend it to multiple people holding multiple keys, so that a=b^c^d^e, etc.

A Look into NaviDial, Japan's Legacy Phone Service (tokyodev.com)

I ran into this recently with ANA (All Nippon Airways). [1] There’s no better way to kill customer loyalty than to put someone on hold for 45 minutes on a charged phone call to make a basic change to a reservation that should be possible on the website (they also refuse to handle reservation-specific inquiries over email). Just had a look and JAL (Japan Airlines) also doesn’t offer a FreeCall number. I haven’t had reason to call them yet though, so not sure if hold times are as bad as ANA. ANA recently redid their website and it’s a flaming pile of garbage (I could go on another whole rant here about all the ways it’s broken), which I suspect may be increasing call volumes for the past year or so. [1] https://www.ana.co.jp/en/jp/guide/contact/international/international_top/

This is bizarre! When you call ANA from Australia they have multiple dedicated 1800 (free) numbers you can reach them at, including a call centre in the Philippines for simple issues that picked up straight away for me. I wonder why they're so much more customer hostile in their own geography?

I try to fly with Skymark when I can because their website is gloriously basic in the best way possible, it's like barebones server-rendered HTML. And you can book your ticket without payment being in the critical flow. You get like 24 hours to pay and that removes SO much stress from booking airline tickets. I hope they never change or "modernize" it with some shitty JS framework.

If the suicide prevention hotline cost $15/hr in the USA people would freak out! But since it's Japan it's whatever.

Because people want to copy the best things of another society, and not the worst.

Darkbloom – Private inference on idle Macs (darkbloom.dev)

Having strong SETI@Home vibes from 25 years ago, except of course, this is not for the greater good of humanity, but a for-profit project. Problem is, from a technical point of view, what kind of made sense back then (most people running desktops, fans always on, energy saving minimal) is kind of stupid today (even if your laptop has no fan, would you want it to be always generating heat?)... I definitely want my laptops to be cool, quiet and idle most of the time.

I think it’s important that systems like this exist, but getting them off the ground is non-trivial. We’ve been building something similar for image/video models for the past few months, and it’s made me think distribution might be the real bottleneck. It’s proving difficult to get enough early usage to reach the point where the system becomes more interesting on its own. Curious how others have approached that bootstrap problem. Thanks in advance.

Why isn’t a MacBook Air M5 on the hardware list?

no fans

I kind of see your point, but I also kind of don't. Sure, it would be great if you'd immediately get hammered with hundreds of requests and start make money quickly. It would also be great if it was a bit more transparent, and you could see more stats (what counts as "idle"? Is my machine currently eligible to serve models?). But it's still very new, I'd say give it some time and let's see how it goes. If you have it running and you get zero requests, it uses close to zero power above what your computer uses anyway. It doesn't cost you anything to have it running, and if you get requests, you make money. Seems like an easy decision to me.

Bootstrapping will be near-impossible (or incredibly costly) unless they offer inference consumers models with established demand arriving at some least-cost router service where they can undercut the competition (if they actually can). And then dogfood the opportunistic provider side on their own Macs, but with a preference to putting third parties first in the queue. Everything else is just wishful thinking.

No it's not the same premise at all. Crypto doesn't do anything useful for legitimate businesses. AI inference is very useful for legitimate businesses, and so are residential IP proxies for scraping. And by definition, residential IPs cannot be centralized. And as building GPUs becomes more expensive, the existing pool of second hand unused hardware becomes more valuable, not less. The problem with crypto mining is that it quickly becomes unprofitable for small scale deployments. I'm not sure if AI inference would be, especially for the decentralized benefits of lower latency.

It is the same premise, because the person you are responding to is not talking about the moral implications at all, only about the financial / hardware implications. Running AI inference increases the power draw, and requires certain hardware. Mining bitcoin increases the power draw, and requires certain hardware. OP's point thus stands: Bad players will find places to get far cheaper power than the intended audience, and will buy dedicated hardware, at which point the money you can earn to do this will soon drop below the costs for power (for folks like you and me). Maybe that won't happen, but why won't that happen?

FSF trying to contact Google about spammer sending 10k+ mails from Gmail account (daedal.io)

I’m old enough to remember when the FSF said that blocking spam was censorship. Good to see them wake up.

I got a human being at Google to look into my problem and take action after sending a police report to Google‘s legal department certified mail return receipt along with a letter describing how someone was impersonating me and my business using a Gmail address in an attempt to commit fraud. Yes, it was a pain to take all of these steps and it probably took about 3 hours but it was absolutely necessary considering there was no avenue for me to shut down this person otherwise.

I was getting spam called constantly every 5 minutes (blocked by Google call screening) and the attackers made an error if sending a message with their AWS bucket url. I was able to submit an abuse report to Amazon and puff Amazon dismantled the entire spam group. No more spam since then. Maybe try saying the spam has porn or inappropriate images?

I'm reporting every spamm mail that I get through Gmail from Gmail accounts but it doesn't seem to help!

someone hooked up their web app to Google Workspace email and the web app got pwned. Google Workspace email is very generous with the kind of outgoing email you can send via their SMTP servers.

I gave up on trying to report abuse to Google, Amazon or Microsoft. It seems reports simply get ignored and the big providers do nothing. I hope the FSF with its weight and media presence can finally do something. Google, Microsoft, and Amazon are my major sources of spam. These days, this is where spam comes from. At this point, they are also too big to block. We allowed this to happen, through neglect and laziness. Even in this discussion: how many people use Gmail as their primary email service?

On YouTube I reported bot accounts for a couple days, the only reaction I got was that at some point it showed a popup that told me too many false reports would lead to a ban. Not sure what Google gets out of it, but there is no way they could be that bad at fighting bots unless they're not even trying. Even trivial tricks like copy-pasted texts keep working.

Bot comments and uploads count in KPIs. Blocking/Removing bots = KPIs look worse.

I hope you realise, it does sound like you are suggesting that salespeople in your company are essentially spammers.

Most of the salespeople in any company are spammers.

Indeed, Mailchimp is a tool specifically built and advertised to send spam .

Where does it say on their website that it is specifically for sending spam?

Someone’s marketing emails are someone else’s spam. Mailchimp is specifically made for mass email emission, for marketing a newsletter and whatnot. So yeah, a lot of people will consider them spammers.

The local LLM ecosystem doesn’t need Ollama (sleepingrobots.com)

It is a bit off-topic, but would it possible to provide a light mode for this blog? I used to work during the day time, and my pupils had to contract to read, making it a very poor reading experience.

MIT license lets you do what you want with the code. That's the deal. The blob storage thing is the real problem though. Nobody talks about it until they try to move their models somewhere else.

Well, yeah, which is why it's silly when people use MIT licenses and then complain that those, for example, with the motto "Build > ask. Disrupt or die.", only take and don't contribute anything back, instead of using a license that demands it.

Has anybody figured some of the best flags to compile llama.cpp for rocm? I'm using the framework desktop and the Vulkan backend, because it was easier to compile out of the box, but I feel there's large peformance gains on the table by swtiching to rocm. Not sure if installing with brew on ubuntu would be easier.

Not huge gains with rocm but you do get a little faster preprocessing speed

The name "llama.cpp" doesn't seem very friendly anymore nowadays... Back then, "llama" probably referred to those models from Facebook, and now those Llama series models clearly can't represent the strongest open-source models anymore...

https://en.wikipedia.org/wiki/List_of_generic_and_genericized_trademarks

>solved the UX problem. >One command Notwithstanding the fact that there's about zero difference between `ollama run model-name` and `llama-cpp -hf model-name`, and that running things in the terminal is already a gigantic UX blocker (Ollama's popularity comes from the fact that it has a GUI), why are you putting the blame back on an open source project that owes you approximately zero communication ?

> Notwithstanding the fact that there's about zero difference between `ollama run model-name` and `llama-cpp -hf model-name` There is a TON of difference. Ollama downloads the model from its own model library server, sticks it somewhere in your home folder with a hashed name and a proprietary configuration that doesn't use the in built metadata specified by the model creator. So you can't share it with any other tool, you can't change parameters like temp on the fly, and you are stuck with whatever quants they offer.

And that's exactly why llama.cpp is not usable by casual users. They follow the "move fast and break things" model. With ollama, you just have to make sure you're getting/building the latest version.

Its not possible to run the latest model architectures without 'moving fast'. The only thing broken here is that they are trying to use an old version with a new model.

The reported alternatives seem pretty User-Friendly to me: > LM Studio gives you a GUI if that’s what you want. It uses llama.cpp under the hood, exposes all the knobs, and supports any GGUF model without lock-in. > Jan( https://www.jan.ai/ ) is another open-source desktop app with a clean chat interface and local-first design. > Msty( https://msty.ai/ ) offers a polished GUI with multi-model support and built-in RAG. koboldcpp is another option with a web UI and extensive configuration options. API wise: LM Studio has REST, OpenAI and more API Compatibilities.

All of those options were either too slow, or didnt work for me (Mac with Intel). I could have spent hours googling, but I downloaded Ollama and it just worked. So no, they are not alternatives to ollama

I think the biggest advantage for me with ollama is the ability to "hotswap" models with different utility instead of restarting the server with different models combined with the simple "ollama pull model". In other words, it has been quite convenient. Due to this post I had to search a bit and it seems that llama.cpp recently got router support[1], so I need to have a look at this. My main use for this is a discord bot where I have different models for different features like replying to messages with images/video or pure text, and non reply generation of sentiment and image descriptions. These all perform best with different models and it has been very convenient for the server to just swap in and out models on request. [1] https://huggingface.co/blog/ggml-org/model-management-in-llamacpp

Llama-server which is part of llamacpp does this for a few months now