I unfortunately have to use GNOME on my work laptop with Ubuntu 24.04 and it is honestly a pain compared to my personal computers running Plasma. The comparison is not entirely fair because I am pitching GNOME from 2024 to the latest version of Plasma, but the difference in UX is night and day. UI is smoother and more fluid, I can configure my system exactly how I want it to be.
The 3B active is small enough that it's decently fast even with experts offloaded to system memory. Any PC with a modern (>=8 GB) GPU and sufficient system memory (at least ~24 GB) will be able to run it okay; I'm pretty happy with just a 7800 XT and DDR4. If you want faster inference you could probably squeeze it into a 24 GB GPU (3090/4090 or 7900 XTX) but 32 GB would be a lot more comfortable (5090 or Radeon Pro).
122B is a more difficult proposition. (Also, keep in mind the 3.6 122B hasn't been released yet and might never be.) With 10B active parameters offloading will be slower - you'd probably want at least 4 channels of DDR5, or 3x 32GB GPUs, or a very expensive Nvidia Pro 6000 Blackwell.
You won't like it, but the answer is Apple. The reason is the unified memory. The GPU can access all 32gb, 64gb, 128gb, 256gb, etc. of RAM.
An easy way (napkin math) to know if you can run a model based on it's parameter size is to consider the parameter size as GB that need to fit in GPU RAM. 35B model needs atleast 35gb of GPU RAM. This is a very simplified way of looking at it and YES, someone is going to say you can offload to CPU, but no one wants to wait 5 seconds for 1 token.
What Strix Halo system has unified memory? A quick google says it's just a static vram allocation in ram, not that CPU and GPU can actively share memory at runtime
You can get tablets, laptops, and desktops. I think windows is more limited and might require static allocation of video memory, not because it's a separate pool, just because windows isn't as flexible.
With linux you can just select the lowest number in bios (usually 256 or 512MB) then let linux balance the needs of the CPU/GPU. So you could easily run a model that requires 96GB or more.
Any good gaming pc can run the 35b-a3 model. Llama cpp with ram offloading. A high end gaming PC can run it at higher speeds.
For your 122b, you need a lot of memory, which is expensive now. And it will be much slower as you need to use mostly system ram.
Seconding this. You can get A3B/A4B models to run with 10+ tok/sec on a modern 6/8GB GPU with 32k context if you optimize things well. The cheapest way to run this model at larger contexts is probably a 12gb RTX 3060.
I can run this on an AMD Framework laptop. A Ryzen 7 (I dont have Ryzen AI, just Ryzen 7 7840U) with 32+48 GB DDR. The Ryzen unified memory is enough, I get 26GB of VRAM at least.
I run Gemma 4 26B-A4B with 256k context (maximum) on Radeon 9070XT 16GB VRAM + 64GB RAM with partial GPU offload (with recommended LMStudio settings) at very reasonable 35 tokens per second, this model is similiar in size so I expect similar performance.
The Q5 quantization (26.6GB) should easily run on a 32GB 5090. The Q4 (22.4GB) should fit on a 24GB 4090, but you may need to drop it down to Q3 (16.8GB) when factoring in the context.
You can also run those on smaller cards by configuring the number of layers on the GPU. That should allow you to run the Q4/Q5 version on a 4090, or on older cards.
You could also run it entirely on the CPU/in RAM if you have 32GB (or ideally 64GB) of RAM.
I currently run the qwen3.5-122B (Q4) on a Strix Halo (Bosgame M5) and am pretty happy with it. Obviously much slower than hosted models. I get ~ 20t/s with empty context and am down to about 14t/s with 100k of context filled.
No tuning at all, just apt install rocm and rebuilding llama.cpp every week or so.
I was planning to build a NAS from OPi 5 to minimise power consumption, but ended up going for a Zen 3 Ryzen CPU and having zero regrets. The savings are miniscule and would not justify the costs.
Same, I used C-a for a while until I found out its default mapping is jumping to the beginning of the line so I rebound tmux prefix to C-Space. Now my only problem is on a few servers where I haven't copied my dotfiles.
This is a great app but it highlights just how insane it is to use apple products where you have to pay a subscription fee for the 3rd party software that provides the basic OS functionality. With KDE Plasma I don't feel the need to install any additional plugins - everything is built-in, coherent and configurable to your liking.
That's a strange argument. Open source software including Plasma Mobile is developed by volunteers who choose to spend their time on a given project. I am quite happy with the pace of Plasma Desktop and the progress made in the past 3 years on its 6th iteration.
As a KDE developer I can say that there are times that we have done things differently or taken the long road because we wanted to support Plasma Mobile.
It would have just been better to continue doing the desktop specific things and let the Plasma Mobile enthusiasts make those changes.
The docs project is part of "La Suite"[1]. They choose Grist[2] as the spreadsheet which is made by an American company but open source and there is a significant contribution from the French it admin.
Interesting. By some luck i've been using Grist for two years and it just feels like the most no nonsense software. But it's a bit different to excel, i would say it's more like airtable. It's more columnar like gui over sqlite database which might be a bit more restricting BUT it greatly helps data integrity.
Actually, it's the exact opposite. There is really no alternative to PowerPoint on Linux, unfortunately. I'm saying this as someone who's used Linux for 20 years now.
Are you just hanging around California startups? I work in big consulting and am inside hundreds of the largest companies in the US, everyone of which is fully Microsoft and only ever seen PowerPoint. I’m in dozens of teams meetings a week across as many organizations and have been in 2 Google meets meeting in the last decade, both of which were California fintech startups.
Yes, most people use MS where I live, too. But most of them only scratch the surface. To this thread's point, 99% of PowerPoint presentations I've seen are just walls of text on a bunch of slides, with the occasional illegible graph.
Now I'm not saying I actually know my way around PPT or that I'm some presentation whiz, but this can probably be done with the browser version. Just like the "new" Outlook is simply a new Edge skin.
I work for a company that has drunk the MS Kool-Aid and then went back for a refill, yet I've never had any issue using the web version of the suite ever since it came out. I don't even run Windows on my work laptop. Teams is the only app that seems marginally better in its heavy version (heh), since it supports separate windows for the calls.
I've been out of the powerpoint loop myself for almost 20 years too; does it actually have any valuable functionality that you can't get on the free alternatives?
I've worked in academia for years (in computer vision labs) and I can confidently say that PowerPoint is the best tool to prepare research presentations.
At least in my field, 90% of presentations are Beamer. PowerPoint is bad at equations just like Word. Besides easily integrating video/animations I can't think of why it would be better.
Could you go into details about why you think this?
I haven't used PowerPoint in years as I think my needs are pretty simple but I wonder what I'm missing.
I can see that the Microsoft ecosystem gives control on who can view files and provides collaboration and control. Both of which would be useful in the corporate world.
Is there's somethnig other than that or is it just ease of use?
For the most part I see people using MS Office tools because it's what they are familar with. They're familar with it because it's the only thing their IT department will allow them to use.
I'm actually constantly surprised by the diversity of experiences I'm seeing here. It's very much not a small bubble, at least not in comparison to any other social network/activity in my life.
Probably just a matter of time, it’s possible the friction will create opportunities. Something in the spirit of iaPresenter, md first would be awesome.
At the moment i have long html page with key event for next and previous, tiny script to check on specif markup for autoscroll.
Huh? There's a ton of PowerPoint alternatives that work on Linux. LibreOffice, OnlyOffice, Collabora Office, Calligra Stage, Google Slides, the online version of PowerPoint, more techy things like LaTeX Beamer or Reveal.js. Maybe these don't have perfect PowerPoint compatibility, or some niche PowerPoint feature you need but there's plenty of slide deck making options that work on Linux.
I tried LibreOffice (Impress) for something simple and it was not good - in fact it would just freeze. Although it did have a feature on MacOS that PowerPoint for Mac didn't, so I ended up using Impress for the first little bit and then PowerPoint for the rest.
Presentation has been a solved problem for more than 2 decades already.
Whenever we are talking migration out of the windows world, there is always a group of MS fanboys that pretend that you can't replace a software with another one if it doesn't even have the exact same set of features down to the smallest details while totally ignoring the interesting features the replacement can have.
The reality is there are never 1:1 replacement and Microsoft would have never had any sort of success in the office area to begin with that sort of nitpicking.
I'd think the only Office part difficult to replace is Excel. It has a lot of functionality, provides a lot of value and is the workhorse of most business processes I see. Now how do you replace THAT?
Poor mother Earth, this race is unsustainable. In order to satiate guys like geohot we are pillaging the natural resources, destroying ecosystems, fucking up the climate.
reply