Apple, to me, is the winner in the end of this AI chapter.
Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.
I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
I think the question to me is whether Apple can ship affordable consumer devices with enough memory to run sufficiently useful models sufficiently fast.
Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.
There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]
Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.
With how expensive memory is, IDK if they can pull this off but I hope they do
> Would that imply the entry-level MacBook will have 128GB
Most people neither need nor want to run large LLMs locally.
The entry level MacBook Neo costs $599. Even if RAM prices revert to the pre-AI mean, the entry level models are not going to be that stacked with RAM.
Eventually we'll get to a doubling where entry level means 16GB of RAM, but most people won't be running anything more than simple models for light local work.
Have you ever tried running large LLMs on a MacBook Pro? It's hot, loud, and makes your battery level disappear at rates you didn't know were possible if you're not plugged in. It's a very niche hobby.
Even if it's 2030 - that's a mere 3 years from now - it will take at least that long for the greater economy to figure out how to use LLMs. And probably a memory boom/bust cycle in that time too.
If Apple, in 2030 has a compute product that a worker can BYO(X) to work that has 2TB RAM, with an encrypted personalized context that can float between agents, then they will be in a very interesting place.
> whether Apple can ship affordable consumer devices with enough memory to run sufficiently useful models sufficiently fast.
i suspect that they will at least do offloading for casual usage as well. if they can ship 24GB in a homeview or appletv style product, it's game over.
The Mac Studio hardware can do 512GB. That might be a hardware limit of the current architecture.
But the family shouldn’t be limited by that. We’re not going to see M6 Max hardware, because the M7 line is being fast-tracked and pulled forward. And I wouldn’t be at all surprised to see M7 capable of 1-2TB or more.
> That might be a hardware limit of the current architecture.
It is, at the very least, a practical limit. There are unlikely to be enough customers willing to pay $50k+ for a hypothetical 1TB RAM version to make it worth manufacturing.
Why is there a desire to have this stuff on the edge on the engineers lap instead of just building some arbitrarily scalable cluster for I'm guessing cheaper than apple's usual premium? Habitual practice from the engineers who would compile on the device getting hot in their lap? Lack of widespread hpc experience compared to running docker?
I don’t want it in my laptop, I want it in my mini server rack at home. I’m not running any of my personal data through an external service that is certainly collecting it as training data.
I want it in neither my laptop nor my home server, which I don't even have, I want it in my phone or something even smaller. I understand that it's currently not possible, but this is the direction I want the technology to be moving into.
Many users would prefer to use a Sonnet-level (or better) LLM without sharing their data with others.
And with a device like this, you pay one single time to buy the product and then never have to think about paying for tokens ever again. For ordinary users that's silly, but for users or businesses chewing through lots of tokens, this means they can create their own turnkey replacement for "the cloud" where the only cost after purchasing the machine is the electricity to run it (which with these machines is very little). Mix that with the privacy benefits and suddenly it makes a lot of sense.
Seems everyone in this thread is assuming I am saying go to private third party compute. No. Build your own cluster on metal you own on property you own, and have your team use that and just some thin clients to write with. Your compute goes way further. Your cluster can be going 100% of the time with jobs queued up by your team. All that memory you buy for your team in macbooks is idle and unavailable when your employee closes the lid and tosses it into the bag into the car on the way home. It could be used 24/7. On hardware you own. For cheaper to build I'm sure than apple hardware tax. On easier OS to manage and no doubt deal with anything involving any sort of heavy compute work than OS X. Are we really going to waste resources with docker when we could be using a node completely? Come on.
Like I said, seems HPC is lost art. All four sub replies missed that in my original comment.
Your main argument is about better allocation for that RAM, which only makes sense in the conditions of a shortage. If RAM is plentiful, like it’s bound to be once the datacenters craze is over, that argument becomes “why does every apartment need a washer/dryer when we can have shared washers/dryers in a common area that are being used 100% of the time”. The answer is “because we can and it’s not that huge of a difference” plus you can have both. Like you can have a server cluster for large batch jobs, and personal ones for personal use.
For individuals though, it’s the same reason plenty of people pay for the 2tb or 4tb SSD for their laptops or desktops even though NAS has been a thing for 40 years. Maintaining a “server” of any sort is a completely separate endeavor for most. Just like a NAS is a no-brainer for many and non-starter for many more.
It's investors making these decisions. They would rather have predictable cloud spend than trying any experiments with high cap-ex. Has been this way since AWS is a thing.
I agree with you, and i would like to set up a server at work for local inference but we simply do not have the knowledge, it's much easier to get a couple of AI Max machines that are good enough for me and the boss which i convinced on the good points on running clankers locally and be done with it (though my plan is to use the AI MAX machines to figure out what the infrastructure and workflow will look like, and at some point ask for the investment)
Because why not? If you can get unlimited inference with roughly Opus 5.5 intelligence for $10k that easily beats current API pricing. Like not even a question.
Apple’s position down the road. When the Chinese company CMXT takes a much bigger share of the memory market worldwide, and they will. Their dominance in memory will probably be pretty close to what BYD is today in the worldwide car industry.
I don’t think Apple can be in a position where all of the Chinese device-making companies can get memory at a significantly cheaper price from CMXT, and Apple is stuck with buying memory from the three hyenas of memory at a significantly higher price.
That’s not going to help Apple sell their devices at a competitive price worldwide over time against the Chinese computer makers (Lenovo, Asus, and Motorola) and make no mistake about it: their share of the market worldwide will get bigger because of this change in the memory market; their focus will be on the long term, while the lawyers/MBA/finance types in the West will have their heads stuck in the short term.
It’s the (Chinese companies’) opportunity, and they’re not going to let it pass; I wouldn’t. In addition, Apple is in competition with Nvidia, Microsoft, Intel, AMD, Qualcomm, Google, Meta, and indirectly with Anthropic and OpenAI (whose CEO, Sam Altman, recently declared that computer ram is wasted on low-level regular computer users), along with a whole legion of other 3D model makers on the procurement of memory.
The combination of all these elements going into the future means that if Apple wants to stay ahead, I believe Apple has no choice but to take memory in house, they have the design and engineering capability in-house, and they have the money because they didn’t burn it on AI models and data centers.
This scenario is no different than replacing Intel or Qualcomm; you can see the writing/destination and the ultimate train wreck up ahead if Apple doesn’t roll up its sleeves and get busy, the market has changed and again Apple has to change and adapt to the new conditions.
No but they can pay for new manufacturing to be built. 2-3 lead time should be enough to bankroll the effort. Apple has to much money too sit around hoping that memory situation resolves itself. They were caught flat footed by the RAM market.
In-house the market has changed, the conditions have changed, the price has changed the only way around this problem is to design and engineer around it in-house. The design and engineering capability isn’t the problem, money isn’t the problem. It’s whether or not the new CEO/Apple have the will to do what’s necessary. That’s pretty much the bottom line… they have done it before (Intel/Qualcomm), and they now have to do it again.
In-house, Apple has no fab. They have labs, but manufacturing memory would make them critically reliant on an expensive fab partner, probably driving them back to be an Intel customer again.
Design, engineering capability and money are all a problem here, and the onus falls on you to prove otherwise.
they didnt realy caught flat, they knew their yoy growth rates and demands. NO AI company had that future demand so certain for all hardware they have. they could easily lock years of ram supply for next years extra demand, becausw they already have cash too, but instead they went after few % margin there or whatever. they now enjoy it
not exactly. I mean they saw increasing ram prices + they knew their demand. They did almost nothing for what months and year or more now? Lack of actions cost us a lot more than apple probably. They are happy to sell at few times more expensive ram, but here I'm thinking they could have great edge, they wasted it with all cash in hand.
I dunno, I reckon they are targeting 64GB and will develop a smaller model thats built in by default that handles a lot of simple tasks quickly.
Probably sitting under a router, so that it can hand off bigger jobs to an Apple DC somewhere, probably with a certain number of credits based on your iCloud sub.
> I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
I don't think so. LLM inference enjoys huge economies of scale. At no point in the foreseeable future will a local model be able to compete with a cloud model in terms of price. Local models will have the privacy angle, but very few people will be willing to pay top dollar for that. There will be some people, for sure, but the vast majority will just use cloud models.
For inference alone they get far better utilization of the hardware because they can keep it fully loaded with maximum concurrency. Other than that they get to collect a lot of data for training.
Most people just Google something or use the free ChatGPT. They have no clue how the answers get there and frankly would be just as pleased with GPT 4o as they would be with Astra. Questions like “what does this light mean on my dashboard” or “which bidet is best”.
Small models inherently have less pre-trained knowledge, which will hurt their performance on queries like this. Moreover a lot of queries benefit from doing web searches, especially if they need to respond to changing conditions such as your bike example. Now you have to send data through a data center anyway for good performance, and probably need to use an API if you don’t want your computer blocked. So what’s the point, when GPT Luna exists if you really want to get a decent model at a rock-bottom price?
My guess is it will probably be both. There was a time when no one could envision computing without the mainframe.
In the mainframe's heyday, it was difficult to grasp a world in which compute became so dense and so cheap that it made sense for each individual to have all processing and memory on their desk, talk less of on their lap and in their hand.
Of course this time could be different. But, when we look at previous relentless progress in compute and efficiency, then also couple that with the breakthroughs AI itself unlocks, it's hard for me to imagine that centralized AI will meet both the scale and flexibility desired for all use cases.
Just as centralized compute never fully disappeared -- it took the form of client-server computing and the cloud, there's no question that centralized inference will continue to exist.
But it wouldn't surprise me if, ten years from now, 90% of global inference happens at the client.
While simultaneously, the 10% of inference that takes place in the datacenter is wildly lucrative and orders of magnitude greater than today.
Put simply: the inference pie may become so large that both client and datacenter win.
Unless cloud PCs take off and kill "AI PCs" and the future of the end-user hardware market, I think it'll be a bit of both.
It'll become easier and more affordable to distill/train/optimize smaller local models to meet your specific work needs. Multi-agent efforts will orchestrate the balance between using on-prem hardware for most tasks, and then defer to cloud models for more demanding/edge-case tasks. Especially if an organization can manage what data is shared with which model, and most of their user base are everyday casual users.
I also expect cloud providers to eventually start charging different rates for on-peak and off-peak usage, which would incentivize running what you can with the local compute you have otherwise sitting idle across dozens to hundreds to thousands of machines in your fleet.
Anthropic just changed their policy to invalidate any unused credits without refund if you buy your API credits annually. I see that as a sign of unpredictable budgets for cloud models for customers.
I could be wrong and all your future hardware purchases for end-users will be thin clients, without the local processing capabilities of today's laptops and towers. I don't see the hardware manufacturers profiting much from that, but maybe we'll see more companies like AMD declaring they prioritize cloud compute customers over everything else.
With a hardware refresh for cloud computing every 3-5 years, we'll likely see today's compute hardware driving Opus and Sol available on the secondary market for companies willing to build more on-prem capabilities.
Cloud compute has been completely commoditized. Yes, local models will get better and better, but I think for the vast majority it will continue to be both cheaper and more productive to use cloud based models, far into the future, where bulk purchasing, oversubscription, and discounted industrial electric rates can be taken advantage of.
Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break over a decade in the future, for comparable cloud models. Yes, there's privacy, etc, but now you're down into fractions of a percent of users who care about the government reading the memory of their computer.
In the end, I think companies like OpenAI and Anthropic will be the nameless and faceless backend provider, and they will get decoupled from the users. At which point, if they haven’t already had their multi-trillion dollar IPO and everyone cashed out, I don’t think they’re going to.
Apple and Microsoft and other companies will own that front-end experience with the customer, and it’s not going to matter to most customers whose backend gets used for what.
I agree with you, even if many of the pundits say that Apple was left in the dust.
At some point, the real cost of inference/intelligence is going to come due, and the existing open source models will be good enough for 95% of tasks users want (or training their own model(s) will be inexpensive for Apple).
Apple will be there, as usual, to provide secure, on-device, faster inference that the vast majority of their customers will be fine with. Let everyone incur the R&D costs.
> Apple gets this and we'll see the result with M7 Max/ultra chips
I'm not sure which chip it'll be, but I now for sure I'm not buying new devices until the AI boom has somewhat settled and we kinda understand what we'll be using it for. Until then, every manufacturer will just be guessing what device to build and will be outdated within a few months.
You don't have to sell it as a local model. Sell it as offline capabilities. Sell it as private background processing while your phone is idle and plugged in. Sell it as un-enshittified LLM once Anthropic and OpenAI inevitably screw everybody.
Take a look at how qwen3.8 27B compares on benchmarks to much, much larger models. A while back I called it the crow of AI models: paradoxically strong given its tiny size, like the brains of crows and other intelligent birds.
It's not comparable to huge models of course, but given the power/size ratio here it shows that serious gains are on the table for models that can run fast on a high-end laptop.
Man, if all that makes Apple a "winner" then I'm dying to hear your assessment of Nvidia.
Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.
The design and engineering of memory is not more difficult than replacing Intel and Qualcomm, Apple certainly has the money, and they certainly have the engineering talent within the Apple Silicon design group. It took 13 years to replace Intel and six years to replace Qualcomm who has already warned their investors that they anticipate losing revenue in the near future. The question is whether or not they have the will to do so, money and engineering ability isn’t the problem, it is simply a matter of will. With CMXT coming online bigger and stronger.
I don’t think there will be a choice long-term for Apple the conditions of the market have changed, and you have to change with it or get left by the wayside. They have shown that they will adapt and they certainly haven’t wasted money like their tech competition over the years.
Difference being, Apple had experience designing CPU cores and wireless radios before ditching Intel and Qualcomm. The transition felt "magical" because it was a generally low-effort layer on top of a muscle that Apple had gradually built over time. Their design approach is not compatible with scaled manufacturing; Apple does not have experience manufacturing DRAM at all. They don't have their own fabs, nor could they bid on fab space when AI companies outspend them 10:1.
Apple would make more money investing in a CUDA-style architecture than they would by becoming a subsistance manufacturer for the pitiful margins of the iPhone, iPad and Mac.
I don't think that's materially different from what you can do with Nvidia and AMD GPUs, besides the fact that you're limited to Thunderbolt interconnect.
Yeah, what apple have frequently got right is the actual product/UX.
Capability is one thing but a focus on the actual product, as an teleological destination that is built rather than something cobbled together as a solution to a problem about how to sell what you have generally has taken them the right way.
For all of those accolades, the Mac is still a minority player in every computer market. They're especially a non-entity in on-prem enterprise buildouts and datacenter construction, which is where the money is going right now.
It's nice to have these types of niches when they make you money. But the Mac isn't profitable like the iPhone is, and probably makes less money than the iPad does if you count aftermarket service sales. The further Apple drifts from their roots as a computer manufacturer, the more the Mac starts to look like a liability to them.
Apple's Last Stand is their prosumer segment, and once that's gone I don't think their vertical integration playbook gets them anywhere further. Their bottom line is dependent on finding new ways to make B2C profitable, which has resulted in subscription services like AppleTV/Apple Music/iCloud so far.
I really do not understand the value of running large models locally. I am absolutely fond of local models but for specific tasks. The agents need to do a lot of work in this case: orchestration, big or small model split of planning and work, split into small tasks, merging, etc.
LLMs with <30B parameter models seem like the sweet spot. I would rather focus on multiple tiny LLMs that are tuned to specific tasks (SQL queries, finance/accounting, backend/frontend dev, etc.).
For larger models, a $10-40 ish Neuralwatt/Opencode Go or any similar subscription goes so far for the average user that it is ridiculous. We are still in the "youth" era of agents. There are many patterns that work well with smaller LLMs. Or with a big-small LLM combination.
My context:
I own the tiny 16GB M4 Mac Mini and I use LLMs (coding agents) to build software and a few of my products use LLMs directly. Full-time user of LLMs, do not read or write code manually. I run the tiny Qwen series (<10B params) sometimes but a couple $20 subscriptions (Claude, Neuralwatt) take care of my full-time agent usage.
Model/harness that runs locally, which you can connect from your phone to do non-dev queries (looking stuff up, recommendations, planning and etc.) can save ~$300/year on various subscriptions.
Agreed but I feel we will see even cheaper annual or prepaid options. Say, a $20 upfront and get the cheapest inference, comparable to Sonnet with subscription. For a personal assistant that can run on laptop and phone, these will be mainstream. $20 may last a couple months for vast majority of users.
True that. But ZDR is also not that expensive. Although for many businesses, buying a Mac with tons of memory is not expensive. Running a dedicated agent on top of private data is certainly valuable.
Yeah. I for example use local inference for tagging up my documents that I self-host in a local Paperless instance, and I don't really feel like sending my medical invoices to clammy Sammy.
I have the same Mac M4 16gb. Do you have the Qwen model assist the larger models for menial tasks/tool calls during coding it can handle? As in have them work in tandem to save on tokens/calls on the large models or are your local agents just meant for their own specific side tasks?
A mix of both. I have been experimenting building agents that focus on small/tiny LLMs. For example, I built a web-app prototyping harness which used Qwen 4/9B models but separate agents (tool calls, prompts and actual code to parse) for CSS management, TSX management, Router, state, etc. It used SolidJS only.
There was a task manager which was given context of the tech stack so it could split the incoming requests. Then branch out the tasks to appropriate agents. Combine the generated code - this was a file/module/import manager.
The other way is let tiny models (again I would use agents for different use cases) do the busy work - tool calling, finding evidence, etc. I usually setup agents to re-write tool call history and create fresh API requests without using multi-turn chat with tool call history.
The addressable market is enthusiasts. The cheapest and least loyal consumer that exists. But I guess it is just a small side project for apple which is fine.
Idk. Once they say our AI app is facilitating AI needs of big number of DAUs, the stock price will likely go up even if there’s no revenue path. It’s a pretty tulip.
"Harbor is currently in a private preview under Apple’s Claris brand"
There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.
If they keep Harbor under the Claris umbrella, that is a nudge that they are doing it for SMB market instead of the consumer market and it will likely be a onerous subscription to use it.
If you ask who would pay for Apple local models, I would. A Mac Studio high end is $30k one-time versus my current run rate of $10k/week for Claude Code Sonnet (if I paid API rates rather than Ultra plan).
If/when Anthropic changes to pure API pricing, that's when I'd need to switch to more local larger models.
I currently run a Mac with 128GB and qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a 200k LOC codebase for security auditing and hardening for many hours at a time. This isn't as strong/fast as Sonnet, yet the cost is only $6K one-time.
yes, but you pay the Ultra plan, that's the whole point, isn't it? Let's be honest also - qwen3.8:27b-mlx is nowhere near Sonnet, both in size and cap, and if you do $10k a week of usage, you should know that.
> Cluster for scale: “Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer’s capacity, or serve more concurrent requests across your cluster.”
This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.
This is what I've wanted for the last few years. I have played around with local models and had some success, but I don't want to deal with the hassle of setting up, updating, etc.
If all of this were easy, I would happily invest in new dedicated hardware to run it. This could be a full-blown Mac or it could be a souped up HomePod-type device, loaded with RAM. Better yet, make it a revitalized Matter-enabled Airport!
I really want capable models local but it’s impossible for me to afford the hardware. If there is a magical way to run glm 5.3 without quantization at 80-100tps with a total spend of less than 10K I’d be happy but that’s impossible now.
> $200 a month is only $2400 a year. That’s 4.16 years until you spend 10k.
I don’t find it reasonable at all. Subsidized Subscriptions or api for cheaper models, is the best value.
If you buy your own hardware, then after those four years you still have that hardware, and it likely still holds significant value. Plus you know the local model you're running today won't change or suddenly degrade in four months, let alone four years. Whereas after four years of paying subscriptions and API costs, you have literally nothing.
Oh? How come the four year old 4090 RTX cannot be purchased for less than (or even at) MSRP then? Can you buy me one? I'm willing to pay $500 even if it is worth zero in your eyes, that's a stellar deal.
After I think about it, this makes sense that Claris would tackle such a project under the auspices of Apple. They're a subsidiary so they can monkey around with this stuff without having to adhere to the draconian everyday operating restrictions of "Apple proper". Good on them if they can pivot into something that hews to AI.
"Harbor is currently in a private preview under Apple’s Claris brand, but it seems likely that we’ll see it launch to the public in the weeks and months ahead."
Claris?? Ok, this sounds like it's just someone's personal project. Something like this that was actually going to ship would be in a different org.
If true, it is an astounding story. That said, I'm happy to see that Apple ships a graphing calculator with macOS 27 (albeit, a different one). I may not use it much these days, but I have fond memories of it when I was in university.
https://getlocalcode.app/ runs local AI models (larger ones of course, DeepSeek 4.1 Flash 1017) easy. You can download and try it now. The problem is elsewhere: you cannot find M5 Ultra that you can purchase and set it up today.
(1) All the headaches of maintaining FileMaker Server on Apple hardware, (2) multiplied with the additional requirement to cluster devices, (3) but without users accustomed to the silliness of misusing consumer devices running a consumer operating system for a service that must be reliable, and (4) an Apple subsidiary unable to bring about meaningful change.
It was pretty fast to get it set up. It could be better for sure, as it's not a first class concept the same way that it is in linux, but an hour of google search ai mode will get you where you need to go.
I know you can set it up to make it seem to work, but from what I’ve seen people say over and over, it will eventually end up asking you some question through a dialog box. And you will have no idea that’s going on other than all work stops. And the machine seems locked up or something until you connect a display and find out it wants you to click OK.
MacOS is a desktop OS. You really REALLY need to treat it like one.
Lots of things pop up and can’t be dismissed any other way than though mouse/keyboard on the display.
OS updated and wants to tell you it worked? No command line way to get rid of that. Program wants to do something and one of those system security prompts pops up (“BBEdit wants to access your Documents folder”)? Same problem.
You can set it to auto login and auto update and grant permissions and everything.
And then, one day, it will stop responding and you’ll have to hook it up and log in to see some pointless dialog that just wanted someone to click “OK”. And until you do no work gets done.
OMG after 25 years, Claris gets something beyond FileMaker (among the most legacy software still alive)? Claris would not have the engineering talent, so why would Apple use them as a vector?
Currently FileMaker is a really expensive but sophisticated developer deployment environment. They stayed pretty true to the original vision in the mid 80’s when they merged their DOS database and added a gui layer for the Macintosh. I started on version 2 and it was at its best at when it went relational in version 3. Since then they have been focusing on deployment of graphical data applications and being a better database.
An Apple M5 Ultra running an app like that? Because Apple hardware is a monolith, unlike the PC space, "Harbor" will be configured out of the box with the ideal settings for many well-known open-weight models.
... Looks like I picked the wrong week to buy parts for a linux-based inference box.
Cheap remote is probably not going to last forever. Memory prices will eventually drop. Local models will be more than enough for many purposes when the hardware is available cheaply enough.
Hopefully it won't be another preinstalled bloatware app like Image Playground. macOS enshittification has been greatly accelerating, even giving Win 11 Home a run for its money.
Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.
I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.
There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]
Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.
With how expensive memory is, IDK if they can pull this off but I hope they do
[1] https://www.tomshardware.com/tech-industry/semiconductors/ap...
Most people neither need nor want to run large LLMs locally.
The entry level MacBook Neo costs $599. Even if RAM prices revert to the pre-AI mean, the entry level models are not going to be that stacked with RAM.
Eventually we'll get to a doubling where entry level means 16GB of RAM, but most people won't be running anything more than simple models for light local work.
Have you ever tried running large LLMs on a MacBook Pro? It's hot, loud, and makes your battery level disappear at rates you didn't know were possible if you're not plugged in. It's a very niche hobby.
I'm old enough to remember the time when most people neither needed nor wanted a phone with a large screen and no buttons
If Apple, in 2030 has a compute product that a worker can BYO(X) to work that has 2TB RAM, with an encrypted personalized context that can float between agents, then they will be in a very interesting place.
holy shit
i suspect that they will at least do offloading for casual usage as well. if they can ship 24GB in a homeview or appletv style product, it's game over.
But the family shouldn’t be limited by that. We’re not going to see M6 Max hardware, because the M7 line is being fast-tracked and pulled forward. And I wouldn’t be at all surprised to see M7 capable of 1-2TB or more.
It is, at the very least, a practical limit. There are unlikely to be enough customers willing to pay $50k+ for a hypothetical 1TB RAM version to make it worth manufacturing.
Spoiler alert: they're supply-constrained too. Their customers will pay to outbid Apple, just like how Apple outbid the industry on TSMC's 5nm node.
And with a device like this, you pay one single time to buy the product and then never have to think about paying for tokens ever again. For ordinary users that's silly, but for users or businesses chewing through lots of tokens, this means they can create their own turnkey replacement for "the cloud" where the only cost after purchasing the machine is the electricity to run it (which with these machines is very little). Mix that with the privacy benefits and suddenly it makes a lot of sense.
Like I said, seems HPC is lost art. All four sub replies missed that in my original comment.
For individuals though, it’s the same reason plenty of people pay for the 2tb or 4tb SSD for their laptops or desktops even though NAS has been a thing for 40 years. Maintaining a “server” of any sort is a completely separate endeavor for most. Just like a NAS is a no-brainer for many and non-starter for many more.
Not saying they're right.
I agree with you, and i would like to set up a server at work for local inference but we simply do not have the knowledge, it's much easier to get a couple of AI Max machines that are good enough for me and the boss which i convinced on the good points on running clankers locally and be done with it (though my plan is to use the AI MAX machines to figure out what the infrastructure and workflow will look like, and at some point ask for the investment)
How is this literally even possible in 2026? In a thread about LLMs, no less?
I don’t think Apple can be in a position where all of the Chinese device-making companies can get memory at a significantly cheaper price from CMXT, and Apple is stuck with buying memory from the three hyenas of memory at a significantly higher price.
That’s not going to help Apple sell their devices at a competitive price worldwide over time against the Chinese computer makers (Lenovo, Asus, and Motorola) and make no mistake about it: their share of the market worldwide will get bigger because of this change in the memory market; their focus will be on the long term, while the lawyers/MBA/finance types in the West will have their heads stuck in the short term.
It’s the (Chinese companies’) opportunity, and they’re not going to let it pass; I wouldn’t. In addition, Apple is in competition with Nvidia, Microsoft, Intel, AMD, Qualcomm, Google, Meta, and indirectly with Anthropic and OpenAI (whose CEO, Sam Altman, recently declared that computer ram is wasted on low-level regular computer users), along with a whole legion of other 3D model makers on the procurement of memory.
The combination of all these elements going into the future means that if Apple wants to stay ahead, I believe Apple has no choice but to take memory in house, they have the design and engineering capability in-house, and they have the money because they didn’t burn it on AI models and data centers.
This scenario is no different than replacing Intel or Qualcomm; you can see the writing/destination and the ultimate train wreck up ahead if Apple doesn’t roll up its sleeves and get busy, the market has changed and again Apple has to change and adapt to the new conditions.
Even so, DDR5 is so high-demand that it's likely that Apple can't outbid customers like Nvidia or OpenAI when they drive the margins they do.
Design, engineering capability and money are all a problem here, and the onus falls on you to prove otherwise.
Probably sitting under a router, so that it can hand off bigger jobs to an Apple DC somewhere, probably with a certain number of credits based on your iCloud sub.
I don't think so. LLM inference enjoys huge economies of scale. At no point in the foreseeable future will a local model be able to compete with a cloud model in terms of price. Local models will have the privacy angle, but very few people will be willing to pay top dollar for that. There will be some people, for sure, but the vast majority will just use cloud models.
Maybe when chip would infer at 10-20x the speed of a 5090 RTX, otherwise it's really..bad.
What new bike should I get? How should I rephrase this email? What to do about my persistent headaches? Recipes based on these ingredients?
In the mainframe's heyday, it was difficult to grasp a world in which compute became so dense and so cheap that it made sense for each individual to have all processing and memory on their desk, talk less of on their lap and in their hand.
Of course this time could be different. But, when we look at previous relentless progress in compute and efficiency, then also couple that with the breakthroughs AI itself unlocks, it's hard for me to imagine that centralized AI will meet both the scale and flexibility desired for all use cases.
Just as centralized compute never fully disappeared -- it took the form of client-server computing and the cloud, there's no question that centralized inference will continue to exist.
But it wouldn't surprise me if, ten years from now, 90% of global inference happens at the client.
While simultaneously, the 10% of inference that takes place in the datacenter is wildly lucrative and orders of magnitude greater than today.
Put simply: the inference pie may become so large that both client and datacenter win.
It'll become easier and more affordable to distill/train/optimize smaller local models to meet your specific work needs. Multi-agent efforts will orchestrate the balance between using on-prem hardware for most tasks, and then defer to cloud models for more demanding/edge-case tasks. Especially if an organization can manage what data is shared with which model, and most of their user base are everyday casual users.
I also expect cloud providers to eventually start charging different rates for on-peak and off-peak usage, which would incentivize running what you can with the local compute you have otherwise sitting idle across dozens to hundreds to thousands of machines in your fleet.
Anthropic just changed their policy to invalidate any unused credits without refund if you buy your API credits annually. I see that as a sign of unpredictable budgets for cloud models for customers.
I could be wrong and all your future hardware purchases for end-users will be thin clients, without the local processing capabilities of today's laptops and towers. I don't see the hardware manufacturers profiting much from that, but maybe we'll see more companies like AMD declaring they prioritize cloud compute customers over everything else.
With a hardware refresh for cloud computing every 3-5 years, we'll likely see today's compute hardware driving Opus and Sol available on the secondary market for companies willing to build more on-prem capabilities.
It's not as if they are short of money.
Or are you saying that Apple should become a memory manufacturer, rather than working with TSMC?
Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break over a decade in the future, for comparable cloud models. Yes, there's privacy, etc, but now you're down into fractions of a percent of users who care about the government reading the memory of their computer.
Apple and Microsoft and other companies will own that front-end experience with the customer, and it’s not going to matter to most customers whose backend gets used for what.
At some point, the real cost of inference/intelligence is going to come due, and the existing open source models will be good enough for 95% of tasks users want (or training their own model(s) will be inexpensive for Apple).
Apple will be there, as usual, to provide secure, on-device, faster inference that the vast majority of their customers will be fine with. Let everyone incur the R&D costs.
I'm not sure which chip it'll be, but I now for sure I'm not buying new devices until the AI boom has somewhat settled and we kinda understand what we'll be using it for. Until then, every manufacturer will just be guessing what device to build and will be outdated within a few months.
It's not comparable to huge models of course, but given the power/size ratio here it shows that serious gains are on the table for models that can run fast on a high-end laptop.
A few years back we never thought open source llm would get as good and even the small and cheap model
Its good to have a race and such competition
Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.
I don’t think there will be a choice long-term for Apple the conditions of the market have changed, and you have to change with it or get left by the wayside. They have shown that they will adapt and they certainly haven’t wasted money like their tech competition over the years.
Apple would make more money investing in a CUDA-style architecture than they would by becoming a subsistance manufacturer for the pitiful margins of the iPhone, iPad and Mac.
Capability is one thing but a focus on the actual product, as an teleological destination that is built rather than something cobbled together as a solution to a problem about how to sell what you have generally has taken them the right way.
It's nice to have these types of niches when they make you money. But the Mac isn't profitable like the iPhone is, and probably makes less money than the iPad does if you count aftermarket service sales. The further Apple drifts from their roots as a computer manufacturer, the more the Mac starts to look like a liability to them.
Apple's Last Stand is their prosumer segment, and once that's gone I don't think their vertical integration playbook gets them anywhere further. Their bottom line is dependent on finding new ways to make B2C profitable, which has resulted in subscription services like AppleTV/Apple Music/iCloud so far.
And Apple really wants to sell people who have no cred in the space their stuff because THEY have no cred in the space.
Its basically a club for stupid f**s
LLMs with <30B parameter models seem like the sweet spot. I would rather focus on multiple tiny LLMs that are tuned to specific tasks (SQL queries, finance/accounting, backend/frontend dev, etc.).
For larger models, a $10-40 ish Neuralwatt/Opencode Go or any similar subscription goes so far for the average user that it is ridiculous. We are still in the "youth" era of agents. There are many patterns that work well with smaller LLMs. Or with a big-small LLM combination.
My context:
I own the tiny 16GB M4 Mac Mini and I use LLMs (coding agents) to build software and a few of my products use LLMs directly. Full-time user of LLMs, do not read or write code manually. I run the tiny Qwen series (<10B params) sometimes but a couple $20 subscriptions (Claude, Neuralwatt) take care of my full-time agent usage.
There was a task manager which was given context of the tech stack so it could split the incoming requests. Then branch out the tasks to appropriate agents. Combine the generated code - this was a file/module/import manager.
The other way is let tiny models (again I would use agents for different use cases) do the busy work - tool calling, finding evidence, etc. I usually setup agents to re-write tool call history and create fresh API requests without using multi-turn chat with tool call history.
There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.
Looks like they've mostly just released annual upgrades to FileMaker for the past decade, but they did add MCP to it last year: https://www.claris.com/blog/2025/why-mcp-matters-for-claris-...
If/when Anthropic changes to pure API pricing, that's when I'd need to switch to more local larger models.
I currently run a Mac with 128GB and qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a 200k LOC codebase for security auditing and hardening for many hours at a time. This isn't as strong/fast as Sonnet, yet the cost is only $6K one-time.
in 10 years your 10 year old Mac will almost certainly be a relic for running cutting edge local models
What am I missing
yes, but you pay the Ultra plan, that's the whole point, isn't it? Let's be honest also - qwen3.8:27b-mlx is nowhere near Sonnet, both in size and cap, and if you do $10k a week of usage, you should know that.
This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.
If all of this were easy, I would happily invest in new dedicated hardware to run it. This could be a full-blown Mac or it could be a souped up HomePod-type device, loaded with RAM. Better yet, make it a revitalized Matter-enabled Airport!
They are going to sell sure 10,000$ and you won’t need anything else for the rest of your life (if you are over 90).
Btw, llama.cpp with llama-swap ain’t that hard to configure if you have access to a potential LLM and have 5h of your life available to that task.
I don’t find it reasonable at all. Subsidized Subscriptions or api for cheaper models, is the best value.
But the future of AI is so uncertain, regarding model sizes. I don’t think making huge hardware investments is reasonable for individuals.
There is no ZDR at that price.
I would not let a non-local AI have access to my financial or health data.
If you buy your own hardware, then after those four years you still have that hardware, and it likely still holds significant value. Plus you know the local model you're running today won't change or suddenly degrade in four months, let alone four years. Whereas after four years of paying subscriptions and API costs, you have literally nothing.
Claris?? Ok, this sounds like it's just someone's personal project. Something like this that was actually going to ship would be in a different org.
https://www.pacifict.com/Story/
If true, it is an astounding story. That said, I'm happy to see that Apple ships a graphing calculator with macOS 27 (albeit, a different one). I may not use it much these days, but I have fond memories of it when I was in university.
They desperately need to fix that.
It SHOULD be totally possible. But it fights you.
It’s too bad I cannot trust it to put it in my rack.
Lots of things pop up and can’t be dismissed any other way than though mouse/keyboard on the display.
OS updated and wants to tell you it worked? No command line way to get rid of that. Program wants to do something and one of those system security prompts pops up (“BBEdit wants to access your Documents folder”)? Same problem.
You can set it to auto login and auto update and grant permissions and everything.
And then, one day, it will stop responding and you’ll have to hook it up and log in to see some pointless dialog that just wanted someone to click “OK”. And until you do no work gets done.
https://en.wikipedia.org/wiki/FileMaker
... Looks like I picked the wrong week to buy parts for a linux-based inference box.