This is a cool way of approaching AI models. I'm a big fan of local LLMs, local specific models like this should be even more powerful.
> Every model is free up to 100k monthly active devices. No tokens, no logins.
I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you.
These local models are like old school software. They're producing the weights, and then giving them to people. If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
The whole "but we need to keep it updated for your security" doesn't really work as well for software designed to run fully offline like these local models are.
I'm not saying they shouldn't get paid, but I guess I feel a personal sadness that it's less obvious how to successfully monetise such a sincerely useful and beneficial approach towards AI models.
> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues:
- almost no software is static. Look at a package like python Requests and even though it does the simplest thing and has barely changed from a user perspective, it gets updated all the time. This is true for most software. This is doubly true for something like local AI models where both the software and the hardware are changing constantly. Subscriptions motivate sellers to keep their software up to date.
- If I’m an app developer, the idea that I can try something out for $X/month is very appealing versus making an upfront investment of (let’s say) $X*20. This is doubly true for something like local models where I will almost certainly want something new when the models improve.
- To add to the first point, I work at a startup. No one asks questions when I want $20/month licenses. But let’s say I want something that’s gonna be in the 5 figure range annually. If I go to my CFO and ask for $50k upfront and then we implement something and the project fails, I look like an idiot. If I ask for $2000/mo budget for something and then we try it for two months and it fails, no one cares. Subscriptions are just safer in this sense.
Very much like map CD-rom updates, I think a lot of people wouldn't mind paying per-update for these models, and would understand the implications. That is, if there weren't so many open models available.
How is it not obvious and fair that they are asking you to pay them when you see success (defined as >100K MAU)? How more aligned can you and them be besides this?
When I buy I chair I don't pay a share of my income to the furniture shop when I get rich. I would buy another chair at some point too, maybe fancier one (or the same). Because chairs are commodity. I do however pay taxes to the government based on my income because it keeps doing ongoing maintenance on everything.
Everyone wants to be paid forever for something they produced once is some kind of a mind virus. Make me a better chair and maybe I will buy it, but don't expect to become a trillion dollar company. It's deeply unfair to everyone who wants to be a trillion dollar company of course.
What is really funny to me -- the ones that do make it to collect the rent indefinitely also decrease their own taxes paid to the government AND also decrease the amount of contribution to society by making less and shittier stuff.
> If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
Because they own the IP and they get to decide the terms of how it’s licensed.
This is like a company taking open source software, saying they like the code the community has given them, and asking why they should continue having to respect the terms of the license after downloading the code. The availability of the software (or models) does not equal a free license to use as you please.
A license allowing 100K devices for free is very generous. The businesses selling more than 100K units of anything will be significant operations. It’s fair that they’re asked to contribute financially.
No, you’re arguing against the first part of my comment without reading the whole thing where I addressed the value proposition.
A 100K free device license is a generous gift to small companies. Past 100K, it’s more than fair to ask that companies contribute. That’s a significant operation at that point and the model has obviously provided value.
If the model doesn’t provide any value to a >100K device company then they should train their own or not use it.
I could see how people unfamiliar with per-device or per-unit licensing fees would be confused, but this is a common business model. Scaling payments with the customer’s business success is one of the more fair ways to align vendor and customer while also providing a nice way to give freely to smaller businesses like this.
It’s a good thing. It’s silly that a company giving away a license for 100K devices, which covers small and medium hardware operations, is being criticized for this of all things.
> why should you continue getting money off me and my customers?
Welcome to the concept of fair market value. Less snarkily you're conflating the concepts of price and cost; theyre not the same thing and theyre not the same for you or the seller.
I definitely think there's a lot to be done with small models dedicated to specific tasks. I've always thought the REAL value is in having large models be able to easily build small models for custom tasks (which I know is kind of a thing), but perhaps just providing the small models directly is the more accessible approach.
>accessible via one SDK for Swift, Kotlin, and JavaScript
Lol well let me know when there's a Python SDK and I'll give it a try then. Obviously this isn't a deal breaker if you have a real case, but as someone who is willing to spin something up and try it out if there's a quick "pip install" command, this is getting put back on the shelf for now.
I love this idea and hope to see more on-device models. How do they make money, though?
I tried out their demo for Clear, the audio quality improvement model.[0] I'm not sure if it's just I don't have refined enough an ear or their demo is broken, but the "raw" and "enhanced" versions sounded exactly the same to me.
The underlying model (DFNet3) is not particularly great but it is very small and fast - imo the best commercially usable denoising model is MossFormer2 (no affiliation - it's just excellent) with one drawback in that it can't remove reverb.
Nvidia's RE-USE model can do what MossFormer2 does _and_ can remove reverb, but it is non-commercial licensed.
at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
It's an ANE optimized version of Parakeet, with our own inference, which enabled us to push performance to about 300x realtime speed on an iPhone 16/17. Our next gen Voz model is trained from scratch and will be at least twice as fast. Android and other platforms will land soon.
I don't think you understood. It means these are specialized models.
Their toxic model could be ideal for video game lobbies without investing a ton of money if you're an indie dev
This also could be ideal if you want your child to play online to have auto-censorship
A lot of the models would be useful in a web context, to improve on the CMS we're making for clients. But they look like most of them are iOS only, few have a node package or something other, and all the benchmark are running it on modern iPhones so I doubt it would be that fast on a 20$ VPS.
This is a cool idea. The most useful one for me would be something that can process pdf files into a json schema. Title and tag generation from a post would also be useful. I'm interested in web app though.
We plan to make most our models available for Android and web too. Some are a bit harder to port to the different platforms and will take a bit longer to properly land on Android or web. Mostly sequencing (Voz, Clips, Title). Soon!
We design in Figma. Build a design system out that with components for all our public facing products, demos and marketing assets. The website is built with the components, and we have a set of skills and tools to keep both in sync. We also use skills to sync between HuggingFace, our SDK on GitHub and the website. We write and review most our core copy by hand and use that to help expand into different pages. We prob do a write-up about this on our blog later.
> Every model is free up to 100k monthly active devices. No tokens, no logins.
I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you.
These local models are like old school software. They're producing the weights, and then giving them to people. If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?
The whole "but we need to keep it updated for your security" doesn't really work as well for software designed to run fully offline like these local models are.
I'm not saying they shouldn't get paid, but I guess I feel a personal sadness that it's less obvious how to successfully monetise such a sincerely useful and beneficial approach towards AI models.
I definitely understand the appeal of the desire to “buy it once”, but I think there are a few issues:
- almost no software is static. Look at a package like python Requests and even though it does the simplest thing and has barely changed from a user perspective, it gets updated all the time. This is true for most software. This is doubly true for something like local AI models where both the software and the hardware are changing constantly. Subscriptions motivate sellers to keep their software up to date.
- If I’m an app developer, the idea that I can try something out for $X/month is very appealing versus making an upfront investment of (let’s say) $X*20. This is doubly true for something like local models where I will almost certainly want something new when the models improve.
- To add to the first point, I work at a startup. No one asks questions when I want $20/month licenses. But let’s say I want something that’s gonna be in the 5 figure range annually. If I go to my CFO and ask for $50k upfront and then we implement something and the project fails, I look like an idiot. If I ask for $2000/mo budget for something and then we try it for two months and it fails, no one cares. Subscriptions are just safer in this sense.
Everyone wants to be paid forever for something they produced once is some kind of a mind virus. Make me a better chair and maybe I will buy it, but don't expect to become a trillion dollar company. It's deeply unfair to everyone who wants to be a trillion dollar company of course.
What is really funny to me -- the ones that do make it to collect the rent indefinitely also decrease their own taxes paid to the government AND also decrease the amount of contribution to society by making less and shittier stuff.
Because they own the IP and they get to decide the terms of how it’s licensed.
This is like a company taking open source software, saying they like the code the community has given them, and asking why they should continue having to respect the terms of the license after downloading the code. The availability of the software (or models) does not equal a free license to use as you please.
A license allowing 100K devices for free is very generous. The businesses selling more than 100K units of anything will be significant operations. It’s fair that they’re asked to contribute financially.
how'd that work out for the rest of IP owners that had their stuff used as training data for use in model creation?
was the material they trained on produced in-house?
I know some people want it to be a legal violation, but it’s not.
Also these are audio models, if you hadn’t noticed. The training sets for this type of work is very different than the corpus of scraped GitHub repos.
Philosophical questions don't make sense if you use them to counter argue about something which is a legal issue.
You made up a position to argue against.
A 100K free device license is a generous gift to small companies. Past 100K, it’s more than fair to ask that companies contribute. That’s a significant operation at that point and the model has obviously provided value.
If the model doesn’t provide any value to a >100K device company then they should train their own or not use it.
I could see how people unfamiliar with per-device or per-unit licensing fees would be confused, but this is a common business model. Scaling payments with the customer’s business success is one of the more fair ways to align vendor and customer while also providing a nice way to give freely to smaller businesses like this.
It’s a good thing. It’s silly that a company giving away a license for 100K devices, which covers small and medium hardware operations, is being criticized for this of all things.
Welcome to the concept of fair market value. Less snarkily you're conflating the concepts of price and cost; theyre not the same thing and theyre not the same for you or the seller.
>accessible via one SDK for Swift, Kotlin, and JavaScript
Lol well let me know when there's a Python SDK and I'll give it a try then. Obviously this isn't a deal breaker if you have a real case, but as someone who is willing to spin something up and try it out if there's a quick "pip install" command, this is getting put back on the shelf for now.
I tried out their demo for Clear, the audio quality improvement model.[0] I'm not sure if it's just I don't have refined enough an ear or their demo is broken, but the "raw" and "enhanced" versions sounded exactly the same to me.
[0] https://desertant.com/models/clear/
Nvidia's RE-USE model can do what MossFormer2 does _and_ can remove reverb, but it is non-commercial licensed.
> Hate speech triage. On-device moderation that flags hateful, abusive and threatening text
What could go wrong here?
This also could be ideal if you want your child to play online to have auto-censorship
Voz is Parakeet 0.6B v3
Clear is DeepFilterNet 3
Ear is the language predictor from whisper-tiny
...