After yesterday's keynote and the changes to Search, it became clear in the near future, Google will cease to direct any traffic to websites and the search results will just become a footnote in Gemini's response.
There is a Netflix documentary about Imfura, Gicurasi and the Pablo descendents narrated by David Attenborough. It just came out last month. Fantastic doc.
I'm literally working on an iOS app right now that needs to infer some input fields from free text typed by the user. Now to take into consideration typos, unstructured text (pricing, dates .. etc), I was pondering a cloud LLM or a basic local parser or even a local on-device LLM (ANE for 15+ devices and a different on-device LLM for the older models)
For the different on-device LLM, I literally went to HuggingFace and filtered by the smallest available models that can do the job, and Granite-4.0-h-1b works just fine, it corrects typos, infers dates, currencies all fields I need.
And it got me thinking how my first reflex was to rely on a cloud LLM which is waaay overkill for my need. Granted, an on-device LLM will need to be loaded on the devices on install or downloaded after the fact (which adds latency when the user needs it for the first time) but still, it's a better tradeoff than a cloud LLM.
I decided on a basic parser, and so far it seems to work fine. granted, it struggles with some words, but I just need to finetune it to have as much coverage as possible in terms of typos without triggering false positives.
A lot of developers have that reflex too and go along with it and then just pass the API costs to the customer. I could have gone that route too but turned out I don't even need an LLM for my usecase.
Yeah but I need broader coverage on older phones. No I'm not going for a 3rd party LLM. Foundation Models for iPhone 15 and newer, and a parser for the older ones. Currently training a Word Tagger in Create ML
reply