Apple Intelligence, Apple Intelligence, Apple Intelligzzzzzzzzzzzzzzzz

[nytta0]

I feel all new tech that requires people to actively wear something hasn’t been that successful. 3D glasses for TVs, VR headsets, AR stuff, even smartwatches to a degree.

Sure, all of those have a following of enthusiasts, but nothing has revolutionized the lives of most people, no matter how high the initial goals of the tech were. I think having to actively wear something will always be an obstacle for a lot of people. As a quick and easy example, just think of all the insurmountable accessibility challenges.

Having something on you in a bag or in a pocket (like a phone) totally works. But having to wear something? Not so much.

 

[Bonusround]

Apple Watch? I'm very glad to have a watch with fall detection attached to my mother's wrist.

 

[cateye]

(warning: Salty over-reaction incoming. It's been a week, y'all)

Wearables are largely an excess of marketing, selling the idea that you're interesting and important enough to need technology integrated into your daily activities to such an ever-present extent. Some efforts are benign—do I need an Apple Watch to exercise effectively? Not at all. But a "watch" is an established and understood format, and ultimately how a smart watch integrates into our day-to-day is relatively harmless. Its sphere of influence is entirely personal.

Others, however, are pure manipulation: AI-enabled pendants and badges necklaces and glasses and whatever else, constantly gathering information under the ruse that we need it to support our complex, important, and busy existence. Horseshit. The busiest and most important among us would struggle to think of 5 things in a single waking day that would require that sort of ever-present and immediate access to our location, surroundings, and thoughts. The wearers, like every house with a Ring doorbell, are just mules, gathering information that will ultimately be abused, and paying for the privilege.

If nothing else, the extent to which Humane's stupid pin, and how it got swallowed by the conceit of its founders, should be a warning about how inessential such a device is and to question why companies are so desperate to create them in the first place. I get Apple's motivation here, if indeed they are developing such a device, but I don't trust it in the slightest, even if it is Apple doing it.

 

[nytta0]

Apple Watch? I'm very glad to have a watch with fall detection attached to my mother's wrist.

Oh, for sure! Some wearable tech can definitely be useful for people in the right context. And, like @cateye says above, smartwatches are maybe the most accessible and accepted of all the 'wearables'.

But to expand on my original point, you really don't see as many people with smartwatches as you see with smartphones. And I'm sure the fact that you actually need to wear the smartwatch is a huge part of the reason why.

 

[Bonusround]

You aren't wrong, @cateye. We lost the privacy game when everyone started carrying Nokias and using electronic payments instead of cash. I used to think of London as the panopticon metropolis what with its cameras everywhere. 'Americans would never stand for that.' Now private companies like Flock have stitched together every camera hanging off the corner of a convenience store, the purchase of which was forced by our insurance industry.

The arrival of glasses or pins is not the start of the slide; I'm not sure it even steepens the slope. A decade ago the Glassholes flooded San Francisco for a quick minute. Watching someone stroll into a crowded bar with camera-adorned spectacles was fascinating. Just observing how people noticed (or didn't) and chose to interact. What is polite eyeline discipline? Is it okay to stare at the lens hovering beside your right eyeball?

I wonder if anyone would even notice today, and the cameras are better concealed anyway. The easiest way to spot a pair of Meta Ray-Bans is by the age of the wearer.

Whether or not they can be trusted, I do expect Apple to go above and beyond with privacy measures for any new camera-equipped device. But ask: who actually benefits from these measures when the lens is pointed outward? We surely can't expect Google's or Meta's or OpenAI's devices to behave in kind, so that means we'll all be implicitly cast as extras in somebody else's film epic. "All the world's a [film] stage"... now who gets to keep the negatives?

In short: we're cooked. Privacy firmly died in the 21st century. Rest In Peace

 

[wrylachlan]

I think it’s not at all controversial that requiring a device to be worn creates a barrier. That’s obvious. The question is how big a barrier and what are the benefits. The barrier is lower for socially acceptable form factors (glasses and watches) than novel form factors. And it remains to be seen whether AI can unlock meaningful benefits that don’t exist now for wearables. I’m optimistic in the long run but pessimistic in the short term.

As for the argument that we’re not “important enough” to need always on tech, I think that’s bullshit. No one needed a personal navigator when we all know how to read maps, but using personal navigation software on our phones has become utterly ubiquitous. No one needs real time access to restaurant reviews but who doesn’t use them when traveling?

There are a lot of things that we perceive as “only the rich and important need that” simply because they’re the only people who have it and our pea brains can’t imagine a world in which everyone has it. I think a personal assistant is like that. Of fucking course I can schedule my kids pediatrician visits myself. But if I could say to my assistant “hey can you book appointments for the boys.” I would totally do it. When AI agents fully mature I don’t see anyone turning their noses up at the functionality out of a perception that’s its “only really needed by the rich and important people.”

 

[wco81]

For this pin thing to sell, Siri would have to be useful and people will want to use Siri to do a lot of the operations that they'd just do manually.

So maybe they have some confidence in the new and improve Siri and that it will cause a significant number of users to change the way they use their devices.

 

[wco81]

Apple Watch? I'm very glad to have a watch with fall detection attached to my mother's wrist.

My Watch detected a fall when I went to retrieve the garbage bins and one of them had rainwater so I flipped it to drain it and then whipped it around to wheel it back inside.

I had to confirm that I didn't fall.

It's that it detected that. I was on my feet, never off balance. People do more sudden action, more range of motion with their limbs, doing various kinds of sports activities.

 

[Scud]

In short: we're cooked. Privacy firmly died in the 21st century. Rest In Peace

Bingo. Every iDevice listens to everything you say. It's the cost of convenience

 

[Hap]

Makes me wish the concepts in Anon were real (the removal part, not the constant surveillance)

 

[nytta0]

OK guys, listen. This amazing video and the few glasses of wine I had tonight make me realize the future of wearable tech (AI or otherwise) is completely analogous to what happened to the Xbox Kinect sensor from a few years ago.


View: https://www.youtube.com/watch?v=AGSRMeHYKOE

 

[Louis XVI]

I’m not sure that any of this is going to prove out. But if it does, I think the idea will be more real time assistant than perpetual memory. As a thought experiment, imagine you were rich enough to have a personal assistant at your side at all times that you could ask things of.
…
“Who is that person over there in the brown jacket? He looks really familiar.”

This one’s the problem.

“Hey Siri, who’s that cute girl in the brown jacket? What’s her name? What’s her phone number? What’s her address? Take a picture of her so I can use Grok to make porn of her.”

 

[Honeybog]

the idea will be more real time assistant than perpetual memory.

“Who is that person over there in the brown jacket? He looks really familiar.”

Not sure how you do the second one without the first. Guy in the brown jacket has to have his image saved somewhere.

“Ooh, that sofa is nice, make a note to show my wife and see if she likes it.”

This is a problem with this entire segment of technology. 1) How often do you shop for a sofa, and 2) do so alone, despite needing your wife’s opinion? 3) Do you do this frequently enough that taking out your phone and texting a picture becomes enough of a recurring inconvenience that it justifies having another device to charge while also walking around looking like an absolute tree stump with an AI pin?

“What time did the johnsons say they were going to come over?”

For those situations when you’re planning a get together, but don’t have access to a Siri-enabled device such as an iPhone, iPad, Apple Watch, HomePod, HomePod Mini, MacBook, or Mac.

I guess instead of having a calendar event, this could be a query based on a conversation you had with the Johnsons last week. There’s probably some value in having a device capture and store conversations like that in a massive database, along with all the stuff you said to your wife about your boss, all of your political views, and the jokes you made about the Johnsons.

Some non-zero amount of the value of a personal assistant is their being able to see and hear what you see and hear and having it be their job to remember the important stuff.

I’m not worried about this product category truly taking off, but if it did, I think it’d be fascinating to watch the average onset of dementia slowly tick its way towards fifty.

 

[wrylachlan]

Not sure how you do the second one without the first. Guy in the brown jacket has to have his image saved somewhere.

The distinction is between capturing and storing everything and doing the analysis on request vs. doing triage in real time and only storing ‘high likelihood of being needed in the future’ things. My assumption is that AI assistants in all their various implementations will be closer to the latter than the former.

 

[wrylachlan]

I’m not worried about this product category truly taking off, but if it did, I think it’d be fascinating to watch the average onset of dementia slowly tick its way towards fifty.

That’s what they said about the printing press and yet somehow the world still created Einstein. The history of outsourcing our memory is very long and at each innovation people worried about what would be lost. It always turns out to be a nothingburger.

And I’ll go one step further, there are plenty of times when a memory is fading and it just doesn’t get reinforced because you can’t pull up the correct recall that would reinforce it. Enter smart glasses that could, for example, label that person whose name is slipping from your memory. Voila - memory reinforced.

Or think about an AI that could watch your eye movements while reading in another language, or could detect that subtle lean in you do when someone speaking in another language says something you don’t quite catch. Those are clues about how your memory of those words is on the verge of or recently failed. That AI could reinforce them for you.

So while I do see a lot of offloading of mental effort in our future with AI, I also see a lot of potential for AI to reinforce and improve our mental processes.

 

[Honeybog]

The distinction is between capturing and storing everything and doing the analysis on request vs. doing triage in real time and only storing ‘high likelihood of being needed in the future’ things.

If it’s the latter case, I don’t see how you ensure that it can identify the man in the brown jacket. It also seems to ignore that current AI systems are inherently maximalist, and that when given the choice between collecting more or less data, none of these companies choose less.

That’s what they said about the printing press and yet somehow the world still created Einstein. The history of outsourcing our memory is very long and at each innovation people worried about what would be lost. It always turns out to be a nothingburger.

That was Socrates and in a very different context. Complaints about the printing press were based around the easier dissemination of ideas (especially ones that challenged orthodoxy) and the threats to scribe’s livelihood. The comparison to the printing press comes up a lot through AI boosters, but the irony is that LLMs are significantly more likely to result in something closer to a pre-Gutenberg system, with circular training data and obscured weights and instructions leading to more orthodox outputs.

And I’ll go one step further, there are plenty of times when a memory is fading and it just doesn’t get reinforced because you can’t pull up the correct recall that would reinforce it. Enter smart glasses that could, for example, label that person whose name is slipping from your memory. Voila - memory reinforced.

That’s not how memory reinforcement works, though. It’s well studied that just being given the answer to something doesn’t assist recall. If anything, it ensures that you won’t reinforce the information at all, since synapse plasticity requires active connections between neurons to strengthen memory. It’s the reason why mnemonics and memory palaces are effective recall techniques.

 

[wrylachlan]

That’s not how memory reinforcement works, though. It’s well studied that just being given the answer to something doesn’t assist recall. If anything, it ensures that you won’t reinforce the information at all, since synapse plasticity requires active connections between neurons to strengthen memory. It’s the reason why mnemonics and memory palaces are effective recall techniques.

That only works for something you actually can recall. Once it’s past that state you NEED to give the answer in order to reinforce. The ideal way to build memory is to challenge that memory just before you’re going to lose it. But once you’re past that point you’re SOL which is a state many of us find ourselves in near constantly with one memory or another.

I could easily imagine a system that only applies the labels when it detects (through eye movements) that you’re unable to recall. My overarching point is that it’s not AI, but how we choose to deploy AI that makes the difference. AI has the potential to be a tool for enhancing our human abilities not just a tool for replacing them. My hope is that some will use them that way, though I’m well aware that others will not.

 

[Honeybog]

I don’t want to keep dragging this further afield, but I think we’re just going to have to agree to disagree on the semantics of what it means to reinforce a memory. You’re absolutely correct that a lapse of knowledge is a lack of knowledge and that you can’t recall something you don’t know. I disagree, however, that being presented with this knowledge can necessarily be labeled reinforcement. There are endless studies that show that passive engagement with information leads to diminished recall, and that this passivity in aggregate leads to earlier cognitive decline. We’re also beginning to see studies that show just how little synaptic activity goes on when interacting with AI.

All of this is a moot point, though, because wearable AI is fundamentally a solution in search of a problem. There just aren’t that many people going through the world needing spousal support while they buy couches on the daily or needing to identify men in brown jackets. Then again, like all good sticky tech, maybe it can just create the problems to solve, like helping a forty year old distinguish her teenage daughter from her memory of her younger sister after those neural pathways all atrophied.

 

[gregatron5]

There just aren’t that many people going through the world needing spousal support while they buy couches on the daily or needing to identify men in brown jackets.

There are a lot of agencies and forces that have that exact second problem, and that's where the money is, so that's where "AI" is likely to go.

 

[Dano40]

Dano, you keep popping up here, parroting this point then ghosting the conversation, that I wonder if you may be trolling. It's been explained to you multiple times that the problem with memory is not design, as was the case with Intel processors and the Mac, forcing Apple's hand to pursue Apple Silicon, it's manufacturing capacity. Unless Apple intends to spend tens if not hundreds of billions of dollars opening its own fabs to produce rote DRAM wafers, they will never bring this "in house." It's nonsensical, financially and procedurally. This manufacturing capacity problem is even spreading to the production of Apple's far more complex and expensive SoCs at TSMC, despite them being custom designed in-house.

Fabs are complex to build and difficult to run profitably. There's a reason why Apple outsources, for all intents and purposes, all of its manufacturing. Jobs championed this, Cook made it happen, and going back is not a realistic or reasonable point of debate.

Much of what you say is true however in Apple history, particularly the last 25 years, Apple has had to do things that they did not want to do but they had to if they were going to move forward. They worked with or tried to work with Motorola, IBM, and Intel to get a processor made for the iPhone they also even tried to work with Google in regard to Google maps. All these instances ended in tears ultimately Apple had to roll up it sleeve and get busy. It was either that or be left by the wayside. Note: Samsung and Qualcomm also on that list of companies that Apple had to work around.

My point is this will be another occasion where Apple will need to roll up their sleeves not because they want to but market circumstances have changed where they have to. Here’s a link to a Korean newspaper/website, where somebody else is thinking about the long range, implications, the Korean memory companies are probably in the process of cutting their own throats in the memory business long-term because once the Chinese get in, they will iterate and in time dominate the market, and that is but another reason why I believe Apple long-term will have to roll up their sleeves again bring it in house.

The Chinese playing it smart in the memory industry, short term, greed (by Koreans and Americans), appears to be getting in the way of maintaining market share.

Long-term if Google can burn/lite up $185 billion dollars just in the next 11 months of this year in pursuit of moat-less AI I think Apple internally with the Apple Silicon crew can handle it and spend a hell of a lot less money over the long-term.

https://www.koreaherald.com/article/10679206 Chinese making in roads in the low end bread and butter memory chips. (glad to see someone sounding the alarm in Korea).

 

[wco81]

Apple that one year did buy up almost all the production of NAND for the iPad Nano and no other manufacturer could produce enough competitive portable music players.

Buying up all the DRAM and probably NVMe storage would be hard to do though. But they can probably corral a lot of it, though Mac is obviously not a big business and they can't absorb all that volume.

 

[Dano40]

Apple brought SoC, Modem and WiFi chip DESIGN in house. The continue to outsource manufacturing. But design isn’t the problem in the RAM space - it’s manufacturing capacity. Apples playbook for dealing with manufacturing capacity constraints is fundamentally different.

That is true, but circumstances in the marketplace have changed (that was in a bygone era before Sam Altman, and OpenAI) when Apple had problems with Samsung Fab in the past they dropped them and went to TSMC. Yes it took time but Apple still had to do it. I believe this will be another roll up your sleeve time for Apple.

https://www.koreaherald.com/article/10679206 The Chinese playing it smart in the memory industry, in the short term, because greed (by Koreans and Americans), appears to be getting in the way of maintaining market share down the road. Once in the Chinese companies won’t be going out in either the low end or the high end. Why give a tough competitor, a easy road in?

 

[wrylachlan]

That is true, but circumstances in the marketplace have changed (that was in a bygone era before Sam Altman, and OpenAI) when Apple had problems with Samsung Fab in the past they dropped them and went to TSMC. Yes it took time but Apple still had to do it. I believe this will be another roll up your sleeve time for Apple.

https://www.koreaherald.com/article/10679206 The Chinese playing it smart in the memory industry, in the short term, because greed (by Koreans and Americans), appears to be getting in the way of maintaining market share down the road. Once in the Chinese companies won’t be going out in either the low end or the high end. Why give a tough competitor, an easy road in?

It’s not “roll up your sleeve” time, it’s “get out your checkbook” time. Breaking into the RAM manufacturing business is a 3 year project at minimum. And at the end of that, you’ve successfully in-houses a totally undifferentiated commodity that is easily available on the open market.

 

[Scud]

I agree; Apple will have to compete for the remaining 60% world memory supply with Google, Anthropic, Amazon, AMD, and NVidia.

 

[gregatron5]

I agree; Apple will have to compete for the remaining 60% world memory supply with Google, Anthropic, Amazon, AMD, and NVidia.

Pretty sure Altman (name seems apropos as he seems sort of not-quite-man) bought up 60%, so everyone has to fight for the remaining 40%. I am dumbfounded that not a single government in the world has even made an attempt to stop this. It affects literally everything in our modern world except perhaps extremely isolated manual labor subsistence farming or hunter-gatherers.

 

[educated_foo]

Pretty sure Altman (name seems apropos as he seems sort of not-quite-man) bought up 60%, so everyone has to fight for the remaining 40%. I am dumbfounded that not a single government in the world has even made an attempt to stop this. It affects literally everything in our modern world except perhaps extremely isolated manual labor subsistence farming or hunter-gatherers.

I'm seeing 40% in a quick search, but potayto, potahto. Where do they find the $71 billion? The gap between what Sam Altman says ("solve physics," "create God and ask him for money," etc.) and what he does (ad-supported chatbots and porn generators) beggars belief. I put the RAM thing squarely in the "says" box.

 

[Chris FOM]

That was Socrates and in a very different context. Complaints about the printing press were based around the easier dissemination of ideas (especially ones that challenged orthodoxy) and the threats to scribe’s livelihood. The comparison to the printing press comes up a lot through AI boosters, but the irony is that LLMs are significantly more likely to result in something closer to a pre-Gutenberg system, with circular training data and obscured weights and instructions leading to more orthodox outputs.



That’s not how memory reinforcement works, though. It’s well studied that just being given the answer to something doesn’t assist recall. If anything, it ensures that you won’t reinforce the information at all, since synapse plasticity requires active connections between neurons to strengthen memory. It’s the reason why mnemonics and memory palaces are effective recall techniques.

Not only that, it’s also well established that off-loading mental activities can reduce our ability to perform them unassisted in the future. Overuse of GPS, for example has been shown to have a meaningful negative impact on people’s spatial memory and navigation skills, including measurable differences in brain function.

 

[wco81]

Not only that, it’s also well established that off-loading mental activities can reduce our ability to perform them unassisted in the future. Overuse of GPS, for example has been shown to have a meaningful negative impact on people’s spatial memory and navigation skills, including measurable differences in brain function.

That toothpaste is long out of the tube.

Even those of us who learned to use paper maps for years aren't going back, not just for driving navigation but also to navigate an unfamiliar city on foot.

I recall pulling over several times to unfold the map and re-locate where I am relative to where I started and my destination. If you have a photographic memory, you're gifted but you're an outlier.

 

[Hap]

Not only that, it’s also well established that off-loading mental activities can reduce our ability to perform them unassisted in the future. Overuse of GPS, for example has been shown to have a meaningful negative impact on people’s spatial memory and navigation skills, including measurable differences in brain function.

True, but the trick is replacing something that can be automated with something else that can’t yet be yet that uses the same type of mental thinking.

IMO - we only have so much mental capacity and if we don’t use some support to free up some of that capacity - we stagnate. The key is to actually USE that freed up capacity for something else. Unfortunately the average person does not have the motivation to do so.

Deleted - a bunch of long winded, rambling, examples

 

[Chris FOM]

That toothpaste is long out of the tube.

Even those of us who learned to use paper maps for years aren't going back, not just for driving navigation but also to navigate an unfamiliar city on foot.

I recall pulling over several times to unfold the map and re-locate where I am relative to where I started and my destination. If you have a photographic memory, you're gifted but you're an outlier.

The toothpaste may be out of the tube but we don’t have to keep squeezing, much less find new tubes to empty as well. I’m not suggesting that we go back to a world before GPS navigation, but if it’s had measurable impacts on our reasoning abilities that’s worth considering before we find more and more things to hand off to AI. Assuming that letting AI handle it is without consequences and waving off those concerns with specious comparisons to how people thought about the printing press is particularly naive. And yes, I think having computers take over our social memory is both a line we haven’t crossed yet and one we shouldn’t. It would be a huge mistake. The state of humanity in WALL-E was a cautionary tail, not an ideal to strive for.

 

[wco81]

Sure I wouldn't have AI compose or summarize text for me.

Maybe someday I will play with image generation tools because I stopped drawing -- more like doodling -- a long time ago and never learned any of the various graphics tools.

Or instead of doing several operations when editing photos, if AI can do what you tell it, it might be worth exploring, more from automation point of view.

Though certainly you are letting some thinking it takes to do such operations lapse.

 

[wrylachlan]

The toothpaste may be out of the tube but we don’t have to keep squeezing, much less find new tubes to empty as well. I’m not suggesting that we go back to a world before GPS navigation, but if it’s had measurable impacts on our reasoning abilities that’s worth considering before we find more and more things to hand off to AI. Assuming that letting AI handle it is without consequences and waving off those concerns with specious comparisons to how people thought about the printing press is particularly naive. And yes, I think having computers take over our social memory is both a line we haven’t crossed yet and one we shouldn’t. It would be a huge mistake. The state of humanity in WALL-E was a cautionary tail, not an ideal to strive for.

Let me give an example. I used to teach a 2 week long intensive malaria prevention training for Peace Corps Volunteers. We would fly them from all over Africa to Senegal where we had our training center and go over everything, from malaria biology to the core preventive interventions to behavior change theory and how it pertains to malaria programs. It was an intensive training and multiple trainees were able to convince their schools to give them credit for it including Yale and Brown (Peace Corps had a program where you could serve while pursuing a Master’s degree.)

Because this thing was so intensive and there were 40 students each session I felt it was my responsibility to hit the ground running. So I used a tool called Cerego to make myself flashcards with each students name, school, Peace Corps country, and a picture and drilled them before the training began. From first meeting I had memorized who they were and what they were about. Many of the names, faces and where they served I can still remember 15 years later.

Making the flashcards took FOREVER! It was like 2 days of work googling, FaceBook, etc and then getting it all into the right formats and what not. Today an AI agent could do that for me in an hour.

And then once we were in the training there were lots of things that were said during sessions that I wish I remembered. I would gladly have reviewed a transcript if that were possible then. Now it is!

The point I’m making here is that you can chose to use technology to supplant your own abilities or you can choose to use technology to unlock your own abilities. I choose the latter.

 

[wco81]

WSJ article shows contrition of the TSMC plant in Arizona, which Apple is subsidizing with commitment to have a lot of production of their chips there.

But it also shows a picture of workers assembling "servers" in a Houston assembly line. They're described as AI servers and you can see they have a rack (1U?) form factor.

https://apple.news/AmOmQKHZFSA291qvy0Ezzkg

But they are going to expand the Houston facility to assemble some Mac minis as well.

 

[esome]

The point I’m making here is that you can chose to use technology to supplant your own abilities or you can choose to use technology to unlock your own abilities. I choose the latter.

I applaud your choice and aspire to have the same approach but I fear those choices will become harder and harder as AI becomes imbued in more facets of daily life. And I wonder if it isn’t the case that many youth who grow up with it will lack the non-AI perspective/context that makes this sort of choice meaningful in the first place.

As an aside, I wish my Peace Corps trainers had put in as much effort as you did! And are you aware of Tech Corps? Most of the RPCVs I know consider this AI-focused initiative grotesque and antithetical to the very nature of humanitarian work in general.

 

[wco81]

So here's a feature I'm curious about.

I have a lot of scans of legal and financial documents. I prefer to store scans of docs, especially those which I've signed, as scans than the physical copies.

I haven't tried OCR in years. But what could be useful is if AI can search text strings in scanned files.

I search for strings in my photos stored on my iPhone and it finds them, so obviously it's doing some image recognition processing. I don't think Apple labels it as one of their Apple Intelligence features though. I don't know with which iOS version it's been able to search text strings in your photos.

Many of these scanned files are sensitive, so I don't want to upload them to ChatGPT or anything online.

Does the latest version of MacOS search images for text? Actually, Spotlight search isn't that great, at least on Ventura, which I'm stuck with on my 2017 iMac for now.

That would be one useful outcome of this AI hype, if MacOS gets a much more useful Spotlight search which will also search through text and be smart, like instead of just searching exact text strings, it finds similar form, like if I searched for "black" it also finds instances of "blacken."

Main thing is perform all this on-device.

A lot of the docs are password-protected. So if I opened one of them, then just search within the document.

Or maybe I need to revisit OCR.

 

[Scud]

LocalLLM with an embedding model is your answer.

 

[Honeybog]

So here's a feature I'm curious about.

I have a lot of scans of legal and financial documents. I prefer to store scans of docs, especially those which I've signed, as scans than the physical copies.

I haven't tried OCR in years. But what could be useful is if AI can search text strings in scanned files.

I search for strings in my photos stored on my iPhone and it finds them, so obviously it's doing some image recognition processing. I don't think Apple labels it as one of their Apple Intelligence features though. I don't know with which iOS version it's been able to search text strings in your photos.

Many of these scanned files are sensitive, so I don't want to upload them to ChatGPT or anything online.

Does the latest version of MacOS search images for text? Actually, Spotlight search isn't that great, at least on Ventura, which I'm stuck with on my 2017 iMac for now.

That would be one useful outcome of this AI hype, if MacOS gets a much more useful Spotlight search which will also search through text and be smart, like instead of just searching exact text strings, it finds similar form, like if I searched for "black" it also finds instances of "blacken."

Main thing is perform all this on-device.

A lot of the docs are password-protected. So if I opened one of them, then just search within the document.

Or maybe I need to revisit OCR.

Live Text is the feature name, and it’s been around since 2020 or so on all platforms. You can search Photos for text on MacOS.

As far as working on MacOS generally, the answer is… kinda? In my experience Spotlight will surface non-OCR’d PDFs with search terms that it couldn’t have found without scanning and indexing, but like everything Spotlight, it’s extremely hit and miss.

Search-ability aside, in terms of OCR quality, Live Text eats Acrobat’s lunch. The accuracy rate is a million times better, and it’s kind of tragic that Apple hasn’t integrated it into Preview or Shortcuts so they could really compete with Adobe.

 

[wco81]

Live Text is the feature name, and it’s been around since 2020 or so on all platforms. You can search Photos for text on MacOS.

As far as working on MacOS generally, the answer is… kinda? In my experience Spotlight will surface non-OCR’d PDFs with search terms that it couldn’t have found without scanning and indexing, but like everything Spotlight, it’s extremely hit and miss.

Search-ability aside, in terms of OCR quality, Live Text eats Acrobat’s lunch. The accuracy rate is a million times better, and it’s kind of tragic that Apple hasn’t integrated it into Preview or Shortcuts so they could really compete with Adobe.

Yeah I tried searching within some scanned docs saved as PDFs. In some cases, it found all instances of the word searched but in others, it only highlighted one instance when there were clearly a couple of others.

It wouldn't be a flashy feature, if they boosted Spotlight indexing with AI to make every document type searchable.

Like do image recognition of photos and identify them, even if there isn't metadata to identify people or places in the photos.

But it would take a lot of CPU cycles, one would think. So maybe a process which the user would have to initiate to have it go through their photo library and tag places and names in the photos.

I actually have been using various geotagging work flows for years so I already have GPS data in almost all of my photos. But imagine being able to automatically tag photos using AI. Supposedly image-processing and recognition is one of the promising areas of AI, for detecting important things in satellite imagery or anomalies in radiographic imagery better than humans.

But again, it's not that flashy a demonstration of AI. Photography outside of phones is slowly but surely withering away.

 

[Horatio]

Like do image recognition of photos and identify them, even if there isn't metadata to identify people or places in the photos.

The iOS photos app already does this for people.

 

[dspariI]

Search-ability aside, in terms of OCR quality, Live Text eats Acrobat’s lunch. The accuracy rate is a million times better, and it’s kind of tragic that Apple hasn’t integrated it into Preview or Shortcuts so they could really compete with Adobe.

Unless I'm misunderstanding what you mean, Live Text is already integrated into Preview and Shortcuts and has been for quite a long time.