Apple Intelligence: Clean Up in Photos

Technology is consistently entertaining new crazes. Some examples include blockchain, subscription juicers, netbooks, 3D televisions, hyperloop, and "hoverboards", just to name a handful of examples. All of these were going to be "the next big thing", but none of these have panned out as the inventors intended.

There has been a term bandied about that people think may be the end-all for computers. Said term is "Artificial Intelligence", or "AI". The term "AI" can mean a variety of different things, depending on whom you ask. However, when most use the term AI, what they are expecting is a fully conscious and sentient entity that can think, act, and rationalize as a human would. This is called "Artificial General Intelligence". Today's technology is nowhere even close to being able to come to this reality. It is not yet known whether or not Artificial Intelligence will actually live up to its ultimate expectations.

Apple is not known for being jumping on bandwagons and being the first to create new categories of technology; they typically leave that to others. However, if there is a technology that they can put their own spin on, they might do so. At their World Wide Developer Conference 24, they introduced one of these types of technologies, called "Apple Intelligence".

Apple Intelligence is not a single item; in fact, it goes against the grain of other AI assistants and only works on your data. Apple Intelligence consists of a variety of tools to help you accomplish a specific task. When introduced, Apple indicated that the initial features of Apple Intelligence would be released over the course of the iOS/iPad 18 and macOS Sequoia releases.

The items that comprise Apple Intelligence include: Writing Tools, Image Generation, and Personalized Requests. Initially, Apple wanted to have the first items available with iOS 18; however, during the beta, Apple realized that the features would not be far enough along for an initial iOS/iPadOS 18.0 and macOS Sequoia (15.0) release, so they were pushed to iOS/iPadOS 18.1 and macOS Sequoia 15.1.

Not every device that can run iOS 18.1, iPadOS 18.1, or macOS Sequoia 15.1 is able to support Apple Intelligence. To be able to run Apple Intelligence you need to have one of the following devices:

• iPhone 16/Plus (A18)
• iPhone 16 Pro/Pro Max (A18 Pro)
• iPhone 15 Pro/Pro Max (A17 Pro)
• iPad mini (A17 Pro or later)
• iPad Air (M1 or later)
• iPad Pro (M1 or later)
• Apple Silicon Mac (M1 or later)

The reason that these devices are the minimum is a combination of needing 8GB of memory, as well as a neural engine.

This article is part of an on-going series that covers the features of Apple Intelligence, as they become available. This article focuses on the Apple Intelligence feature called “Clean Up".

Photo Editing History

There is an old adage that goes "a picture is worth 1000 words", however the original quote is by newspaper editor Arthur Brisbane, who said "Use a picture. It's worth a thousand words". The quote is from 1911, back when newspapers were the prime method of obtaining news and with the limited amount of space, you could easily put a picture in place of 1000 words. The sentiment of either quote is that it would take about a thousand words to adequately describe a scene, when a single photo would be able to convey the same thing.

Fast forward to today and everything has changed. Written text is still important, but it has been supplanted not only by photos, but also by video. We are nearly two centuries from when the first photograph, "View from the Window at Le Gras" was taken. We have come a long, long way from then. Today's technology can easily take multiple pictures per second when you are using burst mode on a camera.

When film cameras became popular, you would take a photo in the hopes that you would get a usable photo. You would not know right away, because you would need to send your film off to be developed and processed. Once the film was processed, there was typically not a lot that you could do with the photo. That is not to say that some people did not manipulate photos, because of course they did, but it was a skill and not something easily accomplished.

Nearly 35 years ago, there was a new piece of software released. That software is called Photoshop. It is quite likely that you have heard of Photoshop, but in case you have not, Photoshop is software created by Adobe that will allow you to not only create images, but also the ability to edit photos. It is this latter functionality that many use the software for. Photoshop is not an easy piece of software to use, at least not for the average user. There are millions who are quite proficient with the software (the author of this post is absolutely not one of them).

While it is no longer necessary to hope that you got a good photo, there may still be instances when you may want to make some modifications to a photo, but you have the skills to use an app like Photoshop. For these situations, you can use a feature within Photos called "Clean Up".

Clean Up

Clean Up is a new tool that can be used to remove various items from a photo. The Clean Up tool can be found within the editing functions of the Photos app. To access the Clean Up tools, perform the following steps:

1. Open the Photos app.
2. Locate the photo that you want to use Clean Up on.
3. Click on the "Edit" button.
4. Click on the "Clean Up" button to bring up the Clean Up tools.

Once you bring up a photo, you will have a sidebar that says "Clean Up". Here you will have a single option: the size of the brush. You can adjust the size of the brush by clicking and dragging along the slider. The further right you go, the bigger the brush.

When you bring up a photo for editing, you may notice some items flashing. These flashing objects indicate what Photos thinks you may want to remove. Sometimes, it is correct; other times, it may not be. Let us look at an example.

In the photo below, you will see that it contains a car, some garbage cans, a white car, and a folded chair. In the screenshot, you will see that the garbage cans and car are automatically selected.

If you double-click on any of the highlighted items, they will be removed and their background will be replaced. Here is an example of what that might look like.

-- INSERT SCREENSHOT OF GARBAGE CANS "CLEANED UP" --

Now, you may initially think "Oh, that's pretty good", and at first blush it might be. However, if you look at it closer, it does not work all that well. As an example, the grass has been expanded onto the street. At the same time, the street has been expanded onto the grass. This is not accurate at all.

The thing that I think is the most aggravating is the fact that you can easily see that there is a curb that is circling around behind the garbage cans, yet it is completely removed from the area that you can easily see. It is somewhat understandable that the area behind the garbage cans, that is not seen, is filled in improperly, but the area that is shown should not really be touched.

Let us look at another example.

In this second photo, you can see a squirrel just chilling on the railing of a deck. Let us say that you want to remove the backing of the chair in the lower portion of the photo. It is the area that is highlighted.

Now, if you remove the chair, you will get something like this:

This is an infinitely better photo. The stiles of the railing on the deck are correct, and it does look very close to what you might expect. The only item that I noticed was that the filled-in area along the far right of the photo is not correct. However, it does make sense given that it does not have any information to fill in that area, besides the dirt at the top of the railing.

Closing Thoughts on Clean Up

Clean Up is a good idea and a tool that can provide mixed results. In some cases, the results are good and acceptable. However, there are also those instances where it does not work all that well. Ultimately, it depends on the image and what you are trying to clean up as to whether the proper item(s) will be removed. Hopefully, Apple is able to improve the way that this functionality works and have it function as expected.

Be sure to check out all of the other articles in the series:

• Introduction to Apple Intelligence
• Apple Intelligence: Type with Siri
• Apple Intelligence: Summarization and Mail
• Apple Intelligence: Writing Tools

You can also check out all articles about Apple Intelligence