AI Sky Replacement in Luminar

Lately, I’ve been using the AI Sky Replacement tool inside Skylum’s Luminar 4.

The idea behind it is to make a scene more engaging by enhancing the sky. It cleverly replaces an uninteresting sky for another of your choosing. Anyone whose done this manually in Photoshop knows it’s a tedious chore to get right.

An image of a harbor entrance light before sky replacement.
The same image after using Luminar’s AI Sky Replacement (click for full size)

Luminar provides several dozen sky images with the tool; all you have to do is try one out. If you don’t like it, try another and keep experimenting until you find one you like.  

In some scenarios, if it doesn’t recognize a sky in your image, the tool becomes disabled. For instance, if you take a picture of your foot, AI Sky Replacement is disabled. But if there is the sky in your image, it works more often than not. I’ve tried cases where I thought it wouldn’t work, and it does.

Here’s an example where Luminar correctly identified the sky through an arch. This surprised me as I thought the software would get confused, but the AI proved me wrong. The photo is of an Instagramer at the Abbey of Montserrat in Catalonia.

Also, there are sliders you can use to tweak the results, but it does the job with or without the sliders. As someone who spends a lot of time looking at details in images, it’s pretty good. 

I won’t go into the arguments of what’s real and what’s fake. I create a lot of images, and I always manipulate them one way or another. I consider photographs straight out of the camera raw material. It is with this resource that I apply post-processing techniques to get the look and tell the story I have in my mind. In that vein, I have no compunction about using software (AI or not) to edit photos. 

Over the years, I’ve taken many thousands of digital images that now sit in a Lightroom library. Less than one percent ever qualify for public consumption. Compositionally, most are just plain bad, but in a few cases, it’s because the sky is too flat.

I’ve started looking at some of these “reject” images with new eyes and asking myself, what if? What if it had a different sky and, perhaps, different lighting? Before I go on, let’s pause here because it’s the second part of this question that I find the most interesting. 

Not only does it (as the name suggests) replace the sky, but it intelligently relights the scene to match the light from the new sky.

Here’s a surreal image of a mountain lake in Oregon after adding a long exposure of the Florida sky? The lighting of the scene below the sky is a result of the AI software.

Think about that for a moment. If your scene was taken in midday, but the sky you’ve chosen as a replacement is from sunset, merely replacing one sky for another might create an unbelievable, if not odd, lighting contrast. As humans, we recognize subtle changes in light, even if we are not always aware of it. But combine a sunset sky with a noontime landscape, and we get a feeling that something is not quite right. 

This is where the AI shines through. Skylums’s software agent works at relighting the non-sky elements with subtle tones of the sky that you selected. If Luminar simply replaced the sky, that would be a cool thing indeed, but Skylum is building upon years of AI experience. They’ve created a machine that combines the expertise of masking (sky replacement) with the techniques of expert toning. That is, given a scene, figure out how to achieve a believable lighting scheme. Of course, the idea of “believability” is subjective, and it depends on your individual preferences. All things considered, it does a pretty good job in a fraction of the time that you could do manually. There are infinite combinations of skies and landscapes, and each result is entirely unique. You could make the argument that it’s not perfect, but then, what or who is? The goal of AI is not perfection, rather accomplishment of things that formerly, only humans could do. In that regard, it’s as good or better and way faster than most humans. 

While we’re on the subject of AI, I just finished Stuart Russell’s book, Human Compatible, Artificial Intelligence and the problem of control.

In it, he explains just how far and fast AI is advancing. Stuart draws the curtain back on the future of our civilization once AI is fully realized. And from the sounds of it, we are much closer to that than most of us wish to believe. I recommend picking it up if you are the least bit interested in what the future holds. 

Skylum’s Luminar is a fascinating tool, and it’s a little uncanny how well of a job it does. But as I’ve come to learn from Russell’s book, this is just the tip of the iceberg, everything is moving in this direction, and we’d all better get used to it. Children born today will never know a world without AI. Any doubts you may have are quickly dispelled the next time you look at your smartphone or ask Google a question. 

So back to the photos, I’ve included some examples of both extreme and subtle applications. In the most extreme case here, the lighting of the ENTIRE scene is changed from daylight to night, as determined by my selection of one of Luminar’s out-of-the-box night skies. I added some additional elements like the light in the lighthouse and some extra toning. But the majority of the scene lighting is done by AI. 

This is how the Oregon coastal scene appears straight out of the camera.
Here, the same image is toned automatically to match the starry sky. (Click for full resolution).

By now, I’ve used it in a lot of different scenarios, some extreme like the previous example, but more often, I use it in subtle ways. Examples of this are where I’m just adding a little bit of texture to a picture where the sky is only a minor background component, not necessarily to be noticed. Here are some examples of that. 

An example of the subtle use of sky replacement on a minor background component of an image
Another example of sky replacement as a minor element to an image.

There’s a lot to like with the combination of replacement and toning, but there’s one more thing that seals it, at least for me. Luminar supplies about thirty skies, from sunrise to the Milkyway galaxy, and everything in-between. So they basically give you enough of a selection for most situations. They also provide addition sky packs from accomplished photographers that you can purchase as plug-ins. But to be frank, I would prefer to use my own skies. And the one feature clinches the deal for me is that Luminar allows you to use your own skies. 

As it happens, I take way too many pictures of the sky. Whenever I’m out shooting landscapes, and I see beautiful clouds, I can’t help but take a picture. By itself, an image of the sky is not that interesting; pretty maybe, but as far as being a complete composition, generally not. 

It’s always been in the back of my mind that one day I’d do something with these images, and that time has come. All those skies I’ve collected can now be used in Luminar AI Sky Replacement. 

I’ve started selecting a few and preparing them for use in the tool. I have morning, daylight, sunrises, and sunsets. I don’t have many night skies, so, for the time being, I’m using nights supplied by Luminar, and truth be told, those are the most fun to try out.

Playing with Luminar’s night skies are perhaps the most fun of all. Here is the silhouette of Mt Tibidabo from Barcelona with one of the out-of-the-box skies supplied by Luminar.

But I’m more of a sunrise and sunset person, so that’s most of what I have. Here’s a snapshot of the ones I’ve prepared so far. 

A snapshot of some of the skies I’ve prepared to be used inside Luminar. Most are from Florida where I live and others are from around the globe.

With my old skies now ready, I’m going back over my archives and revisiting images. Luminar’s AI Sky Replacement is breathing new life into my library. Photos that would never make the cut are being reconsidered with some pleasant results. Anyway, if you’re into post-processing, give it a try. As for me, it’s become a permanent part of my kit. And, like it or not, this is a taste of the future, even as it is upon us now. 

Sarasota Reflections

When there is no breeze, the water on Sarasota Bay turns to glass. I was fortunate enough to show up one night two years ago when the conditions were just right.

Daily Image

Sarasota Reflections
Apartment buildings overlook Sarasota Bay, as seen from Island Park.

Everyone likes the views from Island Park which is why folks come here to walk and take photos. I’m particularly fond of the perspective it provides for nighttime cityscapes across the water.

more from Sarasota

Sarasota is growing, and the skyline changes about every six months or so. That means I need to get my behind down to this spot at least once a year to keep up. But I’m pretty sure nothing has changed in this one particular section.

So Many Stories

I’ve heard it said that eventually, everyone passes through Times Square. There’s no way to describe it unless you’ve been there; it’s electric.

Daily Image

So Many Stories
The stories we tell with pictures are a form of virtual reality

Last week I was talking about telling stories with simple images at the beach, but this is an example of a subject that’s the polar opposite of serenity and sunsets. Regardless of the scene, success comes about by framing an image in a way that allows the viewer to enter it and muse about what is going on.

more urban exploration in the gallery

If you want to tell stories with your photos, it doesn’t matter what the scene is. It could be a beach, a farm, a city or anything in-between. I find that having a sense of depth draws us into the scene. We start at items close up and then wander around establishing distance and placement. It happens so fast we don’t notice, but crafting scenes are what makes photography so enjoyable. It’s a subtle version of virtual reality based on immersion. If we are, even for an instant, immersed in a photo, then we’ve experienced a form of virtual reality. Stories when told by a picture or a book, have always been a way to experience a different reality.

Hot Summer Nights

Last year I was in New York City on the hottest three days of the year. It was unbelievably hot and the only thing to do at night was to walk around Times Square in the pouring rain.

Daily Image

Hot Summer Nights
A hot and steamy night in the rain at Times Square

I should be used to the heat from Florida, but it was no easier. Nevertheless, the rain and lights created fantastic photo opportunities that are entirely different than those I get back home. A nice effect is how the rainwater creates a reflective sheen on the pavement.

more street photography from the gallery

Taking photos at night in a city is a matter of experimentation. With a camera, we have several choices to make. A wide aperture combined with a high ISO allows a type of street photography without a tripod. However, with a tripod, we can take longer exposures if we want to capture light trails. In this case, I just wanted to capture images of the scene without special effects. For me, the most exciting thing was watching people out having fun in the rain; which by the way, was what I was doing also.

Cool Shot on a Hot Day

This is not a product shot, I just thought it was a cool shot on a hot day in our winter and your summer. Well, it seems a few days ago was the shortest day of the year, which also means it’s winter. Since I moved to Florida I can’t seem to get it straight, winter is like summer and summer like winter. Hear me out.

Daily Image

Cool Shot on a Hot Day
Cool Shot on a Hot Day

In winter, we have mild sunny weather, in summer we have storms daily and the tourists are gone. If that sound a little backwards you’re not alone.

some of my favorite photos

This favorite little spot of ours known as Crabby’s Beachwalk Bar & Grill in Clearwater Beach. It’s a good place to contemplate these types of conundrums. When I get disoriented it’s better to just take a seat and chill. That’s exactly what my wife and I did when I got the idea to take this photo.

Nuns on Vacation

I was walking through the old section of Kotor and came upon a group of nuns on vacation. At least I think they were on vacation, I didn’t actually speak with them. But they seemed to be having fun nuntheless.

Daily Image

Nuns on Vacation
Nuns on Vacation – As seen in Kotor Montenegro

I’m not really in the habit of following people around, but I kept running into the same group. On no prior occasion have I seen nuns with cameras and iPhones, just like the rest of us. I don’t know why I found that odd, perhaps they are allowed to order these items on Amazon. But again, with nuns I am a novice and I could have it all wrong.

my favorite images

Anyway, I saw the group of them a couple of times and I took a few pictures of the spectacle. There is a certain ascetic to this scene that I find pleasing. I don’t see a lot of vacationing nuns in Florida. There is nothing much for them to see here except people that lay on the beach. And trust me, we have the mother of all beaches. But, they could raise money by selling French Fries. Anyone could order and they would just put the ingredients in the friar.

What I Saw in My Mind

Walking to the train one morning in Vancouver I noticed this cruise ship at dock. It arrived a few hours earlier and was preparing for another voyage to Alaska. A common site in the summer but nonetheless it looked awesome in the morning light so I took a few photos; this is what I saw in my mind. Let me explain…

Daily Image

What I Saw in My Mind
What I Saw in My Mind – From Vancouver BC

I spent a lot of time on this photo to make it look like what you see here. The original photo didn’t look so pretty. It had a lot of distractions, including other boats and a parking lot in the lower right. So I manipulated it to make it look like what I thought I saw. What we perceive is usually different than what we see. In other words, we perceive what we want to see and have a tendency to screen the rest; so this image is now closer to what I thought I saw.

At the time I only noticed the boat, it stood out because it wasn’t there the day before. I also noticed the soft light of the morning sun reflecting on the side. I thought to myself that it was an awesome scene, but when I looked at the image later there were other things I didn’t remember seeing, like the parking lot for instance. Rather than get disappointed and throw it away, it became a challenge for me to see if I could replicate my initial impression.

more images from Vancouver

To my way of thinking the practice of photography is an expression of art. If I just want to record an event I snap a photo with my phone. But “photography” can be more than that. That doesn’t mean it should always be manipulated, but it should tell a story. In this case I did indeed manipulate it so that I could convey the story what I perceived in my mind on that summer morning in Vancouver.

Composite of Downtown Vancouver

This is a multi-exposure composite of downtown Vancouver. I took this while staying on a high floor at the Marriott Delta Vancouver. My Marriott profile indicates a preference for a high floor. About half of the time, depending on availability, I end up with an amazing view like this.

Daily Image

Composite of Downtown Vancouver
Composite of Downtown Vancouver from the Marriott Delta Vancouver

To get this I setup the camera on a tripod next to the window and left it there for about twelve hours. I took exposures in the afternoon, evening and then upon waking in the morning. I also used a lens skirt so that there wouldn’t be any reflections on the window coming from my room. Later I blended them all together to form this composite image.

The technique is my attempt to counteract my indecision. Often, the images I post are just one of many that I took of the same thing. I suppose I could post them all but that would get boring, so I pick just one. That’s where indecision comes in. I’m left with ten or twenty images of the exact same thing, but in different light.

some other composites

A composite allows me to pick and choose my favorite aspects of each photo and combine then into one image. It’s a little like seafood gumbo; it can be tasty if all the ingredients are nicely blended. And for desert, I get to have my cake and eat it too.

People at One World Observatory

So often when travelling we find ourselves in places where there are a lot of tourists. There’s nothing wrong with that and quite often I am one of them. However from a photography perspective it presents a challenge. For me the challenge is either how to incorporate crowds into an image or avoid them altogether. In this case I sat on a bench watching the people at One World Observatory and noticed the reflections creating this scene. I took several photos and this is my favorite.

Daily Image

People at One World Observatory
People at One World Observatory

This is also a good example of post processing. Because right out of camera the people looked more like silhouettes, you couldn’t see too much detail. I was able to bring that out in post processing, and primarily because I used a Sony camera with a great sensor. That sensor captures much more shadow detail something like an iPhone. So this is closer to what the scene actually looked like because of course our eyes are able to capture a wider range of light. I used post processing to bring the detail back from the shadows.

Getting back to the challenge of crowds, the other approach would be to avoid them altogether. To do that you need to get to places early or stay late. I don’t know about you but for me that’s easier said than done. Sometimes it can be difficult to get up and out early. I still try and sometimes I make it and I’m usually rewarded with softer light and scenes without a lot of people.

other travel images from the gallery

But these are just common sense tips. What makes an interesting photo is entirely in your head. With photography we can take the most common and mundane of scenes and express something transformative. That’s true for any art form, so whether you decide to include crowds or avoid them is just a technique, the thing that’s really important is what you see.

Salt Lake City

A few months ago I was in Salt Lake City visiting some close friends. I am not Mormon but my ancestors were and so are my close friends. So while there I took the time to visit the temple grounds and took a tour of the convention center. This panorama I took while standing on the convention center roof.

Daily Image

Salt Lake City
Salt Lake City panorama

My ancestors were the original settlers of Salt Lake City. So we also visited the cemetery to see where they were buried. Through help from my friends and a little sleuthing we found the graves where my great-great grandparents were buried. They’re in the Salt Lake Cemetery, which is the resting place of many of the original pioneers.

It was for me an amazing experience because I came away learning about my heritage that heretofore I’d only heard from my grandmother before she died. It goes without saying the Mormons are big on ancestry and so they were more than happy to help me fill in the gaps. I am fortunate to have such a recorded history and now that I know a little more I’m eager to share it.

Daily Images

Back home the other day I was taking some photos along the waterfront. Two Mormon missionaries approached me and rather than ignore them I engaged them in conversation. I pulled out my iPhone and shared this photo and told them about my experience and ancestors. They seemed genuinely surprised to meet me and equally happy to see a picture of their main temple. It was a fun encounter. While I am not planning to convert to the Mormon faith, there are many things we share in common. And for that connection and fellowship I am truly grateful.