I think AI-"upscaled" videos are as jarring to look at as a newly bought TV before frame smoothing has been disabled. Who seriously thinks this looks better, even if the original is a slightly grainy recording from the 90's?
I was recently sent a link to this recording of a David Bowie & Nine Inch Nails concert, and I got a serious uneasy feeling as if I was on a psychedelic and couldn't quite trust my perception, especially at the 2:00 mark: https://www.youtube.com/watch?v=7Yyx31HPgfs&list=RD7Yyx31HPg...
It turned out that the video was "AI-upscaled" from an original which is really blurry and sometimes has a low frame rate. These are artistic choices, and I think the original, despite being low resolution, captures the intended atmosphere much better: https://www.youtube.com/watch?v=1X6KF1IkkIc&list=RD1X6KF1Ikk...
We have pretty good cameras and lenses now. We don't need AI to "improve" the quality.
prmoustache · 1h ago
The weird thing is that people are seemingly enjoying this.
Yesterday we went to a store to have a look at a few smartphone for my partner. She primarily wants a good camera above any other parameter. I was seeing her preferring those that were counterfeiting the reality the most: she was like, "look I can zoom and it is still sharp" while obviously there was a delay between zooming and the end result which was a reconstructed, liquid like distorded version similar to the upscaling filters people are using on 8/16bit game console emulators. I was cringing at seeing the person I love the most preferring looking at selfies of picture of us with smoothed faces and a terrible fake bokeh in the background instead of something closer to the reality.
gyomu · 20m ago
I’m a photographer, and am on a bunch of beginner photography groups.
These groups used to be a mix of people being confused at how their camera worked and wanting help, people wanting tips on how to take better pictures, and sometimes there was requests for editing pictures on their behalf (eg “I found this old black and white faded picture of my great grandparents, can anyone help restore it?”)
These days, 99.9% of the posts are requests that involve synthesizing an entirely new picture out of one or more other pictures. Examples: “can someone bring in my grandpa from this picture into this other family picture?”. Or “I love this photo of me with my kids, but I hate how I look. Can someone take the me from this other picture and put it in there? Also please remove the cups from our hands and the trees in the background, and this is my daughter’s ex boyfriend please also remove him”.
What’s even crazier is that the replies of those threads are filled with dozens of people who evidently just copy pasted the prompt + picture into ChatGPT. The results look terrible… but the OP is always pleased as punch!
People don’t care about “reality”. Pictures have lost their status of “visual record of a past event”* and become “visual interpretation of whatever this person happens to want”.
There’s no putting back the genie in the bottle.
*: yes, you can argue they were never 100% that, but still, that’s effectively what they were.
petralithic · 14m ago
But people have be editing photos like that before AI and even before Photoshop, I don't see the big deal. What I've seen recently is synthesizing whole new pictures with AI, by training a LoRA on their face and body and asking the AI to create themselves with a specific setting or background.
ulrikrasmussen · 53m ago
Yes, this is the exact same reason that frame smoothing exists. When you walk into a store, all the TVs are lined up showing some random nature show or sports event, and frame smoothing will make your TV look a little more smooth than the others, even though it completely ruins the content.
It's made for making sales, not for making things actually look good.
windward · 34m ago
At some point it became unacceptably rude to gatekeep, king-make, or be otherwise judgemental of taste. It was at around the same time that subcultures and counterculture melted into an homogenous mass.
I think we lost something in that. Embarrassment can be useful for moving us out of our comfort zones.
brap · 2h ago
I remember watching an episode of one of my favorite shows on my parents’ brand new TV, and thought to myself something about this episode is off, like the production is cheap, the acting feels worse, even the dialog is bad.
Over time I noticed everything looks cheaper on their TV.
It is especially bad for animated shows that have made an explicit artistic choice to let (parts of) the animation progress at a lower frame rate. My kids watched "spider-man: across the spider-verse" at a friends place where smoothing was not turned off, and it completely ruined the artistic feel and made the movie feel like a stuttering video game.
jay_kyburz · 1h ago
I found those spiderverse movies really hard to watch because of the low frame rate. I don't think it was artistic, it was cheap.
pja · 46m ago
> I found those spiderverse movies really hard to watch because of the low frame rate. I don't think it was artistic, it was cheap.
It was absolutely an artistic choice - Sony spent more per frame on those movies than any previous animated film & the directors knew exactly what they were doing when they chose to animate some parts on every second (or even third) frame.
ulrikrasmussen · 56m ago
I think that's a matter of taste, but it definitely doesn't make it easier to watch them when the frame rate randomly switches between low and high all the time :).
spacechild1 · 47m ago
I had the exact same experience watching Goodfellas on my parents' TV. It felt like a cheap soap opera and I was thoroughly confused about what's happening. Afterwards I did some research and learned about motion interpolation in modern TVs.
agumonkey · 22m ago
Thing is, to some population this is seen as better. While to me it feels as journalists camera, too real to pass as a story.
Angostura · 28m ago
I hope you waited until they were out of the room and turned it off in settings?
chneu · 1h ago
It also has to do with how basically everything is filmed for Netflix/streaming nowadays.
topranks · 48m ago
Not really sure I get why that would be a factor
alienbaby · 5m ago
I think its preferred to get this kind of smooth unreal effect for services like youtube, but not because it looks better; but rather because it compresses better for storage. Less fine detail overall helps video compression.
The extreme blur here was obviously a creative choice by the director/editor, the rest of the video has lower resolution but it's not nearly that bad (which is why Bowie still looks like himself in other parts of the upscaled video).
The process used to upscale the video has no subtlety, it's just "make everything look crisp, even if you have to create entirely made-up faces".
ygra · 1h ago
Seems like they ignored the non-square pixel aspect ratio as well for the upscaling, which may have changed face shapes as well.
janfoeh · 39m ago
Between 2:07 and 2:08, the guy on the right loses his glasses. Over the course of a couple frames, they just fade into nothingness.
hliyan · 2h ago
This phenomenon of pushing technology that end consumers don't want, seem to be driven by a simple sequence of incentives: pressure from shareholders to maintain/increase stock price -> pressure on business to increase market share, raise prices, or at least showcase promising future tech -> pressure on PMs to build new features -> combined with developers' desire to try out new technologies -> result: AI chatbots/summaries on things we didn't ask for, touchscreens on car dashboards, AI upscaling etc.
wickedsight · 56m ago
> pushing technology that end consumers don't want
Flashback to when every TV at CES had 3D functionality. Turns out nobody really wanted what. What an immense waste of resources that was.
lifestyleguru · 32m ago
Failure of 3D TVs was one unprecedented glorious victory of a customer, where customers not buying it indeed led to its disappearance. Otherwise I'm furiously not buying other ridiculous stuff but my consumer decision does nothing.
hdgvhicv · 1h ago
When stock price has to grow 8% more than gdp this is inevitable.
Retr0id · 20m ago
I also think it looks like garbage, but I wonder if maybe it looks better on small mobile screens - where you can't actually see the mangled details, but can perceive that it "looks sharper"
cercatrova · 11m ago
I like upscaling and frame interpolation but as always, the TV does not have the hardware to do a good job. If you use neural network models, it works and looks a lot better without looking plastic-like.
omnimus · 1h ago
What makes it uneasy is not only upscaling but they are generating new frames to make it 60fps. 60fps by itself feels fake (check some footage of The Hobbit that tried 48fps). It feels like video games.
It's kinda funny to aim for 60fps because modern video productions will often have 60fps footage that's too sharp and clean. So they heavily post process the videos. You add the film grain and lower the fps to 30 or even 24 (cinema) so it looks much more natural.
The question is if this is just habitual / taste thing. We most likely wouldn't prefer 24fps if the movie industry started with 50fps.
ageitgey · 2h ago
The AI upscaling makes it look like NIN are playing with late-1980s era Rick Astley. Hilarious.
bryanrasmussen · 1h ago
Rick Astley is so ubiquitous now, thanks to Memes, the AI is never gonna give him up.
justinator · 2h ago
That is terrible.
I see this upscaling a lot in Youtube videos about WWII that use very grainy B+W film sources (which themselves aren't using the best sources of) and it just turns the footage into some weird flat paneled cartoonish mess. It's not video anymore, it's an animated approximation.
No comments yet
robotbikes · 43m ago
This reminds me of colorized black and white movies from the 90s although I can know imagine AI being used to do that and upscale the past creating new hyper-real versions of the past.
rightbyte · 3h ago
The closeups of the bass player are like 6 slowmotion frames in the original and look like an interpolated mess with unhuman body joints upscaled.
internet_points · 1h ago
Wow, that is horrible! The 2:07 mark where AI put in some generic Rick Astley-alike for Bowie, just made me feel sick
black_puppydog · 2h ago
Holy... wtf...
At 2:04 the original deliberately has everyone on stage way out of focus, and the AI upscaler (or the person operating it) decided to just replace it with an in-focus version sporting what looks like late 90s video game characters. That is terrible.
MrGilbert · 2h ago
Also, David Bowie looks like a 20-something-year old man in this shot.
energy123 · 2h ago
The most upvoted comment is "Thank you so much for preserving this!!"
SanjayMehta · 2h ago
The first video induced actual physical nausea.
I had to stop playback or I’m sure I would have thrown up. And I don’t suffer from motion sickness etc.
There’s definitely something “uncanny valley” about it.
kg · 3h ago
Wow, you're not kidding. In some shots David Bowie barely looks like David Bowie because the algorithm's taken such liberties with the original image to try and make it look sharp.
petralithic · 3m ago
I'm not sure about this specific instance, but AI generated movies will absolutely be the future, when you can create the exact shots you want with stability of the foreground, background, and characters, and edit it all together, it'll be an explosion of creativity just as with image generation currently.
To be clear, I don't think it'll be telling an AI to "create me a movie with X, Y, and Z" because AI reasoning is not there yet, but for the raw video generation, it's progressing steadily, as seen in r/aivideo.
jader201 · 3h ago
Two root comments (so far) are focusing on YouTube, but the article claims most of the AI was done by Will’s team, using AI to convert stills to video:
> The video features real performances and real audiences, but I believe they were manipulated on two levels:
1. Will Smith’s team generated several short AI image-to-video clips from professionally-shot audience photos
2. YouTube post-processed the resulting Shorts montage, making everything look so much worse
You can see the side-by-side [1] of the YouTube post-processing, and, while definitely altering the original, isn’t what’s causing most of the really bad AI artifacts.
Most of what YouTube appears to be doing is making it less blurry, sometimes successfully, and sometimes not. And, even with that, it is only done on Shorts.
Some PM in Youtube: “ yes let’s make it harder to tell real videos from AI to make people who don’t know better more susceptible and accepting of it”
aleph_minus_one · 1h ago
>
Some PM in Youtube: “ yes let’s make it harder to tell real videos from AI to make people who don’t know better more susceptible and accepting of it”
This can backfire, perhaps making people believe that real, important news is in reality AI-generated to brainwash them, thus making people less susceptible, and more disbelieving.
cyanydeez · 2h ago
Thats the future. Kids arnt going to have the same mental history as older generayions.
petargyurov · 1h ago
On this episode of "Trying to make AI useful"...
Seriously, who's idea was this? It can't be a money saving feature; surely it costs more to upscale all these videos than to just host the HD version.
And even if you argue it can be used only on low res videos to provide a "better experience", the resulting distortion of reality should be very concerning.
nicgrev103 · 1h ago
If I were a marketing person I would also make genuine images look AI generated for the free publicity. Nothing gets attention like mistakes or fakes. The fact that they aren't actually fake means there is no downside for WS and team. I once spoke to a social media manager for a large brand and he said they intentionally put typos in posts on a semi regular basis and it always results in more post engagement (people correcting the typo).
chneu · 1h ago
It's called ragebait and it's pretty common in marketing already.
matsemann · 36m ago
Like the new Naked Gun movie adding extra floating fingers to their poster, heh.
Today on The Verge, GenAI upscaling in YT shorts. Yes, AI is here to stay, but I do hope the icky parts go away soon.
merelysounds · 1h ago
> GenAI upscaling in YT shorts
I cannot watch the linked video, but its description quotes “not generative AI”; is The Verge or someone else showing something different?
recursive · 1h ago
I fear the icky parts might be the only ones staying.
No comments yet
ojagodzinski · 2h ago
Open an company that sells t-shirts with "AI glitched" text on it so people can make every foto of this kind illegitimate.
aitchnyu · 2h ago
In a couple of years, they can go into same junk drawer of sixth finger prosthetics (generative AI problem) and 5-eyes masks (face recognition problem).
foota · 2h ago
I wonder if the fact that the original video was AI generated made the upscaling look worse than it would on a real video? Not that it can certainly be detected, but an actual video is likely different from an AI generated in ways that it seems like could lead astray their "computational photography" processing.
Theres not a single person i know in my life who will want this as a consumer. WHY does the world keep doing things that are so complicated and unnecessary.
lifestyleguru · 37m ago
Because necessary things are equally complicated but less profitable.
black_puppydog · 3h ago
So... the videos showing the difference between the AI-tainted youtube version and the supposedly untainted instagram version are hosted on... youtube?
ch_sm · 3h ago
apparently the sharpening algorithm is only applied on youtube shorts, not on regular youtube videos.
superchink · 3h ago
the experimental post processing was only applied to shorts, according to the post.
bertman · 36m ago
I find it hilarious that the Youtube spokespersons go out of their way to clarify that this is "not the bad GenAI shit that we know everyone hates but the good kind, you know, machine learning and stuff, you know, trust us"
mxschll · 40m ago
From now on, I am going to bring an unreadable AI looking sign to every concert I go.
cobertos · 3h ago
What's the point of using videos like this if it's a risk to reputation just to use them?
hamdingers · 2h ago
The people with whom this is a reputational risk were not going to buy Will Smith concert tickets anyway.
jacquesm · 3h ago
The same reason you don't feed prime vegetables and fruits to pigs.
jonplackett · 3h ago
Saving money of course! That’s the sad truth
hanspeter · 3h ago
What's the point of saving money if it's a risk to reputation?
cyanydeez · 2h ago
Will smith punched a dude on stage, a comedian. I think you are putting a lot on a cage concept with a scatter plot of outliers.
You actually need a reputation of merit for there to be risk. Hes a rapper, not a saint or Ethicist.
hdgvhicv · 49m ago
“AI” editing is one thing
The upscaling seems to be Google doing it without permission of the original uploader. Google however are unaccountable, you can’t complain otherwise you’ll be exiled
Vote with wallet etc /s
14213112 · 3h ago
there are no risk of reputation until you use it. further more, even within creative professions, using Gen AI is already acceptable to some degree.
wordofx · 2h ago
I’m wondering at what point the minority are going to finally accept ai is here to stay.
hettygreen · 3h ago
What happens when AI gets trained on AI slop?
If there's code to stop AI from being trained on AI, I would like to have it from stopping me from seeing it.
energy123 · 1h ago
Google have the untainted video, so it's a competitive advantage
cyanydeez · 2h ago
Better question is what happens when a generation knows nothing but slop
booleandilemma · 2h ago
I hate how everyone thinks we have to use AI now. I wish this trend would end already.
ttflee · 2h ago
TLDR: The video got VAEed, and we are the discriminators being fooled.
bananapub · 2h ago
doesn't seem very blurry - don't generate video of a crowd? seems like an easy rule?
Mistletoe · 2h ago
Who the hell are going to Will Smith concerts in Europe? He’s been dead to me since the slap, I cannot imagine going to a concert of his with a clean conscience.
wordofx · 2h ago
I thought Willy had disappeared into the shadows after his wife embarrassed him to the point he prob won’t step foot in public again…
I was recently sent a link to this recording of a David Bowie & Nine Inch Nails concert, and I got a serious uneasy feeling as if I was on a psychedelic and couldn't quite trust my perception, especially at the 2:00 mark: https://www.youtube.com/watch?v=7Yyx31HPgfs&list=RD7Yyx31HPg...
It turned out that the video was "AI-upscaled" from an original which is really blurry and sometimes has a low frame rate. These are artistic choices, and I think the original, despite being low resolution, captures the intended atmosphere much better: https://www.youtube.com/watch?v=1X6KF1IkkIc&list=RD1X6KF1Ikk...
We have pretty good cameras and lenses now. We don't need AI to "improve" the quality.
Yesterday we went to a store to have a look at a few smartphone for my partner. She primarily wants a good camera above any other parameter. I was seeing her preferring those that were counterfeiting the reality the most: she was like, "look I can zoom and it is still sharp" while obviously there was a delay between zooming and the end result which was a reconstructed, liquid like distorded version similar to the upscaling filters people are using on 8/16bit game console emulators. I was cringing at seeing the person I love the most preferring looking at selfies of picture of us with smoothed faces and a terrible fake bokeh in the background instead of something closer to the reality.
These groups used to be a mix of people being confused at how their camera worked and wanting help, people wanting tips on how to take better pictures, and sometimes there was requests for editing pictures on their behalf (eg “I found this old black and white faded picture of my great grandparents, can anyone help restore it?”)
These days, 99.9% of the posts are requests that involve synthesizing an entirely new picture out of one or more other pictures. Examples: “can someone bring in my grandpa from this picture into this other family picture?”. Or “I love this photo of me with my kids, but I hate how I look. Can someone take the me from this other picture and put it in there? Also please remove the cups from our hands and the trees in the background, and this is my daughter’s ex boyfriend please also remove him”.
What’s even crazier is that the replies of those threads are filled with dozens of people who evidently just copy pasted the prompt + picture into ChatGPT. The results look terrible… but the OP is always pleased as punch!
People don’t care about “reality”. Pictures have lost their status of “visual record of a past event”* and become “visual interpretation of whatever this person happens to want”.
There’s no putting back the genie in the bottle.
*: yes, you can argue they were never 100% that, but still, that’s effectively what they were.
It's made for making sales, not for making things actually look good.
I think we lost something in that. Embarrassment can be useful for moving us out of our comfort zones.
Over time I noticed everything looks cheaper on their TV.
It was the auto-smoothing.
https://en.m.wikipedia.org/wiki/Soap_opera_effect
It was absolutely an artistic choice - Sony spent more per frame on those movies than any previous animated film & the directors knew exactly what they were doing when they chose to animate some parts on every second (or even third) frame.
The extreme blur here was obviously a creative choice by the director/editor, the rest of the video has lower resolution but it's not nearly that bad (which is why Bowie still looks like himself in other parts of the upscaled video).
The process used to upscale the video has no subtlety, it's just "make everything look crisp, even if you have to create entirely made-up faces".
Flashback to when every TV at CES had 3D functionality. Turns out nobody really wanted what. What an immense waste of resources that was.
It's kinda funny to aim for 60fps because modern video productions will often have 60fps footage that's too sharp and clean. So they heavily post process the videos. You add the film grain and lower the fps to 30 or even 24 (cinema) so it looks much more natural.
The question is if this is just habitual / taste thing. We most likely wouldn't prefer 24fps if the movie industry started with 50fps.
I see this upscaling a lot in Youtube videos about WWII that use very grainy B+W film sources (which themselves aren't using the best sources of) and it just turns the footage into some weird flat paneled cartoonish mess. It's not video anymore, it's an animated approximation.
No comments yet
At 2:04 the original deliberately has everyone on stage way out of focus, and the AI upscaler (or the person operating it) decided to just replace it with an in-focus version sporting what looks like late 90s video game characters. That is terrible.
I had to stop playback or I’m sure I would have thrown up. And I don’t suffer from motion sickness etc.
There’s definitely something “uncanny valley” about it.
To be clear, I don't think it'll be telling an AI to "create me a movie with X, Y, and Z" because AI reasoning is not there yet, but for the raw video generation, it's progressing steadily, as seen in r/aivideo.
> The video features real performances and real audiences, but I believe they were manipulated on two levels:
1. Will Smith’s team generated several short AI image-to-video clips from professionally-shot audience photos
2. YouTube post-processed the resulting Shorts montage, making everything look so much worse
You can see the side-by-side [1] of the YouTube post-processing, and, while definitely altering the original, isn’t what’s causing most of the really bad AI artifacts.
Most of what YouTube appears to be doing is making it less blurry, sometimes successfully, and sometimes not. And, even with that, it is only done on Shorts.
[1] https://youtu.be/Bx5GzIsmEBI
This can backfire, perhaps making people believe that real, important news is in reality AI-generated to brainwash them, thus making people less susceptible, and more disbelieving.
Seriously, who's idea was this? It can't be a money saving feature; surely it costs more to upscale all these videos than to just host the HD version.
And even if you argue it can be used only on low res videos to provide a "better experience", the resulting distortion of reality should be very concerning.
Today on The Verge, GenAI upscaling in YT shorts. Yes, AI is here to stay, but I do hope the icky parts go away soon.
I cannot watch the linked video, but its description quotes “not generative AI”; is The Verge or someone else showing something different?
No comments yet
https://www.youtube.com/watch?v=pAQ3JuXkDkU
You actually need a reputation of merit for there to be risk. Hes a rapper, not a saint or Ethicist.
The upscaling seems to be Google doing it without permission of the original uploader. Google however are unaccountable, you can’t complain otherwise you’ll be exiled
Vote with wallet etc /s
If there's code to stop AI from being trained on AI, I would like to have it from stopping me from seeing it.