It already exists for 4k TVs. For the new ones with NVIDIA GPUs.

The method still needs knowledge about the world aka a fairly large training set. And it will make errors when the input is "ambiguous". I think this technique is best suited for post processing old video, where an editor can check for blatent inconsistencies and then "hint" the algorithm. That said, it may be possible to provide a general purpose "video filter" which can filter to super resolution a video with the help of a "hint file" for a specific movie.

@@johnwt7333 Wow, I didn't know that! Just knew about standard upscaling. I'm specifically talking about the TecoGAN from this paper though. That's a whole new level.

Yes, computervision on still images is, in a lot of ways, a _harder_ problem then computer vision on a few sequential frames. We humans don't recognize things from single snapshots in the wild, we use continuous input (from 2 eyes, no less!) to get shape and shading information that a single still image cannot capture. Photographs of objects have to be curated by skilled photographers in order to be as easily discernible to us as the real world. Viva la video!

The NVIDIA SHIELD TV is the first streaming media player to feature AI upscaling. It can upscale 720p or 1080p HD content to 4K at up to 30 frames per second in real time. Trained offline on a dataset of popular TV shows and movies, the model uses SHIELD’s NVIDIA Tegra X1+ processor for real-time inference. AI upscaling makes HD video content for top apps including HBO, Hulu, Netflix, Prime Video and KZfaq appear sharper on 4K TVs, creating a more immersive viewing experience.

Doubling as a Rorschach test surfacing the machine's training bias...

"Over there." "Enhance Image."

I'm gonna doubt that it can't translate illegible details from one frame into another with (accurate) highly legible details. I wonder what happens to accuracy given images of text with varying levels of resolution, for example. I'm going to guess the model, given a woefully low-resolution image of a document, would output an image where some details are refined but the text would still be illegible.

@@brandonbahret5632 text would be an interesting test if it can close the loop on temporal and spacial variation. Have you seen some of the astrophotography software? I can't believe they get such amazing results with so little frame to frame information

"Enhance" from crime movies and TV shows will never happen. Algorithms like this just fill in missing information with convincing fakes. Great for upscaling images and movies, but useless for crime footage, because what you are seeing doesn't actually match reality.

What about 8k cameras? ENHANCED!

And a few papers down the line, and we will even be able to ROTATE!

"Shows here your dad broke in to this warehouse last night." "Yeah but he's dead" "Listen the camera clearly derived your dad's face, and you can tell by the pirate outfit that his intent was to steal items"

@@johntheux9238 Enhance the enhanced!

So we might finally be able to see Bigfoot in all the potato clips?!

Lmao

Now I imagine the algorithms all calling their mums for help "Are you sure you have looked *everywhere*, sweetheart?"

@@NicksStuff "Look, it's right under the test folder"

@@NicksStuff *"just think of where you last put it"*

It would not work very optimally for all types of video. This method works best for material where most of the scenery is "stereotypical" - That is, where detail does not carry essential information or has a particular esthetic quality. Your average sitcom or talkshow? Yes! A game? Yes. A high end production with carefully crafted visual effect? No very well.

This will only work when the output is the exact same as the ground truth

@@michaelrenper796 Thanks for your addon. (also a subset of individuals will suspect sarcasm there, but I assure my tone is genuine)

I'm pretty sure a lot of cloud oriented companies and streaming companies are looking into this. NVIDIA already has a system that upsamples stuff on the card itself, real time, so you can get much more frame rate in games by running lower resolution and upscaling it

it's already what we're doing with compressed videos in a way, but the output is more predictable whereas with ai upscaling the result may be more impressive at times but the outcome risks to be less predictable. What would truly revolutionize video streaming I believe is ai compression and decompression.

Anyone care to donate a GPU to a worthy cause?

@Thomas Baruzier wdym?

nope. too slow for youtube . yet

@@HISEROD Guess he didn't understand that we are talking about videos that don't have a better resolution than 480p available. (Aren't we?)

We have to be careful, though: kzfaq.info/get/bejne/edGppqZ7p8qrpKs.html :)

I wonder how it would react to quake game as an input.

already has try SSAA or high power MSAA (never use FXAA, it just blurs everything without considering game geometry, hence known as the, Vaseline Blur)

@@SBImNotWritingMyNameHere You can use FXAA and sharpening. Some games have terrible anti-aliasing and this is the only way to make them look better.

Success

@@davidmartinek5257 The issue with FXAA is it is only frame contextual and sharpening a blurred image always results in a terrible quality picture. Add to that that it does little to really mitigate aliasing and you still have the issue of temporal shimmering. TAA and TXAA mitigate this to a point but it's still imperfect. DLSS is the current best thing besides downsampled TAA but they're ALL expensive. Take this AI for example. To upscale from 360p to 1080p could take anywhere from 0.5 seconds per frame to maybe even MINUTES per frame. People acting like this kind of thing is going to take off tomorrow and solve our "bandwidth issues" are just plain wrong. The nearest step to fixing bandiwdth concerns is simply better compression algorithms but that's not cheap, either. AV1 encoder is phenomenally good but a lot of lower end systems simply can't decode it efficiently and until good codecs gain traction there simply wont be a hardware implementation for them that is readily available to everyone. Eventually the hardware to do this kind of upscaling will be common place but it's still not as cheap as simply sending much more efficiently compressed frames across a network -- nor is it as cheap as simply having a modern GPU render a game in a higher resolution and downsampling it.

@Lilith does stuff Already exists.

@@DasAntiNaziBroetchen what is it?

@@mihailmilev9909 Google "Audio Super Resolution with Neural Networks". There's at least one more I know of.

I see tears of steel, Me likey.

take that japanese porn

Finally uncensored hentai

This is going to be a problem in Japan, since the law there forbid porn movies which are not pixelated.

I wonder if it even can make little things a little bit larger ...

I didn't even think about that

Open and shut case rookie. It was Mel Gibson all along. How can we stop this Mel Gibson crime spree?

For better, and in large part, worse.

2 days ago..?!?!?!?

@@sans2744 Patreon members get early access to videos.

what's the point of that

@@bookymydoor less rendering time

@@B9poy And just generally use worse textures lol. Wouldn't make a difference.

Nvidia DLSS already does something like that.

@@christoph6055 yeah but some people prefer to use that method it's a bit faster but doesn't give the same results

not to forget the biases introduced will do a lot of harm :D

that wont be reliable. thing is the ai is imaginning what the image should be not what the image actually is. so basically its legally not useful as evidance. the faces could be fully fabricated by ai and not the real face of criminals.

ali 32bit Valid point 👍🏼 hadn’t thought of that!

@@skierpage that is exactly the same as what were doing now just with normal people. if you cant see a proper face then any estimation is false. because its gussing it out of nowwhere. exactly what a human would do . you dont go" i think its this guy is guilty because my imagination thinks that blurrry fottage looks like him" in court.

@@skierpage the ai isn't magic, it won't able to see twenty pixels of pink and figure whose face those pixels belong to. Needs a minimum threshold of quality and quantity of data to be usable.

I've been thinking AMD would have no chance to compete with DLSS in the near future, this video changed my mind.

DLSS 2.0 is quite powerful, maybe when nvidia announces their new gen next week we get an improved DLSS 3.0 or something like that.

You don't have to, the Chinese and the US are already applying this tech to their spy satellites on each others military bases. That was the initial purpose of this algorithm. Games and fun stuff are only the side effect.

@@Fred_Costa Algorithmically it is easy to compete with DLSS, real challenge will be matching nvidias hardware level experiance on building AI accelerators

DLSS2 already exists and it's way better than v1. TecoGAN will need to compete with v2 then.

You are too kind. Thank you so much! 🙏

4K 120 fps. yes, you can

analogic images cannot be properly numerically up-scaled

@@ikagura Once analog video is digitised, the principle is the same. Even digital cameras start off as analog (i.e. light entering through aperture)

You can already get a pretty amazing result with Topaz Video Enhancement AI software. It's expensive tho, but gives results.

@@razeezar I don't think that's the definition of analog video

I took a quick look at the paper, and it seems like that could become a usable option in the near future. It's already on the ms/frame timescale there, so "a few GPUs down the line" we should be able to do this with cheaper devices. :)

Yes. There are already companies doing this. When TecoGAN first came out, I found some, but I don't have the links on hand anymore.

Something else to note is that you could use an AI to compress the image to an internal representation, which may or may have a smaller footprint than traditional compression algorithms. I'm not at all privy to the insides of an AI or traditional compression algorithms, but from all the crazy stuff people have been doing with AI it doesn't sound implausible at all.

@@typistkid9012 Now that's a good point! I guess that if the network is large/good enough to get an understanding of depth and perspective and various kinds of movements, it could do a lot better than current compression algorithms. It might be a lot better at prioritizing elements of importance in a video.

Thing is that I imagine these learning algorithms might be very bloated in size, so aside from taking a long time to decompress that low quality movie (which most likely could have been sent over the net quicker, or maybe even physical copies if you're talking about enough data or a poor enough connection speed), it still might not be particularly viable to "end-users" due to the space and resource hog. It would definitely have a place in remastering old media though, or enhancing security camera footage (granted, I bet that this would be totally inadmissible evidence in court, but still could be useful for investigations)

Looking at how they use CGI fakes, maybe they already cut even more costs than that, by not even investing in innovation?

Good point. True on an information basis, that slow moving pixels are being resampled multiple times. However video does increase the dimensionality, volume of data and general complexity of the algorithm.

@@jamesflames6987 though despite said dimensionality, volume, & complexity, doesn't the time dimension(usually) have exploitable localities like neighboring pixels to a CNN?

Super resolution on single images does already exist and it outperforms video super resolution in visual fidelity. TecoGAN in principle works just as well on images. While your assumption with having more data to work with is valid, the general aim is to produce visually plausible results rather than recovering actual true high-res information in the image. Compared to images, videos have the added complexity of temporal consistency which is the unique selling point of the proposed method.

@@jonasmayer5624 ah yes, temporal consistency as well, kinda reminds me of Obra Dinn's dithering

Rendering time went up by +1000 for one frame

yeah, it would be awesome, im currently using Topaz Gigapixel AI for the same reason but for Images, im a graphic designer.

More likely you would be using this kind of tool to upscale the video and save it. Then edit in whatever you use to edit. Then the integration might come later.

@@gritcrit4385 Ok, so... Is there a way I can use this without programming? is there a GUI?

@@JoelGustafsson I don't know enough to answer that. But I doubt there are any end user ready installer and GUI. Try asking around. Maybe you could find some guided steps to achieve it.

That would be rather revolutionary especially for video games knowing that it would dramatically increase performance..

Nova Verse yeah, in video games its called DLSS2 every new AAA title has it.

@@Nephron89 Yeah, Nvidia's A.I super Sampler.