AI upscaling
When enlarging an image past its native resolution (the resolution that it was captured and then stored at), pixellation is a common issue. The photo becomes grainy and distorted, and you can no longer zoom in because there aren't enough pixels present in the original image. There are numerical methods that can be used to generate new pixels, allowing you to zoom in further. One such method is "blur", where you take the values of the surrounding pixels and average them. This will keep the photo from being noticeably pixellated, but now the photo will look blurry instead of grainy.
Another approach is to use Artificial Intelligence to upscale the image. Essentially, you add more pixels and use an AI algorithm to predict / infer what the values of the pixels should be. The more you upscale the image, the harder the computation becomes because the amount of information you have about the image becomes proportionally smaller compared to the amount of information the algorithm needs to "make up".
It's likely that the AI algorithm is making these "predictions" / inferences based on a weighted combination of properties like tone and balance - but nobody knows for certain. This is because there is no way to audit how a Deep Learning algorithm works - it is a computational black box. Pretty trippy.
I think a cool application of this technology would be to upscale and colorize old photographs and movies and then host them on a free, online archive. It would take a lot of compute power, but it would be a cool project that would allow us to see history in a different light.
Here are some links for further reading ordered in complexity:
https://petapixel.com/2017/10/31/website-uses-ai-enhance-photo-csi-style/
https://thenextweb.com/artificial-intelligence/2017/10/31/ai-can-make-low-resolution-photos-great-again/
https://deepsense.ai/using-deep-learning-for-single-image-super-resolution/
https://arxiv.org/abs/1612.07919
Here's a $100 AI upscaling tool that may or may not work (there's a link to a free one in one of the articles):
https://www.provideocoalition.com/a-i-gigapixel-enlarge-images-using-artificial-intelligence/
Another approach is to use Artificial Intelligence to upscale the image. Essentially, you add more pixels and use an AI algorithm to predict / infer what the values of the pixels should be. The more you upscale the image, the harder the computation becomes because the amount of information you have about the image becomes proportionally smaller compared to the amount of information the algorithm needs to "make up".
It's likely that the AI algorithm is making these "predictions" / inferences based on a weighted combination of properties like tone and balance - but nobody knows for certain. This is because there is no way to audit how a Deep Learning algorithm works - it is a computational black box. Pretty trippy.
I think a cool application of this technology would be to upscale and colorize old photographs and movies and then host them on a free, online archive. It would take a lot of compute power, but it would be a cool project that would allow us to see history in a different light.
Here are some links for further reading ordered in complexity:
https://petapixel.com/2017/10/31/website-uses-ai-enhance-photo-csi-style/
https://thenextweb.com/artificial-intelligence/2017/10/31/ai-can-make-low-resolution-photos-great-again/
https://deepsense.ai/using-deep-learning-for-single-image-super-resolution/
https://arxiv.org/abs/1612.07919
Here's a $100 AI upscaling tool that may or may not work (there's a link to a free one in one of the articles):
https://www.provideocoalition.com/a-i-gigapixel-enlarge-images-using-artificial-intelligence/
As we discussed in class - this is fascinating and perhaps unsettling from the perspective the human image maker. Thanks for sharing!
ReplyDelete