Christine Betterton-Jones : Knowledge junkie
This week we demonstrate Artificial intelligence software and techniques to :
- Enhance and colourise black and white photos to make them look as if they were taken yesterday.
- Animate portraits and photos so that the person in them seems to smile, turn their head and look up and down.
AI technology is also being used to improve old movies and videos so that they actually look better now than when they were first made..and the ultimate tool for eternal youth? . how about a digital avatar of yourself, just like Abba! Welcome to the world of Artificial reality.
Most of us remember the days when we scanned old family photos into our computers, then, noticing all the scratches and blemishes on the image, opened an image editing programme like Paint ShopPro or Photoshop – chose the cloning tool to copy a bit of sky to cover up over a blob on the print, painstakingly drew a line around a selection, then used the colour picker to colourise the area, messed around with sharpening filters….well life is too short for all that, and getting shorter, though the good news is that AI – Artificial Intelligence – is getting smarter and faster so that you can do all that, and more, with a single click of a mouse. This AI is also able to automate the process of digital film restoration, so that old movies can look better now than when they were first made.
Photographs:
There’s a fad these days, not just to enhance old photos and portraits, but to animate then as well:
We thought we would experiment for ourselves, and I used a 14 day free trial of the genealogy website MyHeritage.com . This uses “Deep Nostalgia” artificial intelligence tools to enhance repair, colourise and animate photographs. (The main disadvantage of the trial is that you have to provide payment details before beginning it.)
Not wishing to insult my Grannie, I uploaded a black and white graduation photo of myself – over 50 years old (!) and some other photos that my father had taken in China to play around with.
See this Google Album for the results of the experiment:
https://photos.app.goo.gl/X68KyCayA4QwTpVm9
Conclusion – sometimes very good, but prone to error and exaggeration!
If you want to see and play with some real ghoulish animation stuff: https://www.motionportrait.com/en/indexe.php
(Note – there’s a button to upload your own photo to this site, but this didn’t work for me)
Film restoration:
Film restoration using digital technology has been very active since the early 90’s. The first process is to scan the film frame by frame. Then, until recently, repairs were done manually – a time consuming and expensive process.
Scratches and blemishes, tears, grain, flickering, camera shake, faded colours, hiss on the sound track all have to be corrected. Some of the techniques shown in this video from 2015 are already out of date.
These days, much of the labour intensive work is done by AI.
A famous example is that of YouTuber Denis Shiryaev who took the latest advances in AI technology and used them on one of the earliest works of film: “The Arrival of a Train at La Ciotat Station”, shot by the Lumière Brothers in 1896.
Shiryaev used commercial image-editing software called Gigapixel AI. https://www.topazlabs.com/gigapixel-ai
Here is a video which compares before and after restoration:
Here’s a better video of the enhanced version:
The original film was shot at a low number of frames-per-second, making it jerky. The AI neural networks looked at the original frames and “filled in” the in-between data, creating a more natural effect. People are seen to move on and off the train as they do in real life. It looks like it was shot yesterday.
There are now online services using AI to enhance and restore uploaded video files: e.g. Recuro Media https://www.recuromedia.com/
This is a Swedish company which says its software can do 95% of the manual restoration required at a fraction of the cost. You provide the video files. The high resolution scanning of the raw film footage still needs to be done. However, many such services are available (in Spain too) to digitise both film, VHS tapes, DVDs etc. The scanning of film stock and restoration are normally charged by the minute. Note that 8mm film and VHS media do not give such good results because the original material does not have a high enough resolution.
Many old films are now being re-released in re-mastered high definition. e.g. The Criterion Collection. https://www.criterion.com/shop/collection/327-glorious-restorations This means they can be seen once more on the big screen in modern cinemas which have digital projectors.
A Clockwork Orange has just been remastered in 4K ultra high definition.
https://www.slantmagazine.com/dvd/a-clockwork-orange-4k-review-stanley-kubrick/
One major problem with this and other old films is the mono soundtrack. Converting those to 5.1 surround sound has not been a success – audio is the next challenge.
Abbatars
But forget enhancing old films – why not enhance yourself? Abba has released a new album of songs after 40 years – However, they are all 40 years older, so instead of touring in their ageing bodies, they are appearing as youthful Abbatars of themselves – thanks to some clever camerawork and artificial intelligence