This article first appeared in Digital Edge, The Edge Malaysia Weekly on November 13, 2023 - November 19, 2023

Imagine owning a holo-pet that is able to respond to your commands and play with you, whenever and wherever. Or having a holo-friend that can be your best pal without your having to step out of your home.

The complexities of human relationships often make life unpredictable and difficult at times. So, what if we were able to construct an artificial intelligence (AI) powered companion based on our preferences? One that is able to generate real-time responses in your interactions?

AI Holographic technology has risen to new heights recently, with the Hypervsn SmartV Digital Avatar being released at the start of the year. The AI hologram functions on the SmartV Window Display, a gesture-based 3D display and merchandising system, allowing for real-time interaction with customers.

At home, Universiti Teknologi Malaysia (UTM) has developed its first home-grown real-time holo professor, which is able to project a speech given by a lecturer who is in another place. With Malaysia breaking boundaries with extended reality (XR) technology, is it possible for the next wave of hologram technology to be fully AI-powered without constraints?

The idea of interacting with holograms essentially boils down to humans interacting with computers. Interacting with computers usually comes with interacting with the keyboard or mouse but holograms take it a step further, making computer interaction seamless and more natural.

“So ultimately, it’s just humans interacting with computers. But in the next paradigm shift, it is going to be so easy that at times, we won’t even know that they are there,” says Ivan Gerard Khoo, director of Ministry XR, a spatial computing solutions developer.

With generative AI advancing at a rapid rate, to integrate it into holograms would provide a greater immersive experience of interacting with computers around you.

Khoo shares his thoughts on AI being able to push past the barrier of computer interaction through a device with holographic technology, especially in older communities who might not be tech savvy.

“We’ve got a billion apps here, right? But it’s still not easy to use for everyone (like the handicapped or the elderly). Imagine all the apps in our phone right now [becoming] accessible in the environment around us. And the evolution has begun as the enabling technologies, although nascent, are here today,” says Khoo.

In fact, a lot of researchers are seeing that we are actually moving towards an artificial general intelligence that may even develop sentience, chimes in Andrew Yew, founder and chief technology officer of Ministry XR.

As much as it is promising to develop artificial sentients, Yew mentions that no machine thus far has ever passed the Turing test convincingly, which determines whether AI is capable of thinking like a human being.

With minimalism on the rise, the focus turns to the technology and hardware surrounding integrating AI into holograms. Is it possible to create a hologram which is not restricted by a display enclosure?

In movies, you don’t need anything and you [are able] to interact with the virtual world just like that. But in order to make it happen, you need hardware to make it work. You need to set up those things in such a way that it has all of that, so that it can trick your mind [and you think it is] holographic but actually, it is not, explains Kapil Chhabra, founder of Silver Wings XR Interactive Solutions Pte Ltd.

Holograms demonstrate an illusion of light rays reflected onto a medium. They are three-dimensional images generated by interfering beams of light that reflect real, physical objects.

Now, imagine AI bringing the technology of eye tracking into holographic figures, allowing them to have eye contact with humans. Olaf Kwakman, managing partner of Silver Wings XR Interactive Solutions, thinks that it is a brilliant solution as users do not need glasses anymore. “There’s still technology needed but with eye tracking, you can create some kind of projection. And that works beautifully,” he says.

Now, if you make these screens really large and all around you, you can basically project it any way you like. But we’re not quite there yet, Kwakman says.

The challenge with projecting holograms onto mediums is the ability to project it in such a way that it is invisible to the human eye, so that the holograms are more realistic. Chhabra says this has been a struggle for some time and he hopes that it can be made possible in the future.

Taking inspiration from the Apple VR Headset’s pocket-sized and portable battery solutions, Kwakman says it has a very promising augmented reality visualisation but adds that the hardware needs to be further evolved into something smaller.

“If you ask me, what’s going to happen in the future is that you’re not going to wear glasses anymore, you’re going to wear some kind of small lens, which you can just put in your eye. And with a lens like that, you can project augmented reality in full,” he says.

With AI’s potential, it could bring realistic 3D holograms to new heights, where it fills in the gaps and makes the interactive experience much more engaging and powerful.

In order to realise full holographic and 3D visualisation, “you need a strong connection as well, because there’s a lot of data flowing”, says Kwakman.

The lack of usage of holographic solutions is due to poor understanding and awareness of the benefits of the technology, which in turn hampers progress, he adds.

“It’s very difficult to envision the advantage it can bring to a company to introduce holographics, 3D visualisation solutions, and how it will actually benefit them. And, leaders find that troublesome as well, which means that it is difficult sometimes to get the budget for it,” says Kwakman.

Having created Malaysia’s first home-grown holo professor, Dr Ajune Wanis Ismail, senior lecturer in computer graphics and computer vision at UTM’s Faculty of Computing, shares that XR hologram systems can be complex to set up and maintain. Technical issues, such as connectivity problems or software glitches, could disrupt lessons.

AI algorithms are used to enhance the accuracy of holographic content, reducing artifacts and improving image quality. These holographic solutions in extended reality (XR) technology come as a challenge as the technology is relatively new and is rapidly evolving with new breakthroughs occurring since then.

“Building and deploying AI-powered holographic systems can be costly [in terms of hardware and software components].”

Incorporating AI into holograms could pose an immense demand on computational power. Most of the existing holograms produce non real-time content with a video editing loop, but AI models for holography are computationally intensive, says Ajune.

She emphasises the importance of achieving high-fidelity reconstruction in handling complex dynamic scenes with objects or viewers in motion.

“Researchers are developing more efficient algorithms and leveraging hardware acceleration [such as graphics processing units] to reduce computational demands,” says Ajune on how achieving real-time interaction with holographic content demands low latency.

There is no doubt that XR holograms systems are complicated and a challenge to integrate with AI, however, the prospect of being able to replicate environments and enable real-time global communication without the need for physical presence spurs excitement.

As we advance into the era of digitalisation, people need to start familiarising themselves with this technology and become proficient users, believes Ajune.

“There is a lot of information out there but the teachers are still sticking to conventional methods of teaching, and the students are not paying attention because they are on their phone [and] learning it [the info] themselves,” says Yew.

With AI and XR hologram technology becoming more and more advanced, it is also pertinent to educate users and raise awareness about digital wellbeing.

There must be sensibility and responsibility from business owners and users in utilising XR and AI technology, as society’s mindset drives the continued advancement of such technologies.

“I think [what AI can do] is going to be amazing but at the same time, like many others, I also see the risks there. And sometimes it feels a bit scary, if so much power is given out [of the] hands of humans and with computers being able to do that,” says Kwakman.

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's App Store and Android's Google Play.