This article first appeared in Digital Edge, The Edge Malaysia Weekly on February 26, 2024 - March 3, 2024

When Samsung launched the Galaxy S24 Ultra, the highlight was how this phone brings its very own built-in artificial intelligence (AI), dubbed Galaxy AI, to augment its core offerings natively without the need to download any third party applications. 

That is not to discount the physical attributes of the phone itself. With a bulkier design featuring a fully flat face, a less reflective screen and a titanium frame, the S24 Ultra feels sturdy even without a case and internally features an overall upgrade from the S23 Ultra, sans its battery.

While it uses the same battery as its predecessor, the battery life can easily last about two days of casual use on a single charge. This old battery even gets a longer life, thanks to the new Snapdragon 8 Gen 3 chipset, which not only boosts the phone’s processing speed for normal use and gaming, but its increased efficiency also allows the S24 Ultra to get a little more out of the older battery.

However, the star attraction is Galaxy AI and its host of generative AI offerings that make full use of the phone’s upgraded chipset and processors. Among its noteworthy features are:

1.Circle Search

This is Samsung’s headliner in the new suite of generative AI features. It is an impressive feat of generative AI where holding the home button — or the navigation handle — allows the user to either circle or tap any element on screen to Google Image search it. This can be a picture or icon on the phone, or whatever the camera is capturing.

This has proven to be incredibly diverse in its uses, from finding a replacement for an obscure gift to searching for the history of a curious landmark or even figuring out where that stock image comes from.

2.Chat Assist

Part of Samsung’s goal of innovating the core functions of the smartphone is communication. This adds new icons to the keyboard toolbar, like the Galaxy AI star icon, that allow the generative AI to look through the drafted message to correct grammar and spelling.

When we tested it, the corrections made were inconsistent, with deliberate spelling mistakes for complex words usually leading to incorrect suggestions. However, for simple and short sentences, this feature was reasonably reliable.

It also offers the ability to alter what has been written to conform to specific writing styles that can be useful for work messages or emails, featuring styles like polite, casual, formal and social media — which adds appropriate hashtags to the messages!

Another feature is translation. By tapping the new translation icon, the AI can automatically translate what is being typed into any of its selected language packs. These language packs need to be downloaded and their accuracy is marginally better than Google Translate.

The available language packs are English, Spanish, Chinese, Japanese, Korean, Hindi, Thai, Vietnamese, Polish, Portuguese, Italian, French and German. The English and Spanish packs have support accents, like British, American and Indian for English and Mexican, Spanish and American for Spanish.

3.Note Assist

This feature allows users to quickly organise their notes by creating summaries out of walls of text or by instantly formatting them into bullet points with appropriate headers. All this is done by highlighting the desired text and selecting the summarise or auto-format option from the menu with a minimum of 200 characters.

The entire process was quick and easy, and the results were up to standard, with the key limitations being that this autoformatting had a small and undisclosed maximum word count, making formatted summaries for long-form text difficult and tedious.

The AI is also able to translate text into any of its supported languages, generate a cover for the note and correct the spelling.

4.Generative Edit

On the creative side of Galaxy AI, the biggest showcase of this is in how it brings AI into photo editing. What stood out immediately was how easy, intuitive and fun the entire process can be.

When editing photos, the rightmost icon now features the ability to remove reflections and shadows from photos using generative AI. It can also individually remove elements with impressive results, for example, removing clutter from an office desk.

The leftmost icon, featuring three stars, is where most of the AI functions kick in. This allows the user to also remove objects from the photo, add objects from other photos and even straighten the photo.

For example, the AI was used on a photo of an office desk to replace all elements on the desk with various food items.

Hitting the Generate button leads to the AI filling in the gaps with computer generated detail. While not perfect, it allows a certain degree of creative freedom without much technical knowledge and can be useful for proof of concept images.

On top of all that, the AI also has some impressive passive features, like turning any video into slow-mo video by just holding the video as it plays as well as auto-sharpening and improving the lighting of photos taken, which naturally sharpens blurry photos and brightens dark ones.

5.Live translate and transcribing

The S24 Ultra has the ability to live translate audio coming in, made to be used conversationally either over call or even in person, being quickly accessible via the dropdown menu where the WiFi, Bluetooth and QR reader options are.

Supporting the language models mentioned previously, it is able to translate conversations in real time, although its accuracy is not always consistent. It is difficult to get the AI to recognise stammering and background noise, leading to confusion and, in the case of Chinese, it can only translate traditional Mandarin.

The AI is also able to transcribe audio files from the phone’s audio recording application and summarise the transcription, which it can then export to Notes.

While this is useful, the limitations of the available language models still hold it back, especially when it comes to recognising accents.

The AI struggled to accurately transcribe what was being said by an interviewee with a Malaysian accent when using the English language pack’s British accent option and it was only marginally better with the American accent option.

“Impressive but not quite there yet” perfectly describes the Galaxy AI features. The toolset and opportunity it brings are immense and fun to play around with, but the results often need a lot of adjustment to achieve a satisfactory result.

Overall, the S24 Ultra is an impressive powerhouse of a phone. It delivers great performance as just a smartphone while boasting a slew of new generative AI offerings. While these promise incredible results, the technology — minus Circle Search — still feels too early as it struggles to keep up with expectations.

The S24 Ultra officially hit stores worldwide on Feb 7.

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's App Store and Android's Google Play.