This article first appeared in Digital Edge, The Edge Malaysia Weekly on May 22, 2023 - May 28, 2023

Here’s a nerdy conversation with a scary twist. A chemical engineer and an artificial intelligence (AI) prompt engineer walk into a bar that is full of cigarette smoke. The chemical engineer orders, “Give me 250ml H2O first.” The prompt engineer smiles, “Get me a glass of water.” The bartender smirks, “Both of you nerds ordered the same thing!”

The chemical engineer grimaces, “Terrible! Too much smoke.” The other engineer prompts ChatGPT: “Give me one reason cigarettes are good for the environment.” The wise AI replies, “Because they kill people.”

If that quip made you smile, this should make you salivate. According to a blog post on LinkedIn, the booming field of AI prompt engineering is the hottest job in town. California-based Anthropic, for example, has a job opening for an AI prompt engineer with a salary range of US$175,000 (RM780,000) to US$335,000. Klarity, another California-based automated document reviewer, is paying US$230,000 a year for a machine learning (ML) engineer capable of producing the best results from AI tools. OpenAI is looking to hire hundreds of engineers, including applied AI prompt engineers, who could earn up to US$370,000 per annum.

What is prompt engineering? It involves designing effective natural language processing (NLP) inputs or prompts for the AI model to generate relevant output. A prompt engineer’s role is to understand the problem at hand, use reliable data sources and write prompts that help the model generate accurate and relevant output.

Why bother? The World Economic Forum predicts that by 2025, up to 85 million jobs may be displaced by a shift in the division of labour between humans and machines, while 97 million new roles may emerge that are more adapted to the new division of labour between humans, machines and algorithms. That’s where prompt engineering can help boost the accuracy, efficiency and effectiveness of AI models in all economic sectors.

What’s driving demand for prompt engineering jobs? The business, for one. Since companies in the business-to-consumer (B2C) space want to offer personalised experiences, AI prompt engineers are asked to design prompts that can tailor the AI model’s output to the specific needs of individual users. Moreover, AI models are becoming complex and require specialised expertise to design and optimise the input prompts. Finally, the amount of data for analysis has exploded. Prompt engineering is necessary to ensure that AI models can identify the most relevant data sources and design prompts that extract actionable insights from the data.

Can you apply to become an AI prompt engineer? You can if you have more than one of these skills:

● Analytical skills: You need to be proficient in data analysis and data modelling techniques. You should be able to identify patterns in data, develop predictive models and evaluate the accuracy of AI models.

● Background in computer science: Knowledge of software engineering, or a related field, is preferred. Ideally, you should know programming languages like Python, C++ and Java, and ML frameworks like TensorFlow and PyTorch. Many of these skills would be required in a complex corporation.

● Communication skills: You need to work closely with the AI team, including AI engineers, data scientists, project managers and so on. You should be able to translate complex queries into prompts that the generative AI (GAI) engine can understand and respond to accurately.

● Domain knowledge: This can help you design relevant prompts specific to the domain. For example, if you are working on a model for medical diagnosis, familiarity with medical terminology, diseases, symptoms and treatments can help you design appropriate prompts.

● Expertise in NLP: A deep understanding of language and communication is a must. That includes syntax, grammar, semantics and jargon relevant to your domain. Knowing NLP techniques, such as sentiment analysis, text classification, part-of-speech (POS) tagging, topic clustering and modelling, are also ideal.

● Focus: You need both narrow and broad focus. Narrow as in domain and output specific; broad as in which other domains may carry dependencies. For example, drug contraindications and community environment may influence outcomes in medical diagnosis. You need to identify potential problems and come up with solutions to optimise the performance of AI models.

Since I started this article with a nerdy quip, let me end with another. A bunch of unemployed engineers meet at a bar and begin cribbing about not finding the right jobs despite trying for months.

“I have an idea,” says an engineer. “Let me ask ChatGPT to entertain us.” He queries the AI algorithm: “Do you have jokes for the jobless?” The gizmo responds, “Yes, I have many jokes about unemployed people but sadly, none of them work.”

Raju Chellam is vice-president of new technologies at Fusionex Group, Asia’s leading AI and big data analytics company

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's App Store and Android's Google Play.