Enhancing products with generative AI

Background

There has been no end to the hype surrounding Generative AI.

It's often referred to as generative models or gen AI, is a subset of AI/ML that focuses on creating new data rather than just interpreting existing data or making decisions based on it. 

Traditional AI/ML, on the other hand, typically involves unsupervised, supervised or reinforcement learning algorithms that are trained on large datasets to perform specific tasks such as forecasting, prediction, classification, regression, or decision-making.

It is also possible to also have “Hybrid AI” scenarios, where Gen AI models are used initially to get to market quickly, but then custom traditional AI models are then created and hosted locally for more control and reduced cost.

New modalities for gen AI

Gen AI encompasses various use cases or modalities, each focusing on generating data of different types.

Some key modalities for generative AI include:

• Text Generation

• Image Generation

• Audio Generation

• Video Generation

• 3D Model Generation

• Structured Data Generation

‍

Complementing traditional AI with gen AI

Gen AI is a complement to a traditional AI strategy.

Off the shelf gen-AI powered tools can help optimize user productivity in a variety of scenarios such as content generation, summarization and transcription. Gen AI technologies can also transform traditional products, or they can be used to create entirely new disruptive products. 

Gen AI relies on powerful foundation models that have been trained on staggeringly large datasets using immense cloud computing resources. 

Training these is well beyond the capability of the average business. For example,the popular GPT3 model, that is the basis of chatGPT, was reputedly trained on 50TB of Internet text  using 10,000 nVidia V100 GPUs running in parallel for many weeks. 

Having such models available as “Intelligence-as-a-service" via easily-accessible, and relatively inexpensive, API calls enables you to do the following:

• Instead of spending time and resources on training a custom model you just choose an apt foundation model, e.g., GPT from OpenAI, Gemini from Google, Titan from Amazon and so on. These are designed to have generalized capabilities and can usually be used out of the box via inexpensive API calls.
• Using prompt engineering to validate your hypothesis quickly without having to train a domain-specific model
• Using techniques such as Retrieval Augmented Generation (RAG) to ground the results to your own proprietary databases
• Building a quick prototype for validation against your use cases

Gen AI can complement a traditional AI strategy, and a Hybrid approach brings the best of both worlds. 

‍

Transform a product with a 'Hybrid AI' approach

Once you have validated your hypothesis you could ship your product to your customers without needing custom models. Note that there are cases where you may need to revert to traditional AI and using a hybrid approach.

There could be a variety of reasons to build a hybrid AI-powered product: 

• Cost: While each API call to a foundation model is cheap, it can add up if your application makes a lot of calls - especially as you scale your business
• Unpredictable latency: Hosted foundation models are shared with all customers of the service provider. This can lead to access latency that is not acceptable in a real-time scenario
• Privacy: Since the foundation models are hosted outside your applications security perimeter it may not be acceptable to share sensitive data in the API calls.

Once you have validated that your approach does work with the best-in-class foundation model, you can then train a traditional custom model optimized for your use cases. This will usually be a far smaller and easy-to-host model that will consume data within your own compute clusters and have an accuracy close to what the foundation model gives you. 

In addition you can use the following techniques

• Create domain-specific synthetic data using foundation models to train a traditional AI model
• Benchmark your model by validating the accuracy of your custom model against the results produced by a best-in-class foundation model.

‍

‍

‍