.png)
Agent Spotlight: Anthony Avila, a Machine Learning Operator who Learns as Fast as a Machine
January 17, 2023
Invisible learned in a recent survey that business leaders are prioritizing the adoption of new automation tech in response to growing concerns over economic volatility.
It makes sense why business leaders would turn to automation in this environment: the job market is still hot, investors are clutching their wallets, and automation tech represents a solution that cuts long-term costs and lessens the need for hiring.
Beyond the standard automation approaches businesses might take, one automation technology stands out – the AI language model GPT-3. Developed by the leading AI laboratory Open AI and available via its API, GPT-3 is a next-generation technology that represents a step up from the automation tech most businesses are used to in that it uses deep learning to mimic human text.
Copywriting, for example, has emerged as a primary use case for language models like GPT-3. But the business application of GPT-3 does not have to be siloed within marketing.
In fact, marketing use cases only scratch the surface of what an effectively used GPT machine can do for your organization. With capabilities to automate other business functions like customer service, data operations, and code generation, the flexibility of the technology is what makes it so exciting.
GPT-3 by itself doesn’t automatically start solving your business problem. Unlike a Google Sheet that’s useful right out of the gates, with some new technologies like this one, it is not always clear how to immediately use them.
The AI uses deep learning to produce human-like text from a given text prompt. Trained on a massive amount of data - 45 TB worth of human-generated text datasets - GPT-3 predicts the best response to the prompt based on its interpretation of the prompt and training data.
With those capabilities, GPT-3 does the following things at a high level:
To unlock the flexibility the model has to offer like in the use cases we discuss below, it needs to be expertly trained on sometimes hyper-specific datasets. This is accomplished by a technique called fine-tuning.
Thankfully, Invisible can help with fine-tuning your model.
GPT-3 is extremely flexible, lending itself to any number of use cases. Let’s look at some examples.
Whether you’re interfacing B2B or D2C, your customer support team could be managing thousands of interactions with customers at once. GPT-3 can be a powerful tool in your CRM toolkit by managing these customer interactions for you with a tone and style nearly indistinguishable from yours.
GPT-3 already excels at short interactions — a singular customer inquiry with a singular helpful response from a well-trained model would be an example of this. Naturally, companies have gravitated towards “chatbots” that assume the role of a human customer service representative to manage lengthier customer service inquiries that require more back-and-forth.
That’s one direction that GPT-3 is headed. But, the tech provides two distinct advantages over the average chatbot:
The key to setting up your language model to be successful as a chatbot is in the way you train it. With as few as a thousand sample interactions, you can have a smart, trained bot answering questions and directing customer service inquiries to the right place.
Say your company wants to provide helpful answers to lengthy and complex customer inquiries. Invisible would train GPT-3 chatbot by feeding your model prompt examples that maximized its flexibility to engage with different types of questions within a single conversation.
That means that a customer could ask numerous questions covering unrelated topics and the chatbot would be able to keep up and continue to provide assistance without human intervention.
With an approach like this, a company could develop a GPT-3 driven chatbot to automate customer service at multiple touch points that allows it to offer personalized interactions at scale without hiring an expensive army of human representatives.
Let’s explore how GPT can be used to automate and strengthen your data operations using an example from the logistics industry. Since the start of the COVID-19 pandemic, the industry and the integrity of supply chains have been frequently undermined by disruptions.
With no end to supply chain volatility in sight, logistics leaders need to lean more heavily on data operations as a predictive tool to better prepare for supply chain disruptions before they happen and a reactive tool to improve their response. Adding next-gen automation tech, even just to speed up daily tasks, can make a difference.
Logistics as an industry has not widely adopted GPT - but it should. The AI would generate value in logistics in two key areas: data operations and automated communication between stakeholders.
One intuitive application for GPT-3 in supply chain data operations is by automating reporting and summarizing data. A supply chain analyst might input a complex sheet of data into a model trained to summarize it and receive an intelligent digest to share among managers.
An emerging idea for more technical logistics analysts, however, is using GPT-3 to turn plain English into database-ready SQL, or Structured Query Language. Users in these roles spend an inordinate amount of time writing code to comb databases for actionable data insights.
Brian Kane, a data analyst at SeekWell, demonstrated how he’s using GPT-3 to do just that in a recent blog. He noted he’s automated a tedious aspect of his job by shortening the time it takes to create inputs in SQL syntax.
As supply chain leaders increasingly adopt SQL databases for forecasting and other use cases, Invisible could deploy a GPT model to quickly call upon critical data in time-critical scenarios. In an industry where every moment counts, tech-driven efficiency makes a difference.
Moreover, the tech can be used to report time-critical information among supply chain stakeholders via AI-generated emails and other communication channels.
The best way to implement your fine-tuned model is by using Invisible’s process execution platform. We enable clients across any industry to program business processes that we execute with the combination of a flexible workforce and automation expertise.
Invisible is experienced in preparing data for machine learning use cases, providing data for organizations to make their AI models smarter.
In one example, former Google executives got demand for their trend-discovery platform faster than they could meet it. Invisible processed 10,000 keywords a week to feed their model, helping them expand to new regions and verticals.
If you’re on the fence about introducing GPT-3 into your organization, it could be because you’ve wondered how you’d relate to it, whether it would take your job or enhance it. The reality is that the emerging technology can add valuable flexibility and promote growth in your organization.
If you want to try it to improve workflows in your company, let us know!
01|
02|
03|
.svg)
Invisible learned in a recent survey that business leaders are prioritizing the adoption of new automation tech in response to growing concerns over economic volatility.
It makes sense why business leaders would turn to automation in this environment: the job market is still hot, investors are clutching their wallets, and automation tech represents a solution that cuts long-term costs and lessens the need for hiring.
Beyond the standard automation approaches businesses might take, one automation technology stands out – the AI language model GPT-3. Developed by the leading AI laboratory Open AI and available via its API, GPT-3 is a next-generation technology that represents a step up from the automation tech most businesses are used to in that it uses deep learning to mimic human text.
Copywriting, for example, has emerged as a primary use case for language models like GPT-3. But the business application of GPT-3 does not have to be siloed within marketing.
In fact, marketing use cases only scratch the surface of what an effectively used GPT machine can do for your organization. With capabilities to automate other business functions like customer service, data operations, and code generation, the flexibility of the technology is what makes it so exciting.
GPT-3 by itself doesn’t automatically start solving your business problem. Unlike a Google Sheet that’s useful right out of the gates, with some new technologies like this one, it is not always clear how to immediately use them.
The AI uses deep learning to produce human-like text from a given text prompt. Trained on a massive amount of data - 45 TB worth of human-generated text datasets - GPT-3 predicts the best response to the prompt based on its interpretation of the prompt and training data.
With those capabilities, GPT-3 does the following things at a high level:
To unlock the flexibility the model has to offer like in the use cases we discuss below, it needs to be expertly trained on sometimes hyper-specific datasets. This is accomplished by a technique called fine-tuning.
Thankfully, Invisible can help with fine-tuning your model.
GPT-3 is extremely flexible, lending itself to any number of use cases. Let’s look at some examples.
Whether you’re interfacing B2B or D2C, your customer support team could be managing thousands of interactions with customers at once. GPT-3 can be a powerful tool in your CRM toolkit by managing these customer interactions for you with a tone and style nearly indistinguishable from yours.
GPT-3 already excels at short interactions — a singular customer inquiry with a singular helpful response from a well-trained model would be an example of this. Naturally, companies have gravitated towards “chatbots” that assume the role of a human customer service representative to manage lengthier customer service inquiries that require more back-and-forth.
That’s one direction that GPT-3 is headed. But, the tech provides two distinct advantages over the average chatbot:
The key to setting up your language model to be successful as a chatbot is in the way you train it. With as few as a thousand sample interactions, you can have a smart, trained bot answering questions and directing customer service inquiries to the right place.
Say your company wants to provide helpful answers to lengthy and complex customer inquiries. Invisible would train GPT-3 chatbot by feeding your model prompt examples that maximized its flexibility to engage with different types of questions within a single conversation.
That means that a customer could ask numerous questions covering unrelated topics and the chatbot would be able to keep up and continue to provide assistance without human intervention.
With an approach like this, a company could develop a GPT-3 driven chatbot to automate customer service at multiple touch points that allows it to offer personalized interactions at scale without hiring an expensive army of human representatives.
Let’s explore how GPT can be used to automate and strengthen your data operations using an example from the logistics industry. Since the start of the COVID-19 pandemic, the industry and the integrity of supply chains have been frequently undermined by disruptions.
With no end to supply chain volatility in sight, logistics leaders need to lean more heavily on data operations as a predictive tool to better prepare for supply chain disruptions before they happen and a reactive tool to improve their response. Adding next-gen automation tech, even just to speed up daily tasks, can make a difference.
Logistics as an industry has not widely adopted GPT - but it should. The AI would generate value in logistics in two key areas: data operations and automated communication between stakeholders.
One intuitive application for GPT-3 in supply chain data operations is by automating reporting and summarizing data. A supply chain analyst might input a complex sheet of data into a model trained to summarize it and receive an intelligent digest to share among managers.
An emerging idea for more technical logistics analysts, however, is using GPT-3 to turn plain English into database-ready SQL, or Structured Query Language. Users in these roles spend an inordinate amount of time writing code to comb databases for actionable data insights.
Brian Kane, a data analyst at SeekWell, demonstrated how he’s using GPT-3 to do just that in a recent blog. He noted he’s automated a tedious aspect of his job by shortening the time it takes to create inputs in SQL syntax.
As supply chain leaders increasingly adopt SQL databases for forecasting and other use cases, Invisible could deploy a GPT model to quickly call upon critical data in time-critical scenarios. In an industry where every moment counts, tech-driven efficiency makes a difference.
Moreover, the tech can be used to report time-critical information among supply chain stakeholders via AI-generated emails and other communication channels.
The best way to implement your fine-tuned model is by using Invisible’s process execution platform. We enable clients across any industry to program business processes that we execute with the combination of a flexible workforce and automation expertise.
Invisible is experienced in preparing data for machine learning use cases, providing data for organizations to make their AI models smarter.
In one example, former Google executives got demand for their trend-discovery platform faster than they could meet it. Invisible processed 10,000 keywords a week to feed their model, helping them expand to new regions and verticals.
If you’re on the fence about introducing GPT-3 into your organization, it could be because you’ve wondered how you’d relate to it, whether it would take your job or enhance it. The reality is that the emerging technology can add valuable flexibility and promote growth in your organization.
If you want to try it to improve workflows in your company, let us know!
LLM Task
Benchmark Dataset/Corpus
Common Metric
Dataset available at
Sentiment Analysis
SST-1/SST-2
Accuracy
https://huggingface
.co/datasets/sst2
Natural Language Inference / Recognizing Textual Entailment
Stanford Natural Language Inference Corpus (SNLI)
Accuracy
https://nlp.stanford.edu
projects/snli/
Named Entity Recognition
conll-2003
F1 Score
https://huggingface.co/
datasets/conll2003
Question Answering
SQuAD
F1 Score, Exact Match, ROUGE
https://rajpurkar.github.i
o/SQuAD-explorer/
Machine Translation
WMT
BLEU, METEOR
https://machinetranslate
.org/wmt
Text Summarization
CNN/Daily Mail Dataset
ROUGE
https://www.tensorflow
.org/datasets/catalog/
cnn_dailymail
Text Generation
WikiText
BLEU, ROUGE
Paraphrasing
MRPC
ROUGE, BLEU
https://www.microsoft.
com/en-us/download/details.a
spx?id=52398
Language Modelling
Penn Tree Bank
Perplexity
https://zenodo.org/recor
d/3910021#.ZB3qdHbP
23A
Bias Detection
StereoSet
Bias Score, Differential Performance
Table 1 - Example of some LLM tasks with common benchmark datasets and their respective metrics. Please note for many of these tasks, there are multiple benchmark datasets, some of which have not been mentioned here.
Metric
Usage
Pros
Cons
Accuracy
Measures the proportion of correct predictions made by the model compared to the total number of predictions.
Simple interpretability. Provides an overall measure of model performance.
Sensitive to dataset imbalances, which can make it not informative. Does not take into account false positives and false negatives.
Precision
Measures the proportion of true positives out of all positive predictions.
Useful when the cost of false positives is high. Measures the accuracy of positive predictions.
Does not take into account false negatives.Depends on other metrics to be informative (cannot be used alone).Sensitive to dataset imbalances.
Recall
Measures the proportion of true positives out of all actual positive instances.
Useful when the cost of false negatives is high.
Does not take into account false negatives.Depends on other metrics to be informative (cannot be used alone)and Sensitive to dataset imbalances.
F1 Score
Measures the harmonic mean of precision and recall.
Robust to imbalanced datasets.
Assumes equal importance of precision and recall.May not be suitable for multi-class classification problems with different class distributions.
Perplexity
Measures the model's uncertainty in predicting the next token (common in text generation tasks).
Interpretable as it provides a single value for model performance.
May not directly correlate with human judgment.
BLEU
Measures the similarity between machine-generated text and reference text.
Correlates well with human judgment.Easily interpretable for measuring translation quality.
Does not directly explain the performance on certain tasks (but correlates with human judgment).Lacks sensitivity to word order and semantic meaning.
ROUGE
Measures the similarity between machine-generated and human-generated text.
Has multiple variants to capture different aspects of similarity.
May not capture semantic similarity beyond n-grams or LCS.Limited to measuring surface-level overlap.
METEOR
Measures the similarity between machine-generated translations and reference translations.
Addresses some limitations of BLEU, such as recall and synonyms.
May have higher computational complexity compared to BLEU or ROUGE.Requires linguistic resources for matching, which may not be available for all languages.
Table 2 - Common LLM metrics, their usage as a measurement tool, and their pros and cons. Note that for some of these metrics there exist different versions. For example, some of the versions of ROUGE include ROUGE-N, ROUGE-L, and ROUGE-W. For context, ROUGE-N measures the overlap of sequences of n-length-words between the text reference and the model-generated text. ROUGE-L measures the overlap between the longest common subsequence of tokens in the reference text and generated text, regardless of order. ROUGE-W on the other hand, assigns weights (relative importances) to longer common sub-sequences of common tokens (similar to ROUGE-L but with added weights). A combination of the most relevant variants of a metric, like ROUGE is selected for comprehensive evaluation.
