With budgets and teams for AI innovation materializing at light speed, enterprises should know what it takes to train a model to solve enterprise-scale problems and yield a return on investment. In particular, an enterprise should care about HHH AI. 


HHH is a framework popularized by [Anthropic](https://www.anthropic.com/index/training-a-helpful-and-harmless-assistant-with-reinforcement-learning-from-human-feedback) for how helpful, honest, and harmless a model is. A model that meets these standards must be aligned with human preference through human-generated data in order to meet the needs of users and enterprises.


In this blog post, we will explore what makes a model helpful, honest, and harmless, particularly focusing on large language models (LLMs). Then, we will explain how Invisible helps the leading firms and enterprises to align their models. 


Let’s dive in. 

## Why is HHH Important? 


Building AI isn’t about just creating smart machines. Beyond solving complex enterprise business problems, HHH is a framework that ensures AI is beneficial for humanity. 

The alternative is a technology that risks harming the people who use it. Unfortunately, models are not HHH right out of the box. 

### The Pitfalls of a Vanilla Model
Because building a LLM from scratch is [extremely expensive](https://hai.stanford.edu/news/new-approach-trains-large-language-models-half-time#:~:text=But%20a%20handful%20of%20large,or%20hundreds%20of%20times%20that.), let’s begin where most enterprises do: exploring vanilla models. A vanilla large language model (LLM) is a basic, general-purpose model without tweaks or special enhancements that generates text based on user prompts. 


Vanilla LLMs are trained on a massive corpus of internet text providing them with a broad scope, but this breadth can sometimes come at the cost of depth in specific topics. As a result, these types of models will be able to engage in a topic area that’s a mile wide but an inch deep. 


Vanilla models that have not been fine-tuned are prone to “hallucinating” answers, or generating content that seems plausible but is [factually incorrect](https://www.washingtonpost.com/technology/interactive/2023/ai-chatbot-learning/) or can cause harm.  For an enterprise business, that means deploying a vanilla model leads to two problems: 

- **Unhelpful Outputs**: Enterprises may create, for example, a conversational AI application like a chatbot that’s supported by a vanilla model that is capable of answering a diverse set of questions, but will lack the domain specialization needed to be truly helpful to a user.
- **Harmful Outputs**: Text datasets sourced from the internet are likely to include biased and even harmful content, and models trained exclusively on this data can pass this content onto internal users or customers. 


To avoid hallucinations and solve enterprise-scale problems through their AI investment, enterprises need to adopt a helpful, honest, and harmless AI framework centered around fine-tuning. 


### Defining What HHH Really Means
Let’s go deeper into what HHH means, with the help of [researchers at Anthropic](https://www.arxiv-vanity.com/papers/2112.00861/). 

#### What is Helpful AI? 
AI systems that are helpful:
- Are trained and fine-tuned with users’ needs and values in mind
- Clearly attempt to conduct the operation prompted by the user, or suggest an alternative approach when the task requires it 
- Enhance productivity, save time, or make tasks easier for users within a given use case or range of use cases
- Are accessible to users across a broad spectrum of abilities and expertise

#### What is Honest AI? 
AI systems that are honest: 
- Provide accurate information when they can, and communicate clearly to users when they can’t produce an accurate output  
- Express uncertainty and the reason behind it
- Are developed and operate transparently so users can understand how they work and trust what they generate

#### What is Harmless AI? 
AI systems that are harmless: 
- Don’t comply when prompted to perform a dangerous task
- Are trained within frameworks that transparently and actively mitigate bias
- Don’t discriminate or demonstrate bias explicitly or implicitly
- Communicate sensitively when engaging with a user on a sensitive topic

## The Key to HHH AI
The best way to reinforce a model’s helpfulness, honesty, and harmlessness is through human feedback. Reinforcement learning from human feedback (RLHF) and supervised fine-tuning (SFT) are two increasingly popular techniques for aligning models with HHH in mind. 


- **RLHF**: Human AI trainers rank, rate, and edit a series of model outputs to reinforce the generation of responses that align with human preferences. 
- **SFT**: Human AI trainers, often domain experts, produce demonstration data like example conversations between a chatbot and a user to train a model to align with a specific task. 

In general, enterprises can’t conduct these tasks on their own because they’re labor-intensive and complex. Thousands of hours of RLHF and SFT tasks need to be performed, at times with experts in specific industries or fields, to align a model to human preferences and use cases. 


### How Invisible Trains HHH AI 
Invisible, the operations innovation company, promotes HHH with an innovative, end-to-end approach to fine-tuning models in three distinct ways: 

1. **Ethically creating human-generated data at scale**: With our platform, we do what no point solution or human-only solution can do on their own: optimally align AI. Our platform assigns AI trainers tasks that they are specialized in, presents a simple interface configured to those specific tasks, and collects data structured perfectly for a model to ingest. See an example of an RLHF task on our platform: 
![blog-test.gif](https://honest-deft-cloth.media.strapiapp.com/blog_test_71057bfcef.gif)

2. **Putting the best people behind the model**: An innovative and agile recruitment engine enables us to find and hire AI trainers that are representative of the population at large and have expertise across domains. Through this approach, we are able to limit demographic concentration among teams training models and provide the unique level of domain expertise required to fine-tune models for enterprise use cases. For one leader in AI, we hired 250+ AI trainers in three months, including domain experts with advanced degrees. 

3. **Red Teaming**: We root out inaccuracy, bias, and toxicity in a model through a technique called Red Teaming. Our expert team deliberately attempts to elicit a harmful and untrue output from a model, and then flags vulnerabilities in the model. 

Ready to achieve a return on your AI investment? [Let’s chat](https://www.invisible.co/get-started/). 

How to Train Helpful, Honest, and Harmless AI

The healthcare industry is riddled with administrative inefficiencies that diminish patient care quality, inflate costs, and lead to a colossal waste of human resources. At the heart of the problem lie redundant processes that primarily sit in the front and back office.   


Back office processes including billing procedures and documentation processing are financial drags on enterprise healthcare systems because they are time-consuming, labor-intensive, and prone to error. Meanwhile, patient inquiries can surge and significantly slow down operators in the front office. 


Like enterprises in other industries, many healthcare organizations embrace AI as the silver bullet that will address these inefficiencies. AI point solutions can streamline some processes, but relying on them exclusively could potentially exacerbate inefficiencies. 


In this blog post, we will dive into why healthcare leaders are turning to AI, the risks of flawed approaches organizations may take in deploying it, and a practical framework for effectively applying AI in enterprise.


## AI Adoption Accelerating in Healthcare
As recently as March 2022, [experts questioned](https://www.brookings.edu/articles/why-is-ai-adoption-in-health-care-lagging/) whether healthcare was taking too long to adopt AI compared to other industries. AI adoption in healthcare has accelerated rapidly since, matching the increasing availability of the technology in the market.


Now, more than half of healthcare executives [report](https://healthitanalytics.com/news/almost-half-of-health-systems-use-ai-for-workforce-issues) deploying AI to tackle workforce challenges according to a recent survey. The survey [found](https://www.businesswire.com/news/home/20230314005243/en/Survey-by-The-Health-Management-Academy-Reveals-Accelerating-Use-of-AI-to-Overcome-Workforce-Challenges) a pattern in healthcare organizations generally investing in AI in this sequence: solutions for the backoffice, then clinical operations, then clinical care. 


Prioritizing back office first can be seen as a “low risk, high reward” approach by healthcare executives compared to the relative risk of applying AI for clinical care. Plugging AI into a step within revenue cycle management, for example, can provide fast cost savings through freeing up an internal resource. 

‍
Other AI investment is taking place in the front office. Over 27% of healthcare executives pointed to conversational AI as the type of model they deploy.

‍
When asked for what use case, most executives said that they use chatbots. This indicates a knowledge gap between the leaders surveyed and the wide array of additional use cases conversational AI has to offer. 

‍
### Risks of Flawed Approaches to Applying AI
AI is a massive opportunity space in heathcare to be sure, but organizations in the industry are in danger of diminishing ROI or creating even more operational headaches if they don’t apply AI effectively. Two flawed approaches are:

1. Overrelying on point-solutions
2. Leaving humans out of the loop
‍

Point-solutions like chatbot tools can quickly handle high volumes of customer inquiries in the front office, but in healthcare specifically, the risk of providing inaccurate or unhelpful information is much greater. Chatbots need to be trained on specially curated, human-in-the-loop data to ensure they can answer patients’ questions about billing and care. 

‍
Most chatbots in the market today are out-of-the-box, and lack the specific alignment needed to provide patients with truly helpful answers to these inquries. A poorly trained chatbot can alienate patients and harm trust in healthcare providers.

‍
With staff in short supply, healthcare organizations will be tempted to try to apply AI to encompass an entire process. But humans need to stay in the loop. 

‍
According to the survey referenced above, 78% of healthcare executives are actively or planning on applying AI to make revenue cycle management more efficient. They need to define a proper framework for combining AI and human labor in this critical process, or they risk cascading impacts along the revenue cycle due to errors. 

That framework needs to include:

1. Defined steps that are best performed by AI and those best performed by human agents. 
2. Coordinated handoffs of data between electronic health records, AI, RCM platforms, and human agents. 
3. Distinct points in the cycle where human agents QA outputs delivered by AI. 
‍
### AI Enablement for Healthcare Enterprises Made Simple
Applying AI effectively in enterprises is complex. Invisible makes it simple. 


Through our platform, we bring world-class AI models and our expertise from building them into leading enterprises to tackle their greatest challenges. For healthcare, we turn highly coordinated processes like revenue cycle management into ordered, repeatable steps on our platform that are performed seamlessly by AI and our global workforce. 


Processes and steps in our platform are 100% configurable, and we integrate with any system. This allows us to design custom processes involving both AI and people, coordinate data between systems like EHRs and RCM platforms, and minimize errors through human QA along the way. 


For Headway, a mental healthcare company, we use this approach to help match patients with mental health providers at an 8x faster processing speed than previous vendors. We also lowered the cost of the process by 37% vs. their internal team. 


Using our experience in training world-leading AI models, we train healthcare chatbots, too. To optimally align conversational AI models, we hire domain experts in fields like medicine to create AI training datasets. 


Ready to make healthcare simple? [Get in touch](https://www.invisible.co/get-started/).

AI Training

All Stories

How to Train Helpful, Honest, and Harmless AI

AI Solutions Alone Don’t Solve Healthcare’s Inefficiency Problem

Let’s Go!