Large Language Model (LLM)

LLMs are specialized machine learning models engineered for both understanding and generating text. These models are created using artificial neural networks and feature tunable parameters. The creation process involves two main steps: pre-training and fine-tuning. During pre-training, the model is exposed to a large text dataset and learns to predict the next word in a sequence. This helps it understand language structure and context. In the fine-tuning stage, the model is trained on a more specialized dataset, with adjustments made to its parameters to optimize performance. The end result is a model capable of generating coherent and contextually relevant text based on a given input.

Parameter

A parameter is an internal numerical variable that the model fine-tunes during its training process. These tunable parameters, often consisting of weights and biases in neural networks, are adjusted to capture underlying patterns in the training data. Their optimization is critical for enhancing the model's proficiency in tasks such as prediction, classification, or natural language understanding and generation.

Pre-training

Pre-training is the initial phase where a model is trained on a large, general-purpose dataset before being fine-tuned for specific tasks. During this stage, the model learns basic features and patterns from the data, such as language structure in the case of language models. The goal is to establish a broad understanding that can serve as a foundation for more specialized learning later on. Pre-training enables the model to have a head start, making it easier and quicker to fine-tune for particular applications.

Supervised Learning

A cornerstone methodology in machine learning, supervised learning involves training models on a dataset that includes both input features and corresponding output labels. The model's goal is to discover the relationship or mapping between these inputs and outputs. It adjusts its internal parameters during this learning process to minimize the error in its predictions. Once trained, the model can then apply this learned mapping to new, unlabeled data, aiming to generate accurate and reliable predictions or classifications.

Fine-tuning

The process through which a machine learning model's parameters are further optimized for a specialized task or dataset. This process refines the model's capabilities, allowing it to perform with greater accuracy and relevance in its designated application. Fine-tuning narrows the model's focus, adapting its broad, pre-trained knowledge to excel in a particular domain. An example of fine-tuning for the Glass Health use case would be providing the model with a dataset that consists of example input clinical problem representations and output clinical plans to improve the models ability to generate clinical plans.

Reinforcement Learning from Human Feedback - RLHF

A specialized training methodology for Large Language Models (LLMs) like GPT-4. The process starts with supervised learning on a broad dataset, where the model learns to predict text sequences. After this, the model undergoes a fine-tuning phase that uses human-generated feedback. Typically, human reviewers are presented with multiple model-generated outputs for the same input prompt and are asked to rank them in order of relevance, coherence, or accuracy. Errors flagged or corrections made by these reviewers are then fed back into the model. The model's parameters are adjusted based on this feedback, effectively "teaching" it to produce more accurate and reliable results over time. This method amalgamates machine-driven statistical learning and human judgment to refine the model's capabilities.

Generative Pre-trained Transformer 4 (GPT-4)

GPT-4 is an advanced large language model employing a neural network architecture with billions of tunable parameters. These parameters have been optimized through supervised learning on extensive text datasets. Designed for multi-faceted natural language tasks, GPT-4 is adept at text generation, translation, question-answering, and summarization. Building on its predecessors, GPT-4 offers enhanced scale and performance, aiming to produce outputs that are both coherent and contextually accurate.