Executive Summary: GPT-4, ChatGPT, and LLMs

OpenAI was founded in 2015 as a research organization focused on advancing artificial intelligence in a safe and beneficial manner.

GPT-3 (Generative Pre-trained Transformer 3) is a state-of-the-art natural language processing model (LLM). With 175 billion parameters, it is one of the largest language models to date.

One of the major improvements to GPT was the result of reinforcement learning from human feedback (RLHF). This technique, originally introduced in 2017, is how OpenAI achieves its state-of-the-art LLMs. For example, an RLHF model was preferred over a 100x larger base GPT-3 model

ChatGPT (GPT-3.5) incorporates similar architecture and training principles as GPT-3 but takes a list of messages as its input and response with a message itself. The ChatGPT API was introduced on March 1, 2023 and is x10 cheaper than the GPT-3 API.

GPT-4 is more reliable, creative, and able to handle much more nuanced instructions than GPT-3.5. OpenAI is very protective of the techincal details behind the improvements in GPT-4 and it’s for example not know of how many parameters it consists.

LLMs

Large Language Models (LLMs) like GPT-4 are very powerful and commonly used for:

• Text Completion: input text as a prompt, and the model will generate a text completion that attempts to match whatever context or pattern you gave it. (OpenAI documentation)
• Code Completion: see for example Github Copilot
• Chat Completion: Chat models take a series of messages as input, and return a model-generated message as output. (OpenAI documentation)

Prompt Design

Models can do everything from generating original stories to performing complex text analysis. You have to be explicit in describing what you want. Showing, not just telling, is often the secret to a good prompt.

There are multiple free resources with prompt examples and guides:

• OpenAI Cookbooks - Examples and guides for using the OpenAI API
• Awesome ChatGPT Prompts - ChatGPT prompt curation to use ChatGPT better
• Codex Prompt Engineering - Prompt engineering guide by Microsoft, focused on generating code

Embeddings

An embedding is a vector (list) of floating point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness.

Embeddings are commonly used for:

• Search (where results are ranked by relevance to a query string)
• Clustering (where text strings are grouped by similarity)
• Recommendations (where items with related text strings are recommended)
• Anomaly detection (where outliers with little relatedness are identified)
• Diversity measurement (where similarity distributions are analyzed)
• Classification (where text strings are classified by their most similar label)

Augmenting LLMs

The simplest possible prompt is a “zero-shot” instruction prompt. You just ask the model to do what you want. This works ok but the model still has a tendency to make things up because it doesn’t know about changing world events or private info.

There are multiple interesting ways in which the capacities of LLMs can be expanded.

Fine-tuning

You can fine-tune GPT-3 models on example prompt-completion pairs. It’s currently not possible to fine-tune ChatGPT models. To fine-tune a model you should provide at least a few hundred high-quality examples, ideally vetted by human experts. From there, performance tends to linearly increase with every doubling of the number of examples. Increasing the number of examples is usually the best and most reliable way of improving performance.

Chain-of-Thought (CoT)

Chain of thought – a series of intermediate reasoning steps – significantly improves the ability of large language models to perform complex reasoning.

Adding Actions

Looking up information

Instead of hoping that the LLM hallucinates the correct answer, its access to external knowled APIs enables it to provide factually correct answers.

prompt

Question: what's the world fastest land animal?
Action: {{google[world fastest land animal]}}
Result: Cheetahs
Answer: The world's faster land animal is a cheetah

Question: what's the largest planet in the solar system?

completion

Action: {{google[largest planet in the solar system]}}
Result: Jupiter
Answer: Jupiter is the largest planet in the solar system

Evaluating code

GPT can generate code and as such can be prompted to generate code.

Executing LLMs generated code can be a huge security vulnurability:

• LLM outputs are not always predictable
• A succesfull man-in-the-middle attack now enables the attacker to directly execute malicious code by tampering the prompt or it’s completion. This is a real concern and as of writing, code injection holds the 3rd place in the OWASP top 10.

Nonetheless it is an interesting use case which, while requiring extra attention to security, can be done responsibly.

Your task is to answer questions correctly. You have access to a Python interpreter, so if you are not able to answer a question from memory, you can write a program that will answer the question. Always write your answer as a valid Python program, with helpful comments.

Begin.

Question: What is 37593 * 67?
Answer:
"""
# Multiply the numbers
print(37593 * 67)
"""

Controlling a Browser

Given a simplified state of the browser, GPT can be prompted for a “next action” to take based on a given goal like “make a reservation at Dorsia for 2”

https://twitter.com/natfriedman/status/1575631194032549888

Startup Adept is trying to take this to the next level by building a machine learning model that can interact with everything on your computer.

Chain-of-Thought x Actions

The ReAct paper pioneerd the combination of CoT with actions. It turns out that combining reasoning and access to external knowledge leads to impressive results.

With chain-of-thought or actions only

With chain-of-thought and actions combined

Another interesting example of combining CoT with actions is the PAL paper where the solution step is offloaded to a programmatic runtime (in this case Python)

Integrating Documents

Or how to add your proprietary knowledge to GPT.

Integrating documents enables GPT models to leverage external documents such as manuals, articles, or other documents to generate more accurate and informative completions.

With fine-tuning

You can generate prompt-completion pairs about your documentation and fine-tune a GPT model with them:

1. Generate prompts by asking GPT “Ask a question about the following documentation”
2. Generate completions by asking GPT “Answer the question about the following documentation” where question is the result of the previous step
3. fine-tune a GPT-3 model on the generated question-answer pairs

With embeddings

Documentation snippets similar to your prompt are selected based on embeddings and added to the prompt. Having access to a relevant documentation snippet will enable GPT to generate improved completions.

1. Calculate embeddings for your documentation
2. When prompting, select x releted documentation snippets
3. Add the snippets to the prompt

The Future of AI

The rapid progress in GPT capabilities has opened up tremendous potential, and many companies are racing to seize the opportunities it presents. Only time will tell which companies will ultimately succeed.

One standout company in the AI industry is Humanloop, which offers a user-friendly interface for prompt evaluation, model fine-tuning, and other tools. In the current gold rush of AI, companies like Humanloop are like sellers of shovels, providing critical infrastructure and support services that are essential for success. As such, Humanloop is poised to continue growing and become a highly successful business.

One company stands out is , the company provides a convenient UI for evaluation prompts, fine-tuning models, and other tools. In the gold-rush of AI, they are selling shovels and as such will likely continue growing into a very succesfull business.

Model Improvements

As LLM models become more advanced, they will likely improve in:

• lower prices, making more LLM based businesses viable
• larger context windows, making more LLM apps possible
• faster models, making more apps viable

Multimodal Models

Different models are being incorporated to create Multimodal Models. This will elevate AI capabilities to even more impressive levels. One of the first examples was the Visual ChatGPT paper from Microsoft where they combined a ChatGPT like model with Visual Foundation Models resulting in a ChatGPT that can understand and edit images:

More recently HuggingGPT or JARVIS was introduced (also by Microsoft). It uses LLMs to act as a controller to manage existing models from the machine learning community’s main repository of AI models, Hugging Face. Because of its access to all Hugging Face models, it’s effectively a generic multimodal model. Below is an example of JARVIS’ capabilities. It references the different AI models that were used in yellow.

Data

Data is crucial in AI, and domain-specific user interaction and feedback data is particularly valuable. Companies that collect and use this type of data effectively may have one of the most valuable assets in the AI industry. Startups can differentiate themselves from larger AI companies by training their AI models using application-specific data. The ability to acquire and leverage domain-specific training data will be a key factor in the success of many AI startups.

Conclusion

In conclusion, the development of language models like GPT-3 and ChatGPT has opened up many possibilities for natural language processing, text completion, code completion, chatbots, and other applications. Prompt design and embedding techniques can further enhance the capabilities of these models. As these models continue to improve and become more advanced, we can expect to see even more impressive applications of AI in the future.