最後更新: 2023-04-25

目錄

• LLaMA
• text-davinci-003
• Alpaca

Models: Text, Audio, Vision, Multimodal, Time series, Reinforcement learning, Graphormer

Foundation model
Fine-tuned model

Test

BoolQ is a question answering dataset for yes/no questions containing 15942 examples.
PIQA is a dataset for commonsense reasoning
SIQA: Social Interaction QA

LLaMA

由 Meta 開發

License: noncommercial license

Parameters

65B, 1.4T tokens
33B, 1.4T tokens
13B, 1T tokens
7B, 1T tokens

Training dataset(English)

CCNet [67%],
C4 [15%],                # 783 GB
GitHub [4.5%],         # 328 GB
Wikipedia [4.5%],     # 83 GB
Books [4.5%],          # 85 GB
ArXiv [2.5%],
Stack Exchange[2%] # 78 GB

text-davinci-003

OpenAI, GPT-3

This model builds on top of InstructGPT
trained with humans in the loop
(reinforcement learning from human feedback (RLHF))

Alpaca

instruction-following language model

fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations