Introduce

언어모델과 대규모언어모델을 어떻게 평가해야 하는지 알아봅시다.

• Evaluate Metric

  • MSE

  • RMSE

  • MAE

  • MAPE

  • Confusion Matrix

  • Accuracy

  • Recall

  • Precision

  • F1-Score

  • ROCurve

• PPL

• BenchMark

  • GLUE BenchMark

  • KLUE BenchMark

  • BLEU Score

  • MMLU

    • https://www.kaggle.com/code/debarshichanda/llm-evaluation-mmlu-style

• Korean BenchMark

  • HAERAE BenchMark

  • K-MMLU

  • ko-lm-evaluation-harness

    • https://github.com/Beomi/ko-lm-evaluation-harness