OpenAI Codex (статья года), JuliaCon 2021 starts on July 20

[anhinga_anhinga]

"Evaluating Large Language Models Trained on Code", arxiv.org/abs/2107.03374A detailed description of early (pre-GitHub Copilot) versions of OpenAI Codex. This is the "paper of the year" so far: we finally have real progress in AI-assisted computer programming (and difficulties of computer programming form the key bottleneck limiting the speed of

Comments

[anhinga_anhinga]

Acceleration is rather spectacular this year.

An early-GPT-4-level model is now open source, reasonably compact, fine-tunable (LLama-3-70B in base and instruct configurations).

GPT-4o is two "early GPT-4 minus GPT-3.5" deltas above that, and OpenAI has something much stronger than that internally.

A lot of novel architectural decisions were being published in recent months.

[anhinga_anhinga]

> GPT-4o is two "early GPT-4 minus GPT-3.5" deltas above that

no, actually, only 1.5 deltas above that

[anhinga_anhinga]

Claude 3.5 Sonnet by Anthropic is the smartest model at the moment

[anhinga_anhinga]

Llama-3..1-405B open weights (base and instruct versions)

o1-preview and o1-mini (reasoning and deliberation; System 2 "slow thinking" over System 1 "fast LLM thinking")

a lot of interesting alternative architectures and algorithms

multi-agents over LLMs