OpenAI Codex (статья года), JuliaCon 2021 starts on July 20

[anhinga_anhinga]

"Evaluating Large Language Models Trained on Code", arxiv.org/abs/2107.03374A detailed description of early (pre-GitHub Copilot) versions of OpenAI Codex. This is the "paper of the year" so far: we finally have real progress in AI-assisted computer programming (and difficulties of computer programming form the key bottleneck limiting the speed of

Comments

[anhinga_anhinga]

Acceleration is rather spectacular this year.

An early-GPT-4-level model is now open source, reasonably compact, fine-tunable (LLama-3-70B in base and instruct configurations).

GPT-4o is two "early GPT-4 minus GPT-3.5" deltas above that, and OpenAI has something much stronger than that internally.

A lot of novel architectural decisions were being published in recent months.

[anhinga_anhinga]

> GPT-4o is two "early GPT-4 minus GPT-3.5" deltas above that

no, actually, only 1.5 deltas above that

[anhinga_anhinga]

Claude 3.5 Sonnet by Anthropic is the smartest model at the moment

[anhinga_anhinga]

Llama-3..1-405B open weights (base and instruct versions)

o1-preview and o1-mini (reasoning and deliberation; System 2 "slow thinking" over System 1 "fast LLM thinking")

a lot of interesting alternative architectures and algorithms

multi-agents over LLMs

[anhinga_anhinga]

o1 and o1-pro release

o3 and o3-mini announcement

open-source somewhat weaker reasoners in that spirit

very formidable "new Claude 3.5 Sonnet"

very formidable series of Google models, including Gemini 2.0 Flash and Gemini 2.0 Flash with Thinking

Liquid AI non-standard architectures

~AGI (meaning ~parity with humans in the ability to do AI R&D) is likely in 2025