The State of ML in 2023 – Non_Interactive – Software & ML
I’ve been trying to figure out how to best write this article for most of the last year. Today, I’ve decided to just write down something, rather than continue trying to wordsmith exactly what I mean. I am tremendously excited by everything that is going on in ML right now. The breadth of the problem space to which we can apply generalist learning techniques seems virtually unbounded, and every time we scale something up, we see new capabilities start to emerge. With all that being said, I think it’s worth considering from time to time what we haven’t achieved. Lets take a quick tour or the current state of the art: The image space I know quite well. With DALL-E 3, we cracked spelling (most of the time), which was a major capability gap of text-to-image models when compared to humans. We still have a lot of problems. These models can’t tell left from right, can’t count past three, and still have a hard time getting pose and positions of body parts right. We still have to take hacky app
I’ve been trying to figure out how to best write this article for most of the last year. Today, I’ve decided to just write down something , rather than continue trying to wordsmith exactly what I mean. I am tremendously excited by everything that is going on in ML right now. The breadth of the problem space to which we can apply generalist learning techniques seems virtually unbounded, and every time we scale something up, we see new capabilities start to emerge. With all that being said, I think it’s worth considering from time to time what we haven’t achieved. Lets ta
related reading
- GPT-4openai.com
- The Scaling Hypothesis · Gwern.netgwern.net
- Scaling: The State of Play in AIoneusefulthing.org
- GenAI Handbookgenai-handbook.github.io
- It's time to become an ML engineer • Greg Brockmanblog.gregbrockman.com
- Scaling is subtler than it seemsberen.io
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Alex L. Zhangalexzhang13.github.io
- AI progress is about to speed up | Epoch AIepoch.ai
- larger language models may disappoint you [or, an eternally unfinished draft] — LessWronglesswrong.com
- machine learning imindslice.substack.com
- Things we learned about LLMs in 2024simonwillison.net