Part 1 of the series is here. In our last post we saw how an LLM when given some text to start with could generate more text. This idea forms the basis of a tool like ChatGPT but there are still a few steps to get there. GPT3 by OpenAI was maybe the most powerful model…

Part 1 of the series is here. Machine learning and neural network systems have been around for a while but it seems like everything has really exploded in the last few years. We’ve gone pretty quickly from computers using AI to understand  “Alexa, set a timer for 5 mins” most of the time,  to worrying…

Part 1 of the series is here. Many different techniques have been used for Machine Learning over the years, but since all of the most exciting recent advances have been made using neural network systems that’s what we will focus on in this series. So let’s start from the basics. A neural network is a…

A friend of mine asked me to explain to him what’s going on with all this AI stuff. What should they know about it? etc. This person is very smart but not technical in all the ways we computer people like to use that word so I wanted to try to create an explanation which…

It’s pretty amazing what these LLMs can do in a role as an assistant to a human user. As long as a human is there to check the results, we will see more and more places where LLMs and systems built on top of them will improve the productivity of users. What’s harder to think…

I’ve been thinking about this conception of LLMs as a lossy compression of the internet. Ted Chiang talks about this here. Karpathy also mentions it in his latest video. What does this loss look like? It doesn’t always seem like the “loss” part matters. Maybe its even good. Maybe sometimes what you want is some…

I am a big fan of Karpathy’s style of explaining things so I was very happy to see this video drop. Honestly, Its not my favorite of his but it covers a lot of good topics. I enjoyed the second half more than the first but its still a good starter video for people who…