Category: AI

I wonder sometimes about the difference between how an LLM talks and how I do it. It’s hard to really be confident in introspecting your own cognition but at least it feels like an idea forms in my head and then I render the words.  I don’t know if an LLM starts with an “idea”…

Part 1 of the series is here. AI like any new technology brings with it both promises of great benefits and also many concerns about how it might cause bad outcomes. Since there’s plenty of people who are already trying to get rich by selling the idea that AI will solve all our problems, I think…

Part 1 of the series is here. After spending time talking about LLMs and what they can do with text, Let’s go back to looking at images. Text to image generators allow you to just type something like: “create an image of a cat wearing a blue baseball cap” And you get: There are several tools…

Part 1 of the series is here. In our last post we saw how an LLM when given some text to start with could generate more text. This idea forms the basis of a tool like ChatGPT but there are still a few steps to get there. GPT3 by OpenAI was maybe the most powerful model…

Part 1 of the series is here. Machine learning and neural network systems have been around for a while but it seems like everything has really exploded in the last few years. We’ve gone pretty quickly from computers using AI to understand  “Alexa, set a timer for 5 mins” most of the time,  to worrying…

Part 1 of the series is here. Many different techniques have been used for Machine Learning over the years, but since all of the most exciting recent advances have been made using neural network systems that’s what we will focus on in this series. So let’s start from the basics. A neural network is a…

A friend of mine asked me to explain to him what’s going on with all this AI stuff. What should they know about it? etc. This person is very smart but not technical in all the ways we computer people like to use that word so I wanted to try to create an explanation which…

It’s pretty amazing what these LLMs can do in a role as an assistant to a human user. As long as a human is there to check the results, we will see more and more places where LLMs and systems built on top of them will improve the productivity of users. What’s harder to think…

I’ve been thinking about this conception of LLMs as a lossy compression of the internet. Ted Chiang talks about this here. Karpathy also mentions it in his latest video. What does this loss look like? It doesn’t always seem like the “loss” part matters. Maybe its even good. Maybe sometimes what you want is some…