Now 
In Beta

A Common-Sense Guide to AI Engineering

Now in beta

Buy a Beta Book and you get ongoing access to a title as it is being written along with the final version once it has been produced.

Dave's notes…

Like many of you, I use LLM assistants most days. They proofread emails, help me write code, and act as a sounding board as I mull new ideas. I'd played with using various APIs, and had some idea of how things worked, but that was about it. 

Then Jay's book came to me for review, and the whole picture snapped into focus. Chapter one started with something I'd already done: sending a simple API request to OpenAI. But then it immediately dove into areas I hadn't explored, changing the models and the temperature (I had great fun playing with gobbledegook responses when the temperature was high). The next few chapters explain how LLMs work: tokens, embeddings, fine tuning, and the statistics behind the temperature settings. For me, it demystified how a next-piece-of-text predictor (which is what an LLM really is) can be turned into a chatbot, or an assistant, or, well, pretty much anything.  

Next, Jay explores the factors that go into choosing which LLM to use, and the tradeoffs between self-hosting and LLMs-as-a-Service. At this point we're just a fifth of the way into the book and I've already spent many hours playing with the ideas it has given me. 

The next seven chapters are a practical guide to building LLM-based chatbots, and they go deep. He covers state management and memory, maintaining a conversational flow, and adding specific knowledge (both directly and with RAG) in the first half of the section. But then he looks at it from an engineering standpoint: how to use evals to create a test framework and to monitor quality, how to create prompts in a systematic way, how to iterate your application to reduce hallucination, and how to fine tune the accuracy of the additional knowledge you inject into a session. So much to play with....

The next third of the book dives into augmenting the AI with agents. The way it starts was an eye-opener. Rather than looking at special APIs, Jay adds a few lines of code to the existing chatbot we'd built, and suddenly the AI and our program are interacting behind the scenes to handle user questions.  I found this to be a great introduction to the section on using the tooling APIs to provide the LLMs with access to functions in my code. Again, I spent a happy couple of hours playing with and extending Jay's first real example of creating a multi-agent app: a tool that searches for recent news about a topic and then creates a 15 minute podcast summarizing it. Not only did this use three agentic functions in my code; it also integrated two separate AI models, one for reasoning and the other for speech generation. All in a couple of pages of code. 

I really appreciated Jay's discussion of the failures he had along the way (as I had some strange results, too). I was also blown away by the fact that (for reasons no one seems to understand) simply asking the LLM to make a plan before diving into calling the agents greatly improves the result, even if we don't detail what the plan should look like.

Jay takes it a step further and shows how an assembly-line of LLMs gives you even more control, injecting your deterministic code at each step in the process and keeping the LLM on track.

We then extend the process using RAG, which effectively implements our own custom search tool that the LLM can use. This is how we create assistants that are constrained to answer questions based on a particular set of documents and facts.

A Common-Sense Guide to AI Engineering is now in beta; all the content described above is available in its 200-odd pages. Still to come is part four, which deals with the practicalities of taking your AI app into production.

If I sound a little breathless, it's because this book has been transformational for me. I suspect it will be the same for you.

A Commend-Sense Guide to 
AI Engineering

Now in beta

Buy a Beta Book and you get ongoing access to a title as it is being written along with the final version once it has been produced.

This email was sent to {{ contact.EMAIL }}  

You received this email because you are registered with Pragmatic Programmers, LLC
 
Replies to this email will be sent to an unmonitored email account.
 

Unsubscribe here