When integrating LLMs in your products, you often want to generate structured data, like JSONs. With the help of function calling (released June 13th 2023), this process has become much simpler! In this post I will explore the new API. Thanks for reading Simon’s Substack! Subscribe for free to receive new posts and support my work.
Not sure if it's smart to champion json schemas as a way to go, since its a resource format that brought challenges in the past leading to the emergence of GraphQL — that eventually established capability oriented design over resource oriented imperative implementations. Solving a long standing integration nightmare. So a neat way for now to "get started" with the ChatGPT API specifically, but should not be over invested. OpenAI is aware of this and the guys are working on an implementation that is more similar to https://lmql.ai/#cot ; being able to express natural language prompts that also contains code.
There is no evidence they are using jsonformers. The performance is exactly the same as if you were to just feed it a json schema and tell it to format the output. This is literally exactly the same as prompt engineering they just do it for you.
Not sure if it's smart to champion json schemas as a way to go, since its a resource format that brought challenges in the past leading to the emergence of GraphQL — that eventually established capability oriented design over resource oriented imperative implementations. Solving a long standing integration nightmare. So a neat way for now to "get started" with the ChatGPT API specifically, but should not be over invested. OpenAI is aware of this and the guys are working on an implementation that is more similar to https://lmql.ai/#cot ; being able to express natural language prompts that also contains code.
Just a big thanks for an easy guide on how to use this. :) OpenAIs own documentation is lacking.
Great post. I'm intrigued about how you'd use the schema and function to do chain of thought prompting?
Did you create the image of the Recipe Creator app yourself? It's very well-drawn and aesthetically pleasing.
is there a way to produce json in different languages ? i used to be able to do so with prompt engineering, but now how can we do that ?
Looks like OpenAI added a note since your writing:
> (note: the model may generate invalid JSON or hallucinate parameters)
This suggests that it’s not masking the token posterior with the schema, and just relying on the system message and improved steering.
Sad. I was getting excited at having access to something other than just bulk inference out of these most advanced models
There is no evidence they are using jsonformers. The performance is exactly the same as if you were to just feed it a json schema and tell it to format the output. This is literally exactly the same as prompt engineering they just do it for you.
How could JSON schema be a Turing complete language?