Post-training gets your model to behave the way you want it to. As AMD VP of AI Sharon Zhou explains to Ben on this episode, the frontier labs are convinced, but the average developer is still figuring out how post-training works under the hood and why they should care. In their focused discussion, Sharon and Ben get into the process and trade-offs, techniques like supervised fine-tuning, reinforcement learning, in-context learning, and RAG, and why we still need post-training in the age of agents. (It’s how to get the agent to actually work.) Check it out.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Transcript

This transcript was created with the help of AI and has been lightly edited for clarity.

00.00
Today we have a VP of AI at AMD and old friend, Sharon Zhou. And we’re going to talk about post-training mainly. But obviously we’ll cover other topics of interest in AI. So Sharon, welcome to the podcast. 

00.17
Thanks so much for having me, Ben. 

00.19
All right. So post-training. . . For our listeners, let’s start at the very basics here. Give us your one- to four-sentence definition of what post-training is even at a high level? 

00.35
Yeah, at a high level, post-training is a type of training of a language model that gets it to behave in the way that you want it to. For example, getting the model to chat, like the chat in ChatGPT was done by post-training.

So basically teaching the model to not just have a huge amount of knowledge but actually be able to have a dialogue with you, for it to use tools, hit APIs, use reasoning and think through things step-by-step before giving an answer—a more accurate answer, hopefully. So post-training really makes the models usable. And not just a piece of raw intelligence, but more, I would say, usable intelligence and practical intelligence. 

01.14
So we’re two or three years into this generative AI era. Do you think at this point, Sharon, you still need to convince people that they should do post-training, or that’s done; they’re already convinced?

01.31
Oh, they’re already convinced because I think the biggest shift in generative AI was caused by post-training ChatGPT. The reason why ChatGPT was amazing was actually not because of pretraining or getting all that information into ChatGPT. It was about making it usable so that you could actually chat with it, right?

So the frontier labs are doing a ton of post-training. Now, in terms of convincing, I would say that for the frontier labs, the new labs, they don’t need any convincing for post-training. But I think for the average developer, there is, you know, something to think about on post-training. There are trade-offs, right? So I think it’s really important to learn about the process because then you can actually understand where the future is going with these frontier models.

02.15
But I think there is a question of how much you should do on your own, versus, us[ing] the existing tools that are out there. 

02.23
So by convincing, I mean not the frontier labs or even the tech-forward companies but your mom and pop. . . Not mom and pop. . . I guess your regular enterprise, right?

At this point, I’m assuming they already know that the models are great, but they may not be quite usable off the shelf for their very specific enterprise application or workflow. So is that really what’s driving the interest right now—that people are actually trying to use these models off the shelf, and they can’t make them work off the shelf?

03.04
Well, I was hoping to be able to talk about my neighborhood pizza store post-training. But I think, actually, for your average enterprise, my recommendation is less so trying to do a lot of the post-training on your own—because there’s a lot of infrastructure work to do at scale to run on a ton of GPUs, for example, in a very stable way, and to be able to iterate very effectively.

I think it’s important to learn about this process, however, because I think there are a lot of ways to influence post-training so that your end objective can happen in these frontier models or inside of an open model, for example, to work with people who have that infrastructure set up. So some examples could include: You could design your own RL environment, and what that is is a little sandbox environment for the model to go learn a new type of skill—for example, learning to code. This is how the model learns to code or learns math, for example. And it’s a little environment that you’re able to set up and design. And then you can give that to the different model providers or, for example, APIs can help you with post-training these models. And I think that’s really valuable because that gets the capabilities into the model that you want, that you care about at the end of the day.

04.19
So a few years ago, there was this general excitement about supervised fine-tuning. And then suddenly there were all these services that made it dead simple. All you had to do is come up with labeled examples. Granted, that that can get tedious, right? But once you do that, you upload your labeled examples, go out to lunch, come back, you have an endpoint that’s fine-trained, fine-tuned. So what happened to that? Is that something that people ended up continuing down that path, or are they abandoning it, or are they still using it but with other things?

05.00
Yeah. So I think it’s a bit split. Some people have found that doing in-context learning—essentially putting a lot of information into the prompt context, into the prompt examples, into the prompt—has been fairly effective for their use case. And others have found that that’s not enough, and that actually, doing supervised fine-tuning on the model can get you better results, and you can do so on a smaller model that you can make private and make very low latency. And also like effectively free if you have it on your own hardware, right?

05.30
So I think those are kind of the trade-offs that people are thinking through. It’s obviously very much easier essentially to do in-context learning. And it could actually be more cost-effective if you’re only hitting that API a few times. Your context is quite small.

And the host and models like, for example, like Haiku, a very small model, are quite cheap and low latency already. So I think there’s basically that trade-off. And with all of machine learning, with all of AI, this is something that you have to test empirically.

06.03
So I would say the biggest thing is people are testing these things empirically, the differences between them and those trade-offs. And I’ve seen a bit of a split, and I really think it comes down to expertise. So the more you know how to actually tune the models, the more success you’ll get out of it immediately with a very small timeline. And you’ll understand how long something will take versus if you don’t have that experience, you will struggle and you might not be able to get to the right result in the right time frame, to make sense from an ROI perspective. 

06.35
So where does retrieval-augmented generation fall into the spectrum of the tools in the toolbox?

06.44
Yeah. I think RAG is a way to actually prompt the model and use search basically to search through a bunch of documents and selectively add things into the context, whether it be the context is too small, so like, it can only handle a certain amount of information, or you don’t want to distract the model with a bunch of irrelevant information, only the relevant information from retrieval.

I think retrieval is a very powerful search tool. And I think it’s important to know that while you use it at inference time quite a bit, this is something you teach the model to use better. It’s a tool that the model needs to learn how to use, and it can be taught in post-training for the model to actually do retrieval, do RAG, extremely effectively, in different types of RAG as well.

So I think knowing that is actually fairly important. For example, in the RL environments that I create, and the fine-tuning kind of data that I create, I include RAG examples because I want the model to be able to learn that and be able to use RAG effectively. 

07.46
So besides supervised fine-tuning, the other class of techniques, broadly speaking, falls under reinforcement learning for post-training. But the impression I get—and I’m a big RL fan, and I’m a cheerleader of RL—but it seems always just around the corner, beyond the grasp of regular enterprise. It seems like a class of tools that the labs, the neo labs and the AI labs, can do well, but it just seems like the tooling is not there to make it, you know. . . Like I describe supervised fine-tuning as largely solved if you have a service. There’s no equivalent thing for RL, right? 

08.35
That’s right. And I think SFT (supervised fine-tuning) came first, so then it has been allowed to mature over the years. And so right now RL is kind of seeing that moment as well. It was a very exciting year last year, when we used a bunch of RL at test-time compute, teaching a model to reason, and that was really exciting with RL. And so I think that’s ramped up more, but we don’t have as many services today that are able to help with that. I think it’s only a matter of time, though. 

09.04
So you said earlier, it’s important for enterprises to know that these techniques exist, that there’s companies who can help you with these techniques, but it might be too much of a lift to try to do it yourself. 

09.20
I think maybe fully end to end, it is challenging as an enterprise. I think there are individual developers who are able to do this and actually get a lot of value from it. For example, for vision language models or for models that generate images, people are doing a lot of bits and pieces of fine-tuning, and getting very custom results that they need from these models.

So I think it depends on who you are and what you’re surrounded by. The Tinker API from Thinking Machines is really interesting to me because that enables another set of people to be able to access it. I’m not quite sure it’s quite at the enterprise level, but I know researchers at universities now have access to distributed compute, like doing post-training on distributed compute, and pretty big clusters—which is quite challenging to do for them. And so that makes it actually possible for at least that segment of the market and that user base to actually get started. 

10.21
Yeah. So for our listeners who are familiar with just plain inference, the OpenAI API has become kind of the de facto API for inference. And then the idea is this Tinker API might play that role for fine-tuning inputs, correct? It’s not kind of the whole project that’s there. 

10.43
Correct. Yeah, that’s their intention. And to do it in a heavy like distributed way. 

10.49
So then, if I’m CTO at an enterprise and I have an AI team and, you know, we’re not up to speed on post-training, what are the steps to do that? Do we bring in consultants and they explain to us, here’s your options and these are the vendors, or. . .? What’s the right playbook?

11.15
Well, the strategy I would employ is, given these models change their capabilities constantly, I would obviously have teams testing the boundaries of the latest iteration of model at inference. And then from a post-training perspective, I would also be testing that. I would have a small, hopefully elite team that is looking into what I can do with these models, especially the open ones. And when I post-train, what actually comes from that. And I would think about my use cases and the desired things I would want to see from the model given my understanding of post-training.

11.48
So hopefully you learn about post-training through this book with O’Reilly. But you’re also able to now grasp like, What are the types of capabilities I can add into the model? And as a result, what kinds of things can I then add into the ecosystem such that they get incorporated into the next generation of model as well?

For example, I was at an event recently and someone said, oh, you know, these models are so scary. When you threaten the model, you can get better results. So is that even ethical? You know, the model gets scared and gets you a better result. And I said, actually, you can post-train that out of the model. Where when you threaten it, it actually doesn’t give you a better result. That’s not actually like a valid model behavior. You can change that behavior of the model. So understanding these tools can lend that perspective of, oh, I can change this behavior because I can change what output given this input. Like how the model reacts to this type of input. And I know how. 

I also know the tools right. This type of data. So maybe I should be releasing this type of data more. I should be releasing these types of tutorials more that actually helps the model learn at different levels of difficulty. And I should be releasing these types of files, these types of tools, these types of MCPs and skills such that the model actually does pick that up.

And that will be across all different types of models, whether that be a frontier lab looking at your data or your internal team that is doing some post-training with that information. 

13.20
Let’s say I’m one of these enterprises, and we already have some basic applications that use RAG, and you know, I hear this podcast and say, OK, let’s try this, try to go down the path of post-training. So we already have some familiarity with how to do eval for RAG or some other basic AI application. How does my eval pipeline change in light of post-training? Do I have to change anything there? 

14.03
Yes and no. I think you can expand on what you have right now. And I think your existing eval—hopefully it’s a good eval. There’s also best practices around evals. But essentially let’s say it’s just a list of possible inputs and outputs, a way to grade those outputs, for the model. And it covers a decent distribution over the tasks you care about. Then, yes, you can extend that to post-training. 

For fine-tuning, it’s a pretty straightforward kind of extension. You do need to think about essentially the distribution of what you’re evaluating such that you can trust that the model’s truly better at your tasks. And then for RL, you would think about, How do I effectively grade this at every step of the way, and be able to understand has the model done well or not and be able to catch where the model is, for example, reward hacking when it’s cheating, so to speak?

So I think you can take what you have right now. And that’s kind of the beauty of it. You can take what you have and then you can expand it for post-training. 

15.10
So, Sharon, should people think of something like supervised fine-tuning as something you do for something very narrow? In other words, as you know, one of the challenges with supervised fine-tuning is that first of all, you have to come up with the dataset, and let’s say you can do that, then you do the supervised fine-tuning, and it works, but it only works for kind of that data distribution somehow. And so in other words, you shouldn’t expect miracles, right?

15.44
Yes, actually something I do recommend is thinking through what you want to do that supervised fine-tuning on. And really, I think it should be behavior adaptation. So for example, in pretraining, that’s when the model is learning from a huge amount of data, for example, from the internet, curated. And it’s just gaining raw intelligence across a lot of different tasks and a lot of different domains. And it’s just gaining that information, predicting that next token. But it doesn’t really have any of those behavioral elements to it. 

Now, let’s say it’s only learned about version one of some library. If in fine-tuning, so if in post-training, you now give it examples of chatting with the model, then it’s able to be able to chat over version one and version zero. (Let’s say there’s a version zero.) And you only gave it examples of chatting with version one, but it’s able to generalize that version zero. Great. That’s exactly what you want. That’s a behavior change that you’re making in the model. But we’ve also seen issues where, if you for example now give the model in fine-tuning examples of “oh, here’s something with version two,” but the base model, the pretrained model did not ever see anything about version two, it will learn this behavior of making things up. And so that will generalize as well. And that could actually hurt the model. 

So something that I really encourage people to think about is where to put each step of information. And it’s possible that certain amounts of information are best done as more of a pretraining step. So I’ve seen people take a pretrained model, do some continued pretraining—maybe you call it midtraining, I’m not sure. But like something there—and then you do that fine-tuning step of behavior modification on top. 

17.36
In your previous startup, you folks talked about something. . . I forget. I’m trying to remember. Something called memory tuning, is that right?

17.46
Yeah. A mixture of memory experts. 

17.48
Yeah, yeah. Is it fair to cast that as a form of post-training? 

17.54
Yes, that is absolutely a form of post-training. We were doing it in the adapter space. 

17.59
Yeah. And you should describe for our audience what that is. 

18.02
Okay. Yeah. So we invented something called mixture of memory experts. And essentially, you can hear like the words, except for the word “memory,” it’s a mixture of experts. So it’s a type of MOE. MOEs are typically done in the base layer of a model. And what it basically means is like there are a bunch of different experts, and for particular requests, for a particular input prompt, it routes to only one of those experts or only a couple of those experts instead of the whole model.

And this makes latency really low and makes it really efficient. And the base models are often MOEs today for the frontier models. But what we were doing was thinking about, well, what if we froze your base model, your base pretrained model, and for post-training, we could do an MOE on top? And specifically, we could do an MOE on top through the adapters. So through your LoRA adapters. And so instead of just one LoRA adopter, you could have a mixture of these LoRA adopters. And they would effectively be able to learn multiple different tasks on top of your base model such that you would be able to keep your base model completely frozen and be able to, automatically in a learned way, switch between these adapters.

19.12
So the user experience or developer experience is similar to supervised fine-tuning: I will need labeled datasets for this one, another set of labeled datasets for this one, and so on. 

19.29
So actually, yeah. Similar to supervised fine-tuning, you would just have. . . Well, you could put it into one giant dataset, and it would learn how to figure out which adapters to allocate it to. So let’s say you had 256 adapters or 1024 adapters. It would learn what the optimal routing is. 

19.47
And then you folks tried to explain this in the context of neural plasticity, as I recall.

19.55
Did we? I don’t know. . .

19.58
The idea being that, because of this approach, your model can be much more dynamic. 

20.08
Yeah. I do think there’s a difference between inference, so just going forwards in the model, versus being able to go backwards in some way, whether that be through the entire model or through adapters, but in some way being able to learn something through backprop.

So I do think there is a pretty fundamental difference between those two types of ways to engage with a model. And arguably at inference time, your weights are frozen, so the model’s “brain” is completely frozen, right? And so you can’t really heavily adapt anything towards a different objective. It’s frozen. So being able to continually modify what the model’s objective and thinking and steering and behavior is, I think it’s valuable now. 

20.54
I think there are more approaches to this today, but from a user experience perspective, some people have found it easier to just load a lot of things into the context. And I think there’s. . . I’ve actually recently had this debate with a few people around whether in-context learning truly is somewhere in between just frozen inference forwards and backprop. Obviously it’s not doing backprop directly, but there are ways to mimic certain things. But maybe that is what we’re doing as a human throughout the day. And then I will backprop at night when I’m sleeping. 

So I think people are playing with these ideas and trying to understand what’s going on with the model. I don’t think it’s definitive yet. But we do see some properties, when just playing with the input prompt. But there I think, needless to say, there are 100% fundamental differences when you are able to backprop into the weights.

21.49
So maybe for our listeners, briefly define in-context learning. 

21.55
Oh, yeah. Sorry. So in-context learning is a deceptive term because the word “learning” doesn’t actually. . . Backprop doesn’t happen. All it is is actually putting examples into the prompt of the model and you just run inference. But given that prompt, the model seems to learn from those examples and be able to be nudged by those examples to a different answer.

22.17
By the way, now we have frameworks like DSPy, which comes with tools like GEPA which can optimize your prompts. I know a few years ago, you folks were telling people [that] prompting your way through a problem is not the right approach. But now we have more principled ways, Sharon, of developing the right prompts? So how do tools like that impact post-training? 

22.51
Oh, yeah. Tools like that impact post-training, because you can teach the model in post-training to use those tools more effectively. Especially if they help with optimizing the prompt and optimizing the understanding of what someone is putting into the model.

For example, let me just give a contrast of how far we’ve gotten. So post-training makes the model more resilient to different prompts and be able to handle different types of prompts and to be able to get the intention from the user. So as an extreme example, before ChatGPT, when I was using GPT-3 back in 2020, if I literally put a space by accident at the end of my prompt—like when I said, “How are you?” but I accidentally pressed Space and then Enter, the model completely freaked out. And that’s because of the way things were tokenized, and that just would mess things up. But there are a lot of different weird sensitivities in the model such that it would just completely freak out, and by freak out I mean it would just repeat the same thing over and over, or just go off the rails about something completely irrelevant.

And so that’s what the state of things were, and the model was not post-trained to. . . Well, it wasn’t quite post-trained then, but it also wasn’t generally post-trained to be resilient to any type of prompt, versus now today, I don’t know about you, but the way I code is I just highlight something and just put a question mark into the prompt.

I’m so lazy, or like just put the error in and it’s able to handle it—understand that you’re trying to fix this error because why else would you be talking to it. And so it’s just much more resilient today to different things in the prompt. 

24.26
Remember Google “Did you mean this?” It’s kind of an extreme version of that, where you type something completely misspelled into Google, and it’s able to kind of figure out what you actually meant and give you the results.

It’s the same thing, even more extreme, like super Google, so to speak. But, yeah, it’s resilient to that prompt. But that has to be done through post-training—that is happening in post-training for a lot of these models. It’s showing the model, hey, for these possible inputs that are just gross and messed up, you can still give the user a really well-defined output and understand their intention.

25.05
So the hot thing today, of course, is agents. And agents now, people are using things like tool calling, right? So MCP servers. . . You’re not as dependent on this monolithic model to solve everything for you. So you can just use a model to orchestrate a bunch of little specialized specialist agents.

So do I still need post-training? 

25.39
Oh, absolutely. You use post-training to get the agent to actually work. 

25.43
So get the agent to pull all the right tools. . . 

25.46
Yeah, actually, a huge reason why hallucinations have been, like, much better than before is because now, under the hood, they’ve taught the model to maybe use a calculator tool instead of just output, you know, math on your own, or be able to use the search API instead of make things up from your pretraining data.

So this tool calling is really, really effective, but you do need to teach the model to use it effectively. And I actually think what’s interesting. . . So MCPs have managed to create a great intermediary layer to help models be able to call different things, use different types of tools with a consistent interface. However, I have found that due to probably a little bit lack of post-training on MCPs, or not as much as, say, a Python API, if you have a Python function declaration or a Python API, that’s actually the models actually tend to do empirically, at least for me, better on it because models have seen so many more examples of that. So that’s an example of, oh, actually in post-training I did see more of that than MCPs.

26.52
So weirdly, it’s better using Python APIs for your same tool than an MCP of your own tool, empirically today. And so I think it really depends on what it’s been post-trained on. And understanding that post-training process and also what goes into that will help you understand why these differences occur. And also why we need some of these tools to help us, because it’s a little bit chicken-egg, but like the model is capable of certain things, calling different tools, etc. But having an MCP layer is a way to help everyone organize around a single interface such that we can then do post-training on these models such that they can then do well on it.

I don’t know if that makes sense, but yeah, that’s why it’s so important. 

27.41
Yeah, yeah. In the areas I’m interested in, which I mean, the data engineering, DevOps kind of applications, it seems like there’s new tools like Dex, open source tools, which allow you to kind of save pipelines or playbooks that work so that you don’t constantly have to reinvent the wheel, you know, just because basically, that’s how these things function anyway, right? So someone gets something to work and then everyone kind of benefits from that. But then if you’re constantly starting from scratch, and you prompt and then the agent has to relearn everything from scratch when it turns out there’s already a known way to do this problem, it’s just not efficient, right? 

28.30
Oh, I also think another exciting frontier that’s kind of in the zeitgeist of today is, you know, given Moltbook or OpenClaw stuff, multi-agent has been talked about much more. And that’s also through post-training for the model, to launch subagents and to be able to interface with other agents effectively. These are all types of behavior that we have to teach the model to be able to handle. It’s able to do a lot of this out of the box, just like GPT-3 was able to chat with you if you give it the right nudging prompts, etc., but ChatGPT is so much better at chatting with you.

So it’s the same thing. Like now people are, you know, adding to their post-training mix this multi-agent workflow or subagent workflow. And that’s really, really important for these models to be effective at being able to do that. To be both the main agent, the unified agent at the top, but also to be the subagent to be able to launch its own subagents as well.

29.26
Another trend recently is the emergence of these multimodal models or even, people are starting to talk about world models. I know those are early, but I think even just in the area of multimodality, visual language models, and so forth, what is the state of post-training outside of just LLMs? Just different kinds of this much more multimodal foundation models? Are people doing the post-training in those frontier models as well?

30.04
Oh, absolutely. I actually think one really fun one—I guess this is largely a language model, but they are likely tokenizing very differently—are people who are looking at, for example, life sciences and post-training foundation models for that.

So there you would want to adapt the tokenizer, because you wanted to be able to put different types of tokens in and tokens out, and have the model be very efficient at that. And so you’re doing that during post-training, of course, to be able to teach that new tokenizer. But you’re also thinking about what other feedback loops you can do.

So people are automating things like, I don’t know, the pipetting and testing out the different, you know, molecules, mixing them together and being able to get a result from that. And then, you know, using that as a reward signal back into the model. So that’s a really powerful other type of domain that’s maybe adjacent to how we think about language models, but tokenized differently, and has found an interesting niche where we can get nice, verifiable rewards back into the model that is pretty different from how we think about, for example, coding or math, or even general human preferences. It’s touching the real world or physical world—so it’s probably all real, but the physical world a little bit more.

31.25
So in closing, let’s get your very quick takes on a few of these AI hot topics. First one, reinforcement learning. When will it become mainstream? 

31.38
Mainstream? How is it not mainstream? 

31.40
No, no, I mean, for regular enterprises to be able to do it themselves. 

31.47
This year. People have got to be sprinting. Come on. 

31.50
You think? Do you think there will be tools out there so that I don’t need in-house talent in RL to do it myself?

31.59
Yes. Yeah. 

32.01
Secondly, scaling. Is scaling still the way to go? The frontier labs seem to think so. They think that bigger is better. So are you hearing anything in the research frontiers that tell you, hey, maybe there’s alternatives to just pure scaling?

32.20
I still believe in scaling. I believe we’ve not met a limit yet. Not seen a plateau yet. I think the thing people need to recognize is that it’s always been a “10X compute for 2X intelligence” type of curve. So it’s not exactly like 10X-10X. But yeah, I still believe in scaling, and we haven’t really seen an empirical plateau on that yet.

That being said, I’m really excited about people who challenge it. Because I think it would be really amazing if we could challenge it and get a huge amount of intelligence with less pure dollars, especially now as we start to hit up on trillions of dollars in some of the frontier labs, of like that’s the next level of scale that they’ll be seeing. However, at a compute company, I’m okay with this purchase. Come spend trillions! [laughs]

33.13
By the way, with respect to scaling, so you think the models we have now, even if you stop progress, there’s a lot of adaptation that enterprises can do. And there’s a lot of benefits from the models we already have today?

33.30
Correct. Yes. We’re not even scratching the surface, I think. 

33.34
The third topic I wanted to pick your brain quick is “open”: open source, open weights, whatever. So, there’s still a gap, I think. 

33.49
There are contenders in the US who want to be an open source DeepSeek competitor but American, to make it more amenable when selling into. . .

34.02
They don’t exist, right? I mean, there’s Allen. 

34.06
Oh, like Ai2 for Olmo… Their startup’s doing some stuff. I don’t know if they’ve announced things yet, but yeah hopefully we’ll hear from them soon. 

34.15
Yeah yeah yeah. 

Another interesting thing about these Chinese AI teams is obviously, you have the big companies like Tencent, Baidu, Alibaba—so they’re doing their thing. But then there’s this wave of startups. Set aside DeepSeek. So the other startups in this space, it seems like they’re targeting the West as well, right? Because basically it’s hard to monetize in China, because people tend not to pay, especially the enterprises. [laughs]

I’m just noticing a lot of them are incorporating in Singapore and then trying to build solutions for outside of China. 

35.00
Well, the TAM is quite large here, so. . . It’s quite large in both places. 

35.07
So it’s the final question. So we’ve talked about post-training. We talked about the benefits, but we also talked about the challenges. And as far as I can tell, one of the challenges is, as you pointed out, to do it end to end requires a bit of expertise. First of all, think about just the data. You might need the right data platform or data infrastructure to prep your data to do whatever it is that you’re doing for post-training. And then you get into RL. 

So what are some of the key foundational things that enterprises should invest in to set themselves up for post-training—to get really good at post training? So I mentioned a data platform, maybe invest in the data. What else?

36.01
I think the type of data platform matters. I’m not sure if I totally am bought into how CIOs are approaching it today. I think what matters at that infrastructure layer is actually making sure you deeply understand what tasks you want these models to do. And not only that, but then codifying it in some way—whether that be inputs and outputs and, you know, desired outputs, whether that be a way to grade outputs, whether that be the right environment to have the agent in. Being able to articulate that is extremely powerful and I think is the one of the key ways of getting that task that you want this agent to do, for example, to be actually inside of the model. Whether it’s you doing post-training or someone else doing post-training, no matter what, if you build that, that will be something that gives a high ROI, because anyone will be able to take that and be able to embed it and you’ll be able to get that capability faster than anyone else. 

37.03
And on the hardware side, one interesting thing that comes out of this discussion is if RL truly becomes mainstream, then you need to have a healthy mix of CPUs and GPUs as well. 

37.17
That’s right. And you know, AMD makes both. . .

37.25
It’s great at both of those.

And with that thank you, Sharon.

Read More

Generative AI in the Real World, Podcast

Radar