Zohra: 0:02

Hello friends and welcome to season eight of Inside Tech Comm with Zohra Mutabanna. This season features a collection of conversations that explore the ideas, challenges, and opportunities shaping technical today. From AI and content strategy to leadership, product thinking, and the future of our profession. Each episode offers a fresh perspective from people doing the work. Let's get started. Season eight is proudly sponsored by LavaCon. Use discount code ITC26 to save$200 on registration for this year's conference. I'll share more about LavaCon later in this episode. Hello listeners. Welcome to another episode of Inside Tech Comm. Today we'll be talking with Patrick Bosek from Heretto. And in the past, we've talked about structured content and its impact on how AI ingests that framework. Today we're going to do a deep dive. Patrick is going to be talking to us and educating us on how structured content contributes useful context under the hood and how it works. With that, Patrick, welcome to my show.

Patrick: 1:17

Great to be here, Zohra. It's very nice.

Zohra: 1:19

Patrick, first tell us a little about yourself, what you do, and yourself.

Patrick: 1:24

Sure, happy to. So I'm Patrick Bosek. I am one of the founders of Heretto. I'm also the CEO. So I've been in the structured content industry for quite a while. As the uh some I guess I could be considered the face of the company sometimes. So in that role, I also do quite a bit of media things. So I have a talk show I do with Scott Abel. So that's called Coffee and Content. We've been on a little bit of a hiatus due to a technical issue, but we typically do Coffee and Content every other week. So those are a lot of fun, cover a lot of different topics from the content world. We also have a podcast called Content Components. If you look it up on Spotify, that's got four or five seasons. It's all about documentation, structured content, those kinds of things. So if you're wanting nerd out about structured content, that's a great place to go. Yeah. And then I'm a developer by background. So I was one of the developers and architects of the first version of our software. So I'm I'm fairly technical, but I don't cause problems for our development team anymore. Nowadays, I focus on what's going to make customers happy and those kinds of things.

Zohra: 2:31

What spawned this idea to look into structured content as a developer? I'm curious.

Patrick: 2:37

Oh boy. So that's actually going back a lot further than I'd like to admit. But my co-founder and I, we got into structured content, not DITA, which is where we are today, but XML content with an actually with a custom schema when we were in college, actually, and we just saw that it was a superior way to store and manage content and that it was going to make some of the publishing activities we were working on easier and more capable. At that point in time, the technologies surrounding digital were just far, far less well developed than they are today. So structured content provided it actually provided something quite different at that moment in time, now that I'm reflecting on it, than it does today. At that moment in time, it it basically gave you the ability to put, like, a little bit of a database in a file. The amount of things you had to manage and the flexibility you got when you were using structured content at that point in time, if you're looking to apply that to like a very structured digital publishing environment, there was a lot of advantages there. Um so that was why we picked it up originally. And then over the years, we eventually drifted into, or I guess found our way, is maybe a better way of putting it, into technical content where you still get a lot of those benefits that we we were originally looking at with DITA and in the digital publishing around uh technical content and documentation, but there's a ton more today. So there's there's so much more that you get from it.

Zohra: 4:05

Yeah, we could probably have a whole episode on structured content. And I have, as I mentioned, had some episodes covering what that is. And it seems to me like the context can be different. Of course, DITA comes close to mind, but for those of us that don't use DITA, structured content still follows a certain framework, a certain guidelines, if you will. But for this conversation, I want to ground what that definition means to you, and then we can, I guess, start talking about how we can deep dive into how that lends to AI.

Patrick: 4:42

Yeah, I should think that is really important to start with the definition of structured content because everybody does define it a little bit differently. And I think the individual definition matters. So I'll give you my definition. I'll also tell you why I think mine is right. But that doesn't mean that other people's don't uh can't be right too. So technically speaking, if you put all of your content into Excel or Google Sheets and you put it into, you know, let's say every topic was a column, or maybe better a row, and every portion of that topic or that piece of information was divided up into different columns. So you have the pieces broken up into the different cells. That would be structured content because you would have the content broken up into a structure. Like in the most basic definition, you have a structure for your content. So you're you're labeling the different pieces of it in a methodology, and that's structured content. Now, obviously, that's a terrible choice. Like that wouldn't be a thing that you'd want to do in almost any production environment. Although I will say that we have had customers migrate from that exact environment just like that to our tool, which is DITA. But when you think about like how this comes together, the reason I start there is because there are a lot of systems that are they're effectively that, because if you think of what a form is, so like a digital form, it is just actually a table. You have your question one, question two. So your question one might be what is the heading for this file? Your question two might be who authored this file, question three might be the body. And people will call that structured content because, you know, you can put metadata in one place. And and in certain circumstances, that's what headless CMSs are, for instance. So, like things like Contentful, those kinds, that's that's what they are. In certain circumstances, that actually works quite well. Um, that's a really effective form of structured content. It's backed by a database, uh, it has the ability to create links between those pieces of information, uh, it has the metadata separated out. Like you have like there are certain things, but it in the background, it is functionally a table that is fundamentally different than what we work with, which is XML, because XML is a tree, which means that you can have structure below those big boxes. So when you go into a topic, there's going to be structure at multiple levels. So you'll have a piece, like let's say we'll use a procedure or a task topic, for instance. When you start creating that, you'll have a title, and then you'll have metadata around it, and then you'll have the prerequisites for running this procedure. And then the individual procedure, instead of being an ordered list, it would be steps, right? And then there's a step inside of it, there's structure inside of the steps. And so by having this multiple layers of structure and semantics, it means that you're able to manipulate that in a more granular way beyond just formatting. It's not just HTML and boxes, effectively, but they're both structured. Like there is a, like, both things are valid in terms of the content label. So, what you really need to ask yourself when you're thinking about, okay, well, we're moving to structure, it's beyond just the term structure. Like you should really be thinking about what is the form of that's going to serve our mission? What do we care about in terms of interoperability? What forms of linking are really important to us? What forms of output are going to be really important to us? Because those types of questions will guide you towards the type of structure that's going to be best for your particular implementation and your particular needs.

Zohra: 8:19

I have to say that I thought my definition was going to be very different from yours or my understanding. But at some level, I think we are on the same page.

Patrick: 8:28

Yeah. I mean, I think my definition is pretty broad. Like I said, I think it's right too.

Zohra: 8:32

Yes, it's very inclusive. So that allows me to be kind of okay, that definition sounds close to what I have in my mind. Fantastic. Yes. I have not worked with DITA, and you mentioned DITA, and DITA comes up a lot when we talk about structured content. We've touched upon this before, but I think for the benefit of the audience, it would be nice to quickly do a shout out to what DITA is.

Patrick: 8:58

Yeah, for sure. So DITA, in the most basic definition, it's it's a form of XML. So in that way, you can actually think of it as like a cousin to HTML in some ways. So it actually shares a lot of the same elements that HTML has. The thing to know about DITA is that it has built in many of the patterns that you need to scale up technical content. So there's a ton of stuff that we've pretty much figured out over the last call it 15 or 20 years, are really critical to larger or more complex sets of technical content. And the DITA committee, so the people who work on the open source standard, they put all of those things into the standard. So you don't have to recreate them. And then perhaps just as importantly, beyond not having to recreate them, you don't have any vendor opinion or lock-in. So one of the challenges that the that DITA was trying to solve, and I think has solved pretty effectively, is that when you have vendor-specific formats, it means that okay, well, we've reused content, but we've reused content in this particular system. And once you get to a certain level of reuse, if you didn't have a portable technology underneath it, your ability to migrate someplace else is I mean, it's it's almost impossible. It just becomes the lock in there, it's just so high. So what DITA really solves is it solves having good patterns for the core things that you really need to do with technical content, and then it also solves an interoperability problem. And you know, there's some of the criticisms of DITA are that it comes with you know like a high learning curve. Yeah, I don't know if I actually agree with that. Like I've seen a lot of people learn it pretty quickly. There's people feel that it's too heavy for today's uses. That's another criticism. Those people tend to come from like the Markdown world. So, like in comparison to Markdown, you know, Markdown is a text format, DITA is an is an XML format. I think that's just a use case thing. So like DITA is not a great, is not like you never write the README files for your vibe coded project in DITA. I don't do that. Like when I have small apps and things like that that I build, I don't write any of the documentation in DITA. It doesn't make any sense. DITA is a format that you use when you arrive at a sufficient level of complexity and a sufficient scale. So if you want like really rough and tough numbers, what we've found over the years is that if you're below a thousand pages of content, DITA probably doesn't make sense. Um, there are some exceptions, but even at a thousand pages of content, like it can be questionable there depending on your And so where it really starts to make sense is when you have multiple variations of products or variations of content, audiences. So you have anything that's like personalization, and there's a bunch of different terms that people use for the content changes when somebody looks at it for a different person looks at it for a different reason. Localization can be a driver, although I think less so today. And then just sheer scale with like reuse. So if you have a lot of content that you is used in multiple places, that's when you know DITA starts to really shine.

Zohra: 12:11

In a simplified way. Since we are going to do a deep dive into how structured content lends itself to AI, does DITA lend itself better, or just this the generic definition of the structured content that we ground it, that we are trying to ground this conversation in also lends itself to better AI interactions?

Patrick: 12:31

That is such a good question. So the answer is yes, it depends in here. And I feel like I'm among the right people to give like engineering style, it depends answers. One of the jokes in our leadership team is that when you ask the business people a question, they give you an answer. When you ask the technical people a question, they say it depends. And so I think that I'm I'm filling that role in this particular moment. So, you know, this is kind of like what we we wanted to cover today. So it is a it's a very, very long answer. At the very core of it, if, like, you wanted, like, the too long, didn't listen version of it, structured content is going to give you better structured contents with the broad definition. So like anything that is broken up into labeled pieces that are then stored consistently, like that definition of structured content. So that's inclusive of DITA, inclusive of, uh, headless CMSs, inclusive of anything that has a well-defined schema applied to content. So, why would that be better for AI? And it really just comes down to chunking, like that's the fundamental difference between those things. So the problem with that statement is that when you get to certain forms of structured content, and I'll like I'll give you like an example from a headless CMS. The headless CMS is every headless CMS is gonna have a different schema. Like part of the the appeal of a headless CMS is that you can create your own schema. So you go in and you get a blanks blank page, and all you have to have is a title, and then you can add a body and an author, and you could add you know links to other things. And like the standard thing is like, you know, how would you lay out a one of the standard examples is how would you lay that out for like a music venue? And you'd have one thing that was a band, and then you'd have an artist, and then you'd have like those kinds of things, right? And you'd you build those schemas. Well, in a really well-designed schema, it's gonna function really well as giving you good sized pieces of content for the chunking. In a poorly designed schema, what's gonna happen is you're gonna get people basically putting in a title and a body, and like that's really not any different than a Google Doc, right? So at that point in time, people are just gonna throw huge quantities of content into that body field. So even though technically you have two fields, the structure is two boxes, like it's one box, it's it's two boxes with unstructured content in one of them. And so then you lose the benefits of chunking. So now there's a bunch of other things, but if you like if you want the biggest bang for your buck, so like the I don't know, the 10% that gives you 60% outcome, it's having really well, clean, defined borders around your content, which means smaller pieces, more intentional pieces, pieces of content that contain ideally one piece of information. So in DITA, like a topic is supposed to be about one thing. It like if it's a concept topic, it's supposed to teach you one thing or explain one thing. If it's a task, it's supposed to give you a procedure that does one thing. Um, so it takes you from from the beginning to end of one thing you have to complete, and there's other versions of that as well. So in the AI world, when you go to produce results with an AI system, being able to provide it very clean chunking, so very clean set like pieces of content for the classification on when you're putting it into a RAG system, and then also for retrieval, there's a bunch of other things, but that's the thing that gets you the biggest bang for your buck.

Zohra: 16:05

Modern language models are pretty good at extracting meaning from unstructured content. Does that reduce the need for structured content? Because we're trying to make the case that you need to have chunking with structured content. But I'm kind of trying to flip that question and see if the LLMs have gotten so bad so good, or is there still an advantage to use structured content as we have established?

Patrick: 16:33

Yeah, so I think that is also a great question. It's one of the places that there's I don't want to say because I think that's the wrong way to think about it. But I think that there's so maybe this is a better way to put it. You're right that the the modern LLMs have gotten very good at extracting information when the question is basic. And the reality is that most of the time when people are interacting with these LLMs, their their modalities of interaction fall into a couple of categories. One is coding, and coding is a verifiable tree. So like part of the reason coding has gotten so good is not because the LLMs have gotten so good, it's because the harnesses have gotten so good at managing something that is verifiable, like that. It's logically verifiable. The other modality that we use, we use these in is effectively a replacement for search, right? So you go to Google and like you look at the AI result. Again, a lot of the advancement there is actually in the deterministic harness under the hood. Like the amount of engineering that Google and Anthropic and Chat or and OpenAI have done so that what is being delivered to the AI for consideration is high quality, is probably equivalent to the amount of like engineering they've done in the AI model. Well, maybe not, I don't know, but like it's a lot is the point. And then when you're but when you're thinking about interacting with these systems in a more general sense, typically people are putting very small quantities of information into them. So like if I go to NotebookLM and I throw in a couple of PDFs, it does work quite well because the quantity of information is very small. Now, in the background, it is still chopping that PDF up and it is still indexing it. Like it is not just feeding that PDF full stock to the to the LLM. But what's happening when you ask that the question, or you're working with the system to get information out of that, is that because it's a small number of chunks, and because the system is then reducing down what it's sending to the LLM to a smaller set, it's not that big of a deal. The problem is when you scale up. So there's a reason you can only put, I don't know what it is, it's not a huge quantity of content into NotebookLM. When you start talking about thousands of pages or tens of thousands of pages or more, at that point in time, especially if you're asking more human questions as opposed to like search queries, that's when it absolutely still breaks down. And like we have research that shows this. So like this isn't this isn't just me hypothesizing these things. Like this is this is stuff that we can show. And in those worlds where having really strong RAG retrieval, which is built on the foundation of having good tokenization of the content, you start to see structured content outperform unstructured content by about 50%. Now it's use case dependent. Like there are like simpler questions will do better in both cases, more complex questions will do better. So there's there's a lot of it depends in there. But for things that really matter, especially in like regulated industries, or I mean, I guess everybody should care about their customers getting the right answer, but you know, like some people care more than others, I suppose. It still is a substantial difference in overall quality of retrieval and accuracy.

Zohra: 20:03

So I think this is a great, I guess, segue into the RAG, uh, the retrieve and generate model that we start looking into. Under the hood, retrieval is evolving from a fairly linear model towards more agents that can decide what information they need to go and find. Where does structured content fit into this newer model of agentic retrieval and context engineering?

Patrick: 20:25

Yeah, I so I think that's an interesting area of evolution right now. And I guess the first thing I want to call out here is that the models are themselves getting better. Like there's no doubt about that. But I actually think the advancement in RAG systems is more in the harness sophistication than it actually is in the models right now. So when you think about like how does an AI work with a a RAG like content operations or RAG architecture such that it's going to produce a better outcome. Well, actually, should we stop and like give a definition of what of like what RAG is before?

Zohra: 21:00

Yes, please. Absolutely. Go for it.

Patrick: 21:03

Yeah, okay. So RAG is retrieval augmented generation, which is technical people trying to confuse the world with with fancy words. All it means is that you're running a deterministic on the well, you're running a search. I was about to say deterministic search, but that's not necessarily true. You're running a search, the output of that search is then being put into the context for an AI agent to generate a result. That's all it means. So it's it's just search and then generate, is what it comes down to. Now, most RAG systems are built on vector databases. And why it matters to call that out is because fundamentally, vector databases, what they are is they're just they're calculations around a piece of information as to like kind of where it sits in space, and they what they do is they retrieve things that are close together. Now, those calculations are done by LLMs typically. When you look at like some of the research that we've done, like When you look at, like, some of the research that we've done, like of different ways of calculating the embeddings. So the embeddings are like the position, I guess, if you want to think about it that way. And the way that we we do that is that you can calculate the embeddings with a bunch of different with any of the AI models that are out there. So what we've found is that in our testing, OpenAI tends to be the one that's most cost effective and easiest to do. And so that's the one that we've used. We've tried a bunch, but that's the one we've kind of settled on as like our baseline, which is funny because we use Anthropic everywhere else. But like that's how that comes together. So then you have the content in the vector database, the way you retrieve it is through these vector searches, and then when you retrieve sets of content, that's going to be put into the LLM. So the major difference here between like the first versions of these and the versions that we deal with today, the models are better, yes. But now you may see AI being in a place where you send the question to the LLM, the LLM modifies the question or it breaks it down into some way, or it clarifies the question, changes the question, and then it passes the query to the vector database. The vector database returns results, and that loop can continue. So reasoning is largely just a loop, right? Like they basically feed the results back in. There could be some deterministic logic that does some parsing or extraction between the LLM and the database. Like there's other ways that they, you know, they can basically upgrade the overall loop, but it really fundamentally is a loop. So I think the misconception that a lot of people have is they think all of this is happening inside of the LLM. Like the LLM is making a decision about what information it's retrieving, but like the information is in the LLM. The information that you're when you're talking about RAG is not in the LLM. It's in a separate piece of technology that the LLM is making, is having an interaction with?

Zohra: 23:55

When I started working with structured content, metadata became a centerpiece to how a cornerstone to how search was done. So the the richer the metadata, the better the output was. So organizing information was super critical. What changes when a machine is using that metadata to decide what information belongs in an AI model's context with all the technology breakdown that you've given us?

Patrick: 24:24

Yeah. So this could be a whole series of podcasts in and of itself.

Zohra: 24:30

Of course.

Patrick: 24:30

What changes? Well, oh boy. The first thing I think that's important to think about here is that metadata gives you a lot of options in how you process something. So one of the key aspects of having really high quality metadata on your content is it has hooks and, like, levers that you can pull in deterministic processes that decide how content ends up and how it shows up. Right. So when you're thinking about what you're gonna end up providing to an LLM, you almost always have some type of a pipeline that's going to be coming into whatever your RAG system is or whatever, wherever the LLM is actually going to be touching that content. And metadata means that you can be more specific about what and how those things go into the uh the LLM or the AI-based system. So it gives you a lot more control with traditional code. And I'm emphasizing that because it really shouldn't be undervalued. That is still enormously powerful. The more specific you can be in a deterministic way in the systems that you build, the more confidence you can have and the more sustainable those systems are going to be, because you can write unit tests, right? Like things in deterministic systems have deterministic outcomes. Like that's fundamentally how they're different than AI systems. So metadata still has a ton of value in that way. This is kind of a the other side of that coin, but I think also valuable. There are circumstances where traditional search, if metadata is effectively applied, will perform equally well to vector search, and it's a lot less expensive. So there's certain applications where the old stuff works just as well. And there's actually cases where it works better, but then you know, we don't have to go, we don't have to dig too deeply into those things. But the point is that metadata also gives you options. So if you're in a place where you're completely reliant on AI systems and you don't have really strong metadata around how you're trying to build a retrieval pipeline, um, then you're totally reliant on the inference of context from the AI into a vector database. Whereas if you had that metadata, you might be in a place where you could have a deterministic system or a traditional search that was just as just as good or better, um, and also gave you some other options. So metadata can be really valuable in terms of comparison of not just an optionality for your system. So you have optionality, you have comparison, like you have those types of things. And then when you think about like how does it actually benefit the LLMs themselves, this is where you get into like a very interesting discussion because there are aspects of it that can be really Typically, what it means is that you're transitioning structured metadata into, I would say, like statements for the LLM, such that it is still context, right? But like LLMs receive context differently than like humans and traditional machines to some extent. But if you have everything tagged for, so I'll give you an example. Let's say you have two products. You have a we'll use washing machines, right? Let's say you have a 1000 version and a 1001 version, and those are very, very similar systems, right? They're the same line of washing machine. There's a few differences. Well, you've got a hundred topics for each of those systems, and let's say 95 of them are the same. So they all carry the 1000 and the 1001 tag, and then you have five that diverge. So you have 105 topics in total, and you know, five with a thousand, five with a thousand and one. If you're just stuffing all of that into an LLM, and remember, like this is only 105 topics, so you can probably get away with that. The way you would use metadata there, and notice I'm specifically avoiding RAG and search in this particular example. Like I'm just talking about how you put the whole 105 in, is instead of having a tag on that, you would probably be better off actually having a statement at the top of it where before you put it in, you'd use a deterministic system to take the tag from you know product and then it had the tag name to this is the 1001 version of this washing machine. This is the 1000 version of this washing machine. You just put that at the top of every topic. And an LLM is going to interpret that better than having like the actual tag on there. And so I've done a bunch of testing around this. Like, this also is the same with like glossary terms. So if you have like structured glossaries, um, different LLMs will receive the pieces of that glossary, which are tags, right? Like it's metadata. Like you're like, you know, this is an abbreviation, this is another way of saying it. Like, there's like right, it that's just metadata that describes different pieces of it. If you write it out as prose, even if it's the same prose over and over again, the LLMs tend to work with that version of that content better. So transitioning the structured metadata into clear and simple statements, even if they're repetitive for the LLM, tends to produce a better result today with the LLMs we have today, because of how they've been trained and you know the biases and like the internal weightings that they have. Like you could train one that works better with the actual structure. That's just not the ones that we have today. So metadata touches a ton of different pieces of it. And I haven't even gone into like how metadata can then be used to build graphs and to create like more sophisticated like selection mechanisms. But the point is that once you get to any level of complexity, having the metadata on the content gives you so much more optionality in terms of how you build the systems around the LLM, and then also how you actually instruct the LLM when you're putting inputs into it.

Zohra: 30:31

I think this was such a great overview because my goal here was to kind of step back. And for those amongst us who have very, I guess the as a technical writer, one would have that structured content, the concept But if you don't, if you just have a Google Doc and you're starting off, then adding metadata to that, you've given us a why, a solid why. Why should you still consider it? And how that would kind of work under the hood that I keep going back to. And I would love for us to maybe even dive into the knowledge graphs and how they would relate to metadata. With more than 70 sessions and workshops featuring speakers from companies like Salesforce, TikTok, T-Mobile, and Amazon. LavaCon brings together content strategists, documentation leaders, and content professionals to share practical ideas you can apply right away. And yes, it's known as the Fun Conference, too with networking events, therapy dogs, comfort llamas, storytelling, and karaoke. Use discount code ITC26 to save$200 on registration. You did mention graphs here. So I think it's worth going into that a little bit, if not more.

Patrick: 31:53

Yeah, sure. So I'm happy to go. Well, there are people who are are experts in knowledge graph. I wouldn't consider myself an expert in knowledge graphs. I do know I I would say a pretty good amount about them. So there's probably a limit to how far I can go, but I'm very happy to talk about them in relation to structured content metadata. So maybe uh start with just the definition of what a knowledge graph is. So so a knowledge graph is if you want to think about it in technical terms, it's a database that has typed nodes and typed connections between those nodes. So you could have people and then you could have, like, a 'knows' or like this person knows this person or is married to. Obviously, that would be a one-to-one connection in most cases. But like, you know, so this is like the friend of a friend stuff. Like, so like ontologies, knowledge graphs, those kinds of things. When you put this information into graph databases, what it means is you can start asking multi-level questions of the database. So the thing that people really like about them is if you go back to like the people example, you could ask a question of a knowledge knowledge graph of like, give me all the companies for people that Patrick has been on podcasts with. And a good knowledge graph, like a good knowledge graph database would be able to go and say, Okay, well, Patrick is a podcaster, and then there would be a field that's like has been on, or something like that, and that would link out to a bunch of different podcasts, and then each of those podcasts would have guests and hosts, and each of those guests and hosts would be people, and all the people would have as employment or you know, works for something along those lines, right? And so you could have a query that would say like doop, doop, doop, doop, doop, and hop across all those things and select all of those companies. That's why people like knowledge graphs, is because it allows you to ask those kinds of questions of an information set, which are traditionally very difficult to ask in the context of like relational databases. So you can do some of that stuff in relational databases, but it's much less flexible, I guess is the best way to put it. So you're dealing with very large and you know well tagged or very semantic sets of content, like you're just gonna do better in a knowledge graph. So, how this collides with structured content and metadata is that it's not always that you're typically you're not actually keep maintaining your content in the knowledge graph, right? So you're maintaining your content in another location. So the way that you you correlate, you tie the content to the knowledge graph is through the metadata in many cases. So the metadata that becomes the selection mechanism that the the knowledge graph can use to pull that out. You can also use direct linking, but you oftentimes like the metadata is going to become like a major piece of like how those two things come together, or how the knowledge graph forms the nodes in the graph out of the content, like it contributes to those things.

Zohra: 34:52

I'm thinking in a typical environment, you have documentation coming from product, from engineering, from support. How should the infrastructure establish authority and provenance and a source of truth within the context of that uh of structured content? Because that content may be structured differently. What's your take on that?

Patrick: 35:17

Yeah, so I think this is a really timely question. So the the environment you've described, you know, very briefly, where like technical content comes from support, product engineering, um, the technical writers, like that is one version of companies where the the technical content is kind of like the result of all these contributors. That's how you get to those corpuses of content. The other structure of companies is where you have the technical writers is more of like a clearinghouse, like they're the ones that manage most of the content, and they probably write a large portion of it. And in those environments, you do tend to have knowledge bases that grow up around that, but there's a clear, very clear delineation between like the docs are gold plated, the knowledge bases can be a source, it can be leveraged, but it's not like the level of governance and quality that's expected is is very, very different. And I think it's very situational as to which which is going to work for any one individual company. You know, some companies can work really well in the more almost crowdsourced version of documentation, and then other companies really can't go that route. Like they have to have technical writers as the authorities on truth, more or less, like is the way to think about it. Um, and that's like a that's like um a center of excellence approach. I have a lot more experience in that approach because our technology lends itself to that approach. Like if you have the highly like crowdsourced collaborative version of it, like that's just not where we fit. So I don't work in that world as much. When you're trying to establish authority and governance across multiple groups like that, so again, I guess there's kind of two situations there. One is they're all using the same thing, which in my experience there is almost always a wiki or Google Docs. Like I haven't seen very many variations on that. Um, and that's just because when you have a large group of people who are not writers writing the content, you have to give them a tool that's not a writer's tool. It's like it's super uncommon that you'll see a circumstance where you have engineering product support and technical, and like maybe you know, a couple of technical writers working in technical writing tools. Like I think I've maybe seen that once in my entire career doing this for like I'm not even gonna say how many years, but more than a decade. It's much more common that you'll see you hit you go to the lowest common denominator. Maybe that's that's not exactly what I mean. You go to the most available system, which is almost always a wiki. Wikis are not built for governance, like there's just no way that. Like wikis are built for availability, and governance is an that imposes structure, it needfully imposes roadblocks and workflow and things like that, because that's where quality comes from. It comes from people doing the same things, knowing the patterns, checking themselves in predictable ways, that kind of stuff. Like that's governance, you know. Like you look at any like Six Sigma or Six Sigma, right? Like it's it's a set it's a set of patterns and practices. You can't apply those to a group of people that are not professionals in patterns and practices around that particular environment. So it becomes very challenging, is what it comes down to. You can't really have like broad inputs and high governance. And if you don't have high governance in your content, the work of like reverse engineering authority and like source of truth becomes much more complicated. It's not impossible in all cases. So I can give you an example of this. So like we have technical writers here, obviously, like that would be crazy if we didn't. But like our product team doesn't do the product specifications in Heretto, which is you know, Heretto is a documentation system. Like we do our product specifications in Confluence, and we do that because like we like product specifications are the sorry, these are the the design specs, not the actual, like these are these are not the final specs. We don't do the design specs in Heretto. So the design specs, because like they are working spaces, right? Like you've got a bunch of people in there and like they're throwing stuff at the wall, they're pulling things around, like they're just they're kind of screwing around in the content to figure out what's going to work while they're going through the process, the design process. It's a very creative, free-flowing process of like getting together, and that's just not the right place to apply like high structure and high governance. Like, you need to give people the minimum amount of those things to give them the most creative freedom to get to the end design results they're looking for. Now, at the same time, the outcome of these initial design documents that they do, they tend to be very high quality. So they're totally inconsistent. Like there's no like like they're not, you know, they're not in the same format of the same structure, but like they have really, really good information in them because they are for the initial design of something, like they kind of are the single source of truth. So in our world, what we do there, this is before the thing is built, before the documentation is written, all those kinds of things, is you just lock it down. Like once it's done, you just reduce the number of people that can change it, and the people that can change it are much more mindful. Now, we also have the benefit of the people who are working on this, are people who are who have all of the benefits of knowing structured content and documentation because that's a tool that we work on. So, like inherently, they're gonna be better about some of these practices, and you do see that bleed into you know building just frankly, probably more information architecture, you know, kind of by osmosis in some of these documents. So, like we see a lot of content come from Confluence. It's very common that people move from Confluence to Heretto, and it tends it comes in a wide variety of of qualities and like like positions from the structure and like a governance perspective. So, but we use those as inputs to to different LLM systems that we have, those those Confluence pages. So before a feature is built and before the content is migrated into Heretto and it's put on our internal like our internal point of reference is a thing we have like an internal name for it, that is all designed and built in Heretto, and that is considered the highest order of truth in the organization. Like if it's there, it's true. But before, you know, in the design process, if people have questions about things, like they can ask an AI agent and it will source from those Confluence pages. And it works quite well because it's a very small set of content, it's maintained by a very small team, it has a very known position in the cycle of information. Like it was designed at this point in time by these people for this purpose. So like there is a set of metadata that's assumed about that content simply by the definition of what it is. Like there's all these attributes that come with it that you can build into a system to get good results from.

Zohra: 42:16

I was also thinking about lone writers and systems where they've not evolved. You gave them great examples of how to think about it. So if you don't have those systems in place, you can start small. Where can I start if I don't have it? How do I think about this where you don't have established processes or uh governance frameworks? How do you establish that so that you can get to the source of truth and locking it down and having the right people, get the right gatekeepers, so that eventually your content is gold-plated?

Patrick: 42:49

Um Yeah, I can actually give one other piece of AI-related advice to that. So if you're a lone writer, like, so if I was gonna go be a lone writer in an organization today, almost no chance that you have access to like true structured content. Like that's just not common for lone writers. So I think the most common circumstance for those individuals today is going to be that they're writing in Markdown. There's a bunch of other situations, but like that's probably the most common. So I'll focus on that one. One of the things that I would do after I wrote a very basic style guide, I'm not even gonna call it an information architecture at that point in time, but a set of rules and patterns around how it is that we created content, is I would feed that into an agent and I would say, you're now going to be my editor, and you're gonna hold me to these rules. And so then every time, so again, we're in Markdown, we're probably checking this content into GitHub, which at a you know, at if I'm maintaining a thousand topics or less, that's probably gonna work pretty well. Every time uh it was checked in, it just had the agent go and look look it over and have it give me feedback. Oh, you didn't put enough metadata on the top of this topic. Oh, this topic is too long. Oh, it it this topic contains too many claims. Like you should break this up into more more Markdown files because you know you're mixing concepts and and tasks here. Like this is clearly, you know, this section is clearly a task, and this section contains a bunch of claims. Like there's uh claims facts, sorry. So there's a bunch of things that you could do with a very simple configuration of Claude, or I've actually not done this in uh in Gemini or OpenAI, so like I'm just gonna say Claude, but I'm assuming you can do this in other LLMs. Yeah, yeah. So you could set all those things up and it could basically just help you enforce a number of rules that are going to get you a lot further down the road than you would be if you were you know just writing content as text and Markdown or in YAML files or things like that. And I think you would find that if it did scale up and you hired more people and you did need to transition to a true structured format, you know, for personalization or reuse, those types of things, that you'd still have a lot of work to do, but you'd be in a better place.

Zohra: 45:02

Yeah, I think that's a great suggestion to kind of lean into those systems available to you. If you were designing a content architecture from scratch today. Let's say you've been hired, there are no systems, and you are going to be designing content architecture from scratch. Uh specifically for AI and agentic systems, what would you build from the source content all the way through retrieval context, uh governance, and execution? Oh my god. But you you do you you've been you've done great so far with all the I mean geez.

Patrick: 45:38

Well, so it would depend a ton on there's so many variables that go into that. So I'll give you one version of it. Okay, so if I'm coming into an organization that has, let's say 10 proper technical writers maintaining a hundred thousand pages of content, or well, let's just say fifty thousand pages of content. It's that's a little more on the on the medians there. And that content is you know, has any level of complexity, which it almost always does at that size, like you can almost never get around that. The first thing I'm going to do, if they're not already, if so, like you said, the nothing exists. The first thing I'm gonna do is I'm gonna introduce structured content and a proper CCMS. Like that would be step one. And then from there, I'm going to look at the delivery requirements. So one of the things that I think people undervalue is the importance of having a really high-quality website today. Like that is where most of your AI answers are coming from. Like before you implement any AI, you need to feed the public AI. And to do that, you need a website that has all the different Many of them are just SEO things, but all the different aspects that are going to give you the proper authority with Google, OpenAI, Anthropic, and the other major models, so that when people are asking questions of those systems, they're properly retrieving information from your set of information for your customers. Like that's gonna be 90% of your customers' AI interaction. From there, it's a very it's gonna get very situational. If I've got a field service team, I'm gonna look at how I'm gonna deploy an agent to that field service team that covers the particular issues that they're having a hard time solving that are also their information issues. I would start deploying agents and connections to our support channel. So like there was a feedback loop there. You know, those things are maybe not simple, but fairly straightforward, right? Like that's kind of like, you know, that's known work at this point in time. I'm not gonna call it a solved problem, but it's known work at this point in time. I think if you you kind of started with that path in that you're going to get yourself so far down the road that you're probably gonna be ahead of most people already. That I think that that is a blueprint. I guess the other thing I should say there too is when I said implement a CCMS, I meant do the information architecture work on the content, come up with an information model, get the content into really good shape, and then put it into a CCMS. So that's like the process of implementing structure. One of the biggest mistakes that people make is that they think they're gonna buy a tool and that's gonna solve their problems, and that's not the—not the case. Like the work it takes to go from having a ton of content that's unstructured to good structured content that's actually going to solve problems. That's an information modeling and information architecture and conversion process, and that is work. It has a huge payoff, it's the foundation for everything else. But if you don't do that work, you don't get the benefit out of structured content. So, like that would be the first step.

Zohra: 48:54

First of all, my apologies, Patrick. I did not mean to quote unquote assault you with that question.

Patrick: 49:01

No, no, no, no. It's a great question. It's just it's such a big question.

Zohra: 49:05

So I was on a team where the mindset was that the tool would solve all the problems. And thinking about information architecture and mod—well, let's say that we had to revisit how our content was modeled and architected. The tool did not solve the problem.

Patrick: 49:24

You think about it. If you were gonna go build a house and you showed up at the build site with a bunch of tools but no blueprints, yeah, what would you get? What would you end up with?

Zohra: 49:34

Thank you so much for watching. This was a great conversation, and I hope you had fun as well. I did.

Patrick: 49:39

Thanks again.

Zohra: 49:40

Thank you. Thank you for listening to Inside Tech Comm. If there's a guest you would love to hear on the show, please let me know. And don't forget to follow and catch every episode on your favorite podcast app. See you next time.