XML-powered MUD-side interface


update log:

  • 10-17-12: I've updated read_function to better detect the start of a function, which should resolve some circumstances under which the function definition didn't appear in the read function text.

If you've followed the series up to this point, you've probably been able to successfully generate a decent set of documentation, but it's probably also only useful to you in the HTML format. If you're using the config we discussed in part 2 of this series, you've also been generating an XML copy of your documentation. The XML doxygen spits out is bloated and frustrating to deal with--call this fair warning--but this post should help you lay a good foundation for parsing doxygen's output and turning it into a useful MUD-side manual system.

The structure of this post is as follows:

  • Parsing XML
    • Process
    • Implementation
  • Interfacing
    • Implementation

Processing XML

Depending on the size of your mudlib, and the amount of your mudlib you choose to process, one of the initial issues you're likely to face is the sheer size of the XML output doxygen creates. Our index alone is over 9 megabytes, and LDMud isn't particularly fantastic at working with large files. We're still running on LDMud 3.3.719, so while we can't speak to the usefulness of the xml_parse efun in .720, we did immediately run into the fact that it has some issues actually parsing XML properly which resulted in us needing to implement a pre-parser which was responsible for massaging our XML strings just to get xml_parse working properly.

I did some initial work on an LPC implementation of a SimpleXML emulator hoping to make a better XML parser for LPC in the process--and while it has been somewhat promising, it's not useful for dealing with large XML files.

I've ultimately ended up falling back on parsing our XML in Python using the cElementTree library, and then serializing it into a format the restore_object() efun can use. Now would be a good time to note that I'm still relatively new to Python, and my source will likely not fare well in an evaluation of my use of pythonic style. I will try to spend a fair amount of time discussing procedure, so the code is easier to replicate in another language should you need to do so.

Process

It might be easiest to think of this process in three main phases, one in which we inventory all of our classes and their members, a second in which we go collect details on each of these members, and a third in which we serialize our Python data structures (dicts, lists, strings) into mappings/arrays/strings and save them into a file compatible with LPC/restore_object. Before we get into this, we'll discuss our global class variables.

Setup

We need to compile a decent chunk of data and then serialize it all at once to create our output file, so we're setting up a number of global variables. Here are the current declarations:

 1#USER CONFIGURABLE VARIABLES
 2#should indicate the location of your doxygen XML directory
 3self.url_root = "u:\www\wizards\secure\doxygen_public\\xml\\"
 4#serialized data written here normally
 5self.output_file = "e:/doxy/index"
 6#if <debug> is true, data written here
 7self.output_debug = "e:/doxy/index_debug"
 8#if <debug> is true, a loadable .c will be written here
 9self.debug_loader = "e:/doxy/output_debug.c"
10#extension, but you probably won't need to mess with this...
11self.output_ext = ".o"
12
13#some closures for global use
14self.map_text = lambda x:x.text
15self.scrub = lambda x: len(x) or hasattr(x, "text")
16
17#Data-storage variables
18self.class_members = dict()
19self.class_refs = dict()
20self.class_details = dict()
21self.page_refs = dict()
22self.page_details = dict()
23self.group_members = dict()
24self.group_refs = dict()
25self.group_details = dict()
26self.doc_structure = dict()
27self.member_details = dict()

Inventorying the Index

In the first phase, we walk through our index by opening our XML file and creating an iterative parser with iterparse. The iterparse will be called when each tag terminates and will have a few attributes (where applicable) which we can test for values we want. In our case we're only interested in elements with the tag "compound" and which have a "kind" attribute of either "group", "page" or "class". Depending on which we've found, we store them in the appropriate group_refs, page_refs or class_refs dictionary following the format {name: refid}. In the case of the "group" and "class" which may also have function members, we also use some lambda closures to extract the name of all "function" members into the class_members or group_members dictionaries in the format {class_name: [function_names]}. It might be worth mentioning that doxygen generates stable refids we can use to look-up our members/classes. Aside from file extensions, these are even the same across the html/xml copies, meaning it should be trivial to generate a link from the MUD copy of the documentation to the web copy of the same.

In order to save on memory overhead, we go ahead and call clear() on each of these elements after we're done with them. Now we're ready to go compile detailed descriptions for all classes/pages/groups and their requisite members.

Enumerating class/group/page and member descriptions

Once we've populated self.class_refs, self.group_refs and self.page_refs, we call our requisite parse_ function for each key in these dictionaries., i.e.

1for classname in self.class_refs:
2	self.parse_class(classname)

While these parse_ functions have some different specifics, they all follow the same basic steps. First, we check our _refs dictionary for the key we've been called with, which will give us the refid indicating the location of the XML file we should load. We'll go read this file and once again create an iterparse parser.

  • If we're looking at a class or a group, we're going to check for "memberdef" tags with a "kind" attribute of "function". We'll find the value of the element's "name" tag, which will hold the name of our function, and we'll add it to our self.<type>_members dictionary.
  • If we're looking at a class/group, we're also going to check to see if we have a "compounddef" tag, and if we do it means there's also some head documentation associated with the compound describing it in more detail. We're basically going to follow the same steps described in the next section for function members to parse this description, except we're only going to be looking for the keys: "brief", "detail", "file", "history", "see also", "subpages", and "todo".
  • If we're looking at a page, we're going to check for "compounddef" tags with a kind of "page" and check for that element's "compoundname" tag. While this is fairly similar, this is also the next-to-last step for pages--we can from this point go ahead and parse out the page's "detaileddescription" tag, its "title" tag, and run a search for any subpages we should associate with it.

Returning to class/group parsing, we still have a bit of work to do for each of our function members.

  1. Find the member's "location" tag and make sure it has both a "bodystart" and a "bodyend" attribute. This protects us against accidentally parsing function prototypes, as they have no function body.
  2. Next we start pulling out specific details we'd like to keep about the function.
    1. We pull out the "references" and "referencedby" tags
    2. We run the detaileddescription tag through our self.parse_description function, which saves a dictionary containing a number of values we'll use in a minute. These values include the "detail", the "synopsis", "usage", "history", "see also", and "todo" sections.
    3. We pull out the "definition" and "argstring" tags as is
    4. We run the briefdescription tag through format_description
    5. We pull the "file", "startline" and "endline" attributes out of the "location" tag.
    6. We save all of these values into the self.member_details dictionary in the format: self.member_details[member_refid] = {name: value} such that all of our member functions will have a dictionary of values defining all of the following keys: "definition", "argstring", "brief", "detail", "file", "startline", "endline", "references", "referencedby", "synopsis", "usage", "history", "see also" and "todo". Keep in mind that some of these values may be null, but the keys will be present.

When we're done, we'll clear our elements to free up memory. There are a few relevant functions involved in this process which have gone undiscussed so far.

  • parse_description: We basically look for some expected sub elements in our description which indicate the synopsis, usage, history, see_also and todo elements. We extract these and clear the nodes we got them from and assume everything remaining is part of t he "detail" description for the node.
  • format_description:it is large, cumbersome, annoying, and fairly simple. It basically iterates over every subtag in the description node passed to it and uses a long if/elif/elif/.../elif/else chain to test for the tags we expect, and to properly format the description we'll return. The description returned is a list, though some sub-elements will be strings, nestes lists, and dictionaries. In another language I would've implemented this with a switch statement.
  • translate_ref: When we encounter a tag, we pass its refid and text to this function. Sometimes the text of a ref tag will already be an appropriate reference to use for that member/compound, but in other cases it may not be. We basically reverse-engineer a valid reference for our refid and then if the reference we've come up with doesn't match our text, we return text + {new_reference} to ensure that all references are compatible with the manual command.

Serializing data

In the final phase, we want to write a restore_object compatible file by serializing the dictionaries we've been keeping our data in into lpc-compatible mappings. At the code level (at least in the serialize function...) this looks very simple. We open our output file for writing, begin it with a line reading "#1:0" and write our variables. All of the text of a variable must be on one line (our lines will be very long), but the code to write them is trivial:

1output.write("class_members "+unicode(self.class_members)+"\n")

This, of course, isn't all that's going on. If we did this, our dicts would have a string representation of {key:data} and our lists would be represented as [item, item item]--but in LPC we need ([key:data]) and ({}) respectively. We also need to take care to properly escape some things, and perform some other scrubbing/conversion. For that purpose, we redefine both the "list" and "dict" classes and provide a new unicode method with which to overload their serialization behavior. If you're particularly familiar with python, you will likely notice one of the side-effects of this method of doing things, which is that this script often uses a literal list/dict invocation wrapped with a call to list() or dict() (e.g.: blah = list([1, 2, 3])) to make sure that we're creating our overloaded lists and dictionaries.

Serialization, debug mode

I should also provide some explanation here of the "debug" flag which you may notice in the code. It's used in the class invocation, so you call either DoXML(0) or DoXML(1). In the latter case you're invoking debug mode. One of the problems we ran into was that it is horribly hard to figure out exactly what is going wrong when you're outputting several megabytes of serialized text on a single line. When the restore_object() fails due to malformatted data, it'll give you a useful line number error, but that line will contain hundreds of thousands of characters.

In debug mode, each key is written to its own variable on its own line, and a test .c object is written along side it. You can load/update the test object and when/if an import error is thrown it will give you a line number you can use to actually track down the element (and thus the XML file) containing malformatted text.

Implementation

Following is our source code. Note the variables which you'll need to modify.

  1import codecs
  2import re
  3import itertools
  4from xml.etree.cElementTree import iterparse
  5
  6
  7"""Reimplement the list class so we may alter how lists get serialized."""
  8class list(list):
  9
 10	def __unicode__(self):
 11		trans_table = ''.join( [chr(i) for i in range(128)] + [' '] * 128 )
 12		out = "({"
 13
 14		for element in self:
 15			if isinstance(element, unicode) or isinstance(element, str):
 16				if element == "\\":
 17					out += '"\\\\",'
 18				else:
 19					out += '"'+re.sub(r'([\"\'\\])', r'\\\1', unicode(element).replace("\\", "\\\\")).replace("\n", "\\n").replace("\t", "\\t").replace("\\\\\\n", "\\\\n")+'",'
 20			else:
 21				out += unicode(element)+","
 22		return out + "})"
 23
 24
 25"""Reimplement the dict class so we may alter how lists get serialized."""
 26class dict(dict):
 27
 28	def __unicode__(self):
 29		trans_table = ''.join( [chr(i) for i in range(128)] + [' '] * 128 )
 30		out = "(["
 31		for key in self:
 32			if isinstance(self[key], unicode) or isinstance(self[key], str):
 33				out += '"'+key+'":"'+unicode(self[key]).replace('"', '\\"').replace("\n", "\\n").replace("\t", "\\t")+'",'
 34			else:
 35				out += '"'+key+'":'+unicode(self[key])+','
 36		return out + "])"
 37
 38
 39"""Parse Doxygen's XML and serialize into LPC's restore/save_object format.
 40
 41To execute in normal mode you can use command line 'python doxml.py' or import
 42doxml and execute doxml.DoXML(0).
 43
 44Will run in debug mode when executed as doxml.DoXML(1) which will write the
 45output to an alternate file, and write a companion .c file which you should
 46then load on the MUD. This is only useful for debugging, as it creates
 47numbered variables on separate lines allowing you to isolate the source of
 48an import failure.
 49"""
 50class DoXML:
 51	"""If <debug> is true, the serialized output will be altered to make it
 52	significantly easier to identify seralization issues.
 53	"""
 54	def __init__(self, debug):
 55		#USER CONFIGURABLE VARIABLES
 56		#should indicate the location of your doxygen XML directory
 57		self.url_root = "...\doxygen_public\\xml\\"
 58		#serialized data written here normally
 59		self.output_file = ".../index"
 60		#if <debug> is true, data written here
 61		self.output_debug = ".../index_debug"
 62		#if <debug> is true, a loadable .c will be written here
 63		self.debug_loader = ".../output_debug.c"
 64		#extension, but you probably won't need to mess with this...
 65		self.output_ext = ".o"
 66
 67		#some closures for global use
 68		self.map_text = lambda x:x.text
 69		self.scrub = lambda x: len(x) or hasattr(x, "text")
 70
 71		#Data-storage variables
 72		self.class_members = dict()
 73		self.class_refs = dict()
 74		self.class_details = dict()
 75		self.page_refs = dict()
 76		self.page_details = dict()
 77		self.group_members = dict()
 78		self.group_refs = dict()
 79		self.group_details = dict()
 80		self.doc_structure = dict()
 81		self.member_details = dict()
 82
 83		#Process index, then parse classes, groups, pages, finally serialize
 84		self.parse_index()
 85		for classname in self.class_refs:
 86			self.parse_class(classname)
 87		for groupname in self.group_refs:
 88			self.parse_group(groupname)
 89		for pagename in self.page_refs:
 90			self.parse_page(pagename)
 91		self.serialize(debug)
 92
 93	"""Parse index.xml which enumerates our classes and their members for the
 94	full list of classes/groups/pages and members we need to gather details on.
 95	"""
 96	def parse_index(self):
 97		xml = codecs.open(self.url_root + "index.xml", encoding="ascii", errors="ignore")
 98		parser = iterparse(xml)
 99		for event, elem in parser:
100			if elem.tag == "compound":
101				if "kind" in elem.attrib:
102					if elem.attrib["kind"] == "group":
103						group_name = elem.findtext("name")
104						self.group_refs[group_name] = elem.attrib["refid"]
105						#find function children
106						func_filter = lambda x: "kind" in x.attrib and x.attrib["kind"] == "function"
107						#extract refid and name in a dict
108						func_extractor = lambda x: (x.findtext("name"),x.attrib["refid"])
109						self.group_members[group_name] = dict(map(func_extractor, filter(func_filter, elem.findall("member"))))
110					elif elem.attrib["kind"] == "page":
111						page_name = elem.findtext("name")
112						self.page_refs[page_name] = elem.attrib["refid"]
113					elif elem.attrib["kind"] == "class":
114						class_name = elem.findtext("name")
115						self.class_refs[class_name] = elem.attrib["refid"]
116						#find function children
117						func_filter = lambda x: "kind" in x.attrib and x.attrib["kind"] == "function"
118						#extract refid and name in a dict
119						func_extractor = lambda x: (x.findtext("name"),x.attrib["refid"])
120						self.class_members[class_name] = dict(map(func_extractor, filter(func_filter, elem.findall("member"))))
121				elem.clear()
122			if elem.tag == "doxygenindex":
123				elem.clear()
124
125	"""
126	"""
127	def find_subpages(self, elem, ref):
128		subpages = list()
129		for node in elem.findall("innerpage"):
130			subpages.append(dict({"title":node.text, "name":self.translate_ref(node.text, node.attrib["refid"])}))
131			if node.attrib["refid"] in self.doc_structure:
132				self.doc_structure[node.attrib["refid"]] += list([ref])
133			else:
134				self.doc_structure[node.attrib["refid"]] = list([ref])
135		for node in elem.findall("innergroup"):
136			subpages.append(dict({"title":node.text, "name":self.translate_ref(node.text, node.attrib["refid"])}))
137			if node.attrib["refid"] in self.doc_structure:
138				self.doc_structure[node.attrib["refid"]] += list([ref])
139			else:
140				self.doc_structure[node.attrib["refid"]] = list([ref])
141		return subpages
142
143	"""
144	"""
145	def parse_page(self, page_name):
146		self.page_details[page_name] = dict()
147		print self.url_root + self.page_refs[page_name]
148		xml = codecs.open(self.url_root + self.page_refs[page_name]+".xml", encoding="ascii", errors="ignore")
149		parser = iterparse(xml)
150		for event, elem in parser:
151			if elem.tag == "compounddef" and elem.attrib["kind"] == "page":
152				el_name = elem.findtext("compoundname")
153				self.page_details[el_name] = dict({"detail":self.format_description(elem.find("detaileddescription")), "title":elem.findtext("title"), "subpages":self.find_subpages(elem, self.page_refs[page_name]) })
154
155	"""
156	"""
157	def parse_class(self, class_name):
158		xml = codecs.open(self.url_root + self.class_refs[class_name]+".xml", encoding="ascii", errors="ignore")
159		parser = iterparse(xml)
160		for event, elem in parser:
161			if elem.tag == "memberdef" and elem.attrib["kind"] == "function":
162				el_name = elem.findtext("name")
163				if el_name in self.class_members[class_name]:
164					loc = elem.find("location")
165					if "bodystart" in loc.attrib and "bodyend" in loc.attrib:
166						references = elem.findall("references")
167						referencedby = elem.findall("referencedby")
168						if references is not None and len(references):
169							references = list(map(self.map_text, references))
170						else:
171							references = list()
172
173						if referencedby is not None and len(referencedby):
174							referencedby = list(map(self.map_text, referencedby))
175						else:
176							referencedby = list()
177
178						if self.class_members[class_name][el_name] not in self.member_details:
179							description = self.parse_description(elem.find("detaileddescription"), 0)
180							self.member_details[self.class_members[class_name][el_name]] =  dict({"definition":elem.findtext("definition"), "argsstring":elem.findtext("argsstring"), "brief":self.format_description(elem.find("briefdescription")), "detail":description["detail"], "file":loc.attrib["file"], "startline":loc.attrib["bodystart"], "endline":loc.attrib["bodyend"], "references":references, "referencedby":referencedby, "synopsis":description["synopsis"], "usage":description["usage"], "history":description["history"], "see also":description["see also"], "todo": description["todo"]})
181				elem.clear()
182			elif elem.tag == "compounddef":
183				description = self.parse_description(elem.find("detaileddescription"), 0)
184				loc = elem.find("location")
185				self.class_details[self.class_refs[class_name]] = dict({"brief":self.format_description(elem.find("briefdescription")), "detail":description["detail"], "file":loc.attrib["file"], "history":description["history"], "see also":description["see also"], "todo": description["todo"], "subpages":self.find_subpages(elem, self.class_refs[class_name])})
186			if elem.tag == "doxygen":
187				elem.clear()
188
189	"""Walk each element in the group for functions and parse out details.
190
191	You may need to modify this if you want to be able to read docs for more
192	than just functions.
193	"""
194	def parse_group(self, class_name):
195		xml = codecs.open(self.url_root + self.group_refs[class_name]+".xml", encoding="ascii", errors="ignore")
196		parser = iterparse(xml)
197		for event, elem in parser:
198			if elem.tag == "memberdef" and elem.attrib["kind"] == "function":
199				el_name = elem.findtext("name")
200				if el_name in self.group_members[class_name]:
201					loc = elem.find("location")
202					if "bodystart" in loc.attrib and "bodyend" in loc.attrib:
203						references = elem.findall("references")
204						referencedby = elem.findall("referencedby")
205						if references is not None and len(references):
206							references = list(map(self.map_text, references))
207						else:
208							references = list()
209
210						if referencedby is not None and len(referencedby):
211							referencedby = list(map(self.map_text, referencedby))
212						else:
213							referencedby = list()
214						if self.group_members[class_name][el_name] not in self.member_details:
215							description = self.parse_description(elem.find("detaileddescription"), 0)
216							self.member_details[self.group_members[class_name][el_name]] =  dict({"definition":elem.findtext("definition"), "argsstring":elem.findtext("argsstring"), "brief":self.format_description(elem.find("briefdescription")), "detail":description["detail"], "file":loc.attrib["file"], "startline":loc.attrib["bodystart"], "endline":loc.attrib["bodyend"], "references":references, "referencedby":referencedby, "synopsis":description["synopsis"], "usage":description["usage"], "history":description["history"], "see also":description["see also"], "todo": description["todo"]})
217
218				elem.clear()
219			elif elem.tag == "compounddef":
220				description = self.parse_description(elem.find("detaileddescription"), 0)
221				self.group_details[self.group_refs[class_name]] = dict({"brief":self.format_description(elem.find("briefdescription")), "detail":description["detail"], "history":description["history"], "see also":description["see also"], "todo": description["todo"], "subpages":self.find_subpages(elem, self.group_refs[class_name])})
222
223			elif elem.tag == "doxygen":
224				elem.clear()
225
226	"""If <text> isn't a valid manual reference, use <ref> to resolve a
227	valid reference and append it in curly braces.
228	"""
229	def translate_ref(self, text, ref):
230		refstring = ""
231		#check pages
232		if ref in self.page_refs.values():
233			refstring += filter(lambda x: self.page_refs[x] == ref, self.page_refs)[0]
234		elif ref.rpartition("_1")[0] in self.page_refs.values():
235			refstring += filter(lambda x: self.page_refs[x] == ref.rpartition("_1")[0], self.page_refs)[0]
236		#check groups
237		if ref in self.group_refs.values():
238			refstring += filter(lambda x: self.group_refs[x] == ref, self.group_refs)[0]
239		#check classes
240		if ref in self.class_refs.values():
241			refstring += filter(lambda x: self.class_refs[x] == ref, self.class_refs)[0]
242		#check members
243		#I want the name of the class and the name of the member
244		if ref in self.member_details:
245			classname = self.member_details[ref]["file"].rpartition("/")[2].rpartition(".")[0]
246			membername = self.member_details[ref]["definition"].rpartition(" ")[-1]
247			refstring += classname+"."+membername
248
249		#if the refstring != text, append it
250		if text == refstring:
251			return ""
252		elif len(refstring):
253			return " {"+refstring+"}"
254		else:
255			return ""
256
257	"""Walk the description by tags and convert them to formatted plaintext.
258
259	The method used is a bit dumbfire and our implementation is slowly evolving
260	as we encounter new tags in the xml and figure out how to parse them best.
261	This is far from perfect, and far from a science. You may prefer to alter
262	many of these.
263	"""
264	#convert the descript subtags to plaintext
265	#there are two possible circumstances:
266	#	1 - we need to do something to a description _before_ a tag
267	#	2 - we need to do something to the desc string _after_ a tag
268	def format_description(self, desc_node):
269		desc = list()
270		prefix = None
271		pad = None
272		for elem in filter(self.scrub, desc_node):
273
274			if elem.tag == "para":
275				#para can be text or it can contain tags
276				if hasattr(desc_node, 'tag') and desc_node.tag in ["detaileddescription"]:
277					if len(elem):
278						desc.append("\n")
279					else:
280						desc.append("\n\n")
281				if elem.text is not None:
282					desc.append(elem.text)
283			elif elem.tag == "simplesect":
284				sub_desc = ""
285				if elem.text is not None:
286					sub_desc += elem.text
287				if elem.tail is not None:
288					sub_desc += elem.tail
289				if "kind" in elem.attrib:
290					sub_desc = list(sub_desc)
291					if len(elem):
292						desc.append(dict({elem.attrib["kind"].capitalize()+":": list(sub_desc + self.format_description(filter(self.scrub, elem)))}))
293					else:
294						desc.append(dict({elem.attrib["kind"].capitalize()+":": sub_desc}))
295				continue
296			elif elem.tag == "title":
297				pass
298			elif elem.tag == "linebreak":
299				pass
300			elif elem.tag == "hruler":
301				pass
302			elif elem.tag == "preformatted":
303				pass
304			elif elem.tag == "programlisting":
305				pass
306			elif elem.tag == "verbatim":
307				#verbatim inserts this "*  " before each line, so we're removing
308				desc.append("".join(elem.text.split("*  ")))
309			elif elem.tag == "indexentry":
310				pass
311			elif elem.tag in ["orderedlist", "itemizedlist"]:
312				sub_desc = list()
313				if elem.tag == "itemizedlist":
314					prefix = itertools.repeat("*")
315				else:
316					prefix = itertools.count(1)
317				for sub_el in elem:
318					substr = list()
319					if sub_el.text is not None:
320						substr.append(sub_el.text)
321					if len(sub_el):
322						substr.append(self.format_description(filter(self.scrub, sub_el)))
323					if sub_el.tail is not None:
324						substr.append(sub_el.tail)
325					sub_desc.append(dict({unicode(prefix.next()):substr}))
326				desc.append(sub_desc)
327			elif elem.tag == "listitem":
328				continue
329			elif elem.tag == "variablelist":
330				pass
331			elif elem.tag == "table":
332				pass
333			elif elem.tag == "heading":
334				if elem.text is not None:
335					desc.append("\n"+elem.text+"\n")
336				else:
337					pass
338			elif elem.tag == "image":
339				pass
340			elif elem.tag == "dotfile":
341				pass
342			elif elem.tag == "toclist":
343				pass
344			elif elem.tag == "language":
345				pass
346			elif elem.tag == "parameterlist":
347				desc.append("\n\nParameters:")
348				for sub_el in elem:
349					if sub_el.tag == "parameteritem":
350						desc.append(list(["\n",sub_el.findtext(".//parametername")+" - ",self.format_description(sub_el.find("parameterdescription")),"\n"]))
351					#these should be "parameteritem"
352				continue
353			elif elem.tag == "xrefsect":
354				pass
355			elif elem.tag == "ref":
356				#we need to translate this ref into a format that'll work with our command
357				desc.append("[color=ref]"+elem.text+self.translate_ref(elem.text, elem.attrib["refid"])+"[/color]")
358				pass
359			elif elem.tag == "copydoc":
360				pass
361			elif elem.tag == "blockquote":
362				pass
363			elif elem.tag == "computeroutput":
364				#desc.append("[b]")
365				if elem.text is not None:
366					desc.append("'"+elem.text+"'")
367				if len(elem):
368					desc.append(self.format_description(filter(self.scrub, elem)))
369				if elem.tail is not None:
370					desc.append(elem.tail)
371				continue
372			elif elem.tag == "sp":
373				if len(desc):
374					desc[-1] += " "
375				else:
376					desc.append(" ")
377			elif elem.text is not None:
378				desc.append(elem.text)
379			else:
380				if elem.text is not None:
381					desc.append(elem.text)
382			if len(elem):
383				desc.append(self.format_description(filter(self.scrub, elem)))
384			if elem.tail is not None:
385				desc.append(elem.tail)
386		return desc
387
388	"""Parse some expected subcategories out into variables and assume anything
389	leftover is part of the primary description.
390	"""
391	#parse out some major subcategories
392	def parse_description(self, desc_node, debug):
393		#we're technically getting more than the description out of this
394		desc = dict({"detail":list(), "synopsis":list(), "usage":list(), "history":list(), "see also":list(), "todo":list()})
395
396		#step 1 - go imperatively pull a few things we might see in
397		#		  this node out right now and then use .clear() so we
398		#		  know we can use everything remaining in the ["detail"]
399		simplesects = desc_node.findall(".//simplesect")
400		#handles synopsis, history, usage and see also
401		for sect in simplesects:
402			if "kind" in sect.attrib:
403				if sect.attrib["kind"] == "see":
404					desc["see also"] = list(map(self.format_description, sect))
405					sect.clear()
406				elif sect.attrib["kind"] == "par":
407					if sect.findtext("title") == "Usage:":
408						desc["usage"] = list(map(self.format_description, sect))
409						sect.clear()
410					elif sect.findtext("title") == "Synopsis:":
411						desc["synopsis"] = list(map(self.format_description, sect))
412						sect.clear()
413					elif sect.findtext("title") == "History:":
414						desc["history"] = list(map(self.format_description, sect))
415						sect.clear()
416
417		simplesects = None
418		#all that's left is todo
419		xrefsects = desc_node.findall(".//xrefsect")
420		for sect in xrefsects:
421			if sect.findtext("xreftitle") == "Todo":
422				desc["todo"] = list(map(self.format_description, sect))
423				sect.clear()
424		xrefsects = None
425		desc["detail"] = list(self.format_description(desc_node))
426		desc_node.clear()
427		return desc
428
429	"""Take our data variables and write them to a .o file using LPC's data-
430	serialization format.
431
432	If <debug> is true, they're written to numbered vars which makes it a *lot*
433	easier to track down any load issues you may encounter in LDMud.
434	"""
435	def serialize(self, debug):
436		if not debug:
437			with codecs.open(self.output_file+self.output_ext, mode="w") as output:
438				output.write("#1:0\n")
439				#they're all dicts or dicts of dicts
440				output.write("class_members "+unicode(self.class_members)+"\n")
441				output.write("class_refs "+unicode(self.class_refs)+"\n")
442				output.write("class_details "+unicode(self.class_details)+"\n")
443				output.write("group_members "+unicode(self.group_members)+"\n")
444				output.write("group_refs "+unicode(self.group_refs)+"\n")
445				output.write("group_details "+unicode(self.group_details)+"\n")
446				output.write("page_refs "+unicode(self.page_refs)+"\n")
447				output.write("page_details "+unicode(self.page_details).encode("ascii", "ignore")+"\n")
448				output.write("member_details "+unicode(self.member_details)+"\n")
449				output.write("doc_structure "+unicode(self.doc_structure)+"\n")
450		else:
451			with codecs.open(self.output_debug+self.output_ext, mode="w") as output:
452				with codecs.open(self.debug_loader, mode="w") as loader:
453					output.write("#1:0\n")
454					x = 1
455					for key, value in self.class_members.items():
456						output.write("class_members"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
457						loader.write("mapping class_members"+unicode(x)+" = ([]);\n")
458						x += 1
459
460					x = 1
461					for key, value in self.class_refs.items():
462						output.write("class_refs"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
463						loader.write("mapping class_refs"+unicode(x)+" = ([]);\n")
464						x += 1
465
466					x = 1
467					for key, value in self.class_details.items():
468						output.write("class_details"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
469						loader.write("mapping class_details"+unicode(x)+" = ([]);\n")
470						x += 1
471
472					x = 1
473					for key, value in self.group_members.items():
474						output.write("group_members"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
475						loader.write("mapping group_members"+unicode(x)+" = ([]);\n")
476						x += 1
477
478					x = 1
479					for key, value in self.group_refs.items():
480						output.write("group_refs"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
481						loader.write("mapping group_refs"+unicode(x)+" = ([]);\n")
482						x += 1
483
484					x = 1
485					for key, value in self.group_details.items():
486						output.write("group_details"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
487						loader.write("mapping group_details"+unicode(x)+" = ([]);\n")
488						x += 1
489
490					x = 1
491					for key, value in self.page_refs.items():
492						output.write("page_refs"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
493						loader.write("mapping page_refs"+unicode(x)+" = ([]);\n")
494						x += 1
495
496					x = 1
497					for key, value in self.page_details.items():
498						output.write("page_details"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
499						loader.write("mapping page_details"+unicode(x)+" = ([]);\n")
500						x += 1
501
502					x = 1
503					for key, value in self.member_details.items():
504						output.write("member_details"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
505						loader.write("mapping member_details"+unicode(x)+" = ([]);\n")
506						x += 1
507
508					x = 1
509					for key, value in self.doc_structure.items():
510						output.write("doc_structure"+unicode(x)+" "+unicode(dict({key:value}))+"\n")
511						loader.write("mapping doc_structure"+unicode(x)+" = ([]);\n")
512						x += 1
513
514					loader.write('\n\nvoid reset(int arg)\n{\n\trestore_object("'+self.output_debug+'.c");\n}\n')
515
516if __name__ == "__main__":
517	DoXML(0)

Interfacing

After we generate the restore_object compatible file, all we've really got is a few large mappings containing a bunch of data which we (unfortunately) still need to do some work to massage for final output in a MUD-side manual interface. The good news is that this work is fairly straightforward, but the bad news is that some of the details here will be idiomatic and you'll almost undoubtedly need to rewrite parts of this in order for it to function on your MUD. As interfacing is less complex than parsing, we're just going to dive right into the dirty work. There are three objects of note here:

  1. The new 'man' or 'manual' command
  2. The old 'man' or 'manual' command
  3. The doxy documentation daemon

The first of these is pretty simple, and as a stopgap, it falls back on the second of these. We renamed our "man" command to "manb" (har, har). Here's the code for our new 'man' command:

 1inherit STD_COMMAND;
 2
 3int do_cmd(string str)
 4{
 5	log_file_open("man_syntax", str+"\n");
 6	string modified_command;
 7	if(find_and_load_object("/players/misery/doxy/doxml.c") && "/players/misery/doxy/doxml.c"->read_doc(str, &modified_command))
 8	{
 9		return 1;
10	}
11	else
12	{
13		//we didn't find this in man2, so we'll fall back on man1.
14		return "/bin/wizards/manb.c"->do_cmd(modified_command);
15	}
16}
17
18int help()
19{
20	string halp_fmt = "%30s   %=-40s\n\n";
21	string example_fmt = "%30s   %=-40s\n";
22	string heading_fmt = "%s\n%=-70s\n\n\tExamples: %s\n\n";
23	string halp = "This command provides access to Doxygen-generated "
24		"code documentation.\n\nPRIMARY SYNTAX OPTIONS:\n";
25	halp += sprintf(halp_fmt, "man", "Reads the manual index.");
26	halp += sprintf(halp_fmt, "man <topic>", "Looks in order for a page, a group, or a class by the name of topic.");
27	halp += sprintf(halp_fmt, "man <class>.<member>", "Reads class::member documentation. (:: and a space are also valid separators).");
28	halp += sprintf(halp_fmt, "man pages <topic>", "Explicitly search pages for topic.");
29	halp += sprintf(halp_fmt, "man groups <topic>", "Explicitly search groups for topic. Necessary if a page obscures the group you need.");
30	halp += sprintf(halp_fmt, "man classes <topic>", "Explicitly search classes for topic. Necessary if a page/group obscures the class you need.");
31	halp += sprintf(halp_fmt, "man pages", "Lists all available pages.");
32	halp += sprintf(halp_fmt, "man groups", "Lists all available groups.");
33	halp += sprintf(halp_fmt, "man classes", "Lists all available classes.");
34	halp += "SPECIAL COMMAND MODIFIERS:\n";
35	halp += sprintf(heading_fmt, "READ", "The 'read' command allows you to read a class file or a group/class member's text.", implode(({"'man read <class>'", "'man read <class>.<member>'"}), ", ") );
36	halp += sprintf(heading_fmt, "BRIEF", "The 'brief' command specifies that you'd like less information and behaves a little differently depending on what type of documentation you use it on. For classes/groups, brief mode will omit the member list. For functions, it includes only the synopsis, and the first sentence of the description. For other types it has the default behavior.", implode(({"'man brief efun'", "'man brief efun.present'"}), ", ") );
37	halp += sprintf(heading_fmt, "VERBOSE", "The 'verbose' command provides as much information as possible. For the 'pages', 'classes' and 'groups' lists, this means a 1-sentence description of each item is included (if available.) For classes/groups this has a similar effect, providing brief descriptions where available for all member functions. For functions and pages it retains the default behavior.", implode(({"'man verbose efun'", "'man verbose efun.present'"}), ", ") );
38
39	write(halp);
40	return 1;
41}

You can note there that the doxy documentation daemon is still residing in my personal work directory--once it's finalized I'll move it out into its final home. The biggest point of information here is that the daemon returns 1 if it resolved our request, and 0 if not. If it returns 0, it also checks our command and either writes it directly, or writes a modified copy of it to the modified_command variable which we pass by reference. This is largely because we're trying to encourage a bit more specificity in our manual command syntax--we see it as generally bad if the manual command is allowing you to be lazy and not requiring you to know the difference between efuns, sefuns and lfuns (some of our historical mudlib documentation makes it obvious that "lfun" served a dual use as "local function" and "living function" at the same time, for example.)

Before we jump straight into the full source for the documentation daemon, we're going to briefly discuss what our process looks like. The reset of the daemon contains a restore_object specifying the location of the .o file we generated in the first half of this post. We have a read_doc function which is essentially just a command processor, and then we have a number of query/format functions used to assist the command parser in serving/printing documentation.

The read_doc function has a finite list of "type" keywords and "mode" keywords. In our case, types are "pages", "groups" and "classes" while "modes" are "read", "verbose" and "brief", while we're also entertaining some additional modes. Our first step is to register the presence of and strip any of these encountered out of the string with the obvious caveat that under some circumstances, unless we turn these into flags, legitimate classes/members might get obscured. We'll probably be modifying how this works ourselves, but we're still gathering feedback. If the command has no length, we instead call print_page("index"); and return--"Index" will be the name doxygen gives to your first documentation page. I probably should've thrown this in an earlier part of the series, but for reference you can create an initial page with an .md file like ours, below:

 1Tsunami Mudlib Documentation                                {#mainpage}
 2============
 3
 4## Help Wanted
 5I'm working on tidying up our Doxygen implementation still, though the
 6rate at which I'm identifying new issues has slowed and I've started to
 7roll forward with the task of documenting (in the short term) the most
 8important parts of the mudlib.
 9
10If you're interested in helping out, let _Misery_ know to send you a
11copy of the current documentation standards.
12
13## Guides
14- @subpage new_wizard
15- @subpage qc_standards "QC Standards"
16- @subpage build "Builder's Guide"
17- @subpage actions_guide
18- @subpage driver "Driver Documentation"
19
20## Meta
21- @subpage development_axioms
22- @subpage doxygen_guidelines

At this point we map the parts of our command through a function which downconverts them by stripping the special commands, converting some other keywords into the old format, and saves them to the modified_command variable which is passed by reference from the 'man' command. This step can (and will) be skipped once the old command no longer serves as a fallback. This doesn't modify our working copy of the command--just the one passed by reference.

From here we get into the real meat of the routing logic. We pass our our type_flag variable into a switch statement; either it will match one of our declared types, or it will be null, so we parse all explicit types separately, and then in the default case use a cascading logic that first tries to identify a page, then a group, and finally a class. In all instances, when a successful match is made we save the reference to our "ref" variable, and in all instances we also save an error string which will later get printed should our documentation string come up empty. If our command is currently empty but our type flag is set, we go ahead and set our ref to the value of the type_flag. We could just explicitly print these now, but we want to allow our mode options, like brief and verbose, to have some meaning in the context of the page/group/class lists.

In the final phase of command processing, if we have a ref value of any sort, we pass the value of our mode_flag into a switch, so we can process the commands a little differently for each of our variables. At the end of any given processing tree, we check our documentation variable for length and if there is none, we print the most recently recorded error. In the case of found documentation, we run the document through our paged reading system. Without further adieu, the source:

  1#include <daemons.h>
  2#include <lpctypes.h>
  3#include <regexp.h>
  4#define TABULAR_SPRINTF_FMT "   %#-*s\n"
  5
  6string *key_order = ({"name", "synopsis", "description", "examples", "history", "future", "see also", "meta"});
  7mapping class_members = ([]);
  8mapping class_refs = ([]);
  9mapping class_details = ([]);
 10
 11mapping page_refs = ([]);
 12mapping page_details = ([]);
 13
 14mapping group_members = ([]);
 15mapping group_refs = ([]);
 16mapping group_details = ([]);
 17
 18mapping doc_structure = ([]);
 19mapping member_details = ([]);
 20
 21/* prototypes */
 22
 23int message_doc(string documentation, string error);
 24int read_doc(string command, string modified_command);
 25int tabular_adjust_cols();
 26varargs mixed unpack_description(mapping desc, int level, int last_tagspace);
 27void reset(int arg);
 28
 29mapping query_class_detail(string ref);
 30mapping query_group_detail(string ref);
 31mapping query_member_detail(string ref);
 32mapping query_member_in_classes(string membername);
 33mapping query_member_in_groups(string membername);
 34mapping query_page_details(string pagename);
 35string *query_class_members(string classname);
 36string *query_classes();
 37string *query_group_members(string groupname);
 38string *query_groups();
 39string *query_pages();
 40string *query_sub_pages(string page);
 41string query_class(string classname);
 42string query_class_member(string classname, string funcname);
 43string query_group(string groupname);
 44string query_group_member(string groupname, string funcname);
 45string query_page(string pagename);
 46
 47string format_class_members(string classname);
 48string format_class_members_detailed(string classname);
 49string format_classes();
 50string format_classes_detailed();
 51string format_compound_details(mapping details, string name);
 52//string format_group(string groupname);
 53string format_group_members(string groupname);
 54string format_group_members_detailed(string groupname);
 55string format_groups();
 56string format_groups_detailed();
 57string format_member_brief(mapping details);
 58string format_member_details(mapping details);
 59string format_member_in_classes(string membername);
 60string format_member_in_groups(string membername);
 61string format_others(string search, string current);
 62string format_page(string pagename);
 63string format_pages();
 64string format_pages_detailed();
 65
 66void print_class_members(string classname);
 67void print_class_members_detailed(string classname);
 68void print_classes();
 69void print_compound_details(mapping details, string name);
 70void print_group(string groupname);
 71void print_group_members(string groupname);
 72void print_group_members_detailed(string groupname);
 73void print_groups();
 74void print_member_brief(mapping details);
 75void print_member_details(mapping details);
 76void print_member_in_classes(string membername);
 77void print_member_in_groups(string membername);
 78void print_others(string search, string current);
 79void print_page(string pagename);
 80void print_page_detail(string pagename);
 81void print_pages();
 82
 83void read_class(string ref);
 84void read_function(string ref);
 85/* fin prototypes */
 86
 87//utility/other/manual funcs
 88int tabular_adjust_cols()
 89{
 90	return TERM_D->query_columns()-5;
 91}
 92
 93/**
 94if we haven't yet, load our saved data. This data is generated by a python
 95script.
 96*/
 97void reset(int arg)
 98{
 99	if(arg || sizeof(class_refs)){return;}
100	restore_object("/players/misery/index.c");
101}
102
103///recursively unpack description mappings.
104varargs mixed unpack_description(mapping desc, int level, int last_tagspace)
105{
106	switch(typeof(desc))
107	{
108		case T_STRING:
109			return desc;
110		case T_POINTER:
111			return implode(map(flatten_array(desc), #'unpack_description, level), "");
112		case T_MAPPING:
113			level++;
114			string tag = implode(m_indices(desc), "");
115			int tagspace = sizeof(tag);
116			if(tagspace query_columns() - (tagspace + (level*2) + last_tagspace)-2;
117			if(!cols){cols=68;}
118			return sprintf("\n%=*s  %=-*s", tagspace, tag, cols, implode(map(flatten_array(m_values(desc)), #'unpack_description, level, tagspace+last_tagspace), ""));
119	}
120}
121
122///translate some commands to our old man syntax
123string translate_man_command(string command)
124{
125	switch(command)
126	{
127		case "weapon":
128			return "wfun";
129		case "living":
130			return "lfun";
131		case "room":
132			return "rfun";
133		case "monster":
134			return "mfun";
135		case "read":
136		case "classes":
137		case "pages":
138		case "brief":
139		case "verbose":
140			return 0;
141		default:
142			return command;
143	}
144}
145
146///Return all categories the topic <phrase> is valid for.
147mapping check_others(string phrase)
148{
149	mapping others = ([]);
150	if(query_page(phrase)){others+=(["page"]);}
151	if(query_class(phrase)){others+=(["class"]);}
152	if(query_group(phrase)){others+=(["group"]);}
153	return others;
154}
155
156mapping type_keywords = (["pages", "classes", "groups"]);
157mapping mode_keywords = (["read", "update", "eval", "brief", "verbose"]);
158int read_doc(string command, string modified_command)
159{
160	if(!command)
161	{
162		print_page("index");
163		return 0;
164	}
165	string *parts = explode(command, " ");
166	string type_flag, mode_flag, ref, error;
167	string documentation = "";
168	/* strip out flags, but we'll hop over anything with proper ::/. separators */
169	if(sizeof(parts))
170	{
171		foreach(int i : sizeof(parts[0..<1]))
172		{
173			if(member(type_keywords, parts[i]))
174			{
175				type_flag = type_flag ? type_flag : parts[i];
176				parts[i] = 0;
177			}
178			if(member(mode_keywords, parts[i]))
179			{
180				mode_flag = mode_flag ? mode_flag : parts[i];
181				parts[i] = 0;
182			}
183		}
184		parts = filter(parts, (:$1:));
185		parts = regexplode(implode(parts, " "), "::|\\.|\\s", RE_OMIT_DELIM);
186	}
187	else
188	{
189		print_page("index");
190		return 0;
191	}
192
193	//save a backwards compatible command copy.
194	modified_command = implode(map(parts, #'translate_man_command) - ({""}), " ");
195
196	int num_parts = sizeof(parts);
197
198	/*
199	If the command has parts, we're going to either look for an explicit type
200	flag, or fall back into default parsing if none is present. Default parse
201	is page -> group -> class and then look for member if specified.
202	We save our error messages and our "ref" as we go. Later we'll attempt to
203	use the 'ref' to fill the 'documentation' variable; if we reach the end
204	without any documentation, 'error' will be printed.
205	*/
206	if(num_parts)
207	{
208		switch(type_flag)
209		{
210			case "pages":
211				ref = query_page(implode(parts[0.. 1)
212				{
213					ref = query_group_member(parts[0], implode(parts[1.. 1)
214				{
215					ref = query_class_member(parts[0], implode(parts[1.. 1)
216					{
217						ref = query_group_member(parts[0], implode(parts[1.. 1)
218					{
219						ref = query_class_member(parts[0], implode(parts[1..more_color(documentation);
220		return 1;
221	}
222	else
223	{
224		printf("%s\nPassing call to 'manb' for second attempt.\n", error||"unspecified manual error");
225		return 0;
226	}
227}
228
229//query funcs
230
231///return a sorted array of our classes
232string *query_classes(){return sort_array(m_indices(class_refs), (:$1>$2:));}
233
234///return a sorted array of our groups
235string *query_groups(){return sort_array(m_indices(group_refs), (:$1>$2:));}
236
237///return a sorted array of our pages
238string *query_pages(){return sort_array(m_indices(page_refs), (:$1>$2:));}
239
240///return a sorted array of pages declared as subpages of <page>
241string *query_sub_pages(string page){return sort_array(map(query_page_details(page)["subpages"], (:$1["name"]:)), (:$1>$2:));}
242
243/**
244Returns a mapping in which keys are the refids of entities having the supplied <refid> as a parent.
245This is somewhat obtuse, but the point is that while query_sub_pages just works for pages, query_children
246works for both pages and groups. The value in the pair is an array of all elements which serve as a parent,
247as groups may be children of multiple other groups.
248*/
249mapping query_children(string refid){return filter(doc_structure, (:member($2, refid||"indexpage") > -1:));}
250
251///Returns refid if groupname exists, 0 otherwise.
252string query_group(string groupname){return group_refs[groupname];}
253
254///Returns refid if classname exists, 0 otherwise.
255string query_class(string classname){return class_refs[classname];}
256
257/**
258Returns refid if pagename exists, 0 otherwise. As a backup, this will also
259check to see if replacing spaces with underscores helps make a match.
260*/
261string query_page(string pagename)
262{
263	return page_refs[pagename] ? page_refs[pagename] : page_refs[implode(explode(pagename||"", " "), "_")];
264}
265
266///Returns the refid of <funcname> in <groupname> if both exist.
267string query_group_member(string groupname, string funcname){return group_members[groupname] ? group_members[groupname][funcname] : 0;}
268
269///Returns the refid of <funcname> in <classname> if both exist.
270string query_class_member(string classname, string funcname){return class_members[classname] ? class_members[classname][funcname] : 0;}
271
272///Returns a sorted array of the members of <classname> if it exists.
273string *query_class_members(string classname)
274{
275	return classname && sizeof(class_members[classname]) ? sort_array(m_indices(class_members[classname]), (:$1>$2:)) : ({});
276}
277
278///Returns a sorted array of the members of <groupname> if it exists.
279string *query_group_members(string groupname)
280{
281	return groupname && sizeof(group_members[groupname]) ? sort_array(m_indices(group_members[groupname]), (:$1>$2:)) : ({});
282}
283
284///Return a mapping of classes which have a member matching <membername>.
285mapping query_member_in_classes(string membername){return filter(class_members, (:member(m_indices($2), membername) > -1:));}
286
287///Return a mapping of groups which have a member matching <membername>.
288mapping query_member_in_groups(string membername){return filter(group_members, (:member(m_indices($2), membername) > -1:));}
289
290/**
291Return a mapping of details for <pagename> or an empty mapping otherwise.
292
293@note Unlike the other similar queries, this accepts the pagename, not the ref.
294*/
295mapping query_page_details(string pagename){return sizeof(page_details[pagename]) ? page_details[pagename] + ([]) : ([]);}
296
297///Return a mapping of details for member <ref> or an empty mapping otherwise.
298mapping query_member_detail(string ref){return sizeof(member_details[ref]) ? member_details[ref] + ([]) : ([]);}
299
300///Return a mapping of details for class <ref> or an empty mapping otherwise.
301mapping query_class_detail(string ref){return sizeof(class_details[ref]) ? class_details[ref] + ([]) : ([]);}
302
303///Return a mapping of details for group <ref> or an empty mapping otherwise.
304mapping query_group_detail(string ref){return sizeof(group_details[ref]) ? group_details[ref] + ([]) : ([]);}
305
306///Return a mapping of ([class name: ({brief description}) ]).
307mapping query_class_briefs(){return map(class_refs, (:class_details[$2]["brief"]:));}
308
309///Return a mapping of ([group name: ({brief description}) ]).
310mapping query_group_briefs(){return map(group_refs, (:group_details[$2]["brief"]:));}
311
312///Return a mapping of ([page name: ({title}) ]).
313mapping query_page_briefs(){return map(page_refs, (:page_details[$1]["title"]:));}
314
315//format funcs
316
317/**
318When given a <details> mapping for a member, returns a formatted string briefly
319describing it in man format.
320
321@see doxml.query_member_detail, doxml.format_member_details
322*/
323string format_member_brief(mapping details)
324{
325	string out = "";
326	mapping doc = ([]);
327	doc += (["synopsis":({details["definition"]+details["argsstring"]}) + ({implode(flatten_array(details["synopsis"]), "")})]);
328
329	string brief = implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "");
330
331	doc += (["description":trim(explode(implode(explode(brief, "\n"), " "), ".")[0])+"."]);
332	foreach(string key : key_order)
333	{
334		if(!sizeof(doc[key])){continue;}
335		out += sprintf("%s\n", upper_case(key));
336		switch(typeof(doc[key]))
337		{
338			case T_POINTER:
339				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(implode(filter(doc[key], (:$1 && sizeof($1):)), "\n"), 0, " \t\n"));
340				break;
341			case T_STRING:
342				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(doc[key], 0, " \t\n"));
343				break;
344			case T_MAPPING:
345				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(implode(m_values(map(filter(doc[key], (:$2 && sizeof($2):)), (:sprintf("%s: %s\n", $1, $2):))), "\n"), 0, " \t\n"));
346				break;
347			default:
348		}
349
350	}  //
351	return implode(map(explode(out, "\n"), (:sprintf("[color=long]%s[/color]", $1):)), "\n");
352}
353
354/**
355When given a <details> mapping for a member, returns a formatted string fully
356describing it in man format.
357
358@see doxml.query_member_detail, doxml.format_member_brief
359*/
360string format_member_details(mapping details)
361{
362	string out = "";
363	mapping doc = ([]);
364	doc += (["synopsis":({details["definition"]+details["argsstring"]}) + ({implode(flatten_array(details["synopsis"]), "")})]);
365	doc += (["examples":implode(flatten_array(details["usage"]), "")]);
366	doc += (["see also":implode(flatten_array(details["see also"]), "")]);
367	doc += (["description":implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "")]);
368	doc += (["history":implode(flatten_array(details["history"]), "")]);
369	doc += (["future":implode(flatten_array(details["todo"]), "")]);
370	doc += (["meta":(["location":sprintf("Lines %s-%s of %s.", details["startline"], details["endline"],details["file"][14..query_columns()-10, trim(implode(filter(doc[key], (:$1 && sizeof($1) && $1 != '\n':)), "\n"), 0, " \t\n"));
371				break;
372			case T_STRING:
373				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(doc[key], 0, " \t\n"));
374				break;
375			case T_MAPPING:
376
377				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(implode(m_values(map(filter(doc[key], (:$2 && sizeof($2)&& $2 != '\n':)), (:sprintf("%s: %s\n", $1, $2):))), "\n"), 0, " \t\n"));
378				break;
379			default:
380		}
381
382	}
383	return implode(map(explode(out, "\n"), (:sprintf("[color=long]%s[/color]", $1):)), "\n");
384}
385
386/**
387When given a <details> mapping for a compound (class/group),
388returns a formatted string fully describing it in man format.
389
390@see doxml.query_class_detail, doxml.query_group_detail
391*/
392string format_compound_details(mapping details, string name)
393{
394	string out = "";
395	mapping doc = ([]);
396	if(!sizeof(filter(details, (:sizeof($2):)))){return "Topic not yet documented.\n";}
397	if(name){doc += (["name":({name})]);}
398	doc += (["see also":implode(flatten_array(details["see also"]), "")]);
399	doc += (["description":implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "")]);
400	doc += (["history":flatten_array(details["history"])]);
401	doc += (["future":implode(flatten_array(details["todo"]), "")]);
402	if(details["file"])
403	{
404		doc += (["meta":(["location":sprintf("%s", details["file"][14..query_columns()-10, trim(implode(filter(doc[key], (:$1 && sizeof($1):)), "\n"), 0, " \t\n"));
405				break;
406			case T_STRING:
407				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(doc[key], 0, " \t\n"));
408				break;
409			case T_MAPPING:
410				out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, trim(implode(m_values(map(filter(doc[key], (:$2 && sizeof($2):)), (:sprintf("%s: %s\n", $1, $2):))), "\n"), 0, " \t\n"));
411				break;
412			default:
413		}
414
415	}
416	if(sizeof(details["subpages"]))
417	{
418		out += "\n\nSUBPAGES:\n";
419	}
420	foreach(mapping subpage : details["subpages"])
421	{
422		out += sprintf("%8s%=-*s\n", "",TERM_D->query_columns()-10, sprintf("%s %s\n", subpage["title"], subpage["name"]));
423	}
424	return implode(map(explode(out, "\n"), (:sprintf("[color=long]%s[/color]", $1):)), "\n");
425}
426
427///Returns a string tabular listing of function members for <groupname>.
428string format_group_members(string groupname)
429{
430	return sizeof(query_group_members(groupname)) ? "\n[color=more]MEMBERS[/color]\n[color=ref]"+implode(explode(sprintf(TABULAR_SPRINTF_FMT,tabular_adjust_cols(), implode(query_group_members(groupname), "\n")), "\n"), "[/color]\n[color=ref]")+"[/color]" : "";
431}
432
433///Returns a string tabular listing of function members for <classname>.
434string format_class_members(string classname)
435{
436	return sizeof(query_class_members(classname)) ? "\n[color=more]MEMBERS[/color]\n[color=ref]"+implode(explode(sprintf(TABULAR_SPRINTF_FMT,tabular_adjust_cols(), implode(query_class_members(classname), "\n")), "\n"), "[/color]\n[color=ref]")+"[/color]" : "";
437}
438
439///Returns a string name/brief listing of group member functions
440string format_group_members_detailed(string groupname)
441{
442	string out = "";
443	foreach(string member : query_group_members(groupname))
444	{
445		mapping details = query_member_detail(query_group_member(groupname, member));
446		string brief = implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "");
447		out += sprintf("%30s : %50=-s\n\n", member, trim(regexplode(implode(explode(brief, "\n"), " "), "\\.\\s")[0])+".");
448	}
449	return "\n[color=more]MEMBERS[/color]\n[color=more]"+implode(explode(out, "\n"), "[/color]\n[color=more]")+"[/color]";
450}
451
452///Returns a string name/brief listing of class member functions
453string format_class_members_detailed(string classname)
454{
455	string out = "";
456	foreach(string member : query_class_members(classname))
457	{
458		mapping details = query_member_detail(query_class_member(classname, member));
459		string brief = implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "");
460		out += sprintf("%30s : %50=-s\n\n", member, trim(regexplode(implode(explode(brief, "\n"), " "), "\\.\\s")[0]));
461	}
462	return "\n[color=more]MEMBERS[/color]\n[color=more]"+implode(explode(out, "\n"), "[/color]\n[color=more]")+"[/color]";
463}
464
465///Returns a detailed string listing of all pages/titles.
466string format_pages_detailed()
467{
468	string out = "";
469	foreach(string member : query_pages())
470	{
471		mapping details = query_page_details(member);
472		out += sprintf("[color=ref]%s[/color]:\n%3s%50=-s\n\n", member, "", details["title"]||"");
473	}
474	return "\n[color=more]PAGES[/color]\n[color=more]"+implode(explode(out, "\n"), "[/color]\n[color=more]")+"[/color]";
475}
476
477///Returns a detailed string listing of all groups/briefs.
478string format_groups_detailed()
479{
480	string out = "";
481	foreach(string member : query_groups())
482	{
483		mapping details = query_group_detail(query_group(member));
484		string brief = implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "");
485		out += sprintf("[color=ref]%s[/color]:\n%3s%50=-s\n\n", member, "", trim(regexplode(implode(explode(brief, "\n"), " "), "\\.\\s")[0]));
486	}
487	return "\n[color=more]GROUPS[/color]\n[color=more]"+implode(explode(out, "\n"), "[/color]\n[color=more]")+"[/color]";
488}
489
490///Returns a detailed string listing of all classes/briefs.
491string format_classes_detailed()
492{
493	string out = "";
494	foreach(string member : query_classes())
495	{
496		mapping details = query_class_detail(query_class(member));
497		string brief = implode(flatten_array(details["brief"]), "") + implode(map(flatten_array(details["detail"]), (:mappingp($1) ? sprintf("\n\n%s%s", implode(m_indices($1), ""), implode(flatten_array(m_values($1)), "")): $1:)), "");
498		out += sprintf("[color=ref]%s[/color]:\n%3s%50=-s\n\n", member, "", trim(regexplode(implode(explode(brief, "\n"), " "), "\\.\\s")[0]));
499	}
500	return "\n[color=more]CLASSES[/color]\n[color=more]"+implode(explode(out, "\n"), "[/color]\n[color=more]")+"[/color]";
501}
502
503///Returns a tabular string listing of all pages.
504string format_pages()
505{
506	return "\n[color=more]PAGES[/color]\n[color=ref]"+implode(explode(sprintf(TABULAR_SPRINTF_FMT,tabular_adjust_cols(), implode(query_pages(),"\n")), "\n"), "[/color]\n[color=ref]")+"[/color]";
507}
508
509///Returns a tabular string listing of all groups.
510string format_groups()
511{
512	return "\n[color=more]GROUPS[/color]\n[color=ref]"+implode(explode(sprintf(TABULAR_SPRINTF_FMT,tabular_adjust_cols(), implode(query_groups(),"\n")), "\n"), "[/color]\n[color=ref]")+"[/color]";
513}
514
515///Returns a tabular string listing of all classes.
516string format_classes()
517{
518	return "\n[color=more]CLASSES[/color]\n[color=ref]"+implode(explode(sprintf(TABULAR_SPRINTF_FMT,tabular_adjust_cols(), implode(query_classes(), "\n")), "\n"), "[/color]\n[color=ref]")+"[/color]";
519}
520
521/**
522Returns a string similar to format_compound_details, but customized for pages.
523
524@note Only requires 'pagename' as input rather than full details mapping.
525*/
526string format_page(string pagename)
527{
528	mapping deets = query_page_details(pagename);
529	if(!sizeof(deets))
530	{
531		return 0;
532	}
533	else
534	{
535		string out = "";
536		out += deets["title"];
537		out += unpack_description(deets["detail"]);
538		return implode(map(explode(out, "\n"), (:sprintf("[color=long]%s[/color]", $1):)), "\n");
539	}
540}
541
542/**
543Given a topic as <search>, and the current type (class/group/page),
544will see first if the same topic exists in other categories, and
545also if the topic exists as a member in any other classes.
546
547@todo This should probably also suggest possible matches if we
548hit a threshold in getting close to a longer match, especially given
549our current page nomenclature which is cumbersome.
550*/
551string format_others(string search, string current)
552{
553	string other_info = "";
554	mapping check = check_others(search)-([current]);
555	if(sizeof(check))
556	{
557		other_info += sprintf("%8s%=-*s\n", "", TERM_D->query_columns()-10, sprintf("The topic '%s' also exists in these categories:\n%s", search, sprintf(TABULAR_SPRINTF_FMT, tabular_adjust_cols(), implode(m_indices(check), "\n"))));
558	}
559	other_info += format_member_in_groups(search);
560	other_info += format_member_in_classes(search);
561	return sizeof(other_info) ? "\n[color=more]ELSEWHERE[/color]\n[color=more]"+implode(explode(other_info, "\n"), "[/color]\n[color=more]")+"[/color]" : "";
562}
563
564/**
565Returns a formatted string listing other classes with a member sharing names
566with <membername>. These may or may not be related.
567*/
568string format_member_in_classes(string membername)
569{
570	return sizeof(query_member_in_classes(membername)) ? sprintf("%8s%=-*s\n", "", TERM_D->query_columns()-10, sprintf("The topic '%s' is also a member of the following classes:\n%s", membername, sprintf(TABULAR_SPRINTF_FMT, tabular_adjust_cols(), implode(sort_array(m_indices(query_member_in_classes(membername)), (:$1>$2:)), "\n")))) : "";
571}
572
573/**
574Returns a formatted string listing other groups with a member sharing names
575with <membername>. These may or may not be related.
576*/
577string format_member_in_groups(string membername)
578{
579	return sizeof(query_member_in_groups(membername)) ? sprintf("%8s%=-*s\n", "", TERM_D->query_columns()-10, sprintf("The topic '%s' is also a member of the following groups:\n%s", membername, sprintf(TABULAR_SPRINTF_FMT, tabular_adjust_cols(), implode(sort_array(m_indices(query_member_in_groups(membername)), (:$1>$2:)), "\n")))) : "";
580}
581
582//read funcs
583
584///Given a refid <ref>, will print the member's body if it exists.
585void read_function(string ref)
586{
587	int start_line = to_int(member_details[ref]["startline"]);
588	int end_line = to_int(member_details[ref]["endline"]);
589	int read_lines = end_line - start_line + 1;
590	string func = read_file(member_details[ref]["file"][14.. 0)
591		{
592			start_line--;
593			read_lines++;
594			string new_line = read_file(member_details[ref]["file"][14..more(func, 0, 0, 1, 0, start_line);
595
596}
597
598///Print's the file associated with a given <ref>.
599void read_class(string ref)
600{
601	return MORE_D->more_file(class_details[ref]["file"][14..more_color(format_others(search, current));
602}
603
604///Send the output of doxml.format_page through more_d.more_color
605void print_page(string pagename)
606{
607	return MORE_D->more_color(format_page(pagename));
608}
609
610///Send the output of doxml.format_compound_details (for a group) through more_d.more_color
611void print_group(string groupname)
612{
613	return MORE_D->more_color(format_compound_details(query_group_detail(query_group(groupname)), groupname));
614}
615
616///Send the output of doxml.format_compound_details (for a class) through more_d.more_color
617void print_class(string classname)
618{
619	return MORE_D->more_color(format_compound_details(query_class_detail(query_class(classname)), classname));
620}
621
622///Send the output of doxml.format_classes through more_d.more_color
623void print_classes()
624{
625	return MORE_D->more_color(format_classes());
626}
627
628///Send the output of doxml.format_groups through more_d.more_color
629void print_groups()
630{
631	return MORE_D->more_color(format_groups());
632}
633
634///Send the output of doxml.format_pages through more_d.more_color
635void print_pages()
636{
637	return MORE_D->more_color(format_pages());
638}
639
640///Send the output of doxml.format_classes_detailed through more_d.more_color
641void print_classes_detailed()
642{
643	return MORE_D->more_color(format_classes_detailed());
644}
645
646///Send the output of doxml.format_groups_detailed through more_d.more_color
647void print_groups_detailed()
648{
649	return MORE_D->more_color(format_groups_detailed());
650}
651
652///Send the output of doxml.format_pages_detailed through more_d.more_color
653void print_pages_detailed()
654{
655	return MORE_D->more_color(format_pages_detailed());
656}
657
658///Send the output of doxml.format_class_members through more_d.more_color
659void print_class_members(string classname)
660{
661	return MORE_D->more_color(format_class_members(classname));
662}
663
664///Send the output of doxml.format_group_members through more_d.more_color
665void print_group_members(string groupname)
666{
667	return MORE_D->more_color(format_group_members(groupname));
668}
669
670///Send the output of doxml.format_class_members_detailed through more_d.more_color
671void print_class_members_detailed(string classname)
672{
673	return MORE_D->more_color(format_class_members_detailed(classname));
674}
675
676///Send the output of doxml.format_group_members_detailed through more_d.more_color
677void print_group_members_detailed(string groupname)
678{
679	return MORE_D->more_color(format_group_members_detailed(groupname));
680}
681
682///Send the output of doxml.format_compound_details through more_d.more_color
683void print_compound_details(mapping details, string name)
684{
685	return MORE_D->more_color(format_compound_details(details, name));
686}
687
688///Send the output of doxml.format_member_brief through more_d.more_color
689void print_member_brief(mapping details)
690{
691	return MORE_D->more_color(format_member_brief(details));
692}
693
694///Send the output of doxml.format_member_details through more_d.more_color
695void print_member_details(mapping details)
696{
697	return MORE_D->more_color(format_member_details(details));
698}
699
700///Send the output of doxml.format_member_in_classes through more_d.more_color
701void print_member_in_classes(string membername)
702{
703	return MORE_D->more_color(format_member_in_classes(membername));
704}
705
706///Send the output of doxml.format_member_in_groups through more_d.more_color
707void print_member_in_groups(string membername)
708{
709	return MORE_D->more_color(format_member_in_groups(membername));
710}

This more or less covers the basics of our implementation. The next section will be discussing some planned upgrades and will probably be more theoretical and less code oriented but nonetheless will probably be of interest for those looking at implementing Doxygen on an LDMud.

Discussing this elsewhere?
Enter 'link' for a markdown link
or 'tweet <message>' for a pre-populated Tweet :)
Want to subscribe? Enter 'rss' for a feed URL.
>