Documentation, Documentation, Documentation

Rejected ideas for this blogpost:
a) A meditation on the validity or otherwise of an oft-repeated quote attributed to Dick Brandon on the utility of documentation, whilst delicately avoiding the object of the quote’s comparison.
b) A rehash of Tony Blair’s 2001 speech on education, with the word education replaced everywhere by the word documentation, Labour with LiveCode, etc etc, you see where I’m going with that.
c) An entire blog post written in the style of the documentation format, with some kind of tortured wordplay involving the words “meta” and “documentation” as the title.

Hopefully I made a good decision to reject the above, although as you can see, remnants of those ideas have wormed their way in to this post regardless.

It’s great to hear about various community efforts being organised to improve LiveCode’s User Guide, and translate it into various languages. Here at LiveCode we are continuing to try and find ways to empower our users both in terms of what it’s possible to do with LiveCode, and also ways to contribute to it. Elanor has already written about the user guide in this respect;, and tt would be great if something similar could be done to make it easier to contribute the LiveCode Dictionary.

Currently the dictionary files are in XML, which means they are rather opaquely swathed in tags. Some of these tags are meant to be rendered directly as HTML, while others are merely structural markings. Indeed, some of the older dictionary files contain extremely convoluted passages such as the following:

<description>Use the <b>accept</b> <glossary tag="command">command</glossary> when running a <glossary tag="server">server</glossary>, to accept <glossary tag="TCP">TCP</glossary> connections or <glossary tag="UDP">UDP</glossary> <glossary tag="datagram">datagrams</glossary> from other systems (or other <glossary tag="process">processes</glossary> on the same system).<p>Use the datagram option if you want to accept UDP datagrams.</p><p></p><p><b>Parameters:</b></p><p>The <i>portNumber</i> is the <glossary tag="TCP">TCP</glossary> <glossary tag="port">port</glossary> number on which to accept connections.</p><p>The <i>callbackMessage</i> is the name of a <keyword tag="message box">message</keyword> to be sent when a connection is made or a <glossary tag="datagram">datagram</glossary> is received.</p><p></p><p><b>Comments:</b></p><p>When a connection is made or a datagram is received, the <b>accept</b> <glossary tag="command">command</glossary> creates a new <glossary tag="socket">socket</glossary> that can be used to communicate with the other system (or <glossary tag="process">process</glossary>). When using the <command tag="close socket">close socket</command>, <command tag="read from socket">read from socket</command>, or <command tag="write to socket">write to socket</command> <glossary tag="command">commands</glossary>, you can refer to this <glossary tag="socket">socket</glossary> with a socket identifier that looks like this:</p><p><i>host</i>:<i>port</i>[|<i>connectionID</i>]</p><p>where the <i>connectionID</i> is a number assigned by the <b>accept</b> <glossary tag="command">command</glossary>. (You only need to specify the connection number if there is more than one <glossary tag="socket">socket</glossary> connected to a particular <glossary tag="port">port</glossary> and <glossary tag="disabled">host</glossary>.)</p><p></p><p>The <i>callbackMessage</i> is sent to the <glossary tag="object">object</glossary> whose <property tag="script">script</property> contains the <b>accept</b> <glossary tag="command">command</glossary>. Either one or two <glossary tag="parameter">parameters</glossary> are sent with this <keyword tag="message box">message</keyword>. The first <glossary tag="parameter">parameter</glossary> is the <glossary tag="IP address">IP address</glossary> of the system or <glossary tag="process">process</glossary> making the connection. If a <glossary tag="datagram">datagram</glossary> is being accepted, the second <glossary tag="parameter">parameter</glossary> is the contents of the <glossary tag="datagram">datagram</glossary>.</p><p></p><p>For technical information about sockets, see RFC 147 at <<u>http://www.ietf.org/rfc/rfc147.txt</u>>.</p><p>For technical information about UDP datagrams, see RFC 768 at <<u>http://www.ietf.org/rfc/rfc0768.txt</u>>.</p><p>For technical information about the numbers used to designate standard ports, see the list of port numbers at <<u>http://www.iana.org/assignments/port-numbers</u>><i>,</i> in particular the section entitled "Well Known Port Numbers".</p></description>

…which can’t really be easily understood without, for example, displaying in a browser, or a field using the htmltext property. All of which makes it rather difficult to read and edit, and somewhat fragile too. So I spent some time recently thinking about how we could make dictionary files that are readable, extensible, and easy to adjust. Here is an example of the proposed format, again for the accept command:

Name: accept

Type: command

Syntax: accept [datagram] connections on port <number> with message <callbackMessage> 

Summary: Accepts an internet connection and creates a <socket> for that connection.

Introduced: 1.0

OS: mac,windows,linux

Platforms: desktop,server,web

Security: network

Example:
accept connections on port 80 with message "connectionMade"

Example:
accept datagram connections on port 80 with message "connectionMade"

Example:
on mouseUp
   accept connections on port 80 with message "connectionMade"
end mouseUp

on connectionMade pIPAddress
   put "Connection made:" && pIPAddress
end connectionMade

Parameters:
number: 
callbackMessage: The name of a message to be sent when a connection is made or a datagram is received.
portNumber: The TCP port number on which to accept connections.

Description:
Use the <accept> <command> when running a <server>, to accept <TCP> connections or <UDP> <datagram|datagrams> from other systems (or other <process|processes> on the same system).
Use the datagram option if you want to accept UDP datagrams.

When a connection is made or a datagram is received, the <accept> <command> creates a new <socket> that can be used to communicate with the other system (or <process>). When using the <close socket>, <read from socket>, or <write to socket> <command|commands>, you can refer to this <socket> with a socket identifier that looks like this:
host:port[|connectionID]
where the connectionID is a number assigned by the <accept> <command>. (You only need to specify the connection number if there is more than one <socket> connected to a particular <port> and <host>.)

The <callbackMessage> is sent to the <object> whose <script> contains the <accept> <command>. Either one or two <parameter|parameters> are sent with this <message>. The first <parameter> is the <IP address> of the system or <process> making the connection. If a <datagram> is being accepted, the second <parameter> is the contents of the <datagram>.

For technical information about sockets, see RFC 147 at http://www.ietf.org/rfc/rfc147.txt.
For technical information about UDP datagrams, see RFC 768 at http://www.ietf.org/rfc/rfc0768.txt.
For technical information about the numbers used to designate standard ports, see the list of port numbers at http://www.iana.org/assignments/port-numbers, in particular the section entitled "Well Known Port Numbers".

References: HTTPProxy (property), script (property), read from socket (command), write to socket (command), close socket (command), open socket (command), openSockets (function), hostAddressToName (function), hostName (function), hostAddress (function), peerAddress (function), hostNameToAddress (function), datagram (glossary), IP address (glossary), TCP (glossary), port (glossary), command (glossary), socket (glossary), UDP (glossary), host (glossary), server (glossary), message (glossary), parameter (glossary), process (glossary), object (object)

Tags: networking

There are really only two syntactic features of this docs format. The first is element specification, which is just done using a colon, e.g. Name: or Type:. The second is a general linking markup – enclosing text in < and >. Hopefully you’ll agree that the latter is much more legible, and editable to boot. Moreover, it points up an inconsistency in the original file, namely that there is a variable called ‘number’ specified by the syntax, but the associated parameter being described is ‘portNumber’. Converting the xml files into this new format has uncovered all of these inconsistencies.

The idea is that when we are building a new version of LiveCode, these docs files are parsed into an array which can be displayed by the dictionary in a CEF Browser object:

I created a LiveCode highlight.js build, so that we could display LiveCode script in the colours familiar to you from the script editor. You can see the result of that in the code snippets in the above image.

You will also be able to document LiveCode script using exactly the same syntax, and autogenerate an API for it, if for example you are writing a library for distribution. The idea would be that you could add tabs in your dictionary viewer for each of the stacks that is currently being used as a library, as a handy reference tool.

Much of this is in the early stages at the moment, so feedback is very welcome. If this or something like this does become the format of the dictionary files, we’ll immediately be seeking community help to improve our documentation. One of the primary objectives from my point of view would be to improve the examples. There are many examples in the dictionary which refer to not-necessarily-existant objects, or which rely on variables initialised outside of the example. It would be good if, as far as possible, the examples were self contained, and runnable; then they would hopefully be much more useful. I’d also like every parameter of the syntax to have as full a description as possible.

Another important step we are taking is to include a table in every object’s dictionary entry, containing all of the messages and properties associated with that object. Here, for example is a snippet of how the image object’s new entry would look:

Let me know your thoughts in the comments.