Neil's Place

August 26, 2005

4:20 PM How Templates Work XXXIX - More on RDF Modifications

Often, a new RDF triple is created in the datasource which would only affect a template rule's bindings. Since the bindings section of a rule specifies conditions that may optionally match, the addition or removal or this RDF data would never be able to add or remove a new result. At the very most, the change would cause a label to be filled in with a value, or cleared when removing an RDF triple.

As described earlier, the builder first scans the conditions part of a rule to see if it would cause a change. After this, the bindings are examined. This is done whether the conditions produced a new result, removed one, or the content was not affected, since a binding could have affected any existing results. It's possible, for instance, for every existing row to be affected by a single triple being added to the datasource. Consider the following binding:

<binding subject="?start"
            predicate="http://www.xulplanet.com/rdf/categoryName"
            object="?name"/>

This binding involves a triple pointing out from the starting variable that has been used in these examples. The value for this binding will be the same for every result, so if the category name changes, every result will need to change. However, the builder can use a much simpler process for recalculating the results. Instead of regenerating the content for a result, the builder just looks for attribute values that involve the ?name variable. Those attributes are just recomputed, substituting the new value for ?name instead. This process is repeated for each result that would be affected.

When a template involves multiple rules, the same process is used for each rule as with one rule. As when generating the results initially, only the highest matching rule needs to be applied. The only extra complication to deal with in the multiple rule case is when a particular result's member resource already matches a rule, yet the new RDF triple would cause an earlier rule to match. Since the earlier rule takes precedence, the builder handles this by removing the old content first and then adding the new content.

Next we will look at how the template builder determines where to insert content.

Comments ( 0 )

August 22, 2005

6:55 PM XULPLanet Status

By the way, in case you hadn't noticed, XULPlanet has returned after a brief hiatus. If you're following the template guide, you may have missed the most recent part.

In other news, the element reference has been updated with new information for Mozilla 1.8/Firefox 1.5.

Comments ( 0 )

August 18, 2005

8:03 PM How Templates Work XXXVIII - Adding RDF Triples

Let's say we've just added the triple below to the datasource.

subject: http://www.xulplanet.com/ndeakin/images/t/obelisk.jpg
predicate: http://purl.org/dc/elements/1.1/description
object: One of the thirty or so Egyptian obelisks

The template builder will be notified through the RDF observer mechanism of the change. The template builder will need to check all of the rules to see if this triple could cause a change in what would be displayed. If the triple wouldn't cause any change in the output, the builder won't make any changes. If the output would change, the builder will need to adjust the output, either by adding a new result, removing an old result, or by changing the value of some part of the result. The builder is smart enough to only change what needs to be changed and leave the remaining parts alone. Let's assume we have single rule with conditions as follows:

<conditions>
  <content uri="?start"/>
  <member container="?start" child="?photo"/>
  <triple subject="?photo"
             predicate="http://purl.org/dc/elements/1.1/title"
             object="?title"/>
  <triple subject="?photo"
             predicate="http://purl.org/dc/elements/1.1/description"
             object="?description"/>
</conditions>

These conditions will cause any photos with both a title and a description to be displayed. Assuming that the 'obelisk' photo doesn't have a description already, adding the triple listed above should cause a new result to be available for this photo. The builder scans through the conditions one by one.

The <content> tag can safely be skipped at this part of the process, so the builder moves onto the <member> condition. This type of condition can only cause a change when an item is being added or removed from a container. Since this is a new RDF triple that isn't an addition or removal from a container, this condition can be skipped. Effectively, if the result generation process was to evaluate this member condition, the same output would be supplied for the ?photo variable whether the new data is there or not. Thus, the member condition can be skipped.

The next condition is a <triple> involving the "http://purl.org/dc/elements/1.1/title" predicate. We aren't adding a arc involving this triple so we can ignore this conditon as well. The second triple, however, could cause a change, since the predicate attribute matches the predicate being added. The subject and object are variables so the builder accepts this as a possible change, and moves on to the next step. If the predicate was different, the builder would come to the end of the conditions and could just stop there. For instance, if the predicate of the triple being added was "http://purl.org/dc/elements/1.1/date", the builder could ignore it since the template doesn't even care about the date field. Similarly, if the triple didn't use a variable but a static value, this value would also need to match in order to continue processing.

Now that we know the conditions could cause a change in the template, the second step is to fill in the variables for this condition for what could potentially be a new result. In this situation, it fills in the ?photo and ?description variables using the values from the newly added triple.

(?photo = http://www.xulplanet.com/ndeakin/images/t/obelisk.jpg,
 ?description = 'One of the thirty or so Egyptian obelisks')

Next, the builder works its way backwards through the rules, in order to fill in the remaining variables. It does this in a similar manner as it does when it generates results, but traverses the rules in the opposite order. The previous triple will fill in a value for the ?title variable, since we now have a value for the ?photo variable referred to by the triple's subject attribute. Next, the <member> condition is examined, and, in this situation, the builder fills in the known ?photo variable, and looks for a parent container containing this value. There is a container "http://www.xulplanet.com/rdf/myphotos", so the ?start variable will be filled in with this value. Now, the potential result so far is:

(?photo = http://www.xulplanet.com/ndeakin/images/t/obelisk.jpg,
 ?description = 'One of the thirty or so Egyptian obelisks',
 ?start = http://www.xulplanet.com/rdf/myphotos,
 ?title = 'Obelisk')

As you can see, the result looks to have all the information necessary to create a new item in the output. If a condition hadn't generated a result, for instance if the photo did not have a title, or it wasn't contained in a parent container, there would be no match and the builder could stop processing the new triple. For instance, we might have added a description for a new photo, but haven't added the photo to the container resource. Once we do add it to the container with another RDF assertion, the process described above is applied again and this time it may match.

There are still two more things to do before a result is accepted as a new match. First, once the builder reaches the <content> condition, it checks what the container or reference variable is, in this case ?start, as specified by the uri attribute. The calculated value for the potential new match is "http://www.xulplanet.com/rdf/myphotos". The builder looks to see if this resource is being used as the stating point in the template. As it happens, this resource is being used, since it is the value of the ref attribute we've been using in these examples. This would also be the case for any starting points used in recursive generation. If the calculated ?start variable was something different, naturally we don't need to change the template output, as that resource isn't being used in a template.

Finally, the builder processes any conditions below the one we started at, in order to fill in any remaining variables. In this case, there are no other conditions, so the builder accepts this result as a new match. Since all the variables have been filled in, the action body for the rule can be processed and a new block of content generated and inserted into the output. We'll find out how the builder determines where to insert the new content is an upcoming section. However, this does show that the template builder can update the output upon changes without rebuilding the entire template.

When an unassertion occurs, or data is removed from the datasource, a different process is used. In this case, the builder looks at the results and determines which ones to remove. When it had first generated the results, the builder stored extra information to specify what parts of the graph were navigated over. It uses this information to help determine what results are no longer needed.

We'll look a bit more at how RDF changes affect the template next.

Comments ( 1 )

August 17, 2005

8:32 PM How Templates Work XXXVII - RDF Modifications

The third type of observer involved in a template builder is an nsIRDFObserver. The template builder implements this interface to listen for RDF modifications. When the datasource is modified, the datasource will notify any observers of the change. The template builder uses these notifcations to update the template as necessary based on the new or removed information. You don't need to implement this observer yourself, although you may add an observer to the datasource if you want to be notified when the data changes.

There are two main situations when the notifications are made. The first is when the modification functions on the datasource are called. There are four such functions: 'Assert', to add a new triple (or arrow) to the RDF graph, 'Unassert' to remove a triple, 'Change' to adjust the target of a triple, and 'Move' to adjust the source of a 'triple'. For Mozilla's datasources, the latter two just Unassert the old triple and add a new one, creating the effect of changing the value. However, only one notification is made.

For instance, an Assert call looks like the following:

var source = RDF.GetResource("http://www.xulplanet.com/ndeakin/images/t/obelisk.jpg");
var predicate = RDF.GetResource("http://purl.org/dc/elements/1.1/description");
var target = RDF.GetLiteral("One of the thirty or so Egyptian obelisks");
datasource.Assert(source, predicate, target, true);

The Assert call adds a new triple to the RDF datasource. When this happens, any templates observing the datasource will be notified via the RDF observer's onAssert method.

The second situation when notifications are made is when a datasource is being loaded or reloaded. Actually, internally, this isn't any different than the other notifications, but it is worth discussing separately. When the RDF parser loads RDF/XML, it starts with a new empty datasource, and as the parser parses the input data, it calls the datasource's Assert function to add each found triple. In effect, this isn't any different than adding the same set of triples yourself using the Assert method.

When reloading a datasource, you might think that the RDF parser removes all the existing data, loads the new data, and adds it to the datasource. Or, you might think that it creates a fresh datasource with the new data. Actually, the parser does something smarter. When reloading a datasource, it keeps the existing RDF triples intact, and only modifies the datasource based on what has changed. When parsing, any triples that already exist are not added again. If a triple does not exist yet, it will be added. Any triples that don't exist in the new data but were there before are removed. This means that the observer will be called only for the triples that differ between the new and old version of the data. If the reloaded datasource hasn't changed, the builder won't receive any notifications. This saves a lot of extra work.

The RDF observer also has two methods onBeginUpdateBatch and onEndUpdateBatch. These are called when performing a lot of operations on a datasource. When changing the datasource, the changes are surrounded by begin and end batch calls. Then, rather than notify on every change, the datasource will send one notification when the changes are finished. The template builder then rebuilds the template completely when done. This is useful when making a large amount of changes to avoid having to keep recalculating parts of the template that might change again quickly.

We'll look at some specifics of how the template builder handles changes next.

Comments ( 0 )

August 15, 2005

7:46 PM How Templates Work XXXVI - Tree Builder Listeners

The second type of listener is used to handle particular actions related to trees. The tree builder implements the nsITreeView interface, so handles the gathering of data and passing it on to the tree. The tree widget informs the view when certain operations are performed that might affect the data. The tree view handles all of these operations, but allows an observer to be attached which is invoked during these operations. For instance, the observer may have an onToggleOpenState method which will be called when the user opens or closes a row. The tree builder will handle the adding or removing of rows, but will call the observer so that it can perform some task.

The tree builder observer implements the nsIXULTreeBuilderObserver interface and may be attached to a tree builder using the builder's addObserver method. You can add more than observer if needed, and can remove them again with the builder's removeObserver method.

The observer is always invoked before the appropriate operation is performed. For instance, the onToggleOpenState method of any observers will be called before the tree item is opened. After the observers have finished, the tree builder opens the row and adds any child rows inside. Note that you cannot cancel the operation from within the observer.

Some useful functions of the observer are the drag and drop related callbacks to handle when an item is dragged onto the tree. This makes handling dragging onto a tree fairly simple. All you need to do is implement two methods, canDrop and onDrop. Note that in Firefox 1.0 (Mozilla 1.7) and earlier, the drag functions are slightly different. There, three functions are used, canDropOn, canDropBeforeAfter and onDrop. The two 'can' functions were combined into one with an extra argument. If you want to support both earlier and newer releases, you can implement all of the functions in the observer, sharing code as necessary.

The tree observer receives drag related events in three places: over a container row, before a row, and after a row. This allows you to handle dragging with more flexibility. For example, in some situations you may want to require dragging onto a folder type of row. In other situations, you may wish to allow items to be dragged between (before or after) rows. This would be the situation if you were dragging items from that tree around, for instance dragging a bookmark from one location to another. The tree widget will draw a small line between the rows while dragging. All you need to do is add a tree builder observer which returns true for the canDrop method. Note that the 'drag on' case only allows dragging onto containers, not ordinary rows.

var treeBuilderObserver = {
  canDropBeforeAfter : function(idx, orient) { return false; },
  canDropOn : function(idx, orient) { return true; },
  canDrop : function(idx, orient) { return !orient; },
  onDrop : function(idx, orient) {
    // do something here
  },
};
tree.builder.addObserver(treeBuilderObserver);

This observer implements both the older and newer methods and only allows dragging on rows. The canDropBeforeAfter method returns false since we do not want to allow before and after drops. The canDropOn method returns true however. The Mozilla 1.8 method canDrop checks the orientation and returns the opposite. This works as the 'on' value is 0 and the 'between' values are -1 and 1. Obviously, this code is much simpler than what we would really want to use -- we should be checking what is being dragged to make sure that it is compatible with the tree. Or, we might want to allow dropping on specific rows only; the drop methods are supplied with an index argument so we can check for this.

Comments ( 1 )

August 14, 2005

12:07 AM How Templates Work XXXV - Template Listeners

There are several listeners (or observers) used during the template build process, each used for different purposes. These each implement a different XPCOM interface, as listed below:

The first of these is the simplest and involves two methods, willRebuild and didRebuild. You would implement this object with these two methods if you wish to be notified when the template is rebuilt using the builder's rebuild call. The template builder might also force a rebuild when the underlying data change notifications require it. The primary use of this listener is to store some state before the template is rebuilt and restore it afterwards. Recall that when a template is rebuilt, all of the existing content will be removed and generated fresh. The willRebuild method of any listeners will be called before the content is removed, and didRebuild method will be called when the content has been regenerated. This listener will also work for tree builders, and will call the appropriate methods before and after the tree has been generated.

To assign a builder listener to a builder, use the addListener method.

var someListener = {
  item: null,
  willRebuild : function(builder) {
    this.item = builder.getResourceAtIndex(builder.root.currentIndex);
  },
  didRebuild : function(builder) {
    if (this.item) {
      var idx = builder.getIndexOfResource(this.item)
      if (idx != -1) builder.root.view.selection.select(idx);
    }
  }
};
tree.builder.addListener(someListener);

This example is very simple and just saves and restores the selected index after a rebuild. Since the content goes away during a rebuild, the selection is lost, so it is restored here during the didRebuild method. The Firefox bookmarks window uses this technique. If you try an example using the code above, you will notice that the first tree will maintain the selection when the Rebuild button is pressed, whereas in the second tree does not. This is because the listener is only attached to the first tree.

The example above makes use of the getResourceAtIndex and getIndexOfResource methods. These two methods are available for tree builders and will convert between an index in the tree and the associated member resource for the item at the index. Naturally, we can't store the index as the item may have moved its position. Or, the resource may no longer be part of the results, which is why we need to check the return value of the getIndexOfResource method. (As this example uses the RDF resources directly, it requires elevated privileges so you will need a chrome URL to test it.)

You might also guess that the builder's root property, which is used above, refers to the tree. In a content builder, it will return the element with the datasources attribute, which in the template builder is referred to as the root element.

Finally, you can remove a listener using the builder's removeListener method.

Comments ( 0 )

August 9, 2005

8:00 PM How Templates Work XXXIV - Rebuilding Templates

The datasource associated with the template can be retrieved using the element's "database" property. It implements the nsIRDFCompositeDataSource interface. Since this is a composite datasource, it may contain more than one datasource. These may be listed in the datasources attribute separated by spaces. For example:

<vbox datasources="template-guide-photos5.rdf template-guide-streets.rdf">

Sometimes, you will want to calculate the datasource to be used and attach it to the template later. You can do this using the composite datasource's AddDataSource method. You can add as many datasources as you wish, and you can remove them using the RemoveDataSource method. A common pattern is to use the following:

var RDF = Components.classes["@mozilla.org/rdf/rdf-service;1"].
            getService(Components.interfaces.nsIRDFService);
var ds = RDF.GetDataSource("http://www.xulplanet.com/ndeakin/tests/xul/template-guide-streets.rdf");
var tree = document.getElementById("theTree");
tree.database.AddDataSource(ds);
tree.builder.rebuild();

This is the typical way to add a datasource to an element, in this case to the tree with the id "theTree". The datasource is retrieved using the RDF service's GetDataSource method. After adding a datasource, the tree builder's rebuild method is invoked to rebuild the template with the new data. This doesn't happen automatically when you add the datasource, which is useful, since you will often want to add or remove other datasources at the same time.

The composite datasource can only be accessed from privileged code, regardless of what datasources it contains. Fortunately in this situation, you can just set the datasources attribute (or the corresponding property) to the datasources that you want. For instance:

var tree = document.getElementById("theTree");
tree.datasources = "template-guide-photos5.rdf template-guide-streets.rdf";

This will also change the datasources used. In this case, the template will be rebuilt automatically as you can set all of the datasources at once with this method of changing datasources. So you don't need to call the rebuild method. You can also do the same with the ref attribute (or the ref property) and the template will be rebuilt automatically. In the example above, there will be two datasources attached to the tree. They will both be loaded and the template reconstructed. Note that if one of the datasources is already loaded it will not be loaded again. This is convenient when you want to simply add a new datasource to an existing template without reloading existing data. If you do want to reload the data, you can call the builder's refresh method:

tree.builder.refresh();

This will reload the datasource attached to the template. If there is more than one datasource, as above, all of them will be reloaded. Due to the nature of the way templates are updated, you don't usually need to rebuild a template after a refresh call, although there may situtations where this will be necessary.

If you do plan on determining the datasources dynamically, it is common to start with an empty datasource using the special URI "rdf:null".

<tree datasources="rdf:null" ref="http://www.xulplanet.com/rdf/myphotos">

This will create a composite datasource with no datasources in it. This syntax is necessary as otherwise you wouldn't be able to specify a value for the datasources attribute, and a template builder would not be attached to the element. In a chrome context, the datasource rdf:local-store is always included even if you don't specify it. This is something to watch out for if you are going to be manipulating the composite datasource.

One more note: the datasources attribute may use either absolute or relative URLs. Relative URLs are relative to the XUL document the corresponding element is in. The RDF service's GetDataSource method however, only accepts absolute URLs. So you will need to use the full path in this situation.

Comments ( 0 )

August 6, 2005

10:48 AM How Templates Work XXXIII - Attaching the Template Builder

When inserting an element into a XUL document, the element is checked to see if it has a datasources attribute. If so, a template builder will be created for the element and attached to the element. If the element is a <tree> element and has the flags attribute set to "dont-build-content", a tree builder will be created. Otherwise, a content builder will be created. Both types of builder share much of the same code except for how they generate output to be displayed. Both types of builders implement the nsIXULTemplateBuilder interface, while the tree builder also implements the nsIXULTreeBuilder interface.

The builder associated with an element is accessible via the element's "builder" property both for content builders and for tree builders. An element that does not have a builder will have this property set to null. The processes of creating a builder for an element applies both when an element is created when the window is loaded and when an element is inserted dynamically.

Templates can only be used in XUL documents, however, there is no requirement that the templates generate XUL elements. They could also be used, for example, to generate HTML elements. This isn't a very common technique, however, here is an example of how this can be used:

<html:div id="photosList" datasources="template-guide-photos5.rdf"
          ref="http://www.xulplanet.com/rdf/myphotos"
          xmlns:html="http://www.w3.org/1999/xhtml">
  <html:h1>My Photos</html:h1>
  <template>
    <html:p uri="rdf:*"><textnode value="rdf:http://purl.org/dc/elements/1.1/title"/></html:p>
  </template>
</html:div>

This example generates three paragraphs. Some static content before the <template> element displays a <h1> header. Since templates were designed for creating XUL content, sometimes there can be unusual results when using HTML. Sometimes this is due to different whitespace handling for HTML and XUL, which is why the content to generate in the above example is all on one line. If you do plan on generating non-XUL content with a template, just watch out for issues like this. Note that this particular whitespace issue has been fixed in later Mozilla builds.

Note also that the datasources attribute has been placed on a non-HTML element. This is also allowed. However, one thing to watch out for is that non-XUL elements do not have their content generated lazily so all of the content will be generated at once. Be careful that your templates do not recurse to deep levels.

The builder property is a property of the nsIDOMXULElement interface, so all XUL elements will have this property, although, as mentioned earlier, is will be set to null for most elements. For non-XUL elements, the template builder will be assigned to a builder property on the element using a custom JavaScript property instead.

The main purpose of accessing the builder for an element is to call its "rebuild" method. This method removes any existing generated content and deletes all data in the rule network. Then, the method recompiles the rules and regenerates the content. Essentially, the rebuild method instructs the builder to remove any existing information and reconstruct it from the beginning. The only difference is that the datasource has already loaded so the data will be the same. However, this is often used if you modify the datasource or modify the rules.

The builder's refresh method, however, will reload the datasources. It will not rebuild the template, but we'll see in a later section why this is not usually necessary. To summarize, the refresh method reloads the data, whereas the rebuild method reconstructs the content.

The builder is accessible to unprivileged code, so the rebuild and refresh methods may be called by remote code.

We'll see some examples of rebuilding templates next.

Comments ( 2 )