Neil's Place

June 2, 2005

6:26 PM How Templates Work III

A XUL template is made up of series of rules. Each rule contains two things, a set of instructions for navigating through an RDF graph and some XUL content to generate. Given a starting point, the template builder will evaluate the navigation instructions for each rule and will generate a series of endpoints matching that rule. The corresponding content will then be generated. It's actually more complicated than that but this is the basic idea.

There are two syntaxes for declaring the rules, the simplified syntax and the extended (or full) syntax. As its name implies, the simplified syntax is simpler but is limited in the type of data that it can iterate over. It may be used to iterate over one and only one arc (or arrow) in the RDF graph. The extended syntax may navigate anywhere in the graph. Almost always, the simplified syntax will be used to iterate over a set of children of an RDF container like a Seq. An RDF Seq is a container that contains children in a specific order. The template will generate these results in that order.

The simplified syntax is a subset of the extended syntax, that is, it is possible to rewrite a rule that uses the simplified syntax using the extended syntax. In fact, the template builder will internally convert the simpler rule into an extended syntax rule for use. Neither syntax is preferred; you may use the extended syntax exclusively if it feels more natural to you. Note however, that the simplified syntax uses a slight optimization when using more than one rule which the extended syntax does not use. Well look at this in more detail as we look more at rules.

Each rule is declared using the <rule> tag, which you would place directly inside the <template> element. You may have as many rules as you wish, or you may have only one. When using the simplified syntax, you may omit the <rule> element when you have only one rule as its usage will be implied. The extended syntax, however, always requires the <rule> element.

Here is the outline of the template syntax so far.

<vbox datasources="http://www.xulplanet.com/ds/sample.rdf"
         ref="http://www.xulplanet.com/rdf/A" flex="1">
  <template>
    <rule>
      -- rule content goes here --
    </rule>
    <rule>
      -- rule content goes here --
    </rule>
  </template>
</vbox>

When the template builder starts processing, and after it has started the datasource loading, it first must compile the rules. This step involves working through the rules and processing them into internal structures. Thus, changing the rule elements around dynamically doesn't affect anything. However, rebuilding the template (using the builder.rebuild method) will recompile the rules and reapply the template again. This means that you can change the rules using DOM methods, rebuild the template, and get different results.

Comments ( 7 )

June 1, 2005

2:45 PM How Templates Work II

RDF is, in mathematical terms, a labeled directed graph. That means that RDF is a graph of nodes and arrows between them where each node and arrow has some label. Since it's a graph, arrows can point all over the place and nodes can have any number of arrows pointing out of them and pointing at them. And also because it is a graph, there is no real starting point or root node so you can just start anywhere. In the picture below, you can see that node A at the top has arcs pointing to B, C and D. As well, C has an arc pointing to D. You could have arcs pointing elsewhere, for example node D could have an arc pointing back to A. To navigate around, you could start at node A and navigate around the graph following the arrows to B, C or D. Or you could start at B and go to A and then go to C and D. No requirement exists to follow the arrows in the direction they point; you can easily go the other way. The picture was generated from the W3C's RDF validator, a good place to go to check if your RDF is valid.

The text in red are the labels for the arrows, called predicates. In this example, all the arrows have the same label. Usually, this won't be the case. Templates provide a means of navigating around using only arrows with specific labels. Here is the serialized RDF/XML for this graph.

<?xml version="1.0"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:rel="http://www.xulplanet.com/rdf/">
  <rdf:Description rdf:about="http://www.xulplanet.com/rdf/A">
    <rel:relatedItem rdf:resource="http://www.xulplanet.com/rdf/B"/>
    <rel:relatedItem rdf:resource="http://www.xulplanet.com/rdf/C"/>
    <rel:relatedItem rdf:resource="http://www.xulplanet.com/rdf/D"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://www.xulplanet.com/rdf/C">
    <rel:relatedItem rdf:resource="http://www.xulplanet.com/rdf/D"/>
  </rdf:Description>
</rdf:RDF>

For a XUL template query, you first need to select a starting point in the RDF graph. Once you have selected a starting point, you use a number of statements which indicate where to go next when navigating the graph. Eventually, you will end up with a set of nodes you consider the endpoints of your query. These become the results and content would be generated for each of these results. Say you start at A. You could navigate to B, C and D and generate three blocks of output. Or, you could start at D and follow two arrows back. This will get two results, A and B. Look at the graph to see if you can see why two results would be generated in this case.

In XUL template terminology, the starting point is called the container or reference point and the endpoint is called the member. It is so called because it is most common to gather the list of the members, or children, of a container. But this doesn't have to be the case. Any starting point and ending points will do.

Nodes in RDF are identified by a string value. There are two types of nodes in RDF, resources which usually represent 'things', and literals which are values like the names of things, the dates of things, the sizes of things, and so on. A literal's value is, for example, the name of the thing, such as 'Fred'. A resource's value is a URI which for your own RDF data you can just make up. We'll use the URI of the resource nodes in a template. In the image, the resource URI's are the blue labels of each node. There are no literals in this example, but we'll see some later.

Let's say we want the starting point to be A from the above example graph. We will use A's URI (http://www.xulplanet.com/rdf/A) as the reference starting point. In a XUL template, you specify the starting point using the 'ref' attribute. Here is an example:

<vbox datasources="http://www.xulplanet.com/ds/sample.rdf"
      ref="http://www.xulplanet.com/rdf/A" flex="1">

This is an indicator that we want to construct a XUL template using the reference point with the URI 'http://www.xulplanet.com/rdf/A'. Next, we'll see how to construct the template.

Comments ( 9 )

May 31, 2005

5:22 PM How Templates Work I

XUL templates are apparently difficult for many to understand. Unfortunately there are a lot of bits to have to learn to be able to use them effectively: RDF, the querying syntax, and some additional XUL features. The lack of useful logging and error messages makes things doubly difficult. Perhaps templates may be a bit easier on someone if they had an understanding of how tenplates actually work. This is part one in what will hopefully be a longer series of posts about this topic.

To begin, a XUL template is a means of producing a block of content for each result from some query or query-like operation. A good analogy is to the results of a database query. For each result returned from the query, generate some content. The template syntax allows for different rules to generate different content based on particular criteria as well as set attribute values from returned results. Some other systems call this databinding. Effectively, XUL templates are the XUL way of doing databinding. However, with a template, there are two restrictions. First, you can only generate content from the results of a query. You can't just take a single piece of data and assign it to an attribute, since templates are designed for repeated blocks of data. Second, you can only bind to an RDF datasource. These restrictions should be removed in the future.

In XUL, the RDF datasource is specified by placing the 'datasources' attribute on an element. When the XUL parser sees an element with this attribute, it constructs a template builder for the element and attaches it to the element. It is expected that there will be a template inside the element. The template builder loads the datasource, performs the query on the datasource and generates content for each result. The content is inserted into the XUL just as if you had placed it there yourself. Here is an example:

<vbox datasources="http://www.xulplanet.com/ds/sample.rdf">

This example specifies the datasource 'http://www.xulplanet.com/ds/sample.rdf'.

The template builder loads the datasource using the RDF service in the same way as you would create a datasource through the RDF service directly. The datasource is loaded via its URL. Some RDF datasources are provided with Mozilla -- their URL's start with 'rdf:'. Otherwise, the datasource is loaded as any other URL is loaded. If the datasource is already loaded and cached, the template builder can begin work right away. Otherwise, there isn't anything to do until the data is loaded. Actually, this isn't quite true. The RDF service starts the load of the datasource in the background and the template builder goes through the process of building content anyway. Naturally, since there is no data yet, no results will be available so the builder ends up building nothing.

Once some data starts arriving, the template builder scans its information to see if some results can be created. If so, some content can be generated. If not, nothing gets generated again. An interesting thing to note is that due to the nature of the RDF parsing process, the builder generates results and builds content incrementally while the data arrives. Of course, since the data arrives so quickly from the network, you really don't notice this. If the datasource is already loaded, the builder can construct content all in one step, although even this isn't completely true as we'll see later.

The template builder can also use multiple datasources which are all combined into a single datasource as if they were all in one datasource to begin with. An nsIRDFCompositeDataSource is used for this purpose. You can get this composite datasource in a script by using an element's 'database' property if you want to add or remove datasources from it.

As mentioned, the template builder loads the datasources by passing the URLs to the RDF service. However, the special URL rdf:null is used to indicate that you mean no datasources, or an empty datasource. The composite datasource will still be created but no datasources will be added to it. This is used when you need to specify the datasource dynamically with a script.

In addition, for chrome XUL (such as extensions), the datasource rdf:local-store is always included in the composite. The local store is a datasource which is usually used to hold state information such as window sizes, which columns in a tree are showing and which tree items are open. You can query for any data in the local store in a template although this is rarely done.

When multiple datasources are used the RDF is combined as if it was one large datasource. That means that a template query can grab data from anywhere in all of the datasources. This combining of datasources is often termed aggregation. This can be quite a useful feature and works regardless of the datasource. For instance, you might use the built-in bookmarks datasource which holds the user's browser bookmarks and use your own datasource to add custom data about those bookmarks.

<vbox datasources="rdf:bookmarks http://www.xulplanet.com/ds/sample.rdf">

In part two, we'll look at the ref attribute.

Comments ( 4 )

April 21, 2005

10:48 PM Update for April 21

So I took a bit of a break from the XUL templates stuff for a while. In case you didn't know, there's a patch in the bug. It would be great if someone would make a build and stick it up somewhere so people could try it out.

I've been spending much of the last month on some MDG work. It's Mozilla development naturally, but you won't ever see the results, at least not for a long time anyway. But some of you can at least feel good that you are unknowingly paying for it via taxes.

One thing about being involved in an open source project and having at least some name-recognition in that project is that you get numerous job offers. There certainly seems to be some demand building up for experienced XUL developers, of which there aren't many. I wonder if there would be interest in some XUL training sessions. I suppose that would require enough people all in one place though.

In the meantime I've been working on and off on a new tutorial, not a big one, but it should be useful. Oh, and I also posted the patch I made for the slider tag that I wrote a while ago.

I think I'll go on vacation for two weeks.

Comments ( 5 )

March 29, 2005

7:38 PM Templates and Storage

Hey look! It's a XUL template created without the use of any RDF!

Instead of grabbing the data from an RDF datasource, it performs a query on an SQLite database via mozStorage. OK, there is some RDF being used under the hood, but a template author won't need to worry about this. Here is the XUL code used (I've hard coded the database for now):

<tree datasources="some-db" ref="*" rows="10" flex="1"
      flags="dont-build-content dont-recurse">
  <treecols>
    <treecol id="name" label="Country" flex="1"/>
    <treecol id="capital" label="Capital" flex="1"/>
    <treecol id="population" label="Population" flex="1"/>
  </treecols>
  <template>
    <query>
      select * from Countries where name like '%e%'
    </query>
    <action>
      <treechildren>
        <treeitem uri="?">
          <treerow>
            <treecell label="?name"/>
            <treecell label="?capital"/>
            <treecell label="?population"/>
          </treerow>
        </treeitem>
      </treechildren>
    </action>
  </template>
</tree>

The patch to implement this will be coming soon to a bug 285631 near you.

Comments ( 35 )

March 18, 2005

7:29 PM XUL Tutorial Updates

Some significant updates to the XUL Tutorial today, including rearranging the order of the sections and a rewrite of much of the information pertaining to trees. Also, seven new pages on events, DOM scripting, box objects and tree views have been added.

Comments ( 70 )

March 16, 2005

12:09 PM Firefox books

Lots of Firefox books on the horizon:

  • Firefox and Thunderbird Garage by Chris Hofmann, Marcia Knous and John Hedtke
  • Firefox for Dummies by Blake Ross
  • Firefox Hacks by Nigel McFarlane
  • Kurt Cagle's Firefox Programming Companion by Kurt Cagle
  • Hacking Firefox : More Than X Hacks, Mods and Customizations by Mel Reyes

Comments ( 1 )

March 9, 2005

11:45 AM Die Seamonkey Die! Long Live Firefox!

Can you feel the tension is the air? I sure can. It's the point when Mozilla starts collapsing. There's a growing number of people wanting to support and maintain Seamonkey, instead of Firefox and Thunderbird. After all, if you don't like the way things are going, why not branch off and do things the way you do like? It's the open source way! It also happens to be the best way to kill off a project.

Take a look at the list of Seamonkey supporters. Lots of QA people there. Lots of non-programmers. A small handful of people who would do much of the real development work, all of which already work on Gecko. But no management. Nobody with strong leadership, nobody with the time to coordinate development and testing, nobody with the project management skills needed. Oh, and I'll bet no user interface designers either.

Can you guess who will be expected to fill this void? I'll bet you can! Why, it's the overburdened Mozilla Foundation! Surely, the cost and efforts required to maintain two products that do the same thing would be negligible? After all, they've been doing it for a while, right? Sadly, they have. Of course, there were reasons for this since the brand spanking new Firefox wasn't ready until last November. But now there isn't.

One issue is that Firefox has been marketed as a separate product, "built from the ground up" as they say. Of course, it wasn't really built from the ground up, since much of the code, including the UI code, is actually the same. But it is heavily marketed that way. And guess what? A brand new name, some great logos and a concentrated effort on pushing the strengths of Firefox went a long way. Firefox is a great brand name which may one day become one of the best, like Apple, or Google or the Olsen Twins.

However, the new image has a downside. Firefox still has it's older brother, Seamonkey, still a separate product. So guess what? People still think it's a separate product and that it should still be updated. Well, what did you think would happen when you make two similar products? In this regard, it would have made more sense to change the name from Firefox to Mozilla 1.8 and promote it as the actual upgrade from Mozilla 1.7. Would people upgrade? Sure, it's the next version. Would some people complain about the UI changes? Sure, but they'd upgrade anyway, at least eventually. Why? Because the old version is no longer maintained.

Let's look at Netscape. They've released some browsers over the last ten years too. In the latter half of that time, they've been criticized because they created bizarre colourful UI, added strange bits of UI, stuck in all kinds of links to their web site, put stuff on the toolbars that many don't desire to be there, added millions of preference panels, and popped up all kinds of dialogs and ads. But if there's one thing Netscape has managed to do right, it's getting the name of the browser right. The first version was generally referred to as Netscape 1. Then came Netscape 2, then Netscape 3, Netscape 4, Netscape 6 and then Netscape 7. See a pattern? Apart from the glitch with version 5, things seem quite sensible. Netscape has recently announced a new version of their browser. Can you guess what it's called? Get this: Netscape 8. I'm even going to go out on a limb and predict the future: If Netscape releases another version of their browser, it will be called -- are you ready for it -- Netscape 9.

Easy to tell which version to upgrade to, right? There's only one to choose from. Your only choice is upgrade, or don't. And, guess what? The UI changed significantly between most versions, much more so that the UI changes between Mozilla 1.7 and Firefox. Yet people either upgraded or didn't. No one really expects Netscape or anyone to work on the long lost Netscape 5.

Now, some will be saying they like the integrated products rather than the separate browser and mail, much like how people complained to Netscape they didn't like integrated products. Oh well, you can't please everyone. But wouldn't it be easier to construct a Firefox/Thunderbird suite? I'm no build expert, but I'd imagine that would be easier and cheaper than maintaining two separate products.

What about all those missing features in Firefox that the suite has? Well, I looked, and, ironically, I couldn't find the missing features. People keep saying that Firefox has reduced features, but don't seem to ever mention what they are. OK, let's say there are some. Wouldn't it be easier to port these "missing features" to Firefox and then have a build option to include them? Mozilla's overlay system is quite powerful. There's no need to require the user to install extensions for these things. Let's make a standard set and include them as part of a Firefox/Thunderbird suite package. This means too that the QA people should stop scaring people away by marking enhancement bugs invalid because they think the feature would be better as an extension. Sure some things might be, but I'll bet there are tons of great ideas out there that just need a better design and a bit of work and they would make great built-in features. After all, imagine what Firefox would be like if it didn't have tabbed browsing. I'd imagine someone would mark a request for it as invalid.

Those other little things you don't like about Firefox? Personal taste mostly. Don't like the Options command being on the Tools menu? Like the traditional location on the Edit menu instead? Yeah, whatever. Maybe we should put preferences in several dialogs on the Options menu for those that started out using Netscape 3. Really no reason to support two products for this kind of thing.

There's also the issues with the Firefox review and management process, which is, I suspect, the main reason the key developers on the Seamonkey supporters list are against Firefox. You see, Firefox has different reivew process than the rest of Mozilla code. Is that a good thing? Could be, but probably not. Considering one of the Firefox developer's thoughts on these kinds of things, that the Firefox module owner doesn't really do any reviews, and that the Firefox poster boy hasn't done much at all recently, I'd imagine some changes are needed.

For one, I'm quite interested in the XUL toolkit, but it seems to be intertwined with the Firefox toolkit. Would it be wrong to suggest that there be an app toolkit maintained by the application front end developers, and an underlying XUL toolkit maintained by XUL developers? The app toolkit needs good UI designers. But the XUL toolkit needs good API designers. These are different people. Let's get people good at each task onto each piece.

We need to ensure that both front end UI and back end Gecko developers are communicating with each other. Sure, the front end UI is where all the action is. This is where your work gets noticed. You might even be famous for it. But if fame is what you're after, you've come to the wrong place. But if you are a front end UI developer, you've got to let the Gecko folks do their work. Each group has its place. If you do UI, you leave the other stuff alone. There are better experts for that. But if you don't do UI, leave UI work to those who can. And, no matter how much you think you can do good UI, you're almost guaranteed to be wrong.

Firefox is targeted at normal people as opposed to developers. If you know how to create a makefile, you aren't a normal person. But you can use Firefox too. I wonder who the Seamonkey supporters are trying to target? Can't be normal people - it doesn't make sense to create two products that do the same thing for the same set of people. Corporate users? They can upgrade to a Firefox/Thunderbird suite with extras for those mysterious "missing features" all built-in, or they can stick with Mozilla 1.7. Or are they targeting themselves, all programmers and hackers? Surely not. But I bet that's the answer. It's just, well, there just aren't enough of you to really get anywhere. In fact, it's why most free and open software doesn't go anywhere. It's target audience is too small. Unless, of course, you're writing a C compiler. But don't write a C compiler! We have lots already!

One thing is clear though is that the Mozilla Foundation does need to show some leadership in this. Make some decisons, let people know what those decisons are, sit down and figure out what needs to be fixed, and work out how to fix it. Some people will be disappointed. But if people can't accept that they aren't always going to get their way, they won't get very far.

To summarize, resurrecting Seamonkey is absolutely the wrong thing to do. Instead, let's not be a burden on the Mozilla Foundation and let's build on top of one product, a product we can all get behind. I recently upgraded to this, the new version of the browser from Mozilla, and that browser is Firefox.

Comments ( 50 )