Neil's Place

Aaron’s recent comments about problems a particular user was having with the sliding effect that the Firefox notification bar uses made me think again about how incorporating a easy means of doing transition and animtion-type effects in XUL and in the browser UI is needed. Naturally, if there’s an accessibility problem, a global preference to disable these types of effects is probably the most obvious option here. But whatever solution is used for that, it’s easy compared to finding a general solution to doing the transition effects in the first place. There’s been a strong desire from many that being able to incorporate these types of effects into the UI can open up a number of interesting possibilities to enhance the UI or direct the user’s attention to something of interest.

SVG incorporates an animation mechanism (mostly just by using parts of SMIL) that uses tags to declare animations that can be applied to other elements. In this way, one can specify that certain CSS properties or DOM attributes of an SVG element can be modified over time. The whole SMIL specification is quite complicated though. I could imagine that using a declarative form of animation using tags and attributes to specify what and how to animate to be useful in some cases.

A while ago, Webkit added support for a transition feature by adding some custom CSS properties. It works by smoothly adjusting the value of particular CSS properties whenever they change. This feature appears to have been designed intentionally such that one would be able to use it entirely from with a style sheet, without having to modify the document or use any script (and therefore browsers that don’t support the feature don’t suffer any ill effects). However, without script to accompany it, this transition feature is of somewhat limited value, mainly centered around the use of hover effects and the like, and, indeed, the scarce descriptions of this feature present hovering over an element as the main use case. The author also has little control over the transition, in fact, only the CSS properties that participate in transitions and characteristics about their speed can be controlled. The author has no ability to customize, for example, different behaviours for fading in versus fading out or to make an element glow or pulsate.

There’s also another related specification from Apple that defines a different form of animation, also defined in CSS. This allows one to define specific animations to apply to elements. Personally, I find the syntax for this animation feature rather ugly, and more importantly, it’s mostly only of use when used in conjuction with script that modifies the custom CSS properties the spec defines. It seems though that once script is required, it doesn’t make sense any more to define the behaviour in CSS and instead just provide some script APIs to accomplish what you need to.

Of course, all of this moot for XUL and UI as both SVG animation and the two Webkit animation proposals are intended for animating the value of CSS properties, and for many purposes in XUL, one doesn’t want to animate any CSS properties. Indeed, many slide-in effects like the notification bar actually want to have a transition occur when an element is made visible in some way (whatever way that may be), and smoothly grow to its intrinsic size. Note that this is distinctly different from changing the width or height of the element; in this case, both the width and height always remain set to ‘auto’ during the transition. (Note that the notification bar implementation currently does something quite different and overly complicated that I don’t really like — all the more reason for a better way to do this.) To give a more specific example, the smooth animation that occurs when scrolling through tabs when there are too many to display at once, adjusts the scroll position, a property which isn’t exposed to CSS or in the XML document at all.

I imagine there may be some value in allowing a declarative form of animation as SVG does, although I’d think here someone might complain if we invented a different syntax that SMIL or SVG animation (both W3C recommendations) use. There is also some value in the Webkit idea that transitions can be applied automatically, for example, allowing an element to smoothly change when some value is modified. There’s certainly a use of this to make for easier development in the case where an element might change in a variety of different ways. On the other hand, it’s quite often the case that the time when a animation effect should occur is known already, and should just be triggered by a script. For that, it may be easier just to add a simple API for all XUL elements:

element.transition(arguments)

Ideally, here, arguments would be just some opaque JavaScript object that the method can read properties of for different sorts of transitions. At the moment, I’m preferring an approach more akin to this, but am open to suggestions.

An issue here is performance. Various tests I’ve done show that performance isn’t spectacular for many cases, mainly due to the need to do a layout at each step. The notification bar that asks to save a password in Firefox, for example, pushes the entire content down, requiring it to be relaid out at each interval. My testing also shows that performance is quite slower when rendering native widgets on Mac at least. But some of these are problems that could be solved with a bit of work.

Another potential performance issue is whether to allow all elements to have animations, or just, say, create a special element that only animates itself and its children. This could easily extend one of the other elements (for XUL, presumably the generic box), but would have the disadvantage that it wouldn’t be as general purpose and could limit its use for certain effects. Allowing transitons on all elements, of course, means we need extra code and time to check for transitions on every element, most of which won’t be animating. (If most elements are animating, that UI you’re designing is more properly referred to as a cartoon).

Some interesting possibilities, but no clear solution here.

Support for the HTML5 drag and drop API is now available in Mozilla nightly builds. This is the API that IE and Safari have supported for a while. Now Firefox will support it as well so you create content in your web pages that can be dragged and dropped elsewhere. The same API is also used for Firefox extensions and XUL applications, however some additional features are available to support dragging non-textual data (such as image data and files) as well as dragging mutliple items.

Here is an example of how to make something draggable:

<description ondragstart="event.dataTransfer.setData(
                          'text/plain', this.textContent)">
  This text may be dragged.
<description>

Documentation is available on how to use this new API, which include a number of examples. Specifically, there is also a guide to dragging specific types of data.

This code is new and there are a few bugs reported that will be fixed soon. However, the new API is much easier to use than the old privileged only API. For Firefox extension and XUL application authors, the new drag APIs are what you want to use.

Direct use of the nsIDragService or nsIDragSession interfaces is now deprecated. In addition, the nsDragAndDrop.js script is also deprecated.

The next step is to implement this API for clipboard operations as well, so that cutting and pasting are just as easy. Then, the old API, and more importantly the very messy code which implements it, could be removed.

Just thought I’d mention some of the new XUL features that will appear in the next version of Firefox.

Dão added support for a search textbox which may be used via <textbox type=”search”>. It includes search and clear buttons and may be used in one of two modes. The default mode searches when the field is modified and functions similar to the search field in the Firefox bookmarks organizer. If the ’searchbutton’ attribute is set to true, then the search is only performed when the search button or Enter is pressed. This latter mode would be used for the search field in the Addons dialog where searching is a longer process as it searches the remote addons site. The corresponding bug is 388811.

In bug 111034, I added support for the clientLeft, clientTop, clientWidth, clientHeight, scrollLeft, scrollTop, scrollWidth, scrollHeight properties for all elements. For XUL, this means that you can retrieve and modify the scroll position of any scrollable element without having to use a <scrollbox>.

In addition, ‘width’ and ‘height’ properties were added to the ClientRect object, which are the rectangles returned by the getClientRects and getBoundingClientRects methods. This width returns the displayed width plus the border and padding; the clientWidth returns the displayed width plus the padding only.

Combined, these features make it easier to determine the size of an element without having to use the boxObject. They are described in more detail in the specification.

In bug 437640 I added the ability to remove <keyset> elements and disable them with the ‘disabled’ attribute. This makes it possible to do keyboard shortcut editing dialogs where you insert and remove a keyset containing the defined shortcuts. The motivation behind being able to disable keysets is to allow an application to define different sets of keyboard shortcuts in different environments.

For those creating themes, the <gripper> element that was inside the scrollbar thumb has been removed, such that only the <thumb> is used. See bug 448704 for more details.

So I was at the Mozilla Summit in Whistler this year. Take note, it’s not a good idea to go on any kind of trip to somewhere that involves a two and a half hour bus ride a few days before you’re getting married. As everyone at the summit now knows, there was a large rockslide last night which blocked the only real road back to Vancouver (where the airport is). What was my first reaction to this? Panic! What was my second reaction? More panic!

The other longer route wasn’t a good option; it takes seven hours. Instead, Mozilla folks found a way to get me on a floatplane back. This is one of those tiny planes that lands on the water, and if I was in a movie, would be required to catch by jumping off a dock just as it takes off and hanging onto the pontoon. But since I’m not in a movie, I got a regular seat.

Here’s a picture of one of the planes. It isn’t the plane that I took but a different one waiting for a selection of the large contingent of stuck people.

I’d like to thank Dan and Mark for setting up the plane and for driving me out to the dock. I’m in Vancouver now and should have no problem getting to my wedding on time.

Oh, and here’s a picture of the rockslide which I took while we were flying over it.

After some recent newsgroup posts, I thought it might be a good idea to clarify how text is used in XUL.

XUL provides two elements for displaying text, <label> and <description>. The <label> element should be used when labelling a control or user interface element. It supports a control attribute which may be used to associate the label with another element, and an accesskey attribute to support a key shortcut for focusing the associated element.

The <description> element should be used for all other text in XUL. This element does not support the control or accesskey features.

Apart from that, both elements are the same, and are implemented the same.

There is, however, another characteristic that applies to both the <label> and <description> elements, and determines how the text is rendered.

One form involves placing the text inside the element as a child:

<label>Search:</label>

<description>Enter your login information</description>

The two lines of code above cause the elements to become CSS block elements. This text is displayed in the same way as that for displaying the text in web pages for instance, and supports things like breaking paragraphs into lines, drawing underlines, selection and spellcheck highlighting, and adjusting text as necessary to support right to left text, ligatures and various additional characteristics of certain languages. Thus, you can put whatever elements are allowed in a CSS block within these elements, for instance an html <br> element.

The second form is by using the value attribute:

<label value="Back"/>

<description value="Font Selection"/>

The code that implements this is different, is much simpler and doesn’t support line breaking – everything is rendered on one line – nor does it support selection and spellcheck highlighting or blinking text, although it does support other line decorations, and supports a crop attribute for displaying an ellipsis if there is not enough width available. The width is determined from the amount of text and the default height is determined from the height of the font. This type behaves more like a form control that happens to display read only text. In CSS terms, it would probably be considered a replaced element.

Usually, the most common purpose of using one over the other is to get wrapped text versus text that appears on one line. Naturally you would choose the ‘text as child’ form if you wanted more complex text and choose the ‘value attribute’ form for simple single line text. You would not choose which tag (<label> versus <description>) to use based on this criteria, although historically, many people have (incorrectly) done so. It doesn’t help of course that the XUL tutorial on XULPlanet says to do exactly that though. Boo.

To change the value in the first form with the text as a child, use the textContent property. To change the value in the attribute form, use the value property.

label.textContent = "New Text:";

label.value = "New Text";

Another common technique is to use a plain-styled read only <textbox> element:

<textbox class="plain" readonly="true"/>

This creates text which looks like a single line label or description, but allows selection and provides a context menu. This latter feature is really the only reason to use this form, so it should only be used when it is likely that the user will want to select the text. Specifically, the control and accesskey features won’t work, so you should not use this for labels.

You may see references in some documents to a <text> element. This element is an older element that should not be used. For historical reference, it is equivalent to a <description> element but only supports the value attribute form. Also, it doesn’t support any extra APIs so you should not use this element. I mention this because someone linked to a page which used the <text> element incorrectly. In fact, the author of that page had meant to use a different element, <textnode>, which is intended to be used in templates. Although it looks similar, the <textnode> element isn’t related to the description or label elements; instead it simply generates a DOM Text node in a template, which is useful when you want to use the text as child form:

<description><textnode value="?name"/></description>

This code generates output like the following when used in a template:

<description>Marian Dodson</description>

There are a set of automated tests used to check XUL menus and popups. These tests check to ensure that popups open and close in response to particular events, that the popups appear in the right location and that highlighting and navigation between menu items and submenus work correctly.

However, these tests tend to be very finicky in terms of when they decide to work properly. Usually they do, but they have a habit of breaking down when even a seemingly unrelated change in made. Compounding the problem is that frequently only one platform or even a specific machine will fail to run the tests properly, making the real issue difficult to diagnose. There are several cases where I’ve just guessed at a fix and got lucky, and several tests which have just been disabled since I can’t reproduce the problem locally. For others, the response when the tests fail is: “Look, Enn broke the tests again”.

Once a popup test fails, other tests will often fail as well. In this case, the usual cause is that a popup or a window is still open because the previous test failed, causing later tests, especially those that fire events, to be unable to work properly.

There are several reasons why the popup tests are so picky:

• Popups cannot opened unless the window is focused. This prevents pages in background tabs, for instance, from displaying popups. However, it also means that the window must have the focus in order for a popup test to succeed. One trap here is that a new window is not focused until after the load event, which means that a popup opened during a load event may not open. Documents loaded in an existing window will be able to open popups within the load event as the existing window is already focused. This can cause problems if you’re not aware of it and don’t check the test in both circumstances, for instance if you just run one test individually versus running through the test suite.
• Tooltips are opened when the mouse stops moving for a few moments and disappear when the mouse is moved again. The tooltip tests synthesize mouse movement events. This means however that while running a tooltip test, a spurious movement of the real mouse can cause additional events to fire, causing the test to fail as the tooltip either opens or closes at the wrong time. In fact, just having the mouse inside the window where the test is running can do this as well. The solution: move the mouse pointer somewhere else.
• Some tests check for keyboard shortcuts on menus. Because the tests run inside a browser window, this can interfere with the shortcuts used by the browser itself, and vice versa. For instance, testing a particular key combination might open a browser menu, or activate a browser command, causing later tests to fail. Tricky to diagnose, as the test that fails might occur somewhat later than the key test. This issue isn’t unique to the menu tests though; any test which emulates the keyboard may have this issue. The keys tested in the current tests are carefully chosen to not conflict in this way, but only in English builds. If you are testing on another language, keyboard shortcuts may be different and you may have mysterious test failures.
• The tests normally run in a little frame. Sometimes this frame isn’t large enough. This can be an issue if you are using a larger font for example. This can cause elements that the test uses, especially those tests that emulate mouse clicks, to be hidden. Since you can’t really click on an element that is scrolled offscreen, or because the coordinates may be different, the test can fail.
• Some tests open a separate window to run, either because they need a top level chrome window, or because they need more space. Having a low screen resolution can cause these windows to open smaller than desired. As well, a lower resolution can cause popups to appear in different locations or smaller than expected, preventing tests from working properly.
• The theme you are using can affect how the test runs. For instance, the addition of a one pixel border to an element can affect calculations, causing coordinates to be off by one.
• Theme issues can also come up due to differences between platforms. One such case I discovered recently was due to a seemingly harmless rule in the default Windows theme:

    menubar > menu[_moz-menuactive="true"][open="true"] {
      border-width: 3px 1px 1px 3px;
    }

  This rule causes the border of a menu to change when the open attribute is set to true, which occurs while the popup is being opened. This has the side effect of causing a layout to occur to handle the new border, which in turn causes the popup to be made visible and the popupshown event to be fired. This is fine, but other platforms don’t have this style rule, so the layout to open the popup doesn’t occur until later, so the popupshown event which was being tested for fires at a different point.

These are some of the problems to be aware of when running the popup related tests, and many of these apply to other types of tests as well. If you are having problems with the tests, say on your own machine, (the popup tests are in the toolkit directory and are usually identified with a word like ‘popup’, ‘menu’ or ‘tooltip’ in their name), it may be worth checking if any of these problems apply.

Hey, is that the XPCOM Reference on XULPlanet updated for Mozilla 1.9 and Firefox 3? Yeah, looks like it.

We’ve been thinking a bit about the future of XULPlanet. developer.mozilla.org provides more documentation and is easier to update, both for me and for others, and also includes some of the documentation currently on XULPlanet. Due to the greater difficulty in keeping the XULPlanet version updated, this version of, for example, the tutorial, is a bit out of date.

To be honest, I haven’t updated XULPlanet documentation in quite some time, and probably won’t do much in the future. The only thing I’m likely to do is update the XPCOM reference once Firefox 3 arrives. Some developers prefer the XPCOM reference on XULPlanet instead of the MDC reference. However, the latter is created manually and is only partially complete. Perhaps someone with the time and inclination would offer to convert the XULPlanet Mozilla 1.8 XPCOM reference over to MDC. The scripts I used for this are also available.

XULPlanet at one time was quite useful, however, the time has come where its value no longer seems necessary. So at this stage, we have two options. Either find some way to make XULPlanet useful again, or remove it and have XULPlanet just redirect over to MDC.

Thoughts?

We may just have to get Half-life 2 up here in the northern office just for this. The subway renditions alone are, as Mike Beltzner would say, sooooo awesome.