Neil's Place

June 2, 2003

9:00 PM Haystack Comments

I just tried the long-awaited initial release of Haystack. In case you don't know what that is, it's an information client that allows one to organize and find information in almost infinite ways. For instance, Haystack allows one to set a reminder to prepare for the word 'the' contained in an annotation you made to a textbox label in the search panel for a mail folder. Which is a feature that, if it had existed long ago, would have prevented the fall of the Roman Empire.

OK, I'll ignore that Haystack is very slow and that the release notes contain the text '768 megabytes [of RAM] strongly recommended', and focus more on what it actually does and the technology behind it.

When you first launch Haystack, it takes about 5 minutes to do various sundry tasks. Like set up various default settings and compile a bunch of scripts. It only does this once, though, so subsequent launches are faster. The first thing I noticed when the main window opened was my name in the upper left corner. Hey, I thought, what magic! It knows who I am! On the left side is a bunch of links for various tasks. In the middle is a pile of news about SARS and West Nile Virus (hmmm, must know where I live too). It also displays the weather around MIT. I couldn't figure out how to customize it to display more local weather.

More operations are performed by right clicking on things. In Haystack you can right click on ANYTHING and get a context menu with 100 items on it. The menu has an item for each logical concept you clicked on. So, for instance, if you clicked on some text in a list, you might get commands for text, commands for things in lists, commands for lists as a whole, commands for what the list is in, and so on. Some commands seem to apply to everything, such as Annotate and Remind, whereas others such as Find Synonyms only apply to words.

Haystack never appears to open a second window for anything. Instead, when more information is need to carry out a command, a box appears on the left side of the window. This is a bit jarring as it's not always noticeable, so it often seems that a command had no effect.

Haystack uses lots of RDF. In fact, I think it's pretty much all RDF. Thus, an annotation is just a bit of RDF attached to something. Thus you can annotate anything, including a mail message, a news item, a list, or even a specific word. You can do the same with reminding and a number of other tasks.

Because of these things, the UI tends to look quite different after you've clicked on a few things. Boxes of tasks appear on the left side, the middle area changes often, and occasionally something happens on the right panel as well.

UI-wise it falls into the same problem as many web sites, including almost all big Web portals. It can do lots of things, but the UI tries to show too much of these things at once. Instead, a good UI should focus on only one thing -- what the user is currently doing. News web sites have this problem too. When viewing an article on a news site, only half of the display is used for the article, the rest displays weather, stock tickers and links to sign up to newsletters, none of which are actually relevant to what the user is doing. Of course, since news sites were doing this before the Web really took off, people thought this was normal UI, and learned to just ignore the bits along the four edges of a page.

In Haystack it's hard to tell what you're doing at any given time. It's also not clear whether it's possible to do more than one thing at once.

Anyway, that's what I thought after using it for half and hour or so. I'll look more into it to see if I can see how to use it better, and perhaps look more into how it works.

Comments ( 6 )

May 31, 2003

5:51 PM Semantic Web Client UI Diagram

I'm getting really excited by this Semantic Web stuff I'm doing. here's a screenshot / diagram of how it works to display some dynamic UI based on FOAF, RSS, and some movie information.

The UI is written using a number of small (less than 1K) XUL and XBL files, although any kind of XML file can theoretically be used.

Comments ( 37 )

May 28, 2003

8:05 PM Time to do something

I guess it's time to start looking for a job. I've never really done that before, since in the past they've always found me.

Comments ( 3 )

May 26, 2003

4:51 PM Another example

I just added another example on sorting content. I also fixed a few errors in various parts of XulPlanet people have sent me over the last month.

Comments ( 7 )

May 23, 2003

1:01 PM Data Structures and RDF

Time to chime in on the RDF debate.

There are four general ways of storing information:

  1. A list, in which one has a number of items, which may or not be related to one another.
  2. A table, in which one has a number of items (records), each with a distinct set of properties or columns.
  3. A tree, in which one has a hierarchy of items.
  4. A graph, in which one has a number of items (nodes), with the nodes connected to each other in some way.

There are others, but they are more or less just variations of the same.

There are examples all over of each type. Arrays are examples of lists. Of course, they are used all over the place. Relational databases typically store all of their data in tables. So do spreadsheets. Trees are used for mail or news messages and your bookmarks. XML is a syntax for specifying trees of information. The Windows and Classic Macintosh file systems are presented and/or stored as a tree. The Unix file system however isn't a tree. It's a graph. RDF is a graph. The Web is also a graph -- it's a bunch of pages connected via links.

Each of the four storage methods, lists, tables, trees, and graphs, increase in complexity as you go up. Lists are simple to store. Graphs are the most difficult. Actually, that doesn't need to be the case. But, very few programming languages come with any kind of Graph structure ready to use. Due to the complexity, you should probably store data in the lowest type possible, depending on the kind of data you have.

You can always use one of the structures higher than what is necessary. A list could be stored in a table with only one column, a table can be stored in a tree, where a root node has a set of records, each with a set of properties, and a tree is really a specialized form of graph.

However, the reverse is not true. You can't store a graph in a tree, you can't store a tree in a table, and you can't store a table in a list. Any place where you see someone trying to is a hack.

Many people don't know this though. So they just store everything in a tabular database or in XML, regardless of what it is. This has two problems. First, you get data that can be stored in a simpler format, stored in some more complex format. So you get people passing lists of things around using XML. Or, configuration files stored in XML.

Second, you get people trying to coerce more complex data into a simpler format, so you might see people trying to shove trees of data into a database. Or you get serialized RDF written as XML.

Many people think that XML is the ultimate format for storing data. It isn't. It can represent trees nicely, and it can do tables and lists if you really wanted it to, but it can't represent graphs, not cleanly anyway. Perhaps what is needed is an eXtensible Graph Language, which represents graphs of data. There is RDF-XML, and XGMML but both use a language for describing trees. Actually, it shouldn't be called the eXtensible Graph Language, because then people will get confused thinking it's like XML. Because a tree can be represented as a graph, all data could be represented in the Graph Language (not that it should be, of course), unlike XML which can't. Of course, this assumes there isn't some higher level structure above the graph.

Long, long ago, people stored data in lists, because that was all that was available. Then, someone came up with the idea of storing data in tables. So relational databases came along and people moved up the ladder to tables. A few years ago, XML came along so data moved up again to trees. Can you guess what will happen next? The Semantic Web folks want us to move to using graphs. Should we move to graphs? Seems to be the next logical step in information evolution. What's holding us back? Well, it's probably too soon. The world is still in the tree phase. One day, graphs will start to become more popular -- it will just take time. In 30 years, someone might come up with something beyond graphs, and we'll all slowly switch to it as well.

There's also the RSS in RDF debate. Many people don't see the value in storing RSS data in RDF. This is because the information stored in a single RSS file isn't a graph -- it's a tree, so plain-old XML actually makes more sense.

Of course, the Semantic Web folks don't agree. Why? Because they aren't thinking in terms of a single RSS file - they are thinking of building giant collections of RSS data, all linked together so that it forms one giant - hey, it's not a tree - it's a graph. Then, you can search and navigate it like you can with the existing Web.

But of course, the Semantic Web lets the servers and the software you're using, know more about what you're talking about. This is unlike current popular search engines like Google which are pretty much just guessing. You can make it better, sure, but the best way to acheive accuracy is if someone tells it the answer to begin with.

Anyway, I'm starting to wander a bit -- that means it's a good time to stop.

Comments ( 2 )

May 21, 2003

10:58 PM Some RPath Documentation

OK, I have put together a short list of what I've implemented so far in RPath, which I'm now going to call ReoPath, to give it a less generic sounding name.

Currently, it can be called from within Mozilla using an XPCOM component, or can sort of be used as a standalone utility. I'll provide the code and/or binary soon after I clean up some crashes and all those memory leaks.

Feedback is welcome.

Next, I'll implement some additional functions. I'll also post some info about ReoPath Templates, which is a way to embed ReoPath expressions in a XUL file (or any XML file for that matter), and have content generated, kind of like XUL templates, but more powerful (and at the moment, with more hacks).

Comments ( 35 )

May 14, 2003

4:39 PM Cafe Dream

Last night I had an interesting dream.

I was frantically searching all over town for some of those sticky circle things you're supposed to put around the holes in a sheet of paper. I'm not sure why. I've never used them -- in fact I don't know anyone who has.

Anyway, when I got to an office supplies store, it had turned into a toy store, so I couldn't get any. But the interesting part was that next door, a place had opened called the MSN Cafe. Inside, people were lounging about drinking coffee and talking. On the far wall was a line of PCs one could use. On one side was the counter where people could order various gourmet coffees. Above that, a giant MSN butterfly logo.

Could this be a sign of the future? Is Microsoft planning to open up a chain of coffee shops one day? Will Sun counter with stores with names containing a Java/coffee pun? Who knows?

Comments ( 33 )

May 13, 2003

11:04 PM Now we're getting somewhere

This RPath stuff is really starting to do interesting things:

chain('Meg Ryan'^name,'Christina Ricci'^name,^playedBy*^cast|cast*/playedBy)/name

Meg Ryan was in Anastasia with John Cusack who was in Being John Malkovich with Cameron Diaz who was in Fear and Loathing in Las Vegas with Christina Ricci

OK, there's a tiny bit of JavaScript in there too, to add the 'was ins' and the 'withs' and the bold text.

Comments ( 46 )