Week of Code Day 4 – Timeliners Report on Progress

Nigel, the lead Timeliner, explains that the timeline application has a fundamentally different life-cycle than any other Wonderland application we have built in the past. Timelines include three main features: creation, curation, and navigation. The unique thing about creating a new Timeline is that the cell cannot be instantiated until after the user has specified a set of parameters. The unique aspect of curation is that manual placement of objects on a timeline is extremely difficult, so the application needs to assist the user in placing objects. And finally, as was mentioned in a previous blog post, navigating the spiral is a challenge, so the application also has to help users get around.

In today’s blog post, I’ll give you an update on what each of the Timeliners has been up to and provide a glimpse of what all the pieces look like put together.

Timeline on Day 4 of Wonderland Week of Code

Starting on the visual side, Matt has been working on the math and graphics code to generate a spiral. While it’s conceptually easy to take a simple trapezoid and tesselate it around a core to create a spiral, it’s been a little tricky to accomplish this in Wonderland. Every single point on every trapezoid has to be individually calculated and there was a learning curve to figure out how use quaternions – representations of rotations. In addition, the JME quad meshes didn’t work properly, so after a lot of hair pulling and double-checking the math, Doug suggested that Matt try using tri meshes instead, and finally the timelines started rendering as expected as you can see from the image above of our first automatically generated timeline.

Drew is implementing a cool slider on the HUD for navigating the spiral. As you move the slider up and down, your avatar zips around the spiral. The HUD panel also shows your current position in the form of a date.

Nigel is building and hooking up what seems like a million dialog boxes. There’s the main dialog for creating the timeline. This one collects the basic information about the start and end date of the timeline, the desired units of time (hour, day, week, month, year), and the size of each unit. We spent quite a bit of time deciding how best to express the size of the units. For example, if you set up a timeline by day, do you want one day to take up a 10th of a turn of the spiral, or do you want it to take up 2 full turns. This will depend on how much data you want to display for each day. We finally decided on expressing this in terms of revolutions. So you can set the scale of the timeline to make one revolution equal to values such as 1 hour, .5 days, 2 weeks, 3 months, or 100 years. You can also specify a variety of other parameters that govern the look of the timeline.

Other dialogs include one for manually entering text, images, audio, or 3D models, one for setting up "keyword collections," and one for adding "timeline providers."  Jon is working on these last two features. He has written a timeline provider API that is an extensible mechanism for adding dynamic data to a timeline. Each provider is specialized for a certain type of content. Examples of providers might be news, wikipedia events, Time magazine covers, NPR podcasts, personal calendar information, a twitter feed, stock quotes, or any other information source that is time-ordered. Each provider decides what input it needs from the user. For example, a news provider, which might aggregate news from a variety of sources, might take a user query and ask the user to make some decisions about which news sources to search.

Since images are easy to render in Wonderland, Jon tackled a Flickr provider first. In its simplest form, this provider takes a query and uses the "date taken" field to decide where on the timeline to place the resulting images. The default is to display one image per time slice. In the advanced configuration panel, users can specify the number of hits per time slice to display, whether to search the full text or just the tags, whether to select the images based on relevance or interestingness (that’s a Flickr term), and whether to use just Creative Commons licensed photos.

Drew is responsible for figuring out how to lay out the results of each provider. He is taking a two-step approach to creating a timeline layout manager. The first step is finding the correct cell type to display the data from each provider. For example, if the provider returns images, we use the image viewer cell and if it returns text, we use the sticky note cell – a new feature conveniently contributed to unstable modules by a community member just yesterday! The next step is to put each object into the appropriate time slice and figure out where in that time slice it should go relative to other objects. To the right is a video of the first timeline generated with Drew’s layout manager.

One thing we quickly realized is it can be difficult to populate a whole timeline with a single query. Also some sources of data, such as Wikipedia articles, do not provide date information without requiring complex parsing, and other sources of data, such as Flickr, do not provide reliable date information. For example, I’m working on curating a timeline on the life of Frank Lloyd Wright. I want to include Wikipedia articles about each of Wright’s famous buildings along with photographs and 3D models. If you do a Flickr search for the famous house Wright designed in 1934 known as Fallingwater, you get over 18,000 hits. The oldest photo in the collection, however, was taken in 1971. I want a picture of Fallingwater to appear on the timeline in 1934, not 1971. To address this issue, we came up with the idea of keyword collections. These are a way of doing date-specific queries. For example, here are a few lines from my keyword collection for Frank Lloyd Wright:

1934 Fallingwater
1937 Hanna-Honeycomb House
1940 Goetsch-Winkler House

These queries will be handed to each timeline provider configured for the timeline. For example, if I have a Flickr provider, a Wikipedia provider, and a Google 3D warehouse provider, the same queries will be sent to all three services and I’ll end up with a photo of Fallingwater, the Wikipedia article about Fallingwater, and a 3D model of Fallingwater placed on the timeline at 1934.

Of course, we won’t be able to build all these providers by Friday, but hopefully we’ll have a Flickr provider and a New York Times news archive provider automatically laying out the data on a dynamically generated timeline. Others can then extend the functionality by writing new providers.

The last piece of the puzzle is audio, which Joe is working on. As people navigate around the timeline, we want to have audio associated with time segments. For example, while navigating the Frank Lloyd Wright timeline, you might want to hear top hits of the day, audio recordings of Frank Lloyd Wright talking about his architecture, or recordings of other experts talking about a particular building. The big challenge for Joe is figuring out how to get the audio to play just within a time slice, keeping in mind the irregular shape of spiral timeline segments. In addition to the non-rectangular footprint of the segment, the audio also can’
t bleed above or below the segment, or people will hear it in the wrong place. At first this problem seemed daunting, but Drew came up with a service to send notifications when an avatar transitions from one time segment to another, which dramatically simplified the audio calculations. Another challenge is nicely blending the audio so that it doesn’t sound jarring as you navigate from one segment to the next.

 Just to explain why we’ve been so heads-down, the Timeliners have written 12,333 lines of Java code in 4 days!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: