Zip the good

Zip has a lot going for it. It is ubiquitous and dependable… so long as you don’t want to share files across cultures.

Zip the bad

The Zip spec does not seem to know that there are normalization models for UTF-8, when there are actually 4 (or more, because there is some non-standard ones too!). The Zip file gives no guidance as to how file names inside zip files are to be normalized.

Consider, when a zip file is created on Linux, it just writes the bytes for the file name in the encoding of the underlying file system. So, if the file system is in ISO-8859-1, the bytes are written in ISO-8859-1. This may seem ok, but when you decompress the zip file on Windows, which runs on encoding Windows-1252, the file names get all mangled. If the underlying encoding of the file system on Linux is something else, you won’t be able to share files with other systems at all. So in this case, it is not Window’s fault.

The Zip spec says that the only supported encodings are CP437 and UTF-8, but everyone has ignored that. Implementers just encode file names however they want (usually byte for byte as they are in the OS… see table below).

It gets worst! because MacOS runs on some weird non-standard decomposed Unicode mode, you can only share zip files with other MacOs users. According to this email, the LimeWire guys also ran into a similar problem with regards to encodings in MacOS:

“for example a French, German or Spanish Windows user cannot exchange files that contain [file names with] French, German or Spanish accents with a French, German or Spanish Macintosh users”

The following table illustrates the problem:

Yes! holly crap! three different byte sequences corresponding to different character encodings.

The only way around this would be a *special* custom-built widget zipping tool that normalizes file name strings to NFC. If the widget engine needs to decompress the widget to disk, then it would take the NFC and convert them to the operating system’s native encoding (or store the files in memory, and reference them that way). This affects the URI scheme and DOM normalization of Widgets, so Web Apps will have to deal with it eventually… but not sure exactly how.

I love conspiracy theories… particularly when I get to make one up! Here is my conspiracy theory for how Microsoft will try to force both the W3C and other browser makers to adopt IE8’s XDomainRequest mechanism/API.

A bit of background first: the Web Applications Working Group (WAF) has been working on a spec that allows browsers to do cross-domain requests (basically for creating mashups securely). The spec is called Access-Control, and has been in development for three years. The spec was being edited by Anne van Kesteren of Opera Software, but under heavy influence from Hixie of Google, Jonas Sicking from Mozilla, and Maciej Stachowiak from Apple, to name a few people/companies. Marc Silbey, the representative from Microsoft to the working group, was also participating for a while, but he dropped off the radar as Microsoft shifted into high gear during development of IE8 (actually, Microsoft assigned 3 people to participate in WAF, but only Marc did). A few months ago, to coincide with the release of the IE8 beta, Microsoft announced XDomainRequest… aspects of which look, in a lot of ways, very similar to Access-Control, but with some key differences.Then, to the shock of the working group, they brought XDomainRequest to the W3C for standardization knowing full well that WAF had been working on Access-Control for over three years!

Naturally, Microsoft’s actions pissed a lot of people off because, as I stated in an email, they are just ignoring over three years of work into the Access-Control spec, they created their own proposal and implementation in secret and now are attempting to fast track it through standardization ignoring due process.

To which, Sunava Dutta, from Microsoft, responded by saying “incorrect” and prompting Chris Wilson, Chief Architect of IE, to respond:

You know, there is an idea that perhaps we’re not IGNORING the work on Access Control, and perhaps we simply disagree with some of it.

Which prompted me to respond:

…If Microsoft would have found the time to collaborate [in the WAF WG], all this stuff could have been resolved progressively and the [Access-Control] spec would probably be done by now (as has been shown, the MS proposal has just as many issues, if not more, than the Access-control spec; so trying to do it in-house did not yield a more adequate solution).

Which beckons the question, why did Microsoft stop participating in WAF to go off and create their own version of access-control? And here is the conspiracy theory:

1. Microsoft joins the WAF working group in 2007
2. Microsoft “borrows” Access-Control idea
3. Microsoft implements its own XDomainRequest mechanism in IE8beta
4. Mozilla implements Access-Contol in FireFox 3, but then pulls the feature at the last minute (consequently leaving a gap in the cross-domain request space for Microsoft to jump in)
5. Microsoft delays Access-Control work by sending in comments a year late (just before it was about to go to Last Call) and putting in their XDomainRequest proposal for standardization. Meanwhile…
6. Microsoft rolls out IE8, quickly gains market share (no help from Vista, of course )
7. Other browsers must now implement Microsoft’s solution/spec because business and developers start using it
8. Microsoft’s spec become a W3C Recommendation, Access-Control spec dies in the ass.

We are currently at point 5, with Microsoft using delay tactics to slow down standardization of Access-Control.

Why do I care? I’ve only contributed to Access-Control from the sidelines by attending face-to-face meetings and asking Anne dumb questions. However, a lot of C0₂ has been wasted flying everyone to meetings to talk about this spec; that’s thousands of dollars and thousands of kilos of C0₂ going to waste. Another thing that annoys me is, as I already stated, that Microsoft has every chance to provide feedback to the working group to fix/discuss any issues they’ve had with the Access-Control spec.

• Widgets 1.0: Packaging and Configuration
  • Covers Zip packaging and XML configuration document
  • Defines the processing model for XML and Zip
• Widgets 1.0: Digital Signatures
  • XML Digital Signature profile
• Widgets 1.0: APIs and Events
  • All the functions, properties, and events you know and love.
  • I’ve asked Arve Bersvendsen, of Opera, to be the lead editor. Unfortunately, he is tied up till mid April… probably making something awesome for Opera.

Other specs may also follow, particularly:

• Widgets 1.0: Automatic Updates (I personally think this one is critical)
• Widgets 1.0: Cross-widget Communication

Other documents are still under development too:

• Widgets 1.0: Requirements
• Widgets landscape
  • A look at how widgets are created and what features are supported across popular widget engines.

We are aiming to have all these done (ie. Last Call) by October. However, now that the document split has happened, I should be able to get the packaging format done fairly quickly.

We have more or less now settled on the configuration language format. The elements are going to be:

• <widget width=”" height=”" id=”">
  • <title: the title/name of a widget
  • <description> a description
  • <author email=”" url=”"> some details about the author
  • <license> paste your GPL here!
  • <icon src=”"> the icon
  • <access network=”true|false” plugins=”true|false”> if your widget need to get online
  • <content src=”"> some file in the widget archive

Only <widget> and <content> are mandatory at this point.

The processing model for the XML is going to be quite forgiving. The only thing that will cause an error, is not having a well-formed document.  For example, the following the following would result in “The Awesome Super Dude Widget” as the title:

<widget xmlns="http://www.w3.org/ns/widgets">
   <title>
     The <blink>Awesome</blink> 

     <author email="dude@example.com">Super Dude</author> Widget</title>
</widget>

The unrecognized elements are simply ignored, but their text content is extracted. This makes processing more forgiving and allows for extensibility and some graceful degradation. I also want to push that the widget should function if the namespace is omitted.

We are also currently investigating how we are going to deal with internationalization in the configuration document format. We are looking at following ideas from the Best Practices for XML Internationalization.

According to the new proposed charter, the missions of the new working group is to:

…is to provide specifications that enable improved client-side application development on the Web, including specifications both for application programming interfaces (APIs) for client-side development and for markup vocabularies for describing and controlling client-side application behavior.

The new Web Applications Working Group is chartered with the continual development of the following specifications:

Another cool thing about the new working group is that it is modeled on the HTML Working Group, meaning that is open, transparent (no secret chats on the members list) and anyone will be able to participate via the public mailing list.

I’ll continue to edit the Widget Spec and Requirements, and possibly continue to help out with the XBL Primer.  I’ll continue to be part of this new working group for a least 1 year, as I my PhD program ends in March 2009… and hopefully longer, if someone gives me a job to continue working on specs!

In the email, Dan Connolly wrote:

... and adding 3 months, we get: 2007-11-26 + 3 months = 2007-02-26
for a deadline for publication for the HTML 5 specification.

The W3C Director, Tim Berners-Lee, sees no reason why this
working group should be excused further from the three-month
heartbeat rule, and further, encourages us to publish sooner
if at all possible.

I still think it’s really disappointing that it’s going to take a further two months to publish the document. I was personally wishing it would be published for XMas (a nice present for the web community!). A FPWD is important for both marketing reasons and legal reasons: when a FPWD, all sorts of legal things in the W3C process go into effect. From a marketing perspective, it will be good as lots of media attention. However, from a technical perspective, a FPWD is irrelevant because of the rate at which HTML5 is being edited by Hixie (on a daily, if not hourly basis). The latest draft of the HTML5 document is always available to anyone either via the WHATWG site or theW3C CVS repository.

Update: In a follow-up email, Hixie sees no reason not to publish the document straight away! He writes “Cool. Since we are encouraged to publish sooner rather than later, and since there doesn’t appear to be any reason for us not to publish immediately, I have prepared the document for Working Draft publication.” If we are lucky, we might see the document published for xmas!

Update: According this post by to Anne van Kesteren, the publication wheels are now in motion: Mike(tm) Smith sent the request for publication earlier today! Now pending Chris Lilley’s approval… will Chris be the scrooge that ruins christmas?Lets hope not.

Update: No HTML for xmas I am afraid… In this email, Mike(tm) Smith writes, “after discussion with others on the team, the target publication date I’m requesting for the First Public Working Draft of the HTML5 specification is January 22.”

The main problem I’ve been working on over the last two months is trying to specify a subset of Zip that should be used by widgets. My goal has been to define a subset that is interoperable across all platforms and devices in such a way that it also ensures longevity. As you might imagine, this has proven to be quite a challenge…

The issues with Zip

The Zip file format is what is commonly referred to as a de facto standard: it is not formally specified by any standards body, but of it is so widely implemented that it is interoperable across OSs and devices. This seems great on the surface, but when you try to standardize it, it becomes quite a nightmare. The main issues are these:

• There are competing Zip specifications and there are many versions of each of the Zip specifications.
• Different version of the Zip specification are implemented across different platforms and OSs.
• There are many features in Zip that are desirable (eg. UTF-8 support), but are not widely implemented.
• Zip is not an “open standard”, it is the property of PKWARE.
• Zip is periodically updated and PKWARE does not provide any links to previous versions of their specs.

Competing Zip specifications

There are essentially two Zip Specifications that applications make use of: the “official” PKWARE Zip Application Notes and the “unofficial”Info-Zip Application Notes (mostly on Unix). The unofficial notes basically take whatever PKWARE has officially published, and gets modified, or otherwise clarified, by the guys at Info-Zip. In this sense, much of what one finds in the Info-Zip specs is identical to the PKWARE Zip spec. But, because PKWARE actually maintains the official spec, the PKWARE spec is always more up-to-data than what Info-Zip has on its website (for instance, the latests version of Info-zip covers version 6.2.0 of the official Zip spec (26 April 2004); the latest version of Zip is version 6.3.2 which came out in September 2007!, so InfoZip is three years behind PKWARE!).

Problem: Info-zip contains details that pertain to how info-zip works and may not be compatible/interoperable with the PKZip Spec. For example, Info-zip contains details about how to handle Unix permissions, while PKWARE’s Zip spec does not. This might not make the file formats incompatible, but it does make them physically different. You can try this out yourself: zip up a file using Info-Zip’s zip implementation and then zip up the same file using Windows’ Compressed Folders. The results will be different, but you should still be able to decompress the Info-Zip file using Windows’ native Zip implementation.

Different version of the Zip specification are implemented across different platforms, OSs, Specs

Another significant issue form a standardization perspective is that packaging formats are making use of either some Info-Zip spec or some PWARE spec. Significant examples include:

Java/JAR (including WAR and EAR) :

Info-ZIP Application Note 19970311

Open Document Format (ODF):

Info-ZIP Application Note 19970311

Open Office XML – Open Packaging Convention (OOXML-OPC):

PKWARE Zip Application Note (version 6.2.1), but with a bunch of clarifications.

OEBPS Container Format 1.0:

PKWARE Zip Application Note (no explicit version, but at least version 2.0 needed to extract and version 4.5 needed to extract Zip64).

I still have little idea as to what version of the Zip specification is actually implemented on each OS, let alone on mobile devices (information that seems to be quite difficult to come by!). As a result, and after some discussion with Jon Ferraiolo of IBM, I decided to base the Widget Spec on the OEBPS-OCF’s conformance requirements for Zip packages. I was tempted to make the widgets specification conform to the OOXML-OPC spec (put away your tomatoes!) because, in my opinion, the container aspects and conformance requirements are well specified (even if the rest of OOXML is “evil”).

Desirable features in Zip (6.3.2)

There are a number of really cool features in Zip that would make specifying a container format for widgets much better. They include:

• Strong Encryption (using x.509 digital certificates): basically solves the digital signature problem, I think.
• UTF-8 support: solves a significant part of the internationalization problem.
• Zip64: future proofing.

To require widget engines to actually support these features puts a fair bit of strain on makers of widget engines. At this point, we have required that implementers support UTF-8 and Zip64.

Zip is not an open standard

The fact that Zip is proprietary might be something that comes back to bite us on the ass. I’m no lawyer, but there of patents/IPR issues surrounding Zip. I’m also not sure about how PKWARE will feel about WAF specifying a subset of their specification. I’ve emailed PKWARE and informed them of what we are doing and requested that they review the spec. They have responded and said that they will look into it.

Where to from here…

Looking forward, I’d really like to get all the physical and logical packaging stuff done. That includes:

• Anything Zip related
• The inter-package addressing model
• How to handle decompression
• How to name files in ASCII and UT-8

I’d also really like to nail down the auto-updates model and make sure that the manifest language we are specifying is covers all the common use cases. The security model is the elephant in the room No one wants to touch it at this point; but we know its a massive issue. Another massive issue is the APIs… but that’s not something I want to get into now. A big issue for me is internationalization. I’ve been blocked a number of times when I’ve proposed doing internationalization using folders… every widget engine except Opera does it, so I think we should do it too.

These are my random thought on how I think the DFAUI should have been standardized…and why it failed….

Foundations for a Declarative Language for User Interfaces

Although it is possible to describe user interfaces and the interaction design styles in a colloquial language, the disciplines of Interaction Design and User Interface Design have well-understood theoretical and practical foundations including:

• Cybernetics: stemming from the mathematical understanding of communication theory and feedback and control theory,
• Visual Design: stemming from print design, color theory, aesthetics, semiotics (semantics and communication) theory,
• Multimedia/Hypermedia Design: stemming from literary theory (hypertext),film theory, animation, sound design and motion graphics, and electronic publishing
• Sound Design: stemming from music and audio composition,
• Architecture: which provides structure, theories of form and function, and design patterns,
• Software engineering: the actual means through which user interfaces can be realized,
• Ergonomic Studies: sometimes also applied, as is the case with, for example, Fitt’s law (”the time to acquire a target is a function of the distance to and size of the target” [TUG]).

The effectiveness of a user interface is generally evaluated through the fields of Usability and User-Experience Design, however both Usability [Nielsen] an User-Experience Design [Garrett] lack a ‘design language’ on which to design and build multi-modal user interfaces. The fields that make possible the practice of user interface design employ design processes that are iterative and user-centered so to maximize communication and make user interfaces as usable (or effective) as possible. In addition, these design practices provide designers with the freedom of self expression and the means (tools and language) to effectively achieve their communicative goals. In this context, communication is “the full process by which the behavior of one goal-seeking entity comes to be affected by that of another through the reciprocal exchange of messages or signs over some mediating physical channel” [Mullet].

It is generally excepted that there has be a continuous growth in the market of portable and multi-modal devices, and as a result new interface paradigms are emerging that challenge the traditional means of interacting with data. The modes of interaction afforded by these new devices often challenge tradition user interface metaphors experienced by users on traditional desktop applications. These new interface paradigms offer new innovative modes and modalities through which users interact with information. For example, Apple’s IPod’s tactile wheel control that also provides auditory feedback, the upcoming Nintendo Wii and its gesture recognition system, and the Nintendo DS with its advanced uses of direct manipulation through a pointing device. Note that the focus here on entertainment and gaming devices is intentional as they often represent the state-of-the-art in effective user interface design by providing highly efficient user experiences that are both engaging and aesthetically pleasing. The sophistication in these user interfaces is achieved by the skillfully designed fine-grained coordination of multiple modalities over time that achieve an engaging user experience. In other words, images and sounds are skillfully coordinated together over time in response to create a successfully multimodal experience.

I now provide a brief overview of the fields listed above and explain their relationship to interface design in more detail.

Cybernetics

Cybernetics theory has its foundations in the formal mathematical model of communication as defined by Shannon and Weaver. The field deals specifically with the concept feedback and control. The mathematical model of the feedback and control is easily understood as an agent attempts to change the state of a system by manipulating some variable(s), the amount of change that the agent desires is commonly referred to as the Goal. As the agent changes a variable the system provides feedback to the agent allowing him or her to ‘Control’ the level of change.

A simple example of cybernetic theory in action

A person is driving a car at 50km/h (let speed be variable x = 50). Her goal is for the car to accelerate to 60km/s, so she presses down on the accelerator pedal (while(x< 60){ x++ } ). As the pedal moves downwards, more fuel is injected into the car’s engine causing a greater power output that changes the speed of the car (hence x++). However, as the car accelerates it produces feedback through a number of modalities: audibly she can hear the car accelerating, visually she can see the car’s speedometer changing, kinesthetically she can feel the inertia and gravitational forces on her body as the car accelerates. If the car over accelerates (ie. x > 60), she can hear, sense, and see the changes through the various modalities, so she can then ‘control’ the car’s speed by depressing the fuel pedal, or by applying the break pedal (while(x>60){ x– } ). In simple terms, the agent reaches a desired goal by systematically controlling a system through actions that produce perceivable feedback in the system.

The same principles apply feedback and control mechanisms apply when moving a mouse around the screen. When a user wants to click on a button, he pushes the mouse in the direction of the button. The computer interactively updates the display to reflect the changes in the mouse position in relation to the physical movement of the mouse on a surface. As the user watches the feedback that is the representation of the mouse on the screen, he is able to “control” the speed of the mouse by decreasing the amount of physical force applied. Eventually, the user reaches his goal. However, if feedback from the system is too slow and the representation of the

Visual Design

Mullet states that “Visual Design attempts to solve communication problems in a way that is at once functionally effective and aesthetically pleasing.”

Semiotics is the general study of signs, of which a branch is Semantics (Seead). semiotics studies forms the foundation on which communication can be understood. Parts of communication include: the sign: which has two parts the signifier (the presentation of the sign on some medium), and the signified (what the sign is communicates to the reader). Signs are always found on some medium with the purpose of communicating something. Signs are composed of three distinct parts:

• the medium: or the ‘channel of communication’, for example marks on paper rely on the interaction between light and the contrast with the markings made by the author,
• the modes: the communicative choices the author has when using a particular medium for communication. For example, for plain paper, available modes include colors, lines, markings and pictures, textures, symbols, and even smell. However, temporal or auditory models of communication are unavailable to the medium of paper.
• The modalities: the sensory modalities that the reader uses to perceive the sign (eg, visually through the eyes or tactile as would be the case with brail, or through olfaction if the paper had been perfumed).

Because personal computers primarily use screens (emitted light) as their medium of communication, the must rely on the representation of other mediums to convey their messages (eg, photographic, three-dimensionality, document-based). The medium chosen for communication affects the way the message is understood by a reader. In semiotics, the combination of signs (media, modes, and modalities) is formally known as a ‘text’. Things get more technical from there and beyond the scope of this document.

The field of Semiotics has its roots in formal logic and the philosophy of language, but encompasses all forms of communication. Semiotics also provides a means to understand how time can be used in a communication process. Semiotics, however, can not be used independently to create communication as it lacks a design language – it can only be used as a means of to study and predict how people will interpret the semantics of signs. Thus, semiotics can be used to iteratively evaluate a communication process to make communication more effective, particularly because of semiotics’ reliance on understanding socio-cultural contexts in which signs are created and consumed (what we commonly refer to internationalization and localization of designs). Semiotics has a more sophisticated theoretical foundation than both Usability and User-Experience Design for describing human-computer interactions and has been widely applied in the literature of Human-Computer Interaction (Caceres, User interface design dudes).

Graphic Design

Just like written and spoken languages have a generally understood syntax and grammar, so do the visual arrangement of elements on some medium (a graphic). Graphic design theory is a theory of principles whose aims are to maximize communication through the structured arrangement of semiotic signs. The arrangement forms both a syntax and grammar that has been studied for thousands of years, but particularly since the invention of print and later with the establishment of electronic publishing in the previous century (see in particular Knuth).

Architecture

The applications of Architecture are generally well understood. The idea of design patterns was originally coined by architect Christopher Alexander et al. in the 1970s. The idea of a design pattern is to catalog a proven design solution to a particular design problem in such a way that a description to solution is applicable to lots of situations without being overly prescriptive or restrictive (as can be the case with guidelines or template).

Interaction Styles

• Command: command given as direct input into the computer
• Forms
• Direct manipulation: directly manipulating the representation of an object (eg. drag and drop, icons)
• Menu

The need of a Critical Language

Although it is possible to describe the elements that make up a graphical interface by the function that a group of controls play in an interface, such an approach is limited because it does not fully describe the interaction process. In other words, simply identifying interface elements (menu, button, radio button) does not fully capture the human-computer interaction in terms of input behavior, binding to a data model, eventing, presentation, styling, metadata and navigation (Raman, 2003). To fully describe the interaction process, it necessary to consider at least ask:

• Input behavior:
  
  • what means of interaction are presented to the user use to change the model?
  • or what means of interaction are afforded by the device? (eg, direct manipulation, menu, command line input)
  • how does the input behave as input as it interacts with the user? (for example, does it stop receiving input after a certain number of characters have been entered?)
  • what means of control does the system afford the user in relation to feedback? (eg. speed of the mouse and the rate at which the screen updates, time before another key stroke is displayed on the screen while the user is holding down a key).
• Binding to a data model:
  • what is the goal of the user or the motivation for interaction? ie, what part of the data model do they wish to change or capture? (eg. save a file)
  • what is the conceptual model that user interfaces is allowing the user change? (eg an address book, the set of spells that a character in a game can cast when under attack, etc)
  • what is the actual model that the user interface is allowing the user change? (eg. a database table, a document model)
  • what is the computation model that the user is changing? (eg, a thread inform the renderer to update portions of a screen buffer as a user “scrolls” down a page)
• Eventing:
  • what events are triggered by user interaction? (eg. onclick)
  • what events are triggered by the system? (eg, onload)
  • how do those behaviors affect the data model (eg, event phases and level of granularity or propagation)
• Presentation:
  • what medium, modes and modalities are utilized by the device to provide feedback to the user? (eg, a sonic chime when the file is saved successfully, the sound of paper being crushed when the recycle bin is emptied)
  • what declared stylistic rules go into forming the representation of the interface component? (eg, css rules or XSL:FO for print)
  • what are the stylistic restrictions imposed by the limitations of a particular device? (eg, monochrome display does not allow the use of color as a mode of communication, but may allow the use of shades and texture instead)
  • what are the stylistic restrictions imposed by a particular device because of security concerns? (eg, always displaying ‘*****’ in place of passwords, or not reading passwords out loud as is the case when interacting with ATM machines or passwords fields in html forms)
• metadata:
  • what semantics can be extracted from the underlying model and content? (eg. the data type for this input field is of type date)
  • the interaction layer? (eg, element can be ‘dropped’ onto other elements, but does not except textual input)
  • and the presentational layer? (eg. in a localized Australian context, speak the month after the day: “today is the 20th of October”, but not in an USA context: “today is October 20th”. )
• navigation:
  • has the author imposed a navigational structure on the interface? (eg. steps 1 and 2 must be completed one after the other)
  • does the interface provide the user with a non-linear interaction structure? (for example, by using the Accordion interface design pattern?)
  • when a user interacts with an interface element, must particular conditions be met before feedback is displayed? (eg. searching for a contact in an address book)

From an interaction design perspective, the above questions form what in computer science is commonly referred to as the ‘Model-View-Controller’ (MVC) design pattern (Gamma et al.) (See also wikipedia for a detailed description of the MVC pattern). MVC is well understood withing the Working Group and MVC forms the foundations of the XForms specification. XForms describes the interaction process in terms of inputs, outputs, a model and triggers that manipulate some given abstract data model, and mechanisms to transmit data to and from the server.

XForms also describes the behavior of input types in relation to the model through interaction events, data-typing and error checking. XForms also defines mechanism by which inputs can behave as outputs. Hence, XForms provides the elemental primitives on which complex interactions with dynamic data models can be declared by an author. XForms’ power is in its flexibility and high degree of abstraction, accessible design, and device independence. XForms also defines structuring mechanism for user interfaces that can help structure how the user navigates a user interface. T. V. Raman describes the work done on XForms in the following way:

• “The primary goal as we overhauled HTML forms from 1993 was to make the next generation Web forms technology robust with respect to accessibility, internationalization, and the ability to deliver user interaction to a variety of modalities and end-user devices. Looking forward to a ubiquitous Web that would be accessible using devices and modalities best suited to a user’s needs and abilities at a given time meant that we had to step back from commonly held notions of visual interaction and create an abstract user interface vocabulary capable of withstanding the test of time.”

  “Addressing the need to create an abstract user interface vocabulary had the advantage of contributing directly to our primary goals of accessibility. Once we freed the user interface vocabulary from fixed ideas driven purely by visual interaction, we were able to leverage the separation of the XForms model from the user interface to the fullest degree in creating a set of controls that could be easily re-targeted to different user interaction environments. Cascading Style Sheets (CSS) is now a mature Web technology,and this has made it significantly easier for the XForms working group to defer stylistic and presentational issues to the CSS layer. Having separated out presentation by using CSS, we used the XML binding to DOM Events provided by XML Events to factor out interaction behavior of the various controls. Thus, the XForms user interface design separates content, presentation, and interaction to create an XML vocabulary that lends itself to intent-based authoring of user interaction”

Although logical from an engineering standpoint, the explicit separation of the content, presentation, and interaction is seen by some in the web community as counter intuitive and confusing. The traditional approach for authors has been to to consider the required user interface elements available HTML, collect the required data from the users through these interface components, and then have the user post the data back to the server for processing. In addition, XForms has also being criticized for:

• it’s reliance on XML for the data model
• overly declarative
• developers consider XML to be ‘bloaty’
• DOM interfaces hard to work with (hence the creating of workaround like the handy innerHTML property).

Regardless of these criticisms, XForms provides a foundation built of a proven design pattern (MVC) that has been used in practice by software engineers for approximately 30 years. XForms (together with its dependent technologies and pluggable design) provides the base primitives and structuring mechanisms needed by authors to realize almost any kind of user interface.

Given that we now have a language for describing user interfaces. We can begin an analysis of interface components of various applications. It is not possible in most cases to understand the underlying computational models of software applications, but it should be relatively easy to understand the conceptual models from a user’s perspective.

Understanding the UI space: the competition to a DFAUI

• XAML
• Macromedia MXML (Flex 2)
• XForms
• XUL
• AJAX Frameworks (Prototype, Dojo, Scriptacolous)

Directions away from HTML forms

• XAML Approach:
  • generated XML spaghetti nightmare from hell.
  • Tremendously difficult to read or write and probably maintain (requires an IDE).
  • Does not separate content, structure, style, behavior, data binding(all mashed together)
  • Defines it’s own styling language
  • But apparently, one can make pretty powerful stuff because it uses the .Net. API
  • Provides ready made desktop UI components (slider, progress bar, checkbox, etc)
• Macromedia Approach:
  • XML Elements map to classes in an API (Flex and ActionScript3), or user defined classes
  • Heavily dependent of programming
  • Leverages CSS (although implementation is fairly poor) and ECMAScript
  • Powerful yet simple binding and extension mechanisms
  • Provides ready-made interface components for re-skinning (button,accordion, input fields, etc)
  • Very powerful network and hypermedia capabilities and ready made interface components
• XForms approach:
  • abstracts and separates UI logic, model, presentation, and behavior
  • Clean and simple (when compared to XAML and XUL) declarative language
  • Pluggable and leverages CSS, XPath, XMLEvents, Namespaces
  • RESTful approach
  • Device independence
  • Lacks API and higher level interface abstractions
• XUL:
  • Geared towards desktop application development
  • Leverages ECMAScript and CSS
  • Provides an extensive set of ready made interface components and associated APIs

DFAUI – our current state

• XAL proposal seems to want to compete with established UI languages, but is already years behind in implementation and sophistication

• TID-Mobile is better but kinda sits in the same boat
• DFAUI use cases seem to be following XAL by re-inventing the wheel
• So far, we only have really self evident requirements:
  • must use CSS
  • work with events
  • must be device independent
• the requirements and use cases simply leading to requirements that will be identical to XForms

Why DFAUI Failed:

• Monumental task of defining all possible UI elements: consider how long it took to build all the competing languages (even XAL has been undergoing development for years).
• lack of interest in WG
• lack of interest from implementors (they’ve already got way better solutions and have invested lots $$$,$$$,$$$ into them)
• not really doing anything new

If we want our stuff to be adopted, we obviously need to make use of what has already been implemented; it should slot in with as little effort as possible; and should be at least comparable, if not an order of magnitude better than what is already out there.

Building on XForms: Standing on the Shoulders of Giants

• XForms already resolved complex interface modeling problems through MVC
• Leads developers to an effective software engineering solution (MVC)
• but MVC may not lead to an effective interface design solution:
  • comes with some UI design patterns built in: hint, help, error
  • comes with grouping patterns
  • comes with sequential patterns
  • comes with data typing entry control (pattern)

Why build on top of XForms instead of other languages

• “Pemberton blasted the scripting approach taken in Web Forms 2.0, sayingit doesn’t scale well, is harder to maintain, doesn’t address industry requirements and use cases, and doesn’t provide the ability to take snapshots of each step in a forms-based process for sensitive industrial or governmental applications…” (Festa, 2005)

  “XForms is not a Web standard,” said Brendan Eich, a founding member of Mozilla in charge of technical direction, the creator of JavaScript and a member of WHAT-WG. “It’s a relatively new spec seeing early-adopter use in intranets” (Festa, 2005)

• Maybe they both suck?

• XForms MAY be the way forward (with the useful bits from Web Forms 2.0 added in).

• However, XForms is still very high.

• It will be hard to compete with established frameworks like Dojo, Prototype, etc.

• XForms is more sophisticated than these frameworks, but not (yet) a widely available web standard.

DFAUI – what could it actually be?

Consider the emergent growth of open-source Ajax frameworks as the case study for the DFAUI:

• “The DOM and JavaScript have enabled a huge amount of experimentation that have helped us discover useful design and interaction patterns on the Web; however, if we now fail to move these understood concepts to a declarative means that obviate programming, I would assert that we will largely fail to discover the next set of abstractions.” (comment made T.V. Raman on Mark Birbeck’s blog)

• Web API WG is already covering the non-declarable aspects.
• We could investigate what user interface design patterns are emerging (or have already been defined) and attempt to capture them in a standardized way.
• Yahoo! UI is already doing this, but for Ajax.

• Martijn van Welie and others have been capturing UI patterns for years

Towards an an eXtensible User Interface Patterns language (XUIPL)

• Build higher level user-interface centric patterns using XForm primitives,but allowing them to maintain their device independence, accessibility, and adherence to MVC
• Make the language extensible so developers can define and share their own patterns (Ala semantic web ontology style)
• Overcome some of the complexity of XForms, by providing ready made rea world solutions to UI problems
• Builds on XForms as a foundation, reinforcing it’s applicability as a suitable language to build powerful and usable web applications
• Through XUIPL, patterns can be ‘natively’ supported by XForms thus making them easier for novices and professionals to use

XUIPL: A language for User Interfaces that is solution-driven based on proven UI design patterns that lead to better user interface designs that runson a proven software engineering foundation.

• The approach I am proposing gives us a new noble direction that empowers UI designers, not just programmers
• Has the potential to make a language that is an order of magnitude more usable and applicable than XForms alone
• Is open and extensible, allowing new patterns to emerge and be used on the web community… hopefully speeding up the adoption of XForms
• Does not interfere with the development of XForms, but provides more use cases for XForms and for CSS

From an ‘Application’ development perspective, other minor things that XUIPL could also add to make it more attractive:

• Unit testing and logging module, making it easier to build and test web apps
• applications could still be packaged, with the aid of the widget 1.0 spec

References

Mark Birbeck. “On Adobe and XForms via Declarative Programming, Wizards and Aspects”. September 04, 2005. Available at: http://internet-apps.blogspot.com/2005/09/on-adobe-and-xforms-via-declarative.html

Paul Festa, “Fight over ‘forms’ clouds future of Net applications” ZDNet News: February 17, 2005. Available at: http://news.zdnet.com/2100-9588_22-5581106-4.html?tag=st.num

See also: http://www.welie.com/patterns/literature.html for a good listing of literature related to UI patterns.

UI Pattern Collections

• http://time-tripper.com/uipatterns/
• http://www.cs.helsinki.fi/u/salaakso/patterns/
• http://www.welie.com/patterns/index.html
• http://developer.yahoo.com/ypatterns/

Notes

TV Raman says: * In this context, the key to XForms and XHTML2 is *not* to either obviate or decry imperative/script-based programming; rather, it’s primary goal is to liberate today’s script authors from the minutiae of writing yesterday’s code to focusing on the next level of innovation.

* The DOM and JavaScript have enabled a huge amount of experimentation that have helped us discover useful design and interaction patterns on the Web; however, if we now fail to move these understood concepts to a declarative means that obviate programming, I would assert that we will largely fail to discover the next set of abstractions.

• Went to Sydney for two days to work with Lachlan Hunt on the XBL Primer.
• Finished My PhD confirmation.
• Become a Facebook addict.
• Published Widgets 1.0 Requirements.

July is also going to be pretty intense:

• will try to get the First Public Working Draft of the XBL Primer by the 13th of July.
• Presenting my PhD confirmation on the 25th.
• Going to Melbourne on the 26th to work with Cameron McCormack for two days on a model for the XBL 2.0 Test Suite.

After I come back from Melbourne I go to Oslo for a WAF Face 2 Face to be held at Opera Software.

Share on Facebook ]]> http://datadriven.com.au/2007/07/june-wrap-up/feed/ 0