Several people recently asked me why their scripts, which worked fine
when they were using HTML, stopped working when they started using XHTML
(and actually sent their markup with an XHTML MIME type).
There are plenty of reasons why that might happen. Not using the namespaced
DOM Core methods is one, but few people use the DOM Core methods so it's rarely
the cause. It could also be that their markup wasn't well-formed, but usually
that's pretty obvious since the browser complains about it, and when people
ask me, it's usually because the cause is subtle, so this is rarely it either.
It usually turns out that the reason their code is failing is that they used document.write(),
which doesn't work in XML. Why is that?
XML requires that an XML process abort on any well-formedness error.
Say you have the following:
<foo>
<scrible/>
</bar>
</foo>
Since the </bar> tag is unmatched, the entire document is not well-formed, so the UA has to abort and not display it.
But if we allow document.write(), then you could have
things like this:
Now the document is still not well-formed, so it must still abort
when it gets to the </bar> tag. But the problem is
that if we process that document, then when we get to the script, we
write out a <bar> start tag after the <script> block, and
now when we reach the </bar> tag, it matches
something! So the UA couldn't tell if it was well-formed or not.
So in order to remain compliant with XML, document.write() doesn't
work on documents that are parsed with an XML parser.
To be precise, they invented a new element, a new attribute, and a
new value. The new element is canvas, an element that
does nothing except provide a DOM interface for graphics drawing; the
new attribute is composite, a presentational attribute
that can be applied to images; and the new value is
search on the type attribute of
input elements, which provides a control for incremental
search interfaces.
These new tags are to be used in several places. One of these is Dashboard,
where applications written in an HTML-like interpreted language are
executed in a runtime environment similar to a Web browser's. Another
place where these tags are to be used, I am told, is the real Web.
The response to this announcement from a number of people in the
industry seemed to be outrage... about the syntax.
Hello?
Here is Apple introducing their own proprietary markup to the Web,
without going through any sort of standardisation first (not even
unofficial standardisation like the WHATWG), and what people complain about
is the syntax?
I
hope that what Dave is really saying is that Dashboard widgets are
actually XML, albeit an XML that looks very much like HTML except
they've added some nifty stuff to it. If so, great, fine, no problem.
XML lets you do whatever you want, really.
So let me get this right. It's ok to send proprietary non-standard markup
over the Web, so long as the angle brackets are XML angle brackets and
not HTML angle brackets?
This makes no sense. Proprietary markup is proprietary markup,
whether it is HTML-based, XML-based, or any other language such as
PDF, Microsoft Word, XAML, or Flash. It's not the exact order of the
angle brackets that matters, it's the lack of open, consensus-driven
specifications, the lack of interoperability.
Eric wasn't the only person arguing that Safari's extensions are
abominable because they can be used as if they were HTML elements, I
just picked on him because I know he won't take it personally!
There have been several "solutions" proposed to the "problem" of
proprietary tags being added to HTML.
The first is using namespaces in HTML. There is
one fundamental problem with that idea. HTML doesn't have namespaces.
At all. HTML doesn't allow namespaces. The Namespaces-in-XML spec
doesn't apply to HTML. However you cut it, there simply isn't a way to
allow namespaces in HTML. If you were to add namespaces to HTML, all
you would be doing is adding yet another non-ratified
extension on top of the original extension.
The second proposed solution is to use namespaces in XHTML,
sending it as text/html. This also can't be done, this time
because while you can use namespaces in XHTML (with some caveats; see
below), you can't send such XHTML as text/html. The only
things you can send as text/html (according to the relevant RFC) are HTML
documents (up to version 4.01), and XHTML1 documents complying to the
compatibility
profile. (Although I would strongly recommend
against doing that anyway.)
The third proposed solution is to use namespaces in
XHTML, sending it using a legitimate XML MIME type. This
could work, but it is limited to XML, and Apple wanted to allow
authors to use HTML. Also, it requires that authors use two
namespaces (one for the XHTML content, one for the Apple-proprietary
content), and exposing authors to namespaces is a non-starter. The
entire point of the Dashboard development environment is that
it be trivially easy to write these applications, and namespaces
are not trivial.
The fourth proposed solution is to simply replace the XHTML
namespace with an Apple namespace, so there is only one namespace.
This is even worse. Now not only would Apple be introducing three new
non-standard tags, but they'd be introducing the whole of HTML
as non-standard tags. No other browser would be able to render such
pages. Accessibility tools would be unable to recognise the elements.
Tools like Google would be unable to understand the semantics of the
page for proper indexing.
The fifth proposed solution is to use a DOCTYPE that would enable
the extensions. This doesn't scale. What if another UA wants to
implement the extension as well? For example, marquee is
required by half the Chinese Web. Originally, only IE supported it.
Now take canvas, the Safari extension. If we had used
different DOCTYPEs for each set of extensions, which DOCTYPE would you
use if you wanted both? (And wanting both is quite likely; imagine a
stock ticker with a graph above it).
There are other problems with DOCTYPEs. Eric unintentionally
reminded me of one when he said:
IBM
already does something like this, having created an "IBM XHTML 1.1
Transitional" DTD that's used throughout their Web
presence.
The IBM DOCTYPE actually had to be hard-coded
into Mozilla to trigger quirks mode, otherwise their site wouldn't render right.
It's also non-standard, and violates the HTML spec in several ways.
If
you run an IBM-XHTML compliant document through a validator that loads and uses IBM’s
DTD to check over the document’s markup, then the document will validate.
This rather misses the point. It still won't be conformant. This is a key point.
Creating a new DTD does not magically make you standards compliant. For example,
here is a document that "validates":
<!DOCTYPE LI PUBLIC "-//W3C//DTD HTML 4.01//EN">
<li>Validation is only a small part of conformance.
I'm sure we all agree it's not conformant, though.
I've probably missed some of the other proposals. But the reality
is that it doesn't matter. If you want to make a proprietary
non-standard extension to a standard that doesn't support explicit
namespacing, you do the same as is recommended for
CSS: put your vendor name into the tag. In the case of the
aforementioned Safari extensions, I would have recommended calling the
new names apple-canvas, apple-composite, and
apple-search. The real solution is to
bring these proposals to the table, get some consensus between
the relevant vendors and other interested parties, and then use that.
For specific features like these, it doesn't take long to get
consensus; they are small features whose basic design can be agreed
reasonably quickly. Doing this also usually means getting wider
peer review, which improves the quality of the API.
For instance, it would have been pointed out that composite should
never have been put into the markup. It's presentational. It belongs in
the presentation layer, namely CSS.
Apple were probably concerned that by talking about this stuff, they would
have tipped their hand about Dashboard. WHATWG is the perfect cover, though. Apple
could have put forward their proposals, discussed them, got concensus, and everyone
would have thought they were just planning these features for the Web Application
space.
In fact, they did do this. Apple also introduced the new range
value for the type attribute of input elements, which
introduces a new control for imprecise numeric data entry (e.g. a
slider). This, as I mentioned
before, is not a proprietary value. This value was
discussed, in detail, on the WHATWG open mailing list,
and forms part of the Web Forms
2.0 draft specification. Here, Apple did the right thing: they
created an experimental implementation of a proposal. Their
implementation experience has already helped the draft (several
changes to the original text were made to more closely align with what
Apple found was actually needed).
So why did they not do this with the other extensions?
In other news, Brendan
Eich gives us his take on the WHAT Working Group work and other
issues facing the Mozilla Foundation in the latest edition of the Gillmor Gang.
I highly recommend giving it a listen.
When we announced WHATWG, several people noticed that the list of members also contained, in addition to Mozilla
and Opera employees, a couple of Apple employees who work on Safari. A few weeks into the project we
also added Dean Edwards of IE7 fame to the list. The members of the WHATWG are an
oversight committee intended to make sure we don't get off-track; the specs are actually edited by me
and based directly on the input we get on the mailing
list.
There were also some people who commented that instead of calling it a Working Group we should have
called it a Task Force, because the resulting acronym would have been much more appropriate. All I can
say is that I wish I had thought of that because that would have been a really funny name.
I recently started on some Web Forms demos, to show what Web
Forms can do and how it can do it without losing backwards compatibility. I haven't done much yet, just
a demo of how to make the repetition model
degrade gracefully in HTML4-compliant UAs, but it's a start. (Sadly the only UA I could find that was
compliant enough to degrade in the right way was Mozilla, so we might well have to revisit exactly how
those repetition model buttons work.)
The first public demo of a Web Forms 2.0 page was actually a very high profile demo,
though I highly doubt that anyone present actually realised they were looking at it. It was during Steve
Jobs' keynote speech at Apple's Worldwide Developers Conference, during the Safari RSS spot.
If you look at the top right of the screenshots of Safari showing an RSS feed, you'll see a little
slider control. That, my friends, is a Web Forms 2.0 <input
type="range"> control.
Something I hadn't realised until recently is quite how many Web applications are hidden away
inside intranet sites. I always knew that there were some there, but the sheer numbers of such
applications is quite surprising. A few people have sent me confidential screenshots of their intranet
applications (with the sensitive parts censored, of course), which has really helped get me an idea of
the kinds of features that would be most helpful to people writing such sites. We'll be addressing a
number of these requests in Web Apps 1.0
proposals once I've dealt with all the recent Web Forms 2.0 comments.
Joel Spolksy wrote an interesting
article explaining why Microsoft stopped Internet Explorer development in its
tracks a few years ago, despite it having ridiculously many bugs. Here's the executive
summary: they realised that IE was competing with their OS to be the preferred application deployment
platform. Since they make money from their OS, but don't make money from IE, the choice was clear.
I think the irony of this situation is quite amusing. HTML, a document format, is invented,
eventually Netscape is formed and creates a dramatically successful product that makes Microsoft jump.
Microsoft instinctively react to the creation of this new networked document viewer market by developing
a product to compete with it. They (inevitably) succeed, by the simple mechanism of creating
incrementally better products until one is "good enough", aided by the chronic mismanagement that
characterised Netscape during its last eight or so years. However, in the middle of this process, the
fight stops being over who can make the best document viewer, and slips into who can make the best
dynamic content viewer. And that slowly turns into an application platform. Oops. I'm no business expert
but I would hazard a guess that plunging billions of dollars into a loss-leader that competes with your
primary business model is not the best idea.
I've known all the bits of this story for years, but they never really clicked until Joel explained
it.
As Joel points out, though, Microsoft's moves after they realised their mistake with IE have been
harder to understand. I would have thought the solution most likely to succeed would have been to extend
IE in ways that made it into a better application deployment platform. Eventually, this could have
turned IE into the OS, either natively (making the next version of Windows basically be IE), or
by selling IE with versions for all operating systems. This would have had several advantages:
It would settle once and for all their claim that IE needs to be part of the OS.
It would solve the Linux threat: instead of competing with Linux, Linux would become just another
platform upon which IE can run.
It is what their customers wanted.
It could create lock-in just like Windows has.
They actually did start down that road. IE6 has support for a technology that Microsoft
stopped advertising at the same time as they stopped developing IE, namely HTAs, short for HTML
Applications. Similarly, even without really trying, Microsoft managed to lock large numbers of people
into the IE platform — it has taken years for Opera, Mozilla, et al to catch up to IE, and even
now, there are many sites that simply won't work in any other UA. The problem, I am told, is that the
visionaries at Microsoft want the lock-in platform to be an OS. That makes sense; I mean, that's what
they know.
And so they took all the things that they thought made HTML a good deployment platform, and all the
things that they thought made Win32 a good development platform, and all the ideas that Microsoft
Research said would be cool, and started a huge project: making a completely new OS. (It isn't clear to
me if this project spawned sub-projects like .NET, or if smaller projects like .NET became subsumed in
the larger OS project.)
There are several problems with this approach, as I see it. First, the market is no longer growing
that fast. In the past, when they released a new OS, Microsoft could grow its marketshare primarily by
growing the market, and then once the new market hit critical mass, people on older platforms that
couldn't run the newer software would be forced to upgrade to run the newer software. (They did the same
with Microsoft Office: by selling enough new copies to new users, eventually old users were forced to
upgrade simply to be able to read documents written by new users.)
But in the current market, Microsoft are having to depend mainly on upgrades.
Windows XP didn't do as well as previous releases, as far as market penetration goes. And many people
are questioning whether Longhorn has enough new features to warrant upgrading to it. So Microsoft will
either have to quite radically change their approach, or they will have to settle into a slower revenue
cycle.
But IE is only to play a supportive role in this big project.
Of course, Microsoft aren't alone in thinking that extending IE to be a new platform is a bad plan.
As I previously discussed,
the W3C held a conference on Web Applications just last month, and the majority of people there
disagreed with the concept of the browser being the platform as well.
Then again, I work for a browser vendor, and have been in the browser industry (both voluntarily and
as a job) for years now. So it's not surprise that I think the browser is important as an application
platform. (Obviously, though, as Robert Scoble is eager to tell us, Web apps aren't the answer to
everything. I wouldn't recommend to anyone that they try to write a graphics manipulation package in
HTML.)
The problem with the browser today is that applications based in the browser are constrained
to nightmarish UI idioms and a severe lack of polish stemming from the fact that the platform was not
really developed as a platform, and that no real progress has been made on this path for several
years.
John Gruber points out that users
don't really seem to care about the poor UI, though. The other advantages — especially the true
zero-install cost of Web-based applications — far outweigh the costs.
But that's why we started WHATWG: we want to make it easier to make nicer the kinds
of applications that it makes sense to deploy over the Web. Mail and
news clients. Cinema ticket sales. Book stores. Auction sites. Multiplayer stategy games.
In other news, apparently the Windows IE team is starting to wind down its work on the "let's make IE
secure" Windows XP service pack 2 release, and is starting to look at what people want in the IE that
will ship with Longhorn. They are
even looking for
feedback. Amusingly one of the most popular requests in the forums I have looked at (which is
probably a very skewed subset of the real feedback that Microsoft gets) is "better standards
compliance". And even more amusingly, one of the most often repeated messages from Microsoft people in
those same forums is "we're not going to be working on standards compliance" (although not usually
phrased that way).
I honestly do not expect to see any changes at all to the HTML, CSS, DOM, and JS
components in the Longhorn version of IE compared to Windows IE6. I've had Microsoft employees tell me
to my face that they will not have XHTML or SVG support in the next release. I've only seen excuses for
why features won't be added, never any Microsoft employees saying they are interested in supporting a
new spec. I have thousands of tests available on my site if
they want to find some bugs, but I doubt they will even look.
On the other hand, I can assure you that here at Opera we are implementing all kinds of things and
every release has more CSS/HTML/DOM/JS/XML/etc fixes. And over in Mozilla land Brendan has writtenseveral posts listing the kinds
of things Mozilla will be implementing over the next few years.
Travelling east is painful. I woke up around 05:40 on Thursday, and
went to bed around 15:00 on Friday, nine time zones away. My brain now
thinks that 23:00 is a good waking time. Also, it meant I had three
breakfasts yesterday, which successfully smashed my fast so that I
wasn't hungry even though I should have been. Quite the biological
clock confusion.
The second day of the workshop had more
discussion. We had some straw polls; of the 40 or so people there,
around 8 said they wanted to work on the work Opera and Mozilla have
been proposing recently, and about 11 said that not only did they not
think it would be worth working on this, but they actively thought
that the W3C should not work on it.
In my opinion that's pretty short-sighted, but as Steven Pemberton
pointed out, six year ago, the W3C decided that HTML was dead, and the
way forward was a host of new languages (what is now XHTML2, XForms,
MathML, SVG) that would lead the world's population to a clean new
world. So at least they are consistent.
Of course I had to point out that six years ago, I was in school,
which got a good laugh. My point, though, was that times change. In
the last six years we have seen that authors simply didn't agree:
Mozilla has supported MathML for years, but it is still very rare to
see any MathML content on the Web. Mozilla, Opera and Safari all
support XHTML1, in fact Mozilla has supported XHTML1 since before it
had an assigned namespace and MIME type, but again the amount of
application/xhtml+xml content on the Web is trivial.
The truth is that the real Web, the Web that authors write for, is
the Windows IE6 Web. The only way to change that is to reduce the IE6
market share, and new technologies don't do this. Marketing does. Once
users are primarily using a browser that is being regularly updated,
then we can start introducing radically new technologies. Until then,
such technologies simply aren't going to become popular.
There were a lot of rather confused statements during the meeting.
For example, it is clear that a lot of people think that the
browser is dead and that the way forward is transparent "runtimes"
that execute remote applications securely. But then these same people
demand to know why Mozilla, Opera and Safari don't support XForms and
SVG, saying that their lack of support is crippling their standards'
adoption.
Surely if the browser paradigm is dead, it doesn't matter what we
implement?
What I think most of the people at the meeting actually want is a
standard that combines XHTML, XForms, SVG, and SMIL (and CSS, DOM, and
ECMAScript, although they rarely if ever actually mention those by
name), and then adds enough APIs to make the host into a platform in
its own right.
Java tried the "provide lots of APIs that are interoperable across
lots of platforms" and failed. Some people thought this was because
Java, as a language, is too complex for most applications. And Java
doesn't have a detailed spec (not detailed enough to write an
interoperable implementation accurate to the level that is needed for
applications).
The detailed spec problem is the big issue. There has simply never
been a Web specification written in enough detail for this kind of
work. Even "DHTML", which does just a fraction of the number of APIs
needed for the kinds of applications these people are imagining, is
completely inadequately specified. For example, if you have an
object element followed by a script block,
will the script execute before or after the object has loaded? This is
the kind of behaviour that scripts depend on. (Answer: In IE, the
script will block waiting for the object, and if the object doesn't
load, it will be removed from the DOM. The exact behaviour
depends on the extension of the filename in the data
attribute and the local computer's registry. Feel free to explore this
yourself using
these testcases...)
Making these specs more detailed is the work that Opera and Mozilla
want to do. But to do this for a sophisticated application platform on
par with, say, Longhorn, is simply unfeasible. Notice how WINE has to
reverse engineer Windows to determine how it should work. Or how the
various Java clones have to reverse engineer Sun's Java to get
interoperability.
Of course, if they want to do this, I wish them the best of luck. I
might even want to participate in the working group, since
someone will have to look out for the Opera and Mozilla
interests!
Sadly there does seem to be a growing opinion in certain circles
that the W3C is becoming more and more out of touch with the Web. In
many ways, this makes sense: the membership has many more server-side
people than client-side people, and most of the client-side people are
plug-in vendors, not browser vendors. (All the browser vendors present
at the meeting were in favour of variants on the Opera/Mozilla ideas,
but they were easily out-numbered by the non-browser members.) Since
most people consider "the Web" to be what browsers show, it's only
natural for an organisation of people who are largely not doing Web
browser work to appear to be "out of touch".
Really it's not that the W3C is out of touch with the Web, but that
the W3C membership is solving problems that every day Web users don't
see. For example things like CC/PP and SOAP are very much back-end
technologies.
I've also heard a lot of comments recently from people asking if
the SVG working group realises that SVG 1.2 is becoming a dumping
ground for anything and everything, instead of remaining just a
graphics language. For example, SVG 1.2 drafts feature raw
socket APIs and a
Window interface. So I asked people at the conference whether the
SVG working group shouldn't, instead of adding every feature under the
sun to SVG, simply define how SVG should interact with other languages
like XHTML. The answer I got was highly dismissive. Basically: "Well
we want it in SVG".
You'll note that Robert O'Callahan (one of the core layout
developers for Mozilla) sent
an e-mail last month pointing out the many areas of overlap
between SVG and CSS. He never got a reply to his last message. (Hmm...
this is reminiscent of the way the SVG working group effectively
ignored a
message I sent back in 1999 pointing out a problem in SVG 1.0 that
is still present in SVG 1.2 drafts — reply once, ignore further
e-mails on the thread, and don't actually address any of the
problems...)
Since many of SVG's features (like Window, or like the
way CSS properties are allowed to have lengths with no units) directly
conflict with existing standards (both ad hoc and de
jure), it's quite clear that browsers that implement SVG will only
ever be able to implement a subset of SVG.
Another point that was mentioned at the workshop was that DOM3 core
was a long specification, checking in at some 216 pages. The SVG 1.0
spec is 719 pages long. (More than twice the length of CSS2.1, which I
thought was already a ridiculously long specification.)
Still, it's a nice language if all you want to do is draw vector
graphics.
Anyway. The issues have been discussed, the positions have been
given, everyone knows where everyone else stands, now it's time to get
down and actually start doing work.
What working group is going to work on extending HTML...
Today I listened to a good dozen
presentations from various groups on the subject of Web
Applications and Compound Documents.
Some interesting things came out. First, the only sustained
spontaneous clapping of the entire day came as someone suggested, in
response to my brief statement of how backwards
compatibility is critical, that it was about time to drop HTML and
Windows IE6 from the roadmap.
So I can assume from that that most people don't agree with the
whole backwards-compatibility thing!
Second: I was quite amused to see that, of all companies,
Microsoft, Red Hat, and Sun Microsystems actually agreed on
something. Namely that trying to standardise an API for sophisticated
applications is simply a non-starter. The argument, which I agree
with, is that such APIs are simply insanely complicated, and that making
interoperable implementations is nigh on impossible. Just look at the
trouble WINE has had trying to implement Win32 again — now
imagine if you had to write a spec to actually describe the entire
Win32 API in terms that could actually be implemented interoperably
without reverse engineering the first implementation as the WINE
people do.
What was funny was watching the other people then disagree with
them. Hint: If three of the most bitter rivals in the marketplace
— all of whom have extensive experience in the subject in
question — agree on something, then it is probably
true.
What we (Opera and Mozilla) want to do is simply extend HTML, DOM,
and CSS a bit so that the most common things are easier to do. Things
like my Web
Forms 2 proposal or my
server sent events proposal. These are simple extensions, not an
attempt to provide comprehensive platform APIs.
Another point that came out of the discussions is that, in case
there was any doubt, Internet Explorer in Longhorn will not support
XHTML or SVG. (Microsoft suggested they would need some significantly
more comprehensive test suites before they started working on
standards compliance again.)
After the meeting, a bunch of us had dinner at La Fiesta. Our table had three
Microsoft employees, two Red Hat employees, two Mozilla Foundation
employees, and an Opera Software employee. I bet you won't see
that very often.
Tomorrow we have another dozen presentations. I expect to see more
of the same; mostly people expounding on the virtues of XForms, SVG,
XHTML2, or their own radically new proprietary technologies, and explaining how Web
Applications would all be much better off if the W3C would go down
their chosen route.
We, of course, want the W3C to go down our chosen route.
Since there doesn't seem to be much consensus on doing that, though,
the question is what should we do now? Should we do our own
thing (in public of course) and then submit it to the W3C (or IETF or
ECMA) at some future point once we have initial implementations?
Should we simply do our own thing (Opera, Mozilla, and a few
interested parties) and forget standardisation altogether? Should we
just take part in whatever Web Applications working group the W3C sets
up and implement whatever comes out of that in several years' time,
despite being fully aware that few people will ever use it? (Which is a
foregone conclusion since it wouldn't work in Windows IE6.)
I'm learning towards the first of the three at the moment. I guess
the Opera and Mozilla people will have to discuss this in more detail
before we decide anything though.