We’re pleased to announce the two latest projects from SitePen Labs: Paver and Persevere!

Paver exists because of SitePen’s Kevin Dangoor, and his work on the SitePen Support web application. We found that the build tools that exist with Python just aren’t all that easy to use.

Paver is a Python-based build/distribution/deployment scripting tool along the lines of Make or Rake. What makes Paver unique is its integration with commonly used Python libraries. Common tasks that were easy before remain easy. More importantly, dealing with your applications specific needs and requirements is now much easier.

As is the case with most of the things we work on, Paver made our lives easier, and we thought it might be beneficial for you. Currently an alpha release, Paver is available under the BSD license.

Persevere on the other hand is a more deliberate approach to rethink how web servers work, and development is lead by SitePen’s Kris Zyp:

The Persevere project is an open source set of tools for persistence and distributed computing using intuitive standards-based JSON interfaces of HTTP REST, JSON-RPC, JSONPath, and HTTP Channels. The core of the Persevere project is the Persevere Server. The Persevere server includes a Persevere JavaScript client, but the standards-based interface is intended to be used with any framework or client.

The Persevere Server is an object storage engine and application server (running on Java/Rhino) that provides persistent data storage of dynamic JSON data in an interactive server side JavaScript environment. It is currently in beta, and boasts a very solid feature set that should interest JavaScript, Dojo and Ajax developers:

• Create, read, update, and delete access to persistent data through a standard JSON HTTP/REST web interface
• Dynamic object persistence - expando objects, arrays, and JavaScript functions can be stored, for extensive JavaScript persistence support
• Remote execution of JavaScript methods on the server through JSON-RPC for a consistent client/server language platform
• Flexible and fast indexed query capability through JSONPath
• Comet-based data monitoring capabilities through HTTP Channels with Bayeux transport plugin/negotiation support
• Data-centric capability-based object level security with user management, Persevere is designed to be accessed securely through Ajax with public-facing sites
• Comprehensive referencing capabilities using JSON referencing, including circular, multiple, lazy, non-lazy, cross-data source, and cross-site referencing for a wide variety of object structures
• Data integrity and validation through JSON Schema
• Class-based data hierarchy - typed objects can have methods, inheritance, class-based querying
• Pluggable data source architectures - SQL tables, XML files, remote web services can be used as data stores
• Service discovery through Service Mapping Description

Comet daily has more information about the recent addition of Comet support to Persevere. Also be sure to check out the Persevere live grid demo.

Persevere is dual-licensed under the BSD and AFL. Visit the SitePen Labs for more information or to download Paver and Persevere.

"Ciao" / "Grüezi" / "Hola" / Hello! I am excited with my first world-wide open source project so trying to greeting in many languages! Firstly, let me introduce myself, I am Robert (Robertus Harmawan Johansyah for full name). About the project, I am contributing for GFX library in DOJO Toolkit, 3 dimension graphics in web is getting hot at the moment so we shall not let DOJO be getting behind! In nutshell, this project will enhance GFX 1.1 that already implemented such as adding Canvas3D from Firefox and Opera! Of course API unification will be taken in consideration! If you interested about what i am doing right now, I am studying Computer Science as final year in Macquarie University, Sydney.

Hello Planet Dojo! I'm Jeff Balogh, and I'm doing doing a Google Summer of Code
project for Dojo this summer. I'm an undergrad at the University of Central
Florida, with one more semester to go. I'm living in Toronto this summer and
working with the DrProject folks, so drop me a line if you're in the area. :)

For this SoC, I'm creating a drag & drop form editor for Dojo. It's not
intended to be a full-featured IDE; I want to make a fun,
easy way to interact with Dojo and Dijit in the browser.. I'm planning to use a
lot of the great work already available, like dijit.form for the widgets and
dojo.dnd for all the fancy moves. This project will provide a way for dojo
users to embed dynamic, customizable forms in their apps, and I'm hoping it can
also be a nice introduction to using dojo for newcomers.

Currently, the plan is to have a drawing area where you drag and style your form
elements, an HTML source editor for twiddling with all the little details, and a
high-level property editor that exposes the important aspects of each element,
like custom validation and callback code.

My mentors are Peter Svensson, Liu Cougar, and Greg Wilson.

This is my first big javascript project, so tips and suggestions are very
welcome! I'm looking forward to a fun and illuminating summer.

The latest Dojo release includes critical but low-risk patches to version 1.1.0, plus localizations to bring the list of languages supported by Dojo to 22.

Dojo 1.1.1 is a drop-in replacement for Dojo 1.1. Because this is the release where the localizations landed, and because of the critical bug fixes, it is recommended that all 1.1.0 users upgrade to 1.1.1 immediately. Here are the highlights:

• New localizations donated by IBM -- now 24 in total, including English and two variants each of Chinese and Portuguese -- to accompany cultural support of hundreds of locales by dojo.cldr.
• Dojo mini, a new packaging of Dojo source weighing in at just over 1M. The same Dojo, with no tests, no utilities, no demos, just the unoptimized source code, making it easier than ever to download and explore Dojo.
• Bug fixes -- 26 tickets fixed ranging from look and feel tweaks to loader and cross domain (xd) loader bug fixes to severe bugs like one in dojox.cometd

Hi all,

Last week, I blogged about installing Windows XP SP3 and how it affects different versions of Internet Explorer (See my earlier blog post here). Today I will be discussing installing branded/custom versions of IE7 on machines with Windows XP SP3 installed. This post is primarily aimed towards folks who use the Internet Explorer Administration Kit 7 (IEAK7) to create custom IE7 packages, like Internet Service Providers (ISPs) and web developers. If you ever installed the IEAK7, built a custom version of IE7 or distributed a version of IE7 to others, this post is for you.

When installing a branded version of IE7 (like the one you get on a Comcast or Qwest CD when you sign up for their services) on Windows XP SP3 machine for the first time, the IE7 install might fail with the following error:

“Process 'xmllitesetup.exe /quiet /norestart /er  /log:C:\WINDOWS' exited with exit code 61681”<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

The reason is that the IE7 package you are trying to install uses old IE7 files. As you may recall, in October of 2007 we released an IE7 update, which in addition to turning on the menu bar by default and removing WGA validation also addresses the XMLLite issue above.

XMLLite.dll is one of the components that ships with IE7. This DLL is necessary to run IE7, and IE Setup installs this component as part of IE7 installation. XPSP3 contains an updated version of XMLLite.dll, so when you try to install an older version of IE7 on XPSP3 machines, IE Setup fails to install XMLLite since it’s already on your system; hence, you get the error. In the IE7 update, we modified the install logic to only install XMLLite if it’s not already present on the system.

Call To Action

If you produce custom IE7 packages, you need to ensure that those packages will install successfully on Windows XP SP3. You can either try installing IE7 on a Windows XP SP3 system, or for a quick test, you can verify the cache of the IE7 files that were downloaded when generating custom IE7 packages. To verify the cache, on the machine that has the IEAK7 installed, go to C:\Program Files\Microsoft IEAK 7\Download\Win32\<Language>\iebin and search for IESetup.msi or IEBrand.msi.

If those files are not present, then you need to perform the following:

1. Download the new IEAK7 available at TechNet.
2. Run the new IEAK7 wizard.
3. Open the INS file you generated for custom IE7 packages. (You can re-use an existing ins file or create a new one, in which case this step is optional.)
4. On the Automatic Version Synchronization screen, click on the Synchronize button. This step downloads the latest IE7 setup files that it will use to generate a new branded package.
5. Complete the rest of the wizard, and click Finish.

The new packages will be created in the directory you specified during the beginning of the IEAK Wizard. These new packages will work on XPSP3, so you are ready to distribute them to all your customers.

Thanks,

Jane Maliouta
Program Manager

I had the chance to talk about Dojo at JavaOne last week, and one of the attendees, Kenn Thisted, was nice enough to create OpenSearch plugins for dojotoolkit.org and api.dojotoolkit.org:

http://mycroft.mozdev.org/download.html?name=dojo

If you are using Firefox 2 or 3, or Internet Explorer 7, you can easily add Dojo search to your browser's search bar.

This is important, so please read on ...

A long time ago, I proposed a patch to Dojo, adding in a little gem of JavaScript to each and every test case in Dijit: _testCommon.js ... it was magic. It loaded whichever theme we chose safely, and set a class name on the body tag, just as dijit themeing requires. It also allowed us to test pseudo-a11y mode, and genuine right-to-left mode for each of the widgets at our whim. This kind of testing is important to Dijit, and as seen by the "matrix", is extensive. We have to test at a _minimum_ each of the "tundra" tests in all of dijit's supported browsers: Internet Explorer 6 + 7, FireFox 2 (and FF3 coming), Safari 3 (ideally both MacOS and Windows variants, as they differ some), as well as fun testing on "technically unsupported" browsers like the WebKit in the iPhone, Opera 9 (which Dijit mostly works with, just not guaranteed) ... It's a huge matrix. _testCommon.js helped with that some, but did something entirely anti-Dojo: magic.

This is my plea: grep your html. If you are inadvertently loading in _testCommon.js via a script tag, remove it. It is 2k of JavaScript entirely unnecessary in any production environment, or any website whatsoever. We use it as a test mechanism, though it introduces magic that causes a lot of confusion, especially among new
users of Dojo. So go. grep your html. If you have a reference to _testCommon.js, remove it. I'll wait.

If something broke, you've come back. Taking the "magic" away from our test-matrix exposes the downside of me having introduced testCommon in the first place. Again with the "there is no Dojo magic here", _testCommon.js breaks two (or more) fundamental Dijit paradigms:

The default Dijit theme is called "tundra" ... It is done entirely based on CSS, and uses a selector ".tundra" to prefix all the class definitions. _testCommon.js was adding the appropriate class name to the body tag, the most top level node available. You can technically utilize CSS inheritance / cascading, and simply put a class="themeName" around your widget dom, but some things that are direct descendant of body (Dialog, Tooltip, Popups) need the class on the body.

the safest way:
<body class="tundra"><!-- my page --></body> you need to load the theme.css!

each theme available has a folder in dijit/themes/ based on the theme name. To load the "tundra" theme, you need the file dijit/themes/tundra/tundra.css ... There is an opportunity for optimization there, but currently the only supported theme mechanism is to pull in the theme "rollup" of all the available components. _testCommon.js was using document.write (of all things) to include the link tags necessary to load the theme files. This is "magic", and also left our test cases without any useful CSS being loaded. This "magic" also probably introduced a race condition in several widgets who relied on the CSS being loaded in order to size properly. It is our recommendation you load theme theme css file before loading dojo.js to limit risk of hitting this race condition.

the recommended way:
<head>
        <link rel="stylesheet" href="js/dijit/themes/tundra/tundra.css" />
        <script src="js/dojo/dojo.js" type="text/javascript"></script>
</head>
<body class="tundra"><!-- your markup --></body>

People use our tests to learn, to copy, to start. Most of our tests are copies of previous tests, modified for some new widget. Having this "magic" _testCommon.js in each and every page propagated throughout all of dijit, some of dojox, and into user pages (I'm no exception to this copy/pasting) ... The Above "skeleton" page is all that is needed to use Dijit. Add another script tag for all your dojo.requires(), and flood the body with markup -- you are on your way.

All this said, _testCommon.js is _still_ in every test file! Ignore it, don't use it, remove it from your pages still ... We still use it to test Nihilo and Soria themes, as well as do the automated magic to set Right-to-Left mode and pseudo-a11y mode. It is still very important for us to test all these modes, and we like having the helper. Though now _testCommon.js does nothing magic until you force it, and the default theme and configuration is reflected throughout all the dijit and dojox tests.

I hope this clarifies some stuff for everyone.

Navigating a mobile app can be slow, especially on long pages and slow scrolling phones. Fortunately the xhtml mobile profile markup language supported by mobile phones provides a solution to finding links and starting phone calls inside the mobile browser.

Adding the accesskey attribute to link a lets users “click” on that link by simply pressing a number on their phone’s keypad. Valid values for accesskey can be 0-9,#, and * (all of the keys on a standard phone keypad). Displaying which key will activate a link is up to the application as most phones won’t tell the user that a link has an access key. Web site designers currently need to decide on a consistent way to inform users that an access key is associated with a given link. Most apps will use ordered lists where the order of the links corresponds with the access keys. Putting the number in brackets inside or next to the link is another way to denote the accesskey.

CNN’s mobile site makes good use of accesskeys. They have chosen to associate the main sections of their web site with the access keys 1 through 9:

CNN uses a span with the access key number to highlight to the user what number to press to activate the link:

<span class="rebg_num"> 1 </span>
<a accesskey="1" class="base_text" href="/cnn/item/world">World</a>

Pagination links, home page, and any other page that the user will use often are good candidates for adding an accesskey. Consistency is key: if home is (1) on one page, then it should be (1) on all the other pages where that link exists. If users can rely on a keys behaving constantly they will use them. Just like other information architecture exercises, take the necessary time to properly allocate the 12 accesskey options to the pages in your app.

Like the accesskey, starting a phone call from a link is just as simple. Just as mailto: will open an email message on a desktop web browser, adding href=”tel:5555555555″ to a link will start a call and dial 555-555-5555 automatically for the user. Keep in mind that most phones not developed by Apple cannot handle a web or WAP session and a phone call at the same time, so the session will end when the link is clicked.

Using both the access key and the tel protocol will enhance your mobile applications by saving the user time and integrating the mobile web experience with the voice features on the phone. Consider adding them to your existing mobile applications or using them on your next mobile project.

In March, we introduced the SitePen Support service to provide high-quality support for Dojo, DWR and Cometd from the people who know those projects inside and out.

We’re always watching to see how we can make our services better and after listening to our early support clients we’re making changes to ensure that clients are successful with the products we support.

The biggest change is that all of our Support plans feature “Ask the Experts”, including our least-expensive Boost plan. Instead of solving the “help me out of this hole” types of issues, “Ask the Experts” provides tailored advice for product teams well before they hit issues or go down the wrong path. For sophisticated development projects we understand that what’s needed often isn’t a bug fix but rather answers to questions like “is this the right way to do this?” or “does this architecture make sense?” In our work with development and support clients we’ve found that providing guidance can help save everyone a lot of time and money down the road. Our goal isn’t just to fix bugs, but to make projects successful. Many SitePenners are quite active in answering questions on project forums and mailing lists, but when teams outgrow the types of support the community can provide, SitePen’s support services give priority, confidential advice and assistance when it’s needed most.

“Ask the Experts” is critical for ensuring project success, and it’s now part of all of our Support plans.

We’ve also simplified our plans, and all plans now include support for Dojo, DWR and Cometd. If you’re looking for a bit of help on your current project, SitePen Support is available to get you the help and advice you need.

I recently picked up Ron Paul’s “The Revolution: A Manifesto”. I’ve made my way through the first two chapters which do a great job of simply explaining the problems with our government. A few of the highlights include, for example, the 1996 election won by the Republican party:

The Contract with America was typical of what I have just described: no fundamental questions are evan raised, and even supposedly radical and revolutionary measure turn out to be modest and safe. In fact, the Brookings Institution in effect said that is this is what conservatives consider revolutionary, then they have basically conceded defeat.

Basically the Republican party campaigned on a platform of change, and then changed nothing. That sounds a lot like the 2006 Democratic campaign actually…

Needless to say, I am also unimpressed by the liberal Left. Although they posture as critical thinkers, their confidence in government is inexcusably naive, based as it is on civics-textbook platitudes that bear absolutely no resemblance to reality.

I have long been bothered by the assumption that it is the government’s responsibility to fix every thing that is wrong with the world, from healthcare to foreign aid. Frankly, I don’t trust the government’s ability to spend my hard earned money better than I spend it, nor do I agree with giving the government the authority to look out for the best interests of “society”.

Watching candidates “debate” this past election season was beyond painful, because the debate was pointless at best:

No wonder frustrated Americans have begun referring to our two parties as Republicats. And no wonder the news networks would rather focus on $400 haircuts than matters of substance. There are no maters of substance.

To sum things up, I leave you with thoughts on what government’s role should be:

When we agree not to treat each other merely as means to our own selfish ends, but to respect one another as individuals with rights and goals of our own, cooperation and goodwill suddenly become possible for the first time. My message is one of freedom and individual rights. I believe individuals have a right to life and liberty and that physical aggression should only be used defensively. We shoudl respect each other as rational beings by trying to achieve our goals through reason and persuasion rather than threats and coercion. That, and not a desire for “economic efficiency”, is the primary moral reason for opposing government intrusions into our lives: government is force, not reason.”

I highly recommend picking up a copy of The Revolution as this book does an amazing job of simply focusing on the issues that matter and cutting through the common rhetoric and political platitudes we normally read and watch.

There is a growing list of projects, companies, products, and sites using Dojo and we need your help to fill in the blanks.

If you are using Dojo and won't get fired for talking about it, add your name to the list and it'll show up on the front page of the new design!

Here's my presentation on Memcached and MySQL:

You can download the sample files here:

• memcached-demo.tar.gz [4MB]

Thanks to all the hard work of the Mozilla community as of 9:15 AM PDT today we are code complete for Firefox 3 Release Candidate 1 (RC1). New nightly builds are available - if you are a nightly tester/Minefield user you can help test these builds by selecting “Check for Updates” from your help menu.

Assuming no new issues are found today the build team will start official prep work for Release Candidate 1 (RC1) tomorrow. QA will start their extensive RC1 test pass on Monday. If all goes well we should have the Release Candidate publicly available in late May.

RC1 is intended for wider scale public testing. Our 1.2M+ active beta users will automatically get updated to RC1 when it is released. If no new showstopper issues are found in RC1 it will become Firefox 3 final. If we find any critical issues we will continue to release new Release Candidates until we are ready for final ship.

Recently I was writing a “tips and tricks” blog post that was going to focus on the idea that it is better to use an object as a “string buffer”; the idea was that by passing this object around to various functions and pushing string fragments into it, you can get better performance from a JavaScript engine. My friend and colleague Alex Russell challenged me to show him hard data supporting this hypothesis—and the results were quite eye-opening!

For this analysis, I used two sources for tests: the dojox.string.Builder Builder performance test, and a custom test implementing three versions of a common JavaScript task: a JSON Serializer. The initial concept was to show that pushing strings into an object buffer would be faster than using the simple JavaScript operations available to any developer.

From my friend and colleague Eugene Lazutkin comes this excellent explanation about strings in programming:

Many modern languages (especially functional languages) employ “the immutable object” paradigm, which solves a lot of problems including the memory conservation, the cache localization, addressing concurrency concerns, and so on. The idea is simple: if object is immutable, it can be represented by a reference (a pointer, a handle), and passing a reference is the same as passing an object by value — if object is immutable, its value cannot be changed => a pointer pointing to this object can be dereferenced producing the same original value. It means we can replicate pointers without replicating objects. And all of them would point to the same object. What do we do when we need to change the object? One popular solution is to use Copy-on-write (COW). Under COW principle we have a pointer to the object (a reference, a handle), we clone the object it points to, we change the value of the pointer so now it points to the cloned object, and proceed with our mutating algorithm. All other references are still the same.

JavaScript performs all of “the immutable object” things for strings, but it doesn’t do COW on strings because it uses even simpler trick: there is no way to modify a string. All strings are immutable by virtue of the language specification and the standard library it comes with. For example: str.replace(”a”, “b”) doesn’t modify str at all, it returns a new string with the replacement executed. Or not. If the pattern was not found I suspect it’ll return a pointer to the original string. Every “mutating” operation for strings actually returns a new object leaving the original string unchanged: replace, concat, slice, substr, split, and even exotic anchor, big, bold, and so on.

The issue is usually string operations come with a “hidden cost”; because strings are (in theory) immutable, we would expect that any kind of string mutation will come with the costs of copy and replace, taking up valuable CPU cycles, consuming memory space, and putting strains on a garbage collector (among other things). A typical technique used in many languages is to use something you can pass by reference (like an object or array) to your methods, and push string fragments into that—delaying the cost of string mutation until it is absolutely needed. In fact, this was the exact reason I wrote the original dojox.string.Builder; I borrowed the idea from C# as a way of increasing efficiency for large string operations, such as building a document fragment on the fly or serializing a JavaScript Object Literal.

(A side note: the original has gone through quite a few iterations, the latest one by Ben Lowery, who was initially using Builder extensively with Bloglines.)

After doing some fairly extensive testing, I learned that with the modern browsers, this is not the case at all; in fact, not only were typical string operations optimized cross-browser but the performance of dojox.string.Builder was dismal, particularly when compared against native operations.

The chart at the top of this article is the summary of native operation performance. All tests were done on a MacBook Pro 2.4Ghz Core 2 Duo, with 4GB of installed RAM; tests were run on both OS X and on Windows 2003 Server running under Parallels Desktop (with enough RAM allocated to ensure consistent results).

The first set of tests run was based on the tests made by Ben Lowery (the BuilderPerf test, see link above), using a Lorem Ipsum dictionary with the default numbers (1000 iterations using 100 words). The basic concept is to test performance based on both one-time and iterative operations. As an example, here are the tests for concat() once and concat() for:

{
    name: "concatOnce",
    test: function() {
        var s = "";
        s = String.prototype.concat.apply(s, words);
        return s;
    }
},
{
    name: "concatFor",
    test: function(){
        var s = "";
        for(var i=0; i<words.length; i++){
            s = s.concat(words[i]);
        }
        return s;
    }
}

…where the variable words is the dictionary initially generated used for all the tests.

The results were about what we would expect: one time operations in general executed much faster than iterative ones. Here’s the chart again:

One interesting result: the += operator on strings is actually faster, across the board, than [string].concat().

Taking a closer look, we can also accurately gauge performance by browser. For example, comparing the major browsers with the once operations:

…we can see that the two most common browsers, Internet Explorer and Firefox (under Windows) have some issues; join performance with Firefox is not great but concat performance with Internet Explorer is absolutely dismal.

However, one time operations such as these—while attractive and certainly the best performing—are not nearly as common as operations made during some sort of iteration. Here’s the comparison of major browsers using the three most common operations during iteration (join, concat and the “+=” operator):

We can draw a few conclusions from this particular chart:

1. The += operator is faster—even more than pushing string fragments into an array and joining them at the last minute
2. An array as a string buffer is more efficient on all browsers, with the exception of Firefox 2.0.0.14/Windows, than using String.prototype.concat.apply.

The performance of dojox.string.Builder was also measured with this battery of tests; we’ll present that data later in the article.

The second round of tests was performed using three slightly different versions of a typical JSON serializer, based on the one used in the Dojo Toolkit. The serializer has three versions:

1. Recursive serialization using += (passing strings as the result of a function)
2. Recursive serialization using local arrays as buffers (created internally to a function)
3. Serialization using a single array passed to methods as an argument with no return (i.e. void)

The test takes a large pre-loaded Javascript Object Literal structure (the serialized version weighs in at 19,200 characters), and serializes it using all three serializers 10 times, taking the average result. Each test is run 5 times in succession. Finally, 20 samples were taken for each browser.

The expectation was that a single object as a string buffer would yield better performance results, consistently across browsers.

In all cases (with the exception of Opera 9.27/Win), using a single object as a string buffer performed the worst, with the best general performance coming from using recursive serialization with the “+=” operator.

To shed a little further light—and to accurately gauge the performance of each browser—we can compare each serialization technique:

As detailed elsewhere, Safari (on both OS X and Windows) is by far the fastest executing browser; unexpectedly, Opera 9.27—which historically has an excellent track record in terms of performance—was the least performant in these tests. Also of interest is the fact that the cross-platform browsers (Safari, Firefox and Opera) remain fairly consistent in terms of actual performance.

With the results of both rounds available, an analysis of the performance of dojox.string.Builder was undertaken. Performance numbers (pre-optimization) were captured during the first battery of tests, revealing a number of issues with the Builder.append method (by far the most commonly used method). However, some investigation and testing resulted in up to a 10x increase in performance—in some cases, being either comparable to or even beating certain browser operations!

Because of the very good performance in Safari and Opera, the goal of the optimization process was to address shortcomings in both Internet Explorer and Firefox.

Carefully analyzing the original code with dojox.string.Builder.append:

append: function(s){
    return this.appendArray(dojo._toArray(arguments));
}

…we determined that a number of things were causing performance issues:

• The call to dojo._toArray, with the arguments object
• Passing the results to dojox.string.Builder.appendArray, without testing first for a shorter branch
• Dynamic arguments object access

The first change made introduced a branch, for all browsers, that checks to see if more than one argument was passed.

append: function(s){
    if(arguments.length>1){
        return this.appendArray(dojo._toArray(arguments));
    } else {
        this.b += s;
        return this;
    }
}

By performing this simple check, we eliminate the need for calling both dojo._toArray as well as the passthrough to the method appendArray. This change increased the performance of the append method with dramatic results—referring to the chart above, this was by far the greatest performance enhancer.

However, when passing multiple arguments to the append method, both Internet Explorer and Firefox slowed down dramatically.

The internal string buffer used for Internet Explorer is different than any other browser; based on the test results from our first round, we know that pushing string fragments into an array and then join-ing them only when a complete string is needed is the best method for string handling.

Knowing that the culprit for real performance issues is the call to dojo._toArray with the arguments object, the goal was to eliminate the need for using it. One simple way to deal would be to simply iterate through the arguments and append each one, one by one:

append: function(s){
    if(arguments.length>1){
        for(var i=0, l=arguments.length; i<l; i++){
            this.b.push(arguments[i]);
        }
    } else {
        this.b.push(s);
    }
    return this;
}

This helped a bit, but there was not a dramatic increase in performance. To eek a bit more out of the method, we tested the difference between using [].push and manually specifying the next available index in the array; we found that specifying the next available index was a bit faster. Finally, we resorted to an implementation of Duff’s Device, a technique for loop unrolling. Here’s the final code:

append: function(s){
	if(arguments.length>1){
		//	Some Duff's device love here.
		var n = Math.floor(arguments.length/8), r = arguments.length%8, i=0;
		do {
			switch(r){
				case 0: this.b[this.b.length]=arguments[i++];
				case 7: this.b[this.b.length]=arguments[i++];
				case 6: this.b[this.b.length]=arguments[i++];
				case 5: this.b[this.b.length]=arguments[i++];
				case 4: this.b[this.b.length]=arguments[i++];
				case 3: this.b[this.b.length]=arguments[i++];
				case 2: this.b[this.b.length]=arguments[i++];
				case 1: this.b[this.b.length]=arguments[i++];
			}
			r = 0;
		} while(--n>0);
	} else {
		this.b[this.b.length]=s;
	}
	return this;
}

This loop unroll brought the performance of append with multiple arguments down to the same, or faster, than using append with a single argument multiple times.

The last set of optimizations involved fixing the dismal performance in Firefox. Like with Internet Explorer, Firefox had major problems when using dojo._toArray, so the first order of business was to move iterating over the arguments object within the append method itself:

append: function(s){ 
	if(arguments.length>1){
		var i=0;
		while(i<arguments.length){
			this.b += arguments[i++];
		}
	} else {
		this.b += s;
	}
	return this;
}

What this revealed was that accessing members of the arguments object seemed to be very slow—to the point where just iterating over each argument caused enough of a performance hit to actually decrease the performance over using dojo._toArray!

With that in mind, and the help of my colleague Kris Zyp, what we determined was that the issue in Firefox isn’t so much that accessing members of the arguments object is the issue; it’s that dynamic access of the arguments object is the problem. With that in mind, a different kind of loop unroll was called for:

append: function(s){ 
	if(arguments.length>1){
		var tmp="", l=arguments.length;
		switch(l){
			case 9: tmp=arguments[8]+tmp;
			case 8: tmp=arguments[7]+tmp;
			case 7: tmp=arguments[6]+tmp;
			case 6: tmp=arguments[5]+tmp;
			case 5: tmp=arguments[4]+tmp;
			case 4: tmp=arguments[3]+tmp;
			case 3: tmp=arguments[2]+tmp;
			case 2: {
				this.b+=arguments[0]+arguments[1]+tmp;
				break;
			}
			default: {
				var i=0;
				while(i<arguments.length){
					this.b += arguments[i++];
				}
			}
		}
	} else {
		this.b += s;
	}
	return this;
}

With this unrolling technique, we accomplish the following:

1. We create a temporary string buffer (tmp)
2. We take advantage of Javascript’s fall-through switch…case structure to deal with up to 9 arguments
3. We specify in each case branch the actual argument number
4. We prepend the argument to the temporary buffer
5. …finally appending the result to the internal buffer (this.b).

This unroll did not noticeably affect the performance of either Safari or Opera, but made a huge difference with Firefox; from the chart above, you can see close to a 10x increase in performance with both the Windows and OS X versions of Firefox. Note that passing more than 9 arguments will cause Firefox to fall back to the dynamic arguments access version; in theory we can simply add more case statements to deal with this, but from a practical standpoint 9 arguments should be a good limit.

Finally, we should compare how the new optimized version of dojox.string.Builder compares to the native operations we analyzed in Rounds 1 and 2. Again, the tests (including a new one that tested the performance of Builder.append with mulitple arguments) were run, as they were with Round 1—this time with the optimized Builder.

With Safari and Opera, dojox.string.Builder is still slightly slower when used in iterative situations; however, the difference is close enough so that one should not find a noticeable decrease in performance when using it. With Internet Explorer, we find that the performance of the Builder is still pretty bad when compared to native operations.

However, with Firefox, the only operation faster than using Builder.append is the “+=” operator; both join and concat are close enough to Builder.append that it makes no difference which version you use.

Though this analysis, a number of things about string performance have been observed:

• Native string operations in all browsers have been optimized to the point where borrowing techniques from other languages (such as passing around a single buffer for use by many methods) is for the most part unneeded.
• Array.join still seems to be the fastest method with Internet Explorer; either += or String.prototype.concat.apply(”", arguments) work best for all other browsers.
• Firefox has definite issues with accessing argument members via dynamic/variables
• And of course, the reminder to not ignore the data

For those numerically inclined, you can download the PDF version of all the data collected.

So after yesterdays idea of measuring how long it would take for a node to “turn red”. I came to the decision that that probably isn’t the best way to measure overall techniques and impact of using large amounts of descendant selectors in your CSS.

My new approach for today is sheer volume.

Using my test page from yesterday that has a 20 column by 1000 row table, with each cell having it’s own unique id and class.

I have now extended it to generate CSS in the head 1 style declaration for each cell, same as before the background is just red. This means that in the head of my test page I have 20,000 style declarations.

I am then able to make two versions of this file, one using the format:

1.  
2. .tdxx{
3.   background: red;
4. }
5.  

and one where they are all:

1.  
2. table tr td.tdxx{
3.   background: red;
4. }
5.  

All of the content is generated with PHP loops, so I ran the two test pages through my local apache and saved the generated HTML. I saved the two files to my desktop and then opened them in various browser.

Once open and loaded and refreshed them 3 times and recorded the results for IE6, IE7, Firefox 2 (osx), Firefox 3 (osx), Safari 3 and Opera.

I went through and ran it in all the above browsers and then collated the data. In case you’re curious, Opera 9 (osx) couldn’t even show the page, and would choke and crash, so I don’t have any data on them.

The results were rather interesting:

Which looks like:

The green bar is the average time that the page took to render when you use 20,000 direct class name css declarations, the blue is the same but using lots of descendant selectors with the class name (see the example above).

You can clearly see the trend that single class name declarations are faster. But what really amazed me was how much that seems to impact Safari and to a lesser degree IE7 more than Firefox. Firefox 3 running on a mac had probably a negligible difference (based on this being an average of 3 refreshes rather than hundreds of them).

I was also surprised that the impact on Safari is the reverse of what I expected. I expected the slow one to be as fast as the fast one on Firefox and the fast way to be an improvement. The reverse seems to be the case. Descendent selectors are a major slow down to the Safari browser, rather than single class declarations being a performance gain.

That is surprising to me, considering all of these tests were run on OSX (the IE tests were run in a memory starved Parallels image). We have to be careful doing browser to browser comparison, as the tests weren’t all conducted at the same time under the same circumstances.

The tests show that there is slow down using child selectors over direct class name declarations in IE6, IE7 and Safari 3. Safari 3 being the most impacted by descendant selectors. Firefox 2 has some impact, and Firefox 3 doesn’t seem to be impacted at all.

That said, this is a very extreme test, it is not often you’d have 20,000 class definitions in a single page or that all of them would use 4 levels of child selector.

That said it is something to be aware of and keep an eye on, and if you’re looking at ways to optimize css reduction of descendent selector usage might help.

If you’re interested in running the test, I have the two files as a ZIP for download (they upzip to 2+ megabyte HTML files, so I zipped them). Just run them locally and you can see the results at the top of the page.

The Dojo Feature Explorer now has an Atom feed you can subscribe to to keep up with all the latest demos that have been added.  So, point your browser at http://dojocampus.org/explorer/featureexplorer/latestDemos.xml and subscribe!

For all the Web 2.0 addicts out there, I’ve also added access to the list via SCRIPT IO.  If you point your browser to http://dojocampus.org/explorer/featureexplorer/latestDemos.php, you will receive a file containing a JavaScript array of the last ten demos added to the feature explorer.  By default, it will call the function dojocDemos, passing it the array.  You can specify the callback function you want using the callback parameter. e.g.

http://dojocampus.org/explorer/featureexplorer/latestDemos.php?callback=myFunction

will call the myFunction JavaScript function.  You can load this file, as I mentioned, using SCRIPT IO.  This could be hard coded into your HTML page, e.g.:

<script type=”text/javascript” src=”http://dojocampus.org/explorer/featureexplorer/latestDemos/php?callback=myFunction”></script>

or loaded using dojo.io.script.

Jim brought up an interesting discussion yesterday about CSS performance and if child selectors are inefficient.

The basis for the thinking is something Dave Hyatt mentioned in a Shaun Inman post. Basically that child selectors are ineffecient because they actually work right to left rather than left to right to find the element you’re referring to.

So this…

1.  
2. body table tr td.myTestClass{
3.   background-color: red;
4. }
5.  

is allegedly considerably less efficient than…

1.  
2. .myTestClass{
3.   background-color: red;
4. }
5.  

Now I figured it would be interesting to see if I could throw together a quick test that might show this difference. Testing and benchmarking CSS is not something I’ve seen very much of, so I assumed it was either A: Virtually impossible, B: A silly idea, C: Boring, but that never stopped me before, so I headed to the whiteboard.

At the very top of the a HTML I would use Javascript to create a timestamp, this is the starting timer for everything.

I would then have a function on a setInterval so it loops, the function checks a dom element using getElementById, it checks to see if that node is there, if it is it checks to see if the node has the style applied to it, if it does it fires a new timestamp marks the difference between the start and the end and then writes that out (I wrote it out to the page, as I was initially using console.debug, but that’s not very cross browser compatible).

I then have 2 css declarations, one with lots of child selectors and one direct class reference.

I then have a 20 x 1000 table created using PHP loops.

The CSS is applied to two of the table cells. Next to each other.

The null hypothesis is that running these both in the same page, the child selector way will take longer to be seen as being “red” than the direct style.

The test can be viewed at:

http://test.jpsykes.com/cssParsing/test2.php

There is almost no difference. Funnily enough there’s a fairly consistent 7ms difference which I guess is either the time it takes to do the tests in order, or it’s the fact that the second node is further down the dom.

So no difference. Bit of an anti-climax.

I have a couple of ideas:

The test is fundamentally flawed in a way that’s not immediately obvious to me.

The two CSS that I’m comparing don’t display the issue, maybe I need different example.

Maybe the browser does everything CSS then renders the page, so 1 might be happening faster, I just can’t test it. (I’m going to make a different test where I have two pages with lots of each example and I test overall load time rather than individual node styling time).

One thing I did find that was interesting. During my testing and debugging I added an onload timestamp as well. I was getting the “Node is red” timestamp, before the onload. Which I thought was interesting.

More to come.

A quick entry today. We've done date pickers before, but the code's changed since previous Dojo versions. This demo is simple and on target for adding the required HTML attributes to your Notes date field so that you get a dojo date picker control on the web. It also supports validation of required or not required and can handle international date formats.

See the Demo application, or go straight to the Date Picker demo.

This is the final part of our three part series on porting Dojo methods to Flash.

In part one of our series, we implemented Dojo’s hitch method, and then used hitch in part two, where we made our connect method. In part three, we will be using both of these methods as we connect a JavaScript object to a Flash object.

We’ll start by setting up our test case. We’ll need an HTML page with an embedded SWF that contains a few Flash buttons and a text field, and then a text field in HTML:

Our overall goal is to tell the SWF, “When a Flash button is clicked, trigger a function in the JavaScript environment thats sets my text field”.

Our SWF file needs to provide a channel for JavaScript to communicate with it. In Flash, import the ExternlInterface (the Flash 8+ method to communicate with its container environment) and use that to place a function in JavaScript which we’ll call connectToSwf, and link that to a function in the “this” scope called jsConnect:

// ActionScript
import flash.external.*;
ExternalInterface.addCallback("connectToSwf", this, jsConnect);

The jsConnect is quite simple. It will accept three arguments, all strings, even the object. We have no actual access to the SWF button, so we couldn’t pass a reference to it, even if we could get that reference through the ExternalInterface. So we pass a string for the scope and the event, and then use the lang.connect code that we built in Part 2 to connect to the button and its event. When this connection is made, the ExternalInterface makes a call to JS to an event that we will provide ( swfEvent.connectToJs ), passing strCallback back. swfEvent.connectToJs is hard coded in the SWF, but this could be passed in as well.

// ActionScript
function jsConnect(strSwfScope, strSwfEvent, strCallback){
	lang.connect(_root[strSwfScope], strSwfEvent, function(){
		ExternalInterface.call("swfEvent.connectToJs", strCallback);
	})
}

Okay. I cheated. This code will only work if the strSwfScope is on the root of the timeline. A more robust version might use resolveObject(strSwfScope) where that function could split a string on the dot delineator and loop through each string until the object is found.

The buttons are created and named “flashButton1″ and “flashButton2″, a text field is placed for reference, and the SWF is published. To insert the SWF into the page, we’ll use dojox.av, by requiring the package, waiting for the initialization, and then placing our SWF file.

// JavaScript
dojo.require("dojox.av");
 
var flashMovie;
var testMovieUrl="connect.swf";
dojo.addOnLoad(function(){
	flashMovie = dojox.av.flash.place(
		dojo.byId("fmovie"), 
		{ path: testMovieUrl, width:320, height:125 }
	);
});
flashReady = function(){
	setTimeout(init, 0);
}

We used the dojox.av.flash.place method, which takes two arguments. The first is the node where you wish your SWF to be inserted. The second is an object, which takes several parameters, three of which we are using here: path, is the path to your SWF file, relative from dojo.js, or relative to the file if you are using CDN; and width/height, which is the size of our SWF. This method returns a pointer to the SWF object which we will use later. Note that a SWF can take an undetermined time to load - it’s not ready right away, especially if it’s large. Within Flash I’m calling flashReady, which tells the JS that it is loaded and ready to go - however, there still needs to be a little breathing room before the ExternalInterface is ready. Hence, the setTimeout(init, 0).

The init function is where we will build our code to connect to the Flash object. We’ll use swfEvent as the namespace. Here is the structure, followed by the two connections to the Flash buttons:

// JavaScript
init = function(){
	swfEvent = {
		_swfConnections:{},
 
		connectToJs: function(strCallback){
			//
		},
 
		connectToSwf, function(strSwfScope, strSwfMethod, scope, method){
			//
		}
	};
 
	swfEvent.connectToSwf("flashButton1", "onRelease", function(){
		dojo.byId("tf").value = "Flash Foo";
	});
	swfEvent.connectToSwf("flashButton2", "onRelease", function(){
		dojo.byId("tf").value = "Flash Bar";
	});
 
}

connectToJs is the method that the Flash ExternalInterface is using for it’s return call. connectToSwf is what we will build out, and if you notice in our two connections, we need to handle anonymous functions. We can’t pass them through the ExternalInterface, so we will store them, and pass a reference instead. We’ll hitch our scope and method, and then store them in the _swfConnections object.

// JavaScript
var _hitch = dojo.hitch(scope, method);
var strMethod = "strMethod_" + new Date().getTime();
this._swfConnections[strMethod] = _hitch;

_swfConnections now contains a time-stamped reference to our hitched closure (albeit this is a dangerous way to do it if the code executes very quickly).

We now pass everything to the flashMovie pointer that dojo.av provided for us, targeting the connectToSwf function created by the ExternalInterface. All that remians is for our closure to be executed after the SWF return call to swfEvent.connectToJs. Our final code:

// JavaScript
init = function(){
	swfEvent = {
		_swfConnections:{},
 
		connectToJs: function(strCallback){
			this._swfConnections[strCallback]();
		},
 
		connectToSwf, function(strSwfScope, strSwfMethod, scope, method){
			var _hitch = dojo.hitch(scope, method);
			var strMethod = "strMethod_" + new Date().getTime();
			this._swfConnections[strMethod] = _hitch;
 
			flashMovie.connectToSwf(strSwfScope, strSwfMethod, strMethod);
		}
	};
 
	swfEvent.connectToSwf("flashButton1", "onRelease", function(){
		dojo.byId("tf").value = "Flash Foo";
	});
	swfEvent.connectToSwf("flashButton2", "onRelease", function(){
		dojo.byId("tf").value = "Flash Bar";
	});
 
}

And the test result, after clicking the “Set to Foo” button:

Here’s the demo

With this framework, we now can communicate between ActionScript and JavaScript as easily as we can between JavaScript and HTML!

This Saturday, May 10th, is MinneBar, Minnesota's BarCamp. MinneBar is described as an "(un)Conference" which means it's a free, ad-hoc gathering of technology folks where everyone is encouraged to contribute.

There are a lot of great sessions this year. I'll be giving a presentation titled "Memcached & MySQL Sitting in a Tree." The talk is about the new Memcached Functions for MySQL. I'll talk a bit about the what, why, and how about this set of awesome UDFs.

I'm not sure what time I present and I think I have 50 minutes, but I don't know for sure. I'm trying something new this time around; I'll be publishing my presentation on SlideShare.

We are still 3 days away and there are currently 356 people signed up which is right around how many people were signed up last year. If you are in the Minneapolis/St. Paul area, you should come to participate and learn!

To register, visit their website, click the "login" link in the top right, use the password "c4mp" to login, then edit the main page, and add yourself to the bottom. Registration starts at 8:00am, so remember to set an alarm. :)

Hope to see you there!