It's All Geek to Me!

February 05, 2007

Is GooTunes in the works?

I was just trawling through Google's robots.txt and I found "/music" on the list of blocked-off URLs. I also found this post by Danny Sullivan from over a year ago about Google Music Search, which is linked to "/musicsearch". Neither of the two URLs led to anything useful when I checked, but they do lead to different error pages. If we take this to mean that they are not synonyms, then "/music" points to something other than a music search product. What might it be then? A Google Music Store, which we shall call GooTunes.

There is a lot of money in the rich media content business, one that Google is just flexing its muscles at with YouTube. If Google applies its devil-may-care attitude to selling music, we might actually see variable pricing implemented by a major company for the first time--not because the record labels want it but because consumers want it. Google might even take the user's side and try to offer entirely DRM-free music on GooTunes, but watch out Google--Amazon's forthcoming music store is rumored to offer DRM-free, variably priced music. No matter what sort of goodies Google puts into GooTunes, we can be sure it will make at least half the music industry (and Apple!) sit up and take notice.

One thing that worries me about this is that social applications have never been Google's strong suit. Online music stores have generated a lot of buzz not just because the iPod has been successful, but also because they bring together people with similar musical tastes. When you look at Pandora, last.fm and similar sites, it's easy to see that there is a strong social component inherent in the leading edge of innovation in digital music downloads. The question is, can Google execute on it?

Posted by Vishy at 12:22 AM | Comments (0)

January 22, 2007

Google Personalized Home Update: please bring back the old one!

Google's Personalized Homepage just rolled out an update. With the new features, you can expand the contents of a feed item on the homepage itself, without leaving Google. An example of what this looks like is below:

I can't say I am a huge fan of this update. I have several feeds on my personalized home page -- maybe two tabs, each with 15 feeds. The large number of plus signs indicating that an item can be expanded adds a lot of unnecessary visual noise. The timestamp attached to each item is also mystifying. Even an honest to goodness RSS reader, like Google Reader does not feature them so prominently. On my Google Reader's expanded view, the timestamp appears in grayed out text on the top right of each item's enclosing rectangle. I appreciate why the timestamps were put in. Given the narrow rectangle allotted to each feed though, they do nothing more than add to the visual noise.

The personalized homepage from Windows Live presents expanded feed items much more tastefully than its competitor from Google. I understand Google may want to keep the personalized home close to its minimalist ethos--no Javascript, floating divs or fade-ins and fade-outs. Still, Google can learn a thing or two from how Live.com presents its wares. At the very minimum, I'd like the option to remove timestamps and expand buttons from my Google Homepage so as to reduce my informational and visual overload.

On a somewhat related point, real RSS readers or pretend ones like the Google Personalized Home make me feel awful by actually trying to be an Inbox For The Web (tm). RSS is not email. All the bolded folders with the counts of unread messages only serve to remind me how behind I am on my reading. I subscribe to many closely related feeds, which share a lot of duplicates among themselves. I may not even need to read the 45 items marked unread on the Slashdot feed because I may already have seen them on digg, Techdirt or Techmeme. I have no choice processing email; I must eventually process every single message that lands in my inbox. But RSS feeds are different; I subscribe to a feed because I want to pick and choose what I want to read from it, and not be constantly reminded how much of it I haven't read. Will some UI whiz please come up with an RSS reader with a fluid UI?

Posted by Vishy at 11:12 PM | Comments (0)

January 17, 2007

Google Promotes Google Checkout on Home Page

There have been a number of recent reports about Google according their offerings privileged visibility over competing services even if they are not necessarily the best in their class. Ordinarily I would think it's logical for a company to promote its own products. Google, though, has unfortunately set itself up for all the flak it is getting because of its insistence on user experience to the point of an evangelistic creed.

I was surprised then to find that Google defiled its home page, that Helen of Web pages that inspired a thousand pages like itself, with a promotion of its own Checkout product. Without a doubt, Google has worked hard and earned the right to promote its own product on the most heavily viewed page in the world. Is it in the best interests of its users though, to have yet another link on that formerly pristine Google home page?

Google may have promised never to advertise anything on its front page that is not relevant; they clarified this position in their response to this unaffiliated page. Then came the advertisements for the Google Search toolbar, which I tolerated because at least they are search related. After all, when arriving at a search home page, seeing an ad for a search tool is not all that irrelevant.

Putting a Google Checkout promo on the front page is something else though. Although Google won't release exact numbers, it is fair to estimate that end users of Google outnumber advertisers by a factor of 1000 to 1. Is it really relevant to subject all visitors to google.com to a product promotion when at most 0.1% are in its target market?

Google is facing competition from competing ad platforms as well as from users who have wised up enough not to click on an online ad. I can see why they are more aggressive about putting pointers to their products on search result pages. Heck, even the tips fracas isn't as bad as this, because at the end of the day you can argue for its relevance to the user's search query. An unnecessary Checkout promo on the Google home page can point to only two things: either Google cares less about its front page user experience, or Google Checkout is doing badly. Or is it both?

[Update: I was wrong about assuming that the target of the above promotion were Google's advertisers. The promotion is indeed aimed at its users. Still, waving $10 at users to get their credit card information when they just want to find some information on the web is distracting and not relevant to their intent. And if I eyeballed the above and got annoyed with my assessment, albeit a mistaken one, why wouldn't others?]

Posted by Vishy at 03:49 PM | Comments (0)

January 10, 2007

iPhone's missing piece: the developer platform?

This post is inspired by a brief conversation I had with Raven.

Amid all the breathless hype around Apple's iPhone announcement yesterday, there was a conspicuous lack of coverage about the openness of the platform. Apple kept mum about a developer platform around iPhone and how open a platform it was going to be. Would it be as closed as the iPod/iTunes duo, for which there is no way to write meaningful applications without reverse engineering? Or would it be as open as development for the general Mac platform, complete with an SDK and a developers' network?

We can perhaps divine the answer to this from Wall Street's reaction to this announcement yesterday. Apple stock rose approximately the same amount percentage-wise that RIM's stock fell. In other words, the market expects iPhone to encroach on RIM's customer base, which is largely made up of enterprises. There is no way Apple can keep the iPhone platform closed and expect enterprises to adopt it. If the market is right (as it usually is), then we should see some overtures from Apple towards providing a proper development platform around the iPhone.

Let's look ahead to when an SDK and other platform components for the iPhone will stream forth from Apple.

[Update: I was totally wrong on this count; I spoke too early. The following day, Steve Jobs clarified that the platform will be closed, just like the iPod. There will be no developer toolkits and community as with desktop Mac applications. And so Apple remains a consumer electronics company at heart and in direction.

I think Apple is sabotaging adoption of its iPhone with this move. Enterprises won't pick it up because the platform is not open enough. Consumers won't pick it up because the phone is tied to Cingular, which (at least anecdotally) has the 'fewest dropped calls of any network' because you can't make calls in the first place with its coverage. Apple's two-year exclusive deal with Cingular, and the price tag aren't exactly going to help either.

And as you might expect, RIM and Palm stock have recovered by quite a bit.]

Posted by Vishy at 10:51 AM | Comments (2)

January 03, 2007

Why are there no obituaries for rich client applications on the Mac?

[Happy New Year to all my readers! -V]

The technology press loves to publish eulogies to rich client software that runs on desktop computers. The era of the rich client application is over, they say. People are already storing a lot of their computing lives online, on Web applications hosted by the likes of Google. Microsoft, the middle-aged software company, which has become so obese as to take 5 years to produce an operating system upgrade, cannot stem the tide of people flocking in hordes to online counterparts of their flagship rich client applications.

Yet rich client applications are alive and well on the Macintosh. There is a host of rich client applications on the Mac for very specialized purposes despite the existence of online alternatives: buying music (iTunes instead of mp3.com and emusic.com), arranging photos (iPhoto instead of Picasa), instant messaging (iChat instead of Meebo) and email (Apple Mail instead of GMail or Yahoo! Mail). Even sundries like a programmer's text editor (TextMate) are available as full versions only after paying a fee. In a time when free is the new pink and giving away software and services has become de rigeur, let's consider why Macs appear to tolerate the business model of old school, unhosted, shrink-wrapped software much better than PCs.

1. Most Mac users are passionate and picky about user experience. Owning a Mac has never been about the value. Mac users know they are paying a premium for a superior user experience. No matter how much Web technologies advance, a Web application running inside a general-purpose browser cannot beat the user experience provided by a rich client application that accomplishes a small set of tasks with a user interface built for a particular purpose. Rich client applications, for the most part, also have unrestricted access to the local filesystem, which enables them to store more nuanced user preferences.
2. Mac users in general are mobile than PC users. Laptop sales have exceeded desktop sales for some time now. Most Macs sold are laptops rather than desktops; as a result, a higher percentage of Mac users are mobile compared to PC users. Mobility is the perfect antidote to the limitations of most rich-client applications. Think about it: if your choices were between using rich client applications on a computer that you have with you at all times and having your applications accessible at all times on a Web site, wouldn't you rather pick the first and have a richer user experience not subject to the vagaries of reliable Internet access?
3. Enterprises have generally not adopted Macs. Enterprise adoption of the Macintosh platform has traditionally been low, although an enterprise-capable Unix underlies Mac OS X. Rich client applications are usually more problematic than Web applications for an enterprise because they raise problems of versioning, deployment, support and user training. By using Web applications where appropriate, an enterprise can alleviate most of these problems. Rich client applications, on the other hand, work well enough for individual consumers, who don't face the problems of maintaining a uniform environment and policy across thousands of computers. Faced with no such pressure, rich client applications are still suitable for the largely consumer-facing Macintosh platform.
4. Software for the Mac is a seller's market. There is so little software available specifically for the Mac that its passionate users will gladly adopt even intentionally crippled rich client software, so long as it is well designed for the few tasks it can do. Contrast this to the Windows platform, where dozens of third-party applications and online services--many of them badly written--compete for a given market segment. iTunes is undoubtedly a crippled application, whose several arbitrary restrictions can be overcome by open source alternatives. Yet, it is the only way to buy music from the iTunes Music Store. Apple has chosen not even to augment the iTunes user experience with a Web component because its users are happy enough with the experience of a rich client music store application.

Continued adherence to the creed of rich client applications could well restrict the choices available to Mac users because the cost to switch away from a rich client application is higher than the cost to switch away from a Web application.

We are in an age where the operating system a computer runs is beginning to matter less and less because most common needs can be fulfilled online. Rich client applications on the Mac present an interesting counterexample to this trend. Surely a clique of rich-client applications on the Mac cannot swim against the tide of increasing adoption of Web applications? Yet, the Mac may end up making us all "think different" about this question.

Posted by Vishy at 09:51 PM | Comments (1)

December 24, 2006

My free knowledge management solution

It was my uncle who made me a news junkie. He used to live in an apartment just down the street from mine when I was growing up. Every time I visited, he would open a folder with a crafty gleam in his eyes. In it, he would have religiously noted down all the interesting tidbits he saw on TV, in a newspaper or a magazine. And from it came countless trivia questions that I could never answer correctly.

I wasn't just determined to pass his test; I had to fight back as well! I got into the habit of reading the newspaper regularly and noting down interesting bits so I too could ask him questions. I never could beat him conclusively; to my exasperation, he would manage to answer my questions easily while still managing to come up with obscure bits of his own. Nevertheless, to this day I remain a news junkie, curating my own mental library of facts.

My brain can only hold so much though, and rather than take the risk of forgetting facts—or worse, muddling them together—I have come up with my own el cheapo (or rather, el freeo) system for managing what I know. My "notebook of wisdom" has recently reincarnated in the twenty-first century as three free programs each for a different purpose:

• Public bookmarking. Any webpage I see that I need to access from everywhere, I put on my del.icio.us page. I also use it to share cool links with the world.
• Private bookmarking. Any webpage I see that I don't need to access from everywhere but want to have accessible for future reference, I put into Evernote. I also use Evernote for stashing away things I don't necessarily want others to see, such as my plans for world domination. Evernote is kind of like Microsoft OneNote, only free, and with some neat searching and tagging features; there is also a premium version with full support for handwritten notes on tablet devices.
• Making sense of it all: mind maps. For quite some time now, I've sketched complex ideas on paper as a set of interconnected nodes. I just didn't know enough people did it that it had a name of its own: mind mapping. I've only recently started making mind maps on the computer, for which I've found Freemind very suitable. Freemind may be a Java desktop application, but it is surprisingly pretty well executed. It may not integrate as well with productivity applications as some of its $229 competitors, but given its lovely price tag of $0, I shouldn't be complaining much. In addition to having some nifty drawing features, it can export maps to HTML and PDF. I've yet to build up a sizeable library of digital mind maps, but I am confident this won't be just a passing fad considering I've been doing them on paper for years.

I don't know if many readers of this blog actually care to organize their knowledge in some way (come on—it's a pretty dorky thing to do, isn't it?), but I'd be curious to know what other people have tried and what has worked well for them.

Posted by Vishy at 11:19 AM | Comments (4)

December 20, 2006

Google Book Search, meet Amazon's Mechanical Turk

According to this Washington Post story about Google's ambitious book-scanning initiative, Google is scanning 3,000 books a day, which works out to about a million books annually. This rate is no doubt phenomenal but brings its own set of problems along.

Scanning a book is easy, but optical character recognition is notoriously hard to get right. Google is said to be currently scanning only those books whose copyright term has expired, which would mean books published before the 1920s. Any books published more than a 100 years before that decade have peculiar artifacts like the integral-sign S or f-like S, illustrations signed with artistic scrawls and manuscripts featuring cursive handwriting. No matter how breathlessly the media waxes about Google's 5,000 Ph.D.'s, these are all hard problems in OCR today.

Consider the scale of Google's operation: let's say each book scanned this year had an average of 200 pages. That would make 200 million pages. Even an OCR program with five-nines (99.999%) accuracy would spit out 200,000 pages with mistakes. One way to achieve dramatically higher accuracy is to ask the millions of avid readers on the Web to double check the accuracy of Google's OCR program, kind of like the distributed outsourcing in Amazon's Mechanical Turk project. It's really not as bad as it sounds; it's merely about offering the right incentives. Thankfully, Google doesn't have to look far for ideas.

Google already does quite a bit of revenue sharing with its AdSense partners, in exchange for the chance to display ads based off content on third-party websites. Google's plan to monetize Google Book Search is to show ads beside the content of the books anyway. So how about taking the money gained from ads shown beside, say Alice In Wonderland, and share some of it with whoever helped check the accuracy of Google's OCR program on that book? The revenue sharing infrastructure is already in place and it'll really be just a question of building the right user interface to divvy up the books among people.

I don't have access to Google's vast set of query strings (and neither does the general public), but it doesn't seem like the revenue distribution would be that iniquitable across books if structured the right way. There could be a cap of say $100 per book to control for books that mention "sex" a lot. Other than that, let human judgement take its course and watch the dollars roll in!

Posted by Vishy at 10:51 PM | Comments (0)

December 12, 2006

10 treasures you can get to with Start > Run...

Although I spend nearly all of my day in pointy-clicky Windows and OS X, I am a CLI geek at heart. I launch a bunch of programs merely by typing in things into Windows' pathetic excuse for a command line, the Start > Run... box. Here are some of the niftiest things you can access from the Start > Run... box in Windows XP:

A way to start a screensaver instantly

Start > Run... > scrnsave.scr and the screen blanks instantly. Useful for when you need to step away from the computer and don't want prying eyes to see what you had on your screen.

A comprehensive system information tool

Start > Run... > winmsd, for when you get hardware conflicts, driver problems or other low-level issues. This tool gives it all to you in one place, free of sugarcoating.

The trusty Control Panel at your fingertips

Start > Run... > control brings up the Control Panel without having to point and click your way through the Start Menu.

A way to clean your hard disk of junk

Start > Run... > cleanmgr and a disk cleanup assistant appears. Although storage is hardly an issue anymore, it's really surprising how much space you can reclaim by deleting your browsing history, cookies, downloaded program files and other crap you didn't even know you had.

An on-screen keyboard

Start > Run... > osk, for when you need to shut down the computer after suddenly spilling java into your keyboard messing it up beyond repair. You can even hover over the keys to press them!

A way to transfer files to your BlackBerry

Start > Run... > fsquirt, enables you to "squirt" (eww) files from your Bluetooth-enabled computer to another Bluetooth-enabled device, like your BlackBerry.

A one-stop startup configuration utility

Start > Run... > msconfig, for when you are sick and tired of your startup being delayed by the 18 million teeny-weeny programs that crowd the system notification area (not system tray).

A definitive answer to "What version of Windows am I running?"

Start > Run... > winver, et voila!, a dialog box with your operating system version (including any service packs you applied) appears.

A way to design your own Klingon font

Start > Run... > eudcedit, gives you a drawing surface using to fill up your own Unicode Private Use Areas. This way, even fewer of your Klingon-speaking friends will know when you write that their mother has a smooth forehead.

An el cheapo chat program

Start > Run... > winchat brings up a really basic chat program using which you can talk to users of other computers in the same Windows workgroup or domain. It was clearly written by a Microsoft developer who sorely misses talk from Unix.

You knew there was a bonus! Start > Run > %WINDIR%\media opens up a directory with the Windows sound schemes. The MIDI (*.mid) files in there are the half-decent lost works of an unknown composer (Brian Eno?). They're somewhat dated, which makes them sound tacky today. The file called "flourish.mid" vaguely reminds me of the Seinfeld theme for some reason. Start > Run... > clock.avi pops up a movie of a cheesy-looking clock that counts to 12.

I'd like to honorably mention Start > Run... > osuninst. I thought it was going to end my love-hate relationship with Windows, but all it did was pop up a cryptic error message instead.

Posted by Vishy at 09:28 PM | Comments (0)

November 22, 2006

Date-related hijinks in Microsoft Excel and Google Spreadsheets

For years, Microsoft Excel has been the de facto standard against which all other spreadsheet programs have been judged. Because so many Microsoft customers use Excel, sometimes running their entire business off it, it has some bugs that are well known but will not be fixed so as to preserve backwards compatibility. For example, Microsoft Excel thinks that 1900 was a leap year, when according to Gregorian calendar rules, it is not (years that end in -00 must be divisible by 400 to be leap years). Preserving backwards compatibility is important in Microsoft's business model, or else people would never upgrade to newer versions of their software. In fact, so paramount is that concern that in a move that seems pretty boneheaded to be honest, this bug has been baked into an ECMA standard to ensure its propagation into perpetuity(see last link for a quote).

Google obviously does not have as big an issue with preserving backwards compatibility, because it deploys software on its own machines. Still, in matters like these, Google can't just change certain behaviors and break thousands of user documents overnight. Besides, if Google wants to inflict death by a thousand cuts on Microsoft's Office empire, it might also need to preserve the same quirks to make conversion from Excel to its spreadsheets easier. So, I fired up Google Spreadsheets (seriously guys, awfully boring name choice) to see what it thought of 1900. Screenshots from both Microsoft Excel and Google Spreadsheets are below:

Microsoft Excel	Google Spreadsheets

The images above both show date equivalents of ordinary numbers in both spreadsheet programs. Both spreadsheet applications convert ordinary numbers into dates by counting those many days forward from a starting point. For Microsoft Excel, that point is Jan 1, 1900. So, 2 would be converted to Jan 2, 1900, the second day of 1900 and so on. February 28th is the 59th day of any year, so 59 should correspond to Feb 28, 1900 in Microsoft Excel. The next day, the 60th day of 1900, should be Mar 1, 1900 but because of the bug in Microsoft Excel the 60th day is shown as Feb 29, 1900, a date that never existed.

Now Google is trying to do the 'right thing' and be Excel-compatible at the same time. Notice how in the right image the date after 2/28/1900 is not in February anymore? This is how Google does the 'right thing', by not considering 1900 a leap year.  To maintain Excel compatibility as much as possible, it chooses its starting point as Dec 31, 1899 — one day before Excel's starting point. This way, all the dates in January and February 1900 are assigned a number one higher than what they'd be assigned in Microsoft Excel. Starting Mar 1, 1900 however, all dates will have the same day numbers assigned to them.

Obviously, this is somewhat of a tradeoff. All Excel spreadsheets that contain dates in January and February 1900 will not carry over properly into Google Spreadsheets. In real life though, unless someone is using Microsoft Excel to keep track of interest accruals dating back to 1900, when a state-of-the-art computer was a human being, the total number of such spreadsheets is close to zero, if not zero altogether. This is a fine tradeoff to make, and a smart one too — it retains geek cred but keeps Google Spreadsheets compatible (in practice anyway) with the ones from Microsoft Excel. Kudos to Google for this!

Posted by Vishy at 08:06 PM | Comments (0)

November 21, 2006

Two annoying GMail chat bugs

Because of the increasing IM-ization of email, I am a regular user of Google IM even though I have never even downloaded the actual IM client. I think this is awesome because now all I need is a Web browser to show my presence.

I will still mention a couple of annoying bugs that are truly disruptive — they disrupt my user experience like hell! I experience both bugs fairly frequently in Firefox on Windows, based on my observations on home and work computers. I have yet to encounter these bugs on Macintosh computers.

• The phantom Shift key. Like most other IM clients, I can hit Shift+Enter in a chat window and insert a carriage return in the middle of a message. Of course, if I just hit Enter, I send the message I just typed. If I have popped out a chat window though, hitting just Enter occasionally inserts a carriage return into my message instead of sending it to its recipient--it's almost as if Shift gets pressed for me for free. I have only ever seen this bug when a chat window is popped out. One workaround is to pop the window in and pop it back out. You may be able to send a staggering 5-10 messages before it happens again.
• The weird focus bug. When somebody sends you a chat message in GMail and its browser window is not in focus, the title of the window alternates between say "GMail (56)" and "Bugs Bunny says...". This is the browser version of the flashing taskbar button from desktop IM clients like AIM or MSN Messenger. After you bring the browser window into focus, this flashing disappears, and the browser window only displays the usual GMail title. If a GMail window has popped-out chat windows, sometimes this flashing does not go away even after it has been activated after a new chat message. Even if I have seen all chats rooted in a GMail window, its title continues to flash back and forth and catches my eye unnecessarily. The only workaround I have found so far is to close the popped-out chat window.

Please fix these popped-out chat window bugs and make an already awesome product even more so. (As I was writing this, I kept typing "pooped-out" chat windows instead of "popped-out", a typo that doesn't seem too egregious, given the current state of affairs. Okay, maybe I am being a bit too harsh.)

Posted by Vishy at 10:44 PM | Comments (2)

November 09, 2006

Adobe makes huge bet on Rich Internet Applications

Adobe makes some very popular software. If Photoshop and Acrobat weren't enough, the Flash player is one of the most installed pieces of software in existence. The success of video sharing sites like YouTube is predicated on the prevalence of browser plugins from Adobe. Yet, there appears to be no imminent acquisition of Adobe by any larger software company. Like Nintendo in the gaming market, Adobe stands alone for now. As other software shops extend their reach into either the enterprise or consumer electronics, what wave will Adobe ride to stay competitive with its bulkier peers in the software industry? If its recent moves are any indication, it will be the wave of rich Internet applications (RIAs).

Recently, Adobe decided to contribute code to the Mozilla project from its ActionScript/Javascript engine, which it calls Adobe Virtual Machine 2. This newly revamped scripting engine is part of Flash Player version 9. The source code will be housed in the Mozilla code base as a separate project called Tamarin.

Rich Internet Applications today come in two flavors: Ajax-based and Flash-based. Firefox's first attempt at delivering rich Internet applications through XUL has not had sufficient adoption to be deemed successful. When the Tamarin code finally ships with the Mozilla suite of browsers in 2008, however, Firefox will effectively have native, plugin-less support for Flash applications. Firefox would then have native support for both Flash-based and Ajax-based RIAs, and thus become a vehicle of choice for delivering such applications. If Firefox adoption continues at the present rate, it will be the browser used by a significant minority of end users; indeed, if all goes well, it might even be the browser of choice for a few enterprises by 2008. Adobe's plan to cover both RIA bases is evidenced by its membership in the OpenAjax initiative.

Adobe's involvement in the ECMAScript working group and the OpenAjax initiative brings it into contact with an number of key players in the enterprise sector as well, such as Microsoft, IBM and Sun. Adobe can capitalize on these alliances to influence future Web-service standards activity and expand adoption of its client/SOA applications based on Adobe Flex. Moreover, Adobe could leverage the collaboration features built into Adobe Acrobat to enhance their platform's suitability for collaborative, document-based workflow systems in addition to RIAs.

Whereas Adobe's packaged software will continue to be a steady revenue stream, the forthcoming explosion of RIAs, especially in the enterprise, may well hold the key to its future.

Posted by Vishy at 11:56 PM | Comments (0)

November 07, 2006

XBox, a video game platform -- for videos and games

(image credit: Engadget)

This November is turning out to be a major media month for Microsoft. Today, they announced the availability of video-on-demand over their XBox Live gaming service. Starting Nov 22, customers will be able to download television and movie content from Warner, Paramount and Turner Broadcasting. Also debuting this month is Microsoft's media player Zune and its accompanying music store. Microsoft's lackluster performance in the increasingly influential arena of Web services is putting the software stalwart's feet to the fire, leaving it to examine other avenues. After the less-than-stellar performance of its media center operating system, this somewhat unexpected move is Microsoft's fresh salvo in the battle for control over the digital living room.

Earlier this year, Apple announced a device dubbed 'iTV', which would blur the line between the computer and TV by streaming Internet video into traditional home entertainment devices. Microsoft's announcement today preempts iTV by capitalizing on a foothold Microsoft already has, independent of its flagship Windows and Office products. Apple's strategy blurs the line between the computer and the TV by approaching it from the computer side. Apple's computers and devices have been increasingly resembling consumer electronics products for a while now. Fundamentally though, they have retained a relatively complex, mouse-and-keyboard driven user interface. Microsoft has upped the ante on the problem of PC-TV convergence by approaching it from the consumer electronics side. Even though Microsoft has years of expertise in traditional window-based graphical user interfaces, it has correctly picked the much simpler gaming controller as a means to bring media into the living room. This move is rather compelling from the point of view of the average consumer who is also a gamer: why go through the trouble of adding another set top box to the crowded entertainment center when one box can provide both digital video and games? Besides, what's closer to the effortless haze of channel surfing: A-B-A-A-Start or moving the mouse to a text field and typing in a search query?

This move, the first to combine video-on-demand and video games, underscores how video gaming has outgrown a small, loyal audience of consumers to become a key prong in a comprehensive media strategy. Because it is the first platform to combine video and gaming in this way, speculations on its future direction will abound. By adding video on demand capabilities to the year-old XBox, Microsoft provides a key competitive differentiator to other third-generation consoles being released this holiday season. Video-on-demand is the right add-on to the XBox because it has traditionally trounced its competitors on hardware and performance. If the video-on-demand service sees sufficient pick-up, Microsoft may further reduce the cost of the loss-leading XBox consoles and exert price pressures on its competitors. In-game advertising, which has seen but a ripple of the online advertising wave, is bound to get a fresh look from advertisers. Media houses uneasy about Apple's Disney predilection or Google's emerging dominance of advertising may be more comfortable about cutting deals with Microsoft. This platform may also be an interesting one for Yahoo! to consider, as it strives to differentiate its ad platform from Google's.

Microsoft is no doubt going to try and integrate Zune with XBox and this service in the future. Watch for the WiFi add-on to the XBox 360 become a standard feature in the console's next iteration.  If only Microsoft doesn't screw up the presumed DRM scheme, the XBox could well become its next enduring vehicle for a comprehensive consumer media strategy.

Posted by Vishy at 10:10 PM | Comments (0)

October 19, 2006

How open source has benefited software tools

Good software tools are an essential part of any productive software engineer's arsenal. They deliver key insights into enterprise-grade code and highlight issues that engineers may have missed upon an initial review. Software tools are highly appropriate for code instrumentation and tuning, although they perform essential functions in build and release management as well. We will compare these two kinds of tools separately on open source and proprietary platforms to infer the effects open source has had on software tools.

Code instrumentation tools

Proprietary platforms like Microsoft Windows have had excellent instrumentation features for a while now because they are the only way of introspecting into a system that does not come with source code. There is strong support at the programmatic level for building high quality software instrumentation tools. Perhaps to keep from stepping on Microsoft's toes, not many tool vendors have exploited this support to the fullest extent possible. Microsoft itself, being a software behemoth, focuses on a number of business priorities other than software tools. Barring a few excellent offerings from Compuware and SciTech, software engineers are limited to offerings from Microsoft itself. Thankfully, drawing from the strong client side tradition on Windows, these tools are heavily visual and greatly boost developer productivity. To sum up, proprietary platforms offer a few high-quality tools for code instrumentation.

Open source platforms have the opposite problem: a wide selection of tools that are sometimes lacking in quality. To their credit, most open source platforms permit highly granular examination of how a process behaves, down to the system calls it generates. The trace family of tools on Linux and the truss tools on OpenSolaris enable this kind of deep examination. Unfortunately, because of their checkered history on the client side, there aren't many visually impressive performance tuning tools. True to the Unix tradition, however, the statistics these tools produce can be integrated into larger reports relatively easily. Another factor that might impede the development of high quality code instrumentation tools on open source platforms is the pervasive availability of source code. A software engineer does not need sophisticated instrumentation tools if merely examining the underlying source is enough to diagnose a performance problem.

Build and release management tools

Open source has been highly beneficial for build and release management tools. New ideas have debuted successfully in open source settings and have later been emulated on proprietary platforms.

After decades of build tools hailing from the make tradition, the Apache Foundation's XML-based build system, Ant, took a brave new approach to the problem of generating ready-to-deploy software builds. The prolific Apache community refined this approach into the network-ready Maven. Maven can intelligently compute the right versions of a project's dependencies and download them from a central repository. The key differentiator in Maven's approach is that it confers first class status on a software project using its Project Object Model. The excellent Eclipse development platform and its flexible plugin architecture have been instrumental in integrating these tools into the software engineering experience.

For some time now, Microsoft's Visual Studio IDE has been the primary gateway to development for the Microsoft platform. The Microsoft developer population of old was fragmented: the technically savvy minority of Visual C++ developers had different needs from the Visual Basic developers that formed the majority. Visual Studio's build system was opaque and command line alternatives like nmake were available but not very well supported. The arrival of Microsoft .NET alleviated the first problem but not the second. The proliferation of XML-based build and release tools on Java led to analogous open source efforts like NAnt for Microsoft .NET. Unfortunately, Microsoft hasn't taken to the open source projects around its platform as well as Sun has around Java. It took Microsoft some time before they picked up on the importance of XML build systems; they announced their own take on an XML build system, MSBuild rather than embracing and extending NAnt, which had accumulated a conspicuous user base.

MSBuild offers tight integration with Visual Studio and as such, offers a better software development experience than NAnt. NAnt continues to thrive, but will probably be driven underground as MSBuild continues to gain traction on Microsoft's official build environment. Although Microsoft has considered open source models fundamentally antithetical to its own business model, its recent overhaul of Visual Studio's plugin architecture to make it easier to develop for might stimulate the development of tools outside Microsoft. Microsoft also supports command-line use of MSBuild, which is an unorthodox measure for a company that has heavily favored Visual Studio as a gateway for platform development in the past. Perhaps build and release management outside the Visual Studio IDE will come into its own after PowerShell, Microsoft's first fully scriptable commandline is released. However, unless Microsoft truly supports software development outside Visual Studio and encourages open source projects, the Microsoft developer community will be limited to Microsoft's offerings alone.

Conclusion

Different classes of software tools have benefited to different degrees from open source. Nonetheless, it is fair to say that open source has been and will be a boon to the software tools business.

Posted by Vishy at 08:34 PM | Comments (0)

October 17, 2006

The end of IT in small and medium businesses

Why do small and medium businesses (SMB) have internal IT departments anymore?

Several person-centuries spent in the iterative development of technology stacks have led to the emergence of well understood, reproducible best practices in enterprise IT. A number of hosting providers are now able to channel this expertise adroitly into building complete plug-n-play stacks and hosting them. In many cases, the business models of these managed IT stack providers have the wealth of mature open source software as their cornerstone. The collaboration of hundreds of open source developers on these projects has resulted in powerful network effects on quality and functionality. If you are in the SMB segment with a primary function other than IT, you probably just want to get connected and going. There is little competitive advantage to having an internal IT department put together a complete technology stack, against the proven work of hundreds, when the results of this work are ripe for the picking.

Reliable broadband connectivity is widespread in the North American enterprise context. A completely managed IT stack is not an outlandish proposition even for the most paranoid of SMB owners. The fine tuning of high-availability capabilities and the emergence of high quality administrative and reporting tools have assuaged many an SMB owner's worries that a hosting solution might lead to too little control. Moreover, a hosted solution might even prove to be better for a risk averse proprietor, given the recent emphasis on regulatory compliance in practically every major industry. The ease and uniformity with which it can be applied in a hosted situation as well as the spreading out of liability make hosting only more attractive. In cases where there is no getting around keeping records and IT operations on the premises, we are very near realizing the dream of trailer park computing. Providing a managed service atop these compute cubes would be an easy value add.

It is worth pointing out that large enterprises—especially those for whom IT lies in the critical revenue path—might still maintain enough of a competitive advantage using an internal IT department to justify the costs and upkeep of a homegrown stack. Businesses with large existing technology investments may already have critical business processes centered around them, which would make it harder for them to just switch to outsourced IT. Yet, as adoption of outsourced IT increases, the average total cost of ownership is sure to go down as economies of scale kick in. Then perhaps even non-SMB owners might reëvaluate their technology solutions.

Read the writing on the wall: if you are not in IT, then get out of IT.

Posted by Vishy at 11:16 PM | Comments (0)

October 10, 2006

Five myths of open source

The open source movement today is a lot more than a motley crew of hobbyists hacking in their free time. Based on the simple but rigorously applied principles of technical openness and freedom to tinker, it has become a force to reckon with even where serious money-making businesses are concerned. Major technology companies are committed to open source and are starting to build strategic and tactical business models around it.

Open source (OSS) projects have come to be identified with the values of individual freedom and democracy that arise out of leveling the playing field among users and contributors. This thinking in the geek community is rooted in two factors: a sense of relief that credible, technically sound alternatives to proprietary software are finally cropping up and an instinctive reaction to Microsoft's bullying and anti-competitive practices over the years. Although the birth and evolution of OSS projects is unequivocally a step forward for computing technology, these perceived values are not always borne out by actual experience. Enough negative experiences can alienate prospective user and contributors. Here's a list of myths about OSS projects, which, if dispelled, will help set more realistic expectations about them in prospective users and contributors:

Myth #1: An OSS project is a democracy.

An OSS project is not a democracy but a meritocracy. Anybody is entitled to have ideas, but is not entitled to recognition and wide adoption of those ideas. Ideas compete on their value to the influential people in a project. Successful OSS projects assess the value of an idea upon the right mix of technical and non-technical considerations, such as value to the user community. Linus Torvalds, who leads the development of the Linux kernel, has gone on record that the Linux project follows the model of a benevolent dictatorship. Although the inner circle of Linux developers (yes, there is one) makes every effort to listen to as many voices as it can, ultimately it alone makes executive decisions that influence the project.

Myth #2: Any user of an OSS product can change its direction.

Somewhat paradoxically, new users of an OSS project are best positioned to offer useful feedback on its biggest holes in functionality, because they haven't adapted to its idiosyncrasies. If a novice user suggests to a project that a feature is missing, however, a common retort they hear back is 'write it yourself'. It is not always practical for just anyone to write a feature themselves. The code base of a successful OSS project is often complex; it may take some time before a new user learns its internal conventions and is able to contribute working code. Sometimes they may be just offering a suggestion as a helpful user without having an interest in developing it themselves. If the requestor has enough money (say it was an organization), they may be able to hire enough labor to make their request come to fruition and contribute it back to the community. One benefit of giving a feature back to the community is that the community will often help support and maintain it. Even if an independent user effort does succeed, there is no guarantee that their feature will be merged into the principal distribution of that project (see #1 for why), which would only leave them with an additional support and maintenance burden on their hands.

Myth #3: Contributors work on an OSS project purely on volunteered time.

This may have been true 10 years ago, but isn't the case anymore. As commercial adoption of open source increases, key developers of several successful open source projects are on the payrolls of major corporations. These corporations have an explicit interest in making a project they sponsor succeed by entrenching it, which means that ordinary users simply cannot be ignored. The products ordinary users like best are those that get out of their way and "help them kick ass." If a user complains about some aspect of an OSS product and asks for a fix, they are frequently told that they should have no such expectation because hacker time is donated, or that nobody is forcing them to use that product. These sanctimonious and demeaning responses are incredibly off-putting and ultimately detract from the overall success of a project. Moreover, it's just not true to say that all time on an OSS project is donated; corporations that sponsor an OSS project in particular almost have an obligation to listen to what users say (also see #2).

Myth #4: Decisions regarding an OSS project's direction are made purely on technical merit.

Forks and other schisms in an OSS project can occur because of differences of opinion of a non-technical nature or even personality clashes. The XOrg fork of the X server from the main XFree86 project arose from disagreements over license terms. The IceWeasel fork of Firefox arose from a push to develop a purely free version, untainted by any non-free artwork or plugins. The Wine project maintains a number of concurrent versions targeted at different user bases. It is conventional wisdom that forking a project is inconvenient from a technical standpoint and frequently results in a loss of focus and direction. Sometimes, however, a decision with demonstrably lower technical merit may still win out because it is the realistic thing to do or because of popular support.

Myth #5: The principles of open source are important to users of an OSS project.

This may be a bitter pill to swallow for some, but it must be said out loud. Although the motivations and the initiative behind starting an OSS project may be laudable, average Joes care less about the philosophy behind a project and more about how well it works for them. OSS contributors, who frequently have an analytical bent of mind, love to argue about the correctness of their point, often to the point of acrimony. Their efforts will have been in vain, however, if their OSS product doesn't work well for their users. Functionality and correctness count for a lot more with users than infinite choice and conformance with a particular philosophy.

OSS projects may not always embody the principles of freedom and democracy in the way one would expect, but to be successful, an open source project must draw a line at the right place between the wisdom of the crowds and the tragedy of the commons.

Posted by Vishy at 10:08 PM | Comments (0)

October 04, 2006

Is Ajax appropriate for the enterprise?

Ajax, the catchy moniker coined for Asynchronous Javascript and XML content rendering technologies, is de rigeur in the Web 2.0 wave of online innovation. The more savvy users on the Web today are close to expecting Ajax as a given in the Web services they use. When they go to work and interact with Web applications on their corporate intranets, they may start expecting to see Ajax there as well. Is Ajax as appropriate for the enterprise as it is for hundreds of consumer-facing Web services? The answer: except for a few cases, it won't take the enterprise by storm.

Before we examine why, it is a good idea to take stock of the Ajax development scene today. After a heady 18-month period of growth, which has seen the release of tens of self-styled Ajax development frameworks, some winners are beginning to emerge. Enterprise adoption lags consumer-facing adoption by a little because Ajax frameworks for enterprise-grade environments are coming out of the woodwork only now. ICEFaces for Java Enterprise Edition was released in July this year, whereas Microsoft's Atlas framework is slated for an end-of-year release. Once a few Ajax bindings are available for the most common enterprise development environments, enterprise decision makers will be less guarded about Ajax adoption.

Ajax, unsurprisingly, is particularly suited for the majority of intranet applications that are delivered over the Web today. These typically include firmwide applications with a significant self-service component, such as employee performance evaluations or corporate travel booking. Portals too are suited for eventual conversion to Ajax. Not all enterprise applications, however, are suited to delivery over the Web (notably time-sensitive applications such as a real-time securities pricing and trading system, or multimedia applications, where a rich design and visualization experience is key). If these applications are wrongly delivered over the Web today, adapting them to use Ajax might prove to be a needlessly complex endeavor.

Ajax applications can lead to improperly authorized information disclosure in an enterprise environment. Many enterprises where the flow of information must be strictly controlled operate, for better or for worse, on a doctrine of security through obscurity. If an employee doesn't even know about the existence of resources they are not authorized to see, it is less likely they will see them. Ajax applications may thwart this objective because they are easy to reverse-engineer, thanks to the browser's pesky View Source function. Unless the Javascript source code in the browser is well obfuscated, sensitive details may well be downloaded and captured by insufficiently authorized roving eyes. The solution? Concentrate as much logic as possible on the server.

Wouldn't it then be a shame to move to a model dominated by thin-client Ajax applications, as desktop computers continue to gain processing power in their multi-core motherboards? There is something about concentrating logic on a server that seems to run counter to the spirit of virtualization, a direction in which a lot of enterprises are moving today.

Ajax toolkits in an enterprise setting might even prove too restrictive. Enterprise environments tend to be controlled, which means that application developers can make many more assumptions when developing applications for internal use. Security restrictions commonly associated with downloading and executing code external to a host are relatively lax inside an enterprise environment, due to a clearer demarcation between trusted and untrusted sources. These additional assumptions can be exploited judiciously to provide a computing experience that can be at once rich and effectively distributed between clients and servers. A Web browser's standard code execution safeguards are more likely to encumber a rich user experience than facilitate it.

Ajax applications may generate pages that are harder to index and discover in an enterprise search engine. As we have seen before, nowhere is the need for search bigger than within an enterprise full of information workers. Because they are more responsive to user actions, pages generated by Ajax applications are likely to fall under the purview of the Deep Web, an area in which current search engines publicly acknowledge their shortcomings.

Ajax frameworks undoubtedly have their place in an enterprise setting, especially in those applications that resemble consumer-facing Web services where Ajax is used successfully. Before latching on to Ajax to be buzzword-compliant, enterprise architects and decision makers would be well-advised to examine whether Ajax is really right for their situation.

Posted by Vishy at 12:51 AM | Comments (0)

October 03, 2006

In-game advertising: ready for prime time?

Video gaming is slowly but surely growing into an activity that cannot be ignored. Even Washington lawmakers are taking note of this phenomenon that is sweeping families throughout the United States, with assertions about how video game violence could influence violent behavior in real life. Meanwhile, having long outgrown its traditional young male demographic, video games now command the attention of nearly half the population of 35 to 54-year old women, who play video games with their children. All these are favorable indicators for a play that can yield major returns if executed well: in-game advertising.

The in-game advertising industry is projected to reach more than $400 million by 2009. There are a number of reasons advertisers may want to sit up and take notice of this advertising medium. Video games are no longer computer programs running on isolated devices. Many of the most popular games today, World of Warcraft, Final Fantasy and Second Life are played in hosted, online environments. Being linked with a gamer's experience so intimately provides plenty of opportunities to throw in advertising. Advertising in games may also stand a chance of actually being noticed, because it has a better shot at integrating with, and indeed, appearing as content. Generation Y expects high standards in marketing messages targeted at its members. These messages have to be distinctive and engaging. How much more engaging would an apparel ad be in the context of dressing up a video game character, than as a TV ad that might well be skipped over during a time-delayed viewing?

Currently, Microsoft is the only online advertising player that also has its fingers in the gaming pie. If the XBox Live and MSN teams aren't talking, they might want to look each other up soon. The highly successful online experience offered by XBox Live is something Microsoft has on its competitors in new-media advertising, and can exploit to great profit. Google (or the more media-savvy Yahoo) might still gain by linking up with video game industry players like Sony and Nintendo, whose next-generation consoles, the PS3 and Wii respectively, will debut at the end of this year.

In-game advertisers would also do well to consider the mobile angle to their advertising strategies. Multiplayer mobile gaming is on the uptick, especially with devices like the PSP/mylo and Nintendo DS Lite entering the market. If Microsoft's Zune doesn't do well as a media player, it might make sense to reposition it as a personal gaming console—the four way button can be largely grandfathered into a gaming function. In-game advertising for these consoles can be trickier because they are not connected constantly to a network. This obstacle can still be overcome by skinning a video game with a particular brand, or by downloading a cache of in-game ads on to the device when connectivity is available.

In-game advertising networks like Rivals Massive, Inc. and inGamePartners have been around for some time now, but major new-media advertising networks have yet to leverage their expertise via meaningful link-ups. In-game advertising is still far from attaining the glory lavished on online advertising, but this is exactly when media players might want to start thinking about how they will get their game on.

Strengths	Weaknesses
In-game advertising can be very engaging if done well, and stands a much better chance of integrating with true content.	The true impact of in-game ads may be hard to quantify because responding to in-game ad content can sometimes detract from actual game playing.
Opportunities	Threats
Next generation game consoles, in both handheld and stationary form, are about to hit the market. Moreover, the gaming demographic is a lot wider than it used to be.	Game publishers may view collaborating with in-game ad networks as a compromise of their game's artistic integrity.

Posted by Vishy at 10:57 PM | Comments (0)

September 20, 2006

A saner DRM scheme: marrying the media device and the cellphone

Why doesn't anybody like DRM?

Innovations in media distribution are matched at every step by "innovations" in DRM, short for Digital Rights Management or Digital Restrictions Management, depending on whom you ask. These innovations, sometimes implemented using dangerous methods, consist of ever more draconian restrictions on how a piece of media may be consumed and shared. The media industry continues to be wilfully blind towards how broadly unpopular these restrictions are.

In addition to being rooted in a mentality that treats customers as criminals, DRM schemes are unpopular because of how unnatural they feel compared to earlier forms of media. The purchase of a videotape legally confers rights on consumers to do with it as they pleased--make copies, lend it out to friends or resell it. On the other hand, with most DRM schemes today, media distributors set limits on how customers may consume digital media. These restrictions leave customers entirely beholden to media distributors well after the original purchase transaction has concluded. The de facto effect of these encumbrances is that consumers now license digital media from distributors, like commercial software, rather than purchase it outright. This purchase-but-not-quite leaves some customers feeling cheated out of their money.

Media distributors no doubt feel threatened by how the ease of reproduction of digital media directly hits at the foundation of their earlier business models. It is hard to argue from a legal standpoint that DRM should be eliminated entirely. It is not hard to see, however, that the slew of outrageous DRM restrictions from the entertainment industry stems from a sense of denial and disorientation in a world in which new media has changed the rules of the game. Many DRM schemes today are tied to an actual digital media artifact: a digital audio file, DVD or digital video download. An artifact may be played only until a certain date or only a certain number of times. Other DRM schemes tie media to the device that plays them. This kind of scheme makes it hard to share an artifact illicitly, but also gets in the way of customers who legitimately want to play an artifact on multiple devices they own.

It would be logical and consistent with earlier forms of media consumption to tie digital rights to a person rather than to an artifact or device. The rights to play an artifact are conferred on the owner of the artifact, and are fundamentally separate from the artifact itself and its medium of consumption. Any sane DRM scheme that won't end up imposing inordinate restrictions on its consumers must adhere to this principle.

One possible DRM scheme that fits the bill might be based on biometric data. Every media player might have a fingerprint reader to validate someone's identity before they play an artifact. Such a scheme would be terribly complicated because such biometric information would have to be programmed ahead of time into an artifact or a media player. Moreover, not everybody is going to be thrilled about handing fundamental, impossible-to-change biometric information like fingerprints or eyeball scans over to media distributors or media player manufacturers; the risk of a leak and subsequent abuse is simply too high. Ignoring the complications though, from the customer's point of view, this scheme would get a lot more right than Microsoft decreeing that a video may only be played thrice. For inspiration on how a similar, less complicated DRM scheme might work, we need look no further than everyone's humble companion: the cellphone.

A cellphone's SIM card effectively acts as a lightweight proxy for its owner's identity. Media players could be designed to accept a DRIM card, the digital rights equivalent of a SIM card. Much like SIM cards store their owner's contacts, DRIM cards stores rights pertaining to media artifacts. Media artifacts may reside separately on a media player's storage device but won't play unless there is a right for that artifact in the device's DRIM card. Rights on a DRIM would be truly offline. Unlike a SIM, a DRIM should not be required to contact the media distributor in any way before a media artifact is played. Rights stored on the DRIM should be mere tokens, which can be transferred as required.

It is easy to see how this system would be less infuriating than existing DRM schemes. You could play your media artifacts on any device you own by using your DRIM card in them. You could make as many copies of a media artifact as you want for backup purposes because no copying restrictions are 'baked into' the artifact. You could lend the artifact to a friend by copying it over to their device and then beaming its rights from your DRIM to thers, like some phones let you send your contacts to others via SMS. You could limit the term of the loan by specifying that such a rights transfer expire after some time. If you wish to resell a media artifact instead, you could have the new owner copy it from your device; they could then pay you to beam them a non-expiring rights transfer.

With its similarities to cellphones and SIM cards, mobile carriers surely have the know-how to implement a DRIM scheme to share media among mobile phones. With the growing buzz around mobile media, mobile payments and other mobile applications, mobile carriers might want think of how they can enter the value chain in this lucrative market. If mobile and media player manufacturers collaborate on a DRIM standard, we will be closer to a world where the power equation between a media distributor and a customer achieves some more balance. There is already a way; now all that's missing is the will.

Posted by Vishy at 10:55 PM | Comments (0)

September 14, 2006

Sun hires JRuby developers and wakes up to enterprise realities

Sun recently announced that they hired the chief developers of JRuby, a port of the Ruby programming language to the Java virtual machine. If this overture of Sun towards a non-Java language is any indication of future developments, backers of enterprise Java should be mighty pleased.

One of the things that has bothered me for a while about Java's platform philosophy is its dogma about everything being written in Java. Platforms in an enterprise setting tend to be heterogenous for a number of reasons. In a typical enterprise, where a variety of programming and scripting languages glue systems together into a cogent platform, Sun's official insistence on all-Java-or-nothing seems unrealistic at best and unnatural at worst.

Enterprise customers who wish to switch to a fully functional Java platform have two options: wait for Sun and the Java open source community to release Java libraries that do what you you need, or rewrite significant amounts of functionality in Java themselves, possibly mixing in several direct non-Java dependencies along the way to save time. The first option may be subject to Sun's exasperatingly slow Java Community Process. The second option negates the whole point of switching to a Java platform. To Java's credit, a significant amount of functionality available prior to its advent has already been ported to pure Java. But ask any software engineer and they'll tell you that Java isn't the best language for everything. A policy that officially supports only Java on the Java virtual machine ignores the reality of an enterprise—the mind-boggling heterogeneity in the kinds of technical tasks that need to be accomplished.

Microsoft .NET adopts a more pluralistic approach to languages on its platform. So long as someone makes the effort to write a platform-compliant compiler for a programming language, most of the existing libraries in that programming language can be adopted without too much effort. Writing a good compiler is harder than writing just any library, but the effort pays off immensely with regard to code reusability. More importantly, it is still less work than rewriting everything known to the industry in one language. Today, Microsoft .NET counts among its supported languages a systems programming language (C++), an object-oriented language (C#), a scripting language (IronPython), a functional language (F#) and even a dying language (COBOL). By embracing such a myriad of programming styles and languages, Microsoft .NET's approach reflects the reality of an enterprise better than a monolingual one.

If Sun has hired the JRuby developers because it has learned from Microsoft .NET's approach to programming languages, they are undoubtedly taking a step in the right direction. Before someone points out that I am ignoring Jython, a perfectly robust port of Python to the Java virtual machine, let me point out that Jython was never officially blessed by Sun. And we all know where Jim Hugunin, the creator of Jython, went. The Java platform could gain many more converts if it doesn't impose a reinvent-the-wheel-in-Java requirement. For this to happen, Sun could start out by officially bringing Ruby into the Java fold, along with other languages as time goes on. They could also (genuinely) open source Java platform specifications and let the open source community decide which languages get co-opted, as a recognition of its tireless efforts.

Here's to more announcements like this from Sun!

Posted by Vishy at 08:40 PM | Comments (0)

September 13, 2006

While you're shopping for that display, throw in some RAM too

Paul Boutin recently wrote a piece on Slate about how the best computer upgrade these days is a bigger display rather than a faster computer. While I agree with him, I think adding more memory to your computer might be an equally worthy contender for 'best computer upgrade'.

Setting aside the demanding gaming segment, personal computers, fueled by the gigahertz myth, have long been way more powerful than they need to be for basic tasks like Web browsing, checking email and light word processing. Boutin points out that switching around a set of related applications silently soaks up far more time than we realize. He claims noticeable gains in productivity after he switched to a higher display because he can now view the applications he reads with and writes with simultaneously. Boutin's arguments make a lot of sense, but are a little journalist-centric. Most other people use more than two applications at a time: one or more browser windows, an instant messenger, a media player and other applications as needed, such as a video editor or a spreadsheet. For example, a financial services professional may cycle through Microsoft Excel, Microsoft Outlook, a trading application and a Bloomberg news terminal. A software developer may cycle through a browser, Microsoft Outlook, an IDE and the application being developed. These are just two examples, but typical office or home tasks involve more than just two applications at a time.

With limited screen real estate and multiple open applications, a computer with say, Windows XP's supported minimum memory—64MB of RAM, slows to a drag because applications not being used actively get paged out to disk. Switching among a set of applications gets painful as portions of windows take forever to be read back from disk, be redrawn and be usable. Adding more memory to your system can bring about amazing increases in performance. To be sure, there is a limit beyond which adding more memory yields diminishing returns. Continuing to add more RAM is hardly a bad idea because with enough RAM, there may be no need to swap applications out to disk at all. Rather than agonizing over what exactly the sweet spot, if you're in the market for an large display anyway, just drop another $150 on a gigabyte of RAM. It is at least four times as cheap as a 24+" LCD display and offers quite a performance boost, especially if you use more than two applications at a time.

The problem with this advice is that adding RAM to a computer is not always as straightforward as hooking up a computer to a larger display. In the case of many PCs, it involves peeking under the hood and figuring out where to install RAM modules, but only after reading through several ominous warnings to ground yourself before handling them. My most painless memory upgrade experience (see page 79) has been with my MacBook: remove battery, pop out old modules, insert new ones, replace battery. The slightly more involved process of adding RAM to most computers means less people will be willing to try it, which is really a shame. I am sure most people will wise up to the benefits of added RAM once it becomes easier to augment a computer's memory. Windows Vista has a distinctive new feature called ReadyBoost™, which enables users to expand a system's available memory simply by sticking a memory key into its USB port. Once features like these become more commonplace, memory issues are bound to (*cough*) become a distant memory. Anyone who is particular about how much memory they will need could simply carry their own around. Internet cafés may just start putting up 'BYOB, BYOM' signs.

Oh, and don't forget to enjoy that new display also!

Posted by Vishy at 11:38 PM | Comments (0)

September 11, 2006

Video on demand: perspectives and opportunities

Video on demand has seen a bit of action lately as broadband penetration continues to increase and old media roll out business models that take new realities into account. There is no dearth of players in the field, be they large companies or small, young companies or old. Sometimes, the players are not even those that one would expect. It is clear that this sector is quite immature and that early movers can do a lot to shape it in their image and grab a big slice of a multi-billion dollar market. This post will provide an overview of recent developments in video on demand (particularly in TV over IP), identify opportunities and speculate on future directions.

Video on demand players such as YouTube and Google Video are reaching stratospheric levels of user activity. When a startup that is little more than Flickr for video is valued at 600 million, it is safe to say that video on demand is hot. If the last presidential elections in the United States were powered by blogs, this one might well be powered by YouTube. At the moment, both these services are clearly in high demand, but they are both driven by a new media approach. Their continued success may well depend on how they integrate with other shades in the consumer media spectrum.

Integration with other types of media is what pre-Web 2.0 technology players in concert with old media can offer. Last week Amazon introduced Unbox, a movie and TV episode download service. The $9.99 movies and $1.99 TV episodes are owned by several traditional media houses. All videos come encoded with a Windows-only digital rights management system. Amazon is the least experienced player in this field and it almost seems they released a movie download service right now just to throw their hat into the ring and to steal some of Apple's thunder in the process. Apple, however, has always pulled off its media plays with remarkable finesse and timing. Their announcement of a movie download service and video-related enhancements to the iPod product line comes at a time when demand for consumer video is at a point of inflexion. The video iPod has enough of an installed base now that having the option of downloading movies from iTunes to them seems like a natural progression from downloading music and TV shows. Another key announcement among Apple's latest product releases is the one about a forthcoming set top box that will act as a media hub. This long awaited wireless device might well mark the convergence of PC and TV, a goal other players have tried to attain and failed.

Now for the more surprising aspects of this space. Netflix pioneered online video rentals and it was widely expected that it would tie up with somebody like TiVo to offer on-demand movie downloads to DVRs. For lack of will or execution, this partnership never materialized, and two companies that might have been uniquely positioned to move ahead of big players like Apple, Google and Microsoft just never capitalized on their advantages. As DVRs and movies on demand continue to be increasingly commoditized by basic cable providers, TiVo has probably lost the optimal window in which it could best reap the benefits of a collaboration with an online movie rental provider. Another surprise entrant in the video-on-demand arena is AT&T, with its Broadband TV service. Subscribers to this service can watch approximately 20 channels within the comfort of their browser. This service may not take off very well though because people want to watch TV content on a television rather than on a PC. Unless there is some way for downloaded content to be streamed to other media devices, AT&T's TV may well be just turn out to be the stunted arm in a triple-play offering. Another product line to watch is Microsoft's beachhead in the personal media space via its Zune media player. Although Microsoft is by no means early to the video on demand game, its existing relationships with content providers, relatively established content verticals and virtually limitless budget might portend a video on demand strategy that cannot be instantly dismissed.

In the context of cross-device integration, video on demand players might do well to evaluate their offerings with respect to the mobile and automotive segments as well. Media devices such as the PSP, mylo and iPods will continue to make steady inroads into the personal electronics segment. More and more cars come equipped with in-cabin video these days; having a plan to capture this portion of the market might yield large payoffs. Having coherent and interoperable Internet, mobile and automotive strategies might well prove to be the triple play of online video. The appointment of Google's Eric Schmidt to Apple's board might open up interesting ways in which the two companies can collaborate over video on demand. Even if Google's nationwide WiFi doesn't take off as expected, Google and Apple might be able to work out what to do with all the dark fiber the former has bought up.

Over all this potential for innovation hangs the specter of old media companies imposing excessively rigid digital rights restrictions on their content. Even after the realignment and introspection that old media is going through in response to challenges posed by new media, it is easy for them to revert to their content-is-king credo once video on demand has taken root sufficiently. One can only hope that their experiences collaborating with new media and technology players over video on demand will teach them that content alone is worthless without a good business model to accompany it.

Strengths	Weaknesses
The increasing penetration of broadband and the coming of age of delivery technologies means video on demand is poised to take off.	Video on demand won't supplant existing delivery methods and become indispensable to consumers until they have the flexibility to stream content from and play content on any device they own.
Opportunities	Threats
The precise shape of a video on demand offering is still open to definition, and moving fast can still help shape the sector significantly.	Old media can stunt innovation in business models and delivery methods by drawing the old content-is-king canard. They can get away with it too because intellectual property laws currently work in their favor. If they don't adapt to the new ways of consuming content, they risk alienating users and failing to attain critical mass on their new media initiatives.

Posted by Vishy at 12:24 AM | Comments (0)

September 01, 2006

Is platform heterogeneity in the enterprise inevitable?

Many small to midsize enterprises tend to be pragmatic about their choice of technology platform. Such enterprises would see no harm in operating on a single platform, such as Linux or Windows, so long as all its components work. Such an enterprise is willing to tolerate less than "best of breed" components so long as they all blend into a cohesive technology solution that satisfies its needs.

Then there are other enterprises who are determined to use only those components, which, by some measure of goodness, have surpassed all others in their class. These enterprises are willing to stitch a platform together from these components themselves, even if it means that it will be heterogeneous and more complex to manage. Platform heterogeneity is unavoidable given the present enterprise software environment, so long as such enterprises exist. We will identify a few factors, which, when resolved, might enable one platform to claim the entire enterprise market for itself. We will also see why vibrant open-source communities are key to reducing the complexities associated with platform heterogeneity.

The second kind of enterprises identified above (let's call them "best-of-breed-only"--BOBO, for short) seem to accept platform heterogeneity as a fact of life. Unsurprisingly, these enterprises have significant technology investments and large internal development teams to support their internal IT operations. Year after year, major technology stack providers vie for their hearts and minds by commissioning industry analyst firms to conduct ROI/TCO studies that case their own offering in a better light.

Typically, one finds that BOBO enterprises choose Java on the server side and Windows/Microsoft .NET on the client side. Java, having been around twice as long as Microsoft .NET, offers a compelling server platform with a flourishing open source community around it. In many cases, the open source community brings to the table significant design experience and maturity arising from language traditions that predate Java. This development heritage has led to a superior set of administration and management tools that can be combined in new and interesting ways to provide the data administrators need. Moreover, a significant number of infrastructure and middleware players have thrown in their lot with Java, resulting in, for instance, multiple viable implementations of the Java virtual machine standard.

Microsoft .NET may not have as vibrant an open source community around it as Java, but it has a stranglehold on the client that is only cemented by vendors like Infragistics Inc., who produce professional-looking user interface controls around the basic windowing toolkit. The narrowly defined user interface standards of the Windows platform ultimately lead to a more predictable look-and-feel for end-users, thus making Microsoft .NET the best-of-breed client platform. The Windows/Microsoft .NET platform is also preferred by end-users because it can best emulate the look and feel of familiar Microsoft Office applications. Java rich client applications, in contrast, continue to look out of place amid other applications that share the desktop with them. However, because user expectations for Web interfaces are more flexible, Java continues to show considerable promise as a Web-based client platform. A definitive victory for Java Web client frameworks like Seam is not in sight though, because Microsoft .NET's ASP.NET Web development platform also offers a compelling platform on which software developers can be highly productive.

Java has arguably less of an uphill task than Microsoft .NET in taking over the entire enterprise segment. Its major areas of improvement are clearly seen to lie on the client side. Because of early positioning and quality control mistakes from Sun, Java has acquired a reputation of awkwardness on the client that has become hard to shake off. However, open source initiatives like the Apache Foundation have captured the talent, the momentum and the mindshare to produce a compelling, pure-Java client side solution if they only look beyond producing the next application server framework that conforms to some more JSRs. In other words, just because new projects keep satisfying some Java community-defined notion of cool doesn't mean they are always appropriate in an enterprise setting. This sort of "server-is-king" attitude is a holdover from an ancient mentality that treats the client and presentation layer as an afterthought once the core engine has been built. This attitude is outdated today, in an environment where it is necessary to access server-side business logic from disparate kinds of clients, including rich client and mobile profiles. Java's insistence on write-once-run-everywhere to the fullest extent possible might well prove to be its undoing in this regard, because writing versatile client frameworks requires a flexibility server-only engineers can afford not to have.

If Microsoft .NET's aim is to have the enterprise segment in its back pocket, there needs to be a much better sense of community and ownership around the platform. Microsoft is doing nothing to prevent the death of major open-source .NET projects like NDoc. If anything, it releases products that supplant these projects, such as Sandcastle in the case of NDoc. However, the blame for this state of affairs does not lie solely with Microsoft. Once Microsoft releases a competing implementation, community members consider their efforts necessarily nullified, a practice that does not bode well for the health of the platform. Open source projects on the Microsoft .NET platform must have a life of their own, independent of Microsoft's efforts, rather than being stop-gap efforts until an officially sanctioned product that supplants them comes along. Understandably, developers are wary of the Microsoft juggernaut's track record of entering a domain and instantly quenching all its competition. However, if the records of major .NET open source projects like Mono are to be believed, Microsoft's Developer Division, the division behind Microsoft .NET, has been remarkably open about and supportive of Microsoft .NET development originating outside Microsoft. It is, at least to some extent, the responsibility of the .NET open source community to consider accepting this hand of friendship, even if cautiously at first.

Then again, there may be those in the open-source community that won't have anything to do with Microsoft because "strong-arming is in Microsoft's DNA". For those incorrigible developers, there is the flourishing Mono project, which brings Microsoft .NET bindings to Linux. Currently, its focus is to provide a broad set of development stacks centered around the Common Language Runtime standard. As such, it is concentrating more on breadth of functionality rather than aggressive performance optimizations. If an aggressive corporate backer throws in its lot with Mono, as IBM once did with Java, it can work wonders for the entire Microsoft .NET development community. Microsoft would be forced to fine tune its own runtime, which is not aggressively optimized in all cases. The momentum gained from such aggressive optimization could infect the rest of the open-source community, thus spurring it on to developing a compelling administrative and management toolset itself, rather than relying on Microsoft to do so.

Platform heterogeneity may have its benefits, chiefly of not having to put all of one's eggs in one basket. However, it racks up significant technology and human costs and is necessarily more complex to manage. The custom platforms of many BOBO enterprises, for most of whom IT is not a core line of business, prevent them from simply outsourcing their IT function to completely managed hosting providers. The key to reducing platform heterogeneity lies in creating vibrant open-source communities around existing enterprise platforms, each with an objective to fix that platform's shortcomings. These open-source communities would no doubt compete for mindshare, but would also learn from each other and ultimately render platform heterogeneity less inevitable and technology costs more manageable.

[Update: For the sake of completeness, I should mention another category of firms, who, being afflicted with NIH syndrome, conclude that nothing out there is good enough and simply develop their entire technology stack from scratch. This harks back to the days when firms would write their own database systems and have programmers code up queries for each kind of report they wanted. Thankfully, due to the maturity of both commercial and open source software, not too many companies think this way anymore. Today even business analysts with a decent knowledge of SQL can define queries for reports. The firms in this category are mostly ISVs and other technology-heavy outfits that already have a significant investment in technology. It is difficult to call their platforms homogenuous or heterogenous, but it does add complexity simply because very few people know their proprietary systems.]

Posted by Vishy at 07:48 PM | Comments (0)

August 26, 2006

Five things that bug me about a Mac

PCs are not pretty. As if their clunky exteriors are not enough, they run Windows. Yet, I used them exclusively for my home computing needs for over 12 years.

This changed recently when I finally sipped the Kool-Aid and got a Mac. Let me be clear that I have been really happy with my purchase. That still does not put the Mac above criticism for some things it could do better. Here is a list of things a former Windows user (who is something of a control freak when it comes to computers) coming to a Mac wishes it would do better:

1. Quit applications. In Windows, if I close all windows of an application, I quit it, generally speaking. On a Mac, however, even after I close all Firefox windows (say), it sits around consuming system resources. If I switch to another application after closing all my Firefox windows, there is no way for me to tell that Firefox is still running, save for the little triangle under the Firefox dock icon. Can't I just close all the windows of an application and forget about it?
2. Navigate among windows. Don't take my Alt-Tab away! Apple-Tab on the Mac only switches between applications, not individual windows of the same application. For the latter, I have to use the decidedly clunky Apple-`. Can't I just have one way to flip through all my open windows?
3. Show meaningful context menus. I know this is a common peeve, so I won't dwell on it. It's not that I mind that there's only one mouse button. It's just that one of the handiest features of Windows is right-clicking on an object and finding out what you can do with it. On a Mac, no matter what icon I click on, I get the same context menu and I am none the wiser about what I can do with it. How about something handy, like Send To on Windows context menus?
4. Kill a misbehaving application. The only way seems to be by choosing Force Quit from the context menu of a window's icon in the dock. But wait -- what happens if an application has no windows (see #1)? I have to open up a Terminal window and grep for the process ID from a ps listing because there is no graphical task manager like in Windows. It's not a big problem for me, but what about grandma who bought a Mac because you told her it just works?
5. Be more keyboard friendly. I am a keyboard czar -- I pick up the keyboard shortcuts of a Windows application quickly and try to use them as much as possible. When I use the same applications on a Mac though, they simply don't have as rich a keyboard-centric user interface. For example, before I send off a Word document to the printer, I hit Ctrl+F2 to see a print preview. What's the keyboard shortcut for Print Preview on Word 2004 for Mac? Zilch. Zip. Nada. To be fair, this is more an application writer's problem than the Mac's, but it still bugs me. Oh, and while we are on the subject of keyboard friendliness, how about replacing a rarely used key (anyone know the Mac equivalent of SysReq?) with a Delete key? No I don't mean the existing delete key, which does the same thing as the windows Backspace key. I want something that behaves like the Windows Delete key.

In due course I'll get over these annoyances and actually begin to grok the Mac Kool-Aid. Then I can go around looking like the unshaven, hoodie-wearing, hands-in-pockets hipster from the Mac ads. But while I am not over these yet, why not write about them using a Mac?

Posted by Vishy at 03:11 PM | Comments (28)

August 09, 2006

The success of Sony's mylo: A Gordian knot

Sony recently announced the imminent release of mylo, a $350 personal communicator that supports WiFi, Skype, instant messaging and MP3s. This device is unique in many respects, but it may doom itself by being a little ahead of its time.

mylo is the killer app of VoIP and WiFi -- the first device that truly unifies voice and data communication. Earlier devices have taken a shot at this goal, but have been stunted by the antics of slow-moving telecom carriers, who want to preserve the status quo. The mylo is different from a PDA because of its much improved communications capabilities. mylo has a full QWERTY keyboard, which differentiates it from Sony's PSP game console. Together with Google's much-touted WiFi service and VoIP-enabled applications such as Google Talk, it may well be the Mobile Skype Cable that challenges the stranglehold of traditional wireless telecom providers on the airwaves.

Although the prospect of a WiFi-enabled personal mobile communicator sounds great, Sony has bet the success of mylo upon WiFi, whose own success has yet to be proven. Freely available WiFi is far from being universal. The lack of ubiquitous free WiFi outside public access areas like college campuses would stunt mylo's voice and IM communication features. Think of all the stories about abysmal cellular phone network coverage you've heard and magnify them a few times to get an idea of the situation. Until ubiquitous WiFi becomes a reality, mylo would be reduced to an expensive MP3 player.

This device is different from Sony's previous forays into personal electronics. It's almost as if the Vaio and PSP teams produced this device after a night of passion. The closest device that comes to mind is Sharp's Sidekick, marketed heavily by wireless carrier T-Mobile. The Sidekick too has a QWERTY keyboard and IM integration.It has a 2.6-inch screen form factor as against mylo's 2.4 inches. It may not support Skype but it has a camera instead. Nonetheless, mylo would beat the Sidekick for the average consumer because it is not tied to a carrier.

Sony's announcement raises a number of questions. Will mylo influence the PSP product line at all? Why is Sony entering the market of general-purpose personal electronics? It may be trying to differentiate itself from Nintendo, its closest competitor in personal electronic devices. Or perhaps it may be positioning itself to capture the detritus left behind by the peaking of the iPod--folks for whom a glorified hard drive with a video screen just doesn't cut it anymore.

If Sony's track record alone were any indication, it can sabotage the success of mylo in many ways. It can exercise proprietary control over technical aspects of the device and stunt its blossoming as a platform. It can try and pull off something as dastardly as the rootkit fiasco from last year. Or it can overprice a device that without WiFi would be a fancy MP3 player. Unless price cuts come soon for this device, lukewarm sales early on can cast a shadow over its future success, just as in the case of N-Gage. The big Internet companies, Google, Yahoo and Ebay, stand to profit considerably from usage of their mobile services on this device. Perhaps they can come to Sony's aid somehow and cut the Gordian knot that will make a seriously cool device like the mylo a success.

Posted by Vishy at 07:33 PM | Comments (1)

July 13, 2006

Dynamic languages in the finance industry

Lisp, the ancient patriarch of computer languages, is derided as much in industrial circles as it is revered in academia. "What good is Lisp when you can't design real production systems with it?", its detractors have asked for a number of years. For all these years—multiple generations in the accelerated timescales of technology—such derision was hard to counter. Lisp and related languages were respected for the way in which they molded the thought processes of young computer scientists, but not for much else. And yet, now, if you look in the right places, Lisp and other languages that owe an intellectual debt to the Lisp programming environment are making a comeback. Recognized names in industry now employ functional and dynamically typed languages, including Python, Ruby, OCaml and Lisp itself as cornerstones of their operations. The financial services industry, one of the powerhouses of the modern economy, stands to profit immensely from this trend in particular. Let us see why functional and dynamically typed languages and the financial services industry can be such a great match for each other.

Before we proceed, there is a technical distinction to be made between functional languages and dynamically typed languages. Functional languages are exponents of a style of programming that considers any computer program as a function mapping input values to output values. On the other hand, dynamically typed languages, also known as dynamic languages, treat program objects differently in different execution contexts rather than requiring them to identify their type at compile time. Typically, dynamic languages erase the distinction between compile time and runtime because their compiler and runtime can be invoked in one go. The dynamic languages referred to above as being in vogue also support functional programming. Keeping this in mind, in the context of this essay only, we will use these terms interchangeably.

Computer applications in the financial services industry can be classified broadly into modeling, reporting and transaction-executing applications. Consider applications that model financial instruments and price them for the open market. Aside from sundry trading of stocks, bonds and other financial instruments, financial services firms profit by inventing new kinds of securities in-house and building armies of financial services experts around them. Once a new security starts trading in the open market, the technical infrastructure to model and price it also follows close behind. Currently, a financial services professional (FSP) must explain the financial concepts underpinning a new type of security to an applications programmer. The application programmer then produces an application that is aware of this new type of security by modeling it in object-oriented languages like C++, Java or C#. Building a compile-time type scaffolding around a new conceptual domain is not a trivial task. Most of the time, the sole purpose of such work is to pacify the compiler and has nothing to do with the conceptual domain of financial instruments.

Dynamic languages forgo strict compile time checking and view an object's type more as the set of operations it can perform rather than an inherent property of the object. Ruby, for instance, calls its dynamic typing 'duck typing', which comes from the adage 'If it quacks like a duck, it is a duck.' In other words, an object need only perform the operations characteristic of a type to qualify as an instance of that type. This feature turns out to be unusually handy in the financial services world. When modeling and pricing a complex security like an option on an option on a swap or a collateralized debt obligation (CDO) backed by another (also known as a "CDO-squared"), it's more important to know what behavior that security is capable of rather than ascribing it a meaningful type ahead of time. For example, the same set of operations apply to disparate kinds of derivative securities, regardless of whether they are based on an actual asset or another derivative. Correspondingly, in an object model representing these derivatives, it is more important to be able to put them in a collection and apply the same operation to them rather than care about the specific security from which they derive their value. A statically typed environment with compile time checks turns out to be more an encumbrance than a boon in such a situation. This is exactly where dynamic languages step in. They get out of the way of application developers by freeing them of strictly technical type-related concerns. Thus, they reduce the amount of work that goes into modeling a new financial product and save valuable application development time.

In fact, with the right choice of dynamic language, the line between FSPs and application programmers can be blurred, which brings us to the second benefit of dynamic programming languages to the financial industry. Time is easily the most valuable commodity in the financial services industry. For a company in this industry, a faster development cycle for financial computer applications could mean stealing that edge over its competitors in a hotly contested marketplace, where secrets don't stay secret for very long. The effort involved in developing financial software is split in a dichotomy between FSPs and technical-minded application programmers. Inevitable misunderstandings of requirements, along with other communication-related overhead, can quickly eat up valuable time in development cycles. Most dynamic languages today provide highly integrated development and execution environments, where it is possible to freeze the execution of a production application, introspect objects and their behaviors, and make changes to the system even as it is running. Back-and-forth communication between business units and application programmers can thus be greatly reduced. Additionally, today's dynamic languages are far easier to embed in larger applications than their statically typed counterparts. By using them, the application programmer can furnish the FSP with building blocks that let them program in a mini-language of their own. The FSPs do what they know best: modeling financial instruments; application programmers also sidestep any possibility of miscommunication by also doing what they know best: building good layers of abstraction.

Let us build upon the idea of abstraction a bit further to arrive at the third advantage of dynamic languages. An abstraction layer, it bears repeating, is a very powerful concept. For evidence of this claim, look no further than the copy of Microsoft Excel running on an FSP's desktop. Sometimes, the entire business of a financial services firm hinges upon Microsoft Excel. FSPs use Excel in a big way to get things done because they tell it what quantities they want it to calculate rather than how it should calculate them. This style of programming, called "declarative" programming, as opposed to the "procedural" variety application programmers are intimate with, is powerful because it operates at a higher level. It allows its users to ignore the details that don't ultimately matter. Functional languages are excellent at modeling an operation as a sequence of logical steps, where each step can be seen as transforming its input in some way before passing it along to its successor. Programs written in functional languages can inspect their own structure, modify it and generate other programs with astonishing ease. Imagine a "program of programs", a toolbox that contains other little programs to perform different kinds of operations. An FSP can model a workflow in this program of programs by dragging and dropping boxes, connecting them with arrows and defining how information flows across the boxes. Application programmers using visual development environments such as Visual Studio do this already when developing graphical user interfaces. The possibilities are essentially infinite if a high-level modeling approach such as this was adopted by FSPs by harnessing the power of functional languages.

Functional and dynamic languages are not appropriate for every kind of financial application. In cases such as trade engines, where raw performance is paramount, there is no substitute for the nose-to-the-ground approach of highly optimized programs written in traditional programming languages. Some firms may also be squeamish on multiple counts about using dynamic languages because most of them are open-source and don't have large corporations backing them. Yet, it is no coincidence that hedge funds and other financial services firms of the nimbler variety have built entire businesses around dynamic languages. It is ultimately poetic that the programming languages so well suited to an industry as dynamic as the financial services industry are also called dynamic languages.

Posted by Vishy at 10:49 PM | Comments (5)

July 08, 2006

Google's future lies in its enterprise play

It has almost become too predictable. Every few months the traditional media and the blogosphere fall all over themselves trying to analyze the impact of the newest Web service to stream out of the Googleplex. Arguments, conversations and rebuttals rage for weeks around what else might be under drapes inside the sleek technology execution machine that is Google. Google's present success and future potential are widely acclaimed because of its impressive track record, the caliber of its employees and the quality and speed of its execution. It is clear that Google has changed the rules of the game in Internet business. Right from the inception of a new toy by Google—sometimes when it has just barely been released into the Google Labs playpen—speculation abounds on how it will revolutionize the market. However, the facts don't always bear out all the speculation. Apart from its core business, search, Google has made little headway in getting users to adopt its other services. With the induction of each new service, typically free of charge, into Google's portfolio, investors worry about how it will actually make money for Google. There is plenty of money to be made for sure if Google would only reexamine its strategies and wipe the dust off one: its enterprise play.

Corporate intranets and internal messaging systems for easier collaboration are no longer a rarity among medium-sized to large enterprises. Having said that, not all business processes within a company are automated yet. Consider the finance industry. There are a number of process inefficiencies such as manual verification and other labor-intensive checks surrounding the sale of practically any financial instrument. Due to the diligent access hygiene imposed by a number of regulatory requirements, several of these labor intensive steps are now done using electronic means. What used to generate a paper trail now generates an electronic trail of documents. Whether or not industry regulations require them, complex workflows and document management systems are gaining traction in other industries as well, notably healthcare. As the number of electronic documents floating around increases, so does the need to find them and organize them. Google has had a search appliance that solves exactly this problem. As physical processes increasingly get converted to their electronic equivalents, the information associated with them, which has remained 'dark' and unsearchable before, becomes searchable—analogous to the 'light matter' that makes up the visible Universe. The so-called dark Webs inside corporate firewalls continue to lighten. Google can focus a bit more on its enterprise offerings to capitalize upon this wave of change. To their credit, they have shown signs of noticing this swath of uncharted intranet territory by partnering with a number of organizations over a set of tools dubbed Google OneBox for Enterprise.

Google must also seek to expand another crucial arm of its enterprise play: mobile. Information workers are increasingly expected to have access to information while on the go. Currently, Google's mobile offerings outside the consumer space have been minimal. Consumers may not always choose the most Internet-enabled mobile devices for their personal use, but would use a highly Internet-enabled mobile device, such as a BlackBerry, if required to do so by their employer. This captive audience of users with powerful mobile devices is more than just a fertile testing ground for new Google mobile products that will later be targeted at consumers. Providing a viable mobile search solution that integrates well with that organization's search appliance would make Google's value proposition a lot more compelling.

We have so far considered only Google's existing offerings in the enterprise space; consider what new plays Google could make in this area. Google has its fingers on the collective pulse of the search habits of millions of users around the world. Let us suppose that one day it decided to mine all submitted queries that return less than a hundred results. During the course of this analysis, it is discovered that the phrases [walmart] and [health insurance] keep occurring together, although WalMart is not in the insurance business. How much would Walmart be willing to pay Google for the knowledge that its customers expect it to enter the healthcare industry? The real-life story of how HP got into the projector business bears more than a passing resemblance to the above scenario. A market research survey found that even before HP got into the projector business, it was being named as one of the top 3 brands of projectors in America. How about using the mounds of data Google collects to provide market intelligence to enterprises in the form of 'Zeitgeist consultancy services'? Think of an enterprise-grade, competitive intelligence version of Google Trends. Another piece of speculation recently seen in the press has been around Google's formidable infrastructure, which has proven equal to the challenge of Google's increasing user base. Why not provide comprehensive application hosting solutions to enterprises that wish to outsource their infrastructure entirely to a Google server farm?

Google has always allied itself strongly with the end user, the average Joe who typically connects to its services from his personal computer or mobile device. Anecdotal accounts—too many to be just isolated complaints—seem to indicate that Google isn't making enterprise search as much of a priority as its consumer search. Perhaps Google wishes to make the point that putting out high-quality services for end users alone is sufficient to keep them coming back and sustain its core business. Or perhaps Google views a major enterprise push as described above as repudiating its essence—going from innovating constantly in a creative work environment (but being somewhat of a lone wolf about partnering with others) to entering high-maintenance relationships with other large enterprises. Most importantly though, it comes down to a question of control. An expanded suite of enterprise services means Google would have to run its software in environments that are not fully under its control. Google's software would have to play reasonably well with other enterprise systems and there will inevitably be problems, including security holes, even with Google software. Google would have to be very careful getting into the business of off-site software installation and support—perhaps the hundreds of former Microsoft employees now in its ranks can be some help there.

If Google continues to dominate search in the consumer segment and provides a compelling and integrated value proposition to enterprises at the same time, large enterprises would be amenable to signing support and consulting contracts around enterprise search that could prove quite lucrative for Google. Fixing the flaws in its current enterprise search offering and recognizing the money other strategies can bring in would enable Google to diversify its revenue streams and continue rolling out free services, whose only ostensible purpose seems to be to increase the 'stickiness' of Google's Web properties. While some might argue that this would go against Google's essential line of business so far, enterprise search may well prove to be its only reliable revenue stream except online advertising, because the need for organizing the world's enterprise information is immune to the vagaries of business cycles, click fraud allegations and increased competition that plague Google's core business.

Posted by Vishy at 11:42 AM | Comments (0)

February 13, 2006

HOWTO: Make acts_as_taggable work with PostgreSQL

I recently had a chance to play with acts_as_taggable, a sweet little mixin written in Ruby for ActiveRecord/Ruby on Rails model classes that gets them associated with keywords (aka tags or metadata) with relatively little effort. This post won't be particularly useful unless you first read a basic tutorial about using acts_as_taggable.

My setup is centered around the PostgreSQL database product, even though MySQL is far more popular. When I followed the instructions in the tutorial above, I found that I was getting some fairly basic PostgreSQL errors which went something like:

column " posts.title" must appear in the GROUP BY clause or be used in an aggregate function: SELECT posts.* FROM tags_posts, posts, tags WHERE tags_posts.tag_id = tags.id AND ( tags.name = 'voip') AND posts.id = tags_posts.post_id GROUP BY posts.id HAVING COUNT( posts.id) = 1

Apparently this is a known issue in the acts_as_taggable project's bug tracker but when I searched for a patch, one was not available. With lots of help from Goynang on #rubyonrails, I've put together a patch that would make acts_as_taggable work properly with PostgreSQL. Hopefully this will be integrated into the main project soon, but until then, use it for your needs. I impose no restrictions on the use of this patch especially considering very little of it is my code.

Posted by Vishy at 09:33 PM | Comments (0)

December 28, 2005

A travel search engine that truly rocks

Web applications never fail to surprise me. Even in well-trodden domains like Web search or online travel booking, something comes along every once in a while that makes me sit up and take notice.

Most of my travel is international, for which roundtrip tickets cost over $1000. Right now, there are three ways in which I can book tickets in principle:

a) buy online directly from the website of an airline (let's call these direct tickets)
b) buy from an online travel agent like Orbitz or Expedia (let's call these indirect tickets) or
c) buy from a human airfare consolidator or travel agent.

I almost never use the first option because it shows flights only from one airline and some part of me thinks I might not be getting the best price out there. I have used the second option with for most of my travel bookings. I have been able use to the third option with limited success, pricewise, when online travel agents like Expedia and Orbitz don't give me good enough fares.

With existing online travel agents, I find the user interface to be a bit restrictive for planning my flights. For instance, in my search parameters, I am unable to specify a time window within which I would like to arrive and depart. I have to look at each result and filter out the ones that work with my schedule myself. Moreover, these sites offer fares only for indirect tickets that have been made available to them and do not let the user comparison shop for direct tickets that may be an equivalent or better deal. Because they sell fares, I can only look at the fares they have on sale; this restriction also applies when I buy my tickets from a human travel agent.

I recently heard of kayak.com, an online travel search engine that has blown me over with its feature set and user interface. It acts as a meta-search across websites that offer direct tickets as well as indirect tickets. It does not sell any airfares itself but directs users to fares that it finds from airline websites as well as travel agents. In so doing, it has shown me direct ticket deals that I wouldn't ever have encountered on Travelocity, Orbitz or Expedia. Kayak's user interface employs Ajax skilfully to give a smooth user experience. Kayak also provides Ajaxified sliders, an apposite user interface metaphor for selecting continuums, such as price ranges or time windows for arrival and departure.

The only gripe I have about Kayak is that the outbound links to the actual websites from which you can buy the tickets occasional result in session or cookie-related errors. This doesn't prevent me from purchasing the fare -- it's just an extra step that makes the process a bit less seamless than I would like it. Nonetheless, Kayak has quickly become my favorite travel search engine because of its smart user interface and its coverage of websites from which I would never have thought of buying tickets. I highly recommend checking it out the next time you need to buy flight tickets.

Posted by Vishy at 08:31 PM | Comments (0)

December 18, 2005

HOWTO: Get Ruby on Rails to map nonstandard column types correctly into your model

I have been trying my hand at Ruby on Rails, the latest and greatest Web development platform to hit the block. Ruby on Rails is often breathlessly praised by the bloggerati for its 'configuration by convention' philosophy and has quickly become a Web 2.0 buzzword. Heck, some even tout it as the beginnings of a hitherto unspoken conversation about how to keep programmers happy. The Net is fulminating with impressive videos that show you how to build a blog application with Rails in 15 minutes. I find that a lot of this material is directed at absolute beginners; beyond that, anyone who wants to meddle in Rails is presented mostly with a steaming pile of API documentation (much to the credit of the Rails developers, I have rarely seen a better-documented framework). Although I find API documentation useful, I, along with many others, learn a lot better from HOWTOs and tutorials. This HOWTO is written in that spirit.

Rails' wow! moment comes when it successfully generates basic CRUD pages for your data model using a single line of code, via a process known as scaffolding. Based on the data types of the columns in your database table, it successfully figures out how to display and accept input for the values of that column. For example, a varchar column in your database table would result in the automatic generation of a HTML text field, whereas a text/ntext/clob field would result in an HTML textarea. This works more than 80% of the time, because there aren't that many standard column types. However, powerful databases like PostgreSQL allow the database user to define custom types and reference them in table definitions. As you would expect, Rails doesn't map these custom types correctly to user interface elements in the scaffold.

Let's say you're building a Web application that is concerned with some domain object, say a widget. Let's suppose that every widget instance has an attribute, :widget_blick, of a nonstandard data type, blick. The actual representation of blick values in the database is irrelevant. Let's assume though that the database provides TextToBlick and BlickToText functions that convert back and forth between a blick and some textual representation of a blick. After you create the widgets table and use Rails' script/generate model Widget, you'll get something like so in widget.rb

class Widget < ActiveRecord::Base
end

Now, you want each widget instance's :widget_blick attribute to return the textual representation of a blick and also to accept values in this representation and perform translations between this representation and blicks using the functions provided by the database. First you need to override the default accessor and mutator functions for the :widget_blick attribute

class Widget < ActiveRecord::Base
        def widget_blick=(textrepresentation)
           # call TextToBlick
        end

        def widget_blick
          # call BlickToText
        end
end

To call the database-provided functions, each model class derived from ActiveRecord::Base provides a find_by_sql method which nominally returns an object of the same type as the model class. In other words, Widget#find_by_sql would return the result of the database function call wrapped in a Widget instance. All you need to do is to reach inside this Widget instance and grab the underlying value to make your accessor and mutator methods functional. Your code would look something like so

class Widget < ActiveRecord::Base
        def widget_blick=(textrepresentation)
           write_attribute(:widget_blick, 
           Widget.find_by_sql(["SELECT TextToBlick(?) 
as value", textrepresentation]).first.value)
        end

        def widget_blick
          Widget.find_by_sql(["SELECT BlickToText(widget_blick) as value 
FROM widgets WHERE id = ?", id]).first.value
        end
end

To verify that this works, fire up the really handy script/console. You should be able to see the following sequence (type things in at the ">>" prompt. The console's responses are marked with a '=>')

$ ./script/console
Loading development environment
>> w = Widget.find_first
=> #
>> w.widget_blick
=> "BLICK(0, 1)" <-- This is the textual representation
>> w.widget_blick = 'BLICK(999, 1000)'
=> "BLICK(999, 1000)"
>> w.save
=> true

Now, check your database table to see that w's widget_blick property indeed got updated to "BLICK(999, 1000)".

We successfully got our model to print as output and accept as input the textual representation for a blick and convert it into a blick behind the scenes for us. For what it's worth, I found this procedure necessary when getting Rails to talk successfully to PostGIS, which uses PostgreSQL's custom types to implement spatial and geographic extensions to the core PostgreSQL database. This HOWTO was made possible with the help of "argv[0]" on the channel #rubyonrails at Freenode IRC.

Posted by Vishy at 12:00 PM | Comments (1)

October 12, 2005

Battle lines in instant messaging?

Today, Yahoo! and Microsoft came to an agreement to let their instant messaging networks interoperate. Widely prevalent analysis has it that by doing so they are taking aim at AOL, who has the largest market share in instant messaging in the United States. With their networks put together, Yahoo! and Microsoft will have more users than AOL's network -- a clear challenge to IM's biggest player at the moment, AOL.

But is something else going on here? This year has shown unprecedented investment by Internet companies into the realm of instant messaging. Yahoo! has introduced voice capabilities into its IM client. Google entered the instant messaging market with its offering, Google Talk. Google Talk debuted with Jabber, an open IM protocol that promised eventual interoperability between the walled-off instant messaging networks of the time. Google Talk is currently in its infancy, but it could prove to be a force to reckon with for Yahoo! and Microsoft if some of the other projects executed by Google are any indication.

This interoperability move by Yahoo! and Microsoft comes at a time when significant units in both companies are in head to head competition with Google in the area of online advertising through their portals. In bringing their IM networks together they have effectively united against their common rival, Google. By interoperating between themselves, they have stolen some of the thunder of Google Talk, the upstart that promised network interoperability. It also opens up the path to future cooperation between these two companies in initiatives to contain Google. This alliance is essentially the instant messaging equivalent of the Google-Sun partnership announced two weeks ago. I wouldn't be surprised if Yahoo! and Microsoft decided to go steady in other future initiatives.

It would of course be foolish to think that the Microsoft-Yahoo! partnership isn't taking aim at AOL because the overt purpose of their alliance is to dwarf the single largest player in instant messaging (moreover, given the significant contribution of AOL to Google's ad revenue -- AOL is the only named third party in Google's filings with the SEC -- targeting AOL's market share may strangle one of Google's major revenue streams after all). However, under the radars of many observers, alliances have formed and battle lines have been drawn on both sides. It should make for a very interesting battle between the upstart and the established players for the hearts and minds of Internet users.

Posted by Vishy at 11:11 PM | Comments (2)

September 29, 2005

Foople

From the point of view of a geek who likes to hack, the best part of Google Maps isn't just the stunning user interface, but also the cool things you can do with the Google Maps API. Recently, after a pleasant dinner appointment with friends, my digestive system had to face less than pleasant consequences. It wasn't anything serious and in any case, I really am not exposed to as much danger as my animal-eating friends. At that point, hopeless geek that I am, I was already itching to hack something up with the Google Maps API. Suddenly a project idea popped into my head.

Foople!

Foople combines location data from Google Maps and restaurant inspection data from the New York Department of Health and Mental Health to provide an overview of the hygienic practices of New York area restaurants. I used Perl to screen scrape Department of Health pages for restaurant inspection data and generate an XML data file with all this information. This data file is the input to some code that displays this information on Google Maps. I'll do a more detailed writeup about the various hacks I had to do to get Foople working. But before I get around to memoizing the hacking experience properly, I thought I'd push out this alpha version so people can play around with it.

Eat safely!

Posted by Vishy at 08:49 PM | Comments (0)

September 18, 2005

The Rise of Javascript

If technology companies could pick a martial art to associate themselves with, Google would surely pick Aikido. Instead of blocking attacker's movements, Aikido chooses to deflect, control and redirect the attacker's energy. Likewise, instead of expending considerable energies to build a brand new Web application platform, Google merely wove together existing technologies such as Javascript and the XMLHttpRequest object to create Google Maps and GMail, now viewed as formidable threats to the traditional model of desktop software. This move by Google has spurred considerable interest in Web-based user interface innovation. The Javascript-XMLHttpRequest combination, now catchily christened Ajax (Asynchronous Javascript And XML), will form the basis of a new framework that will debut in Microsoft's ASP.NET v2.0.

Major Internet companies have also recently built or bought products that give a user greater control over their browsing experience using client-side extensions. A month and a half ago, Yahoo! bought a small company called Konfabulator that develops so-called widgets that draw information from web sites but look and feel like applications running on the local computer. Microsoft has developed a similar product called Microsoft Gadgets. The closest direct answer that Google has to these products is the sidebar in Google Desktop 2. The Greasemonkey extension to the Firefox web browser gives users considerable control over what they want to see on their pages, and has potential to be developed into a similar platform for client-side widgets. As it so happens, the original developer of Greasemonkey is in Google's employ.

Central to the emergence of Ajax and client-side widgets has been the rise of Javascript. Javascript, born as LiveScript, was developed at Netscape Communications Corp. by Brendan Eich. Naming the language Javascript was partly a marketing ploy to whip up hype by association with the Java programming language, which was making waves all over the software community. It was also named Javascript to reflect its original intention -- to give Web developers with no formal Java training access to Netscape's Java engine and browser object model. In a sense, it was similar to the function of Visual Basic on the Windows platform. Just as developers on Microsoft's platform could avoid the arcane traps of C++ by writing in Visual Basic, Web designers could use Javascript to program for the Web while avoiding the barrier to entry that Java presented.

Javascript quickly grew to have a sophisticated object and event model. Unfortunately, it never had a powerful integrated development environment (IDE) or reliable cross-platform debugging because of the browser wars of the 90s. Programmers also perceived it as a simple-minded, second-tier language because it had a low barrier to entry and did not have features of industrial strength programming languages, such as compile-time checking. Javascript can boast of having several distinctive features of modern programming languages, such as function objects, dynamic typing, regular expressions and native support for compound data types such as hashes. Even after Java applets on client browsers withered away, Javascript's object model continued to be used in Dynamic HTML, a lightweight (read non-Macromedia Flash) technique of creating dazzling user interface effects on Web applications.

Thankfully, companies like Google recognized the tasks towards which Javascript's innate technical merits could be applied. By capitalizing on Javascript's versatile user interface event model and the power of XML to send data across the Web, it was possible to create a responsive user interface like that of Google Maps. Javascript seems to be experiencing a resurgence in the software community. There are a few things that Javascript always got right as a platform. Some of these lessons are valuable when considering what could make a new platform successful.

1. Lower barriers to entry. Make the language accessible to software types without requiring them to learn a new development environment and a new set of development tools. Structure the workflow of the language so that it is easy to create quick and dirty prototypes with it. In other words, make complex things simple to express and simple things even simpler. Dynamically typed languages work well for this kind of thing because the developer needn't worry so much about the formal type of a value as what type it should have when used in a certain syntactic context. Even if you provide a runtime environment that disallows raw memory addresses for security reasons, make it easy for developers to insert their own behaviors using concepts like events and function objects.
2. Embed. Embed. Embed. Embedding a programming language into another application gives the programming language an automatic raison d'être. There is a manifest platform against which programmers can begin coding immediately. Nobody has to come up with a conceivable use for the language because it has not been developed in a vacuum. Of course, the developers of the language should take care not to tie the language to the application in which it is embedded. Taking Javascript's case as an example, it shouldn't include native language features, such as keywords, that refer to browsers or windows.
3. Remove barriers between users and developers. Devolve control over the language's development and intended use cases to the users of the language. There should be no rigid one-way flow of technology from a privileged group of language developers to a larger set of users. The ideal goal of a platform's evolution should be to combine as much of the language's developer and user communities as possible. Passionate outreach and teaching will go much further towards this goal than traditional flashy advertising. A smart user may well come up with a use for your language that you never foresaw. Take advantage of the conversations that happen among the users in your market because users of a product can sometimes be a lot more passionate about changing it than developers who have adapted to some of its quirks. This principle may vaguely remind readers of the cluetrain manifesto. The lessons from the manifesto apply as much to technology platforms as to marketing the new wave of products.

It is said that the technology industry has matured to such an extent that completely new platforms must break through extraordinarily high inertia to make an impact. However, a smart platform still can, Aikido-like, redirect the energies of its users to propel itself to greatness.

Posted by Vishy at 01:10 PM | Comments (0)

September 14, 2005

A Shift in Microsoft's Code Names?

Microsoft's PDC has just begun. Today they demonstrated the upcoming version of Office. This version radically changes the user interface by making it task-oriented rather than command oriented. Menus and toolbars have been eliminated in favor of, well, a menutoollbar known as a ribbon. Add to this a vaguely iTunes-like brushed aluminum look and you have the makings of a major visual overhaul of the Office Suite. Yet, the powers that be at Microsoft have chosen to christen the project with the codename "12".

How lame!

The starry-eyed Microsoft, who chooses code names for projects before they have even begun, comes up with this?! What happened to the mystique of Chicago, the promise of Memphis and the Pacific Northwest's culturally relevant Whistler and Longhorn?

Recent delays in product cycles may have taught Microsoft to be less ostentatious with their code names. Much touted operating system releases such as Longhorn have been delayed for so long that their code names themselves started to command some brand equity. In the old Longhorn Developer Center pages, the code name has been used in several places completely idiomatically, not even including so much as a pair of quotes. The name "Longhorn" may not fit with the marketing vision that Windows seeks to portray to its customers. Yet, viewed purely as a product name, "Longhorn" isn't entirely bad for an operating system release. It is perhaps as whimsical as the seemingly unoriginal "Vista", which will be the official name of the Windows release formerly code named "Longhorn". Microsoft kept promising (and later cutting) so many features with Longhorn that it has come to be associated with failure to execute and deliver products on time. Even early betas of the operating system were so delayed that the code names became de facto release names. Yet, a plausible sounding name that had gained some brand equity of its own had to be buried and obliterated completely from public memory.

As if learning from this mistake, Microsoft has deliberately picked a lame code name for the next version of Office. A code name that's just a mundane number downplays the impression that much more effort is put into marketing a Microsoft product than into developing it. It would be a lot less painful to dispose of should delays in development cycles correspondingly delay the choosing of an official product name for Office. Code naming the next version of Office with a number puts considerably less cultural stock in the name and may be telling Microsoft's customers that they are doing away with such niceties in favor of focusing on getting the release out the door. Perhaps it may just be that because Office is to be released along with Windows Vista at the end of 2006, the code name wouldn't need to survive for as long as Longhorn did; the marketing department may have just taken the easy way out.

What are they going to call it eventually anyway? Office Vista?

Posted by Vishy at 12:58 AM | Comments (0)

August 24, 2005

Windows Remixes

Windows has its detractors. Windows also has its fans. Much like how the fan fiction surrounding a literary franchise occasionally reinvents its subject in new and interesting ways, Windows users have gone about providing remixed versions of the operating system everyone loves to hate.

Wired News ran a story about Windows remixes currently available on the software underground. These remixes enhance Windows by paring it down to the bare minimum or by modifying it to serve a specific purpose. There are Knoppix LiveCD-like distributions, which make it easy to boot a computer into a diagnostic mode, as well as bare bones versions such as "Windows XP SP2 Lite Edition". All these redistributions violate Microsoft's license terms and are essentially fly-by-night operations. Moreover, Microsoft is kind enough to remind you that accepting modifications from anyone other than the original publishers of Windows, themselves, poses a security risk.

If you have the crippled Windows XP Home Edition and you continually ache for something better, you can assuage some of that pain legally. An article I came across the other day describes how to convert your copy of Windows XP Home into Windows XP HomePro. This HomePro Edition re-enables those features that had been crippled in Windows XP Professional to yield Windows XP Home. Most importantly, it makes your formerly XP Home computer start believing that it runs Windows XP Professional. Of course, the HomePro edition does not provide all the features of Windows XP Professional, but it definitely adds back several useful features that power users miss.

Posted by Vishy at 11:59 PM | Comments (0)

July 02, 2005

Separation between an application and its host runtime: a case study

The free availability of source code for modern object runtimes such as the .NET CLR and JVM is very helpful in figuring out the internals of these industrial strength runtimes. If you are an independent learner, perusing the source code and somewhat terse specifications of these runtimes goes quite a bit further than visiting your local bookstore for a technical book. Sometimes you end up learning things that you wouldn't otherwise notice when working with these runtimes.

'Hello world' programs in both C# and Java look nearly identical, except for keyword differences. However, whereas Java requires the main method of a class to be both static and public, C# is satisfied with a main method of any accessibility level, so long as it is static. I will quote here two code snippets taken from the source code to the Java and .NET runtimes. In the file ${SRC_ROOT}/java.c from the source jar of the Java Virtual Machine, we see the snippet:

  /* Make sure the main method is public */


  jint mods;

  jmethodID mid;

  jobject obj = (*env)->ToReflectedMethod(env, mainClass, 

mainID, JNI_TRUE);



  ...

  mid = (*env)->GetMethodID(env, (*env)->GetObjectClass(env, obj),

 "getModifiers", "()I");

  ...

  mods = (*env)->CallIntMethod(env, obj, mid);

	

  if ((mods & 1) == 0) { /* if (!Modifier.isPublic(mods)) ... */

                    message = "Main method not public.";

	    messageDest = JNI_TRUE;

	    goto leave;

	

  }

Compare this to a snippet from ${SRC_ROOT}/clr/src/vm/clsload.cpp:

 // Returns true if this is a valid main method?

 void ValidateMainMethod(MethodDesc * pFD, CorEntryPointType *pType)

 {

         _ASSERTE(pType);

                 // Must be static, but we don't care about accessibility

     THROWSCOMPLUSEXCEPTION();

     if ((pFD->GetAttrs() & mdStatic) == 0) 

         ThrowMainMethodException(pFD, IDS_EE_MAIN_METHOD_MUST_BE_STATIC);

These snippets show the main method of an executable class being looked up and validated for passing control into by the object runtime. The comments in the code should make things sufficiently clear. In the JVM implementation, it is mandatory for the main method to be both public and static. In the C# implementation, the comment declares that the runtime doesn't care about the accessibility of the Main method so long as it is static. With the additional restrictions imposed by the JVM implementation, you could, in theory, have multiple methods at the bytecode level named main, all of which take an array of strings as formal parameters, with different degrees of visibility. However, when emitting bytecode/IL, both Java and C# compilers prohibit methods with identical signatures differing solely in accessibility at the language level. In other words, the extra constraint imposed by the JVM implementation isn't because Java offers expanded method-naming features over C#. So, with that argument out of the way, which runtime implementation has the better approach?

In a managed object runtime, it is very easy to coalesce the logic of the runtime and the code it executes into one system, from a point of view external to both. The key question is to what extent internal boundaries are still maintained despite this apparent coalescence. The JVM implementation adopts a purist object-oriented approach. In other words, if the main method is not declared by the class as being visible to the world, it is not visible to the runtime either. The JVM implementation relies on and adheres to well-known definitions of member visibility from object-oriented theory. It does not make any assumptions about the function of a method that just happens to be named main, unless the method fulfils visibility (public) and instantiation (static) criteria. In stark contrast, the CLR implementation drops all pretense and says that if there is a static Main method available at all, it has got to be the entry point for the .NET application. The CLR approach too has its own merits. Naming static methods 'Main' when you don't intend them to be application entry points is extremely poor form for a programmer. Nonetheless, the JVM still makes fewer assumptions about the code it will host. In so doing, it demonstrates a philosophy of maintaining a more rigorous separation between the runtime and the code it is executing. Further evidence of this way of thinking of the runtime and an application as separate entities is seen in how the JVM has to be configured and invoked explicitly with the name of the main class ('$ java [options] MainClass'), rather than simply executing the main class directly ('$ MainClass.exe')

Applications can be written very efficiently if they make many assumptions about the underlying runtime and development environment. For example, exploiting implementation-specific quirks in how compilers generate code leads to application code that is as brittle and unportable as it is efficient. The problem is that such applications have effectively made the compiler and runtime part of themselves. I do not claim that the JVM necessarily maintains a better separation between itself and application code than the CLR. For example, the JVM does not have a counterpart to AppDomains, handy units of code protection and isolation in the CLR. The CLR can set up an application to run in an AppDomain separate from its own, thereby completely isolating itself from unexpected application behavior. However, in the simple matter of the main method of an application, we see a slight divergence in the philosophies of the Java runtime and the CLR. The JVM demonstrates a stricter separation between an application and its host runtime and takes the better approach.

[C# fans: the bad karma I have generated for myself by portraying my primary programming language these days in a negative light will be neutralized by vicious Java-bashing in a future blog entry. Worry not. --V]

Posted by Vishy at 01:12 PM | Comments (0)

May 30, 2005

The cast system in C++

A long time ago, when the prevalent language was C, all casts were equal in the eyes of society. Then Stroustrup created C++. As C++ grew, its cast system became explicit and rigid. The cast system in C++ has four casts -- dynamic_cast, static_cast, const_cast and reinterpret_cast. The function of these casts is to convert from type to another. The types involved in a cast are usually pointer types, though this is by no means necessary. Even though the intent of the cast system in C++ was merely to mark the different kinds of type conversion, programmers have accorded the four casts different levels of status and desirability. We will learn about these casts in order of their desirability and the destabilizing effects they have on programs.

The highest cast is the dynamic_cast. When converting a pointer of one type to a pointer of another type, dynamic_cast performs a runtime check to make sure that the cast is safe. If the cast is unsafe, a dynamic_cast results in a null pointer. Other programming languages, such as C^# with its as operator, have emulated this cast, evidently as a form of Sanskritization. By performing a runtime check, the dynamic_cast distinguished itself as the safest of all the casts. It is least likely to cause access violations or other forms of unrest in a program and therefore leads to the least violent death for a program.

The next most prestigious cast we will consider is the static_cast. This cast does not believe in incurring performance penalties by performing runtime checks. When static_cast-ing from a Fruit* to an Orange*, the conversion will succeed, but if the pointer was really pointing to an Apple, then invoking the getNumSegments() method on it would cause an access violation. The static_cast is less reflective and more worldly because it takes the type information provided in the static_cast statement as the literal truth, even if it results in a little violence down the road.

The runtime information available to the dynamic_cast combined with the authoritarianism of the static_cast has given the nexus of these two casts enormous influence over the other casts. A variable may be created as a Fruit*, but it has the opportunity to be reborn legitimately through the rituals of the upper casts as a Banana*. This phenomenon of being born again has led to this nexus being called the twice-born casts.

The next cast we will examine is the const_cast. The function of this cast is to cast away the const-ness of a pointer. Const-correctness, when used correctly, is a very important and useful feature in C++. In the illusory universe of the stack, where variables and pointers are constantly in flux, it imbues them with an unchanging beauty for as long as they are in scope. The const_cast is willing to trade away this const-ness in return for the convenience of changing a const variable's value. Unless blessed by the mutable keyword, casting away the const-ness of a value is a serious risk to the correctness of a program, which may result in either violent death or unpredictable behavior.

The cast at the very bottom of the cast system is the reinterpret_cast. Faint-of-heart programmers shrink away in horror from the mere shadow of the reinterpret_cast. The reinterpret_cast does the dirty work of converting between unrelated types. Through the menial hands of the reinterpret_cast, a Fish* may be reinterpreted as a Bicycle*. Most coding guidelines in C++ either prohibit the reinterpret_cast or impose severe restrictions on its life. It should be clear why this is the case. If an object is arbitrarily transformed into another of an unrelated type at random, the natural order inherent in society would be transformed to chaos.

Coming across each of these casts designated explicitly in C++ code is an eyesore to code readers and is a testament to the ugliness of the explicit cast system in C++. It is important to realize that each of these casts has its place in an advanced systems programming language like C++. Any type conversion is necessarily an unnatural act to some degree. Explicitly marking them as such is merely a constant reminder to writers and readers of code alike that objects of all different kinds must live together in harmony.

(If the extended metaphor in the above explanation seems entirely lost on you, you are probably not familiar enough with Indian culture. Feel free to email me or talk to me about it. I hope you will still find the technical content useful, because it is accurate. If you are familiar with Indian culture and know what I am referring to, you should know that I don't believe in it one bit. This is intended as merely an extended inside joke that occurred to me in a moment of inspiration.)

Posted by Vishy at 12:19 PM | Comments (1)

May 11, 2005

The influence of organizational structure on software

It isn't a stretch to say that the high-level structure of software is influenced by the structure of the organization that produces it. Most successful software today is organized around several high-level components or modules. Each of these modules display the following characteristics:

1. It has a published, well-defined interface for other modules and processes to access its functionality.
2. It tries to minimize external dependencies so that as an application evolves, changes occur in few sharply defined parts rather than diffusively through the application.
3. It hides internal implementation details so that changes and extensions can occur without breaking code that already depends on it.

When the initial design for an application is created, it makes sense for an independent programming team to take on bringing a module from idea to product. Note that this doesn't always mean there are as many teams as there are modules. All it means is that every module typically has one team working on it and that not every team needs to know the internal details of modules it does not own. Thus, in early stages of the application, the design of the application influences the structure of the teams that build it. As the application evolves, this organizational structure becomes more entrenched and the incentive to reorganize the application development team grows smaller because things are working well presently. The cause-effect relationship begins to change and the structure of the application begins to reflect who talks to whom (more on this below).

The question this essay tries to address is how close this organizationally-influenced module structure comes to the technically optimal structure for an application.

(Before you read any further, I would like to note that the idea of organizational structure influencing application structure is not an original idea from me. I had heard it being mentioned in my software engineering class at MIT, but when I looked for it recently on Google, I was unable to find a good attribution for it. If anybody could point me to a good formal statement of this principle, I'd greatly appreciate it.)

Inherent in a company's organizational structure is the organization of its knowledge and intellectual capital. Despite working within the same company, ostensibly towards the same goals, it is hard to eliminate redundant work entirely and make every team perform tasks that perfectly complement other teams' tasks. Personal and team competitiveness ensure that some amount of work gets duplicated between teams that have similar goals. Furthermore, the same competitive spirit leads teams working on similar projects to be secretive and not share details with each other. Several examples of this occurred when HP merged with Compaq under the watch of Carly Fiorina. According to some accounts, for a while, teams from HP and Compaq that were in the same field competed against each other to ensure their survival.

Let's consider how exactly a company's organization can affect the organization of its software products over the course of time. In companies that have a product-centric organization, such as Microsoft, the structure of their software is related to who talks to whom. We have seen some products that worked well, that were designed for each other, such as Windows 3.1 and MS-DOS 5. Other well-integrated technologies like the Office suite work well together because there were direct lines of communication between those development teams. The depth of communication between the various Office teams led to technologies like Object Linking and Embedding, and indirectly influenced the later development of COM, COM+ and the .NET runtime.

Yet, from the same company, we have seen two data access technologies -- DAO and RDO around the same time. I am not familiar with the intimate details of their programming models, but at first glance, it would seem that there is significant overlap in their functions. (Of course, we are not mentioning ADO here, which was another data access library that came later.) Why should all this redundancy in data access libraries have occurred in the first place? My best guess is that the development teams didn't talk to each other all that much. Even today, when I peruse MSDN Blogs, I see comments from Microsoft developers about how they don't know enough about what other groups are doing in such a large software company. When components from teams who don't communicate are brought together to form an application, the components won't always fit well together. In other words, such an application would not always have a technically optimal design. Once a application is released, opportunities to fix these chinks in the proverbial armor become scarce. Kludges and hacks are introduced that fix the problem while maintaining compatibility with the status quo. The evolution and technical optimality of an application are thus influenced by the communication (or lack thereof) between development teams within a software company -- a factor that is often influenced by contingencies having nothing to do with the technical aspects of an application.

Another factor influencing the evolution of an application is its present state. The way an application currently behaves is surely the most obvious source from which to harvest ideas for what can be added, removed or improved in it. Looking closely at an application and analyzing how closely its behavior approximates its idealized requirements may lead to refactoring and code-level interface enhancements. Adding new features to the application may involve clever reuse of code from other components. All together, these activities may boost the technical optimality of an application's design. There isn't much need to belabor the point that an application's evolution is highly influenced by its present design.

So far, we have concluded that the technical optimality of an application's design is influenced by some combination of its present design and the communications structure of the organization that built it. Let us assume for a moment that there exists a truly ideal technical design for an application. It is worth taking a moment to consider which of the two factors we have considered is more likely to push the application's current design towards this truly ideal design. Development teams not talking to each other may lead to some missed opportunities in improving a design technically. Some ideas may never make through the pipeline. As market dynamics and organizational priorities change, these missed opportunities 'stored' somewhere in the collective brain of a team may fall out of focus and never be realized in future iterations of the application. On the other hand, technical shortcomings in the current design of an application will always be present and are governed by such contingencies to a lesser degree. In other words, the current source code of an application and associated documentation acts as a viable and sufficiently stable medium of communication between developers and among development teams, without its being adversely influenced by communications shenanigans within and among development teams.

The above question becomes relevant in the case of open source software, where there are usually no well-defined product teams. Control is decentralized among multiple developers, who may be spread across multiple timezones. Far from even a semblance of a team working in a development cycle, all the developers have to work off is the current source code snapshot of the application and associated documentation. With such a development model, intellectual capital that isn't written out isn't intellectual capital at all. As a result, hardly anything goes without being written down somewhere, where other developers might read it and act upon it. All communication between developers happens in only two ways: explicitly in the aforementioned rigid, restricted way or implicitly through the current state of the application as expressed by its design and source code. Because there is no intellectual capital lost to organizational communications structure, a decentralized, open source software development model is more likely to result in a technically optimal software design.

Posted by Vishy at 12:44 AM | Comments (0)

January 21, 2005

Useless Factoid of the Day #2: The Golden Ratio in the C++ STL

The Golden Ratio is any ratio where the following holds true:

a:b == (a+b):a

The Golden Ratio can also be defined as the ratio of successive Fibonacci numbers. Its approximate value is 1.618:1 and its exact value is (1 + √5)/2. This ratio occurs in the radii of successive spirals in the shell of a nautilus and in the numbers of successive generations of wild rabbits. The Golden Ratio is also said to resonate very well with the human visual system's sense of esthetics. The navel is said to divide the length of the human body into sections related by the Golden Ratio. Indeed, the ancient Greeks used the Golden Ratio in several of their buildings. Mathematicians and philosophers have cited the Golden Ratio as proof of how apparently unrelated aspects of the universe still speak the universal language of mathematics.

Did you know how the Golden Ratio is linked to the C++ collections implementation in the Standard Template Library? What in the world do wild rabbits, nautiluses and buildings in ancient Greece have to do with code to grow a resizable collection of objects in C++? Read on for what is probably one of the more unusual places in which the Golden Ratio appears.

Dynamic memory allocation

Computer programs can allocate memory to store values in two ways: statically and dynamically. Static memory allocation refers to a situation where the amount of memory needed is known in advance and can be specified in the source code of the program itself. Dynamic memory allocation refers to a situation where a program may request an amount of memory which may change depending on conditions during its execution. Dynamic memory allocators dole out memory from an area of program-accessible memory called the heap. Typically, they maintain a linked list of available blocks of memory, using which they respond to dynamic memory allocation requests. Given a request for a certain number of bytes of heap memory, an allocation could take place in several ways. An allocator could traverse its list of available blocks and return the first block that would accommodate the number of bytes requested. Such an allocation is called a first-fit allocation. Alternatively, an allocator could traverse its list of available blocks until it finds an available block, whose size is closest to the number of bytes requested. This type of allocation is called a best-fit allocation. For the sake of simplicity, we will assume for the rest of this entry that allocators perform first-fit allocations.

Resizable collections

Collections are entities in a program that can be treated as one cohesive object and addressed in a unified and predictable manner. For example, arrays are contiguous areas of memory which start at a well-known address in memory and have a fixed length once they are allocated. Dynamically resizable collections are those that grow to accommodate any number of items that are added to them, even if the number of items to be added was not known in advance. An example of a dynamically resizable collection is the ArrayList class in Java and C#. This collection is based upon an array which is usually larger than the number of items in the collection. It is time to resize an array when it gets "too crowded" and the number of items in the collection comes too close to the size of the array. A new array of a bigger size is dynamically allocated from the heap and the members of the old array are copied to the new array.

Let's consider how the size of the new array is related to the size of the old array. Let us suppose the size of the old array is x. The size of the new array could then be some multiple of x, say k · x. A naive implementation would request the new array to be simply twice the size of the old array. This expansion would guarantee that all the items from the old array would fit into the new array, leaving plenty of room for any new additions. In this case, k = 2. However, let us see if we can get away with smaller values of k to save some space.

Let us say the initial allocation of an array is x.
Upon the first resize, the allocation would be k · x.
Upon the second resize, the allocation would be k · k · x.

We have to ensure that the second resize can accommodate the results of the previous two allocations. In other words, x + k · x <= k · k · x. Assuming a nonzero initial allocation (why would anyone want to dynamically allocate *no* memory?), we can factor out x, leaving the quadratic inequality

k2 - k - 1 <= 0

which can be solved to produce k <= (1 + √5)/2. In other words, we can get away with a resizing factor as low as the Golden Ratio, or even lower! A few STL implementations resize their collections using a factor of 1.5, for a good mix of efficiency and convenient memory.

This has been a presentation by the World's Largest Repository of Useless Knowledge, me. For more information, look up Andrew Koenig's paper on this subject using Google.

Posted by Vishy at 01:13 AM | Comments (1)

January 05, 2005

Perl 6: A Cool Programming Language

Perl may inspire strong feelings of love and hate in people, but it is unarguably a unique language.

On the surface it sort of looks like C and was initially meant to be an awk-killer. However, you quickly realize that except for the semicolons, braces and a few functions names, Perl has little in common with C. Perl was designed by Larry Wall, a linguist by training. If you read Perl scripts you'll quickly realize that they read like some other languages you may know, like *gasp* English! Heck, Perl is probably the only programming language that has a culture of poetry!.

Programming in Perl can be a LOT of fun (more on that in the extended entry) Nevertheless, Perl 5, which is the latest and greatest Perl out there, suffers from some stodgy constructs that seem to take some of the fun out of hacking up a quick and dirty Perl script. Perl 6 goes a long way in fixing some of it, as can be seen in this great article. This blog entry is about why programming in Perl is fun and about what I think are the best new features of Perl 6!

Perl is fun!

Just compare
open(INPUT, $file) or die("Cannot open $file!");

with
try {   File f = new File(filename);
  FileInputStream stream = new FileInputStream(f);
} catch(FileNotFoundException e1) {
  // Handle this exception
} catch(SecurityException e) {
  // Handle this exception
}


The code excerpt on the top is a line of Perl, while the code excerpt below accomplishes the same thing in Java. It is clear that the Perl reads like something you would say in everyday speech. "Open the file or die trying!" The Java example has much more structured exception handling and a more formal way of opening a stream to a file. The Perl script, on the other hand, Does What You Mean (tm) and worries about the error handling in a different way entirely.

Perl has a type system that may seem weird if you come from a C/C++/Java/C# background. However, very soon you'll learn to like it because Perl's types correspond roughly to how our linguistic system is wired. Perl has a scalar type, which could be a number, a word or "one of anything". In Perl you would say, "Pass me the ball", in C you'd be saying "Pass me the ball which is exactly 14.75 inches in diameter". Do you really care so long as it's clear you want just one ball?

Perl also has an array type, which is "something made of other things". In Perl, you'd be passing around a basket of apples. The basket may be empty or full and if you need to put in more apples at any time, you just add in the apples. What's more, you can also put in apples, pears, cupcakes, knives and anything that could go into a basket. In C and C++, you'd be required to indicate exactly how many apples that basket can take before filling it up with apples. If you have to put in an extra apple, you either can't do it or you move all the apples from this basket into an entirely new basket on your own. If you didn't put in enough apples to fill up the basket, random undesirable things may appear in the space that is left over. But no, you can put in only apples and not anything that could go into a basket.

Perl has a hash type, which is "a bunch of pairs of things" which are associated with each other in some way. You could use a hash to indicate the concept of relationships. You could use a hash to say "George Bush is related to America in the same way as Queen Elizabeth is related to the United Kingdom" And later, you could say "Oh wait! Jacques Chirac is related to France in the same way too!" Without using a special data structure called a hashtable in C, you'd have to come up with your own way of keeping track. You could keep two arrays of things where a country and its head of state are stored at the same place in each array, but that doesn't make their relationship explicit enough.

Perl deals in things, groups of things and relationships between things. It doesn't much care what the thing is, how it is stored and so on. Contrast this with a C-like language, where the language imposes terms on what a thing could mean. Is the thing stored using 16 bits, 32 bits or 64 bits? Or, is it a letter instead? Or *gasp*, a word? Perl is this magic language, but ultimately it runs on real computers that unfortunately deal in ugly things like 16, 32 and 64 bit chunks. But rather than expose all that to you, it runs around pasting pieces of duct tape everywhere so your program holds together.

Another big feature of Perl is that it is a programming language with pronouns. This fact plays a big role in the natural feeling you get when you read Perl, because we use a lot of pronouns to refer to the immediate context of a sentence, when we understand and process English. In a set of instructions like "Soak shirt in laundry detergent for two hours. Wash. Rinse. Repeat until stain is gone.", there is an implicit 'it' after the verbs at the start of the last three sentences. We know exactly what each 'it' refers to. The first two its refer to the shirt and the last it refers to the set of instructions itself. Consider what would happen if we were really explicit about everything and did not use the implicit 'it's at all. The instructions would read "Soak shirt in laundry detergent for two hours. Wash the shirt. Rinse the shirt. Repeat 'Soak shirt in laundry detergent for two hours. Wash the shirt. Rinse the shirt.' until the stain is gone." Sounds a bit convoluted, doesn't it?

If you wanted to cut every apple in a basket of apples in C, your code would look something like this:
Apple basket[10];
// ...
for(int j = 0; j < 10; ++j)
{
  Apple an_apple = basket[j];
  cut(an_apple);
}


In English, your code would read something like "Counting from 0 upwards until 10 (because that is *exactly* how many apples this basket can hold), let us assign it to a number J. Pick up the Jth apple from the basket and set it on the table. Let's call this apple A. Now, cut A"

In Perl, your code would read like this:
my @basket;
foreach(@basket)
{
  cut;
}

You can practically read it out aloud and figure out what it says! Notice that you don't ever explicitly refer to the apple you're cutting right now. All you say is "for each apple in the basket, cut it". Perl has a magic variable called the "topic variable", to which it automatically assigns the apple you're working on right now. Its value can be accessed through the variable name $_. The same $_ acts as the 'it' and is passed into the subroutine 'cut'. See how simple and natural it is? So natural that even a C-style language like C# has borrowed from Perl and introduced a foreach loop, in addition to the usual for, while and do...while loops.
Perl also has flexible clause ordering, a distinctive feature that makes it seem chatty and colloquial compared to other programming languages that mandate a fixed order of if(condition) action(); You can write valid Perl that reads like so:
print("$variable not defined!") if !defined($variable);
or
print("$variable has a value!") unless !defined($variable);
Perl allows an if or a while checking for a condition to be placed after the set of actions associated with that condition. The statement has exactly the same effect as if the if or while were placed before the set of actions for a condition. This flexibility mimics how we occasionally put ifs or whiles at the end of our sentences when we speak English. Note that this flexibility parallels English in another subtle way. If we are about to issue a complicated set of instructions for some condition, we usually express the condition before the actions. For example, we might say "If the cabin pressure goes down, grab the oxygen masks that appear. Put the mask over your mouth and nose. Continue breathing normally" However, if we're describing only one action to take when a condition is true, we might say "Turn off the A/C if it gets too cold" Perl supports the post-action if only if the action consists of one statement.

Another reason Perl is so much fun to write is that the wonderful hacker culture around Perl has given rise to remarkable function names, such as croak, carp, warn and bless. Variables can be "my" variables or "our" variables. The language really starts to be fun when you think you can get away with common words like my and our in something as official, boring and formal as a computer language.

So by now, you must be convinced Perl is a lot of fun to write, but it is definitely not fun to read, especially if you didn't write the code. Perl is well reputed to be a write-only language. Code once written, even by you, can be really hard to figure out, if you're trying to make sense of it later. Perl also lacks important constructs like C's switch..case, which takes a set of actions based on the different values that can be taken by a variable. It could be achieved with an onerous sequences of ifs and elses, but that's the best Perl can offer in terms of language support, without going into complicated constructs like hashes of function references. This language, with a culture of poetry, is also one with a tradition of programmers competing to write the most obfuscated code.

And Along Comes Perl 6...

Perl 6 is the next redesign of Perl. It is a rather ambitious departure from the currently available versions of Perl. It adds a few new language constructs, changes the feel of the language a little bit and makes it easier for programmers to avoid common mistakes. I'll briefly mention a few things I love about Perl 6 and let you read about the rest at http://www.linux-mag.com/2003-04/perl_6_01.html.

The given...when construct

Perl 5 lacks a switch...case, but Perl 6 has one. It's called given...when. If you want to take different courses of action based on the values of a variable, you can write Perl 6 code that reads like so:
given $country
{
$nbsp; when 'U.S.' { print 'North America' }
$nbsp; when 'India' { print 'Asia' }
$nbsp; when 'France' { print 'Europe' }
$nbsp; when 'Egypt' { print 'Africa' }
}

See how wonderfully and naturally it reads? The designers of Perl 6 have clearly prioritized a natural feel for this construct over sticking with the analogous switch...case construct in Perl's forebears.

Junctions

Perl 6 introduces a new fundamental type in the language, called a junction, which converts long and onerous boolean conditions in other programming languages to statements that practically read like English! Let's say you wanted to see if any of the numbers in a list was greater than 10 and print a message if that were the case. In classic Perl, you could do something like
foreach(@numbers) { print "Too big!" if $_ > 10; }

of if you wanted to get fancier,
map { print "Too big!" if $_ > 10; }, @numbers;


But, with Perl 6, you can say
print "Too big" if(any(@numbers) > 10);


The function any returns a disjunction (a collective OR) of all the variables in the list, so that each number, in effect, is compared against 10 and the condition is true if any one of them is greater than 10. Another kind of junction is a conjunction, which lets you write code like
print "$num is the biggest so far!" if(all(@numbers)) < $num;


What I am most excited about is that this will let you reduce the somewhat clunky
if($foo > 1 && $foo > 2 && $foo > 3)
{
  do something
}

to the considerably less clunky
if($foo > all((1, 2, 3))
{
  do something
}

where you don't keep mentioning $foo repeatedly. If you aren't impressed, try an abjunction:
if(one(@roots) == 0)
{
  print "This polynomial has a unique root"
}


Now, if reading code with junctions doesn't remind you of another language you know (*cough* English *cough*), then who knows, maybe you don't know it that well? ;) Junctions are also ideal candidates for optimization, because the inherent parallelism in comparisons and other operations scripts perform with junctions lets compilers allocate them to different threads or processors as it sees appropriate. This junction stuff packs a powerful punch.

Consistent sigils

This is a fairly major change that helps Perl programmers avoid common and hard to spot mistakes and brings the language more in line with our own linguistic intuitions. This would have to be my favorite feature of them all.

As you may know, Perl uses different prefixes on variables depending on the context in which the variable is used. For example, you'd declare scalars, arrays and hashes as follows:
my $scalar;
my @apples;
my %hash;


These prefixes are known as sigils. Perl programmers: did you even know they were called sigils? A sigil in a non-Perl context means a symbol or a sign.

However, when referring to the 5th element of @apples, you might think you could use @apples[4], but Perl 5 makes you use $apples[4]. In other words, you are supposed to use the scalar sigil even if you are referring to an element from an array. As you will see explained in the article about Perl 6above, sigils are like demonstrative pronouns. $ would roughly correspond to 'that' and @ would roughly correspond to 'those'. $number would be 'that number' and @apples would be 'those apples'. In English you'd refer to an apple as 'one of those apples'. However, Perl 5, in its infinite wisdom, makes you refer to it roughly as 'one of that apples'. This convention in Perl 5 isn't bad once you get used to it, but it is counterintuitive to begin with. This counterintuitiveness alone lets inadvertent mistakes slip in every once in a while. Sometimes these mistakes are caught by the Perl compiler. Otherwise, depending on which sigil you mix up with which other, these mistakes can cause your script to fail far away from the site of the mix-up, or worse, produce utterly unpredictable behavior.

Perl 6 fixes sigils to bring them in line with our linguistic intuitions. In other words, you can refer to the fifth apple in a basket of @apples as @apples[4]!

Summary and conclusion

I hope the above gave you some flavor of why Perl is a fun programming language in which to write code. Perl 5, the latest Perl out there, is undoubtedly fun, but also a bit rough around the edges. Perl 6 is an ambitious redesign of the language which adds a few new language features, fixes some of the rough edges and adds considerable power.

Posted by Vishy at 09:31 PM | Comments (0)