Brian Lam reports that the rumored Apple tablet will be launched with a variety of multimedia-enhanced text sources. If correct, iTunes LP and iTunes Extras are merely the first steps in an envisioned world of enhanced digital formats.

Would such formats be open to anyone other than Apple? iTunes LP is, but Lam writes:

The logic here is that textbooks are sold new at a few hundred dollars, and resold by local stores without any kickbacks to publishers. A DRM’d one-time-use book would not only be attractive because publishers would earn more money, but electronic text books would be able to be sold for a fraction of the cost, cutting out book stores and creating a landslide marketshare shift by means of that huge price differential.

While Apple does seem rather taken with HTML these days — again, the iTunes LP format and the Safari 4 welcome page — the invocation of DRM means there has to be some kind of lockdown preventing the world at large from using the format.

As much as I love the idea of spiffy multimedia content, I don’t want Apple (or anyone else) controlling that future.

With both the new buttonless trackpads and the new iPod Shuffle, it seems that Apple’s going on an all-out war to eliminate as many buttons as possible from their products.

My opinion, from December of last year:

Keeping with Steve Jobs’ seeming fear of buttons …

Or maybe machines should be smarter so buttons are irrelevant.

David Bennahum, 01995:

Thanks to the microprocessor things get so much more powerful so quickly that there is a real fear of not being “backwards compatible”, meaning the files from 01982 should still somehow run in the word-processor from 01982, even if it is 01995. This kind of uncertainty scares people with really big pockets, people who have thousands of employees and can spend thousands on one order. From day one those people were just *not* going to spend money on a company called Apple best known for a machine built in a garage by two hackers, a machine that kids loved because of the video games. We seem to forget those days.

A little bit of history.

In 01981 IBM launched its first personal computer, the aptly named Personal Computer. Back then there was only one kind of personal computer — the video game machine. You had three choices: Atari, Apple and Commodore. Weirdoes who liked monochrome bought TRS-80s (aka the “trash eighty”) from Radio Shack with an audio-cassette tape drive. In this kind of universe, you’re Apple and you’re thinking “how are we going to convince companies that a personal computer belongs in their offices?” The answer is, you’re not. No way. Not Apple. Not Atari. Not anyone but IBM is ever going to convince corporate America to buy this thing called a PC. So Apple, rightfully, walked away from a market they never were a contender in; and from 01981 to 01994 the office market drove the expansion of personal computer industry.

…

The Pippin is part of a project with Bandai, a Japanese game company known for its effluvium of Saturday morning cartoons, video games, and other brain candy for kids. The logic goes like this. As computers become household commodities, like consumer electronics, Apple stands heads above the other PC makers as the only one with any concept of design, marketing and the shopper’s impulse — the kind of stuff Walkmen, Swatches and GameBoys are made of. With the increasing popularity of the Internet, the demand for cheap household computers will dovetail nicely with the demand for ever more immersive video-game machines. Who else but Apple has the skill to enter this market? Who? If Apple is lucky they will forge an alliance with Sony and build these $300 hybrid PC/video-game/Internet gateways together. Could you imagine how many they would sell at Christmas? Tons.

The copy is fun, but it’s certainly not the modern Apple voice — it’s written from the perspective of somebody who knows that you have to deal with the whims of flawed technology, but hey, here’s something to help. The current style is more of one that says ‘We are artists and life is beautiful. Our products will help you appreciate the wonderful world.’ It’s more restrained, but no less human.

Today I’ve been looking for information about the origins of Apple’s ‘data detector’ technology. Although it was available in some form starting in the late 90s, it seems to have disappeared at some point, resurfacing with the release of Leopard. (See also.)

In Leopard, there’s a private (i.e. Apple-only) framework, which can be found at /System/Library/PrivateFrameworks/DataDetectorsCore.framework/. The detectors are in plain-text format, and can be seen in the Resources/Patterns subdirectory.

The system seems to have a loose library structure. There is a common.ddn file that defines keywords in a manner similar to POSIX’s character classes; this file is often imported into more specific detectors. There also appears to be some minor similarity to the syntax of Pascal, but I’m not a programmer so don’t assume this means anything.

Anyhow, here are some historic mentions of the subject I’ve found, sorted roughly by publication date:

Collaborative, programmable intelligent agents (Bonnie Nardi, James Miller, David Wright; 01998)

This is presumably the first public mention of the technology Update: it is not; see here, and the paper was published in Communications of the ACM in March of 1998.

It seems that even from the start, the system was intended to be a plugin architecture — there would perhaps be some included detectors, but users could easily create their own detectors with the help of a fairly simple syntax that loosely resembles regular expressions. The present syntax seems to have a generally similar format, although it may be complexified somewhat for greater power.

The goal of our research on intelligent agents was to create something useful for our customers, but something with that sprinkling of pixie dust that would make it seem “intelligent.” Ben Shneiderman observed that claims about intelligent software agents are vague, dreamy and unrealized. We started from a simple but focused approach to agents, that they should have the ability to infer appropriate high-level goals from user actions and requests, and take action to achieve those goals. Further, based on a study of reference librarians as exemplary human agents we wanted to build a system in which the user would not have to state goals explicitly and in detail — we learned from librarians that a large part of the value they provide to clients is in working with imprecise requests. Beyond this, our general design strategy was to keep the user’s question in front of us at all times: Will this software do something useful for me, in an intelligent way that makes me more productive? The system we describe here — Apple Data Detectors — meets our criteria of being unobtrusive, having the ability to infer user needs, and doing useful work. Apple Data Detectors will ship as a product in 01997.

…

Past work on intelligent agents has been multi-faceted, to the point where it is difficult to find consensus on exactly what constitutes an agent. Researchers have used machine learning techniques to track user actions and construct models of user preferences, created agents that employ user models, consulting a set of parameters that describe the user, implemented planning systems to make the leap from a user’s stated intention to the specific actions that are required to achieve that intention, and built agents that act as “eager assistants”. The locality of agents also varies across different agent-based systems: some act only within one’s own machine, while others autonomously crawl the Web, searching for interesting content, for example. Apple Data Detectors works on the user’s own machine, and falls into the eager assistant category, enabling rapid user action with minimal input on the part of the user.

…

In an investigation of how people file information on their computer-based “desktops”, we discovered that a common complaint of users is that they cannot easily take action on the structured information found in everyday documents. By structured information we mean data recognizable by a grammar. Ordinary documents are full of such structured information: phone numbers, fax numbers, street addresses, email addresses, email signatures, abstracts, tables of contents, lists of references, tables, figures, captions, meeting announcements, Web addresses, and so forth. In addition, there are countless domain-specific structures such as ISBN numbers, stock symbols, chemical structures, mathematical equations and so forth. These structures are not only relevant to users, but are also recognizable by present day parsing technologies. The type of a structure can be used to identify appropriate actions that might be carried out on the structure — place a meeting on a calendar, add an address to an address book, dial a phone number, open a URL, find the current price of a stock, file an ISBN number, compile a list of abstracts, and so forth. The system we developed to enable people to work more fluidly with structured information is called Apple Data Detectors.

I’ve put together a comparison shot of the data detectors being used on an email message. The top capture demonstrates the system in use in 01997, and is included with the article; the lower one demonstrates the system as it appears in Leopard.

The plan included the ability to build on other detectors, which created something of a dependency jumble:

The solution for shared, compatible detectors requires developers to register their detectors. Our registry is supported by Component Integration Labs, a company responsible for maintaining the Apple Event Suites (among other standards). Apple Data Detectors will make use of the registry which contains definitions for classes of data objects and for events that operate on the objects. Classes of detectors will be defined as data objects and developers must write detectors that detect required fields of the detector class. Detectors can detect more information than the class requires, but they must detect at least the data that the class requires.

This may still be an issue as a result of the import command; I don’t know what happens if a detector attempts to pull an external detector that doesn’t exist — and because this is a private framework there’s no documentation available other than the included patterns.

This article was briefly mentioned in a paper on teaching machines how to understand human language:

Recognizing textual entities

[…] These entities may be detected using various techniques. Regular expressions and pattern matching are often used. Apple Data Detectors (Nardi, Miller, & Wright, 01998) use context-free grammars.

AppleScript in a Nutshell, pp. 35 & 36 (Bruce Perry; 02001)

Script Editor Controls/Commands

[…] In addition, when creating a script to use an Apple Data Detector (ADD), use the description field to contain the type of detector that will be referenced in the script and other values. ADD is an intriguing Apple technology that allows you to run scripts that respond to contextual menu selections. Chapter 20, Apple Data Detectors Extension, is devoted to ADD.

Unfortunately, only two pages of chapter 20 are available in Google’s online viewer, and Amazon’s preview functionality won’t work with me. With the examples given on the pages that are available, it seems as though one would define detectors externally and ask for them via AppleScript. The detector presumably gives a return value which is treated as any other AppleScript object data. The sample code uses the detector patterns as a way to create custom detector actions.

Infrastructure for Electronic Business on the Internet, p. 105 (Veljko Milutinović; 02001)

5.2.3 The Apple Data Detector

The Apple Data Detector Software Package was shipped as a product first in 01997. Later on, newer and more sophisticated versions were introduced.

The essence of the product is in being able to extract semantics from everyday documents, without asking the user to recreate the documents in a new way. Documents are automatically redefined from a stream of characters, and reformatted so that the specific requests stated by the user become explicitly visible.

In other words, the document is changed so that the implicit goals of the user (ambiguous from the initial document point of view) become explicit (from the modified document point of view). The essence of the success is in the application of sophisticated parsing and recognition techniques, as well as in the application of appropriate response and collaboration methods.

Onwards

I don’t know anything about future OS X releases. Perhaps Snow Leopard will promote this to a publicly-available technology; perhaps we’ll have to wait for a release beyond that.

The technology involved is certainly an interesting one, and could integrate nicely with microformats — for example, automatically detecting hCards and offering to put them in one’s Address Book. I guess we’ll just have to see.

Today, among a number of other things, Apple announced MobileMe — a service based around the idea of personal ‘cloud computing’. In essence, the idea is that any time one updates certain information — email, photos, addressbook, &c. — at one machine, it will be updated almost instantaneously for all the user’s devices.

MobileMe’s foundations come from .Mac, which offered the ability to use Apple’s servers to store and sync personal data. Over the past four years or so, .Mac had come to be seen as outdated and over-priced, with customers hoping each year that an overhaul would be announced, but to no avail.

Hints of interest in cloud storage came with the Macworld 02008 announcements of the ‘Back to My Mac’ service (remote access and syncing) and the ability to push videos between devices. Much of the discussion revolved around the AppleTV and the MacBook Air, but I felt that there was something deeper going on.

The iPhone is really the motivator for much of this direction, in my opinion. MobileMe is generally about easy access and instant propagation, yes, but at the core it’s about being able to use the iPhone as a ultra-mobile computer that can call people. Digital media, contacts, email, data storage, and native applications — everything about the iPhone centers on the idea of the cell phone being subservient to the user.

However: Apple is far from being first with any of this. Cloud computing? Google’s entire application suite depends on it, and offers most of the functionality. Microsoft produce the only significant competitor in the category, Exchange (which, in fact, Apple are licensing to pull in business users). Dropbox are just one of the companies offering instant personal file synchronization. There’s been a fair amount of movement towards cloud computing over the past year, too.

Apple differs from these others due to how completeness of the service. It has Google’s web-based data access; Microsoft’s automatic syncing; Dropbox’s file management; it works equally well with desktop computers, the web, and — most importantly — the iPhone.

I think this is part of why ‘Mac’ was missing from the WWDC banners — the iPhone is a major part of Apple’s plans, and OS Ⅹ is more about users than hardware.