More about Lillian Schwartz.

Media companies won’t move forward. They’re being left behind.

Verizon and RIM, determined to release the Storm in time for the holidays, rushed the device to market despite glitches in the stability of the phone’s operating system, according to people close to the launch.

RIM co-Chief Executive Jim Balsillie said the companies made the crucial Black Friday deadline by the skin of their teeth, after missing a planned October debut. Mr. Balsillie said such scrambles — and the subsequent software glitches that need to be fixed — are part of the “new reality” of making complex cellphones in large volumes.

Verizon and RIM provided a software update for the Storm in December that resolved some user concerns.

I guess the alternative of ‘not getting into a situation where we’re scrambling’ didn’t go over so well with investors.

David Bennahum, 01995:

Thanks to the microprocessor things get so much more powerful so quickly that there is a real fear of not being “backwards compatible”, meaning the files from 01982 should still somehow run in the word-processor from 01982, even if it is 01995. This kind of uncertainty scares people with really big pockets, people who have thousands of employees and can spend thousands on one order. From day one those people were just *not* going to spend money on a company called Apple best known for a machine built in a garage by two hackers, a machine that kids loved because of the video games. We seem to forget those days.

A little bit of history.

In 01981 IBM launched its first personal computer, the aptly named Personal Computer. Back then there was only one kind of personal computer — the video game machine. You had three choices: Atari, Apple and Commodore. Weirdoes who liked monochrome bought TRS-80s (aka the “trash eighty”) from Radio Shack with an audio-cassette tape drive. In this kind of universe, you’re Apple and you’re thinking “how are we going to convince companies that a personal computer belongs in their offices?” The answer is, you’re not. No way. Not Apple. Not Atari. Not anyone but IBM is ever going to convince corporate America to buy this thing called a PC. So Apple, rightfully, walked away from a market they never were a contender in; and from 01981 to 01994 the office market drove the expansion of personal computer industry.

…

The Pippin is part of a project with Bandai, a Japanese game company known for its effluvium of Saturday morning cartoons, video games, and other brain candy for kids. The logic goes like this. As computers become household commodities, like consumer electronics, Apple stands heads above the other PC makers as the only one with any concept of design, marketing and the shopper’s impulse — the kind of stuff Walkmen, Swatches and GameBoys are made of. With the increasing popularity of the Internet, the demand for cheap household computers will dovetail nicely with the demand for ever more immersive video-game machines. Who else but Apple has the skill to enter this market? Who? If Apple is lucky they will forge an alliance with Sony and build these $300 hybrid PC/video-game/Internet gateways together. Could you imagine how many they would sell at Christmas? Tons.

Just a quick one adapted from a few of my tweets (1, 2, 3).

Dan Hanttula, 01999:

Furthermore, a business teacher once told me, “you don’t get investment funding unless you’re taking a company public, or planning to be acquired.”

The first internet bubble burst hard, but everybody was happy to get right back on for the second bubble (roughly 02005–02008) and throw around just as much money on companies that clearly didn’t deserve more than a ten dollar bill. As a result of this craziness of giving out funding to everybody, it looks like venture capital might fall in on itself.

Now we just need to wait for the third round to hit so we can ride the carousel again…

I tend to forget how fortunate we are that the Internet is largely decentralized, free from the control of organizations that would force us to pay for the right to do anything.

There are still plenty of walled gardens, but over time they open or crumble.

Vannevar Bush’s famous essay predicting the course of technology in the twentieth century.

His section on the memex contains a paragraph that serves as an effective mission statement for many of the sites that inspire me, and one I hope to prove worthy of:

The historian, with a vast chronological account of a people, parallels it with a skip trail which stops only on the salient items, and can follow at any time contemporary trails which lead him all over civilization at a particular epoch. There is a new profession of trail blazers, those who find delight in the task of establishing useful trails through the enormous mass of the common record. The inheritance from the master becomes, not only his additions to the world’s record, but for his disciples the entire scaffolding by which they were erected.

Today I’ve been looking for information about the origins of Apple’s ‘data detector’ technology. Although it was available in some form starting in the late 90s, it seems to have disappeared at some point, resurfacing with the release of Leopard. (See also.)

In Leopard, there’s a private (i.e. Apple-only) framework, which can be found at /System/Library/PrivateFrameworks/DataDetectorsCore.framework/. The detectors are in plain-text format, and can be seen in the Resources/Patterns subdirectory.

The system seems to have a loose library structure. There is a common.ddn file that defines keywords in a manner similar to POSIX’s character classes; this file is often imported into more specific detectors. There also appears to be some minor similarity to the syntax of Pascal, but I’m not a programmer so don’t assume this means anything.

Anyhow, here are some historic mentions of the subject I’ve found, sorted roughly by publication date:

Collaborative, programmable intelligent agents (Bonnie Nardi, James Miller, David Wright; 01998)

This is presumably the first public mention of the technology Update: it is not; see here, and the paper was published in Communications of the ACM in March of 1998.

It seems that even from the start, the system was intended to be a plugin architecture — there would perhaps be some included detectors, but users could easily create their own detectors with the help of a fairly simple syntax that loosely resembles regular expressions. The present syntax seems to have a generally similar format, although it may be complexified somewhat for greater power.

The goal of our research on intelligent agents was to create something useful for our customers, but something with that sprinkling of pixie dust that would make it seem “intelligent.” Ben Shneiderman observed that claims about intelligent software agents are vague, dreamy and unrealized. We started from a simple but focused approach to agents, that they should have the ability to infer appropriate high-level goals from user actions and requests, and take action to achieve those goals. Further, based on a study of reference librarians as exemplary human agents we wanted to build a system in which the user would not have to state goals explicitly and in detail — we learned from librarians that a large part of the value they provide to clients is in working with imprecise requests. Beyond this, our general design strategy was to keep the user’s question in front of us at all times: Will this software do something useful for me, in an intelligent way that makes me more productive? The system we describe here — Apple Data Detectors — meets our criteria of being unobtrusive, having the ability to infer user needs, and doing useful work. Apple Data Detectors will ship as a product in 01997.

…

Past work on intelligent agents has been multi-faceted, to the point where it is difficult to find consensus on exactly what constitutes an agent. Researchers have used machine learning techniques to track user actions and construct models of user preferences, created agents that employ user models, consulting a set of parameters that describe the user, implemented planning systems to make the leap from a user’s stated intention to the specific actions that are required to achieve that intention, and built agents that act as “eager assistants”. The locality of agents also varies across different agent-based systems: some act only within one’s own machine, while others autonomously crawl the Web, searching for interesting content, for example. Apple Data Detectors works on the user’s own machine, and falls into the eager assistant category, enabling rapid user action with minimal input on the part of the user.

…

In an investigation of how people file information on their computer-based “desktops”, we discovered that a common complaint of users is that they cannot easily take action on the structured information found in everyday documents. By structured information we mean data recognizable by a grammar. Ordinary documents are full of such structured information: phone numbers, fax numbers, street addresses, email addresses, email signatures, abstracts, tables of contents, lists of references, tables, figures, captions, meeting announcements, Web addresses, and so forth. In addition, there are countless domain-specific structures such as ISBN numbers, stock symbols, chemical structures, mathematical equations and so forth. These structures are not only relevant to users, but are also recognizable by present day parsing technologies. The type of a structure can be used to identify appropriate actions that might be carried out on the structure — place a meeting on a calendar, add an address to an address book, dial a phone number, open a URL, find the current price of a stock, file an ISBN number, compile a list of abstracts, and so forth. The system we developed to enable people to work more fluidly with structured information is called Apple Data Detectors.

I’ve put together a comparison shot of the data detectors being used on an email message. The top capture demonstrates the system in use in 01997, and is included with the article; the lower one demonstrates the system as it appears in Leopard.

The plan included the ability to build on other detectors, which created something of a dependency jumble:

The solution for shared, compatible detectors requires developers to register their detectors. Our registry is supported by Component Integration Labs, a company responsible for maintaining the Apple Event Suites (among other standards). Apple Data Detectors will make use of the registry which contains definitions for classes of data objects and for events that operate on the objects. Classes of detectors will be defined as data objects and developers must write detectors that detect required fields of the detector class. Detectors can detect more information than the class requires, but they must detect at least the data that the class requires.

This may still be an issue as a result of the import command; I don’t know what happens if a detector attempts to pull an external detector that doesn’t exist — and because this is a private framework there’s no documentation available other than the included patterns.

This article was briefly mentioned in a paper on teaching machines how to understand human language:

Recognizing textual entities

[…] These entities may be detected using various techniques. Regular expressions and pattern matching are often used. Apple Data Detectors (Nardi, Miller, & Wright, 01998) use context-free grammars.

AppleScript in a Nutshell, pp. 35 & 36 (Bruce Perry; 02001)

Script Editor Controls/Commands

[…] In addition, when creating a script to use an Apple Data Detector (ADD), use the description field to contain the type of detector that will be referenced in the script and other values. ADD is an intriguing Apple technology that allows you to run scripts that respond to contextual menu selections. Chapter 20, Apple Data Detectors Extension, is devoted to ADD.

Unfortunately, only two pages of chapter 20 are available in Google’s online viewer, and Amazon’s preview functionality won’t work with me. With the examples given on the pages that are available, it seems as though one would define detectors externally and ask for them via AppleScript. The detector presumably gives a return value which is treated as any other AppleScript object data. The sample code uses the detector patterns as a way to create custom detector actions.

Infrastructure for Electronic Business on the Internet, p. 105 (Veljko Milutinović; 02001)

5.2.3 The Apple Data Detector

The Apple Data Detector Software Package was shipped as a product first in 01997. Later on, newer and more sophisticated versions were introduced.

The essence of the product is in being able to extract semantics from everyday documents, without asking the user to recreate the documents in a new way. Documents are automatically redefined from a stream of characters, and reformatted so that the specific requests stated by the user become explicitly visible.

In other words, the document is changed so that the implicit goals of the user (ambiguous from the initial document point of view) become explicit (from the modified document point of view). The essence of the success is in the application of sophisticated parsing and recognition techniques, as well as in the application of appropriate response and collaboration methods.

Onwards

I don’t know anything about future OS X releases. Perhaps Snow Leopard will promote this to a publicly-available technology; perhaps we’ll have to wait for a release beyond that.

The technology involved is certainly an interesting one, and could integrate nicely with microformats — for example, automatically detecting hCards and offering to put them in one’s Address Book. I guess we’ll just have to see.