The Search and Marketing SIGs held their first joint meeting on September 9, 2010 at the Mountain View offices of LinkedIn, the new host of the Search SIG. Y.Y. Lee, the COO of FirstRain, spoke on the transformation of search from the traditional keyword-based search that returns document sets to a more powerful subject-based search that connects the dots scattered through millions of documents to return actionable intelligence.
Referring to John Battelle’s description of the web as a database of intentions, Lee stated that “there is massive intelligence latent in the web,” though widely dispersed and all but hidden by a high noise to signal ratio. Organizing this amorphous data into a structured data set capable of answering complex business questions, Lee explained, requires a search engine that understands the concept of a company as something more than a textual string as well as the relationships between corporations and their subsidiaries, product lines, and competitors.
In addition to this conceptual understanding, Lee said, the search tool must understand time, enabling it to capture trends, clusters of events, and anomalies, uncovering opportunities as they emerge. Business search services, such as that provided by FirstRain, process millions of documents a day, effectively time sequencing the web.
According to Lee, the technology of business search begins with crawl, as does consumer search, but it processes retrieved content using linguistic fingerprinting, statistical techniques, and heuristics, structuring the data into a complex dataset that models business relationships. And business search goes beyond keywords, which are frustratingly ambiguous, to query using a rich taxonomy of business concepts.
Business search, Lee explained, lets business users ask questions such as what new competitors are emerging, what are the concerns of my customer’s customers, and what changes are afoot in a company that I do business with. Citing an example, Lee told of how a user spotted unusual management changes at a major business partner, changes that tipped him off to a reorganization and enabled him to capitalize on the change.
Responding to audience questions, Lee said that FirstRain sells its services through a subscription model. FirstRain first sold to financial firms, where analysts accustomed to powerful tools for analyzing quantitative data welcomed a tool that would get them beyond Google and the telephone for collecting and analyzing qualitative data. More recently, Lee said, FirstRain began selling into corporate sales and marketing departments.
The field of search, Lee said, has not seen significant innovation since the failure of several Google competitors. We have accepted search as nothing more than keywords and document sets. Business search, concluded Lee, represents a rejuvenation of the field.
Jim Downey 