Actually, reader, I’m a little tired of all these search posts. But new things keep happening and this one is compelling enough to note. I really miss Google’s notebook feature (actually a lot of people do). It was like BasKet for the Web. It sounds like Yahoo! is about to launch a new app called Search Pad that will be like Google’s notebook but with a teensy bit of intelligence.

This sounds like a right combination. If the search engine can be intelligent enough to figure out that you’re doing some sort of research and then help you with an easy-to-use note-taking, organizing system, fantastic. But if it becomes even more intelligent and can offer even more useful things than just archiving notes, that would be a powerful assistant.

There is some nice potential here. I wonder if Yahoo! will take advantage or underwhelm. Either way Google please take note, your competitors’ efforts to improve how people use the search results they get are becoming more sophisticated and intelligent. Will Wave make up for the loss of notebook?

“…search engines do a decent job of helping people navigate the Web and find information, but they don’t do a very good job of enabling people to use the information they find… Bing is an important first step forward in our long-term effort to deliver innovations in search that enable people to find information quickly and use the information they’ve found to accomplish tasks and make smart decisions.”

This goes hand-in-hand with the idea I was calling for when I said we need to ask How do I assemble knowledge from the information I find? In Erick Schonfeld’s detailed article on TechCrunch, he shows a number of examples of how this better guided search, with a decision-oriented result can work. It seems Bing will provide a lot of smart contextual information around searches to better identify what people are looking for and deliver results that a person will find useful for taking some sort of action.

I said before that we lack intelligent authors to assemble a structured knowledge for acquisition from the content of the Internet. The search engine should morph or augment itself into something that will provide that or come close to providing that (or else be usurped by an alternate application).

Microsoft is making a lot of a statements about changing the search game. Considering the launch of Bing occurs in such close proximity to the launch of Wolfram|Alpha, I think it’s fair to say that there is a real chance people are recognizing the need to change the search game in this direction. Wolfram|Alpha calls itself a computational knowledge engine and makes a point of differentiating itself from search engines. Wolfram|Alpha explains that

“You enter your question or calculation, and Wolfram|Alpha uses its built-in algorithms and growing collection of data to compute the answer. Based on a new kind of knowledge-based computing…”

From what I can see in Microsoft’s Bing and Wolfram|Alpha, neither have fully accomplished the game-change that is needed. However they’re each on the path. It’s clear that Google has a wealth of apps and interconnected information, which they’re subtly making visible and more-and-more intelligent over time. If you spend any time flipping back and forth between Google’s variety of apps, you start to see some interesting connections that overlay search results. Rather than launch an entirely new system, I think Google is evolving itself in this direction. I like that Bing and Wolfram|Alpha have become quite public, I think it will push Google more and it’ll be fun to see how these companies finally deliver on CASAA.

Some of the article meshes in interesting ways with what I was thinking about in my last few posts. I enjoyed reading what he had to say, so I felt like commenting on it. For example,

“And just as the Web once emerged on top of the Internet, now something new is emerging on top of the Web: I call this the Stream. The Stream is what the Web is thinking and doing, right now. It’s our collective stream of consciousness.”

It does feel intuitively right to me that something will emerge on top of the Web but I don’t really think the Web emerged on top of the Internet in an equivalent way. I think of the Web as a conscientiously designed markup language, protocols, and interfaces (browsers) that “resided” on the Internet as their medium. The result of all that was the huge variety we see today. Whereas the stream, also residing on the Internet medium, does not have an analogue in terms of the Web programming I just mentioned. The stream seems to be a lot of different applications or meta-applications that serves various purposes. Some use common protocols or other standards but many might also be considered rivulets within larger streams. So if we want to say there is a stream as something emerging on top of the Web, I think we have to envision it wholly differently than the Web that emerged on top of the Internet. Perhaps it would be more appropriate to think of the stream is an emergent property of the Web?

I’ve been dissecting this stream idea without even saying what it is. Spivack calls out an example in microblogging, of sites like Twitter he says “…they are literally streams of thinking and conversation…” And later he points out that “The Stream is a world of even shorter attention spans, online viral sensations, instant fame, sudden trends, and intense volatility. It is also a world of extremly short-term conversations and thinking.”

That may touch on the same phenomenon I was thinking about when I claimed that people now have short attention spans by necessity. To acquire knowledge people have to pull together many shallow atoms. Spivack’s article recognizes a problem with inexorable onslaught of the stream. So many streams of information, constantly streaming away, how can we cope? The answer seems to be that we’ll need tools to aggregate, filter, and manage our streams for us. [update 14 May: after posting this I read a great post on the Emergent Chaos blog about this very issue... twitter bankruptcy]

But I wonder, we’ve had “constant now” technology for a long time in the form of a telephone. Phone conversations are essentially immediate. There are reasons that we do not stay constantly connected with the now of the phone. What is compelling us to think that we should attempt that with the Web? Just because the Internet is accessible constantly, the Web is updateable immediately, and a computer device can open access to innumerable streams, do I want to be exposed and engaged with that? Increasingly, people seem to be saying yes. I don’t know that that will continue but this is worth exploring more. Nevertheless, because we can transmit information so rapidly I don’t think it means we necessarily will start focusing on the Web for what is happening as opposed to what happened. It may be the case that all of this access, these streams are valuable to us because they keep us rapidly up-to-date with what just happened not what is happening.

“Just as the Web is not any one particular site or service, the Stream is not any one site or service — it’s the collective movement that is taking place across them all.”

I get why it’s so tempting to compare the Stream to streams of consciousness. On the whole, you seem to get a constantly flowing array of stuff with all sorts of lateral connections. Maybe if we place ourselves way above the Internet, and think of it as a mind, these things taken as a whole, would indeed seem fluid like a stream of consciousness. But if we were to focus any one particular person’s Twitter stream for example, it’s much more like an archive than a fluid stream. People select bits of their thoughts and convey them to Twitter, where they appear and remain. Spivack recognizes this because he goes on to discuss the linear nature of most streamlike services. He also states that

“The transition from a slow Web to a fast-moving Stream is happening quickly. And as this happens we are shifting our attention from the past to the present, and our ‘now’ is getting shorter.”

What’s the value in an RSS feed or following a microblog? I think it may be, at least in part, the fact that it is conveying, not the now, but a constant archive of the now. It’s value is that we can look on what has passed, very easily, in a sequential or somehow ordered context and still communicate with it in our own now. Perhaps what we want from the stream is not to be engaged in the now but to be engaged at our own command, with the recently passed. That’s all very unlike a telephone. In this, I don’t see a fluid now like a stream of conscious, but something more like a conveyor of discrete selections.

Let me delineate a few parameters. I do not intend to make a value judgement on any method of acquiring knowledge. My main interest is to examine current changes in our popular methods of knowledge acquisition and what these mean for our understanding. I’m not thinking of experts in a domain of research (though there are some interesting challenges). Rather, I’m considering the everyday person that wants to learn something substantial about a subject or issue in his or her world.

Prior to the Internet if someone wanted to learn about a subject to any non-superficial degree, one would probably read a book. Documentary films could also provide some solid depth of information (or sometimes television shows, though these may also move us away from the deeper end of the continuum). Each of these examples is a single, self-contained, deep unit covering the necessary amount of information for the layperson to learn a satisfactory bulk of what he needs on the desired subject.

I call something a deep unit if it is a self-contained gathering of information related to a specific subject. A book that explains the history of Canada, a documentary on the social structure of ant colonies, a television magazine that devotes an episode to the unsafe build quality of a famous auto manufacturer, these are all examples of units about a topic that, anyone familiar with these mediums will recognize provide a well-focused gathering of information structured in a way that the person apprehending the media will be able to learn a relatively deep amount of knowledge about their subjects.

In contrast to the deep unit, consider the shallow atom. I call something a shallow atom when it provides a, discrete quantity of information in a concise scope. Consider an online encyclopedia (for example Wikipedia) entry about Canada, while it will provide some history it will not be sufficient in itself to provide the deeper knowledge the book does.

Consider a scientist’s blog talking about her day-to-day research insights on her observations of ant colonies. Each individual blog post will provide some information but it won’t provide the wholistic depth of a documentary on the subject. Consider, a consumer web site that allows its visitors to vote on the quality of different cars, it may indicate a low rating for a particular car but it won’t identify the systematic administrative cover-ups of safety violations the manufacturer engaged in, it won’t reveal tests confirming the safety problems. These are all examples of shallow atoms.

I would like to emphasize that although I’ve set up what seems to be a comparison between deep units and shallow atoms, it is not a comparison of the value or capacity. Rather I want to identify distinguishing examples. If my theory is right it won’t be because a shallow atom is equivalent or better than a deep unit but because a great, enmeshed quantity of shallow atoms can provide equivalent knowledge as an individual deep unit.

Design and Assembly

There is both a significant qualitative difference and a difference in apprehension, in virtue of the design and assembly of deep units versus shallow atoms. The deep unit is designed by one or more individuals to have a certain continuity of structure. Its entire quantity of information is conveyed via a predesigned (meaning not left to the person acquiring it) conceptual skeleton. The shallow atom, on the other hand is but a bone within the conceptual skeleton and requires that the person acquiring it design his or her own conceptual skeleton for assembling the required information.

To return to my example of the documentary piece on the subject of ant colonies’ social structures. A team of people no doubt developed a well-defined and edited structure for how they’d convey the information to their audience. The documentary would likely have some sort of introduction teasing peoples’ interests as well as providing an overview of the show’s topic. Later the documentary would go further and further in-depth about the issue, probably interviewing different researchers, explaining scientific analyses, connecting commentary to provide a more extensive context to the point. It may discuss parallels with human behaviour or technological innovations, and ultimately it would wrap up with a conclusion and some sort of summary of the its contents. The point is, there is a directed message in the deep unit that those producing it attempt to convey.

Compare that with the scientist publishing a blog about her research on ant colonies. She may post daily, a new observation she’s made while studying her colony. One post might identify a certain behaviour between worker ants and their queen. Another post might discuss the food storage strategy of the colony. These posts would continue for so long as she’s doing her research. It might be tempting to argue that the entire blog, covering the topic of ant colony research is a deep unit but I don’t think it is. Aside from the overarching topic of the blog, it is not a unit assembled along a structure that has been defined to convey a unit of knowledge. In fact, if it were, it would lose its value as a blog.

Each blog post is a more-or-less self-contained entity-an atom. The blog post contains usually, a bit of information about something which encourages people to comment on it or link to it (also a form of interactive commentary). Blogs are designed to encourage conversation. If the blog post was instead a whole book, the threshold for commentary would be much greater. That is, to comment, people would have to invest much more time and effort to read it. However, the blog post’s appeal and success at developing conversation lies in large part with the fact that it has a relatively low threshold to acquire. Someone can read it without fear that he or she is not getting the full picture by not reading the rest of the blog posts. That enables each post to be commented on, as its own discrete unit. It also means each post is only likely to deliver a small portion of information, so if someone wants to acquire knowledge, say about ants, he’ll only get a bit from the blog post. He’ll have to follow its links to videos of the colony, other scholarly publications, perhaps a government-funded research site from some other country, to round out the full scope of knowledge he wants to acquire. Of course, before the widespread existence of all these information media that scope of knowledge would likely have come from a deep unit, like a book.

But what does that mean to the person acquiring knowledge? It means he or she has a lot of additional work to do. There is no one author or group that has designed and assembled the appropriate information into a deep unit ready for knowledge acquisition. I can easily think of a number of tasks the knowledge acquirer will have to do, there are probably many others.

1. must validate the trustworthiness of each atom
2. must decide upon the utility of each atom
3. must select which atoms to pursue for further related information
4. must seek the right sources to use for discovering atoms

What about the message of the deep unit’s author? Is that another thing that the knowledge acquirer must take upon him or herself to develop? I’d argue that any person wanting to acquire some knowledge has a responsibility to do some critical thinking, analysis, synthesis, etc. regardless of whether information comes in the form of a deep unit or a shallow atom. Still, I feel that in the case of the deep unit, the author’s design and build, implicit with its message, is an added value in virtue of the perspective provided by the author. It might also be argued that that muddies the waters for the knowledge acquirer. Now, with all the new methods for communication we’ve developed, that perspective might also be obtained through later discussion.

In any case, the design and assembly of the knowledge acquired through shallow atoms is a very different thing (structure, quality, onus of labour, etc.) from that acquired through deep units.

Have you noticed how those belonging to the generation labeled “millenials” are often accused of having short attention spans? They’re always connected for social and information acquiring reasons. They use mobile phones, Web search engines, type short frequent text messages, maintain instant messenger windows on their computer screens, worship metadata (though not always consciously), use blogs, microblogs, social networks, view short video clips online as opposed to television, mix music in collage fashion from small clips of other sounds, etc. The items I just listed provide instantaneous access to shallow atoms of knowledge.

If we think of how the millenials acquire knowledge with these media, then they have short attention spans by necessity. In order to acquire the knowledge they seek on a subject, they have to pull together many shallow atoms.

Imagining ways of knowledge acquisition along a continuum from the very deep, dense sources to the manifold shallow, smaller sources reveals that people not only are taking advantage of newer technologies and media but also are changing their behaviours in response.

I have more to post about this subject in the next little bit. The implications of how we’re changing our methods for acquiring knowledge extend to information archival, opinion making, our responses to news, certain social interactions, and as I’d like to talk about in another post, the next most important change for the Internet. Search engines accomplished their job. And they created a big problem in helping us find all the shallow atoms we want. But there is an incredible, unexploited opportunity for search engines to evolve into something new, solving the problem they created.