The Coming Age of the Context Engine

People talk a lot about information overload, but I think the worst problem we have in information management today is memory overload -- the inability of the human brain to retain all the important information we run into in our careers.  There's more stuff we need to remember than you can possibly hold in your head.  The more successful you are, the more information you need to remember -- and the worse the problem becomes.

I think what we need is a context engine, an app that helps you recall the context around any bit of information in your life.  Unlike a search engine, a context engine indexes just the information in your life, and supplements your own memory.  "How do I know this person?"  "What's the agenda for my next meeting?"  "Who sent me that article last year, and where the heck is the article?"  A context engine will help you answer these questions quickly, anytime and anywhere you need the information. 

The product that I'm working on, Zekira, is a first generation context engine.  In this post I'll discuss the need for a context engine, how it would work, and our status with Zekira.  I'll also give some information on how you can help, if you're interested.

The trouble with information overload

Information overload is a hot topic with a long history.  The term was coined in the 1960s, and popularized in 1970 by Alvin Toffler's book Future Shock, according to an excellent article in Wikipedia (link).  But the idea goes back further.  Xerox implies that it invented information sharing through the development of the photocopier in the mid-1900s (link), And there have apparently been complaints about too much information for as long as we've had writing.  The Bible complains about the proliferation of books, the Romans worried about it, and so did the ancient Chinese.  Once Gutenberg got going with movable type, the complaints increased (link).

Information overload is a popular subject online.  The Wall Street Journal said Google returned 2.92 million hits for it in 2009; the same search today returns 3.76 million, an increase of about 770 references per day.

Prominent authorities opinig on the hazards of information overload include the New York Times (link), Wired (link), and none other than the big consultancy McKinsey, which says it is "killing productivity...and making us unhappy" (link).

The critics of information overload complain that it bombards people with so much data that they are stunned into stupidity.  They become low-grade data zombies, incapable of making intelligent decisions.

The answer, we're told, is to take in less information.  The experts tell us to delete e-mails and limit our exposure to information online so that we can reserve time for thinking deep thoughts and forming long-term memories.

Okay.  It makes sense that we should set aside time to think.  But I believe distraction isn't a function of how much information you bring in, it's a function of how much self-discipline you lack.  There's always something you can distract yourself with; if it's not e-mail it'll be Angry Birds.  You could have the same problem if you had five e-mails a day or five hundred.

I think blaming "information overload" for the problem of distracted people is like blaming "water overload" for the problem of drowning.  The fact is, modern society runs on the flow of information.  The more information you can handle, the more productive you'll be, and the further you'll go in your career.  Given the way the economy works, telling people to limit their information flow is a little like telling them to make themselves stupid.  Instead, I think, we should be increasing our ability to manage that information, so we can be more productive.

The real problem is memory overload

Once you step back from demonizing information itself, it's easier to identify the problems that we have in dealing with so much information.  I think the biggest problem in information management today is the limitation of human memory. 

Think about how you remember things.  It's usually through associations -- I saw it in the newspaper when I was at that cafe, I read it in that article on The Register while I was riding the bus, etc.  When people have more information to remember than their brains can hold onto, those chains of association start to break down.  You remember the fact that you once knew something, but can't recall the information itself.

As I talk with busy knowledge workers -- the type of people who manage the most information -- I hear stories about half-remembered information all the time.  You'll see a person and know that you've met them, but can't recall the details about how you know them or what you discussed with them.  Or a topic will come up and you'll remember that you read something important about it, but you won't recall where you saw it or how you could find that information again. 

Often you know the information is stored somewhere on a computer or smartphone or website, but you have no way to look for it in the moment you need it.  Even if you remember to look up the information later, it's usually extremely hard to find, and you can't take the time to do it. 

The more successful you become in your career, the more information you have, and the more overloaded your memory gets.  Of course it eventually overflows.  The problem is so ubiquitous that most of us don't even think of it as a problem; it's just a feature of life.  We shrug it off as a "senior moment" and uneasily move on.

But it has nothing to do with age; it's a function of experience.  Take all the information held by a mid-career professional and stuff it into a 20-year-old's head and he or she will have the same problems. 

When you add up all those "senior moments" across all the people they happen to, they constitute a huge loss in productivity among the busiest and most pivotal people in the economy.  The only reason we tolerate this situation is because we assume there's nothing we can do about it. 

But I think we can.  The combination of mobile technology, low-cost computer storage, and web services makes it possible to build what I call a context engine -- an app designed specifically to help you recall the information in your life, and all the context around it.

You'll use a context engine to quickly recall:
    -All the details of your relationship with someone -- how you met them, messages and documents you've exchanged, and meetings you've been in together.
    -The backstory to a meeting, including the messages that led up to it, attendees, notes and pictures you took during the meeting, and followup messages afterward.
    -A tweet or Facebook post or e-mail you saw months ago mentioning a great new restaurant that you want to try.
    -That report sent to you five years ago by some guy you met at a half-remembered conference in Boston.

How the context engine will work

A context engine needs to do three things with your information:  Collect, connect, and communicate.

1. Collect.  To build a map of all your information, the context engine needs to gather it from all the places where your information is stored.  That means, first, scanning the hard drives and other storage devices connected to your personal computer.  E-mails, contacts, and meeting records all need to be extracted from whatever messaging and calendar system you use.  For most mid-career professionals, that means digging into old Microsoft Outlook archives, called PST files.  Other documents -- especially presentations and word processing files -- need to be sucked in as well, along with the most ubiquitous file format in business and academics, the PDF.

But you can't stop with the PC.  The context engine needs to reach out to your web apps, to extract things like gMail messages, tweets, and Facebook posts and contacts.  And the information on your smartphone needs to be included, everything from contacts to text messages to pictures.

This process should be automatic and comprehensive.  Everything should be indexed.  You shouldn't be asked to choose which files you want to remember, because you can't know in advance what you'll need.

2.  Connect.  Once all that information has been collected, it must be organized.  That means indexing it not just by keywords, the way we would for a traditional web search, but by all of its attributes, including date, time, location, type of content, and so on.

This is a key difference between a web search engine and a context engine.  In web searches, we look almost exclusively for keywords, and we use the wisdom of crowds to determine which matches are most important.  That works great for searches of publicly-available content, but it breaks down when searching your personal archive.  You may not remember the right keyword for a document or message, and the wisdom of crowds is much less useful for ranking results, because everyone's context is unique.  Instead, a context engine needs to offer many search paths through the archive, so people can search using whatever bits of information they do remember about a topic.

The context engine should also present information to you in a way that lets you jump between bits of related data.  Say you're looking for the record of a lunch meeting.  You might be looking for it because you want to find the name of the person you met with, or some messages you exchanged with that person.  Or maybe you just want the name of the restaurant so you can eat there again.  All of that information needs to be cued up so you can jump to it easily.  Again, the goal is to help you re-create those half-remembered chains of association. 

Many of the products that in the past have tried to organize personal information (such as Google Desktop) have mimicked the keyword-centric searching we do on the web.  Web search is so ubiquitous that we're all a bit like the man with the proverbial hammer -- every problem looks like a nail.  But I think personal context requires a radically different structure to the database and UI.  It's not about searching for things, it's about navigating through your context.

3.  Communicate.  You don't know when you'll need to remember something, so the context engine needs to be available on your mobile devices.  In particular, I think a context engine is a killer app for tablets in business.  Imagine always having your entire information history at your fingertips in every meeting and every conversation.  How much more productive could you be if you had a perfect memory all day long?

I can't tell you how many people in Silicon Valley have told me sheepishly that they don't know what to do with their iPads at work.  They generally love them at home, where they access entertainment and informational content.  But at the office, particularly in meetings, they tend to turn into tools for covertly checking messages and browsing when the meeting gets slow.  Please don't misunderstand, I know there are many things you can do with an iPad.  But I'm reporting what I hear from a lot of iPad users.

Far be it from me to judge the way others fill their time, but I think the context engine would give you a good business reason to carry your tablet all day.

That means the database needs to be hosted in the cloud, which creates all sorts of important security challenges.  Having your extended memory hacked is utterly unacceptable.

Building the context engine

As you know if you've been following my earlier updates, the startup that I'm working on, Zekira, is building a context engine.  The company consists of four engineers plus myself, and we've been working on it for more than a year.  Zekira is the fulfillment of a dream for us.  One of us, Rudi Diezmann, has been working on personal search products since the 1980s.  Others of us first thought about this problem when we were working at Palm almost ten years ago.  We were looking at user problems a PDA or smartphone could solve, beyond helping you manage your calendar and contacts.  There was a group of customers who responded very strongly to any product that could help them recall information and the context around it.

But only recently have mobile computers and wireless networks become powerful enough to let you build a full-function context engine. 

The first version of Zekira is in early beta.  It runs on Macs and PCs, and right now it indexes information found on your computer and any storage attached to it.  Our goal is to take Zekira mobile, and to add web data sources, as soon as possible.  But we did the first version on personal computers so we could get started testing the database and search capability.  Besides, there are a lot of people with old Outlook and Office archives who would be happy to turn a context engine loose on them.

Zekira gives you a little search window that you can leave up on the screen, or minimize: 

After you do a search, your results appear in this window:

 The four stacks in the center show you all the items that matched your current search.  In this case, we're seeing things related to Tom Shannon, including documents that he wrote or that mention his name, messages you've exchanged with him, and his contact record.  Click on any of those items and you'll see information related to them.

The tabs on the left are filters that let you narrow the search.  Currently they let you search by time/date (the filter shown), name, word, document type, and folder: 

 You can combine multiple filters to do complex layered searches.  The filters are extensible, and we plan to add additional search tools in the future.

We're doing a crowdfunding campaign for Zekira on the funding website Indiegogo.  If you don't know how crowdfunding works, people can make small financial contributions to a project and receive benefits in return, such as a discounted copy of the program when it's finished.  Supporters of Zekira can also get access to the beta version of the program, and listing as a sponsor in the about box of the finished app. 

Corporate sponsors of Zekira can get advertising here on Mobile Opportunity, a unique offer since I don't generally accept ads (except for one tiny Google ad that gets me access to Google's excellent traffic monitoring tools).  The advertising sponsorship offer is a great way for a company that has a little bit of advertising budget left at the end of the quarter to help itself, and also help support a great product.  The ad offer is limited to three companies, and is first-come, first served.

If you'd like to learn more about Zekira, you can visit our crowdfunding site here, and our website here.  And here's a video of Zekira in action:

If you have feedback and suggestions for Zekira, I'd welcome your comments.  And if you like the idea, please help spread the word about our crowdfunding campaign.  The more support we get, the faster we can move on the project.

No matter what you think of Zekira, I hope you'll agree that the time is right for a context engine.  With that and an info pad, I'd be one very happy camper.


Chan said...

Bravo Mace! Someone is really sweating on this, the nightmare I engulf with!

I remember Google Desktop tried to be this without much of a go, mainly it wasn't the time. I suspect Google can do it with Android at any given moment. But searching context is a daunting task if you really don't get the security aspect resolved.

iPhones and latest gen mobiles do solved this issue somewhat. At least for emails and photos.

But when I think of me, myself and stuff about me scattered all across the board.

1. In some physical files, albums
2. Flopp,Omega Zip files (Now lost, unreadable)
3. CD and DVDs
4. Old Hard-disks
5. Redundant/ forgotten old email accounts
6. OutlookExpress Folders
7. Propriety formats some I can't even open now (texnotes, , DAT, PDF, Pagemaker)
8. Among friends and colleagues
9. Completely forgotten where there were

It's truly a nightmare but yours like tools will emerge the next generation at least will be better off

keep it up!

BillSeitz said...

I assume you're familiar with Gelernter's old LifeStreams/ScopeWare product...

uno said...

include zip files for sure. They are nightmare to keep track

Brian Hall said...

Michael -- given the domain space, what do you think of the approach Evernote Hello has taken to the problem?

Michael Mace said...

Thanks for the suggestions, everybody! Please keep it up.

Brian, I think Evernote Hello is really intriguing. It looks beautiful. I might nitpick bits of their approach, but actually because context management is a new space, I think there's a lot of room for a variety of approaches, and customers will get to pick the ones that work best for them. Evernote, Xobni, X1, Gist, and a ton of other companies are all working on different pieces of the problem.

I think one of our goals should be to try to make those products work well together. So what I really want to do is integrate our stuff with Evernote. We'll be exposing APIs in our product over time, I know Evernote already does, and I hope other folks will do so as well.

TDC_123 said...

How hard would it be to integrate voice commands, like with SIRI, S Voice, Vlingo etc. ? that would make this awesome!

mallwitt said...

Fantastic Concept, but (and I know this is nitpicking for beta/alpha), your UI needs work.

Might I suggest a construct more akin to a spiderweb with dew drops? Put your search terms in the middle, with linked context items outside in a circle? Represent higher context items by putting them closer in to the center. have simple icons to represent classes of information. Click/drag to add context items to the search.

As an example I start with my client's name because I want to know how why another client brought them up, I get that in the center with a 'web' of items representing other items contextual proximity, one of those is labeled 'meetings' I click on that, and suddenly that zooms to the center and now what I'm seeing is the client's meetings and on the outer edge, any meetings related to the client - bingo, I found a cross-over lunch meeting with both clients, and the common project they were working on, now I can look at that project and what other clients are connected.

Let me know if that isn't the way that you've been envisioning using a context engine, but to me, it's sort of like wikipedia in reverse (instead of ending up at invasive species, you start there and wind up at which clients caught the biggest fish at the conference).

I'm looking forward to something that will help people understand why I associate things with them! Cheers!

Michael Mace said...

Hi, Mallwitt.

I don't mind you nitpicking the UI at all. This is the time to do it, and I really appreciate the help.

We've experimented with a huge number of different UI approaches, and there are problems with every one of them. We played with fans, spinning disks, webs, flying cityscapes, and linked blobs (like Google Image Swirl, if you've seen it). The problem is that the good-looking designs break down when you have more than about eight connections to something. You run out of screen space.

Although the lists we use are not aesthetically pleasing, they are extremely functional, because they maximize the number of connections you can see, and we think that's the key to productivity.

But I still want to find something that works well and looks good, so I'll think more about your suggestion, and please keep it up.

mallwitt said...

Ooo, I get that, I suppose I hadn't considered just how cluttered things could get. I suppose it matters more if you are looking for tangential information.

How about a sort of zoom in/out functionality? You've got your center blob/swirl thing that all of your search terms go into, and then other items outside of it in their own swirl blob - in fact, maybe something like a molecule diagram, with the vertecies representing the information 9and you hover over them to see all the detail, with thinner and thicker lines connecting them to represent greater or lessor contextual links/probability. Heck, why settle for one interface? You can have lists and things in the zoom view, and a swirl/molecular diagram for the search/link view.

Michael Graff said...

I'd be interested in an update to your Info Pad article. A lot has changed since then: Kindle, Android, iPad.

Could a Zekira app on current (or imminent) tablet hardware get you close to your original vision?

Michael Graff said...

I forgot you had already done an update in 2011:

Any thoughts since then?

Anonymous said...

I can see some issues with this:

1. What if I have the situation identified where I recognise the person in the room but don't know where from? I can't search based on photos even if the person is in them unless there's going to be some sort of face recognition built into smartphone apps, and I can't look them up by name because I don't know it. So, even with this app I'm no better off.

2. Think of how calculators made us lazy. I don't really need to do mental arithmetic since a calculator can do it for me. If I don't have to think about stuff because this app will do it for me, it's simply going to make me even worse at remembering than before.

3. Facebook will let you tag photos - will this app index the tags. Indeed will it index any tags on anything that will allow indexing?

4. Unless I missed something, this works only if I have something with the keyword in it. So, unless I named a document, for instance, as "Dave_Smith_Financial_report.doc" then it will hve no comprehension who it was authored by and thus it slips through the context crack. If it actually inspected content to look for words then I don't see how thisn can work.

5. What of media files, like video and audio. If I have a video of the last company kick-off day and the CEO - Fred Bloggs - and others took turns in presenting, how will this app know this and return this video in my context results when I search for Fred Bloggs?

Don't get me wrong, I definitely think this is a step in the right direction but it needs to be extensible and seriously intelligent.

Michael Mace said...

Thanks for the questions, Anonymous. They're good ones. Here are my thoughts.

On the facial recognition thing, you're right, the technology today can't handle it. But I think just finding context around a name would be helpful in many situations, especially if you can look that up on a mobile device.

Regarding us getting dependent on our tools, that's a criticism of almost any personal technology. Not to be flippant, but if you want to go back to writing longhand with a quill pen, be my guest. (The more serious answer is that I think a context engine would free us up to remember more important information and insights.)

Yes, we'll index the tags from anything we can read.

We index all of the document, looking at every word. We also scan things like the author field (in Word), and who wrote an e-mail message that the document was enclosed in. Obviously this can't find every document perfectly, but catching most of them is still an improvement over what you can do today with a traditional search product.

We're not trying to index video today, other than by file name and date. In the future I think there are some interesting possibilities, but the technology (and memory capacity, and network capacity) is not there yet.