Direct link: http://dx.doi.org/10.1571/psgp03-10-11cc

Patricia Seybold Group / Perspective

Toward Boosting Our Collective IQ:
A Knowledge-Centric Approach

A Selection of Writings by Douglas C. Engelbart
[Editing, Preface, and Conclusion by Christina Engelbart]
March 10, 2011

NETTING IT OUT (PREFACE)

This paper is adapted from “Boosting Our Collective IQ,” 1995, a selection of readings by Dr. Douglas C. Engelbart produced for the 4th International World Wide Web Conference in 1995. Editing and preface for this new edition by Christina Engelbart, March 2011.

In the early 1990s, Doug Engelbart, renowned innovator who, among many other credits, invented the computer mouse, described the basic components of a dynamic knowledge ecosystem, and the knowledgebases that are constantly evolving from within these ecosystems, as a wake-up call for IT professionals as well as business users. His call to action is even more viable and pressing today as the scope and method of interaction and collaboration become increasingly distributed, networked, cross-functional, cross-disciplinary, cross-organizational, cross-cultural, interdependent, complex, and urgent on a global scale — just as he predicted. His prescription for the technology is essentially a checklist of what is still missing in today's tools that, when embedded in our knowledge ecosystems, will enable us to make quantum leap improvements in the way we work together on important challenges in business and society.

Innovating our dynamic knowledge ecosystems, as Doug describes in this seminal work, would be a great benefit to anyone who is charged with managing multiple projects in the areas of innovation, customer experience, or any other knowledge-intensive initiative. As we work to effectively innovate with, collaborate with, and respond to our customers’ (and our own organizations’) need, we should work toward increasing our Collective IQ—to best leverage our group brain, as it were. And we do this by sharing our ideas and solutions, and by finding tools and methods by which we can work together to capture, manage, access and learn from the shared knowledge, reusing it and evolving it in an effort to become more effective at addressing the problems facing ourselves, our businesses, our customers, and our society.

I hope you find my father’s thoughts compelling and helpful as you work on increasing your own Collective IQ by designing and cultivating your own thriving knowledge ecosystems.

~ Christina Englelbart

I became motivated (committed) in 1951 to improving mankind's capability for dealing with its pressing problems, especially those over-taxing our collective capability to cope with complexity and urgency. I visualized people collaborating interactively on visual displays connected to a computer complex.

By 1962 I'd published “Augmenting Human Intellect: A Conceptual Framework.” Its fundamental concepts still guide my pursuit. The design principles I concentrate on here are but secondary derivatives of that larger-goal conceptual framework. In fact, all our own computer developments into the 1970s (synchronous distributed shared-screen conferencing, windowing systems, outlining tools, hypermedia, the mouse, among others) were secondary derivatives of this conceptual framework; I view the computer merely as a supportive tool.

My goal here is to sketch out basic requirements for supporting the dynamic knowledge ecosystems of teams, initiatives, organizations networks and society, and to encourage the further development of the framework's technological cornerstone – a world-wide open hyperdocument system (OHS) enabling seamless interoperability within and between distributed knowledge ecosystems.

My goal here is to sketch out basic requirements for supporting the dynamic knowledge ecosystems of teams, initiatives, organizations networks and society, and to encourage the further development of the framework's technological cornerstone – a world-wide open hyperdocument system (OHS) enabling seamless interoperability within and between distributed knowledge ecosystems.

Increased pressure for reduced product cycle time, and for more and more work to be done concurrently, is forcing unprecedented coordination across project functions and organizational boundaries. The need for highly effective knowledge work capabilities is becoming increasingly urgent. Yet most organizations do not have a comprehensive picture of what knowledge work is, and of what aspects would be most profitable to improve. In my mind, the point of greatest leverage is what I have come to call "CoDIAK" for the COncurrent Development, Integration and Application of Knowledge.

Figure 1 represents the high-level core of a CoDIAK process, such as can be found in any complex knowledge-intensive task or project. In the center is a basic organizational unit, representing the interactive knowledge domains of a single individual, or of individuals or groups within a project team, department, functional unit, division, task force, whole organization, community, or nation.

Each organizational unit is continuously analyzing, digesting, integrating, collaborating, developing, applying, and re-using its knowledge, much of which is ingested from its external environment.

 

© 2011 Doug Engelbart Institute

Figure 1. The high-level Core of a CoDIAK Process.

 

 

A result of this continuous knowledge process is a dynamically evolving knowledge base, or dynamic knowledge repository (DKR), consisting of three primary knowledge domains (see Figure 2):

•          Intelligence Collection: An alert project group keeps a watchful eye on its external environment, actively surveying, ingesting, and interacting with it. The resulting intelligence is integrated with other project knowledge on an ongoing basis.

•          Dialog Records: Responding effectively to needs and opportunities involves a high degree of coordination and dialog within and across project groups. This dialog, along with resulting decisions, is integrated with other project knowledge on a continuing basis.

•          Knowledge Product: The resulting documents generated by or for the project group provide a comprehensive picture of the project at hand. These documents, which are iteratively and collaboratively developed, represent the knowledge products of the project team, and constitute both the current project status and a roadmap for implementation and deployment.

© 2011 Doug Engelbart Institute

Figure 2. Examples of a Dynamic Knowledge Repository (DKR).

 

 

The CoDIAK process is rarely a one-shot effort. Lessons learned, as well as intelligence and dialog, must be constantly analyzed, digested, and integrated into the knowledge products throughout the life cycle of the project.

With minor adjustments in the boxed lists in Figure 2, this basic generic CoDIAK model seems to apply equally well to academic scholarship, heavy industry, government, medical research, social institutions, philanthropic initiatives, consumer product business, consulting firms, trade associations and consortia, small non-profits, and so on.

The CoDIAK capability is not only the basic machinery that propels our organizations, it also provides the key capabilities for their steering, navigating and self repair. And the body of the applicable knowledge developed represents a critically valuable asset. The CoDIAK capability is crucial across the organization, whether in strategic planning, marketing, R & D, production, customer service, or operations. The CoDIAK capability should be considered a core business competency in the organization's capability infrastructure.

This paper anticipates that new tools and methods supporting the CoDIAK capability will inevitably become harnessed with revolutionary benefit to the ongoing, everyday knowledge work within and between larger organizations. Toward that end, the following concerns about interoperability between knowledge-work domains will have to be met.

Interoperability in an Individual's “Knowledge Workshop”

Consider your own personal CoDIAK environment. Assume that you have evolved a fairly comprehensive, online "knowledge workshop", equipped with better and better software to support the kinds of tasks shown in Figure 3.

 

© 2011 Doug Engelbart Institute

Figure 3. Each functional domain is a candidate for working interchange with all others.

 

 

Consider what you will some day have when your individual workshop inevitably becomes truly integrated. Between the email and the task management files, or the status reports, or whatever, you really would like to tie these functional domains together with a flexible free-flow of information and connectivity.

What kind of interoperability do you have now? I happen to think that the interoperability provided today has a great deal of improvement yet to be pursued.

Interoperability in a Group's Knowledge Workshop

Suppose that you and your colleagues each have a fully integrated online CoDIAK domain, comprised of nicely interoperable sub-domains as in Figure 3. Consider the interoperability needed between your respective knowledge-work domains when you work together online, as in Figure 4.

 

© 2011 Doug Engelbart Institute

Figure 4. Close cooperation between compound knowledge domains puts new demands on knowledge-work interchange.

 

 

Now you're faced with a new challenge and a new problem. You might set it up so you have a few lines that cross between domains, but why stop there? When do two people in intense cooperative work not need total interoperability? In fact, they depend on it heavily in the paper world. Why not online?

Interoperability across Time and Space

Now consider the multiple domains represented in the familiar time-place matrix shown in Figure 5. Is knowledge-work interoperability between the quadrant domains an issue? Very much so. For example, face-to-face meetings need to flexibly utilize anything from the whole organizational knowledge base, and the meeting's records should immediately become an integral part of that same base for later-time work.

© 2011 Doug Engelbart Institute

Figure 5. Collaborative processes commonly considered.

A Point about Online Group Knowledge Work

The matrix in Figure 5 is very neat and ordered. Here in Figure 6, I offer another picture of group knowledge work which reflects how I feel about the various knowledge-work domains with which my personal CoDIAK domain must interoperate.

 

© 2011 Doug Engelbart Institute

Figure 6. Consider also the interactions among them.

The purpose of interoperability is to avoid having information islands between which information cannot flow effectively. Since we grew up in a paper-based framework, we've given little thought about how much exchange and interoperability support we really have had, and how much we depend upon it. To be interoperable in our online CoDIAK world, we could simply print out and hand over the hard copy. With WYSIWYG, word processing and fancy graphics, we're doing that with nicer paper-based formats, faster.

When we inevitably move from computer-supported paper generation and exchange to computer-supported online knowledge creation, exchange and re-use, we will need the same level of interoperability. And, as the number and scale of knowledge domains involved in a given CoDIAK "web" increases, so does the need for "online interoperability" of our knowledge and tools.

The purpose of interoperability is to avoid having information islands between which information cannot flow effectively.

Interoperability across Knowledge Domains

To appreciate the extraordinary complexity of heavy knowledge work at scale, and the associated requirements for interoperability, consider the key functional domains within a large manufacturing organization producing a complex product, such as an airplane. It is difficult enough to provide effective interoperability among the knowledge workers within any of the domains in Figure 7. Now consider the inter-domain challenge, especially considering that some of these domains, such as customers, suppliers and partners exist "outside" the organization, each with its own equally complex multi-domain ecosystem.

 

© 2011 Doug Engelbart Institute

Figure 7. Again, each functional domain is a candidate for working interchange with all others.

With little modification, this model can be adapted to represent any large program, initiative, organization, institution, or society.

It has become apparent to me that someday all of our basic knowledge-work domains will be integrated within one coherent "knowledge workshop." This leads to thinking about an overall, integrated architectural approach to the growing set of knowledge-work capabilities emerging within a multi-vendor environment. Interoperability of tools, processes, and knowledge domains will be increasingly important for a workable knowledge management framework

Assuming an inevitably gigantic scale for our inter-knit "CoDIAK world" provides some important guidance for the continuing investment of our business resources and professional time.

For one thing, each year earlier that an effective degree of knowledge-domain interoperability is in place within important organizational or institutional domains could be worth hundreds of millions of dollars – could mean the difference between increased vitality and sluggishness.

And, for another, we would prefer to avoid investing our research, product development, or organizational-change resources toward ends that won't be interoperably compatible within that future, radically different paradigm.

Someday all of our basic knowledge-work domains will be integrated within one coherent "knowledge workshop."

The foregoing dictates some very significant requirements for any system that attempts to support the CoDIAK processes within and across our future, high-performance organizations. Inevitably, these tools will need to be fully integrated and fully interoperable, supporting very flexible, wide-area sharing of pieces of the knowledge base. This poses the associated need for a new way of thinking about the nature of the knowledge packages we have called "documents." This above requirement for flexibly arranged sharing of essentially arbitrary knowledge chunks provides a very strong argument for documents becoming built from modular-concept nodes with arbitrary inter-node linking – hypertext.

My early assumption, amply borne out by subsequent experience, is that the basic supporting technology for future high-performance knowledge work will be an integrated system based upon multi-media hyperdocuments.

Furthermore, there will be critical issues of interoperability within and between our organizations and their distributed knowledge domains. The ever-greater value derived from online, interactive work within a hyperdocument environment will require a significantly higher degree of standardization in document architecture and usage conventions than heretofore contemplated.

It is inevitable that this service be provided by an "open system" of hyperdocuments and associated network and server architectures. The basic arguments for this Open Hyperdocument System (OHS) are elaborated in “Knowledge-Domain Interoperability and an Open Hyperdocument System.” The hyperdocument system features described below are assumed by me to be strong candidates for requirements for the eventual OHS whose evolution will be so critical to the productivity of industries and nations.

Following is a brief general description of the system design that has evolved from the conceptual orientation described in this paper, through the experience of many years and trial events. Please note that the term "system" is very important here.

•          Shared Files/Documents – the most fundamental requirement. Generalized file sharing is to be available across the entire global domain in which any online collaborative working relationship is established (e.g., world-wide).

•          Mixed-Object Documents – to provide for an arbitrary mix of text, diagrams, equations, tables, raster-scan images (single frame or live video), spread sheets, recorded sound, etc. – all bundled within a common "envelope" to be edited, stored, transmitted, read (played) and printed as a coherent entity called a "document".

•          Explicitly Structured Documents – where the objects comprising a document may be easily arranged in an explicit hierarchical structure, and compound-object substructures may be explicitly addressed for access or to manipulate the structural relationships.

•          Global, Human-Understandable, Object Addresses – in principle, every object that someone might validly want/need to cite should have an unambiguous address supplied automatically by the system as well as optionally by the author/editor, capable of being portrayed in a manner as to be human readable and interpretable.

My early assumption, amply borne out by subsequent experience, is that the basic supporting technology for future high-performance knowledge work will be an integrated system based upon multi-media hyperdocuments.

•          View Control of Objects' Form, Sequence and Content – where a structured, mixed-object document may be displayed in a window according to flexible choice of viewing options – especially by selective level clipping (outline for viewing), but also by filtering of content, by truncation or some algorithmic or graphical view that provides a more useful portrayal of structure and/or object content (including new sequences or groupings of objects, some of which may actually reside in other documents). Editing on structure or object content directly from such special views would be allowed whenever appropriate.

•          The Basic "Hyper" Characteristics – where links can point to any arbitrary object within the document, or within another document in a specified domain of documents – and the link can be actuated by a user or an automatic process to "go see what is at the other end", or "bring the other-end object to this location", or "execute the process identified at the other end". (These executable processes may control peripheral devices such as CD-ROM, video-disk players, etc.). A link may optionally include a specified custom view upon landing in or retrieving the objects pointed to.

•          Hyperdocument "Back-Link" Capability – when reading a hyperdocument online, a worker can utilize information about links from other objects within this or other hyperdocuments that point to this hyperdocument – or to designate objects or passages of interest in this hyperdocument.

•          Link Addresses That Are Readable and Interpretable by Humans – one of the "viewing options" for displaying/printing a link object should provide a human-readable description of the "address path" leading to the cited object; and, the human must be able to read the path description, interpret it, and follow it (find the destination "by hand" so to speak).

•          Personal Signature Encryption – where a user can affix his personal signature to a document, or a specified segment within the document, using a private signature key. Users can verify that the signature is authentic and that no bit of the signed document or document segment has been altered since it was signed. Signed document segments can be copied or moved in full without interfering with later signature verification.

•          Hard-Copy Print Options to Show Address of Objects and Address Specification of Links – so that people working with associated hard copy can read and interpret the link-citation, and follow the indicated path to the cited object in the designated hard-copy document. Also, suppose that a hard-copy worker wants to have a link to a given object inserted in the online file. By visual inspection of the hard copy, he should be able to determine a valid address path to that object and for instance hand-write an appropriate link specification for later online entry, or dictate it over a phone to a colleague.

•          Hyperdocument Mail – where an integrated, general-purpose mail service enables a hyperdocument of any size to be mailed (each email is essentially a hyperdocument as described above, no matter how short or insignificant) . Any embedded links are also faithfully transmitted – and any recipient can then follow those links to their designated targets that may be in other mail items, in common-access files, or in "library" items. In addition, each email item is automatically assigned a unique identifier, enabling participants to later cite or link back to any object in any email.

•          The Hyperdocument "Journal System" – an integrated library-like system where a hyperdocument message or document can be submitted using a submittal form (technically an email message form), and an automated "clerk" assigns a catalog number, stores the item, notifies recipients with a link for easy retrieval, notifies of supercessions, catalogs it for future searching, and manages document collections. Access is guaranteed when referenced by its catalog number, or "jumped to" with an appropriate link. Links within newly submitted hyperdocuments can cite any passages within any of the prior documents, and the back-link service lets the online reader of a document detect and "go examine" any passage of a subsequent document that has a link citing that passage.

•          Access Control – Hyperdocuments in personal, group, and library files can have access restrictions down to the object level.

•          External Document Control (XDoc) – (Not exactly a "hyperdocument" issue, but an important system issue here.) Documents not integrated into the above online and interactive environment (e.g. hard-copy documents and other records otherwise external to the OHS) can very effectively be managed by employing the same "catalog system" as for hyperdocument libraries – with back-link service to indicate citations to these "offline" records from hyperdocument (and other) data bases. OHS users can find out what is being said about these "XDoc" records in the hyperdocument world.

The overview portrayal in Figure 8 shows the working relationships between the major system elements described above. Note the shared catalog service that supports use of the Journal and External Document services.

 

© 2011 Doug Engelbart Institute

Figure 8. "Hyperdocument" provides flexible linkages to any object in any multi-media file; "Open" provides vendor independent access within and across work groups.

 

Details of features and designs for well-developed prototypes of some of the above may be found in previous writings (“Toward High-Performance Knowledge Workers,” “Authorship Provisions in Augment,” and “Collaboration Support Provisions in Augment.”)

Four General Architectural Requirements

Besides the aforementioned Hyperdocument Mail and Hyperdocument Library features that depend upon special larger-scale architectural features, there are at least four other important tool-system capabilities that are very important to wide-area groupware services such as those being considered here:

GLOBAL AND INDIVIDUAL VOCABULARY CONTROL. Somewhat new in the history of computer services are issues regarding the evolution and use of a common "workshop vocabulary" among all the users of the forthcoming "global knowledge workshop." Common data dictionaries have been at issue, of course, but for a much more limited range of users, and for a more limited and stable vocabulary than we will face in the exploding groupware world.

Our own architectural approach has been to introduce into every user-interface environment a common Command-Language Interpreter (CLI) module that derives the user's available operations (verbs) as applied to the available classes of objects (nouns) from a grammar file (individualized if desired with respect to the size and nature of the verbs and nouns utilized from the common vocabulary). The CLI interprets user actions, based upon the contents of the currently attached grammar file, and executes appropriate actions via remote procedure calls to a common application program interface of the "open system environment".

Each of us knowledge workers will become involved in an even richer online environment, collaborating more and more closely within an ever global "knowledge workshop", with multi-organizational users of widely divergent skills and application orientations who are using hardware and software from a wide mix of vendors.

Without some global architectural capability such as suggested above, I can't see a practical way to support and control the evolving global "workshop vocabulary" in a manner necessary for effectively integrating wide-area groupware services.

MULTIPLICITY OF LOOK-AND-FEEL INTERFACE CHOICES. Based upon the same Command-Language Interpreter (CLI) architecture as above, a "look-and-feel interface" software module would be located between the CLI and the window system. Providing optional modules for selected look-and-feel interface characteristics would serve an important practical as well as evolutionary need.

There would be a basic constraint necessary here. When working interactively, no matter what particular look-and-feel style is being used, a user has a particular mental model in mind for the significance of every menu item, icon, typed command or "hot, command-key combination" employed.

The necessary constraint needed here is that the resulting action, via the interface module that is being employed for this user, must be produced through the underlying execution of processes provided by the Command-Language Interpreter module as derived from use of common-vocabulary terms. And the users should learn about their tools and materials, and do their discussing with others about their work, using the underlying common-vocabulary terms no matter what form of user interface they employ.

So far, the evolution of popular graphical user interfaces has been heavily affected by the "easy to use" dictum. This has served well to facilitate wide acceptance, but it is quite unlikely that the road to truly high performance can effectively be traveled by people who are stuck with vehicular controls designed to be easy to use by a past generation.

Besides relaxing the troublesome need to make people conform to a standard look-and-feel, this approach has a very positive potential outcome. So far, the evolution of popular graphical user interfaces has been heavily affected by the "easy to use" dictum. This has served well to facilitate wide acceptance, but it is quite unlikely that the road to truly high performance can effectively be traveled by people who are stuck with vehicular controls designed to be easy to use by a past generation.

As important classes of users develop larger and larger workshop vocabularies, and exercise greater process skill in employing them, they will undoubtedly begin to benefit from significant changes in look-and-feel. The above approach will provide open opportunity for that important aspect of evolution toward truly high performance.

SHARED-WINDOW TELECONFERENCING. This is where remote distributed workers can each execute a related support service that provides the "viewing" workers with a complete dynamic image of the "showing" worker's window(s). Used in conjunction with a phone call (or conference call), the parties can work as if they are sitting side-by-side, to review, draft, or modify a document, provide coaching or consulting, support meetings, and so on. Control of the application program (residing in the "showing" worker's environment) can be passed around freely among the participants. Generic provisions of this service are discussed elsewhere.

Inter-Linkage Between Hyperdocuments and Other Data Systems. For instance, a CAD system's data base can have links from annotations/comments associated with a design object that point to relevant specifications, requirements, arguments, etc., of relevance in a hyperdocument data base – and back-link service would show hyperdocument readers which passages were cited from the CAD data base (or specified parts thereof).

Similarly, links in hyperdocuments may point to objects within the CAD bases. And, during later study of some object within the CAD model, the black-link service could inform the CAD worker as to which hyperdocument passages cited that object.

The CoDIAK Process Supported by an OHS

With the above tool capabilities, together with well-developed methods and other human-system elements, the organization's capability infrastructure could support the following types of online CoDIAK scenarios.

Note that the following online interactions are designed to work, even if the users are in different organizational units, in different organizations, using different application packages on different workstations (assuming access to the data is not barred by stringent privacy features, naturally). The real test of an OHS is when you can click on a link you received via email from someone in a different organization, jumping directly to the passages cited, and then comfortably maneuver through the "foreign" knowledge domain, possibly jumping up a level with an outline view to see the context of the given passage, following other links you find there, and so on, without having to fumble through unfamiliar processes.

The real test of an OHS is when you can click on a link you received via email from someone in a different organization, jumping directly to the passages cited, and then comfortably maneuver through the "foreign" knowledge domain.

INTELLIGENCE COLLECTION. Now an alert project group, whether classified as core business, or improvement activity, can keep a much enhanced watchful eye on its external environment, actively surveying, ingesting, and interacting with it mostly online. Much of the external intelligence is now available in hyperdocument, multimedia form, having been captured in an OHS Journal facility. When I send you an email to let you know about an upcoming conference, I can cite the sessions I think you'd be interested in, and you can click on the enclosed citation links to quickly access the cited passages (taking advantage of hypertext links and object addressability). When I do a search through the Journal catalogs to research a question for the proposal I am writing, I can see who has cited the material and what they had to say about it. If the material is offline (i.e., in XDoc), I can quickly discover where it is stored and how to obtain a copy, probably requesting it via email. If the material is online, I can access it instantly, usually starting with a top-level outline view of the document's titles (taking advantage of the OHS document structure and custom viewing features), possibly setting a simple filter to narrow the field, then quickly zooming in on the specific information I require. I can quickly build an annotated index to the intelligence documents, or objects within those documents, that I want to keep track of. I can share with you a macro I wrote to trap certain incoming intelligence items and reformat them in a certain way, and you could fire this up in your own environment to work off your pet key-words (taking advantage of the common-vocabulary architectural feature). All the intelligence collected is easily integrated with other project knowledge.

DIALOG RECORDS. Responding effectively to needs and opportunities involves a high degree of coordination and dialog within and across project groups. In an OHS environment, most of the dialog will be conducted online via the Journal. Email would be used mostly for "throw-away" communiqués, such as meeting reminders. All memos, status reports, meeting minutes, design change requests, field support logs, bug reports, and so on, would be submitted to the Journal for distribution. Asynchronous online conferencing would be supported by the Journal, with each entry tagged and cataloged for easy future reference. Document exchange would be a matter of submitting the document to the Journal with a comment such as "Here's the latest version – please note especially the changes in Section G, differences are listed in File Y" including links to that section and that file for easy access. The reviewers would click on the links, and proceed to review the document. To make a comment, the reviewer would click on the object in question, and enter the comment, such as "Replace with 'XYZ'," or "Watch out for inconsistencies with Para G4!" with a link to the passage in G4. The author then gets back the indexed comments, and has many options for quickly reviewing and integrating them into the document. Such dialog support will obviate the need for many same-time meetings.

Everyone is but one quick "link hop" away from any piece of knowledge representation anywhere in the whole knowledge collection. Smart retrieval tools can rapidly comb part or all of the collection to provide lists of "hit links" with rated relevance probabilities.

Same-time meetings, when needed, would be greatly enhanced by an OHS. The dialog motivating the meeting would already be in the Journal. Agenda items would be solicited, and the agenda distributed via the Journal. At the meeting, the agenda and real-time group notes can be projected on a large screen, as well as displayed on each participant's monitor (using the "shared screen" feature), and any participant can point to the displayed material (e.g. using a mouse). Controls can be passed to any participant to scribble, type, or draw on this virtual chalkboard. Any presentation materials and supporting documents can be instantly retrieved directly from the knowledge base for presentation. All resulting meeting documents, along with references to supporting documents cited, would subsequently be submitted to the Journal for immediate access by all authorized users.

In addition, tools will soon become generally available for flexibly contributing, integrating, and interlinking digitized speech into the OHS knowledge base. Early tools would be available for speaker recognition, for special-word recognition, and even for basic transcription to text – and for installing and following links between modules as small as a word embedded in a long speech string. This will greatly enhance the development, integration, and application of dialog records. More elegant tools will follow, and as human conventions and methods evolve to make effective use of technology, the quantity and completeness of recorded dialog will become much more significant.

KNOWLEDGE PRODUCT. Throughout the life cycle of the project, the online OHS knowledge product will provide a truly comprehensive picture of the project at hand. Intermediate project states, including supporting intelligence and dialog trails, can be bundled as document collections in the Journal for document version management. All knowledge products will be developed, integrated, and applied within an OHS, with concurrent contributions from many diverse and widely distributed users. These users can also work as if sitting side by side, reviewing a design, marking up a document, finalizing the changes, etc. (using the shared screen feature). Finding what you need among the thousands of project documents will be a simple matter of clicking on a link (provided by the Journal catalogs, or by your project's indices), and zooming in and out of the detail, or by having someone else "take you there" (using the shared screen feature). Accountability is absolute – Journal submittals are guaranteed to be authentic, and each object can be tagged by the system with the date and time of the last write, plus the user who made the change. Documents can be signed with verifiable signatures.

Everyone is but one quick "link hop" away from any piece of knowledge representation anywhere in the whole knowledge collection. Smart retrieval tools can rapidly comb part or all of the collection to provide lists of "hit links" with rated relevance probabilities.

Conventions for structuring, categorizing, labeling and linking within their common knowledge domain will be well established and supportive of a high degree of mobility and navigational flexibility to experienced participants – much as residents get to know their way effectively around their city if they get much practice at it.

As a group adapts its ways of working to take better advantage of a tool system such as projected here, the classes of knowledge objects will grow, as will the functions available to operate upon them – and that growth will be paralleled by the concurrent evolution of an ever richer repertoire of the humans' "workshop knowledge, vocabulary, methodology and skills".

Now that the internet has opened people's eyes to (a small portion of) hypermedia's potential, we should expect active penetration into the hypermedia frontier – where something such as OHS is available for widespread use within and across our dynamic knowledge ecosystems.

There is tremendous potential here, and many methods, procedures, conventions, organizational roles to be developed in close association with the tools. And, if the OHS is to be open, there is much deep exploration to be done into different application domains, such as knowledge management, groupware, organizational learning, Total Quality Management (TQM), Enterprise Integration (EI), program management, Computer-Aided Software Engineering (CASE), Computer-Aided Engineering (CAE), Concurrent Engineering (CE), organizational memory, online document delivery and CALS, and so on.

Potential customers for augmented CoDIAK capabilities can be seen everywhere in today's global society: e.g., all of the "Grand Challenges" earmarked in the U.N. for special support. Essentially every professional society will eventually operate this way; as will legislative bodies and government agencies, and university research programs.

In short, our solutions to every other challenging problem that is critical to our society will become significantly facilitated by high-performance CoDIAK capabilities.

In closing, I would like to emphasize the importance of the evolution of perceptions, visions and paradigms. I am convinced that cultivating the appropriate paradigm about how to view and approach the future will in the pursuit of high-performance organizations be the single most critical success factor of all.

My goal in this short piece is to encourage the further development of an open hyperdocument system (OHS) — an integrated, seamless multi-vendor architecture in which knowledge workers share hyperdocuments of all kinds on easily shareable screens. Hyperdocuments are multimedia files supporting linking and multiple object types. The hyperdocument system should enable flexible, on-line collaborative development, integration, and application, study and reuse of knowledge (CoDIAK).

Now that the internet has opened people's eyes to (a small portion of) hypermedia's potential, we should expect active penetration into the hypermedia frontier – where something such as OHS is available for widespread use within and across our dynamic knowledge ecosystems.

In the limited space here, I'll express some design thoughts for the World Wide Web's continued evolution. The references describe a more complete set, including very important non-hypermedia aspects for system architectures.

EVERYTHING IN THE WORK ENVIRONMENT IS LIVE HYPERDOCUMENT STUFF. All knowledge products, such as plans, reports, email, notes, source code, to-do lists, work breakdown structures, status reports, design documents, user guides, trouble reports, and others are inherently hyperdocument objects. The infrastructure provides knowledge products with all the hyperdocument capabilities described here.

INTEGRATED APPLICATIONS. A tool system facilitating a universal knowledge base and generic CoDIAK processes replaces the standard application or function-based paradigm. Individual application subsystems (graphical editors, program language editors, spreadsheets) work with knowledge products, but do not "own" hyperdocuments in the sense of being responsible for their storage or representation. For instance, one could create a Gantt chart within a project management system, and manipulate it as a graph in a charting application or as mail in an email application. An integrated core application package provides base capabilities of composing, reading, annotating, linking and manipulating knowledge products. All knowledge workers – authors and users – modify and incorporate other knowledge products into their own information bases and knowledge products.

EXPLICITLY STRUCTURED DOCUMENTS. Objects within a hyperdocument have an explicit structure in which structural and logical substructures may be addressed and manipulated. For example, one can manipulate any statement in a hierarchical structure as an aggregate branch of all its substatements (with each maintaining its individual identity). This greatly extends the notion of manipulating sections and subsections in today's outlining tools.

EVERY OBJECT INTRINSICALLY ADDRESSABLE (LINKABLE TO). Every knowledge object – from the largest documents, to aggregate branches, down to content units such as characters – has an unambiguous address, understandable and readable by a user, and referenceable anywhere in the hyperdocument system. Such intrinsic addressability should be integrated deeply into commands for editing, structuring, jumping. Intrinsic addressing options not only are natural to learn and embed in links, but serve as parameters for direct, user-invoked jumping and manipulation commands. This addressing scheme allows direct or indirect addressing (absolute or relative, and through aliases; indeed we allow unlimited indirect address chaining) and working with objects not currently displayed. For instance, one can copy a structure without finding and opening the file containing it. Meta-level referencing (addresses on links themselves) enables knowledge workers to comment upon links and otherwise reference them.

VIEW CONTROL OF FORM, SEQUENCE AND CONTENT. A structured, mixed-object hyperdocument may be displayed with a flexible choice of viewing options: selective level clipping, filtering on content, truncation or other transformation of object content, new sequences or groupings of objects including those residing in other documents, etc. Links may specify views so traversal retrieves the destination object with a prespecified presentation view (e.g., display as a high-level outline or display only a particular statement). View specification becomes a natural and constantly employed part of a user's vocabulary.

HYPERDOCUMENT LIBRARY SYSTEM. Hyperdocuments may be submitted to a library-like service (an administratively established AUGMENT Journal) that catalogs them, and provides a permanent, linkable address and guaranteed as-published content retrieval. This Journal system handles version and access control, provides notifications of supercessions and generally manages open-ended document collections.

Open hyperdocument software concepts are but a small part of a larger Bootstrapping Initiative. We are fostering a cooperative community of organizations interested in strategically improving their collective improvement capabilities, and thereby augmenting each organization's – and indeed society's – potential to excel in our rapidly changing world.

The issue of maximizing Collective IQ in business and society presents a grand challenge with tremendous potential payoff. Indeed the team, company, institution, community, or whole nation which goes after this critical pursuit most aggressively will come out far ahead of its meandering counterparts in productivity, effectiveness, and competitiveness. The World Wide Web provides an excellent springboard for this pursuit, assuming it can move from archival publishing into supporting our dynamic, malleable, heterogeneous CoDIAK work environments.

Every knowledge object – from the largest documents, to aggregate branches, down to content units such as characters – has an unambiguous address, understandable and readable by a user, and referenceable anywhere in the hyperdocument system.

The architecture of the future electronic documents, and of the supportive application systems for working with them, will be a critical factor in limiting or enhancing the performance levels attainable. These architectures are too important to allow their evolution to be dominated by simple, vendor-customer market forces. For any real hope of gaining significant progress, explicit advocacy and pursuit of high-performance collective knowledge-work capability has to be taken up by a "critical mass" of stakeholders, funds, and organizational backing.

As a starting point, the critical yet-to-emerge technology for pursuing these goals is the Open Hyperdocument System (OHS). A highly evolvable advanced prototype OHS which implements the above-stated requirements should be made available for experimentation. This would provide a starting point for exploring the associated work practices, conventions, roles and policies for online collaborative planning, authoring, structuring, linking and managing dynamically evolving repositories. Furthermore, a number of prototypical high-performance teams should be cultivated as models of future workmodes, which can be deployed to support particularly complex mission-critical work.

The ongoing results of such a program would be immediately applicable within many knowledge-intensive activities and grand challenges in government, industry, and society, including software engineering, manufacturing, procurement, concurrent engineering, total quality, legislation, networked improvement communities, re-engineering, telecommuting, digital libraries, distance learning, crisis action, and so on. Cultivating industry and community participation in the formulation of requirements and standards, and collaborating on pilot trials and lessons learned, would provide a much-needed accelerative boost to this effort. And furthermore, boosting the Collective IQ of industry-government collaboration with the emerging tools and practices would offer further leverage to this critical pursuit.

There have been many extraordinary developments in information technology since Doug Engelbart’s 1995 call to action. The World Wide Web, personal computers, laptops, cellphones, etc. are now essentially ubiquitous. Software applications have come into widespread use in almost every conceivable (and sometimes inconceivable) functional domain. As a result, more and more of our information, thoughts, and images are generated, captured, and/or shared online, just as Doug predicted. And yet we have barely scratched the surface of potential that he envisioned as a baseline for enabling information technology.

If I want to send you a link to a specific passage or section in a document, or a specific sequence of frames in a video, or a specific object in a photo, I probably can’t. Unless, in the case of a document, the author happened to mark that phrase or section with an “anchor name” label or ID, and I’m clever enough to discover its name and location in the file.

Hyperdocuments Haven’t Happened Yet

For example, the “hyperdocument” as Doug defined it, does not yet exist. Yes, today’s multimedia documents and Web pages allow us to link to any file available on the Internet, be it a photo, video, text document (privacy provisions permitting). But if I want to send you a link to a specific passage or section in a document, or a specific sequence of frames in a video, or a specific object in a photo, I probably can’t. Unless, in the case of a document, the author happened to mark that phrase or section with an “anchor name” label or ID, and I’m clever enough to discover its name and location in the file. (I personally do this today by trolling the html source code of the file in question for a label that, if not marking the spot exactly, might land the user close enough — not a wonderful user experience).

On the other hand, spreadsheets have always offered granular addressability — I can reference or go to a specific position in a spreadsheet because the spreadsheet application provides a cell reference number for each entry, which it displays in plain sight so I don’t have to dig for it. With PDF formats, I can theoretically link to a specific page of a document using a specified syntax (why are we still using “page” when the computer can give us much more relevant and useful ways to view and access the information?). In Wikipedia, I can link to a specific section of any wiki page, using the section title, but what if someone changes the section title? Will my link still work?

In Doug’s idea of an open hyperdocument system, no matter whose software we’re using, no matter what type of media we’re editing, viewing, or linking to, each object in every file would have an internal reference number automatically assigned to it that we could use to manipulate or link to from anywhere. We could click on the object and select something like “Create Link to this object,” and let the computer look up the reference number and figure out the right syntax. We could also optionally view or hide the automatic numbering and labels for easy reference, among the many custom views available, and create our own links.

Still Needed: True Interoperability

Years ago, my best friend’s grandmother told me that when people were first having telephones installed in their homes, there were two phone companies. Whichever phone company you went with, you would then be able to call any of its customers. However, you would not be able to call the other phone company’s customers. So you needed to go with the company that served your most important friends and family. Because her family was well off, she was one of a lucky few to have a phone line from each company installed her family home, to call and receive calls from friends and family on either line. They only had to keep straight who was on which line. Today this seems incredible, because, lucky for us, the phone companies later agreed on an interoperability standard.
Today’s software arena is still largely in the early, pre-interoperability stages of development, where sharing and exchanging still often requires finagling on the users’ part — exporting files to rigid bit-mapped formats or negotiating with people you wish to share with. For example, my sister recently sent me an external drive of family photos which I could not access on my PC because she had used a Mac to format the drive. I had to borrow a Mac.

Today’s software arena is still largely in the early, pre-interoperability stages of development, where sharing and exchanging still often requires finagling on the users’ part — exporting files to rigid bit-mapped formats or negotiating with people you wish to share with.

Another example is shared-screen teleconferencing, the one requirement on Doug’s OHS checklist that has begun to mature into the mainstream. If you and I wish to have a conference call with shared desktop capability, we have to pick one brand of conference software to use during the call because the software you use, and the software I use, are not interoperable (unless we happen to use the same brand). In other words, if I log into the software I use, and you log into the software you use, we will not be able to locate each other, let alone connect. And even when we use identical software to connect, we cannot trust that we can each access our respective knowledge domains from within that shared conference environment. Participants may need to email to the host any files not published on the Web that we might want to access during the meeting, to be preloaded into the conference system. In other words, we are still in the early stages, the technology has yet to mature, and its true power and potential has yet to be realized.

What’s Holding Us Back?

The OHS requirements and usage scenarios described in this paper are still largely science fiction. This is due in large part to an outmoded paradigm in the industry that has largely been focused on solving for vertical functional domains such as word processing, graphics, email, project management, engineering, etc., often to the exclusion of all others. This creates knowledge barriers. What’s still needed is a horizontal ecosystems approach that solves for the generic cross-cutting knowledge processes – CoDIAK and DKR – to support a thriving foundational dynamic knowledge ecosystem, and completely rethink the vertical applications accordingly. This creates knowledge free-flow. This is what the OHS approach is all about – shifting from an application-centric to a knowledge-centric approach.

At some point, the industry did settle on a (limited) standardized model of basic graphical user interface mechanics such as point and click object selection, pull-down menus, selected commands like cut and paste, and selected command-key shortcuts. Now it’s time to meet end-users’ needs by taking it to the next level.

Technically speaking, implementation of OHS functionality would be relatively trivial, while the potential benefits would be tremendous. We know this because with a handful of researchers using very primitive technology in the mid 1960s, Doug Engelbart implemented 80 percent of these requirements in the span of four years! He used the fully operational prototype in his lab extensively to revolutionize every aspect of their individual and collaborative work, including customer-focused R&D and other novel methodologies, all of which continuously increased their Collective IQ and innovation capacity in ways that would astound today’s business strategists. They first demonstrated their work in 1968 in what is now called the “Mother of All Demos” and went on for another decade of breakthrough innovation. They did all that under the constraints of 1960s computer technology. They first had to develop their own software language using teletypes, push the envelope in display technology, invent the mouse, and more, just to get the first prototype up and running. Here we are in 2011. Imagine what we could accomplish using today’s technology as a springboard.

What will it take? I would recommend pushing along four vectors in parallel:

1.      ![endif]>Transform the provider to end-user relationship. The computer industry has largely operated as a separate entity, tossing inventions and new versions over the wall to consumers, who then scramble to work out how to adapt it. Let’s work together to help developers transform into customer-oriented enterprises practicing outside innovation in close collaboration with their marketplace. Encourage proactive participation from a broad variety of heavy-usage end-user communities (see “About Networked Improvement Communities (NICs)” on our Web site). Expect providers to walk their talk, to be the poster child success story of using their own wares to greatest effect. Are they increasing their own Collective IQ demonstrably and, with it, their capacity to innovate with their customers?

2.      Challenge developers to implement pieces of OHS functionality in existing platforms. Start with applications that package information for us, like wiki, blogs, forum software, email, Google apps, office apps, digital libraries (see “About Open Hyper Tools” for pointers). Use your new relationships from #1 to start the interoperability conversation that will lead toward industry standards.

3.      Get some end-to-end rapid prototypes going that enable broad-based collaboration in experimental pilots to try out whole new ways of working together. Piece together a workable dynamic knowledge ecosystem prototype of best practices and prototypical tools to support participants as they collaborate on how to run the pilot, engage the stakeholders, capture requirements, research options, track dialog and commentary, develop and manage drafts and revisions, take measurements, and keep iterating. Start small, and scale up. Spin off whatever works. Join forces to share costs and learnings. Doug was fond of saying it’s like the computer industry is building better and better windshields, engines, and seats, but no one is considering how all the parts need to work together in a vehicle that we can fly around our information space in with agility, precision, and speed. Here’s where you get to architect, experiment, and test-drive the high performance dynamic knowledge ecosystem supported by more and more OHS functionality as well as revolutionary methodologies. See “Bootstrapping Innovation” for pointers on recommended strategic approach for pilot teams.

4.      Start now experimenting with new methodologies, envisioning how you could work together if you had OHS, adapting whatever tools are available today. Network with like-minded others to share the work of researching what best fits the bill, along with ideas and lessons learned (see “About Networked Improvement Communities (NICs)”, as well as “Bootstrapping Innovation” for a great case example of an organization that leverages largely off-the-shelf technologies in revolutionary ways).

 

Throughout his 50-year career, Doug Engelbart established an unparalleled track record in predicting, designing and implementing the future of organizational computing. Best known for inventing the computer mouse, he pioneered human-computer interaction, hypertext, groupware, knowledge management, and online communities, as well as strategic organizing principles for customer-oriented continuous improvement and innovation. Integrated prototypes were in full operation within his research lab as early as 1968, which later evolved to support thousands of users benefitting from its unique team support capabilities. Doug authored numerous seminal articles, and 21 patents, although he considered his strategic approach to be his most important breakthrough, which inspired all his other pioneering firsts. After nearly 40 years in industry and research, he founded the Bootstrap Institute (now the Doug Engelbart Institute) with daughter/partner Christina Engelbart, to carry on his visionary work. He was awarded numerous honors for lifetime achievement and innovation, including the National Medal of Technology, the Lemelson-MIT Prize, and ACM's A.M. Turing Award.