Need 1

Information-intense textual and multimedia environments need more powerful ways to organize and share information. In order to make better use of information, we need to be able to create information representations that better map our mental concepts. Such a capability would support more refined concepts, and along with better accessibility, enhance their shareability, communicability, and reuse. Reusing information across processing-tool sets (interoperability), individuals, teams, and organizations (enterprise integration) would yield increased efficiencies in generating and documenting higher quality knowledge products (reports, designs, etc.). 1A

Current commercial-off-the-shelf softcopy environments are typically closed-world systems allowing little or no interoperability with information or tools outside of their environment. Only limited tool sets are available to work with information in these environments. 1B

What is needed is for different individuals and organizations working on different types of information with different tool sets, to find, reference, work with, and build on one anothers' information. Environments need to be developed which offer strong interoperability among document types and tools, with a diverse tool set, and which address importing and exporting of external tools' information to and from the environment. Such knowledge-intensive environments must support the ability of users to collaboratively find, reference, generate, interrelate, and reuse information products. Eventually, it is expected that open standards will evolve for document structures and tool functionality which will be available at both the user and programming level. Once this occurs, standards-based commercial products can emerge to address the tool shortage problem. 1C

In summary, organizations need to dramatically improve their information-handling capabilities in order to remain effective in a significantly faster paced and more complicated world. We believe that the technologies that we have investigated, more than any other, hold the greatest promise of supporting these knowledge-intensive activities. This project obtained hands-on experience with a powerful unique (not generally available) collaborative environment (Augment) that was created and evolved to address these issues. From this experience, we gained insight into needed functionality and work methods for future high-productivity collaborative work environments. 1D

Progress 2

Task 1. Investigated the structuring of information to better support referencing, retrieving, generating, interrelating, and reusing information by multiple classes of users (manager, developer, maintainer, accountant) in ways that only a collaborative hypermedia system would allow. 2A

Task 2. The investigation of this particular high-capability system allowed us to determine functionality and work-practices for future collaborative information systems. 2B

Summary of Results 3

Task 1. A past software development project's information was moved to the Augment system. During this importing process, the documents were given standard Augment outline structure. This supports the system's uniform referencing (addressing) capabilities used when referring to information in documents from commands or scripts. 3A

The documents were interlinked where appropriate using the system's hypertext linking features. This allows a reference to a passage in the same or another document to be embedded at a specific point in the referencing document. Tools exist to "follow" a link to the target to which it points, retrieving it in the same or some other user-specified window. For example, requirements in one document were linked to the corresponding descriptions in preliminary and detailed design documents. This traceability trail was extended to each item's implementation in source code files. In addition to formal deliverable documents, informal e-mail design discussions, working notes, etc. were also interlinked where appropriate. 3B

In the demonstration scenario, problem reports were linked to status reports, to the problem area in the source code, and to e-mail discussion about the problem or other related problems. In this environment, it is relatively easy for a maintenance person to find related problems and discussions about them, or for management to follow the details of the issues to the extent desired, for example. One has the opportunity to trace back to the source of the problem in earlier design and requirements documents as necessary, and by following other design trail links, discover what other areas of the system might yet be affected by the problem. The potential increase in quality and efficiency afforded in the creation and maintenance of systems is precisely why such hypertext capabilities are frequently proposed to meet ISO 9000 system development and documentation standards. 3C

Task 2. In working with the Augment system, some of the basic findings were as follows. 3D

In this report, the phrase "work practices" includes such things as ways of doing business, methodologies, practices, procedures, conventions, paradigms, roles, customs, attitudes, and habits. Work practices will evolve over time, and a system must be malleable enough to quickly accommodate desired changes or experiments in support of long-term growth of an individual's, team's, or organization's capabilities. Hence, a major requirement for such support systems is that they should be very plastic; extensible by in-house staff if not by the users themselves. Lotus Notes is an example of a groupwork product that supports this kind of extensibility in many areas. However typical commercial software is not flexible enough to support such change, resulting in tools that soon become obsolete, or stymie work-process evolution. The Augment system supports a user-extensible command language, with provisions for sharing those extensions in a group environment (so that everybody is speaking the same language and can build on previous extensions to the command set). A number of such tool extensions were easily created during this project in support of automatic file importing and restructuring, the creation of hypertext links, and unique e-mail referencing and management functionality. 3E

A major lesson learned is that a broad set of simple capabilities goes a long way toward providing useful end-user solutions, as long as the capabilities are interoperable and extensible. For example in Augment, the document structuring and hypertext linking functionality are available uniformly in all tools including internet e-mail. 3F

This is in contrast to typical commercial packages which provide rigid point solutions consisting of high functionality but which are very narrowly focused. Lotus Notes and typical CASE tools would be examples. E.g. our software development scenario indicated that a software development project would probably be better supported using Augment's broad capabilities with suitable extensions, than by conventional CASE tools, specifically because it would allow interoperability of the CASE functionality with all other aspects of the system's broad capabilities (screen sharing, work flow management, library journal, e-mail, etc.). One might realistically trade off some very specific CASE functionality for improved group productivity and reusability. 3G

Much of Augment's capability comes from usage conventions - agreed upon work practices and procedures - with simple tool extensions where appropriate to support them. One example is the use of two side-by-side windows to work with large files; one narrow window showing only the top levels of the document's structure as an overview or map, and a second full sized window zoomed in to the detail of the particular area of interest. The Augment user interface functionality allows easy selection of the level of outline detail to be displayed in a window, and the specific area from the overview map to be displayed in the detail window. Fast navigation is then supported by mousing the spot in the overview window that the detail window is to display (via the Jump command). In the future, tool support would be extended to directly support this way of working and navigating, with automatic setup of such paired windows' relative screen positions and the appropriate levels of detail to be displayed in each. 3H

A second example was in a large McDonnell Douglas project, where on-line status reports were in a specific format by convention. Command scripts were created and periodically executed to automatically gather together particular types of information from the reports of whole groups of people (e.g. everyones' Schedule and Problems sections), and bundle them into higher-level or more tailored reports. 3I

Augment was probably the first system to implement hypertext. Some observations about Augment's linking capabilities follow. 3J

• The document structuring (hierarchical outlining) and non-hierarchical hyperlinking capabilities complement each other very well. Outlines capture hierarchical relationships in a natural and familiar manner, but are somewhat limited compared to links (e.g. the same item can't appear in multiple places in an outline without duplicating it). Hyperlinks offer a more general though less structured or familiar way of capturing arbitrary semantic relationships among a set of items (referred to as a "semantic network"). Outlines require collocation of the related items in the same document whereas links do not. But outlines can contain links, and links can reference outline structure in the same or another document, so these two capabilities can work together in a synergistic manner. Both scale well to support structuring of very large collections of information. 3J1

• Links are very powerful organization and retrieval tools. In Augment, links are basically point-to-point, though with support for limited specification of the extent of the item being referred to. I.e. a link descriptor appears at a specific point in the referring document as specially demarcated text (which is the "address" of the item being referred to). There is no specific demarcation of any surrounding document text which would explicitly define the context for the link. Both this and the exact extent of the target are either inferred from context by the reader, or in an attempt to avoid ambiguities, additional description or commentary is required at the link by the author. What is needed is fine-grained specification and optionally visible demarcation of the location and extent of both ends of a link. 3J2

• Additionally, links in Augment are unidirectional. In reading a document, there is no way to see what links if any point into it. In the demonstration scenario, this required creating two links for every pair of interrelated items, one link at each item pointing to the other item, e.g. so that one could follow a design description to its implementation in the source code, or from the source code follow a link back to the section of the design document that describes the functionality that was to have been implemented. In newer systems some capability for discovering backlinks (links pointing into a document) would be required, along with the ability to follow the links in either direction (from either linked item to the other). 3J3

• Though links should be followable in both directions, they need a semantic direction to support link "types"; e.g. to capture the relation that one item "preceded" the item to which it's linked. The link with type "preceded" needs a semantic direction so that a user can sort out which item preceded which. Augment supports this, but only because of the unidirectionality of its links. Primitive "typing" support exists in Augment via the link's comment field - which was originally created to capture such information. 3J4

• Augment limits linking capabilities to the document's author. I.e. only users with write access to a document may embed links to other items in it. This probably arose from links' original usage as formal references to other documents. This author-vs-reader distinction is still maintained in modern hypertext systems which are oriented towards authoring of documentation (e.g. in SGML or World Wide Web's HTML format) or on-line help systems. But more general hypertext systems allow symmetric creation of links among any documents to which a user has read-access, typically storing link data as metainformation separate from the document's text, e.g. in a DBMS. This more open access requires some sort of "creator-private vs public" visibility control of links, as well as functionality to enhance document readability (decluttering of link mark-up), including filtering options available to any reader for turning off display of undesired links. 3J5
  The Brown University Intermedia hypertext system is an example of one such modern system which supported all of the above desirable linking characteristics. 3J5A

• A subtle but significant feature of Augment links is that they are implemented as human-readable text. This provides immediate cues as to the content and whereabouts of the referenced item. It also supports manual following of the link when reading hardcopy e.g. in draft review commentary. There is the additional immediate benefit that usable links can be sent in e-mail over the internet, to other Augment systems. 3J6
  Many if not most modern hypertext systems provide only an iconic cue/control for a link. Future systems should support optionally readable display of the link information within the document. This would support working with hardcopy printout, and in stand-alone portable PC applications (such as Augment's MiniBase), allowing link references to be read and created even though link referents may well not be available in the very small subset of documents currently accessible. 3J6A

• Finally, Augment links support referencing material over a wide area network. The link specification has an optional field for specifying the host computer node of another Augment system on which the referenced material resides. Following the link returns the referenced item transparently. 3J7

Unique Results 4

Task 1. A particular form of organizational tool that was extremely powerful was the "Locator File". This is an outline file structured by the major topic areas of the particular area of information to be organized. Within each topic area are hypertext links leading off to specific items in other files and directories, along with comments as to what each item is about. This can be thought of as the master index or catalog for some collection of information, or as a generalization of the file directory or folder concept allowing more flexible and fine-grained organization. 4A

For example, the scenario project task had its own locator file with sections including: Project Synopsis, Staff, Concept of Operations, Requirements, Design Discussions, Meeting Notes, Status Reports, Test and Evaluation Records, Problem Reports, discussions of Future Enhancements, Documentation and Code, and the Project Glossary. Each section contained commented links to all significant related items of information in the system. This served as a "project handbook" containing pointers to all of the up to date information about the task, and so captured its current state. 4B

Another locator file was developed for the project as a whole (which consisted of four such tasks). It contained sections such as Background (with links to customer contact reports and concept papers), Contractual Documentation (with links to the original statement of work and customer ECPs), Project Plan (linking to the work breakdown structure, schedules, and budgets), project Tasks (with links to each task's locator file), and links to project level Status Reports. 4C

Finally a skeletal corporate-level locator file was created, to be suggestive of enterprise integration. It contained sections for ongoing and completed Projects (with links to individual project's locator files), similarly for IR&D projects, Personnel (which would link to an extended company phone book - including individuals' history of projects, experience, tool/technology expertise, etc.), Organization Chart (which would be another structuring of the "phone book" information), Quality Circles, company Forms Database, Technology Repositories (design and module libraries, system development methodologies, etc.), and External Intelligence (links to publicly available information about various technologies, business partners, and competitors). 4D

Task 2. Augment has a large amount of unique functionality that would be useful in current systems. 4E

Augment provides an integrated environment; i.e. a given capability is available to the user in a uniform fashion wherever it makes sense. Various document addressing styles are uniformly available wherever addresses are called for in commands, links, or scripts. Likewise the content filtering patterns and window view specifications (discussed below) are available not just for window set-up, but in commands (e.g. Copy Filtered), and in links (activated on the window in which the referenced information is displayed when the link is followed). 4F

Though a number of integrated systems have existed in the past, most are relatively narrowly targeted, e.g. to software development environments or office automation. Augment's environment is very broadly aimed at collaborative knowledge work. 4G

Augment provides a document-centered environment as opposed to tool centered. Any tool can be applied to any document to the extent it understands the operand (content) specified in the requested operation. In modern systems, window and tool functionality as well as document structure are all enmeshed in, and hidden by, the specific tool which "owns" them. In Augment, the functionality of windows, tools, and documents is orthogonal. Windows are not tied to tools and are relatively long-lived. They are in charge of (flexible) display of information from any document, and providing access to its content to any tool that the user wants to bring to bear on the information. This decoupling of responsibilities among documents, tools, and windows creates extremely high interoperability among tools (data), and allows greater flexibility in creating powerful capabilities at the user level. 4H

This architecture is more general than current attempts at a "document-centered paradigm" such as the Object Linking and Embedding architecture. In OLE, though one can embed items (objects) from one kind of tool into another tool's document, the former tool is still the only one that can operate on it. In fact, Augment provides a more general and flexible linking and embedding capability with its basic hypertext linking functionality and "include links" respectively (discussed below), though with few data types. 4I

The user has complete control of the viewing format of the information in a window, independent of the application program. There are per-window settings for the level of detail to be displayed for any file from any application that is displayed in the window. Also outline addresses, structure indenting, and labels (bookmarks) may be turned off and on at will. 4J

In conjunction with these window "view specifications", the user can create and activate/deactivate "content filters". There is a sophisticated string pattern matching / generating language which allows the user to specify that any information containing specific patterns is to be presented to, or filtered from, the window. E.g. one can easily specify filters to present only messages from specific users or about specific subjects when viewing large e-mail archives. 4K

In addition to the content filtering mechanism, which sits between a document's internal representation and the window in which it is displayed, an underlying "sequence generator" mechanism is available for programmable control. A sequence generator module passes the "next" portion of the underlying document structure up to the content filter and window when requested by the latter for display. What is "next" (and how it is to be displayed) is a decision of the sequence generator (and only later mediated by content filtering and extant window view specifications). The default sequence generator delivers standard outline structure for display. Other sequence generators could be developed for uniquely handling hypertext webs or trails, or mathematical sets of information for example. A (non-default) sequence generator is activated in a particular window as part of the latter's view specifications, and so is easily and uniformly controllable in user commands, scripts, and link specifications. 4L

The sequence generator and content filter provide a large domain of very general and powerful dynamic display functionality that is completely nonexistent in current commercial or research systems. 4M

One such sequence generator supports "include links". When activated by the user (on a per window basis) the include-link specification (the reference) is replaced with the information to which it points. This transformation occurs dynamically in the window display only and does not affect the document's contents (which still contains only the include-link specification itself). However, the information is available for editing and printout as viewed. These are known as hot-links or transclusion links in the hypertext literature, and provide the above-mentioned "embedding" functionality analogous to OLE, though with more flexibility. This is a powerful capability which supports different views or organizations of the same information in different windows or documents, supporting more sophisticated group analysis and reuse of information than ubiquitous copying (where some copies grow stale as others are updated). 4N

In order to reference and reuse information effectively, a large number of naming (addressing) capabilities are supported which allow specification of the exact location of referenced information. Every document is hierarchically (outline) structured, though this capability need not be used or viewed if not desired. Each item in an outline has two addresses. One is its relative address in the document's structure (e.g. II.C.4.e.vi). Over the life of a document, this address always resolves to that spot in the document - whatever its contents. This type of address is used to naturally refer to sections of a document in the read-only Journal document library (whose contents can no longer be rearranged). 4O

The second type of addressing is a unique id assigned to an item when it is created in a document. It stays with the associated content over its life span, regardless of where it may move in the document's overall structure. This form of address would be used to refer to specific content in a working draft (which could frequently move within the document's structure). For example, it would be used in a link when making some comments on a specific item in a draft. 4P

In the course of this study, our newly developed link creation commands were programmed to automatically use outline structure addresses for references into read-only documents (e.g. in a Journal), and use unique ids in editable documents. 4Q

Viewing specifications support easily turning on and off either kind of addresses when viewing a document, and are available in non-outline-indented hardcopy printout also (as unobtrusive tags in the right margin if desired). Hence this supports manually following a link as read in hardcopy documents. 4R

Another kind of addressing is supported with named labels which serve as bookmarks, placed at the beginning of an item. Labels also may be hidden from viewing while still supporting references. Finally, relative addressing is supported, e.g. allowing a reference to the third item following or preceding some given address in the outline. All addressing forms and features are available ubiquitously in commands, scripts, and links. 4S

As a navigation aid, per-window visitation trails are maintained by the system so that a user can jump back to any file at the position it was once viewed. This is a necessary backtracking capability, frequently needed after following some sequence of links. Many hypertext systems have a link-oriented backtracking capability. Augment's is uniquely generalized to support backtracking through any and all viewing of information in a given window, regardless of what functionality was originally used to display it (following a link, opening a file, or directing display of documents' contents to specified windows with the mouse (an operation known as "jumping", and which includes the Augment form of scrolling). In a next-generation system, one would want to be able to save, edit, link, mail, and generally interoperate with these visitation trails. In Augment they can only be displayed, and a single location chosen to return to. 4T

Another unique window feature is the ability, while working on one document, to place specified text items from the same or other documents in a flexibly-sized demarcated region at the top of the window. These items stay put in that region as the main document scrolls below. Items are easily added and removed from this region via the Freeze and Unfreeze commands. This freezes the item's position in that portion of the window, not its content. It can still be edited if desired (if access protections allow) just like any other text in a window, and the containing document is updated concurrently. The frozen items are subject to the same view specifications as the rest of the window. 4U

This feature allows quick reference to, or update of, key items. In modern systems, one would basically have to have a separate window for each item, which could be awkward to set up. 4V

Collaborative real-time screen sharing is provided as a meta tool. I.e. it is transparently operable independent of the normal Augment applications with which users are currently working. This allows full operability of the Augment environment with this capability. Many commercial screensharing applications require entering a unique application program or mode where other applications' functionality or information is not available. 4W

In Augment, one user controls the screen with mouse and commands, but this control can be passed to any other participating user in turn. Anyone may type comments at any time, which are displayed in a special window of the shared screen along with the identity of the commentor. Collections of users from any number of geographically distributed Augment host computers may participate in the same screen sharing session, viewing and interacting with the same information. With simple conference-phone support, this provides a very powerful collaborative capability that studies have shown is as effective as video-conferencing for detailed discussions, while being much less resource intensive (see Chapter 6 of Greif's book, referenced below). 4X

Future systems should also support private windows, e.g. where each participant could take their own notes during the conferencing session. 4Y

Augment has formal support for "roles". These are basically password protected accounts which represent some formal activity in an organization (e.g. "Whirlwind Project Manager", or "Graphics Lab Database Administrator"). One uses the Act command to take on a role. Encrypted electronic signatures are supported on (portions of) documents. When someone other than the role's owner signs, the system identifies the "acting for" status of the current user. 4Z

Augment supports an extremely fast HCI. It is command-line oriented, but with single-character command abbreviations entered with one hand on a five-key chordset in parallel with operand selection with the mouse in the other. This allows attention to be continuously fixed on the information content, with commands being issued "in the background" (requiring minimal attention). This is in contrast to modern moused-menu systems where attention must be alternately refocused to the mouse to navigate through command menus, or to the keyboard to select a command-key, option-key, or function-key "short cut". One must learn the chord combinations for the alphabet, numbers, and punctuation, but this is not difficult. Formal studies have shown that novices are facile within about two hours of dedicated practice. Keyboard entry is available if the chordset is not used. The keyboard is used for text entry of more than a few characters in any case. Also, context-sensitive help or mouseable pop-up menus are available at any point by entering a single "?" character. 4AA

Additionally, any command that can be entered directly by the user, can be specified (in the same language) in a command script, along with additional control structure if desired (testing for conditions, and altering the flow of the script based on the result), an important capability missing or deficient in all modern GUIs. 4AB

In order to address the "closed-world" issue, the Augment environment supports flexible import and export facilities (e.g. to DOS files). More significantly, it provides a "Reach" application which (with programmer extension) allows Augment users to operate and capture information from non-Augment tools (e.g. a commercial DBMS or CAD system). The intent is to allow interoperability with expensive commercial or legacy applications that would be impractical to port into the Augment environment. This allows capture of their value-added information processing capabilities, bringing desired results back into the Augment environment where they can be integrated into the collaborative information environment and reused. 4AC

A major capability for the management of shared information is provided by the Augment journaling facility. A journal is a named permanent archive or library of read-only copies of submissions. A submission is given a series of attributes (author, submittor, date, keywords) which can later be searched. Additional attributes specify if the item is Part-of, In-reply-to, an Addendum-to, or Supercedes a prior submission (and the prior submission receives cross reference links to the new submission in its Parts, Replies, Addenda, or Superceded-by fields respectively) providing valuable in-library structuring of related information. Then, for example, if a link is followed to a superceded item, the system asks the user if they want to go to the latest version, or to the original. 4AD

The journal is highly integrated with e-mail and work process conventions. One makes submissions using e-mail, at the same time automatically sending a link to it to everyone on the distribution list. Users are automatically notified by e-mail if a new version is submitted. 4AE

Items which have been unreferenced for a long (or specified) time are automatically migrated off-line to free up storage, but their searchable catalog entry remains. When following a link to an off-line item, the system informs the user of the situation, at which point they may abort the link-following operation. Otherwise e-mail is automatically sent to the system administrator describing what archive tape to mount. When the system has retrieved the information, it automatically sends e-mail to the requestor with a link to it. 4AF

The journal is also used to catalog hardcopy items (e.g. conference proceedings). The catalog entry is keyworded and searched like any other, and typically contains a brief description of the worth of the item and its physical location. 4AG

Augment allows a user to access or subscribe to journals on other Augment host computer nodes in a local or wide area network. The local Augment system caches requested items from the remote nodes, and provides them transparently to the user. 4AH

A final feature to be mentioned is the automatic tagging of each sentence / paragraph of a document with the date and name of the last person to change it. This capability has already been found useful in software maintenance situations, including more fine-grained configuration management of items within source code files. In future systems it would serve as the foundation for a change detection and notification capability. This functionality is of great importance in large shared information environments, where users need to be able to quickly discern what information has changed since last seen. For particular items, users would want automatic notification that a (specified portion of a) file, directory, or linking structure has been updated, and further support for evaluating exactly how the item has changed. 4AI

Summary 5

Despite all of its powerful unique functionality, the superior effectiveness of the Augment system derives from more general concerns. It provides a highly integrated environment with complete data interoperability among tools, and provisions for interoperating with tools (not just data) outside of its environment. This allows different classes of users to work with the same shared data, using their own tools in a manner suited to their tasks and roles in the organization. The system is extensible at the user level in functionality and vocabulary, which encourages work practice evolution for individuals and teams. When these capabilities are applied over a large and expandable tool set, extensive coverage of a broad range of users' daily tasks is provided, and the promise of enterprise integration becomes a realistic possibility. 5A

The Augment system is still a rich source of implemented and suggested ideas for useful functionality in systems that support information-intense collaborative work. To the extent that future systems can integrate these ideas in a uniform, interoperable, extensible manner in support of evolving work practices, organizations which depend upon extensive knowledge work over large and changing information bases and which integrate such systems into their corporate culture can expect dramatic improvements in quality and efficiency of operation. 5B

References 6

There are many other aspects of the Augment system, as well as motivations and details for some of the above, which are described in: 6A

• Computer-Supported Cooperative Work: A Book of Readings, I. Greif (Ed.), Chapters 2-5, Morgan Kaufman Pub., 1988, ISBN 0-934613-57-5. 6A1

• The Augmentation Papers: A Collection Since 1960, D.C. Engelbart, available from the Bootstrap Institute 510.713.3550. 6A2

For comparison, the Brown University Intermedia system is described in: 6B

• IRIS Hypermedia Services by B. Haan, Communications of the ACM, Jan. '92, Vol. 35 #1, pp. 36-51. 6B1

• Intermedia: The Concept and the Construction of a Seamless Information Environment by N. Yankelovich, IEEE Computer magazine, Jan. '88, Vol. 21 #1, pp. 81-96. 6B2

An excellent set of top-level requirements for system support of collaborative knowledge work is given in: 6C

• Toward High-Performance Organizations: A Strategic Role for Groupware, D. C. Engelbart, appearing in the *GroupWare '92: conference proceedings pp. 77-100, Morgan Kaufman Pub. This paper is included in The Augmentation Papers collection (above), and in the following. It also provides a terse overview of the broader context that is motivated more fully in the following. 6C1

For a more holistic view of where and why computer system support (such as that provided by Augment) fits into the larger issues of organizational process improvement and enterprise integration, see: 6D

• Bootstrapping Organizations into the 21st Century, the Bootstrap Seminar binder, available from the Bootstrap Institute 510.713-3550. 6D1