Forms of Presentation and the Future of Comprehension

Use of interactive graphic display techniques

Anthony Judge


The suggestion has been made (see previous section) that structuring the relationship between theoretical entities (concepts, propositions, problems, etc.) could best be accomplished using graph theory methods. There are three disadvantages to this approach:

  • graphic relationships are tiresome and time-consuming to draw (and are costly if budgeted as "art work").
  • once drawn, there is a strong resistance to updating them (because of the previous point) and therefore they quickly become useless.
  • when the graph is complex, multidimensional, and carries much information, it is difficult to draw satisfactorily in two dimensions. The mass of information cannot be filtered to highlight particular features - unless yet another diagram is prepared.

These three difficulties can be overcome by making use of what is known as "interactive graphics" (27). This is basically a TV screen attached to a computer. The user sits at a keyboard in front of the screen and has at his disposal what is known as a light-pen (or some equivalent device) which allows him to point to elements of the network of concepts displayed on the screen and instruct the computer to manipulate them in useful ways. In other words the user can interact with the representation of the conceptual network using the full power of the computer to take care of the drudgery of

  • drawing in neat lines
  • making amendments
  • displaying only part of the network so that the user is not overloaded with
  • "relevant" information

In effect the graphics device provides the user with a window or viewport onto the network of concepts. He can instruct the computer, via the keyboard, to:

1. move the window to give him, effectively, a view onto a different part of the network - another conceptual domain

2. introduce a magnification so that he can examine (or "zoom in" on) some detailed sections of the network

3. introduce diminution so that he can gain an overall view of the structure of the conceptual domain in which he is interested

4. introduce filters so that only certain types of relationships and entities are displayed - either he can switch between models or he can impose restrictions on the relationships displayed within a model, i.e. he has a hierarchy of filters at his disposal.

5. modify parts of the network displayed to him by inserting or deleting entities and relationships. Security codes can be arranged to that (a) he can modify the display for his own immediate use without permanently affecting the basic store of data, (b) he can permanently modify features of the model for which he is a member of the responsible body, (c) and so on.

6. supply text labels to features of the network which are unfamiliar to him. If necessary he can split his viewport into two (or more) parts and have the parts of the network displayed in one (or more) part(s). He can then use the light pen to point to each entity or relationship on which he wants a longer text description (e.g. the justifying argument for an entity or the mathematical function, if applicable, governing a relationship, and have it displayed in an adjoining viewport.)

7. track along the relationships between one entity and the next by moving the viewport to focus on each new entity. In this way the user moves through a representation of "semantic space" with each move, changing the constellation of entities displayed and bringing new entities and relationships into view.

8. move up or down levels or "ladders of abstraction". The user can demand that the computer track the display (see point 7) between levels of abstraction, moving from sub-system to system, at each move bringing into view the semantic context of the system displayed.

9. distinguish between entities and relationships on the basis of user-selected characteristics. The user can have the "relevant" (to him) entities displayed with more prominent symbols, and the relevant relationships with heavier lines.

10. select an alternative form of presentation. Some users may prefer block diagram flow charts, others may prefer a matrix display, others may prefer Venn diagrams (or "Venn spheres" in 3 dimensions) to illustrate the relationship between entities. These are all interconvertible (e.g. the Venn circles are computed taking each network node as a centre and giving a radius to include all the sub-branches of the network from that node).

11. copy a particular display currently on the screen. A user may want to keep a personal record of parts of the network which are of interest to him. (He can either arrange for a dump onto a tape which can drive a graph plotter, a microfilm plotter, or copy onto a videocassette, or, in the future, obtain a direct photocopy.)

12. arrange for a simultaneous search through a coded microfilm to provide appropriate slide images or lengthy text which can in its turn be photocopied.

13. simulate a three-dimensional presentation of the network by introducing an extra coordinate axis.

14. rotate a three-dimensional structure (about the X or Y axis) in order to heighten the 3-D effect and obtain a better overall view "around" the structure.

15. simulate a four-dimensional presentation of the network by using various techniques for distinguishing entities and relationships (e.g. "flashing" relationships at frequencies corresponding to their importance in terms of the fourth dimension.)

16. change the speed at which the magnification from the viewport is modified as a particular structure is rotated.

17. simulate the consequences of various changes introduced by the user in terms of his conditions. This is particularly useful for cybernetic displays.

18. perform various topological analyses on particular parts of the network and display the results in a secondary viewport (e.g., the user might point a light-pen at an entity and request its centrality or request an indication of the interconnectedness of a particular domain delimited with the light pen.).

In every current use of interactive graphics there is some notion of geometry and space, but the geometry is always the three-dimensional conventional space. There is no reason why "non-physical spaces" should not be displayed instead and this is the domain of topology. The argument has been developed by Dean Brown and Joan Lewis (28).

"Both geometry and topology deal with the notion of space, but geometry's preoccupation with shapes and measure is replaced in topology by more abstract, less restrictive ideas of the qualities of things...Being more abstract and less insistent on fine points such as size, topology gives a richer formalism to adapt as a tool for the contemplation of ideas....

Concepts can be viewed as manifolds in the multidimensional variate space spanned by the parameters describing the situation. If a correspondence is established that represents our incomplete knowledge by altitude functions, we can seek the terrae incognitae, plateaus, enclaves of knowledge, cusps, peaks, and saddles by a conceptual photogrammetry. Exploring the face of a new concept would be comparable to exploring the topography of the back of the moon. Commonly heard remarks such as "Now I'm beginning to get the picture" are perhaps an indication that these processes already play an unsuspected role in conceptualization....

By sketching tentative three-dimensional perspectives on the screen and "rotating thesis on the tips of his fingers", one internalizes ideas non-verbally and acquires a sensation of sailing through structures of concepts much as a cosmonaut sailing through constellations of stars.

Such new ways of creating representations break ingrained thought patterns and force re-examination of preconceived notions. A mapping is a correspondence is an analogy. Teaching by analogy, always a fertile device, can be carried out beautifully by topological means....Topological techniques are useful at even the most advanced levels of scientific conceptualization...."

The fundamental importance of interactive graphics, in whatever form, is its ability to facilitate understanding. Progress in understanding is made through the development of mental models or symbolic notations that permit a simple representation of a mass of complexities not previously understood. There is nothing new in the use of models to represent psycho-social abstractions. Jay Forrester (29), making this same point with respect to social systems, states

"Every person in his private life and in his community life uses models for decision making. The mental image of the world around one, carried in each individual's head, is a model. One does not have a family, a business, a city, a government, or a country in his head. He has only selected concepts and relationships which he uses to represent the real system. The human mind selects a few perceptions, which may be right or wrong, and uses them as a description of the world around us. On the basis of these assumptions a person estimates the system behaviour that he believes is implied....The human mind is, excellent in its ability to observe the elementary forces and actions of which a system is composed. The human mind is effective in identifying the structure into which separate scraps of information can be fitted. But when the pieces of the system have been assembled, the mind is nearly useless for anticipating the dynamic behaviour that the system implies. Here the computer is ideal. It will trace the interactions of any specified set of relationships without doubt or error. The mental model is fuzzy. It is incomplete. It is imprecisely stated. Furthermore, even within one individual, the mental model changes with time and with the flow of conversation. The human mind assembles a few relationships to fit the context of a discussion. As the subject shifts, so does the model. Even as a single topic is being discussed, each participant in a conversation is using a different mental model through which to interpret the subject. And it is not surprising that consensus leads to actions which produce unintended results. Fundamental assumptions differ but are never brought out into the open."

These structured models have to be applied to any serially ordered data in card files, computer printout or reference books to make sense of that data. Is there any reason why these invisible structural models should not be made visible to clarify differences and build a more comprehensive visible model? The greater the complexity, however, the more difficult it is to use mental models. For example, in discussing his examination of an electronic circuit diagram, Ivan Sutherland writes (30):

"Unfortunately, my abstract model tends to fade out when I get a circuit that is a little bit too complex. I can't remember what is happening in one place long enough to see what is going to happen somewhere else. My model evaporates. If I could somehow represent that abstract model in the computer to see a circuit in animation, my abstraction wouldn't evaporate. I could take the vague notion that "fades out at the edges" and solidify it. I could analyze bigger circuits. In all fields there are such abstractions. We haven't vet made any use of the computer's capability to "firm up" these abstractions. The scientist of today is limited by his pencil and paper and mired. He can draw abstractions, or he can think about them. If he draws them, they will be static, and if he just visualizes them they won't have very good mathematical properties and will fade out. With a computer, we could give him a great deal more. We could give him drawings that move, drawings in three or four dimensions which he can rotate, and drawings with great mathematical accuracy. We could let him work with them in a way that he has never been able to do before. I think that really trig gains in the substantive scientific areas ore going to come when somebody invents nets abstractions which can only be represented in computer graphical form."

Implications of computer augmentation of intellect

There are important intellectual implications emerging from work on advanced computer systems. Of particular interest is the work of Douglas Engelbart's team at the Center for Augmentation of Human Intellect (Stanford Research Institute) which is the centre for the U.S. ARPA Data Network (which links the computers of major universities in the U.S.A.) Engelbart has worked on the means of creating an intellectual workshop to facilitate interaction between conceptual structures. The following extracts are from Engelbart, D.C., Augmenting Human Intellect; a conceptual framework. (32)., He considers that:

"Concepts seem to be structurable, in that a new concept can be composed of an organization of established concepts and that a concept structure is something which we might try to develop on paper for ourselves or work with by conscious thought processes, or as something which we try to communicate to one another in serious discussion....A given structure of concepts can be represented by any of an infinite number of different symbol structures, some of which would be much better than others for enabling the human perceptual and cognitive apparatus to search out and comprehend the conceptual matter of significance and/or interest to the human.

But it is not only the form of a symbol structure that is important. A problem solver is involved in a stream of conceptual activity whose course serves his mental needs of the moment. The sequence and nature of these needs are quite variable, and yet for each need he may benefit significantly from a form of symbol structuring that is uniquely efficient for that need.

Therefore, besides the forms of symbol structures that can de constructed and portrayed, we are very much concerned with the speed and flexibility with which one form can be transformed into another, and with which new material can be located and portrayed.

We are generally used to thinking of our symbol structures as a pattern of marks on a sheet of paper. When we want a different symbol-structure view, we think of shifting our point of attention on the sheet, or moving a new sheet into position.

With a computer manipulating our symbols and generating their portrayals to us on a display, we no longer need think of our looking at the symbol structure which is stored -- as we think of looking at the symbol structures stored in notebooks, memos, and books. What the computer actually stores need be none of our concern, assuming that it can portray symbol structures to us that are consistent with the form in which we think our information is structured.

A given concept structure can be represented with a symbol structure that is completely compatible with the computer's internal way of handling symbols, with all sorts of characteristics and relationships given explicit identifications that the user may never directly see. In fact, this struc turing has immensely greater potential for accurately mapping a complex concept structure than does a structure an individual would find it pr tical to construct or use on paper .

The computer can transform back and forth between the two-dimensional portrayal on the screen, of some limited view of the total structure, and the aspect of the n-dimensional internal image that represents this "view". If the human adds to or modifies such a "view", the computer integrates the change into the internal-image symbol structure (in terms of the computer's favored symbols and structuring) and thereby automatically detects a certain proportion of his possible conceptual inconsistencies.

Thus, inside this instrument (the computer) there is an internal-image, computer-symbol structure whose convolutions and multi-dimensionality we can learn to shape to represent to hitherto unattainable accuracy the concept structure we might be building or working with. This internal structure may have a form that is nearly incomprehensible to the direct inspection of a human (except in minute chunks)".

These insights have been incorporated into the design of an operational computer system which is now being developed so that it will be possible to use computer devices as a sort of:

"electronic vehicle with which one could drive around with extraordinary freedom through the information domain. Imagine driving a car through a landscape which, instead of buildings, roads, and trees, had groves of facts, structures of ideas, and so on, relevant to your professional interests? But this information landscape is a remarkably organized one; not only can you drive around a grove of certain arranged facts, and look at it from many aspects, you have the capability of totally reorganizing that grove almost instantaneously. You could put a road right through the center of it, under it, or over it, giving you, say, a bird's eye view of how its components might be arranged for your greater usefulness arid ease of comprehension. This vehicle gives you a flexible method for separating, as it were, the woods from the tiess." (33)

Clearly some possibilities of this system could be used to explore the concept structures resulting from this project.

Relationship to artificial intelligence projects

In considering the possibility of coding definitions of concepts, propositions and like entities, it is important to benefit as much as possible from related work on artificial intelligence, and possibly pattern recognition. Artificial intelligence projects to simulate human personality or belief systems have had to develop methods and computer techniques which can handle and interrelate entities such as concepts and propositions. Clearly the object of such projects is not attained once an inventory of entities can be examined, even if it is highly structured in the form of a thesaurus. It is therefore interesting to look at both the techniques used to handle concepts and the types of computer based interrogations that are then possible (34-37).

The suggestion that techniques of handling individuals' "beliefs" should have some parallel to a community of scholars' attitudes towards the concepts, propositions, etc., which constitute its territory, may appear somewhat provocative. Does a school of thought constitute a belief system?

T.S. Kuhn (38) uses the terms "belief", "metaphysic", "commitment", and "conversion" in connection with a scientific community's attitude towards a paradigm and paradigm change. It might be useful for disciplines to examine their own conceptual structures in the same way as an aid to the development of the discipline. It could be particularly important as a means of highlighting tensions within the conceptual structures which lead up to Kuhn's paradigmatic changes.

This approach suggests a number of stages of sophistication in the possible development of this project.

  • A static inventory of concepts and propositions
  • A static network of interrelated concepts and propositions
  • "Activation" of propositions as rules governing the relationships between entities
  • Treatment of a school of' thought as a belief system
  • Extension to natural language interaction

On this last point, it may be possible to allow a (non-computer-oiiented) specialist in a particular field to "dialogue" with the concept data base to permit him to discover and indicate where he differs from its contents and what new he thinks should be included (39). This approach might be a useful method of getting around the behavioural problems associated with the power position of official classifiers in committees.
It may eventually be possible to have many such people interacting in natural language wi - th the data base via terminals to facilitate communication (e.g., at a special seminar).