The Society Of Mind by Marvin Minsky
Marvin Minsky gives us his take on the question "How does the Mind work ?" in his book Society of Minds. Minsky believes that the intelligence (or not so common 'common sense') of the Mind cannot be explained by some universal all-purpose theory, but by a diverse set of structures, each unique, coming together as a Society - A Society of Minds.
Just like how a complex task is divided into sub-tasks, which is then further divided into sub-sub-tasks and so on until we are left with unit unique indivisible simplest entities, Minsky thinks of the Mind as a bunch of unique simple units called 'Agents' that don't have any intelligence on their own, but come together as a Society of Agents (Agencies). Agents are not intelligent, Agencies are. Agents are unique and specific in that they have a particular function or purpose. Example - Seeing agents, Grasping agents, Moving agents, Releasing agents in the context of a child playing with building blocks.
This division is very similar to computer programs that you and I write - Splitting up a bigger intelligent function into smaller sub-routines or individual lines of code that have minimal intelligence associated with them as single units, but perform an intelligent task as a whole.
But this coming together of Agents into Agencies is not as simple as it sounds - its not objective or as simple as A + B = C. Think about these two questions -
Just like the former objective question can be answered by 'interactions' between the links of the chain, the latter subject question can also be answered by interactions. Agents interact with each other through connections. There is an organization to Agents.
Minsky believes that Agents interact with each other in patterns. Example - Right now, my Work agents are exploiting my Fear agents (to turn in this write-up) to overcome Sleep. These patterns of interaction are what I believe gives every mind some form of identity - A character, A Self. This Self helps keep Agencies in track, in a path, and helps avoiding Minds from changing too rapidly. It would be interesting to use some of the most common of these patterns while designing the interface of a system. For example - Finding ways in which Grasping agents interact with other agents might enable a designer to desing a good handle to graps or a hand for a robot that grasps
Agents also communicate by sending signals and recognizing signs. Some of these signals are interpreted sub-consciously, inherently by Agents and some are explicit. This internal communication may have multiple signal/language levels where a high-level agency may not be able to understand a low-level agency's (sub-machine) signal or language. For example - How can you explain how you balance your bi-cycle? If an external system (such as a User Interface) can communicate to a user's mind using these same signals (something as simple as using the color red to communicate danger or caution), the User could possibly communicate with the system sub-consciously
Moving on to explore perceptual experiences in the mind, Minsky talks about every image or piece of information in our mind as a 'frame' - a template or a skeleton for stored information that is like an empty form with 'terminals' in them to hold the unit level pieces of information. Example - A frame to represent a person consists of terminals that contain the body, legs,hands etc. These frames can come together in a scene that was evoked by the mind perceiving something as simple as a small phrase such as - "its raining frogs". Minsky explains the process of how words, phrases, sentences, stories stir frames into activity and how the mind immediately perceives the entire scene by filling up existing frames with new terminals or creating new frames entirely.This perception of language, particularly identifying frames and associating them, is very similar to the Association step of Brad's 7-Step Knowledge Acquisition Pipeline. When one reads or hears a sentence (when the Expert talks about a Task or a sub-task), this process of immediately associating images to words and constructing a scene can be understood easily by Minsky's frame theory.
Spatial Schemas in Depictions by Barbara Tversky
Tversky describes and supports with examples the reasons for using visual representations and graphic tools as modes for conveying meaning. She suggests that graphic inventions (sign-boards, data charts etc) convey meaning by using space and elements naturally. Spatial relations are metamorphically used to convey relations based on closeness, at nominal ordinal and interval levels for example. Graphics attracts attention, augments memory by facilitating inference & discovery, and makes communication and understanding easier.
Such tools were employed as a means of communication since ancient history when no written language existed (like they say a Picture is worth a thousand words). Pictures represent meaning more directly and are easily comprehensible and hence the olden day petroglyphs can still be inferred in today’s time without much effort. In contrast to ancient depictions, most of the modern day depictions are visualizations of things that are not inherently visualizable - such as temporal, quantitative, casual or social relations. These depictions rely more on analogy rather than schematic miniaturization or enlargement. Examples of contemporary depiction tools include graphs, tables, icons, charts, diagrams etc.
The way graphics convey meaning is by its natural cognitive correspondence to language and gestures in real life. As graphics displays are external they augment many of the functions by reducing demand on memory and thus facilitating information processing, improving communication by letting several people simultaneously inspect the same graphic display. Spatially organized information is easier to access, operate on and integrate. The main elements of a graphic displays are:
- Figures of Depiction
- Spatial Arrays of Elements
In representing complex concepts, only icons may sometimes fall short, so figures of depiction are used instead. The two common types of figures of speech are Metonymy - a picture of a file represents a folder in computer interfaces and Synecdoche, where a part is used to represent a whole. Examples are icon of a gas pump on highway to depict a nearby gas station, emoticons in computer vernacular etc. The main idea is that Icons can be read by people who do not read the local language.
These are graphs, charts and diagrams which convey qualitative and quantitative information. These convey meaning using natural correspondence and spatial metaphors with different levels of precision, corresponding to 4 traditional scale types - nominal, ordinal, interval and ratio.
Nominal Scales essentially depict clustering by category usign the norms such as indentation, lists, tables, parentheses, circles, boxes, Venn-diagrams, color coding, font, etc.
Ordinal Scale represent relations along with the order and direction of the relation using lines, arrow, highlighting, super-positioning, warm colors, cold colors, etc.
Interval and Ratio relations apply more constraints of the spatial proximity than ordinal scales for eg. distance between elements. Common examples of these are X-Y plots, musical notation, pie charts, etc.
As perceiving and comprehending depth information from flat images is somewhat challenging, the concept of 3D graphics and animation comes in handy. Since animation entails changes in graphics over time, actions and processes which are inherently dynamic can be represented using animation. But compared to an animation, static diagrams offer the flexibility of re-inspect-ibility and determining inter-relationship.
Tversky's study helps us identify ways of exploiting visual representation and graphic displays to facilitate a User's interaction with an interface, by conveying information more intuitively.