1. Introduction
1.1 A Brief History 1.1.1 Artificial Languages 1.1.2 Non-Artificial Languages 1.1.3 Visual Languages 1.2 The Organization of the Proposal 1.3 Expected Contributions 1.4 Criteria for a new language 2. Universal Languages 2.1 Pidgins and Creoles 2.1.1 Simplified Speech 2.1.2 Pidgins and Creoles 2.2 Basic English 2.2.1 Arguments against Basic English 2.3 Visual Communication Languages 2.3.1 Why a Visual Interlanguage? 2.3.2 Background on Semiotics and Icons 2.3.3 Visual Communication Systems (Pictorial and Iconic) 2.3.4 CD-Icon: an Iconic Communication Language 3. Language Design Semantics 3.1 Principles of VIL 3.1.1 Principles 3.1.2 Parts of Speech 3.2 Grammar 3.2.1 Organization of the Iconicon 3.2.2 VIL’s Grammar 4. Visual Representation of the Language 4.1 Icon Design 4.1.1 Improving Design, Aesthetics, and Context Functionality 4.1.2 Bad Icons 4.2 Designing Cross-Cultural Icons 4.3 VIL’s Icon Language 4.3.1 Determine what needs to be represented 4.3.2 Design Basic Symbols 4.3.3 Rules for Combining Visual Combining Features 5. Evaluation 5.1 The Principles 5.2 Testing the Design of the Language and the Icons 5.2.1 What if the icons are not 100% universal? 5.3 Testing the implementation 5.4 Evaluating viability 5.5 Identify Possible Enhancements and Future Work 6. Schedule 7. Bibliography
In Europe, it is planned that the ECU will become a currency that will be recognized in all countries in the European Community. By means of this currency, people will have, even in the smallest and most remote places, funds for immediate use. The intrinsic value of their national currency will be just as great as it ever was, but because of the difficulties and delays of exchange, an international currency medium which is immediately available and which passes at a definitely understood and agreed upon rate, will be a great time saver and aid to travel.
Does this situation parallel that in languages? By means of an international auxiliary language, man could have the benefit of an exchange of ideas without the delay and inconvenience of translation. The exchange of ideas would then become immediate and in terms which would be commonly understood.
The urge for a simple language, easy to learn, and the desire to communicate with people speaking a foreign language made itself distinctly felt about three hundred years ago. Since then people have speculated on the possibility of creating and adopting a universal language.
This dissertation is concerned with building such a language.
In the past several attempts have been made to create a universal language. Most of these languages were artificial languages, but also some existing languages have been proposed as a universal language. The attempts for universal languages can roughly be divided into three groups: artificial languages (i.e. written, non-visual), non-artificial languages, and the visual languages.
It has been suggested that the best plan to get a world language adopted would be to have some great living language, e.g. English, to absorb all the other languages, since English is already the native tongue hundreds of millions of people. It might also be Chinese, German, Russian, Italian, Portuguese, or French which served as an international diplomatic language for about 500 years. Some even tried to resuscitate Sanskrit. But each of these languages has its peculiar difficulties. English, for example, has a fairly simple grammar, but its pronunciation and spelling are notoriously irregular. Some internationally minded people have suggested that the leading nations of Europe agree on a common auxiliary language, but nationalism and racial pride are insuperable obstacles. For example, the French are purposely limiting the amount of English on French television, because they want their people exposed to more French.
"The adoption of any modern national language by the common consent of the chief nations is therefore unlikely, as it would confer undue advantages and excite jealousy, however impartial the promoters of the language might be."
Another plan was to create a language ad hoc, a plan already suggested by Descartes in a letter of November 20, 1629, but a practical start of large proportions did not begin until Schleyer initiated Volapü k in 1879. It spread rapidly and soon there were hundreds of Volapü k clubs, a dozen or more periodicals, and 300 to 400 publications in ten countries. Three international congresses were held and, according to Schleyer, two and a half million people were interested in it, but the boom suddenly collapsed. Schleyer remained adamant to any suggestions of reform and his followers left him; however, a committee proceeded to revise Volapü k, and under the leadership of the Russian engineer Rosenberger, it became "Idiom Neutral," which in turn became "Reformed Neutral," which was for the most part Neo-Latin. Thus Volapü k was born in 1879, reached its peak in 1888, and was practically dead in 1890. It has an ingenious grammar, but an artificial vocabulary. Some other artificial languages which either corrected or exaggerated defects of Volapü k were Spelin, Dil, Dilpox, Balla, Tal, and Pankel.
The best known and most successful of the artificial languages is Esperanto instituted in 1887 by the young Jewish oculist Louis L. Zamenhof, a native form Bialystok, Poland, where Jew and German, Pole and Russian jostled each other on the street and spoke mutually unintelligible languages and differed further in religion and custom. His brochure published in 1887 was signed "Dr. Esperanto" meaning Dr. Hopeful. Esperanto spread rapidly throughout Europe and the world; about four thousand books and one hundred periodicals were published in it. Twenty-seven European broadcasting stations were using it in 1927, and several world conferences were held including one at Washington at which it was spoken. However, World War II gave it a serious setback. Critics say it has too many circumflexed or hooded letters, too many sibilants or hushing sounds, and unnecessary accusative endings, and is too Russian in character. Furthermore, many well known words are mutilated and its correlatives are obscure. Others say it looks like spoiled Latin and not unlike a travesty on Italian.
Besides Esperanto, there are five other artificial languages which have won public attention. They are Ido, Esperanto-II, Occidental, Novial, and Latin without inflections. Like Esperanto all have regular grammars and vocabularies based on Indo-European languages. The main descendant of Esperanto, Ido, is an international scientific language based on the leading languages of Europe. Its chief protagonists were Dr. Couturat and M. Beaufront. Ido corrected the main defects of Esperanto, but it began to degenerate in 1914 when an ill advised ten-year period of stability was decreed. In that same year, Dr. Couturat died and no competent leader arose to fill his shoes. Dr. Talmey eventually revised it and it became Arulo, and later Gloro. It has ceased to prosper. About 200 schemes and projects appeared thereafter, but none has attained a great and lasting popularity or importance.
Esperanto-II is a scheme for adapting Esperanto and was promulgated by Dr. Rene de Saussure from Switzerland.
While Esperanto, Esperanto-II, and Ido bear an obvious family likeness, Occidental, advocated in 1922 by Estonian Edgar von Wahl, grew out of efforts to construct a more "natural" simplified language. Novial, of which the author was Prof. O Jespersen, seeks to reconcile certain tendencies of Ido and Occidental while embodying a number of fresh features. Latin without inflections, propagated in 1903 by the Italian Prof. G. Peano, is what its name implies. It uses classical Latin stem for forms, supplemented by a certain number of modern international words, and almost completely discards grammatical paraphernalia.
Besides the artificial languages there have also been attempts to simplify existing languages. These are the Basic languages, of which Basic English has attracted the most popularity. Although, as mentioned, Basic English is not considered an artificial language, it was still designed by someone (C. K. Ogden), and did not grow naturally.
Resembling simplified English, but quite different, are the (English) Pidgins. These are languages that come into existence in multi-lingual areas. They are not made up by any one individual, but develop naturally where people with different linguistic backgrounds come into contact with each other.
Pidgins and Basic English will be discussed in order to show that it is very well possible to communicate using a restricted grammar and a limited vocabulary.
Lastly, there are the visual communication languages. These are languages that do not use words, but solely rely on signs, symbols, pictures, or icons. The Chinese writing system started out as a pictorial language (but too complex to serve as a universal language). The ancient Egyptians with their hieroglyphs and the American Indian also communicated through a pictorial language. We will argue for a universal computer-based iconic language. Visual languages overcome most of the problems that written languages encounter. Furthermore, with the technological developments nowadays (powerful computers, the World Wide Web), it is now possible to design an iconic communication system through which people with different linguistic backgrounds can communicate. This iconic language should be designed to take advantage of the technology. People will be able to communicate with an iconic or "picture-based" language without the need to draw the pictures themselves, since they can choose these picture from the screen.
The proposal consists of five chapters and a bibliography. Chapter one is this introductory chapter. The four remaining chapters can roughly be divided into two parts. Part one only consists of chapter two and concentrates on previous work, and part two consists of chapters three, four, and five, and concentrates on iconic communication. Various principles for VIL will be given throughout the proposal. In order to distinguish our principles from other people’s work, we will by convention number our principles in the following manner:
Principle X.Y: Name of the Principle
..........................
..........................
..........................
where X stands for the chapter number the principle occurs in, and Y stands for the number of the principle in that chapter.
Part I: Universal Languages
In this part of the proposal the non-artificial languages and visual languages, from section 1.1.2 and 1.1.3 respectively, will be discussed. For the non-artificial languages, the emphasis will be on Basic English and Pidgins and Creoles. We will study these in order to show that it is very well possible to communicate using a restricted grammar and a limited vocabulary. The rest of chapter two is about visual communication languages. The languages studied are not all attempts at creating a universal language, e.g. hieroglyphs and Chinese, but are discussed to explore the world of pictorial languages. Traffic signs, which form a language of their own will be addressed as well, and various examples of traffic signs are given in Appendix C. The last part of the chapter discusses icons and computer-based iconic languages, since this is the area that is of most interest to us, and describes a system called CD-Icon. CD-Icon is a computer-based iconic communication language developed at the University of Exeter and is based on Shank’s conceptual dependency theory.
Part II: VIL: Visual Inter Language
This part of the proposal, which consists of chapters three, four, and five, focuses on the development of our iconic communication language. Chapter three discusses the semantics of the language. Various principles for VIL are stated and VIL’s grammar is described in detail in Appendix G. Verbs are researched and a the resulting hierarchy for VIL’s verb system is given in Appendix F. Classifications of the nouns and adjectives are also researched and given in Appendices D and E respectively.
In chapter four, the focus is on the representation of the system. The chapter starts off by discussing icon design, and gives various examples of good and bad icons along with principles to improve icon design. At the end of the chapter the representation for the grammatical entities (e.g. verbs, nouns, adjectives, etc.) will be discussed (i.e. what will be represented visually and what will be represented iconically). We will talk about how to represent abstract versus concrete meaning, and how to represent categories as opposed to instances of a category.
The last chapter, chapter five, proposes an evaluation of the system. It sugggests how the various criteria and principles can be tested. It will also describe how to test the icons that will be developed for VIL.
Our research into universal languages and iconic communication, together with our program that implements an iconic language, will have several contributions:
Before we start by describing some universal languages, we will first give some conditions and criteria that a new world language must conform to.
Richards mentions three conditions that have to be met in order for a world language to be accepted:
Besides the previously mentioned conditions, there are four principles that we consider important when designing a new visual language:
Principle 1.1: Learnability
The language and the use of its delivery system should be easy to learn.
Principle 1.2: Encodability
It should be easy to compose messages using the language and its delivery system.
Principle 1.3: Decodability
Messages composed in the language should be easy to comprehend, i.e. should let the user decode the messages without much effort.
Principle 1.4: Extensibility/Evolvability
The language should have the possibility to be extended and to evolve.
This chapter mainly focuses on other people’s work. It researches Pidgins and Creoles and Basic English to show that it is possible to communicate with a restricted grammar and a limited vocabulary, and to research ways in which these restricted languages can be extended. Pidgins and Creoles will be the topic of section 2.1 and Basic English will be the topic of section 2.2.
In section 2.3 the visual communication languages are described. The section starts with an explanation of why a visual language is a desirable approach and then gives a brief background on semiotics and icons. It will then describe some pictorial and iconic languages. Chinese, hieroglyphs, and traffic signs are divided into three groups: concrete, abstract, and composite signs. Then some attempts at universal languages that did not use computers are described, i.e. Semantography and Isotype. All these are described in Appendix C.
The last subsection of the chapter, section 2.3.4 about CD-Icon, is of great interest to us since it is also an attempt to create a computer-based iconic communication language. CD-Icon was developed at the University of Exeter and is based on Shank’s Conceptual Dependency theory.
Unlike Esperanto or Ido, which are universal languages that are designed a priori by certain people, pidgins and creoles are multilingual communication languages that develop naturally and come into existence out of economic necessity in multilingual areas where people do not speak each others language.
Research on pidgins and creoles with the emphasis on pidgins will show that it is very well possible to communicate with a drastically reduced grammatical structure and lexicon. This will support the design of VIL, which much like pidgins will use the greatest common denominator of several languages. Some of the simplifications used in pidgins (e.g. uninflected forms, no articles, no copula) might be appropriate in a visual communication language like ours.
The notion of simplicity in language is important, since it may be related to theories of language universals and language acquisition. Jacobson and others have assumed that the simpler of two comparable features is likely to be more widespread among languages of the world, and also the earlier acquired in child language development.
One term that comes to mind when discussing multilingual contact situations and pidgins is the term "lingua franca", so let me start by giving some definitions.
It often happens that, to communicate with each other, two or more people use a language whose grammar and vocabulary are very much reduced and which is native to neither side. This is what constitutes a pidgin.
All natural languages have some degree of redundancy. Consider for example the sentence "Les deux grands journaux". Here there are four overt markers of plurality (in written form, three in spoken form). English is a little less redundant than French, but still in "The two big newspapers" there are two markers of plurality. The pidgins Neo-Melanesian (The pidgin English of Papua New Guinea) and Cameroon pidgin are less redundant still. The only mark plurality by the numeral, tupela bikpela pepa and di tu big pepa (resp. two big newspaper and the two big newspaper).
A creole arises when a pidgin becomes the mother tongue of a speech community. The simple structure of the pidgin is carried over into the creole, but since a creole, being a mother tongue, must be capable of expressing the whole range of human experience, the lexicon is expanded and frequently a more elaborate syntactic system evolves.
This extension comes in the form of reduplication, word compounding, and using verbs as nouns (see Appendix A for a detailed discussion on how to extend a pidgin).
Pidgins generally have the following two functions:
So, a pidgin serves the function of facilitating mutual understanding on a limited range of topics. They usually do not serve the following functions:
Much of the complexity of native languages is associated with the expression of these latter functions.
A pidgin, however, would never be accepted as a universal language, because a lot of people think that pidgins are socially inferior. It does, however, show that it is very well possible to communicate with a drastically reduced grammatical structure and vocabulary.
Basic English, designed by Ogden, is a simplified version of English. Basic English is English made simple by limiting the number of words to 850 and by cutting down the rules for using them to the smallest number necessary for the clear statement of ideas.
Ogden was struck by the fact that whatever you are defining, certain words keep coming back into your definitions. Define them, and with them you could define anything. This suggests that there might be some limited set of words in terms of which the meanings of all other words might be stated. If so, then a very limited language (limited in its vocabulary but comprehensive in its scope) would be possible.
What resulted was the wordlist of 850 words and, more important, the ordered system which restricted their use and idioms to a limited range. Most of the 850 words can be taught in their first and root senses by pictures, pointing, or going through the motions.
The 850 words in Basic English are divided into three main classes. There are 600 names of things, of which 400 are general things and 200 are things that may be pictured. There are 150 names of qualities (‘adjectives"), mostly given in pairs of opposites, such as "first-last," and "early-late." (Basic encourages teaching words in logical groupings - it is possible to carry grouping by opposites into the nouns (front-back) and the prepositions (before-after, under-over), and the verbs (come-go, etc.).
The most striking feature of Basic is its limitation of the verbs to sixteen. Ogden came to this small set of verbs, because all English verbs may be broken down into the operations and directions they are in fact talking about, e.g. to immerse your hands in water, is to put them under water. This made it possible for Basic English to have only 18 verbs (the words for these necessary physical motions and have and be).
There are several reasons of why Basic is not the way to go. Some of them are given here (see Appendix B for a detailed discussion on Basic English):
get up in the morning,
get out of the window,
get over an illness,
get your hair cut.
These are essentially different words. The get stem does not make them the one word.
I think they will not make learning easier, but rather confuse it.
Visual writing is the process of communicating with pictorial symbols (predominately icons). Sentences constructed visually could be the ideal form to convey what is being said, if they represent the deep structure of a sentence, thus bypassing the problems posed by its expression in natural language, like ambiguity and diversity of form. Consider the diversity of form in the following sentences,
Ilan gave a book to Mary.
A book was given to Mary by Ilan.
Mary was given a book by Ilan.
They all have the same meaning, although the surface structures are different and they may differ in emphasis or focus. They would have the same representation if represented by pictographic symbols. This section, and the rest of the dissertation, will mainly focus on iconic communication.
One may think that it would be easier to have the whole world learn the same spoken language. This could be an artificial language or a natural language. Artificial languages like Esperanto have met little success, largely due to the initial effort required to learn them (by a worthwhile number of people). We have also shown why a natural language, like English would not work. It would never be accepted by some countries, they will think that we are extending our culture. For example, in France they are already limiting the amount of English on television, because they do not want the people to learn too much English. Besides this political problem, there are some other disadvantages of learning a spoken language to write in.
An artificial language that is based on deep structure, would express every element exactly in one way.
Furthermore, easier and faster learning is achieved through recognition, the fact that it is easier to recognize visual representations (enough to choose from those available in the system) than to come up with words without system assistance, like when one learns a new (spoken) language (this also becomes obvious when you keep in mind that whenever people do not understand each other, the try to make themselves clear by use of mime and gestures). The user only has to recognize the icons to construct sentences. Icons are also easier to remember. It is a well-known aid to associate a thing to be remembered with a simple object to which it has a defined relationship.
The table shows several iconic signs that are becoming fairly standardized in international usage, along with an approximation of their meaning in English. Note how the signs convey information in a culture- and language-free mode. Note also how the eye is instinctively drawn to the iconic symbols and only later browses back to the natural language interpretation. The iconic symbols often relate at a "primitive" level to one or more of our other senses such as hearing, touch, and smell which also requires low-level cognitive processing. Meaningful alphanumeric symbols, on the other hand, are "compound" symbols (words and sentences) comprising strings of more elementary symbols (letters) that require a higher level of cognitive processing to extract their meaning.
If iconic communication can be shown to be more efficient, quicker, easier, and less ambiguous than conventional communication means, like speaking or writing, then it might be a solution to the problem of multi-lingual communication (although it does not have to be all of these to be useful). Secondly, new powerful computers and artificial intelligence provide hope for solving the old problem of a universal language.
Another reason why now is the time for a computer-based universal language is the explosive growth in people using the Internet. People from any country now have access to the Internet. They sent each other e-mail, read news, or use the World Wide Web (WWW). Much more people would have access (mentally) to the Internet and be able to communicate if they had a common language. This is a perfect opportunity for an iconic language.
"Every blink of the eye brings a picture to the human mind."
Images can be recognized quickly and committed to memory with surprising persistence (many people "never forget a face"). Images are particularly important in two areas (Mullet and Sano):
Representation provides the basis for all communication. We can convey ideas about things that are not materially in our presence only by calling forth an appropriate mental representation. The manner in which such representations are interpreted by participants in a communication system to create shared meaning can be fully understood only within the interdisciplinary context of semiotics. Semiotics is the study of signs and symbols.
The critical process of representation (the focus of analysis at the semantic level) depends on establishing a clear relationship between a representamen and its object. Peirce identifies three forms this relationship can take:
Pure icons rely initially on recall of a previous visual experience on the part of the user, with enough distinction to make their use in a particular context clear to him. They may be used in conjunction with conventionalized symbols, e.g. a diagonal bar through an icon could indicate negation.
In common usage, the term icon, has come to denote any small raster image appearing in a GUI display. In the next sections we will study some systems that are based on visual representations. Some of the systems are pictographic and some of them are iconic.
"Iconic communication deals mainly with non-verbal communication between human beings by the use of visual signs and representations (such as pictures) that stand for an idea by virtue of resemblance or analogy to it in contrast to symbolic communications where the meaning of a symbol is entirely nominal (such as English text describing a picture)."
The earlier attempts at iconic communication were systems in which the icons were drawn on paper. The computer provides an ideal device for the implementation of a flexible iconic communication system, since the icons do not have to be drawn, but merely recognized. Also, icons can be manipulated easier (moved around, added, removed, etc.). Computer-based iconic systems are of the most interest to us and they will be addressed in the section 2.3.4.
Even in visual communication systems, there is a division in natural and artificial languages. The Chinese writing system was originally pictorial. Furthermore, both the old Egyptians (hieroglyphs) and the North American Indian also communicated through pictures. In addition, there are existing signs, symbols, and icons that are understood internationally, but were not intended as a language. These are for example arrows to indicate direction and overlaid diagonals to indicate negation, most of which are found in traffic signs. A study of these can be useful in the design of our icons, and can be found in Appendix C.
In the past, some approaches have been made to come up with an iconic language. Some examples are: Semantography, Isotype, and Worldsign by Jones and Cregan (1986), a language created for mentally handicapped children which allows dynamic representation (see Appendix C). The most recent attempts in computer-based iconic systems are the Hotel Booking System (Appendix C), which allows communication on a restricted domain (that of hotel booking), and CD-Icon, which is based on Schank's conceptual dependency theory, and designed as a pure person-to-person communication system.
CD-Icon was developed by Colin Beardon at the University of Brighton, with the purpose of exploring unrestricted person-to-person communication. CD-Icon is based on Schank’s conceptual dependency theory.
CD-Icon is a means of testing the validity of conceptual dependency directly by using it as the basis of a communication system that uses only icons and no words.
A message is composed by selecting options from a series of interconnected screens. The message is then transmitted, also as a set of interconnected screens, but not showing the options that were not selected.
Four basic types of construct can be specified from the structure of conceptual dependency diagrams:
The CD-Icon system reflects these divisions. It is constructed as four modules, each processing one type of construct with clearly defined interactions (see figure 4).
CD-Icon is described in detail in Appendix C, along with an example of a message composed using the system. It also discusses some of the probelems of CD-Icon. For one thing, there is an encodability/decodability problem, since messages composed with CD-Icon tend to be long and cover many screens.
In the next chapter our proposal will be discussed. It is called VIL, which stands for Visual Inter Language, and is also a computer-based iconic communication system, designed to allow people to communicate with each other when they share no common language. It allows a user to construct sentences, without actually having to type in words, i.e. solely relying on icons. The goal is to make the system language independent so that it can be used universally. So, the system should be able to be used to construct sentences irrespective of the languages known by the users. We will show how it differs from CD-Icon, for one thing it is not based on Schank’s conceptual dependency theory, and it will have its own grammar. We will discuss the design of the system.
This chapter will study the semantics of the VIL. Various principles for VIL are stated in section 3.1. In order to come to VIL’s grammar, we have researched verbs, nouns, and adjectives. VIL’s grammar is given in Appendix G. The organization of the verb system is used in composing VIL’s grammar.
Principles of VIL
VIL’s main principle is that it uses the greatest common denominator of all languages to design its grammar. The grammar will then be drastically reduced in complexity.
Principle 3.1: Greatest Common Denominator or Minimal Grammar and Vocabulary
VIL will adopt a minimal grammar. Every grammatical feature which is encountered in all languages, shall be retained in the grammar of VIL, i.e. no grammatical feature shall be so retained if it is missing from any languages. Thus, for instance, the feature of grammatical gender can be dispensed with because it is missing in at least one languages, i.e. English.
In section 3.1.2 the different parts of normal speech and if and how they will be presented in VIL will be discussed. But first let us review some more principles of VIL
Almost all of the following principles of VIL have been compiled from the information in chapter 2. Just like Pidgins, VIL primarily has the following two functions:
Principle 3.2: Directive Function
to get people to perform desired actions (this might be in, for example, a tourist or trading context).
Principle 3.3: Referential Function
to describe a concrete situation to bring about a desired end.
As mentioned, VIL adopts the principle of the greatest common denominator. The next table shows some of the simplifications VIL will adopt.
|
Simplified linguistic structure in VIL |
|
Lexicon Syntax
Morphology
|
We will now give a more detailed explanation of the simplifications mentioned in the table, presented as a number of principles.
Principle 3.4: No Linear Word Order
Another major principle is that VIL will have no linear word order. The icons can be placed in any order since the different cases (e.g. subject, object, indirect object, etc. will be clearly visually distinguished).
Principle 3.5: Number
There are no synthetic plurals of the ‘man/men’ or ‘computer/computers’ type. Instead nouns are invariable, like ‘sheep’. Plurality is normally implicit in the context and can be indicated by a QUANTIFIER or a NUMBER, e.g. ‘all book’ or ‘five book’.
Principle 3.6: Gender
Gender distinctions are reduced or eliminated in both nouns and pronouns.
Principle 3.7: Inflection
There is no concordial agreement between subject and predicate. Both noun and verb are invariable, where the singular is used for nouns and the imperative for verbs. Since the verb form is invariable, distinctions relating to time and continuity of action are either understood from the context or are indicated by adverbials. Adjectives are invariable as well.
Principle 3.8: Negation
Negation is done by having an invariant way to "modify" an icon, for example by a diagonal bar through the icon, and will not change any order in a message.
Principle 3.9: Verbs and tense
All contrasts of number, person, and tense have been lost in the verb, and every verb has only one form used in inflection. Tense is implicit in the meaning of the sentence when indicated by TIME-DEMONSTRATIVES (last, this, next) or explicit when indicated by TIME-PRONOUNS (past, present, yesterday, tomorrow, etc.).
Principle 3.10: Auxiliary verbs
As a consequence of Principle 3.9, no auxiliary verbs will be used in tenses (as in "I have eaten").
Principle 3.11: Article
VIL will not have a specific article. This means that there is not a specific icon for ‘article’.
The definite article (‘the’) is used in normal speech to refer to something specific that both the speaker and hearer know about. When a speaker says "did you receive the book?", the hearer knows what book the speaker is referring to. Definiteness may be represented by NOMINAL-DEMONSTRATIVES (this, that).
VIL’s parts of speech are reflected in its grammar. The categories that VIL’s distinguishes are: TIME-WHEN, PLACE-WHERE, VERB, AUX-MODAL, AKTIONSARTEN, ADVERB, INTENSIFIER, NOUN, NOM-PRONOUN, NOM_DEMONSTRATIVE, QUANTIFIER, NUMBER, ADJECTIVE, SUBORDINATOR, and COORDINATOR. For VIL’s grammar see Appendix G.
In this section, we will list some principles for the major categories: noun, adjective, abverb, and verb.
Principle 3.12: Distinguishing Roles/Cases
In VIL, a visual distinction will be made between nominal constituents that denote particular semantic roles or cases, like subject, direct object, and indirect object (see section 4.3.2.1).
Principle 3.13: Distinguishing the Major Categories
In VIL, the major categories noun, adjective, adverb, and verb will be distinguished by some invariant visual entity, for example different borders around an icon. For example, an icon with a square border may indicate that the icon is a noun, and an icon with a square border with rounded edges may indicate that the icon is a verb. This principle makes the next principle possible.
Principle 3.14: Derivational Morphology: Changing a Category
In VIL, the meaning of an icon can be changed by changing its category marker, for example the border of the icon. This means that categories can be changed, for example a noun can now become a verb by adding a verb border to a noun icon. So, an icon can have more than one border and the outermost border determines its category. In the case of a verb derived from a noun, this would mean that the icon would have an outer border specific for verbs, and an inner border specific for nouns. For example, an icon for the noun "food" could have a noun-specific border, and the same icon with a verb-specific outer border could stand for the verb "to eat". Thus the outer border denotes what part of speech is meant (i.e. a verb), and the inner border denotes what part of speech it is derived from (i.e. a noun). There are several ways a category can change:
1. De-Verbal Nouns and De-Nominal Verbs
Some verbs will be used as nouns, but usually it will be the other way around, because it is easier to represent nouns visually, since nouns are more likely to be concrete objects.
2. De-Adjectival Nouns and De-Nominal Adjectives
Some adjectives will be used as nouns, but, for the same reason as mentioned in 1, usually it will be the other way around. To keep the example of the icon for the noun "food", this noun can be turned into the adjective "edible" by putting an adjective-specific border around the nominal icon.
3. De-Verbal Adjectives and De-Adjectival Verbs
Adjectives can also be de-verbal as in "the running man", and the same principle of distinction apply. An example of a de-adjectival verb could be "to whiten", derived from the adjective "white".
4. De-Adjectival Adverbs
In VIL an adjective can be used as an adverb too and vice versa. In the sentence "He solved the problem wisely", the adjective "wise" is used as basis for the adverb "wisely" (in a wise manner).
Verbs
The following principles all have to do with verbs.
Principle 3.16: Modality: Imperative
Imperative sentences will have the same construction as assertive sentences, but the are distinguished by a exclamation mark ("!"). Remember from Principle 3.4 that there is no linear word order.
Principle 3.17: Modality: Passive
There will be no difference between passive and active sentences as in the following example:
"I kicked the ball to Ilan."
"The ball was kicked to Ilan."
These sentences have the same deep structure, and will be represented the same way.
Principle 3.18: Yes/No Questions
Yes/No questions will, like imperatives, have the same construction as assertive sentences, but are distinguished by a question mark ("?").
Question in English |
Question in VIL |
Do you like soccer? |
You like soccer? |
Principle 3.19: Wh-Questions
When the question word is the subject, then in VIL a question mark is also put in place of a noun phrase denoting the particular role/case:
Question in English |
Question in VIL |
What knocks at my window? |
"?" knocks at my window (subject) |
What did you buy? |
You buy "?" (object) |
When does the train come in? |
The train comes in ("?" in the time-when field) |
Where did you buy the sweater? |
You buy the sweater ("?" in the place-where field) |
This section will focus on designing a grammar for VIL, and will also describe the organization of the iconicon (section 3.2.1), which deals with the hierarchical organization of three substantive grammatical categories (nouns, adjectives, verbs). First we will give some background in the study of syntax.
In the 60s and 70s, the syntax of natural languages was generally described by specifying a set of rules (phrase structure rules and/or transformations) and a way in which the rules interacted (how, for instance, they were ordered with respect to one another in application) to define derivations. Phrase structure rules expanded the initial symbol S to create an underlying constituent representation for the SENTENCE (deep structure). Lexical items, as well as syntactic categories, were introduced by these rules. Transformations performed a variety of functions. For example, they changed the underlying grammatical relations, as in the case of passives, which were derived from actives; and they created complex sentences out of simple ones, e.g. the sentence "John thinks that Bill will leave" was derived by an embedding transformation that combined "John thinks" and "Bill will leave" phrase structure rules and transformations were introduced by Chomsky’s Syntactic Structures). Grammatical sentences were those for which legal derivations were defined: ungrammatical sentences were those for which no legal derivation was defined. The rules proposed tended in general to be quite specific, in that they were particular to individual constructions in individual languages.
Semantics was assumed to be derived from the deep structure (i.e. result of the phrase structure rules). However, some researchers believed that the semantics was basic and that the syntactic structure was derived from that. This led to the approach called "Generative Semantics" (McCawley).
More recently, syntactic representations are seen as being constructed by very general mechanisms (such as the principles of X-bar theory, which define what is to be a well-formed phrase structure) which are then subjected to conditions on well-formedness which filter out unwanted structures, i.e. those corresponding to ungrammatical sentences. There was is also a tendency to explain particular facts as fas as is possible in terms of general principles of grammar rather than in terms of principles unique to the language being investigated.
There are five frameworks that developed from Chomsky’s Syntactic Structures and Standard Theory of grammar: government and binding (GB), generalized phrase structure grammar (GPSG), lexical functional grammar (LFG), relational grammar (RG), and head-driven phrase structure grammar (HPSG). GB has seen a steady diminuition of language-specific rules (transformational or otherwise), and their replacement by general universal principles and characterization of the ‘parameters’ of possible variation among languages. GPSG and LFG are competitors to GB and are more oriented toward surface structure than GB. GPSG is a framework for writing fully explicit formal grammars for natural languages (language-particular). HPSG developed from GPSG. RG uses grammatical relations to bring out what is common to clause structures in languages which differ in word order, phrase-structure configurations, and case-marking patterns. Rather than being defined in terms of other notions, grammatical relations are primitives of the theory and are used to represent clause structure and to formulate linguistic universals as well as language-particular rules and condition. In LFG, the role of the lexicon is central and grammatical functions are seen as syntactically primitive. Many syntactic generalizations (e.g. passive, ‘raising’, or ‘equi’) are formulated as LEXICAL REDUNDANCY RULES rather than as operations on phrase structure, like transformations. Their use allows monotonic, order-free composition on the sentnece level. LFG postulates a system of interdependent representations with different formal characteristics, which encode different types of linguistic information. The most important representations are C[ONSTITUENT]-STRUCTURE and F[UNCTIONAL]-STRUCTURE. A c-structure encodes linear order, phrasal hierarchy, and syntactic category (part of speech) in a tree. This representation is used to link syntax and phonology. The f-structure encodes information about grammatical functions in hierarchical ATTRIBUTE-VALUE MATRICES (where most grammatical functions have traditional names like ‘subject’ and ‘object’).
In the 60s and 70s, the grammar of a given language had as its base a list of phrase structure rules of the general form:
Such rules define as well-formed a substructure like:
A
X Y Z
The rule specifices precedence relations (e.g. X precedes Y and Z) and hierarchical relations (e.g. that A immediately dominates X, Y, and Z). Much recent work separates out these two functions, hierarchical structure and linear order being specified by different subsystems. Hierarchical relations are defined by the general principles of phrase structure, known collectively as X-bar theory. Linear order of constituents is defined by separate conditions - linear precedence in GPSG, and in GB principles that have to do with the direction in which case or semantic roles are assigned.
In a universal language like VIL, we are not concerned with linear order, and are only concerned with the hierarchical composition. VIL’s grammar will be the simplest possible to govern the use of the chosen vocabulary. For this reason, before we can create VIL’s grammar, we will describe the organization of the verb system, also the organization of the nouns and adjectives are described.
One approach closely related to semantic grammars is of particular interest, as is one AI semantic representation. We consider Fillmore’s case grammar and Schank’s conceptual dependency theory to be of spcial interest. In case grammar, the verb is, just like in VIL, the main part of the sentence and all other parts revolve around it. A sentence in its basic structure consists of a verb and one or more noun phrases, each associated with the verb in a particular case relationship. Conceptual dependency theory is of interest because it is used in the iconic system CD-Icon, and because it restricts the verb system to eleven primitive ACTs. These ACTs are used to represent more complicated verbs. Case grammar and Conceptual Dependency theory are discussed in Appendix H.
The disadvantage against the use of conceptual dependency as a representation formalism, is it requires that all knowledge be decomposed into fairly low-level components. Conceptual dependency is designed ot ease inference, and is a theory for representing fairly simple actions. To express, for example, "John bet Sam fifty dollars that the Mets would win the World Series" takes about two pages of conceptual dependency forms. From an encodability and decodability point of view, this does not seem reasonable.
In theory on grammar, the word lexicon is used to refer to vocabulary. In the report we will adopt the word Iconicon to denote a lexicon of icons. In this section we discuss the organization of three substantive categories: nouns, adjectives, and the verbs. These organizations will eventually determine VIL’s grammar. First we will give a principle about the depth of hierarchies and explain some of the terminology used.
Studies by Liebelt et al. (1983) have shown the importance of a meaningful organization of items. Semantically meaningful categories - such as food, animals, minerals, and cities - led to shorter response times than did random or alphabetic organization. A tree structure is a convenient way of organizing a large set of items. One can partition the items into (mutually) exclusive groups, although VIL actually will use lattice (more than one way to get to an item, cross-classification). If each level has eight items, then a tree with four levels has the capacity to lead the user to the correct item out of a collection of 4096 items. If there are a lot of items to classify, then there is a depth-breadth trade-off. Should one go deeper or wider? Kiger (1964) has shown that breadth is preferred over depth. In his test, in which he grouped 64 items, the deep narrow tree (2x6) produced the slowest, least accurate, and least preferred version, whereas the 8x2 tree was among those rated highest for speed, accuracy, and preference. Wallace et al. (1987) confirmed that broader, shallower trees (4x3 vs. 2x6) produced superior performance, and showed that, when users were stressed to work quickly, they made 96 percent more errors and took 16 percent longer when they used larger/deeper trees. Furthermore, Rosch has found that natural categories are 5 layers deep.
Principle 3.20: Depth of Hierarchies
To facilitate speed of access, in VIL we will adopt that each tree or lattice will not be deeper than 5 levels.
One should also take into account that it is important to "reduce short-term memory load". Humans can remember seven plus or minus two chunks of information. This would suggest that having more than nine items at one level in the tree would negatively influence the previous test results.
Principle 3.21: Width of Hierarchies
Taking short-term memory into account, each non-terminal level of a hierarchy in VIL will have between 7 and 9 objects in it. The terminal levels with the actual objects to choose can have more than 9 items. Since the user can now more easily recognize the terminals and do not have to recognize the items as standing for a category, we decided that it would be justified to present more than 9 items.
A tree that has five levels and has eight objects per level, can already hold 32768 items. We expect the size of the vocabulary of VIL to be much smaller than this, let alone of any single category.
Let’s first explain some of the terminology that will be used when talking about hierarchies.
There has been considerable speculation about the semantic organization of nominal
concepts. It is generally agreed that they are organized hierarchically into levels, from
specific to generic.
According to Miller and Fellbaum, the topmost, or most generic level of the hierarchy is almost vacuous semantically: it is possible to put some empty abstraction designated {entity} at the top; to make {object} and {idea} its immediate hyponyms, and so to continue down to more specific meanings, thus pulling all nouns together into a single hierarchical structure. In practice, however, these abstract concepts carry little semantic information. The alternative is to select a relatively small number of generic concepts and to treat each one as the unique beginner of a separate hierarchy.
Miller and Fellbaum have come up with 26 unique beginners that capture all the English nouns. They have made distinctions between abstract concepts and physical objects. The unique beginners are:
{act, action, activity} {natural object}
{animal, fauna} {natural phenomenon}
{artifact} {person, human being}
{attribute, property} {plant, flora}
{body, corpus} {possession, property}
{cognition, ideation} {process}
{communication} {quantity, amount}
{event, happening} {relation}
{feeling, emotion} {shape}
{food} {society}
{group, collection} {state, condition}
{location, place} {substance}
{motive} {time}
The problem with their hierarchies is that they are too exhaustive and sometimes go about 10 levels deep. Remember from Principle 3.20: the Depth of the Levels, that the hierarchies must not be very deep, because the user must be able to pick nouns reasonably fast, and the deeper the levels the more complicated the distinctions, and the harder it will be for the user to choose which branch to follow. In order to pick the nominal concept car, the user would have to go the following path: artifact -> conveyance -> vehicle -> wheeled vehicle -> motor vehicle -> car. Whereas in our hierarchy, we will see that the user takes a much shorter path: Transportation -> by Road -> car. Furthermore, having 26 unique beginners is too many. Keeping short-term memory requirements in mind (Principle 3.21: the Width of Levels), the ideal amount for a user to choose from is between 7 and 9. This means that artifact will have a superordinate class, which in turn means that the path to car is going to be even longer.
The preliminary results of our study of the nominal system, including the resulting classification can be found in Appendix D.
Two kinds of modifiers are usually distinguished. Roughly, those that modify nouns are called adjectives, and those that modify anything else are called adverbs. We have not yet explored the semantics of adverbs.
The semantic organization of adjectives is different from that of nouns. There is no hyponymic relation (ISA) for adjectives: it is not clear what it would mean to say that one adjective is a kind of some other adjective. The basic relation among adjectives is antonymy. When a probe word in a word association test is a familiar adjective, the response commonly given is the antonym of the probe. For example, when the probe is good, the common response is bad; when the probe is bad, the common response is good. This mutuality of association is acquired as a consequence of these pairs of words being used frequently in the same phrases and sentences.
Principle 3.22: Antonymy Relation
The basic relation among adjectives is antonymy. These antonyms should be represented together for maximum clarity (since antonyms may not be very clear when presented in isolation).
The preliminary results of our study of the adjectival system, including the resulting classification can be found in Appendix E.
This section will focus on finding some general principles by which to categorize verbs. Verbs will be organized according to their meaning. The resulting hierarchical ordering of verbs will serve as a basis for choosing verbs in our Visual Inter Language (VIL) system. Since our system is an iconic system, we are not only concerned with how people choose and think of verbs, but also how to represent them (iconically). How to represent the verbs will be discussed in the next chapter.
Verbs are decomposed into primitive elements. This means that more specific verbs will have to be described by their more general version, modified by some adverbs of manner, e.g. "to run" would be "to walk fast". So, only the basic or most general verb ("to walk") will be stored and this basic verb can be modified with adverbs (to denote "to run"). The modified verb is then "to basic verb in some manner".
Principle 3.23: Composition of Verbs
In VIL, only the most basic/general verbs are stored. These verbs can then be modified, for example by adverbs.
In order for us to find out what the ideal classification for our purpose would be, we have compared various classifications to see what are the categories that they all share. These categories are bound to be among the ones we also want to incorporate in our hierarchy. Also, in order to get a better overview of the different approaches, we have made a distinction in whether or not the approaches can be distinguished by the fact that the do or do not consider arguments or cases when classifying verbs. This division yields the following three groups:
Approaches that do not use arguments or cases during classification:
1) Speech Act Classification (Ballmer & Brennenstuhl)
2) Basic English Verb System
3) Schank’s Primitive ACTs. Schank uses cases in so far that they are defined or assumed by the definition of the primitive ACTs. For example, "to apply force to" (PROPEL) takes a physical object and a directive case (by definition). Similar with GRASP ("to physically grasp an object"). He does not explicitly state subject: animate, object: inanimate, etc. This is also because of the definition of ACT, "something an actor does to an object". The abstract nodes are high up the hierarchy, so not too many restrictions need to be made on the cases.
Approaches that start out without, but consider arguments or cases at the deeper levels:
4) Levin: English Verb Classes. Only in the deepest levels of the hierarchy does she use arguments to distinguish verbs. And even then not really arguments, but diathesis alternations. She does not say anything about how many arguments, or the nature of them. She says something about the alternation the verbs can occur in, and says what the sentence framework looks like in that case. For example, consider the Locative Alternation: it contains a locatum argument (entity whose location is changed) and a location argument. In each variant one is object of a PP, and the second is not.
Jack sprayed the paint on the wall (locative variant)
Jack sprayed the wall with paint (with variant)
5) Matsukawa & Yokota: Development of Concept Dictionary. Real use of deep cases at the third level (non-object categories).
Approaches that use arguments or cases during classification:
6) Ray Jackendoff’s Nonspatial Semantic Fields & the Thematic Relations Hypothesis.
The results of our study of the verb system, including the resulting classification can be found in Appendix F.
VIL’s grammar, designed to govern the use of the chosen vocabulary, is based upon the results of the studies into the various classifications (verbs, nouns, and adjectives). It is described in detail in Appendix G.
In this chapter the focus is on the visual representation of the system. Section 4.1 will research icon design, and its principles. Section 4.2 is about designing cross-cultural icons, and gives some dos and don’ts. In Section 4.3 the visual representation for the grammatical categories (e.g. verbs, nouns, adjectives, etc.) will be studied (i.e. what will be represented visually and what will be represented iconically). We will talk about how to represent abstract versus concrete meanings, and how to represent non-terminal categories as opposed to terminals.
Icon Design
Good graphic design is much more than just pretty colors and pictures. It can significantly improve the communicative value of the interface, which leads to increased usability (ease of learning, efficiency of use, memorability, reduced number of errors, and subjective satisfaction). Representation provides the basis for all communication. We can convey ideas about things that are not materially in our presence only by calling forth an appropriate mental representation. Icons are of central importance to the successful utilization and acceptance of a graphical user interface and its related applications. Icons can lead to faster decision making and improve long-term memorability. Cross-cultural elements must be taken into account, though. For example, a system may represent the concept of a "queue" by an icon of a billiard ball (i.e. cue ball). Such visual representations might be fun to put in a design, but they are often harmful to the novice user trying to make sense of a new visual environment. Also, the product would be dead on arrival if it were ever exported to a non-English speaking country. Understanding what a thing represents (a queue) - as opposed to merely what it depicts (a billiard ball)- is a prerequisite for using the icon correctly. Users must recognize the icon on the display screen as a sign for the thing (e.g. an icon for a calculator program), rather than the thing itself.
Interpretation is the process of reconstructing the meaning of a sign by identifying the sign object and grasping the significance of the connection between the sign object and representamen. As shown in figure 27, the interpretation becomes easier up to a point as the representation becomes more schematic. As the level of abstraction increases, the sign becomes progressively more generic and less complex.
A given sign representation can be characterized by its degree of abstraction - the extent to which the essential qualities upon which the representation is based are isolated from the literal perceptual characteristics of the sign object. A photograph or realistic illustration provides a high degree of fidelity to a particular sign object, and is usually easy to recognize as a result. More schematic representations, because they permit the selective omission of detail, are better able to represent a broader class of objects (as opposed to one of its instances) or to focus on some characteristic aspect of the object (rather than on the perceptual reality of a specific individual).
No matter how important icons are to the usability of a system, icons are also one of the biggest problems in interface design. Imagery that distracts, confuses, or simply bewilders the uninitiated user is all too common in the current generation of graphical applications.
Visual metaphors can help users understand the working of their environment, but only to the extent that the mapping between image states and system states corresponds to the user’s understanding of the real-world analogy.
This section will give some guidelines that help in the design of better and more consistent icons. Shneiderman gives icon-specific guidelines for graphical user interfaces (GUI’s):
Marcus applies semiotics as a guide to four levels of icon design:
He recommends starting by creating quick sketches, pushing for consistent style, designing a layout grid, simplifying appearance, and evaluating the designs by testing with users.
Shneiderman considers a fifth level of icon design:
5. Dynamics: receptivity to clicks - highlighting, dragging, combining, sound, animation.
The American Institute of Graphic Arts (AIGA) uses three of Marcus’ dimensions in evaluating over 300 pedestrian oriented symbols from all over the world. They evaluated the strengths and weaknesses of every symbol in relation to these dimensions:
Bad icons that can be caused at three different levels:
In Appendix I, examples of bad icons are be given.
Effective imagery must possess a perceptual immediacy that allows it to be recognized at a glance. For most images, this involves a process of careful abstraction in which all but the elements that most characterize the sign object are removed. When developing multiple images, care must be taken to maintain cohesion within the image set and to consider the physical, conceptual, and cultural context in which the images will ultimately be displayed.
The following are some principles in icon design. They are explained in detail in Appendix J: simplicity, reduction, generality, cohesiveness, regularity, characterization, and communicability.
To create icons that people around the world will recognize and understand, we must try to understand these people as well as we can. What symbols and objects will they recognize? Are they familiar with the same symbols as we are? Can we assume they have traveled extensively? Can we assume they watch movies and television?
If an icon relies on inside jokes, figures of speech, slang, or other terminology that is well-known only within a particular subculture, then it will be intelligible only to members of that group, and that icons that might be culturally offensive should always be avoided (e.g. the "thumbs up" for "OK").
The following are some principles when designing cross-cultural icons. They are explained in detail in Appendix K: remove or translate text, avoid culture-specific symbols. Be careful when using gestures, religious symbols, animals, colors, and body parts. Consider the reading and scanning direction, and do not imply any order or sequence.
In chapter 3 we discussed the grammar and vocabulary of the communication language VIL. In this section, we will discuss a different sort of grammar and vocabulary, namely that of the icons themselves. We will discuss the process of creating a vocabulary of primitive (or visual) combining features, which are elements of an icon that we can combine according to some rules to produce the icons that we need. In the designing our icons we will combine these elementary symbols to represent complex concepts.
Just like VIL, the icons consist of two parts: a vocabulary and a grammar. The vocabulary is a collection of primitive combining features that are combined according to the rules of the grammar to form icons. For VIL itself, the grammar records how to put icons together to form a message. For icons, the grammar records how to put primitive combining features (also called visual combining features) together to form a comprehensible icon. This icon will be part of VIL’s grammar, e.g. it might represent a noun.
Besides iconic representations, VIL will also have terminal categories that are not represented as icons. These are categories that are represented visually. For example, if the location is a country, one would select the country on a map of the world. This map is not an icon but an image, and we call this visually or graphically. Another example is that of gradable attibutes. These are attributes that will be represented by sliders/scales, and are represented, not iconically but visually. The next figure represents the concept of speed and determines how fast someone or something is going, as in "the fast man" or "the man walked at quite a pace":
Principle 4.1: Iconic vs. Visual Representation in VIL
In VIL, in addition to icons, visual representations are used.
To set up the language for the icons and visual representations, we must design the visual combining features and set rules for combining them. The next few sections will describe the steps involved in setting up the language.
The first step in the design of the language for icons is to determine all the things for which we need icons and visual representations. These will primarily be categories of the grammar of VIL. Let’s list these categories again and, for each one, mention how they will be represented (for examples of members of these categories see Appendix G):
TIME-WHEN: graphically,
PLACE-WHERE: graphically,
VERB: abstract icons,
AUX-MODAL: iconic,
AKTIONSARTEN: graphically,
ADVERB: graphically,
INTENSIFIER: graphically,
NOUN: concrete and abstract icons,
NOM-PRONOUN: graphically or iconic,
NOM_DEMONSTRATIVE: graphically or iconic,
QUANTIFIER: graphically,
NUMBER: numerals,
ADJECTIVE: concrete and abstract icons,
SUBORDINATOR: graphically,
COORDINATOR: graphically.
Note that, for example, adjectives can be iconic and verbs can be concrete icons if they are de-nominal and the noun is concrete.
Besides icons for abstract meanings of terminals nodes in the heirarchies for the substantive categories (e.g. "memory" or "to throw"), we also need abstract icons for non-terminals of the hierarchy (category names), e.g. an icon for "verbs of motion" (where the generalized form is the verb "to move") or "living things".
For visual representations it holds that, besides the categories mentioned above, in general, all gradable concepts are represented graphically, e.g. the concept of size is gradable. It will be represented visually by a slider, with values ranging from small to big. However, the two ranges at the end of the slider are generally icons themselves. Furthermore, PLACE-WHERE can also be chosen from a map, and TIME-WHEN can also be chosen from a clock or calender.
The next step is to build a collection of basic visual symbols that will appear in the icons. The goal in designing these basic symbols is so that they can be combined. The symbols should be simple, small, flexible (i.e. easily resized and reshaped), and easy to combine.
Appendix L discusses some icons from existing applications that represent some of the items mentioned in section 3.3.1 (verbs, nouns, adverbs, adjectives), and also how to represent relationships among objects, and states of being.
In commercial products, visual combining features are often combined with other classes of icons to associate them with a particular manufacturer or product line of which they are a part. For example, assume a company is designing icons for a program used to create multimedia presentations. The visual element:
which could be some company trademark, may be incorporated into the icons for the various media handled by the product (from Horton’s Icon Book):
In VIL, visual combining features can be used differently. They can be put to use in several ways:
The grammar for icons is a set of rules for combining visual combining features, and ensures that basic visual combining features are combined in simple, consistent, and predictable ways that users readily understand. Horton states that a graphical grammar should specify:
Just as words or, in VIL, icons can be combined into verbal statements, questions, and commands, there are four basic ways of combining visual combining features. Horton mentions antithesis, addition, overlap, and specification (see Appendix M for an explanation).
This chapter describes how we will evaluate VIL. We will reiterate the expected contributions (section 1.3) and describe how to evaluate them. In addition to arguing for our principles (like learnability, encodability, decodability, and extensibility), we will argue that VIL satisfies Richards’ criteria, as mentioned in section 1.4, that are needed in order for a universal language to be acepted: the political, the psychological, and the linguistical aspects.
The expected contributions from section 1.3 were:
The following sections will address each of the expected contributions in turn.
As stated in section 1.3, most of the principles are hard to test and have to be argued for. This was done in the previous chapters and in the appendices. In this section we will argue for how VIL meets the four primary capability principles learnability, encodability, decodability, and extensibility/evolvability, and for Richards’ three criterias the political, the psychological, and the linguistical aspect.
Let’s look at Richards’ criteria first:
The Political Aspect
One of the major reason that a natural language cannot serve as a universal language is the political problem. It would never be accepted by other countries. Nationalism and racial pride are insuperable obstacles. If the universal language were to be an iconic one, then there would not be a political problem, since visual representations are universal. The actions in pictures take the place of references to any mother tongue.
"By and large pictures are universal. People in the remotest corners of the earth and living in cultures least akin to ours learn our pictorial dialects with surprising ease and become thereby accessible to what we find to say with our pictures."
The Psychological Aspect
The psychological aspect entails that if people have to learn a new language, then it should be easy to learn. The root criticism of any revived or artificial written language was that it was to hard to learn. VIL uses a restricted grammar and vocabulary to make the system not only universal but also easy to acquire. Also, the arguments for learnability, encodability, and decodability will show that the system is easy to learn and use, so the user should have no fear to learn the system.
The Linguistical Aspect
Ogden argued that "languages are shaped by use rather than by design. Long-established languages have been hammered and wrought, broken and remade, in countless ways which only the biggest and best of dictionaries can show in detail." He stated that "no artificial language can acquire a tithe of such richness of interdependence, without centuries of wide and varied use. With a simplified form of a living language these discrepancies are reduced. Its meanings are held in place by the extent of the common use its words have been put to." It is true that only a non-artificial natural language can accomplish this. However, as stated, political aspects keep a natural language from becoming universally accepted.
Let us now argue for how VIL meets the four primary capability principles: learnability, encodability, decodability, and extensibility.
This means that the system should be easy to learn. We argue that with a visual language the learning process will be much faster than with a written language. This is because of the fact that it is easier to recognize visual representations than to come up with words for them. Richards and Ogden supported this view. Richards argued that the proportion of the meanings of a language that can be visually presented is an enormously important factor in determining the ease with which it can be learned and retained. Section 5.5 on future work describes ways in which learning can be facilitated. Furthermore, learnability is supported by having a drastically reduced grammar and vocabulary.
This means that it should be be fairly easy to generate messages. A computer-based iconic language makes this a lot easier by not requiring the user to draw icons, but simply letting him or her choose the icons from the screen. The design of the grammar (we take the greatest common denominator of the things to be expressed, i.e. no inflection and no gender, etc.), the design of the hierarchies for our grammatical categories, and the design of the program itself all facilitate encoding. Furthermore, the intelligent capabilities talked about in section 5.5 on future work should speed up message generation drastically.
This means that the messages that are composed on the system should be easy to comprehend. Our simple grammar and our self-explanatory icons and consistent visual combining features should account for easy decoding of messages. Possible help features like explanation of the different visual combining features of an icon, possibly animation, could help in decodability.
This means that the system should have the possibility to be extended easily. The design of the hierarchies makes it possible to easily add more icons, without it having to undergo a drastic redesign. Icon compounding is also a possible way to extend the system. Icons for adjective and noun could be compounded to form a new word that has a transferred or figurative meaning (e.g. large and mouth for conceited). Or icons for verb and adverb could be compounded to form a new verb (e.g. walk and fast for run).
The design of the language and the grammar can best be evaluated when people are actually learning and using the system. This will be addressed in the next section. But before we can test the system, we must ensure that the icons are developed well. Did we follow the principles we stated for icon design in chapter 3? Are the icons truly universal? How many of the icons are conventional, and how much time does it take to learn these icons? Are the conventional icons consistent enough so one can predict the meaning of other conventional icons?
To answer all these questions, we need to test the icons with people from various parts of the world. We need to set up a survey for testing the icons. The creation of a survey consists of several steps.
Step 1: Determinig the Target Audience
Ideally, the icons would be tested with speakers of all languages in the world. This would be, of course, extremely difficult to do. We need ways to test the icons people from as many cultures as we can. America is a multi-cultural country and even in our own neighborhood we find various clubs or organizations of people from different cultures (Italians, Greeks, Koreans, etc.). There are also lots of ESL classes (English as a Second Language) where we find a diversity of people. The problem with this, however, is that these people may already be biased since they live in the US and may have become familiar with our culture or at least our way of representing things.
Step 2: Determinig the Distribution Medium
To be sure that the tests are not biased, we need to test with people that not only speak different languages, but also live in other countries. One good solution is the Internet. Through the World Wide Web, we can reach many people of different cultures. We could make an electronic survey and put it on the Web or cross-post it to newsgroups that deal with certain cultures (e.g. soc.culture.netherlands). We recognize that this approach is not ideal in that people that use the Web are technologicallly literate people, and presumably relatively aware of other cultures. Furthermore, these people, being familiar with computers, have an unfair advantage of already being familiar and exposed to lots of icons.
Step 3: Designing Questions
There are several different kinds of questions we could ask when testing the icons in our
survey:
These questions assume that the test subjects are familiar with the English language. It would be better to translate the surveys into different languages to ensure more people will understand and participate in the survey. When doing a translation, it is wise to always translate back (preferably by a different person) to ensure that no meaning was lost in the translation. One could also have two people do the translation and then compare the translations to see the similarities. For reasons of translation problems, open ended questions may not be such a good idea.
When choosing icons to test it is important to realize that people will have much more difficulty understanding and recognizing abstract icons (i.e. icons that represent some abstract meaning like to believe). There are several sorts of surveys that can be set up when you make a distinction between abstract and concrete icons. When considering concrete icons, besides the types of questions mentioned above, there is another type of question one can ask the subject. With concrete icons, the major thing we need to find is the most prototypical version of the icon. We can do this by testing in the following way:
5. prototypical image: Give the subject several versions of icons that are generally recognized everywhere, like a tree, and ask him or her "which icon do you think best represents a tree?" Also, give the subject several versions of icons that are represented differently accross cultures, like a mailbox, and ask the same question.
For abstract icons we cannot use this approach. Since abstract icons are conventional (i.e. must be learned), they are harder to test and may require some explanation about what their primitive combining features or identifiers stand for. The test would incorporate a learning/teaching phase first. For example, we need to tell the subject that he/she is looking at icons that represent verbs, or that icons with an oval border are verbs. We also need to tell him/her that, for example, an icon with an arrow in it is a verb of motion. Once we have instructed the subjects, we can present them with the questions like the four mentioned above. We could of course have a separate group who would test the icons without any instruction as well, for comparison.
Step 4: Advertising the Survey
Once our web survey is set up and put on the Web, we need to let as many people know about it as we can. We can advertise the survey in newsgroups (soc.culture.*) and newspapers. We can ask cultural clubs and organizations to take the survey. We can ask classes in schools to participate in the survey. These classes can be from schools all around the world. We can write these schools and ask them to give the survey in some classes and then send the results back. We may need to give people some incentive to participate in surveys, like giving away prizes or award students with college credit.
For a complex system like VIL, designing icons that are totally language independent and culturally neutral may prove impossible.
Fortunately users do not require perfect icons, only recognizable ones. One can use images not common to every culture provided that users can still recognize them. Most users will still recognize the following icons, though not common to every culture:
An Arab will still recognize this icon as being an icon of a book, even though books in Arabic, which reads right-to-left, would have the spine on the right.
A Japanese will still recognize this icon as being a symbol of greeting among people, even though in the orient a bow is more common as a greeting than a handshake.
A Chinese user will still recognize this icon as an icon for silverware (concrete) or food or restaurant (abstract), even though chopsticks and a bowl may be more common in China.
Once we feel comfortable about the appropriateness and recognizability of our icons, we can test the system (VIL) itself for learnability, encodability, and decodability aspects.
Testing the implementation
Besides testing if the icons are universal, we also need to test the system itself and see if it is easy to learn and use. Shneiderman mentions five measurable quality criteria:
Time to learn the system and retention over time are ways to test our principle of learnability. Speed of performance and rate of errors by users are ways of testing our principles of encodability and decodability. Subjective satisfaction is not easily tested, and has to be asked the users in a survey.
The following are some possible scenarios for tests. The specifics of the tests (e.g. times of instruction, error rates, speeds) are preliminary and subject to change, but are included to give some idea of how tests could be done. An acceptance test may then be:
Let’s assume we have 35 test subjects who are unfamiliar with the system, but are familiar with computers. They will be given one hour of training on how the program works (composing some simple sentences). At least 25 of the 35 should be able to produce and decode, within half an hour, 70 percent of the messages in the enclosed benchmark test (the messages that are in VIL need to be decoded, the message in natural language need to be encoded in VIL).
A second test may be:
After 4 half-days of regular use of VIL, 20 of these 35 subjects should be able to produce, within 20 minutes, some advanced messages in the second benchmark test, and it should take less than 10 errors. This second acceptance test captures performance after regular use.
A third test may focus on retention:
After two weeks, at least 15 of the test subjects should be recalled, and should perform a third benchmark test. In 45 minutes, at least 8 of the subjects should be able to complete 70 percent of the tasks correctly.
Test Subjects and Instructors
Like with testing icons, the test subjects have to be people with different linguistic backgrounds. There could be several test groups to tell us what can be achieved with different degrees of learning. The subjects might be divided up into groups that have:
Teaching should be done by teachers with different linguistic backgrounds as well to find out if language determines the way one composes messages. We may also want to see if having subjects taught by a teacher with the same linguistic background as the subjects makes a difference or not, since this teacher is familiar with the way in which the subjects think when they compose messages. We need to find out if the order in which the program takes you through the steps of composing a message is equally advantageous to all people.
Teaching may be done in various ways:
It is expected that the taught subjects in the last group above will do best, followed by the subjects that were taught the grammatical structure.
The resuslts of the icon tests, time to learn, speed of performance, rate of errors, and subjective satisfaction can then be used in determining the viability of a visual interlanguage and to demonstrate how the computer can be used to facilitate a visual interlanguage language.
One of the results of this work will be to identify areas in which the language and the system could and should be extended. It is clear that the evaluation of the tests will indicate areas in which work needs to be done, both to the design of the language and to the implementation of the system. Furthermore, as indicated in section 5.1, learning the system and encoding and decoding messages could greatly be improved by adding help and intelligent features to the system:
This chapter discusses the schedule for VIL after the proposal. It will identify the things that need to be done and will try to set a time frame for each. These times are just an estimate of how long each item will take and may be subject to change.
|
Things that need to be done |
Expected time frame |
|
1. Decide upon the final vocabulary |
1-3 months |
|
2. Design the icons and visual combining features |
2-4 months |
|
3. Design the survey |
1-2 weeks |
|
4. Put the survey on the Web |
1 month |
|
5. Evaluate the survey and making possible changes |
1 month |
|
6. Implement the program |
3-6 months |
|
7. Test and evaluate the program |
1-2 months |
|
8. Possible corrections to language and program |
0-4 months |
|
9. Complete writeup of the dissertation |
2-4 months |
From the table we see that the total estimated time to finish the dissertation is anywhere from 11 months and 1 week to 25 months and 2 weeks. This is probably a too high of an estimate, since many things can be done simultaneously. For example, we do not have to wait for the survey on the Web to finish, but can start the implementation of the program in the meantime.
AIGA, "Symbol Signs," The American Institute of Graphic Arts, 1993.
Aiken, J. R. "Little English," Journal of Adult Education, 1:11-14, June 1936.
Andrews, A.D. "Lexical Structure," in "Linguistics: the Cambridge Survey," Vol1 Linguistic Theory: Foundations, F.J. Newmeyer, Cambridge University, 1988, pp. 60-88.
Ballmer, Th. and Brennenstuhl, W. "Speech Act Classification: A study in the Lexical Analysis of English Speech Activity Verbs," Ed. W. Levelt, Spring-Verlag, 1981.
Beardon, C. "CD-Icon: an iconic language based on conceptual dependency," Intelligent Tutoring Media, Vol. 3, No. 4, November, 1992.
Beardon, C., Dormann, C., Mealing, S. and Yazdani, M. "Talking with pictures: exploring the possibilities of iconic communication," Association for Learning Technology Journal, 1, 1, pp. 26-32, 1993.
Berlin, B. and Kay, P. "Basic Color Terms," 1960s.
Bertin, J. "Semiology of Graphics," translated by P. Berg, Madison: University of Wisconsin Press, 1983.
Bliss, C. K. "Semantography," Semantography publications, Australia, 1965.
Bohne, S., Bunar, M., Gardella, S. and Hynes, P. "Cross-Cultural Icon Test" An Interactive Qualifying Project Report, WPI, May, 1996.
Brown, R. W. and Bellugi, U. "Three processes in the child’s acquisition of syntax." Harvard Educ. Rev. 34, pp. 133-151 (Repr. in "New directions in the study of language," Lenneberg, E. (ed.), Cambridge, Mass., 1964).
Cazden, C. B. "The Acquisition of Noun and Verb Inflections," in Ferguson, C.A. and Slobin, D. I. (eds), Studies of Child Language Development, Publ: Holt, Rinehart and Winston, 1973, pp. 226-240.
Cliff, N. "Adverbs as Multipliers," Psychological Review, 66, pp. 27-44, 1959.
Corbeil, J. "Visual Dictionary," ISBN: 0-8160-1542-9, 1986.
Corder, S. P. "Simple codes’ and the source of the second language learner’s initial heuristic hypothesis." Paper presented at the Colloque ‘Theoretical Models in Applied Linguistics’ IV, Universite de Neuchatel, 1975.
D’Andrade, R. "The development of Cognitive Anthropology," chapter 5: Folk Taxonomies, Cambridge University Press, ISBN: 0-521-45976-1, 1995.
Dorling Kindersley, "Visual Encyclopedia," ISBN: 1-56458-985-4, 1995.
Dutton, T. "Police Motu: iena sivarai (its story)." Port Moresby: University of Papua New Guinea Press, 1985.
Feierbaugh, W. C. "Computer Graphics: Tools for Visualization," Publisher: William C. Brown, 1993.
Ferguson, C. A. "Absence of Copula and the Notion of Simplicity: A study of Normal Speech, Baby Talk, Foreigner Talk, and Pidgins," in Language Structure and Language Use, Stanford University Press, 1971, pp. 277-292.
Ferguson, C. A. "Baby Talk in Six Languages," in Language Structure and Language Use, Stanford University Press, 1971, pp. 113-133.
Ferguson, C. A. "Simplified registers and linguistic theory," in L. Obler & L. Wise (eds.) Exceptional language and linguistics, New York: Academic Press, 1982.
Foley, W. A. "Language birth: the processes of pidginization and creolization," Chapter 9.
Frutiger, A. "Signs & Symbols: Their Design and Meaning," New York: Van Nostrand Reinholt, 1989.
Gittins, D. "Icon-based human-computer interaction," International Journal for Man-Machine Studies 24, 1986, pp. 519-543.
Gode, A. and Blair, H. E. "Interlingua: a grammar of the international language," Storm Publishers, New York, 1955.
Hall, R. "Pidgin and Creole Languages," Cornell University Press, 1966.
Head, W. D. "Needed: A Spare Tire Language," Rotarian, 50:28-30, March 1937.
Horton, W. "The Icon Book: Visual Symbols for Computer Systems and Documentation," John Wiley & Sons, ISBN: 0-471-59900-X, 1994.
Huggins, W. H. and Entwisle, D. R. "Iconic Communication: an annotated biography," The John Hopkins University Press, 1974.
Hymes, D. Commentary. In A. Valdman and A. Highfiled (eds.) "Theoretical orientations in language studies." New York: Academic Press, 1980.
International Encyclopedia of Linguistics, William Bright (ed.), Oxford University Press, 1992.
Jackendoff, R. "Semantics and Cognition", Chapter 10 "Nonspatial Semantic Fields and the Thematic Relations Hypothesis," The MIT Press, 1983.
Johnson, J. E. "Excerpts," from Basic English, The Reference Shelf. Vol. 17, No. 1, The H. W. Wilson Company, N. Y., 1944, p. 58-68.
Justeson, J. and Katz, S. "Co-occurences of antonymous adjectives and their contexts," Yorktown Heights, NY: IBM Research Report, 1989.
Kamman, W. F. "The Problem of a Universal Language," Modern Language Journal, 26:177-182, March 1942.
Kiger, J. "The depth/breath trade-off in the design of menu-driven user interfaces," International Journal of Man-Machine Studies 20, 1984, pp. 201-213.
Kummel, P. "Formalization of Natural Language," Springer-Verlag.
Levi, J.N. "The Syntax and Semantics of Comples Nominals," New York: Academic Press, 1978.
Levin, B. "English Verb Classes and Alternations: A Preliminary Investigation," The University of Chicago Press, 1993.
Liebelt, L., McDonald, J., and Stone, J. "Searching for items in menus: the effects of organization and type of target," Proc. Human Factors Society, Twenty-Seventh Annual Meeting, 1983, pp. 834-837.
MacMillian, "The MacMillian Visual Desk Reference," MacMillian Publishing Group, ISBN: 0-02-531310-X, 1993.
Marcus, A. "Graphic Design for Electronic Documents and User Interfaces," ACM Press, New York, 1992.
Matsukawa, T. and Yokota, E. in Pustejovsky, J. and Bergler, S. (Eds.) "Lexical Semantics and Knowledge Representation," Proc. of the First SIGLEX Workshop, Berkeley, CA, June 17, 1991, Springer-Verlag, Chapter 21 "Development of the Concept Dictionary -- Implementation of Lexical Knowledge".
McCawley, J.D. "Lexical insertion in a transformational grammar without deep structure," in CLS 4, 1968 reprinted in McCawley "Grammar and Meaning," 1973.
McCloskey, J. "Syntactic Theory," in "Linguistics: the Cambridge Survey," Vol1 Linguistic Theory: Foundations, F.J. Newmeyer, Cambridge University, 1988, pp. 18-59.
Mealing, S and Yazdani, M. "A Computer-based Iconic Language," Intelligent Tutoring Media, 1(3), pp. 133-136, 1992.
Mealing, S and Yazdani, M. "Communicating Through Pictures," from the World Wide Web, Department of Computer Science, University of Exeter, England, 1992.
Mencken, H. L. "The American Language," Copyright, Alfred A. Knopf, New York, 1936, p. 603-607.
Miller, G.A. and Fellbaum, C. "Semantic Networks of English," in Levin and Pinker
(Eds.), Lexical & Conceptual Semantics, Blackwell Publishers Ltd, ISBN: 1-55786-354-7, 1995.
Morris, A. V. "A Language Cut to Fit the World," Rotarian, 53:36-39, November 1938.
Mullet, K. and Sano, D. "Designing Visual Interfaces," SunSoft Press, 1995.
Nadin, M. "Interface Design and Evaluation-Semiotic Implications," in Hartson, R. and Hix, D. (eds.), Advances in Human-Computer Interaction, Volume 2, 1989, pp. 45-100.
Neurath, O. "International Picture Language," Department of Typography and Graphic Communication, University of Reading, England, 1978.
Ogden, C. K. "Basic English: A General Introduction with rules and grammar," London: Kegan Paul, Trench, Trubner & Co. Ltd., Broadway House, Carter Lanes E.C., 1930.
Ogden, C. K. "Basic for Science," London: Kegan Paul, Trench, Trubner & Co. Ltd., Broadway House, Carter Lanes E.C., 1942, 514p.
Peirce, C. "Collected Papers," Cambridge, MA: Harvard University Press, 1931 (excerpted in Buckler, Justus, ed., Philosophical Writings of Peirce, New York: Dover, 1955).
Richards, I. A. "Basic English and Its Applications," Royal Society of Arts. Journal (London), 87:735-755, June 2, 1939.
Richards, I. A. "Basic English and Its Uses," W. W. Norton & Company Inc., New York, 1943.
Rosch, E., Mervis, C.B., Gray, W., Johnson, D. and Boyes-Braem, P. "Basic Objects in Natural Categories," Cognitive Psychology, 9, pp. 382-439, 1976.
Schank, R. "Computer Models of Thought and Language," W. H. Freeman, San Francisco, 1973.
Schank, R. "Conceptual Information Processing," North-Holland, New York, 1975.
Schumann, J. "The Pidginization Process: A Model for Second Language Acquisition," Newbury House Publishers, 1978.
Shneiderman, B. "Designing the User Interface: Strategies for Effective Human-Computer Interaction," Addison-Wesley, 1992.
Smith, D. M. "Pidginization and language socialization: the role of marking," Unpublished paper, Georgetown University, 1973.
Smith, E.E. "Theories of Semantic Memory," in W.K. Estes (Ed.), Handbook of Learning and Cognitive Processes (vol. 5), Hillsdale, NJ: Erlbaum, 1978.
The National Commission on the Enrichment of Adult Life, "A General View of the System of Basic English," Committee Report on Basic English for Adult Education, Washington, D. C. mineo p. 2-19.
The Orthological Committee, "Notes on Basic English," Number One, A short Account of the System, October 1940.
Todd, L. "Pidgins and Creoles," Language and Society Series, Publ: Routledge & Kegan Paul Ltd, 1974.
Tversky, B. and Hemenway, K. "Objects, Parts, and Categories," Journal of Experimental Psychology: General, 113, pp. 169-193, 1984.
Wallace, D., Anderson, N., and Shneiderman, B. "Time stress effects on two menu selection systems," Proc. Human Factors Society, Thirty-First Annual Meeting, 1987, pp. 727-731.
Walsh, C. "The Verb System in Basic English," American Speech, 8:137-143, December 1933.
West, M. P., Swenson, E. et al. "Vocabulary Limitation," Journal of Education, London, 71:582-584, September 1939.
West, M. P., Swenson, E. et al. "A Critical Examination of Basic English," Toronto 1934.
Winston, M.E., Chaffin, R. and Herrmann, D. "A Taxonomy of Part-Whole Relations," Cognitive Science, 11, pp. 417-444, 1987.
Yazdani, M. and Goring, D. "Iconic Communication," Internal Report, Department of Computer Science, University of Exeter, England, 1990.