Teiresias Profile Sam Holmes Mon, 9 Feb 2004 THE PROFILE QUESTIONS GENERAL Domain: Teiresias operates in the area of medical diagnosis. Specifically the area of identifying bacteremia infections in patients. Main General Function: Teiresias' main function I would say is the ability to form meta-data about it's own knowledge base for use in interaction with a human expert. However, the system has a multitude of small, specialized code chunks to make the human interface has useful and intelligent as possible. System Name: Teiresias Dates: Mycin (which Teiresias uses heavily): 1972-1980 Teiresias: 1974-1977 Researchers: Randall Davis Douglas B. Lenat Location: Stanford University Language: Lisp Machine: DEC PDP-10 under Tenex Brief Summary: Teiresias is a system designed to replace the traditional role of a computer-literate assistant who's task is to act as an intermediary between the domain expert and the expert system. The system has two general components, an explanation module which produces the reasoning behind a diagnosis and a knowledge acquisition module which interactively assists the domain expert in expressing their knowledge about the domain. Related Systems: a) Teiresias makes extensive use of the Mycin system, often adding code directly into Mycin to accomplish its goals. b) Components of Teiresias were included in the Emycin system. c) I am unaware of similar systems working with production rules in this manner. CATEGORY TWO Characterization of Givens: When handling explanation of the expert system's reasoning to the domain expert, the system is given a trace of the mycin's reasoning and what sort of information (up or down the and/or tree) the expert needs. The system has the basic structure of and/or trees and production rules built in. When interactively doing knowledge acquisition with the domain expert the system is given the current system knowledge base and natural language answers to questions from the domain expert. The system has much information about production rules and their structure built in for this component. Characterization of Output: In the explanation module, the system is outputting a natural language explanation of the and/or tree produced by Mycin. In the knowledge acquisition phase the system is producing natural language representations of rules entered by the expert, direction for the knowledge acquisition process to proceed in, and meta information about the knowledge base it is given to work with in the form of suggestions posed to the domain expert. Characterization of Data: Is the data reliable? Teiresias does not have any functionality to debug or detect conflicting rules or rule stepping on one another in the knowledge base. The data in assumed to be reliable in these regards. Is the data complete? No. Teiresias function at its best when it is aiding the domain expert in filling out knowledge bases that are not complete. Generic Tasks: Which Generic Tasks it obviously includes, explicitly or implicitly? Teiresias obviously includes abstraction as a core method, as seen by it's formation of meta-knowledge obtained from the expert system's knowledge base. Theoretical Commitment: Does the system have any theoretical underpinning? Is it claiming to show that some theory of its type of problem-solving is correct? Is the method used claimed to work for other similar domains? Teiresias is showing the power of meta-knowledge in the context of dealing with what is 'expected' in a given situation. These expectations are used to make suggestions to the expert as well as understand the intention of the domain expert using the system. Reality: Is there any psychological validity to the method used, the structure of the knowledge, the control mechanisms? That is, is it a system that is merely a simulation of result or is it in any way a simulation of method at a cognitive level? For the most part, no. Although abstracting and process of elimination are human qualities, their use in the system is from a clear statistical slant and not intended to model human thought. CATEGORY THREE Completeness: Has the system been fully implemented? Yes. The system is fully implemented and functioning. Use: Has the system been used with real users from outside the original development situation? Mycin was used with real users outside of the development system, but Teiresias specifically was not. Has the system been used with real users in the user's own working environment? No. Teiresias was only used by the developers in their environment. Performance: Are there any performance measures available? How was the system evaluated? How did it fare? The system was evaluated subjectively by the developers themselves. They system displayed the power of the approach and the value behind having meta-knowledge at an expert system's disposal. As the 'users' had knowledge about the system's construction they could see many situations where the system would not perform as well as it did in their setting. CATEGORY FOUR Phases: Is the system organized into distinct phases of different activity? Distinct subtasks? What are they? Teiresias has many small subtasks to perform. During the explanation phase Teiresias takes a reasoning trace from Mycin and produces an interactive explanation to the domain expert. During the knowledge acquisition phase Teiresias first determines cause of the error in the production rules with the help of the domain expert. When a new rule or concept is added Teiresias needs to interpret the natural language entered by the domain expert in the current working context. Teiresias then has the task of checking for overlooked or inaccurate information based on the meta-knowledge it forms. Subfunctions: This is not particularally applicable to Teiresias because although Teiresias -works in the general area of meta-knowledge it branches out and used a wide variety of approaches, they are not problem-solving in nature. Use of Simulation or Analysis: Does the system use a numerical simulation or analysis, either done by itself or by some package, during its operation? Teiresias uses simple statistical methods to determine what a prototypical instance of a situation may look like based on the expert program's knowledge base. System/Control Implementation Architecture: The Mycin architecture over which Teiresias works is a production rule system. Within Teiresias the meta-knowledge takes the form of prototypical instances of a situation of data object. This can be used to locate incorrect or missing information as well as provide expected information based on the situation. CATEGORY FIVE Characterization of Structure Knowledge: The prototypical instances which form Teiresias' meta-knowledge are organized in a hierarchy based on specificity. This hierarchy is used to check information entered by the expert by traversing down from general to specific instances. Characterization of Process Knowledge: The meta-knowledge possessed by Teiresias allows the system to form expectations about the current situation which can be applied to suggest additions or deletions from the domain expert's input. Deep or Surface: Teiresias doesn't use the domain information directly, rather the knowledge base is used to form meta-knowledge. My opinion is that the concepts of 'deep' and 'shallow' could be interpreted either way in this case. CATEGORY SIX Search Space: The search space Teiresias uses is the knowledge base itself. The knowledge base is processed using simple statistical approaches to form the meta-information used by Teiresias. The mostly-complete knowledge base used to test Teiresias was from Mycin and contained in the area of 450 production rules. Space Traversal and Search Control Strategy: Teiresias narrows the search space by only considering those production rules which could apply to the current situation based on the addition or deletion of a conclusion supplied by the domain expert. All production rules in the knowledge base which are deemed to apply are used in forming the meta-knowledge. Standard Search Strategies: Teiresias does not use a standard AI search strategy when forming it's meta-knowledge outside of the techniques described above. Search Control Characterization: There is no specific control characterization in Teiresias other than the narrowing of the search space described above. Subproblems: Is evaluation of partial solutions possible? The hierarchy of prototypical instances can be traversed from general to specific and at each point Teiresias checks it's progress, either against the knowledge base and the current situation, or by asking for verification from the domain expert. Are the subproblems independent? The prototypical instances could be considered separate from one-another although when the hierarchy is used the search is dramatically narrowed and because of the structure of the hierarchy the system is able to move in a positive direction from instance to instance. Search Control Representation: The search control representation in Teiresias is represented in the actual code of the system, although primarily the system consists of many specific solutions to small and specific problems which occur when bridging the interaction between the domain expert and the expert systerm. Search Control Strength: The idea of using meta-knowledge to provide situation expectations is a 'weak' approach because it can be applied to many systems independent of the domain. Much of the actual coding of Teiresias however would be considered a 'strong' approach because it contains many methods for solving problems specific and only occurring in the area of domain experts interfacing with expert systems. CATEGORY SEVEN Failure Method: During the interpretation of the domain expert's natural language input, if the system is told it is incorrect, the first failure option is to consider alternative meanings or connotations for the words it identified during the first pass. If that fails the system falls back to simply selecting the next closest match from its list of possibilities. Uncertainty: The uncertainty present in the Teiresias system is the actual meaning behind the domain expert's entered natural language. The knowledge base of the expert system Teiresias is operating above is not considered by Teiresias at the level of uncertainty or accuracy. Management of Uncertainty: The uncertainty in the domain expert's input is handled by the Teiresias-created meta-knowledge as described above. Uncertainty or inaccuracy in the knowledge base itself is not considered or handled by Teiresias. Management of Time: Time is not a critical factor for the Teiresias system. In the interest of providing a responsive interface when the system needs to perform a computation which is going to take a significant amount of time the expert is notified. CATEGORY EIGHT Knowledge Representation Method: The meta-knowledge produced by Teiresias about the knowledge base is represented in a set of prototypical instances of the concept or rule that is being studied. These are stored in a hierarchy descending specificity which provides adequate access and search navigation. Knowledge Representation Generality: The knowledge represented in the Teiresias system is of a form created specifically for the task at hand and the information to be represented. The data structures within Teiresias are not from a standard expert system toolkit or set of resources which could be easily applied to other systems. Knowledge Structuring: The meta-knowedge employed by Teiresias is organized in a hierarchy, organized by item specificity. This organization is general and would apply to a similar approach, regardless of the specific domain the system was performing over. CATEGORY NINE Alternative Representations: Teiresias does not use multiple representations of knowledge it has formed from the knowledge base. Alternative Solution Methods: Teiresias does not use alternate search or solution methods. This was one aspect of the system that struck me through the presentation work was that if the system was not able to produce the correct answer, it was then stuck and did not include mechanisms to handle this contingency. Optimization: The information collected by Teiresias is not guaranteed to be completely accurate and rather is taken to be suggestions for the domain expert based on the knowledge base already entered into the system. Multiple Results: The system produces multiple interpretations of the natural language entered by the domain expert. The order of evaluation is ranked by likely interpretation based on the connotation and meaning of words as well as the expected responses of the domain expert taken from the meta-knowledge. CATEGORY TEN Interaction: User interaction (and the following comprehension) is the main focus of Teiresias. The system is able to interact with the domain expert using only natural language and provide a bridge between the expert system and the computer-illiterate domain expert. Data collection: Teiresias performs with the highest accuracy when there is a large amount of information in the expert system's knowledge base. This is due to the fact that Teiresias used the knowledge base to construct the meta-knowledge from which is produces its interpretations. When the knowledge base is lacking, the performance of Teiresias decreases as well as it relies on this information heavily. Data format: Input to the system is given in the form of natural language. Acquisition: Teiresias is a system for knowledge acquisition. The acquisition process is handled as described above. Learning: Teiresias in theory is able to 'learn' from its performance because as each new rule or concept is added to the knowledge base, more information is available for Teiresias to derive its prototypical instances from. Explanation: One key component of the Teiresias system is its ability to explain Mycin's reasoning behind a particular output. Further explanation is in the section above under subcomponents. CATEGORY ELEVEN Strengths: Teiresias displays an effective use of meta-knowledge to understand natural language from a domain expert within the scope of the expert system. The meta-knowledge produced by Teiresias shows the strength of 'knowing about what you know'. Under ideal circumstances the Teiresias system is able to guide a domain expert through the process of knowledge acquisition through the use of just natural language. Weaknesses: The natural language methods applied in Teiresias are not very adaptable and outside the area of Mycin and a specific domain would not be useful. The process by which meta-knowledge is formed makes use of only basic statistical techniques and although is adequate, has much room for improvement. I was disappointed by how much of the system was 'hard-coded' where it could not be applied to any area outside of production rules, and could also not be applied without heavy modification of the expert system it was running over. Teiresias is dependent on trace information from the expert system it is running over and therefore if there is a code change to the expert system it requires Teiresias to undergo a code change as well to remain compatible. Other: Teiresias is an effective approach to interfacing a domain expert with an expert system, showing the strength of meta-knowledge and what can be accomplished if the task and domain are narrow enough. -------