WPI Computer Science Department Spring Semester 2000 CS 538 æ Expert Systems Prof. Dave Brown Office, Fuller Lab. 131 Phone: (508) 831-5618 Email: DCB@CS.WPI.EDU EXPERT SYSTEM PROFILE For every Exp.Sys. we discuss, the group presenting will be expected to provide the class with a typed profile of the system on paper according to the format provided below. This is to form the basis of the presentation of the system to the class, but this need not be strictly adhered to. For some systems some questions may be easy to answer, for other systems they will be hard. Despite my best intentions some of the questions may overlap. As I expect the discussion in class to contribute to our understanding of each system I think that it's better that the profile be completed after the class in which that system is presented. Get the completed profile to me prior to the next class (by mail or hand delivered) so that I can run off copies. The collected profiles will be an important benefit of this class. THE PROFILE QUESTIONS GENERAL Domain: e.g. Chemistry, Electronics, Computers, Medicine, ... (plus specific sub-area). Main General Function: e.g. Diagnosis, Prediction, Planning, Data Interpretation, ... System Name: e.g. AM, DENDRAL, MYCIN, ... Dates: Rough period of development. Researchers: Main people responsible for R&D. Location: Where R&D took place (maybe more than one). Language: LISP, OPS5, KEE, LOOPS, AGE, ... (maybe more than one). Machine: e.g. Xerox Lisp Machine, Symbolics LISP Machine, VAX, ... Brief Summary: a SHORT paragraph describing the system. Related Systems: a) Systems from which this system was developed; b) Systems developed from this system; c) Systems by others using same methods (clones). CATEGORY TWO Characterization of Givens: What is the information: - given - built into the system; express as abstractly as possible. Characterization of Output: What is the information produced by the system; express as abstractly as possible. Characterization of Data: Is the data reliable? Is the data complete? Generic Tasks: Which Generic Tasks it obviously includes, explicitly or implicitly? classification, state abstraction, knowledge-directed information passing, object synthesis, hypothesis matching, abductive assembly, ... (?) Theoretical Commitment: Does the system have any theoretical underpinning? Is it claiming to show that some theory of its type of problem-solving is correct? Is the method used claimed to work for other similar domains? Reality: Is there any psychological validity to the method used, the structure of the knowledge, the control mechanisms? That is, is it a system that is merely a simulation of result or is it in any way a simulation of method at a cognitive level? CATEGORY THREE Completeness: Has the system been fully implemented? Use: Has the system been used with real users from outside the original development situation? Has the system been used with real users in the user's own working environment? Performance: Are there any performance measures available? How was the system evaluated? How did it fare? CATEGORY FOUR Phases: Is the system organized into distinct phases of different activity? Distinct subtasks? What are they? Subfunctions: Despite the fact that the system has a single main function (e.g., diagnosis) does it use other types of problem-solving as part of the system? (see the answers to the last question for clues as to whether this answer might be yes). What kind(s)? e.g. Diagnostic systems sometimes select therapy too; this can be done by synthesis, or by selection (which is often a kind of classificationæthat is, can this disease situation be classified as one that this drug can help?) Use of Simulation or Analysis: Does the system use a numerical simulation or analysis, either done by itself or by some package, during its operation? System/Control Implementation Architecture: i.e. The overall architecture e.g. meta-rules + rules, blackboard + knowledge sources, production rules, active agents, activation nets ... CATEGORY FIVE Characterization of Structure Knowledge: i.e. Is it grouped into types? What types? What are the types used for? Do they correspond to phases of the system? e.g. component knowledge, chemical knowledge, functional knowledge, causal knowledge, ... Characterization of Process Knowledge: i.e. For active knowledge, how would you characterize the effect of that knowledge? e.g. --> . Deep or Surface: i.e. Is the system using deep knowledge or is it only using surface knowledge? How would you characterize it? Are there levels of representation or reasoning? e.g. If there is a qualitative simulation then it could be argued that it is using deep knowl- edge. CATEGORY SIX Search Space: What space or spaces does the system search through? Is the search explicit? Are the states represented explicitly? How? What do the states represent? Complete alternative solutions, Solution refinements, Plans, ...? How big is the space? Space Traversal: How is the space traversed? What does it mean, in terms of the problem, to move from one part of the space to another? Sub-problem decomposition, gradual refinement, down pre- determined hierarchy, instantiation, ... Search Control Strategy: Does the system use a strategy that is expressible in terms of the problem? or does it appear just to be an AI "technique" that happens to fit? Standard Search Strategies: Does the system explicity use Generate & Test or Means-Ends Analysis? (N.B. be careful, almost everything is a form of G&T!) Search Control Characterization: e.g. depth first, best first, breadth first, knowledge-based, random, ... Subproblems: Is evaluation of partial solutions possible? (that is, can the system know when it is on the right track?) Are the subproblems independent? totally? partially? Search Control Representation: Where is the search control knowledge? How is it expressed? Explicitly? Search Control Strength: Is it based on a very domain independent and knowledge-free method (a "weak" method) or is it very domain dependent and knowledge-full (a "strong" method). CATEGORY SEVEN Failure Method: When part of the system fails, or reaches an incorrect conclusion, how does the system attempt recovery? What type of knowledge does it use? is it global or local? Is there a significant vari- ation from the normal flow of control? If it uses back-tracking, what kind does it use? Uncertainty: What is it that is uncertain? - a piece of data? - a piece of knowledge? - a solution to a subproblem? If all are used, how do they interact? Management of Uncertainty: Does the system use probabilities, scoring values, a fixed range of certainty values, ...? What does a value mean? What is the method of combination? Is it a global method, or does it vary? (i.e., a local method). Are there any apparent problems? Management of Time: Is there any time-dependent data? How does it affect the problem-solving? CATEGORY EIGHT Knowledge Representation Method: e.g. rules, procedures, tabular, semantic nets, logic, ... Knowledge Representation Generality: Is there a special language for this system or does it use a general method? Is the method provided by some Expert System building tool? Knowledge Structuring: Is the system based on a hierarchy, a network, ...? Does this structure correspond to the domain in some way? Does it correspond to the problem-solving being carried out? CATEGORY NINE Alternative Representations: Does the system use alternative representations for the same piece of knowledge in order to allow for alternative solution methods? Alternative Solution Methods: Does the system use alternative methods to reach the same solution(s)? Optimization: Does the system produce the best answer? Sometimes? Always? Multiple Results: Does the system produce more than one result? If it obtains several does it try to order them by evaluating them? Does it try to combine them? If so, how? CATEGORY TEN Interaction: i.e. Is there anything interesting about the way the system interacts with the user? (for both input and output). e.g. menus, pictures, diagrams, picking, color, sound, ... Data collection: Does the system require all its data (that is, details of this particular problem) before execution or does it allow/require incremental addition? If incremental, does it vary the type/number/ order of data gathering actions (e.g., questions of a data-base or user) depending on the prob- lem being solved? Data format: i.e. In what form is the data given? e.g. patient records, LISP, Natural Language, ... Acquisition: Does the system have any way of acquiring knowledge from the expert user? Does it guide the user? Does it have a way of validating the knowledge? Learning: Does the system learn from its own performance Explanation: Does the system have the ability to explain where its result came from? Is this obtained from a trace of the goals and subgoals formed during system execution? If not, how is it formed? CATEGORY ELEVEN Strengths: What do you think are the strong points of the system? Weaknesses: what do you think are the weak points of the system? Other: Any other comments that would help to characterize this system. ------------------------------------------------------------------ profile.txt