Greg Milette: Profile of CSRL
WPI Computer Science Department
Spring Semester 2000
CS 538 Expert Systems
Profile form by Prof. Dave Brown
THE PROFILE QUESTIONS
GENERAL
Domain:
e.g. Chemistry, Electronics, Computers, Medicine, ... (plus specific
sub-area).
MDX (Cholestasis) - a liver disease in which there is a lack of bile flow from the liver to the intestine
CSRL - any classification task
Main General Function:
e.g. Diagnosis, Prediction, Planning, Data Interpretation, ...
Diagnosis
System Name:
CSRL - Conceptual Structures Representation Language
Dates:
1980 - 1986 (inferred from the dates of the research papers)
Researchers:
Chandrasekarban, Gomez, Bylander, Mittal
Location:
Ohio State University
Language:
CSRL (which has a lisp-like syntax)
Machine:
Unknown
Brief Summary:
When someone has a simple illness a doctor is able to arrive at a diagnosis and send that person home with the appropriate treatment. At other times when the illness is complex, the doctor may send the sick person to a specialist who can better determine the exact treatment. This is the principle behind which CSRL is based. Notice that the doctor is able to handle "simple" cases, however when it becomes too complex he knows only enough to send the person to the correct specialist. Knowledge and reasoning in CSRL are modeled as a group of "specialists" who work just as real doctors do.
Related Systems:
a) MDX
b) Red, Automech, Weldex
c) None.
CATEGORY TWO
Characterization of Givens:
What is the information:
List of specific symptoms the patient is suffering from, mostly relevant to the liver disease Cholestasis.
Or
Acquired through asking the user or a database questions.
Characterization of Output:
The output is all the specialists who were activated. It is a list of likely diagnosis, and unlikely diagnosis.
Here is an example of the output:
(answer (rejected delivery -2)
(rejected mixture -3)
(established vacuum 3)
(rejected bad-gas -3)
(established carburetor-gasket 3)
(unknown vacuum-hoses 1)
(established fuel-problems 2))
Fuel problems, carburetor-gasket, and vacuum were all established. The others were rejected or unknown.
Characterization of Data:
Is the data reliable?
The data is assumed to be reliable.
Is the data complete?
Data is complete as possible.
If there is incomplete data, the system will not give an incorrect solution. Part of the specialists, known as the knowledge groups, will have to be encoded with the knowledge on how to handle incomplete data.
If a specialist has not been fully implemented the system will query the user to answer for the missing implemented expert. A sample question of this type would be:
For each disease listed indicate whether it is established
Unlikely or unknown . Answer Y-N-U
Post-Oper?$N
Anesthetics?#N
Message (Answer (Unlikely Anesthetics By-User)
(Unlikely Post-Oper By-User))
Since the specialists Post-Oper and Anesthetics are not implemented the system asks the user if it is establish instead.
Generic Tasks:
Which Generic Tasks it obviously includes, explicitly or implicitly?
Classification(diagnosis)
Hypothesis matching (uncertainty handling methods).
Not heuristic classification because it does not do data abstraction.
Theoretical Commitment:
Does the system have any theoretical underpinning? Is it claiming to
show that some theory of its type of problem-solving is correct? Is
the method used claimed to work for other similar domains?
CSRL is intended to demonstrate the effectiveness of having knowledge directly influence the strategic decisions made during the problem solving.
Reality:
Is there any psychological validity to the method used the structure
of the knowledge, the control mechanisms? That is, is it a system
that is merely a simulation of result or is it in any way a simulation
of method at a cognitive level?
The system models the way doctors utilize specialists. General doctors have the ability to diagnose and treat common cases. In addition to this, they have enough knowledge about the diseases they do not know how to treat to identify the correct specialist who can. The language was developed to embody this concept.
CATEGORY THREE
Completeness:
Has the system been fully implemented?
Yes.
Use:
Has the system been used with real users from outside the original
development situation?
Yes. Several systems were developed using CSRL.
Automech -
Domain: Car fuel system diagnosis
Organization: 34 specialists.
Results:
Easy for non-computer experts to understand.
Easy to debug because the code is modular and so can be tested incrementally
Took 4 people (2 car experts and 2 computer experts) five man-months of time to develop, 30% was expert time.
Red -
Domain: - red blood cell antibody identification
Organization: Three components.
- Database - maintains patient records
- Overview - assembles a composite hypothesis
- CSRL - rules out antibodies to simplify overview
Has the system been used with real users in the user's own working
environment?
MDX: no
Automech: no
Red: no
Some industrial systems were developed such as WELDEX.
Weldex identifies defects in welding. It works because they usually happen in specific areas so CSRL specialists can be designed specifically for each area and problem.
Performance:
Are there any performance measures available?
For MDX, several cases were "tried" and the results were "in all cases the diagnostic behavior was natural and closely corresponded to the physicians reasoning."
How was the system evaluated? How did it fare?
Automech provides a glimpse into how the language performs when it is applied to non-medical domains. The system tends to be over cautious. This is because in the medical domain, the system had to be absolutely certain a hypothesis was true, however in an automobile domain, it was not so urgent. Also, the meaning of "established" had different meaning in the auto domain. It meant "this hypothesis is worth pursuing" rather than "the hypothesis is certain." As a result, in automech it is unclear which specialist to pursue next. Some mechanism should be provided to settle this issue. Despite these faults, Automech is still viewed as a successful system.
CATEGORY FOUR
Phases:
Is the system organized into distinct phases of different activity?
Distinct subtasks? What are they?
CSRL has one main phase of activity. That is establishing or rejecting all it=92s hypotheses. During this process, the subtasks started by using the different messages that the system passes, such as establish and refine, to activate other specialists. The order of these is determined by the individual specialists. There is another phase, which might be omitted in some CSRL systems, is to synthesize the results with an overview critic that decides which members of the established hypotheses are required to account for a patient=92s symptoms.
Subfunctions:
Despite the fact that the system has a single main function (e.g.,
diagnosis) does it use other types of problem-solving as part of the
system? (see the answers to the last question for clues as to whether
this answer might be yes). What kind(s)?
e.g. Diagnostic systems sometimes select therapy too; this can be
done by synthesis, or by selection (which is often a kind of
classification that is, can this disease situation be classified
as one that this drug can help?)
CSRL is very specific, it only does diagnosis. Other components need to be added to do more synthesis and to handle data. Handling data might involving making generalizations or reasoning about how reliable it is.
Use of Simulation or Analysis:
Does the system use a numerical simulation or analysis, either done by
itself or by some package, during its operation?
No, everything is symbolic, including the certainty factors.
System/Control Implementation Architecture:
i.e. The overall architecture
e.g. meta-rules + rules, blackboard + knowledge sources, production
rules, active agents, activation nets ...
The specialists are organized into a classification hierarchy, where the top nodes are general and the leaf nodes are very specific. The implementation uses a blackboard to coordinate the specialists. The blackboard has four main sections:
=95Active-hypotheses
=95Established-hypotheses
=95Rejected-hypotheses
=95Suspended-hypotheses
A hypothesis is suspended if there was insufficient evidence to continue exploring it. However, the blackboard can be used to determine that these are never the less, worth exploring to establish all reported symptoms.
CATEGORY FIVE
Characterization of Structure Knowledge:
i.e. Is it grouped into types? What types? What are the types used
for? Do they correspond to phases of the system?
e.g. component knowledge, chemical knowledge, functional knowledge,
causal knowledge, ...
Each specialist contains a group of knowledge. The detail of this knowledge is determined by how high the specialist is in the hierarchy. The higher the specialist is in the hierarchy the more general the knowledge. There are several types of diagnostic knowledge stored in each specialist. Knowledge needed to confirm or rule out a hypothesis are included and used as part of the Establish-Refine phase of the system. Another type of knowledge is about how to reason with the uncertainties in the system. Lastly, there is control knowledge. This allows the specialist to control which subspecialist to activate next.
Characterization of Process Knowledge:
i.e. For active knowledge, how would you characterize the effect of
that knowledge?
e.g. <Partial Situation Description> --> <Classification>.
<patient data> -> <disease category>
Patient data is analyzed until a disease category can be confirmed.
Deep or Surface:
i.e. Is the system using deep knowledge or is it only using surface
knowledge? How would you characterize it? Are there levels of
representation or reasoning?
e.g. If there is a qualitative simulation then it could be argued that
it is using deep knowledge.
Each specialist contains shallow knowledge about it=92s particular specialty. These specialists are organized in levels of reasoning, which give the system an overall deep knowledge of the subject.
CATEGORY SIX
Search Space:
What space or spaces does the system search through? Is the search
explicit? Are the states represented explicitly? How? What do the
states represent? Complete alternative solutions, Solution
refinements, Plans, ...? How big is the space?
The search space for MDX is a set of causes for the liver disease Cholestasis and is described as being complicated and large enough to test. The reason that Cholestasis was chosen was that the determinators that show whether a symptom is present or not can be acquired easily. An example of a cause would be "duct stone" or "tumor". Multiple states could be included in a solution to produce a complete diagnosis. The search is explicit and driven by the specialists' knowledge as described in the next question answer.
Space Traversal:
How is the space traversed? What does it mean, in terms of the
problem, to move from one part of the space to another? Sub-problem
decomposition, gradual refinement, down pre- determined hierarchy,
instantiation, ...
The space is traversed through the guidance of the specialists. The system starts with a hypothesis which is general enough to cover the entire search space. Then the hypothesis is refined as it is passed between specialists, which reduces the search space. For example, when the system starts, the search space is large because it starts at the highest level specialist named "gp" or "internist." The space is further refined when the lower level specialist "liver" establishes. Thus, the search space was refined from containing all disease causes in the specialist "gp" to just liver disease causes within "liver." The liver subspecialist then takes over and further refines the search space by activating other subspecialists. As this process continues the hypothesis is continually refined until a specific hypothesis is reached.
Search Control Strategy:
Does the system use a strategy that is expressible in terms of the
problem? or does it appear just to be an AI "technique" that happens
to fit?
Uses a blackboard to coordinate between specialists and also to allow the specialists to work in parallel. This can prevent situations such as, one disease being secondary to another or a disease having more than one cause.
The control strategy that decides which specialist to activate next is expressible in terms of the problem and models the expert thinking of medical doctors.
Standard Search Strategies:
Does the system explicity use Generate & Test or Means-Ends Analysis?
(N.B. be careful, almost everything is a form of G&T!)
The hypotheses are generated by the specialists based on the patient data and then they are tested by being either established or rejected by other specialists in the system. An example of one iteration of a generate and test loop is an establish message passed to a subordinate specialist. The higher level specialist suggests a condition to the lower level specialist saying "(Establish Mixture)
" and the mixture specialist evaluates this generated hypothesis with the message "(Rejected Mixture -3)
". The next step is to generate another hypothesis based on this new information.
Search Control Characterization:
e.g. depth first, best first, breadth first, knowledge-based, random, ...
It is a knowledge-based search. Each specialist knows what order to explore the classification tree based on domain knowledge or by what certainty values resulted were returned from its subspecialist .
Subproblems:
Is evaluation of partial solutions possible?
(that is, can the system know when it is on the right track?)
Partial solutions are possible since each specialist can conclude something about the hypothesis at some level of detail based on where it is in the classificaion hierarchy.
Are the subproblems independent? totally? partially?
The sub problems are not usually independent, since each specialist usually assumes its parent has been established before it evaluates the problem. Once that is taken into consideration, a specialist could solve a sub problem independently.
Search Control Representation:
Where is the search control knowledge? How is it expressed? Explicitly?
The knowledge is defined explicitly and differently for each specialist. The specialists use a general procedure for establish and refine, however this may be modified to suit the needs of the system. Here are some possible establish and refine procedures:
=95(Establish (if (GE relevant 0)
then (SetCertainty self summary)
else (SetCertainty self relevant)
If the certainty value of the relevant knowledge group is greater than 0, then set the value to the knowledge group summary, otherwise use relevant.
=95(Refine (for specialist in subspecialists
do (Call specialist with Establish)
(if (+? Specialist)
then (Call specialist with refine)))
Try to establish all specialists, if they become established then refine them.
Search Control Strength:
Is it based on a very domain independent and knowledge-free method (a
"weak" method) or is it very domain dependent and knowledge-full (a
"strong" method).
CSRL has a strong method of control and is very domain dependent. Each specialist must decide, based on its knowledge, which specialist to pass control to next.
CATEGORY SEVEN
Failure Method:
When part of the system fails, or reaches an incorrect conclusion, how
does the system attempt recovery? What type of knowledge does it use?
is it global or local? Is there a significant vari- ation from the
normal flow of control? If it uses back-tracking, what kind does it
use?
A "failure" could be defined as when a specialist does not establish and rejects itself. In this case the parent specialist handles this failure by determining which specialist will be activated next based on domain knowledge. The parent specialist is used as a medium for deciding back-tracking.
There are three other types of failures that the system could face, etraneaous evidence, incorrect solutions, and incomplete solutions. If there is extraneous evidence, the system will not suffer because none of the specialists will use this evidence thereby marking this knowledge as irrellivant If there is an incorrect solution the system will need to be debugged to correct that portion of the classification hierarchy. The debugging process continues until the system gives correct answers. If there is an incomplete solution the system may look to explore any hypothesis that was temporarily suspended and that may, if explored, complete the solution. If the solution is still incomplete, then unfortunately the system will only be able to produce a partial solution that may not explain all the observed evidence.
Uncertainty:
What is it that is uncertain?
- a piece of data?
- a piece of knowledge?
- a solution to a subproblem?
The certainty of the solutions is what is considered uncertain. An example of an uncertainty within the domain of automech is since the gas tank has been refueled several times, bad fuel is rejected. Rejected is the term used to mean highly uncertain, when is then further symbolized by -3.
Data uncertainty is different from the solution uncertainty and is not accounted for because it is out of the scope of CSRL. An example of this type of uncertainty is how certain the system is that the data is true. Another component should do this work for CSRL although it could be incorporated into the reasoning of the specialists.
Management of Uncertainty:
Does the system use probabilities, scoring values, a fixed range of
certainty values, ...? What does a value mean? What is the method of
combination? Is it a global method, or does it vary? (i.e., a local
method). Are there any apparent problems?
Uncertainties are described symbolically on a scale from -3 to 3 and are interpreted as a degree of fit. Symbolic means that these numbers represent symbolic values such as reject(-3), unlikely, possible, very likely, and definite(3). There are no calculations made based on these symbols. This is in contrast to other systems, which implement a uniform way of combining certainties by way of a formula. The disadvantage of using a formula to combine certainties is that symbolic measures of certainty must be converted to and from a numeric scale which could lead the system to make false assumptions. CSRL ensures that the experts explicitly state how the probabilities will be combined so that the system makes no assumptions and therefore makes less errors.
Management of Time:
Is there any time-dependent data? How does it affect the problem-solving?
No.
CATEGORY EIGHT
Knowledge Representation Method:
e.g. rules, procedures, tabular, semantic nets, logic, ...
Knowledge is represented as procedures within entities known as knowledge groups. The knowledge is similar to having rules with boolean terms. Here is an example of a knowledge group:
(knowledge-groups
(summary (table
(conditions (ask-tfu? (did the problem start after the last fillup?))
(ask-tfu? (has the problem gotten worse since the last fillup?))
(ask-tfu? (have you tried a higher grade of gas?)))
(match (if (t ? ?) then 3)
(if (f t ?) then 3)
(if (f f f) then 3)
(if (u u u) then 0)
(if (? ? ?) then -3)))))
The name of the group is summary. It makes decisions on three "conditions." Here the system will either ask the user or a database for the answer. The result is interpreted within the "match" statement. Each term of the match corresponds to the answers to the three conditions. The first one (t ? ?) means if the first condition is true then the certainty of knowledge group summary is 3 because it doesn=92t care about the other two conditions. "u" stands for unknown, "f" stands for false, and "?" stands for don't care.
Knowledge Representation Generality:
Is there a special language for this system or does it use a general method?
Is the method provided by some Expert System building tool?
This is a general language for building classification systems. It is not a full programming language, but has facilities to execute lisp statements. It is flexible enough to handle many classification problems, but any system developed with it must subscribe to the methodology of using specialists. If a CSRL program diverges too greatly from this methodology another language should be used.
Knowledge Structuring:
Is the system based on a hierarchy, a network, ...? Does this
structure correspond to the domain in some way? Does it correspond to
the problem-solving being carried out?
Knowledge is structured in a hierarchy and is distributed among each specialist within that hierarchy. This is similar to the domain of medical diagnosis, where at the top of the hierarchy is a general physician, and below him are more specialized physicians who have more specialized knowledge. Just like the general physician will review a patient=92s case and sometimes send him to a specialist, CSRL problem solves by having each specialist review a hypothesis and then select which subspecialist to further refine the hypothesis.
CATEGORY NINE
Alternative Representations:
Does the system use alternative representations for the same piece of
knowledge in order to allow for alternative solution methods?
No. The diagnosis knowledge must be in the form of knowledge groups.
Alternative Solution Methods:
Does the system use alternative methods to reach the same solution(s)?
The system has many ways to modify the Establish and Refine procedures to find alternate solutions. For example, suggestion rules can find a solution fast because the system does not have to review ever specialist, just the one that is suggested. If the system did not use suggestion rules, it would still find the answer, but it would take the system longer and perhaps through a different path.
Optimization:
Does the system produce the best answer? Sometimes? Always?
The system only sometimes finds the best answer because some hypotheses may be left suspended when the user terminates the system or when the system terminates automatically.
Multiple Results:
Does the system produce more than one result? If it obtains several
does it try to order them by evaluating them? Does it try to combine
them? If so, how?
Yes, it produces all the established and rejected specialists. There may be multiple hypotheses contained in this list so the system must try to deduce if it is a complete diagnosis or not. This would be done by a component, not included in CSRL, that would review the resulting knowledge and ensure that all symptoms have been explained by established hypothesis. One way of ensuring that all symptoms are explained is by processing all the suspended hypotheses to check if all the evidence explained in them are included in the established hypotheses. Any hypotheses found to contain symptoms that are not explained by establised hypotheses should be reactivated.
CATEGORY TEN
Interaction:
i.e. Is there anything interesting about the way the system interacts
with the user? (for both input and output).
e.g. menus, pictures, diagrams, picking, color, sound, ...
There is a graphic interface to display and modify the specialist hierarchy, but the system uses text input and output.
Data collection:
Does the system require all its data (that is, details of this
particular problem) before execution or does it allow/require
incremental addition? If incremental, does it vary the type/number/
order of data gathering actions (e.g., questions of a data-base or
user) depending on the prob-lem being solved?
The system can run starting with no data or with a list of symptoms. While it is executing if it is missing some information it will query the user each time. It will also query the user if there is an specialist missing that the system requires.
Here is an example of the system querying the user for information:
Can you see cracks in the carburetor gasket?
If carburetor gasket was not implemented the system would have asked:
Is carburetor gasket established?
The user can enter Y/N/or U(unknown).
Data format:
i.e. In what form is the data given?
e.g. patient records, LISP, Natural Language, ...
The data is given as answers to the system=92s questions or as a symptom description. Here are some example symptom descriptions:
Janudice
Vomitting
(wtloss (amount (lb 16.0)))
(pain f abdomen)
Acquisition:
Does the system have any way of acquiring knowledge from the expert
user? Does it guide the user? Does it have a way of validating the
knowledge?
There is no automated way for the system to acquire knowledge from the user. A knowledge engineer must acquire the knowledge and code it in CSRL.
Learning:
Does the system learn from its own performance
No.
Explanation:
Does the system have the ability to explain where its result came
from? Is this obtained from a trace of the goals and subgoals formed
during system execution?
It can produce an execution tree and the list of specialists it established and rejected.
CATEGORY ELEVEN
Strengths:
What do you think are the strong points of the system?
The system does not use a global calculus to combine certainty values and therefore incurs less error.
It has readable and code that models experts reasoning. A consequence of this is that human experts are able to understand how the system is operating and can therefore be more involved in the system development.
It is easy to debug because it can be developed in a modular fashion. The system does not have to be completely implemented so each part can be tested individually.
Weaknesses:
What do you think are the weak points of the system?
The system is limited to classification and hypothesis matching. As a result, many systems that use CSRL will need to include other components in the design.
CSRL is not a full programming language and thus lacks control structures. This is also somewhat of a strength because the system is highly specialized for the task that it does. However, while the system is being used for its type of classification and hypothesis matching it works well.
CSRL does not have a built in way to prioritize
which specialist will be invoked next. This would include additional control that is needed to speed up search in the cases where several specialists are established with equal certainty and the system needs to know which to expand next.
CSRL does not have a well defined way to stop the program. Although an overview critic can be used to examine the results, currently, it can only function after the CSRL has completed.
Other:
Any other comments that would help to characterize this system.
The main contribution of CSRL is that the knowledge is involved in all parts of the problem solving. This is in contrast to systems that keep the knowledge separate from the problem solving engine as the diagram below shows.