The evaluation of this work is rather difficult since it is mostly exploratory and the main goal is to extend the SiFA paradigm to include a model of conflicts and negotiations.
The goals of this work were previously identified and expressed in the form of the following questions:
The proposed conflict hierarchy, we believe, covers all meaningful SiFA conflicts. Although negotiation graphs for all conflict types have not been presented, COSINE has demonstration all conflict types and their resolution with simple negotiation strategies. Sample runs of COSINE are in appendix B.
The structure of the conflict hierarchy assures that the nodes higher in the hierarchy have all the general aspects of the nodes below them. The hierarchy is an is-a hierarchy, so a leaf node inherits all aspects of its parent node. The resolution strategies have not been fully associated with the conflict hierarchy. Most of the negotiation process has been discussed in relation to the hierarchy but the execution of the negotiation strategy has not been linked to the conflict hierarchy.
COSINE implements the proposed SiFA model successfully. Sample rules and demonstration runs can be found in the appendices. Although COSINE does not take advantage of the possibility of using inheritance in defining the agents, especially their negotiation knowledge, the implementation is simple enough to allow for automatic generation of the code by a knowledge acquisition tool that provides a graphical front end to the designer of a SiFA system.
It is not easy to evaluate this since the implementation was not based on the SINE platform. COSINE is not a platform to build SiFAs, rather it is a rule based implementation of a SiFA based system. It is possible to replicate most of the functionality of a SINE-based SiFA system using a purely rule based approach like COSINE. The inverse is not true though. The negotiation knowledge built into the COSINE agents are not readily available in SINE agents. Also, the knowledge of the agents in the SINE platform are not as clearly isolated and declarative as they are in COSINE.
The model does not have any restrictive features, neither does the implementation. The model does not make any claims about learning and can definitely be extended to include it. The implementation would allow history keeping, and learning simple constants, constraints, preferences, etc, but learning new methods, that are currently implemented as rules, may not be possible.
The sample runs of COSINE are very easy to follow. The agents are operating in an interleaved fashion, and it would be possible to have more than one conflict detection and negotiation going on at once if the conflicts did not involve any related entities, i.e., if they were criticism conflicts in different parameter blocks. In the current state of SINE though, one conflict is resolved before a second conflict is detected and its negotiation started. The only reason for this is to make COSINE easier to understand for human experts.
Since the negotiation behavior of the agents are still very simple, there are very definite observed patterns of communication between the agents. This is a result of the extremely restricted vocabulary and strategies currently available to the agents rather than the patterns that emerge by the nature of the conflicts. Still, it is possible for the same patterns to be observed even if the agents are given more powerful negotiation strategies and a broader vocabulary.
Almost all of the goals set forth for this thesis have been met by this body of work as seen from the answers given to the questions above. It is not possible to show more solid results than the demonstration of the ideas presented in chapter 7 using excerpts from sample runs of COSINE. This is due to the fact that the work on SiFA's is still new and the model keeps evolving. A full implementation of a SiFA-based system that is capable of solving real life design problems does not yet exist, therefore it is not possible to say that SiFA-based systems perform better than others in terms of any metric. Even if the SiFA paradigm does not prove to be useful in a practical sense, we claim that it is a very interesting research tool that enables the study of all aspects of multi-agent systems, especially conflict management.
The next chapter discusses some of the possible future research directions using the SiFA paradigm.