R. P. Clement

University of Westminster

Expert Systems and other knowledge-based programming paradigms are based on the idea that complex real-world tasks can be performed by simple methods that interact with a large amount of 'knowledge'. For example, given a large enough description of symptoms that a patient might exhibit and what diseases (if any) they correspond to, a simple program could be written that compared the current patient's symptoms to those on the list. The program could then find the closest match and print the attached conclusion as a diagnosis.

The program that searches the list is a very simple form of an Expert System Reasoning Engine. The engine described here would be very easy to implement.. Given enough knowledge it should give accurate diagnoses in a fair number of cases (patients).

Does this mean that building such an Expert System is easy? No. Expert System development is usually long, often taking man-years of effort. Most of this effort is spent in acquiring the knowledge (list of symptoms and conclusions).

Knowledge acqusition has been identified as the limiting factor in Expert Systems development by Feigenbaum, and described as the "Knowledge Acquisition Bottleneck"

Before we discuss the acquisition of knowledge, we should ask ourselves the question, "what is knowledge". It is much easier to state examples of what knowledge is, rather than a global (and probably circular) definition. Knowledge includes:

§ Declarative facts. "The moon circles the earth"

¨ Statements of propensity. "It is more likely to rain during winter"

© Methods. "To bake a cake, you need flour, sugar, eggs, ....."

ª Heuristics. "It's usually faster to take a subway than a bus"

§ Stereotypes. "Cars produce pollution."

¨ Pictures, Graphs

© Explanations/Justifications of other knowledge

¨ etc., etc., etc.

Expert Systems for real-world tasks may have to make use of many different types of knowledge. For example, declarative facts may be used as rules to state the linkage between groups of symptoms and a diagnosis. Or more likely these would be statements of propensity. Methods are important in an Expert System that performs some active, rather than passive, task. E.g. an Expert System that designs elevators probably needs to have many different methods implemented. Stereotypes of various typical problems may be stored for use in analogies. E.g. for an Expert System to aid students choose courses, a number of typical student courses could be chosen from, and then modified to suit a new student.

Knowledge may also be explicit or implicit. People, especially experts are often unaware of the knowledge they use. Think about your own native language. Many people are fluent in a language, and use correct grammar, but are unable to express the grammar rules they use. This knowledge is unavailable to consciousness, which has serious consequences for knowledge acquisition (more on this later). Other knowledge is explicit. Most of you can probably immediately verbalise which room this lecture is in, and at what time.

We need to obtain a large amount of information to build an Expert system. This knowledge may also appear in a number of forms and be obtained from a number of sources. The most common method of obtaining the knowledge necessary for an Expert System is by interviewing an Expert in the field concerned. E.g. we might interview a medical doctor to obtain knowledge for a medical Expert System.

Knowledge once obtained can be coded in some knowledge representation format, and executed as a program (the finished Expert System). We are getting ahead of ourselves here, and we need to ask the question:

Expert Systems, by definition, do not deal with everyday, common sense, knowledge that everyone knows. They deal with complex domains such as designing statistical experiments, searching for oil or other minerals, predicting environmental consequences of development, and many other domains.

Someone able to implement such a system on a computer requires a large amount of computing knowledge, depending on the application, including such areas as:

§ Knowledge Representation

¨ Human-Computer Interaction

© Natural Language Processing

ª Good Old Programming

§ Logic and Reasoning

¨ Uncertain Reasoning Mechanisms

© etc. etc. etc.

But it also requires knowledge of the problem domain, which may be diseases affecting soybean crops. The chance of finding a single person knowledgeable in all these areas is near zero.

Because of this we often find a situation where a Knowledge Engineer, trained in Expert Systems development repeatedly interviews an Expert in the field. In effect, two individuals, both highly trained but unsure of each other's field must co-operate over a long period of time. In the following section we will look at the roles and characteristics of each.

Finding a suitable, knowledgeable, and co-operative Expert is one of the harder parts of beginning development of an Expert System. Some common characteristics of Experts include:

§ An implicit rather than explicit body of knowledge

Studies have shown that it is mainly beginners at a task who are able to freely verbalise facts, relationships, and other knowledge. Returning to the native language example, people starting to learn a language often spend much time learning grammar rules. After a few years of using the language, the grammar rules themselves tend to recede into unconsciousness.

Experts in many fields are the same. This leads to the unfortunate circumstance where precisely the people who have the knowledge we want are those who are least able to tell us.

¨ Experts see past surface details and recognise underlying relationships.

As an example of this, when presented with chess games in progress, master chess players were able to memorise the board positions far better than novices. When both groups were presented with random positionings of pieces on the board, chess masters were no better than the novices.

© Experts are able to explain their knowledge

Explanation is one of the major requirements of an Expert. For example, a doctor or lawyer is expected to not only give advice on either medical or legal matters, but also to explain and justify that advice. We wouldn't continue visiting a doctor who never explained his/her decisions.

ª Experts are effective and efficient in their tasks

An Expert usually is able to apply this body of knowledge and is able to perform tasks that other people cannot. Also, there is often the matter of efficiency. An Expert such as a motor mechanic is expected to solve problems (repair cars) rapidly.

There are many other characteristics of Experts, but most of the above are a selection that are especially relevant to computerised Expert Systems.

The major problems with arranging for an Expert to co-operate in Expert Systems development include:

§ Time. Experts are usually busy people.

If an Expert has a skill in demand (and we are of course more likely to be designing Expert Systems simulating skills that are in demand rather than not in demand) then s/he is likely to be busy.

¨ Scarcity. Experts are hard to find.

Especially good Experts.

The title 'Knowledge Engineer' is the person who is, in general, speaking to the Expert, and asking for the knowledge for the Expert System. On the surface this may appear to be a simple task. But there are many pitfalls that an unprepared Knowledge Engineer may fall into.

Some of the requirements for a good Knowledge Engineer (and listing some of the pitfalls at the same time) are:

§ Intelligence.

A Knowledge Engineer will not spend an entire career on one Expert System with one application domain. After one project eliciting knowledge on Microbiology, the next project may be Power Station maintenance. A Knowledge Engineer must be able to adapt to a new problem fast, or precious (and expensive) time may be wasted.

¨ Tact.

Experts are human beings, and like many human beings, sometimes rather sensitive. It is not uncommon for an Expert to become dissatisfied with the Expert System project. If this happens, even if the Expert does not refuse to co-operate, knowledge acquisition can slow dramatically as the Expert loses interest. The best example of this is that Experts are (perhaps unsurprisingly) resistant to the idea that they can be replaced by a machine. A Knowledge Engineer should never suggest this as possible.

© Empathy

As we will see, there are many interviewing techniques available for eliciting knowledge in a structured manner. The Knowledge Engineer should be able to spot when an Expert is having difficulty with one method and suggest another method for the transfer of knowledge.

ª Programming Skills

Knowledge Acquisition itself is not always directly linked to implementation. Sometimes the Expert System may be implemented by a group. The Knowledge Engineer who performs the interview might not write any code. However, the knowledge being elicited will at some point end up in a program, and the Knowledge Engineer should be aware of what this entails.

§ Logicality

Knowledge gained by interviewing an Expert may be fuzzy, ill-defined, contradictory, inconsistent, superstitious, or downright incorrect. The Knowledge Engineer must be able to recognise these cases and prompt the Expert for further clarification.

Interviewing itself is an active area for research, and many results from psychology are relevant here. The speed of knowledge transfer differs greatly as different types of interview methodology are applied. An Expert is also likely to have a better opinion of the value of interviews where the Knowledge Engineer is prepared, and guides the interview skilfully.

Unstructured interviews are common early on. There may be little direction or specific purpose for the interview. The Knowledge Engineer (and probably management of the company building the Expert System) usually have little understanding of the domain.

An unstructured interview has little pre-planned purpose, and it is common that knowledge is elicited with little thought as to what purpose (if any) it might serve in the final Expert System. One reason for this lack of purpose is that without some knowledge of the domain, it is difficult to know what is implementable as an Expert System. For example, I could ask the students in this lecture what tasks undertaken at a superconductor research facility could be assisted by Expert Systems. It is not very likely that any (or me for that matter) would know.

Even when a task is pre-decided, a very unstructured approach is common at first. E.g. in one Expert System development, a statistician at a research facility complained that he spent most of his time assisting others with 'trivial' (to him) problems. The initial interviews were concerned with finding out exactly what he was doing, before it was considered what should be implemented.

The interview should be recorded in a way that is comfortable for the Expert. Rough notes may be taken, the interview may be tape recorded, or possibly a video of the interview may result. Some Experts may find a tape recorder or video intimidating, and the Knowledge Engineer should be sensitive to this and use more traditional techniques if necessary.

An easy way to annoy an Expert is to repeat the same question. This is highly likely to happen if only some information is recorded at initial (or any) interviews. For example, some Knowledge Engineers believe that declarative knowledge is the basis of an Expert System. It is vitally important to listen to the way the Expert uses knowledge. This may contain heuristics that lead to an efficient implementation. In extreme cases, declarative knowledge by itself may be inapplicable. E.g. a medical doctor may run several tests in a row to determine a diagnosis for a patient. There may be an implicit assumption that the previous tests have not returned a diagnosis. The Expert his/herself may not be conscious of this assumption, and no amount of questioning will reveal it. The requirement for this knowledge may only become apparant much longer.

Unstructured interviews can result in some knowledge being acquired, but for efficient acquisition of knowledge, especially knowledge not available to consciousness, other methods can be used.

Structured interviews have a specific methodology, and are used to elicit particular types of knowledge. Some examples of structured interview methodologies are:

§ Talk-through.

The Expert performs a task (e.g. diagnosing a patient) while verbalising everything that passes through his/her mind). This helps if the Expert has difficulty remembering or expressing small details of the task which may appear minor (but be of major importance for a computer implementation). If the Expert is actually performing the task (as opposed to running a stereotyped task through in his/her head) then the Knowledge Engineer can ask about any details performed but left unexplained.

¨ Dividing the domain.

Here, there are some objects to be classified or in some way treated in groups. E.g. credit applications to be accepted, rejected, or referred. The Expert may divide them into two or more groups (e.g. those from people who have banked at the issuing bank for more than one year and applications from non-account holders). The resulting groups may be subdivided again. In simple applications, solely this division may be enough to identify which applications should be accepted/rejected/referred.

© Card sort

Card sort is where examples of objects (or concepts) are written on cards, and the Expert sorts them into piles on any criteria s/he likes. This is a variation of Dividing the Domain, but has the additional prompting of examples written on the cards. E.g. many people given a set of playing cards would sort them into four piles, one for each suit. This illustrates something basic about the domain. In more complex cases, the Expert may be unaware as to what relationships s/he is illustrating.

ª 20 questions

20 questions is particularly useful in finding whether the Knowledge Engineer's understanding of the domain is complete. The Knowledge Engineer (using knowledge elicited during previous sessions) creates his/her own example from the domain. E.g. for an Expert System to design elevators, the Knowledge engineer may design a particular elevator. The Expert then asks questions about this imaginary example. E.g. 'what is the maximum number of people that can be carried?' 'How many floors does it call at?' 'Is it an express elevator?' And so on. The Expert often asks questions that identify a hole in the model of the domain so far. E.g., 'Describe the emergency button'. The Knowledge Engineer may be unaware of the presence of an 'emergency button' and reply '????'. These 'holes' can be noted down for further discussion at a later time.

§ Interesting cases

Often it is difficult for an Expert to remember general knowledge about his/her area of expertise. Interesting cases are easier to remember, and more details will be forthcoming. E.g. ask a motor mechanic about the Ford Fiesta that s/he repaired last week and few details are likely to be forthcoming. Ask the same mechanic about the time s/he repaired Lord Buck's Rolls Royce, and a far more detailed account is likely. Asking the Expert to talk about cases that s/he found memorable can also be a pleasant experience for the Expert.

¨ Examples

Sometimes it is easier to not transfer any general knowledge at all. E.g. in the PUFF pulmonary function Expert System the Expert gave the Knowledge Engineers typical and representative samples of pulmonary function disorders. The Knowledge Engineers learnt the general information for the Expert System rules from these examples. Despite reinventing the wheel in this way, overall the task was felt to be easier than asking the Expert directly, which says a lot for the difficulty of interviewing Experts.

Knowledge gained during interviews (even structured ones) can be large and ill-defined. E.g. a tape recording of a conversation may include several descriptions of a single concept that may be partially contradictory. The middle stages of an Expert Systems project may involve formalising knowledge gained, and recorded on paper, into different forms on different pieces of paper.

A transcript of a conversation must at some time be summarised into a form that can be implemented as rules, graphs, or some other form. Converting directly from transcripts to rules is possible only in very simple cases. In more complex domains, intermediate forms will be necessary before an implementation is attempted.

One method of writing down organised knowledge, especially that resulting from a structured interview is a decision tree. To read the 'tree', start at the root and work downwards towards the 'leaves', which store some sort of result or classification. For example, the conversation:

As well as a decision tree, the same knowledge can be represented as a graph.

Any analyses of knowledge should be cross-referenced to previous analyses (or interview transcripts) for easy reference. As is common in computer programming, initial attempts at an implementation may be misguided. To put it simply, they might not work. Any attempt at restructuring an implementation will be greatly aided if the relevant parts of the original interviews can be found immediately.

Expert systems of any worth are usually large. The knowledge acquisition process itself can a very long time, and result in piles of transcripts. If these transcripts are not highly organised, then it will become more and more difficult to find the necessary knowledge.

There are many knowledge acquisition tools available that run on computers. These take many different forms depending on what kind of knowledge they are designed to help elicit. Some are quite specific, and are only useful for one particular task. E.g. Marcus's SALT system helps design Expert Systems for building elevators.

One "general purpose" automated knowledge acquisition method is to use machine learning programs that can induce rules for an Expert System from sets of examples. This is similar to the development of the PUFF system, except that a computer learns the rules, rather than human knowledge engineers.

For example, for an Expert System to learn rules to recognise various fruits and vegetables (and possibly to drive a robot arm that picks them from a conveyor belt and puts them in correct boxes), examples might look like:

By supplying an adequate number of such examples, a computer can build a rule base that correctly classifies new, unseen, examples, and can function as an Expert System (with a suitable engine attached).

Generally the generation of Expert Systems from examples has its own problems. An initial set of examples chosen by an Expert are often incomplete. What generally happens is that a set of examples is chosen, rules are learnt, and then the Expert checks the resulting Expert System. And, finds it completely inadequate. Additional examples are added, and new, hopefully better rules are generated. This continues until the Expert is satisfied that the rules are correct.

The number of training instances needed for a computer to 'learn' an Expert System rule base is very high, and the overall volume of input required much larger than the final rules. It is still easier to generate the examples as they are logically less complex.

In an attempt to reduce the amount of input necessary to automatically acquire knowledge for an Expert System, I invented KABCO, Knowledge Acquisition by Being Corrected.

KABCO learns how to make rules indirectly. First it learns how to make examples. Once it has learnt this, a very large number of examples can be generated and fed to a machine learning program as before.

KABCO learns by starting with almost no knowledge of the domain, and trying to generate examples. At first, since KABCO is ignorant, these examples are very bad. E.g.

is a red, long, furry covered turnip that's 32 kilometres large. Prompted by these errors, the Expert enters 'corrections'. E.g.

if class = turnip then shape = round and size = 13 .. 21cm.

KABCO learns from these examples and never makes the same mistake again. This approach combines the logical simplicity of examples with a much smaller volume of input.

Both machine learning and KABCO only solve part of the Knowledge Acquisition bottleneck problem, how to tell various vegetables apart. Both rely on a pre-existing list of 'attributes' (things used for describing examples) and 'values' (that they may take). One method that tries to elicit these attributes and values is an automatic repertory grid elicitor.

Repertory grid elicitors ask the Expert for 'objects' (concepts, things to be reasoned about, whatever) and for bipolar constructs (a form of attributes) that can be used to describe them. A bipolar construct is something like 'size = small .. medium .. large', an attribute with an ordered list of values. Objects (e.g. animals) are given values for each attribute known (e.g. an elephant is large, a mouse is small).

Using only bipolar constructs, and limiting all such constructs to have the same number of values (e.g. here, 3), allows easy comparison of the objects. (e.g. mice and rats tend to have more similar values than mice and whales). This allows the program to build hierarchies automatically. These can be shown to the Expert which gives him/her feedback about the descriptive power of the bipolar constructs elicited so far.

Both KABCO and a sample repertory grid elicitor are available from me on request if you wish to play around with them.

Knowledge acquisition is a highly complex task. Methods used during interviews will be different in different domains. Even within the same task, different methods may be more or less appropriate depending on the personalities of the Expert and Knowledge Engineer. Knowledge itself comes in many forms, and needs to be handled in different ways depending on its form.

Some books you may wish to look at for further information include "Knowledge Acquisition for Expert Systems" by Anna Hart. There are several copies of this in the library.

Experts are those who work in a knowledge-rich domain, able to apply that knowledge to be effective and efficient at some task. Experts are used to explaining and justifying their decisions. However, they are often not familiar with computers and computing technology. Also, much of the knowledge they use in their problem-solving may be unavailable to consciousness.

Knowledge may be elicited by interviewing the Experts. This interviewing is complex and should be carefully recorded. Knowledge that initially appears unimportant may turn out to be vitally important in later stages of Expert System Development.

Interviews can be divided into unstructured and structured interviews. Unstructured interviews are common early on, while structured interviews aid efficient knowledge acquisition when a specific body of necessary knowledge is to be elicited.

Knowledge Engineers need to be flexible, intelligent, and knowledgeable about many domains. They also need to adapt rapidly to the terminology and content of an unfamiliar domain.

Once knowledge has been elicited, it must be analysed and summarised. This not only prepares it for implementation as an Expert System, but can reveal inadequacies in the Knowledge acquired.

A variety of computerised Knowledge Acquisition aids have been designed that can reduce the effort required compared to the fully manual approach.

Most of the material found in this summary is expaned upon in the book "Knowledge Acquisition for Expert Systems" by Anna Hart, which can be found in the library. My paper on KABCO is available on request. There are general books on Machine Learning in the library, though the classic in the field remains "Machine Learning, An Artificial Intelligence Approach" edited by Michalski, Carbonell, and Mitchell.