Atomistic questions

1. Does the ILO describe how subject knowledge, practical skills or general competencies will

be exhibited (manifest rather than latent) by an individual upon successful completion of the

educational activity?

2. Is the level of the ILO appropriate to the exit point of the educational activity to which it

refers?

3. Can the ILO confidently be placed within a single learning domain (cognitive, psychomotor or

affective)?

Holistic questions

1. Do the ILOs collectively and concisely describe the technical and non-technical graduate

outcomes (refraining from listing implied foundation and intermediary outcomes)?

2. Are the general competencies transferable to other disciplines or professional contexts?

Background and motivation

This addendum to the key quality assessment functions gives a short introduction to the three

learning domains that learning outcomes are related to. It further exemplifies how the questions

may be used to detect quality issues in ILOs.

The three learning domains

Cognitive: In the original 1956 version of the taxonomy, commonly referred to Bloom’s taxonomy,

the cognitive domain is broken into six levels of educational objectives: knowledge, comprehension,

application, analysis, synthesis and evaluation. The taxonomy was revised by Anderson and

Krathwohl in 2001, with notable changes to levels five and six (synthesis and evaluation). Anderson

and Krathwohl define the revised level five as Evaluate (making judgements based on criteria and

standards) and level six as Create (putting elements together to form a novel, coherent whole or

make an original product).

Psychomotor: There are three commonly referenced taxonomies within the psychomotor domain:

Dave (1970), Harrow (1972) and Simpson (1972). These are also recognised in Krathwohl’s overview

of the 2001 revision of Bloom’s taxonomy. The Simpson and Harrow psychomotor taxonomies are

especially useful for describing the development of the relationship between cognition and physical

movement from birth through to adulthood, or for developing skills within the performing arts and

sports. The psychomotor taxonomy by Dave is perhaps more transferable across disciplines as there

is less emphasis on movement, and rather describes progressive degrees of competence in

performative actions.

Affective: The Affective Domain was classified and described by Krathwohl, Bloom and Masia (1964)

in Handbook II of the Taxonomy of Educational Objectives and presents a taxonomy for developing

attitudes, character and conscience – also expressed in various literature as interests, appreciations,

values or emotional sets. Central to the affective domain is the concept of internalisation: whereby

interacting with external stimuli progressively shapes the individual’s perspective of the world.Example quality assessment

Example programme: Computer Science - master's program (Master of Technology)

A student who has completed the program is expected to have achieved the following learning

outcomes, defined in terms of knowledge, skills and general competence:

Knowledge: Broad mathematical-natural science, technological and data engineering basic

knowledge as a basis for understanding methods, applications, professional renewal and

restructuring.

• This ILO describes possessing a broad, basic knowledge as a basis for understanding. It does

not describe how that knowledge will be exhibited. Possessing knowledge is a latent term

that cannot readily be observed or measured.

• The areas of knowledge described as mathematical-natural science and technological are

somewhat vague and perhaps need to be described within their own ILO.

• Understanding, as a graduate outcome, is somewhat below the study programme level of a

master’s degree. At level 7, you would expect graduates to be capable of demonstrating a

higher-level knowledge such as analysis and evaluation.

• Professional renewal is included in this ILO which is perhaps out of place. Understanding

professional renewal, as suggested in this ILO, might be an awareness of continuous

professional development (CPD) as a concept; however, it is anticipated that a preparedness

to engage in CPD is the more likely intended outcome. This would be better mapped to the

affective domain of learning and placed under general competence rather than subject

knowledge.

Skills: Define, model and analyse complex engineering problems within computer technology,

including choosing relevant models and methods, and carry out analyses, build and evaluate

solutions independently and critically, in relation to both technical and non-technical factors.

• Defining a problem would be an expression of knowledge rather than a skill, as would

choosing a relevant model and method.

• Carrying out an analysis (analysing something) would generally be considered to be within

the cognitive domain; however, this ILO description suggests that computational modelling

will be used which might require certain technical skills. The necessary technical skills should

be described separately from a general description of analysis.

• Building a solution to a complex engineering problem would assumably require a thorough

understanding of the problem, a validated methodology necessary for solving the problem

and the technical skills to build the architecture of the solution. The knowledge components

and skill components need to be separated out here.

• Evaluation of a solution can be done without having built it. It can therefore be considered as

a standalone outcome. Evaluation, in the context described here, should likely be considered

under the cognitive domain and be described as an expression of knowledge. Critique forms

part of an evaluation and does not need to be additionally listed. As the learning outcomes

here describe the graduate as an individual, the inclusion of the word “independently” is not

necessary as it is implicit throughout.• Evaluation would usually consider the whole. If there are specific “technical and non-

technical factors” that are important to include, then these should be clearly described.

• The levels of analysis and evaluation seem appropriate for a master’s degree graduate.

Building a solution to a complex engineering problem might be venturing further into the

generation of new knowledge. It perhaps needs to be considered whether this involves

building an implementation of a known solution or actually solving a problem through a

novel invention, and thereby whether it is an appropriate expectation for this level of study.

General competence: Understand the role of engineering and computer technology in a holistic

societal perspective, have insight into ethical requirements and considerations for sustainable

development, and be able to analyse ethical issues related to engineering work.

• “The role of engineering and computer technology” seems to be subject knowledge rather

than general competence, and therefore misplaced. Of course, it is useful to consider these

things from a social, as well as a technical, perspective, but this is still about understanding

the subject area rather than describing a transferrable competence.

• Ethics and sustainable development could certainly be considered transferable, but what

exactly are the competencies expected here? “Having insight” is a latent term that cannot

readily be observed or measured.

• Ethics and sustainable development should be considered independently and described as

two separate ILOs.

• Ethical requirements might also be separated from ethical issues. It might be fundamentally

important, and a basic knowledge component, to recognise applicable ethical regulations.

Considering ethical issues in terms of moral and social values might instead be considered

within the affective domain and more of a general competence.

• Sustainable development needs elaborating on, and with a clearer outcome described.

“Considerations for sustainable development” is another latent term that cannot readily be

observed or measured.