Further reading
Intelligent Systems for Robotics · optional
Home · Syllabus
None of this is taught and none of it is examined. It is here
because the lectures name ideas that have a literature behind them, and some of
you will want to follow them further than a slide goes.
Cognitive architectures
Week 1 derives a three-layer architecture from the fetch task and then names it.
The wider question — what a complete architecture for an intelligent agent should
contain — has its own field, and this course deliberately does not enter it.
-
Kotseruba, I., & Tsotsos, J. K. (2020). 40 years of cognitive architectures:
core cognitive abilities and practical applications. Artificial Intelligence
Review, 53(1), 17–94.
doi:10.1007/s10462-018-9646-y
— the survey to start with: eighty-four architectures, compared on what they
actually do rather than on what they claim.
-
Langley, P., Laird, J. E., & Rogers, S. (2009). Cognitive architectures:
Research issues and challenges. Cognitive Systems Research, 10(2),
141–160.
doi:10.1016/j.cogsys.2006.07.004
— the open problems, from people who built two of the architectures.
-
Newell, A. (1990). Unified Theories of Cognition. Harvard University
Press. ISBN 0-674-92099-6 — the source of the argument that one architecture
should account for all of cognition. Long, and worth it only if the question
already interests you.
-
Vernon, D. (2014). Artificial Cognitive Systems: A Primer. MIT Press.
ISBN 978-0-262-02838-7 — the textbook route in, if the survey and the
primary literature are too steep a first step.
Situation awareness
Week 1 overlays Endsley's three levels on the course's own layers. That mapping is
a lens, not an identity: her model describes an operator's state of knowledge, not
a specification for software components. The original paper is worth reading for
how carefully it is scoped.
-
Endsley, M. R. (1995). Toward a Theory of Situation Awareness in Dynamic Systems.
Human Factors: The Journal of the Human Factors and Ergonomics Society,
37(1), 32–64.
doi:10.1518/001872095779049543
Reinforcement learning
Week 2 teaches enough reinforcement learning to train the skills this course needs,
and the week 2 notes page carries the reading that is taught. The two entries below
are not taught. One is the survey of what the field has done on real robots, and the
other is the classical controller the lecture measures a learned policy against.
-
Kober, J., Bagnell, J. A., & Peters, J. (2013). Reinforcement learning in
robotics: A survey. The International Journal of Robotics Research,
32(11), 1238–1274.
doi:10.1177/0278364913495721
— what the field had actually got working on physical robots, and where the
difficulties lie. Read it for the gap between a benchmark task and a robot.
-
Åström, K. J., & Furuta, K. (2000). Swinging up a pendulum by energy
control. Automatica, 36(2), 287–295.
doi:10.1016/S0005-1098(99)00140-5
— the energy-shaping controller the week 2 lecture puts beside a trained
policy. Worth reading to see how much of the pendulum's model the classical method
needs.
Knowledge representation
Week 4 teaches RDF, RDFS, description logics, OWL 2 and SPARQL far enough to model the
apartment, and the week 4 notes page carries the reading that is taught. The entries
below are the standards the lecture cites, the method behind competency questions, and
a robot system built on an ontology.
-
Poole, D. L., & Mackworth, A. K. (2023). Artificial Intelligence: Foundations
of Computational Agents (3rd ed.). Cambridge University Press. Section 15.9,
Complete Knowledge
Assumption, with 15.9.1 on its proof procedures. The closed-world assumption and
negation as failure, which is what
FILTER NOT EXISTS applies.
-
W3C. (2012). OWL 2 Web Ontology Language Primer (Second Edition). W3C
Recommendation, 11 December 2012. Eds. P. Hitzler, M. Krötzsch, B. Parsia,
P. F. Patel-Schneider and S. Rudolph.
w3.org/TR/2012/REC-owl2-primer-20121211.
The language by example, with a section on the profiles.
-
W3C. (2012). OWL 2 Web Ontology Language Profiles (Second Edition). W3C
Recommendation, 11 December 2012. Eds. B. Motik, B. Cuenca Grau, I. Horrocks, Z. Wu,
A. Fokoue and C. Lutz.
w3.org/TR/2012/REC-owl2-profiles-20121211.
Section 2 is OWL 2 EL, 3 is OWL 2 QL, and 4 is OWL 2 RL, with the rules
owlrl implements in section 4.3.
-
W3C. (2014). RDF 1.1 Primer. W3C Working Group Note, 24 June 2014. Eds.
G. Schreiber and Y. Raimond.
w3.org/TR/2014/NOTE-rdf11-primer-20140624.
-
W3C. (2013). SPARQL 1.1 Query Language. W3C Recommendation, 21 March 2013.
Eds. S. Harris and A. Seaborne.
w3.org/TR/2013/REC-sparql11-query-20130321.
Section 6, "Including Optional Values", and section 8.1.1, "Testing For the Absence
of a Pattern".
-
Grüninger, M., & Fox, M. S. (1995). Methodology for the design and evaluation of
ontologies. Workshop on Basic Ontological Issues in Knowledge Sharing, IJCAI-95,
Montreal.
PDF from the authors' laboratory.
The paper the ontology engineering literature attributes competency questions to.
-
Tenorth, M., & Beetz, M. (2013). KnowRob: A knowledge processing infrastructure
for cognition-enabled robots. The International Journal of Robotics Research,
32(5), 566–590.
doi:10.1177/0278364913481635.
Not open access. For the current system, Beetz, M., Beßler, D., Haidu, A.,
Pomarlan, M., Bozcuoğlu, A. K., & Bartels, G. (2018). KnowRob 2.0: A 2nd
generation knowledge processing framework for cognition-enabled robotic agents. In
2018 IEEE International Conference on Robotics and Automation (ICRA)
(pp. 512–519).
doi:10.1109/ICRA.2018.8460964.
Maps and composed skills
Week 5 builds the map as a graph of places with approximate relative poses and searches
it by uniform cost. Week 7 composes skills into meta-actions in the way of the STRIPS
macro-operators. The first two entries are the week 5 notes reading. The rest are the
sources behind the lecture's claims, and each has a free copy where one is noted.
-
Poole, D. L., & Mackworth, A. K. (2023). Artificial Intelligence: Foundations
of Computational Agents (3rd ed.). Cambridge University Press.
doi:10.1017/9781009258227.
Section 3.5,
Uninformed Search
Strategies, where 3.5.4 names lowest-cost-first search, also called
uniform-cost search, as the simplest method guaranteed to find a minimum-cost path.
Section 3.6,
Informed (Heuristic)
Search, which defines an admissible heuristic. Section 6.1,
Representing States,
Actions, and Goals, where 6.1.2 describes the STRIPS representation.
-
Warren, W. H., Rothman, D. B., Schnapp, B. H., & Ericson, J. D. (2017).
Wormholes in virtual space: From cognitive maps to cognitive graphs.
Cognition, 166, 152–163.
doi:10.1016/j.cognition.2017.05.020.
Not open access. The
abstract at PubMed reports
walkers in a virtual maze whose shortcuts a labelled-graph model fitted better than
a Euclidean map model.
-
Peer, M., Brunec, I. K., Newcombe, N. S., & Epstein, R. A. (2021). Structuring
knowledge with cognitive maps and cognitive graphs. Trends in Cognitive
Sciences, 25(1), 37–54.
doi:10.1016/j.tics.2020.10.004.
Free at PubMed Central.
A review of the evidence that map-like and graph-like representations both exist and
can operate together or apart.
-
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972). Learning and executing
generalized robot plans. Artificial Intelligence, 3, 251–288.
doi:10.1016/0004-3702(72)90051-3.
Not open access. The origin of macro-operators and of the triangle table.
-
Fikes, R. E., & Nilsson, N. J. (1993). STRIPS, a retrospective.
Artificial Intelligence, 59(1–2), 227–232.
doi:10.1016/0004-3702(93)90190-M.
Free PDF from the author.
The authors' own account of macro-operators and of the execution monitor, which
checks the latest kernel of a triangle table first and replans when none holds.
-
Nilsson, N. J. (1990). Triangle-table trees. Technical material from
research proposals, August 1990.
ai.stanford.edu/~nilsson/tritable.pdf.
Eight pages. Section B describes the triangle table, in which the preconditions that
no earlier action realises form column 0. This is the sequential composition that
week 7 uses to derive a meta-action's preconditions.
-
Kuipers, B. (2000). The Spatial Semantic Hierarchy. Artificial
Intelligence, 119(1–2), 191–233.
doi:10.1016/S0004-3702(00)00017-5.
Published version on the author's site.
Section 4 is the topological level, and section 5.2 is patchwork mapping, where local
frames linked by topological connections are merged into one global frame only when
enough information and computation are available.
Classical planning
Week 6 treats a skill as a STRIPS operator, writes the fetch task in PDDL, and
searches for a plan with a heuristic. The first four entries are the week 6 notes
reading. The rest are the sources behind the lecture's claims and the tools you use
in the exercise. Each has a free copy where one is noted.
-
Poole, D. L., & Mackworth, A. K. (2023). Artificial Intelligence: Foundations
of Computational Agents (3rd ed.). Cambridge University Press.
doi:10.1017/9781009258227.
Section 6.1,
Representing States,
Actions, and Goals, where 6.1.2 describes the STRIPS representation and the
STRIPS assumption. Section 6.2,
Forward Planning,
which treats planning as search in the state space and gives an admissible heuristic
for the delivery robot. Section 5.7,
Complete Knowledge
Assumption, which names the closed-world assumption.
-
Fikes, R. E., & Nilsson, N. J. (1971). STRIPS: A new approach to the application
of theorem proving to problem solving. Artificial Intelligence, 2(3–4),
189–208.
doi:10.1016/0004-3702(71)90010-5.
Free PDF from the author.
Section 3.2 states an operator by its preconditions, a delete list and an add list,
with
push(k, m, n) as the example.
-
Bonet, B., & Geffner, H. (2001). Planning as heuristic search.
Artificial Intelligence, 129(1–2), 5–33.
doi:10.1016/S0004-3702(01)00108-4.
Published version on a course page.
Section 4 defines the additive heuristic, which is not admissible, and the max
heuristic, which is. Section 5.3.3 reports that the max heuristic is almost useless
on Gripper, where the subgoals are mostly independent.
-
Hoffmann, J., & Nebel, B. (2001). The FF planning system: Fast plan generation
through heuristic search. Journal of Artificial Intelligence Research, 14,
253–302.
doi:10.1613/jair.855.
Free at arXiv:1106.0675.
The relaxation ignores the delete lists of all actions. Computing the optimal relaxed
plan is NP-hard, and FF's relaxed plan is not guaranteed optimal, so its heuristic is
not admissible.
-
Fox, M., & Long, D. (2003). PDDL2.1: An extension to PDDL for expressing temporal
planning domains. Journal of Artificial Intelligence Research, 20,
61–124.
doi:10.1613/jair.1129.
Free at arXiv:1106.4561.
Definition 13 in section 7.1 gives the state transition, in which the deletes apply
before the adds. Page 61 credits PDDL to McDermott and the AIPS-98 Planning
Competition Committee.
-
Helmert, M. (2006). The Fast Downward planning system. Journal of Artificial
Intelligence Research, 26, 191–246.
doi:10.1613/jair.1705.
Free at arXiv:1109.6051.
A progression planner based on heuristic search, which translates the propositional
task into a multi-valued one. The paper does not describe the
lama-first
configuration that part 3 of the exercise uses.
-
Ghallab, M., Nau, D., & Traverso, P. (2025). Acting, Planning, and
Learning. Cambridge University Press.
doi:10.1017/9781009579346.
The authors offer a
pre-publication manuscript, free to
view and download for personal use only. Chapter 3, Planning with Deterministic
Models, covers forward state-space search in 3.1 and heuristic functions in 3.2,
including delete-relaxation and relaxed planning graph heuristics.
-
Haslum, P., Lipovetzky, N., Magazzeni, D., & Muise, C. (2019). An
Introduction to the Planning Domain Definition Language. Synthesis Lectures on
Artificial Intelligence and Machine Learning. Springer.
doi:10.1007/978-3-031-01584-7.
The reference for the language. The lectures cite no detail from it.
Meta-actions and macro-operators
Week 7 composes skills into meta-actions, derives their preconditions and effects in
the way of the STRIPS macro-operators, and measures what a planner gains from them. The
week 7 notes reading starts with three works listed under maps above: Fikes, Hart and
Nilsson (1972), Nilsson (1990) and Fikes and Nilsson (1993). Week 7 read the 1972
paper in the
author's copy,
and its derivation rule is in section 3. The entries below are the rest of the notes
reading and the further reading they name. The notes also cite Fikes and Nilsson
(1971), Fox and Long (2003), Helmert (2006) and section 6.1 of Poole and Mackworth,
all listed under classical planning above.
-
Minton, S. (1988). Quantitative results concerning the utility of explanation-based
learning. In Proceedings of the Seventh National Conference on Artificial
Intelligence (AAAI-88), 564–569.
Free at cdn.aaai.org.
Page 564 names the utility problem, the cost of testing whether learned knowledge
applies, which can outweigh what it saves. The paper is about search control rules,
and applying it to macros is the course's step.
-
Botea, A., Enzenberger, M., Müller, M., & Schaeffer, J. (2005). Macro-FF:
Improving AI planning with automatically learned macro-operators. Journal of
Artificial Intelligence Research, 24, 581–621.
doi:10.1613/jair.1696.
Free at arXiv:1109.2154.
Section 2.2.2 is the utility problem for macros, section 3.3 the branching factor a
macro adds, and section 3.4.1 the search depth it removes in exchange.
-
Minton, S. (1988). Learning Search Control Knowledge: An Explanation-Based
Approach. Kluwer Academic Publishers.
doi:10.1007/978-1-4613-1703-6.
The book behind the paper above, and the work Botea and colleagues cite for the
utility problem.
-
Korf, R. E. (1985). Macro-operators: A weak method for learning. Artificial
Intelligence, 26(1), 35–77.
doi:10.1016/0004-3702(85)90012-8.
Not open access. A second account of learning macro-operators.
-
Micheli, A., Bit-Monnot, A., Röger, G., Scala, E., Valentini, A., Framba, L.,
Rovetta, A., Trapasso, A., Bonassi, L., Gerevini, A. E., Iocchi, L., Ingrand, F.,
Köckemann, U., Patrizi, F., Saetti, A., Serina, I., & Stock, S. (2025). Unified
Planning: Modeling, manipulating and solving AI planning problems in Python.
SoftwareX, 29, 102012.
doi:10.1016/j.softx.2024.102012.
The library that writes the domain and calls the engines in weeks 6 and 7.
Intelligent Systems for Robotics · course materials