Intelligent Systems for Robotics · Week 1
isfr.robot.Reacting maps what you sense onto what you do next; it needs to know nothing. Reasoning is what a robot does when the answer is not in front of it, when the order has to be worked out, when it must conclude more than it was told, and when what it expected has stopped being true. All four are handled above a line. The skills beneath it are learned or provided, everything above it is symbolic, and a skill declaring what it needs and what it changes is what joins them.
That is the whole course. Every week from here expands one part of it.
The assessed work is integration, meaning parts that were built separately made into one system, and an account of why it is put together the way it is. The weekly micro-projects are practice and carry no marks.
The weekly work is independent by design. Every micro-project runs from the inputs
it ships with, missing week 3 does not strand you in week 6, and a reference
artifact is published in data/reference/ after each deadline.
The robot is TIAGo: a differential-drive base, a lifting torso, a seven-joint arm, a Robotiq parallel gripper, and a camera, lidar and sonar. It runs in Webots R2025a on your own machine, and it is the same robot for all ten weeks.
Simulation is not a concession. A reset costs nothing, every student has a robot rather than a place in a queue for a shared one, and week 3 trains policies over more episodes than any physical lab could run.
The live example drives the base, returns a camera frame, moves the arm to a
commanded configuration, and closes the gripper. Six of its lines drive the base and
read the camera, and what matters is what is missing from them: no import of the
Webots controller module, no world file, no simulator handle.
The surface the platform gives you is small: drive(),
sleep(), pose(), camera(),
scan(), sonar(), set_joints(), and the
gripper verbs. It is instructor-provided, and you never talk to Webots directly. The
boundary sits there so that everything above it, meaning every layer this course is
about, is written in Python you can read, against an interface that is the same in
week 1 and in week 10.
The task for the term: somebody asks for a drink; the robot finds it, standing on a table or a worktop somewhere in a furnished apartment, and brings it back to whoever asked. One apartment, one person, one can.
The robot in the demo drives, sees, reaches and grips, and it cannot do this. Five questions stand in the way, and four layers answer them:
Everything up to here has been concrete. Everything after it is machinery, and each piece of machinery arrives as the answer to one of those questions, in that order.
How does it get across the apartment? Skills answer that. A skill is an option: an initiation set saying where it may start, a policy saying what it does while it runs, and a termination condition saying when it stops and whether it succeeded. To that, this course adds declared preconditions and effects. Week 2 owns the formalism. It is defined here only because "skill" has to mean something specific before the three layers above it can be described at all.
Some of those skills are learned and some are given. Navigation, base placement and collision-free reaching are learned. Inverse kinematics, grasping and detection are provided. The criterion is to learn only what classical methods do badly. A learned setpoint-tracker is strictly worse than the PD controller beneath it.
Which room is the kitchen, and what counts as a drink? The world model answers both. A detection is instantaneous; a task is not. The robot cannot look at the kitchen and the living room at once, and it cannot re-derive the apartment every time it turns around. Something has to hold what was seen after it has left the frame, and hold it in terms the rest of the system can use: rooms, surfaces, objects, and the relations between them, as a knowledge graph over an ontology. Weeks 4 and 5.
The two questions are not the same problem. A detector, pointed at the right shelf, will return a can; no detector returns a kitchen. A kitchen is a region of space that has been named, and the name has to be attached to somewhere the robot can navigate to. Recognising a type and recognising a place are answered by the same layer and by different machinery within it: the ontology says what a drink is, and the topological map says where the kitchen is and how to get there from here. Week 5 builds both.
In what order? Planning answers that, and it is where the seam of the course sits. The preconditions and effects a skill declares are exactly what the planner consumes. The layer below publishes a contract, and the layer above searches over it. Weeks 6 and 7.
What happens when it fails? Execution and monitoring answer that. A skill terminates with a verdict, succeeded or failed, rather than with the bare fact that it stopped, and a failed termination is what triggers replanning. Suppose the route to the kitchen is blocked. Whether the right response is to drive around the obstacle or to declare that edge impassable is genuinely unsettled in this course, and week 8 takes the question up.
Drawn as a stack, those four layers sit above isfr.robot: skills, then
the world model, then planning, then execution and monitoring, with failure running
back from the top into a replan.
The stack also has a name. It is the three-layer architecture, and the position it argues against is subsumption, which has no world model and no plan, and derives behaviour from layered reactive control. Subsumption was right about the problems it took on. It is the reason the world model in this course holds no metric map. The map is authored. A robot could also learn it by exploring, but that is outside the scope of this course. Every place in the map is one the robot can find again with its own sensors, a doorway as a gap in the lidar scan or an object the detector names. Nothing in it has a position in a world frame. The hybrid won for tasks with a symbolic goal, because "fetch a drink" is a sentence and something has to hold it.
Two ideas sit alongside it. The first is symbol grounding. The plan says
drink and the camera returns pixels, and connecting the two is Harnad's
problem. This course does the engineering version of it rather than the philosophy,
in week 5. The second is situation awareness. Endsley's three levels, perception of
elements, comprehension and projection, overlay the course's layers readably, with
detections at L1, the knowledge graph at L2, and planning and monitoring at L3. The
overlay is a lens, and the levels are not components. Endsley's model describes what
an operator knows, and it makes no claim about how a system should be built. If you
read the paper, you will see how carefully it is scoped.
Perception is tooling in this course, and it is not taught. Off-the-shelf YOLO gives the robot something to ground symbols in. You will use object detection without studying it. Grounding what it detects is a course topic. That distinction is the whole of the position, and it is not a dismissal of vision.
The same picture, with module names on it. isfr.robot is drive, sense,
arm and gripper. isfr.skills is the option contract and the skills that
implement it. The knowledge graph is the world model. The planner works over skills
and meta-actions. The executive runs the plan and notices when it stops being true.
Underneath, isfr_bridge is a small C++ controller running inside Webots.
It publishes raw sensor readings and accepts motor setpoints, and it does nothing
else. Differential drive, the gripper verbs, joint-limit clamping and wheel odometry
all live in Python, where you can read them. Your code talks to the bridge over a
socket and never imports controller, so it runs on a machine with no
Webots install and no WEBOTS_HOME. You will notice the benefit in the
lab, where the shell running your Python needs nothing from the simulator.
Time is simulated rather than wall-clock. robot.sleep(1.0) waits one
simulated second, so running the world at half speed still drives the robot
the same distance, and an opt-in synchronous mode makes a command sequence
reproducible from run to run. This matters because results are then comparable. Week
3's training and week 8's evaluation both depend on it.
Three contracts are fixed for the whole term: the Skill interface, the
Atom shape for preconditions and effects, and the ontology's base
namespace. Everything else is yours, including file formats, how a policy is loaded,
and how a plan step finds its skill. That is deliberate. The micro-projects are
independent by design, so you will arrive at the project with eight components that
have never been connected and no agreed interfaces between them. Reconciling those
choices is the project. Handing you a pre-built seam would teach everything
except the thing this course is about. It is said now, in week 1, so the term is
legible in advance.
The week list at the start of the lecture was a timetable. Read against the diagram it is an argument. Weeks 2 and 3 build skills and learn them, weeks 4 and 5 build the world model, weeks 6 and 7 plan over both, week 8 executes and monitors, and weeks 9 and 10 are your project. It runs bottom-up because each layer needs the one beneath it to mean anything. A planner over skills needs skills that declare something, and a world model earns its keep only once there is something to do with what it holds.
This week has three tasks. Install the environment, run the example, and drive a closed square. Then look at where the robot actually is, and ask whether you believe the pose it reports.
A runnable example accompanies this lecture in
examples/01-robot-devices/. Run it from the repository root:
PYTHONPATH=. python3 examples/01-robot-devices/main.py
Run it exactly that way. python3 examples/01-robot-devices/main.py puts the
script's own directory on sys.path rather than the current one, so the
bare form fails with ModuleNotFoundError: No module named 'isfr' no
matter which directory you start from.
This week's exercise is exercises/01-introduction/. Its
README.md states the task and ends in a Self Check list you can verify
unaided. A solution/ sits alongside the starter code. Do not open it
until you are done.
The micro-project runs from the inputs it provides. It does not need any earlier week's output.
Further, optional, and not examined: background reading.
Intelligent Systems for Robotics · course materials