Course content

Elements of Visual Perception, Image Sampling and Quantization, related Mathematical tools applied to Image Processing and Analysis (array, matrix, linear and non-linear operations, arithmetic and geometric operations, morphology, spatial and temporal operations, frequency analysis, linear algebra, probabilistic methods, image transformations and geometric operations) Image Formation: Camera Models, Calibration, Single view geometry, Multiple view geometry, Epipolar geometry, Feature extraction, Bundle adjustment Position and Orientation: Feature based alignment; Pose estimation; Time varying pose and trajectories, Structure from motion, dense Motion Estimation, Visual Odometry (Semi-direct VO, direct sparse odometry) Localization and Mapping: Initialization, Tracking, Mapping, geometric SLAM formulations (indirect vs. direct error formulation, geometry parameterization, sparse vs. dense model, optimization approach), Relocalization and map Optimization, Examples: Indirect (Feature based) methods (MonoSLAM, PTAM, ORB-SLAM), Direct methods (DTAM, LSD-SLAM), Sensor combinations (IMU, mono vs. Stereo, RGB-Depth), Analysis and parameter studies Recognition and Interpretation: Object detection, Instance recognition, Category recognition, Context and Scene understanding.

Special note on Artificial Intelligence (AI): AI is integrated into the course through the use of learning-based methods for object detection, classification, and scene analysis. Students will work with modern AI tools, including deep neural networks used in visual perception and robotics, and discuss how AI-based methods can be combined with geometric and photogrammetric techniques in VO and SLAM. Emphasis is placed on developing a critical understanding of when AI methods are appropriate, what limitations they have, and how they affect the robustness and explainability of robot vision systems.

Learning outcome

Knowledge: Knowledge about core applications in Robotic Vision. Knowledge about fundamental (physical) concepts about visual perception. Knowledge about image formation, image representation and camera models. Knowledge about image sampling, quantization and processing. Knowledge about structure from motion concepts for pose, tracking, motion estimation as well as visual odometry (VO) simultaneous localization an mapping (SLAM) strategies exploring popular methods. Basic knowledge about feature extraction, object recognition, context awareness/semantics and scene understanding. Insight into AI-based methods for visual perception, image analysis, object detection, and scene understanding, as well as how such techniques can be combined with classical geometric approaches in robot vision.

Skills: Be able to choose imaging systems with respect to specific applications. Calibrate the imaging systems. Modify different imaging setups with respect to environmental conditions. Manipulate and implement pose, tracking and motion estimation techniques. Implement, tune and evaluate SLAM algorithms. Implement object recognition and classification methods. At the end of the semester a successful student should have skills in processing and analysis of digital images, videos and video streams and be able to design simple robot vision and machine vision systems. Be able to apply and evaluate AI-based algorithms for detection, classification, tracking and scene understanding, and assess their strengths and weaknesses compared to traditional methods. Be able to use AI tools in a professionally responsible, critical, and transparent manner.

General competence: Be able to apply the fundamental imaging and perception principles. Consciousness about the role of visual sensing in robotic applications. Be able to analyze strength and weaknesses of different vision based approaches. Develop awareness of ethical challenges, data sensitivity, and academic integrity related to the use of AI tools in robot vision.

Learning methods and activities

The course is given as a mixture of lectures, and assignments. 75% of the assignments must be approved to enter the final exam.

AI tools may be used in coursework and projects, but their use must be clearly documented and justified. The assignments are structured so that students must reflect on their choice of methods, including when and how AI-based approaches are appropriate, and what limitations they have.

Compulsory assignments

• Assignments

Recommended previous knowledge

TTK4105 - Control Systems, TTK4115 - Linear Systems Theory, TTT4275 - Estimering, deteksjon og klassifisering, TDT4195 - Grunnleggende visuell databehandling, or a comparable background.

Required previous knowledge

At least one of the following subjects (or equivalent from other universities):

TTT4275 Estimation, Detection and Classification,

TMA4268 Statistical Learning,

TMA4267 Linear Statistical Models, or

TMA4245/TMA4240 - Statistics.

Course materials

Information on this is given at the start of the semester.