Verfügbare Projektthemen

Diplomarbeiten, Masterarbeiten, Bachelorarbeiten, Studienarbeiten, Fachstudien

Project Topics

On this page, we provide the current project topics we offer. The project topics can be adapted for Diploma Theses, Master Theses, Bachelor Theses, Research Projects, Student Reports (Fachstudien), and Study Projects. It is also possible to submit your own proposal for a project.

Interested in our project topics or you have your own proposal? Please send your inquiry to

When sending the inquiry, please consider the following:

  • You are a student enrolled in a study program offered by or associated with the Computer Science department.
  • The inquiry includes one or more of the topics below.
  • In the case of your own proposal, describe the topic and its relevance to our research interests.

We receive many inquiries, so we ask for your patience in waiting for an answer. If you do not hear from us within two weeks, please send a reminder to the same email address.

We are looking for a student interested in technology transfer for which funds are available. Please contact Prof. Aiello if you are interested in exploring this opportunity.

Self-adaptive Operating Strategies for Pitting on Gears

Gear failure caused by pitting is one of the leading reasons of downtime of gear units in industrial plants and systems. As soon as a pitting exceeds a size of 4 % in relation to the size of the tooth flank, the gear is considered as failed. An adaptive operating strategy applies a load reduction of a damaged tooth by the means of torque variation to increase the Remaining Useful Life. A usage scenario for such an adaptive operating strategy can be, e.g., the gears of wind power drives to make them last longer and reduce the need for repairs or replacements.
In cooperation with the Institute of Machine Elements, we offer the opportunity to develop a holistic operating strategy to reduce gear wearing. The objective of this work is, therefore, to design and implement a software solution that realizes the adaptive operating strategy by evaluating sensor data and proposing correcting operating modes, e.g., by using machine learning, control theory, or any other means suitable for self-adaptive systems. The resulting system can then be evaluated using real hardware on our test bench.

Keywords: Self-adaptive Systems, Operating Strategies, Gears, Wind Power Drives, Sustainability


[1] D. Weyns, An introduction to self-adaptive systems - A comtemporary software engineering perspective, ser. Wiley - IEEE. Hoboken, NJ: Wiley-Blackwell, Oct. 2020. [Online]. Available:

[2] Y. Gretzinger, “Steigerung der nutzbaren Restlebensdauer von Zahnrädern durch
eine adaptive Betriebsstrategie,” 2022. [Online]. Available:

Supervisors: Robin Pesl, M.Sc. / Lisa Binanzer, M. Sc.


Quantum Computing for Smart Energy Optimizations

The advent of practical Quantum Computing promises to speed up many computation processes, especially in the field of optimization. This means that some large scale simulations and optimizations that were unfeasible with traditional von Neumann architectures become practically available. The proposed research topic focuses on optimisations in the fields of energy distribution under modern models of Smart Grids [2]. The project is open ended. Possible goals include: feasibility of using quantum computing for energy optimization [1], encoding MILP energy optimisations as quantum problems, energy use cases implemented on quantum computers. The interested student can also propose his/her own topic within the sub-field of Quantum Computing for Smart Energy Optimizations.

Keywords: Quantum Computing, Smart Energy Systems, Energy Distribution, MILP


[1] Ajagekar, Akshay, and Fengqi You. “Quantum Computing for Energy Systems Optimization: Challenges and Opportunities.” Energy 179 (2019): 76–89. Crossref. Web.

[2] M. Aiello and G.A. Pagani (2016) How energy distribution will change: an ICT perspective, In Smart Grids from a Global Perspective, J. de Wilde, A. Beaulieu, and J. Scherpen (eds), Springer.

Supervisors: Prof. Dr. Marco Aiello / Daniel Vietz, M. Sc.


Unit Commitment Problem

Power systems operation is based on the matching of generation with demand under physical power flow constraints. Power systems operators need to carefully plan and react to changes in demand while attempting to minimize the costs of operation. The Unit Commitment Problem refers to a family of power systems optimization problems that have to do with such matching of demand with generation. The proposed research is about the exploration of applying quantum computing to address the problem and to compare it with classic computing solutions. In particular, tasks include: (1) exploring the current classical and quantum approaches to solve the Unit Commitment problem; (2) implementing a mathematical model which realizes the Unit Commitment problem using MNLIP with any programming language (preferably Python); (3) extending an existing quantum annealing proposal developed at the department; (4) testing and evaluating on current quantum computers (e.g. via Amazon Braket).

Supervisors: Prof. Dr. Marco Aiello 


Modeling the Behavior of Platoons of Motorcyclists Driving in the Black Forest

In 1992, Nagel and Schreckenberg presented a cellular model of traffic that with just four rules describes quite well highway traffic behavior. In the proposed project, the idea is to research a similar model for motorcycle platooning. Starting from maps of winding roads of the black forest, the goal is to build a cellular automata model of a platoon of motorcycles driving through those roads; i.e., to model the behaviour of a group of motorcyclists that are driving together without passing each other. 

Possible metrics for evaluation are the acceleration/deceleration of vehicles depending on position in the platoon, how fun a road is based on the patter of curves and the size of the platoon and how often it is necessary to brake beacuse of a motorcycle in front, comparison with original car traffic model.


[1] Nagel, K.; Schreckenberg, M. (1992). "A cellular automaton model for freeway traffic" (PDF). Journal de Physique I. 2 (12): 2221. Bibcode:1992JPhy1...2.2221N. doi:10.1051/jp1:1992277. Archived from the original (PDF) on 2014-03-11.

[2] Meng, J. P., Dai, S. Q., Dong, L. Y., & Zhang, J. F. (2007). Cellular automaton model for mixed traffic flow with motorcycles. Physica A: Statistical Mechanics and Its Applications, 380, 470-480.

[3] Lan, L. W., Chiou, Y. C., Lin, Z. S., & Hsu, C. C. (2010). Cellular automaton simulations for mixed traffic with erratic motorcycles’ behaviours. Physica A: Statistical Mechanics and its Applications, 389(10), 2077-2089.


Supervisors: Prof. Dr. Marco Aiello 


The Integration of Electric Vehicles in the Smart Grid

Electric Vehicles (eV) have a battery size that can vary from few KWh to almost 100 KWh. These are frequently connected to the power grid to charge. Several researchers have proposed the idea that these batteries can also be used to store grid electricity and provided it back to grid during peak demand periods. The research proposed here is to assess technical solutions to make the eV an integral part of the power grid, to study individual strategies based on dynamic pricing for incentivising eV integration, and to assess the economic viability of such approaches.


P. P. Malya, L. Fiorini, M. Rouhani, and M. Aiello (2021) Electric Vehicles as Distribution Grid Batteries: A Reality Check, Energy Informatics, Springer, 4(Suppl 2):2.

A. P. Lopes, F. J. Soares and P. M. R. Almeida, "Integration of Electric Vehicles in the Electric Power System," in Proceedings of the IEEE, vol. 99, no. 1, pp. 168-183, Jan. 2011.

Francis Mwasilu, Jackson John Justo, Eun-Kyung Kim, Ton Duc Do, Jin-Woo Jung, “Electric vehicles and smart grid interaction: A review on vehicle to grid and renewable energy sources integration,” in Renewable and Sustainable Energy Reviews, vol. 34, no. 6, pp 501-516, Jun. 2014.

Supervisor: Prof. Dr. Marco Aiello


On the energy accountability of running software

Software running on data centres and personal computers is responsible for a large amount of energy consumption and consequent emissions worldwide. At the same time, it is very difficult to perform an accurate measure of energy consumption for individual software programs and their individual components, this is also due to the current software engineering practices of layering and virtualisation. For this research project you are asked to look into energy models for software, energy accountability for software, comparing the running of the same algorithm on different platforms for measuring the energy consumption, etc.


[1] S. Miller and C. Lerche (2022) Sustainability with Rust. AWS Open Source Blog. Online: 

[2] Green Software Lab (2018) Energy Efficiency in Programming Languages (source code of 10 benchmarks implemented in 28 programming languages). GitHub. Online: 

Supervisor: Prof. Dr. Marco Aiello


Cellular Automata for Modelling Resource Allocation

Cellular automata are simple models that can represent complex even chaotic systems. The goal of this project is to provide simple models for resource allocation that help identifying efficient allocation rules. The project entails both a modelling and simulation component to it.

Supervisor: Prof. Dr. Marco Aiello


A Complex Network Analysis of Models to be Checked

Model Checking is the field concerned with the automatic process of checking whether a given model (or family of models) meets certain specifications. Model checkers work in various ways usually exploring large models explicitly represented as states or symbolically represented. Your task is to evaluate the impact of using complex network analysis techniques to improve and extend model checking approaches.

Supervisor: Prof. Dr. Marco Aiello


AI Planning for Poker Player

The game of poker is interesting in that it combines elements of stochasticity, reasoning, risk awareness, opponent modeling, and decision making with hidden information. The goal of this project is to study components that are necessary to build a successful AI poker player. Most notably:

1) HTN Planning techniques that include risk-awareness

2) Accurate opponent modeling based on game hand history

3) Dynamic evaluation of the current state of the game based on hand history, stack values, and position

4) Adaptation of strategies to type of game played (number of tables, speed, limits, etc.)

The case study will be online No-Limit Hold 'em Multi-Table Tournaments.

Keywords: AI Poker Player, AI Planning, Risk Awareness


[1] Jonathan Rubin, Ian Watson (2011) Computer poker: A review, Artificial Intelligence, 172(5-6): 958-987

[2] Darse Billings, Aaron Davidson, Jonathan Schaeffer, Duane Szafron (2002) The challenge of poker, Artificial Intelligence, 134(1-2):201-240

[3] Dan Harrington, Bill Robertie (2014) Harrington on Modern Tournament Poker: How to Play No-Limit Hold 'em Multi-Table Tournaments, Perfect Paperback 

Supervisors: Prof. Dr. Marco Aiello / Dr. Ilche Georgievski / Ebaa Alnazer, M.Sc.


Topics on Distribution Systems and Sustainability

  • Distribution Systems Modelling and analysis
  • Failures Modelling/Prediction on Power Systems
  • Extreme Climate Events
  • Renewable Sources of Electricity Production
  • Resource Allocation on Distribution Systems

Topic descriptions are available upon request.

Supervisor: Matheus de Souza Sant'Anna Fogliatto, MSc

Topics on Probabilistic Modelling and Energy Systems

  • Application of probabilistic graphical models to Critical Infrastructure
  • Analysis of failure response in energy systems using probabilistic models
  • Resource allocation for smart and/or microgrids using probabilistic models
  • Utilization of Bayesian Networks, Bayesian Inference, and Bayesian Models in engineering complex systems

Topic descriptions are available upon request.

Supervisor: Henrique de Oliveira Caetano, MSc

Topics on Engineering AI Planning Systems

  • Enhancement of the Search Algorithm of the SH Planning System
  • Evaluating the Performance of the SH Planning System Against Benchmarks Planning Problems
  • User Study on the Benefits and Challenges of Engineering AI Planning Systems with the PlanX Toolbox
  • A Multi-Criteria Decision Support System for Selecting AI Planning Software and Models
  • Towards Leveraging Graph Neural Networks for Enhancing HTN Planning
  • LLM-Based Improvements of Domain Models for AI Planning
  • Automating the Generation of AI Planning Problem Instances: Designing a Universal Service
  • Towards Value Cards for AI planning

Topic descriptions are available upon request.

Supervisor: Dr. Ilche Georgievski

Topics on Applications of AI Planning

  • Automated reconfiguration of component-based distributed applications by means of HTN planning

Topic descriptions are available upon request.

Supervisor: Dr. Ilche Georgievski

HTN Planning for Electric Vehicles- Knowledge Acquisition and Modelling

Substituting Electric Vehicles (EVs) for traditional ones keeps increasing due to many factors such as price reduction, and climate and environmental awareness [1]. Due to their importance, a lot of research has been conducted to address planning tasks in the domain of EVs. These planning tasks include route planning and battery charging tasks [2]. In fact, planning the driving task of EVs while considering the consumption (costs) of energy, money, and time, induced by the different actions is a complicated task due to the non-determinism of this domain. In particular, the uncertainty of multiple factors such as weather conditions, road conditions, and obstacles existence makes it hard to deterministically predefine the cost of the actions. This leads to variability of action costs, which constitutes a source of risk and uncertainty in these domains [3]. Moreover, the planning task has to also consider other factors such as the location and the availability of charging stations, the driving speed, the state of charge of the vehicle, the charging speed, route traffic, and many more. Lastly, planning tasks should consider the driver’s risk attitude, which reflects on the choices that the driver makes. For example, planning the task of battery charging has to account for the risk attitude of the driver. Some risk-averse drivers might prefer to charge their EVs to 100% as soon as possible to deal with any unanticipated need for driving, while risk-seeking drivers might prefer to charge exactly the amount needed for planned driving [4].

All these complexities call for the need to automate the planning process, a task that is powerfully handled by Artificial Intelligence (AI) planning, which is a subarea of Artificial Intelligence (AI). This subarea investigates the process of computing a course of action that satisfies a given user goal [5]. In the area of AI planning, Hierarchical Task Network (HTN) planning is well-known for its suitability to efficiently solve planning problems in real-world domains [6].

The aim of this project is to model the domain of EVs for HTN planning within the framework
proposed in [3]. The project includes the following topics:

  • Defining the problem of planning the driving task for EVs as an HTN planning problem
  • Acquiring knowledge about the domain of EVs in a systematic way (knowledge acquisition)
  • Defining the requirements of the EVs domain
  • Defining the entities and tasks that are part of this domain
  • Defining the sources of uncertainty that exist in this domain and their effect on the variability of action costs
  • Modelling the domain of EVs using a standard syntax, such as HPDL, by mapping the defined entities, tasks, and variable costs to HTN planning constructs
  • Adapting an existing HTN planning approach to solve planning problems in this domain optimally while taking different risk attitudes into account
  • Evaluating the proposed approach

Keywords: HTN planning, Electric vehicles, risk, uncertainty


[1] J. A. Sanguesa, V. Torres-Sanz, P. Garrido, F. J. Martinez, and J. M. Marquez-Barja, “A review on electric vehicles: Technologies and challenges,” Smart Cities, vol. 4, no. 1, pp. 372–404, 2021.

[2] J. Eisner, S. Funke, and S. Storandt, “Optimal route planning for electric vehicles in large networks,” in Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.

[3] E. Alnazer, I. Georgievski, and M. Aiello, “Risk Awareness in HTN Planning,” arXiv preprint arXiv:2204.10669, 2022.

[4] K. Valogianni, W. Ketter, and J. Collins, “Smart charging of electric vehicles using reinforcement learning,” in Workshops at the Twenty-Seventh AAAI Conference on Artificial Intelligence, 2013.

[5] M. Ghallab, D. Nau, and P. Traverso, Automated Planning: theory and practice. Elsevier, 2004.

[6] I. Georgievski and M. Aiello, “HTN planning: Overview, comparison, and beyond,” Artificial Intelligence, vol. 222, pp. 124–156, 2015.

Supervisor: M.Sc. Ebaa Alnazer


AI Planning Under Uncertainty for Sustainable Buildings

With the increment of CO2 emission, the world energy consumption, and the diversity in energy sources (finite and renewable), the goals of building management systems are widened to include cutting energy consumption; hence preserving finite resources, using non-carbon sources when possible to lower the CO2 footprint, and lowering costs for consumers and businesses while preserving users comfort [2]. This is especially true for buildings that are connected to a smart grid, which makes it possible to include renewable sources and provides dynamic pricing and energy offers coming from competing providers [4]. The advent of the Internet of Things (IoT) offers significant opportunities to improve the limited control capabilities offered by current building management systems [5]. In other words, equipping commercial buildings with different sensors and actuators allows management systems to make informed and "smart" decisions to control the environment and adjust it towards satisfying predefined objectives, such as energy saving. This "smartness" implies automating the decision-making process. Artificial Intelligence (AI) planning fits perfectly for this purpose since it offers automation techniques to intelligently search and compose a sequence of actions, i.e., plans when executed, can achieve a predefined set of goals [6].

Planning in such domains is challenging due to several types of uncertainty, e.g., weather forecasting, demand forecasting, and market prices forecasting [3, 2]. Consider the following example. A smart building is connected to a smart grid and has the goal of maintaining economical and environmental sustainability. The building can get its electricity from two different sources; either from stored electricity generated locally by renewable resources or from electric utilities with varying prices and energy offers. The choice of whether to consume the locally generated electricity or the purchasing it from outside sources should be made under uncertainty about the future market prices and the future weather forecast. In many approaches, AI planning problems are modelled with uncertainties about the action outcomes in mind; hence the resulting plans in such approaches are of a probabilistic nature [7, 1].

The aim of this project is to develop an AI planning-based approach in such domains. This involves defining the entities that are part of these domains, defining the types of uncertainty that can arise in these domains, modelling the AI planning problem by using a standard syntax that can model some sort of uncertainty, e.g., PPDDL [8], solving the planning problem using existing AI planners that can deal with uncertainties, and evaluating the efficiency and effectiveness of the proposed approach.

Keywords: sustainable buildings, Internet of Things, AI planning


[1] Blythe, J.: An overview of planning under uncertainty. Artificial Intelligence Today, pp. 85-110 (1999).

[2] Fiorini, L., Aiello, M.: Energy management for user’s thermal and power needs: A survey. Energy Reports 5, 1048-1076 (2019).

[3] Fiorini, L., Aiello, M.: Predictive multi-objective scheduling with dynamic prices and marginal CO2-emission intensities. In: Proceedings of the Eleventh ACM International Conference on Future Energy Systems. pp. 196-207 (2020).

[4] Georgievski, I.: Coordinating services embedded everywhere via hierarchical planning. Ph.D. thesis, University of Groningen (2015).

[5] Georgievski, I., Nguyen, T.A., Nizamic, F., Setz, B., Lazovik, A., Aiello, M.: Planning meets activity recognition: Service coordination for intelligent buildings. Pervasive and Mobile Computing 38, 110-139 (2017).

[6] Ghallab, M., Nau, D., Traverso, P.: Automated Planning: theory and practice. Elsevier (2004).

[7] Koenig, S., Simmons, R.G.: Risk-sensitive planning with probabilistic decision graphs. In: Principles of Knowledge Representation and Reasoning. pp. 363-373. Elsevier (1994).

[8] Younes, H.L., Littman, M.L.: PPDDL1. 0: An extension to PDDL for expressing planning domains with probabilistic effects. Techn. Rep. CMU-CS-04-162 2, 99 (2004).

Supervisor: M.Sc. Ebaa Alnazer


Topology and Semantic Maps for Spatial Recognition, Robot Navigation, and Path Planning in indoor environments

Autonomous Mobile robots can navigate through indoor spaces to support and render services to humans through interaction and collaboration with them. As the use of service robots has become popular, beneficial, and has a relatively low purchase cost, we are exploring the use of service robots in intelligent office buildings. This project aims to study Topology and Semantic Mapping, a crucial aspect of autonomous mobile robot navigation and path planning. The work is mostly experimental but requires a study of related work to guide you through the research project's progress.

The main research question is: Can Topological and Semantic Mapping of an indoor environment improve Human to Robot Interaction and Collaboration of a service robot, and what are the advantages and disadvantages of such Maps?

The answer to the question would involve carrying out the following tasks;

  • A Study of Literature on the Conversion of Grid Map to Topology and Semantic Mapping. State of the Art is not limited to research using papers. One can find Github repositories with similar implementations using gazebo simulation or in real-life conditions. Media Channels such as Youtube could be of assistance.
  • Building a Grid Map using ROS SLAM package.
  • Designing a Topological Map from the Grid Map composed of Nodes (Places) and Edges (Paths connecting Places), i.e., a graph to Map the paths and Static Objects/Destinations.
  • Designing a Semantic Map from the Topological Map and using it for path planning for robot navigation with an obstacle detection feature.
  • Employing the use of ROSPlan package for robot navigation using a Semantic Map.
  • Employing the use of ROS localization package for sensor fusion.
  • Lidar Sensor and Sonar sensors will be used in this project for pose estimation, navigation, and obstacle detection and avoidance.

Keywords: Service Robotics, SLAM, Robot Navigation, Path Planning, AI Planning

Prerequisites: The Student requires a Knowledge of AI Planning, ROS, Docker, Python or C++ programming language, and Microcontrollers.


[1] Chen, Y., Zhang, J., Lou, Y.: Topological and semantic map generation for
mobile robot indoor navigation. In: Intelligent Robotics and Applications:
14th International Conference, ICIRA 2021, Yantai, China, October 22–
25, 2021, Proceedings, Part I 14. pp. 337–347. Springer (2021)

[2] Deeken, H., Puetz, S., Wiemann, T., Lingemann, K., Hertzberg, J.: In-
tegrating semantic information in navigational planning. In: ISR/Robotik
2014; 41st International Symposium on Robotics. pp. 1–8. VDE (2014)

[3] Katsumata, Y., Taniguchi, A., Hagiwara, Y., Taniguchi, T.: Semantic mapping based on spatial concepts for grounding words related to places
in daily environments. Frontiers in Robotics and AI 6, 31 (2019)
[4] Kostavelis, I., Charalampous, K., Gasteratos, A., Tsotsos, J.K.: Robot navigation via spatial and temporal coherent semantic maps. Engineering
Applications of Artificial Intelligence 48, 173–187 (2016)
[5] Landsiedel, C., Wollherr, D., Beetz, M.: Semantic Mapping for Autonomous Robots in Urban Environ-
ments. Universit ̈atsbibliothek der TU M ̈unchen (2018),
[6] Qi, X., Wang, W., Yuan, M., Wang, Y., Li, M., Xue, L., Sun, Y.: Building semantic grid maps for domestic robot navigation. International Journal of Advanced Robotic Systems 17(1), 1729881419900066 (2020)
[7] Taniguchi, A., Ito, S., Taniguchi, T.: Spatial concept-based topometric semantic mapping for hierarchical path-planning from speech instructions. arXiv preprint arXiv:2203.10820 (2022)
[8] Wu, H., Tian, G.h., Li, Y., Zhou, F.y., Duan, P.: Spatial semantic hybrid map building and application of mobile service robot. Robotics and Autonomous Systems 62(6), 923–941 (2014)
[9] Zhang, J., Wang, W., Qi, X., Liao, Z.: Social and robust navigation for indoor robots based on object semantic grid and topological map. Applied Sciences 10(24), 8991 (2020

Supervisor: Nasiru Aboki (M.Sc.)


Spatio-Temporal Object Recognition and Control and the use of Augmented Reality (AR) for robot navigation and location assistance

Augmented Reality (AR) involves superimposing digital content into real-life environments. The AR content is usually interactive and is used to enhance the real-time experience of the user. 
AR is used in various aspects of our lives, from entertainment to healthcare. For navigational purposes in outdoor environments, a Google Maps application uses a GPS data signal and AR in a live-view mode in the form of a directional arrow to guide a user to a destination while also displaying distance information to the goal and with pit point accuracy.
The ease of path planning in an indoor environment to perform a task can be a daunting task, especially when one has to navigate to and from a waypoint or an already visited destination and also recognize objects or points of interest while going about an activity.

This project aims to explore the use of AR to ease the challenges of real-time navigation, path planning, and location assistance in an indoor environment.
To accomplish the objective, the idea is to investigate using a hardware sensor for mapping an indoor environment using AR technology and identify points of interest while mapping the environment. Recognizing static objects as points of interest (POI) could also teach the robot about its position and orientation on a given map.
The implementation needs to be evaluated to compare its merits and demerits against state of the art in indoor robot navigation.

The project tasks include:

  • Study papers on AR and how it can be used with a Map to aid location assistance and navigation in indoor environments.
  • With a Pre-Map of an indoor environment, implement robot navigation using AR.
  • Employ the use of the ROSPlan package for robot navigation.
  • Kinetic RGB-D Cam and Lidar sensors will be used for the project.
  • An AR arrow displays the robot's distance from objects on a screen (RviZ) and can be used for object avoidance. A Lidar provides scan messages with ranges of identified objects with its field of view (FOV).
  • Having Points of interest (POI) can be an easy way to identify static content, e.g., tables, and this is possible with technology like Positional optical character recognition (OCR).
    Platforms like INDOAR: Indoor Navigation with Augmented Reality provide AR services that can be used in living spaces.

Keywords: Service Robotics, SLAM, Robot Navigation, Path Planning, AI Planning, Augmented Reality, Computer Vision

Prerequisites: The Student requires a Knowledge of AI Planning, ROS, MQTT, Docker, Python, or C++ programming language.


[1] Huang, B.-C.; Hsu, J.; Chu, E.T.-H.; Wu, H.-M. ARBIN: Augmented Reality Based Indoor Navigation System. Sensors 2020, 20, 5890.

[2] A. Corotan and J. J. Z. Irgen-Gioro, "An Indoor Navigation Robot Using Augmented Reality," 2019 5th International Conference on Control, Automation and Robotics (ICCAR), Beijing, China, 2019, pp. 111-116, doi: 10.1109/ICCAR.2019.8813348.

[3]  Makhataeva, Z.; Varol, H.A. Augmented Reality for Robotics: A Review. Robotics 2020, 9, 21.

[4] Neges, M., Koch, C., K ̈onig, M., Abramovici, M.: Combining visual natural markers and imu for improved ar based indoor navigation. Advanced Engineering Informatics 31, 18–31 (2017).,, towards a new generation of the smart built environment

[5] Patel, V.: Augmented reality-based indoor navigation using point cloud localization. Ph.D. thesis, Laurentian University of Sudbury (2021)

[6] Papcun, P., Cabadaj, J., Kaj ́ati, E., Romero, D., Landryov ́a, L., Vascak, J., Zolotov ́a, I.: Augmented reality for humans-robots interaction in dynamic slotting “chaotic storage” smart warehouses. pp. 633–641 (2019).

Supervisor: Nasiru Aboki (M.Sc.)


Utilizing Robot Virtual (Voice) Assistant, Navigation, and Object Detection Modules for the Recognition and Control of Intelligent devices

Deep learning has enabled machines with the ability to use computer vision to classify objects to make decisions. Many of the classified objects have similar features, which have enabled them to be grouped under a general name, such as a dog, cat, or a building. Some object classifications even go into extra detail about the objects to identify their color, pose, dimensions, etc.
In an intelligent living space involving sensors and actuators, a robot can use visual perception to recognize the smart environments in its environment and use intents to control a set of devices.
This project aims to enable a robot to identify specific devices within its field of view (FOV) to control them collectively or individually or to provide data about the device's state, for example, if a lamp is in an off or an on State.
To address this task, knowledge of how a service-oriented architecture operates is necessary, as well as the various technologies that can be used within it.
The project tasks include:

  • A design and implementation of a system based on a service-oriented architecture that will be used to discover and control smart devices by a robot.
  • The student will be given perception modules to study and use for service composition and orchestration. 
  • The system design should allow a human to query the robot to identify and control objects in its field of view (FOV) or provide its state.
  • An evaluation of the system's performance will be investigated and reported, also stating significance and comparison or contrast to the state of the art of how robots perceive and control smart systems.

Keywords: Service-Oriented Architecture, AI Planning, MQTT, Computer Vision, Python Programming, Docker, NLP

Prerequisites: The Student requires a Knowledge of AI Planning, ROS, MQTT, Docker, Python, or C++ programming language.


[1] Taniguchi, A., Ito, S., Taniguchi, T.: Spatial concept-based topometric semantic mapping for hierarchical path-planning from speech instructions. arXiv preprint arXiv:2203.10820 (2022)

[2] Landsiedel, C., Wollherr, D., Beetz, M.: Semantic Mapping for Autonomous Robots in Urban Environ-
ments. Universit ̈atsbibliothek der TU M ̈unchen (2018),
[3] Wang, J., Shi, Z.: A speech interaction system based on cloud service under ros. In: 2019 Chinese Control Conference (CCC). pp. 4721–4725. IEEE (2019)
[4] Ghit ̧ ̆a, S ̧ .A., Barbu, M.S ̧ ., Gavril, A., Tr ̆asc ̆au, M., Sorici, A., Florea,
A.M.: User detection, tracking, and recognition in robot assistive care scenarios. In: Towards Autonomous Robotic Systems: 19th Annual Conference, TAROS 2018, Bristol, UK July 25-27, 2018, Proceedings 19. pp. 271–283. Springer (2018)
[5] Ghit, ̆a, A., Gavril, A.F., Nan, M., Hoteit, B., Awada, I.A., Sorici, A., Mocanu, I.G., Florea, A.M.: The amiro social robotics framework: Deployment and evaluation on the pepper robot. Sensors 20(24), 7271 (2020)


[6] J. Hua, S. Lee, G. -C. Roman and C. Julien, "ArcIoT: Enabling Intuitive Device Control in the Internet of Things through Augmented Reality," 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Kassel, Germany, 2021, pp. 558-564, doi: 10.1109/PerComWorkshops51409.2021.9431115.

Supervisor: Nasiru Aboki (M.Sc.)

A Large Language Model (LLM) as an AI Virtual Assistant for a Robot

An AI Virtual assistant can understand natural language commands to accomplish a set of tasks for a user.
Google Assistant, Alexa, and Siri are some of the popular Virtual Assistants that are commercially available and are widely used in phones, smart speakers and TVs, and car Infotainment Systems.
AI Large Language models (LLM) have become a hot topic since the release of Open-AI's Chat-GPT (3 or 4) and then Google Bard, based on LaMDA (Language Model for Dialog Applications).
Building a Voice Assistant requires using many open-sourced libraries. Still, it takes a lot of effort to set the frameworks up for training, to recognize a wake-up word, or to recognize specific topics.
Given the vast knowledge of an LLM, it is expected that you would require less effort to set it up a Voice Assistant than the aforementioned approach. In the proposed research, we would like to explore the possibility of using one of the free versions available (ChatGPT or Google Bard) as a virtual assistant for a robot.

Tasks will involve research in the field of LLM, and the virtual assistant capabilities expected are; the virtual assistant being able to make a conversation once it is in contact with an individual, having the ability to respond to a name when called, having the ability to provide answers to general questions, having the ability to make use of intents to control IoT devices in the smart lab, having access to a database and service repository to store and/or retrieve data, and having access to a service repository to store retrieve information to inform a user or orchestrate an actuation.

Keywords: Virtual Voice Assistant, Large Language Models, ROS, ChatGPT, Google Bard, NLP

Prerequisites: The Student requires a Knowledge of ROS, MQTT, and Docker, MQTT, Docker, Python, or C++ programming language.


[1] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners (2020)

[2] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding (2019)
[3] Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)
[4] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
[5] Zhang, Y., Sun, S., Galley, M., Chen, Y.C., Brockett, C., Gao, X., Gao, J., Liu, J., Dolan, B.: Dialogpt: Large-scale generative pre-training for conversational response generation (2020)


Supervisor: Nasiru Aboki (M.Sc.)



Dieses Bild zeigt Marco Aiello

Marco Aiello

Prof. Dr.


Elisabeth Ibach



Dieses Bild zeigt Ilche Georgievski

Ilche Georgievski


Leitung Arbeitsbereich Planning, Learning, and Intelligent Systems

Zum Seitenanfang