Full project name: Research and development methods for autonomous calibration and analysis of the position of limbs of a humanoid robot based on the image obtained from a single camera.

To achieve the research goal, the calibration of an humanoid robot manipulator using the built-in cameras and developed autonomous calibration algorithm. The first priority task is to calibrate the onboard cameras of the robot (Fig. 1). The research  focuses on the issues of autonomous camera calibration and determining the tools and algorithms necessary for the calibration process.

Fig. 1 Humanoid robot AR-601M

According to the results of the literature review analysis of Russian and foreign sources on the research topic, we singled out a method for autonomous calibration of mono and stereo cameras using fiducial markers as the most promising direction for solving the tasks. This technology allows to automatically detect markers in a frame using the corresponding individual recognition algorithm. Fiducial markers have a wide range of applications and, due to the variability of the capabilities of the technology, could be effectively used in one area, but they are completely unsuitable for another. There are a number of criteria for evaluating the performance of these systems in order to determine the appropriate marker for the use in particular case and the conditions for working with it. Within the framework of the study, the criteria of resistance of the marker to occlusion overlapping of various intensity by extraneous objects was selected. This criteria is critical in the intended use of the marker when fixing it on the robot arm; as a result, the marker may be partially blocked by other parts of the robot and have different angles of rotation relative to the robot camera.

Three systems were selected for comparison out of the several dozens of fiducial marker systems that were considered: ARTag, AprilTag and CALTag (Fig. 2).

Fig. 2 ARTag, AprilTag и CALTag systems (left to right)

After experiment design of research of the marker performance for overlapping, we conducted pilot experiments using a simple Genius FaceCam 1000X webcam, followed by experiments with the real humanoid robot AR-601M. According to the results of experimental work, the following properties of each of the marker systems were identified:

1. The AprilTag system showed high sensitivity to marker border overlapping. The overlap of borders does not allow to detect the marker edges, which leads to the impossibility of further marker recognition. However, the system showed satisfactory results in overlapping its internal pattern.

2. The ARTag system is also sensitive to marker overlap. This system is applicable for a number of tasks in which the probability of marker overlap is minimized and only a slight overlap of the internal marker pattern is allowed.

3. The CALTag system showed high results during all experiments. This system is resistant to any type of overlap studied and could be used in the case of possible partial marker overlap and various turns of the mark relative to the camera.

As a result of conducted study we selected the CALTag system for further additional research in order to assess the potential for further integration of the system into the control system of the robot AR-601M. It is also planned to investigate the BlurTag and RuneTag systems in relation to marker overlap, as the authors of these systems indicate resistance to overlap and large turning angles of the lmarker relatively to the camera, but there are no documented confirmations from independent sources about the qualitative results of experimental work on these systems.

The works were sponsored by the Russian Foundation for Basic Research and the Government of the Republic of Tatarstan in the framework of the research project No. 17-48-160879.

Publications on Project:

1. A. Sagitov, K. Shabalina, E. Magid. ARTag, AprilTag and CALTag Fiducial Marker Systems: Comparison in a Presence of Partial Marker Occlusion and Rotation. International Conference on Informatics in Control, Automation and Robotics, pp.182-191, 2017.

2. K. Shabalina, A. Sagitov, E. Magid, "Comparing Fiducial Marker Systems Occlusion Resilience Through a Robot Eye." Int. Conf. Developments in eSystems Engineering, 2017.

3. A. Sagitov, K. Shabalina, L. Hongbing, E. Magid. "Effects of rotation and systematic occlusion on fiducial marker recognition." In MATEC Web of Conferences, vol. 113, p. 02006, 2017

4. К. Шабалина, А. Сагитов, Е. Магид. «Сравнение систем координатных меток для калибровки камер мобильного робота в условиях перекрытий» Беспилотные транспортные средства с элементами искусственного интеллекта, с. 65-75, 2017

5. K.Shabalina, A. Sagitov, H. Li, Edgar A. Martinez-Garcia, E. Magid. Virtual Experimental Stand for Automated Fiducial Marker Comparison in Gazebo Environment. The 2018 International Conference on Artificial ALife and Robotics (ICAROB 2018), pp. 411-414, 2018.

6. Р.Н. Сафин, Р.О. Лавренов, С.К. Саха, Е.А. Магид. Эксперименты по калибровке камер мобильного робота при наличии аппаратных дефектов в системе технического зрения. Научно-методический и информационный журнал Вестник НЦБЖД, 2018 (в печати)

7. К. Шабалина, Е. Магид, А. Сагитов. Виртуальный подход для проведения автоматизированных экспериментов сравнения систем координатных меток в среде GAZEBO. Научно-методический и информационный журнал Вестник НЦБЖД, 2018 
8. Е. Магид, А. Сагитов, К. Шабалина, Р. Лавренов, Л. Сабирова. Исследование методов автономной калибровки камеры антропоморфного робота AR-601M. Сборник докладов Научно-технической конференции по итогам совместного конкурса фундаментальных исследований РФФИ – РТ, с.173-177, 2017.