||Along with the developments in robotics, robots have started to being used in other domains than industry. The usage of robots in domestic, hospital and similar environments revealed various problems related to human-machine interaction. This work proposes to complement traditional ways for controlling robots, such as speech recognition or graphical user interfaces, by automatic recognition of human actions and behaviors, allowing robots to guess intended actions and to assist them in their daily tasks. Mobile robots may perceive human activities from any position and any viewpoint angle. To this end, new viewpoint-invariant representations are explored. A system with multiple robots needs to incorporate information describing the same scene. Our goal is to establish a framework which will allow early fusion of the information acquired, instead of simply voting individual recognition results.