This thesis investigates the application of semantic web techniques to multimedia. The objective of the semantic Web is to make (descriptions of) Web resources interpretable by machines and programs, by making use of shared descriptions (i.e. vocabularies, ontologies). Resources can then be described by annotations using ontologies' concepts and relations. Conventional semantic Web languages were designed to address all types of resources (text, images …), but they are restricted to the description of abstract objects and their relationships. Using those languages to describe multimedia data requires extensions able to take into account visual features. These extensions should enable to describe: 1) the mappings between the concepts of an ontology and their visual representations in images and video sequences, and 2) the mappings between the high level visual representations (building, person…) and the low level features (color, shape, texture, motion…) in an image or a video sequence. Such a language should be equiped with mechanisms to perform spatial and temporal reasoning on visual representations of objects. An ontology containing this kind of knowledge allows generating automatically high level symbolic annotations of multimedia resources, i.e. to recognize the entities present in images and videos by combining techniques from image processing that provide low level descriptors and constraint propagation.