Multi-agent scene interpretation

A multi-agent architecture is presented for implementing scene understanding algorithms in the visual surveillance domain. To achieve a high level description of events observed by multiple cameras, many inter-related, event-driven processes must be executed. We use the agent paradigm to provide a framework in which these processes can be managed. Each camera has an associated camera agent, which detects and tracks moving events (or regions of interest). Each camera is calibrated so that image co-ordinates can be transformed into ground plane locations. Each camera agent instantiates and updates object agents for each stable image event it detects. Object agents are responsible for continually updating a 3D trajectory, a view-independent chromatic appearance model, a description of the event’s behaviour, and from these a classification of the object type itself. Camera agents synchronously supply each of its associated object agents with current chromatic and 3D positional observations of the tracked events. Each object agent classifies itself from a range of predefined activities each evaluated using a trained hidden Markov model. The combination of the agent framework, and visual surveillance application provides an excellent environment for development and evaluation of scene understanding algorithms.