Abstract : In this paper, we address the analysis of activities from long range video sequences. We present a method to automatically extract spatial and temporal structure from a video sequence from low level motion features. The scene layout is first extracted, with a set of regions that have homogeneous activities called Motion Patterns. These regions are then analyzed and the recurrent temporal motifs are extracted for each Motion Patterns. Preliminary results show that our method can accurately extract important temporal motifs from video surveillance sequences.