Overview
ASO-Temba extends MS-Temba, a Mamba-based model for understanding long, untrimmed videos. The project investigates how temporal features should be combined across scales and how action detection can operate as video arrives.
Two Experimental Directions
Multi-Scale Feature Fusion
The main branch replaces the original state-space fuser with learned weighted sums and several attention-based alternatives. These variants explore different ways to aggregate temporal information from multiple scales.
Causal Streaming
A separate branch uses forward-only state-space models and a causal fuser, removing access to future frames. Frame- and chunk-level inference supports online detection, while a causal consistency objective replaces the original diversity loss.
Experiments and Resources
The pipeline uses pre-extracted I3D or CLIP features from Charades and Toyota Smarthome Untrimmed. Training and evaluation scripts support comparisons between the fusion and streaming variants.
See the repository, streaming branch, and project poster for implementation details and experimental context.