audio-visual

A curated list of different papers and datasets in various areas of audio-visual processing

ACM MM 2021: 'Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection'

[NeurIPS 2025] OmniTalker: Real-Time Text-Driven Talking Head Generation with In-Context Audio-Visual Style Replication

🪐 HelixWorld: real-time interactive audio-visual world model.

Write small programs with tools & libraries you know and love - and patch them together. Make interactive visuals, sounds and play with computation on the web.

An audio visualizer for React. Provides separate components to visualize both live audio and audio blobs.

🎙 Generator waveform paths for SVG 🎶

Source code for "Synchformer: Efficient Synchronization from Sparse Cues" (ICASSP 2024)

[AAAI 2024] The official PyTorch implementation of "Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation"

Browser-based AV signal flow diagram tool for broadcast, live production, and AV integration

Human Emotion Understanding using multimodal dataset.

Implementation of "EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition, ICCV, 2019" in PyTorch

Libvisual Audio Visualization

Awesome Audio-Visual Intelligence, Survey of Audio-Visual Intelligence

OmniAgent (ICML 2026): the first native omni-modal agent for active video perception — a 7B agent that beats Qwen2.5-VL-72B with 73% fewer frames on LVBench.

Source code for "Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors." (Spotlight at the BMVC 2022)

Programmatic minimalistic audio visualizations.

ICLR 2026 Oral: WAVE: Learning Unified & Versatile Audio-Visual Embeddings with Multimodal LLM

Official code for WACV 2024 paper, "Annotation-free Audio-Visual Segmentation"

[CVPR 2023] Collecting Cross-Modal Presence-Absence Evidence for Weakly-Supervised Audio-Visual Event Perception