Autism is a complex neurodevelopmental disorder whose causative mechanisms are unclear. Taking advantage of a unique cohort with recessively inherited autism, we identified six families with biallelic mutation of the neuronal-specific subunit of the BAF complex, ACTL6B (also known as BAF53b). Relative to all other genes, ACTL6B was the most statistically significant mutated gene in the recessive autism cohort. We describe autism-relevant phenotypes in human brain organoids and in mouse and fly models. We foresee the outcomes from this study will be the following: 1) a link between neuronal activity-dependent transcriptional repression and autism; 2) a characterization of mouse and fly models to study ACTL6B mutant autism; and 3) an understanding the role of ACTL6B and nBAF complexes in neuronal transcriptional regulation.