To elucidate the pathogenesis of vein of Galen malformations (VOGMs), the most common and most severe of congenital brain arteriovenous malformations, we performed an integrated analysis of 310 VOGM proband-family exomes and 336,326 human cerebrovasculature single-cell transcriptomes. We found the Ras suppressor p120 RasGAP (RASA1) harbored a genome-wide significant burden of loss-of-function de novo variants (2042.5-fold, p = 4.79 x 10−7). Rare, damaging transmitted variants were enriched in Ephrin receptor-B4 (EPHB4) (17.5-fold, p = 1.22 x 10−5), which cooperates with p120 RasGAP to regulate vascular development. Additional probands had damaging variants in ACVRL1, NOTCH1, ITGB1, and PTPN11. ACVRL1 variants were also identified in a multi-generational VOGM pedigree. Integrative genomic analysis defined developing endothelial cells as a likely spatio-temporal locus of VOGM pathophysiology. Mice expressing a VOGM-specific EPHB4 kinase-domain missense variant (Phe867Leu) exhibited disrupted developmental angiogenesis and impaired hierarchical development of arterial-capillary-venous networks, but only in the presence of a “second-hit” allele. These results illuminate human arterio-venous development and VOGM pathobiology and have implications for patients and their families.