Objective: Tau neurofibrillary tangles (T) are the primary driver of downstream neurodegeneration (N) and subsequent cognitive impairment in Alzheimer's disease (AD). However, there is substantial variability in the T-N relationship – manifested in higher or lower atrophy than expected for level of tau in a given brain region. The goal of this study was to determine if region-based quantitation of this variability allows for identification of underlying modulatory factors, including polypathology. Methods: Cortical thickness (N) and 18F-Flortaucipir SUVR (T) were computed in 104 gray matter regions from a cohort of cognitively-impaired, amyloid-positive (A+) individuals. Region-specific residuals from a robust linear fit between SUVR and cortical thickness were computed as a surrogate for T-N mismatch. A summary T-N mismatch metric defined using residuals were correlated with demographic and imaging-based modulatory factors, and to partition the cohort into data-driven subgroups. Results: The summary T-N mismatch metric correlated with underlying factors such as age and burden of white matter hyperintensity lesions. Data-driven subgroups based on clustering of residuals appear to represent different biologically relevant phenotypes, with groups showing distinct spatial patterns of higher or lower atrophy than expected. Interpretation: These data support the notion that a measure of deviation from a normative relationship between tau burden and neurodegeneration across brain regions in individuals on the AD continuum captures variability due to multiple underlying factors, and can reveal phenotypes, which if validated, may help identify possible contributors to neurodegeneration in addition to tau, which may ultimately be useful for cohort selection in clinical trials. ANN NEUROL 2021;90:751–762.