Temporal lobe epilepsy (TLE) affects multiple brain regions through evidence from both structural (gray matter; GM) and functional connectivity (FC) studies. We tested whether these structural abnormalities were associated with FC abnormalities, and assessed the ability of these measures to explain episodic memory impairments in this population. A resting-state and T1 sequences were acquired on 94 (45 with mesial temporal pathology) TLE patients and 50 controls, using magnetic resonance imaging (MRI) technique. A voxel-based morphometry analysis was computed to determine the GM volume differences between groups (right, left TLE, controls). Resting-state FC between the abnormal GM volume regions was computed, and compared between groups. Finally, we investigated the relation between EM, GM and FC findings. Patients with and without temporal pathology were analyzed separately. The results revealed reduced GM volume in multiple regions in the patients relative to the controls. Using FC, we found the abnormal GM regions did not display abnormal functional connectivity. Lastly, we found in left TLE patients, verbal episodic memory was associated with abnormal left posterior hippocampus volume, while in right TLE, non-verbal episodic memory was better predicted by resting-state FC measures. This study investigated TLE abnormalities using a multi-modal approach combining GM, FC and neurocognitive measures. We did not find that the GM abnormalities were functionally or abnormally connected during an interictal resting state, which may reflect a weak sensitivity of functional connectivity to the epileptic network. We provided evidence that verbal and non-verbal episodic memory in left and right TLE patients may have distinct relationships with structural and functional measures. Lastly, we provide data suggesting that in the setting of occult, non-lesional right TLE pathology, a coupling of structural and functional abnormalities in extra-temporal/non-ictal regions is necessary to produce reductions in episodic memory recall. The latter, in particular, demonstrates the complex structure/function interactions at work when trying to understand cognition in TLE, suggesting that subtle network effects can emerge bearing specific relationships to hemisphere and the type of pathology.