Background Molecular testing with the Thyroseq v2 next generation sequencing panel (“Thyroseq”) is used to estimate the risk of cancer in indeterminate thyroid nodules. Methods We analyzed 156 indeterminate thyroid nodules evaluated with Thyroseq, across 3 institutions. Thyroseq data and surgical pathology were matched via pathologic re-review. A result was considered Thyroseq positive if molecular alterations were annotated on the report with malignancy probability >30%. Performance characteristics were estimated using Bayes theorem. Results The Thyroseq-negative call rate was 65% (102/156). On surgical pathology, 16% (10/63) of nodules were malignant. The positive predictive value of a Thyroseq-positive result was 22% (8/37; if 2 noninvasive follicular thyroid neoplasm with papillary-like nuclear features are counted as malignant, 27%, 10/37). There was 1 false-negative result (negative predictive value 96%, 22/23). The most common mutation was NRAS (19/37) with a positive predictive value of 7% (1/15). The positive predictive value of all RAS mutations (HRAS, KRAS, NRAS) was 9% (2/22). The second most common mutation, BRAF V600E, had positive predictive value of 100% (3/3). Conclusion We report an external analysis of Thyroseq performance in the evaluation of indeterminate thyroid nodules. These data indicate that Thyroseq is likely to offer high negative predictive value but low positive predictive value. Many genetic alterations appear to be nonspecific for malignancy, and positive results should be interpreted with caution. These findings have implications for the management of indeterminate thyroid nodules profiled with Thyroseq.