Background: This study was designed to evaluate the prognostic value of three systemic inflammation markers, neutrophil to lymphocyte ratio (NLR), platelet to lymphocyte ratio (PLR), and prognostic nutritional index (PNI), for hepatocellular carcinoma (HCC) associated with hepatitis B (HBV). Methods: This analysis included 234 HBV-HCC patients who underwent primary surgical resection at the Mount Sinai Medical Center between 1988 and 2013. Serum albumin and circulating neutrophil, lymphocyte, and platelet counts immediately before surgery were obtained to calculate NLR, PLR, and PNI. Results: Patients with larger tumor size (>3 cm) had higher NLR, higher PLR, and lower PNI. Stratified analysis showed that the impact of three markers on outcome depends on the severity of liver fibrosis. High NLR, high PLR, or low PNI was associated with poor outcome only in patients without end-stage fibrosis (Ishak stage 0–5) and not in those with cirrhosis (Ishak stage 6). Multivariate analysis in Ishak stage 0–5 patients showed that only high NLR was associated with poor outcome independent of tumor size. Of the three markers, only NLR correlated with PD-L1 expression in center of tumor, but not in nonneoplastic liver. Conclusions: The prognostic value of these three markers following surgery was only significant for HBV-HCC patients without end-stage fibrosis, and among the three markers, only NLR remained a significant prognostic indicator independent of tumor size. The correlation of NLR with intratumoral PD-L1 expression raises a hypothesis for shared pathways leading to PD-L1-mediated local tolerance within tumor and systemic inflammatory responses represented by elevated NLR in HBV-HCC.