The purpose of this study was to examine the combined association of sitting time and physical activity with cognitive function and to determine whether moderate-to-vigorous physical activity (MVPA) is a mediator of the association between sitting time and cognitive function in a nationally representative sample of older adults from Chile. Data from 989 older adults (≥65 years old, 61.3% female) from the 2009–2010 Chilean Health Survey were analyzed. Physical activity and sitting time were measured using the Global Physical Activity questionnaire. Cognitive function was assessed using the modified Mini-Mental State levels Examination. Physical activity levels were categorized as “inactive” (<600 metabolic equivalent value minutes per week) or “active” (≥600 metabolic equivalent value minutes per week). Sitting time was categorized as “sedentary”, defined as ≥4 h of reported sitting time per day, or “non-sedentary”, defined as <4 h. We created the following groups (i) non-sedentary/active; (ii) non-sedentary/inactive; (iii) sedentary/active; and (iv) sedentary/inactive. Hayes's PROCESS macro was used for the simple mediation analysis. Compared with the reference group (individuals classified as non-sedentary/active), older adults who were classified as sedentary/active had elevated odds of cognitive impairment (OR = 1.90, [95% CI, 1.84 to 3.85]). However, the odds ratio for cognitive impairment was substantially increased in those classified as sedentary/inactive (OR = 4.85 [95% CI, 2.54 to 6.24]) compared with the reference group. MVPA was found to mediate the relationship between sitting time and cognitive function (Indirect Effect = −0.070 [95% CI, −0.012 to −0.004]). Conclusion: The present findings suggest that, whether overall physical activity is high or low, spending large amounts of time sitting is associated with elevated odds of cognitive impairment and that MVPA slightly weakens the relationship between sitting time and cognitive function.