This study evaluates the ability of the Community Multiscale Air Quality (CMAQ) model to simulate the spatial variability of summertime ozone (O3) at the surface and in the free troposphere over the continental United States. Simulated surface O3 concentrations are compared with 987 Air Quality System (AQS) sites and 123 Clean Air Status and Trends Network (CASTNet) sites. CMAQ’s ability to reproduce surface observations varies with O3 concentration. The model best simulates observed O3 for intermediate concentrations (40–60 ppbv), while over-(under-) predicting at lower (higher) levels. CMAQ reproduces surface O3 for a wide range of conditions (30–80 ppbv) with a normalized mean error (NME) less than 35% and normalized mean bias (NMB) lying between 715% for the whole domain. Although systematically over-predicting O3 in the east and under-predicting it in the western United States, CMAQ is able to reproduce 1- and 8-h daily maxima with a cross-domain mean bias (MB) of 1 and 8 ppbv, or NMB of 8% and 25%, respectively. The model underestimates observed O3 at rural sites (MB ¼ 5 ppbv, NMB ¼ 5% and NME ¼ 23% with a 40 ppbv cut-off value) and over-predicts it at urban and suburban sites by a similar magnitude (MB ¼ 6 ppbv, NMB ¼ 7% and NME ¼ 25%). Apparent errors and biases decrease when data is averaged over longer periods, suggesting that most evaluation statistics are dependent on the time scale of data aggregation. Therefore, performance criteria should specify an averaging period (e.g., 1- or 8- h) and not be independent of averaging period as some current model evaluation studies imply. Comparisons of vertical profiles of simulated O3 with ozonesonde data show both overestimation and underestimation by 10–20 ppbv in the lower troposphere and a consistent under-prediction in the upper troposphere. Vertical O3 distributions are better simulated when lateral boundary conditions obtained from the global Model of Ozone and Related Tracers version 2 (MOZART-2) are used, but under-prediction remains. The assumption of zero-flux at the top boundary and the resulting exclusion of the contribution of stratosphere–troposphere exchange to tropospheric O3 concentrations limit the ability of CMAQ to reproduce O3 concentrations in the upper troposphere. r 2006 Elsevier Ltd. All rights reserved.