burlington_brt_rebuild · checked on the baseline scenario
Model validation
How closely this study's model reproduces what was observed: traffic
counts, federal traffic volumes, commuting and transit ridership. Checked
on the baseline scenario, the
study's baseline.
Published guidelines are shown for reference: they are guidelines, not
pass/fail standards, and the manuals that publish them say so.
Independent check the model never saw this dataPartly independent some of this data was used to fit the modelConsistency only the model was fitted to this dataNo observed data a check on the run itself
Across 26 count locations in the study area, the model carries 45 % of the counted traffic in the analysis hour.
Technical detail
Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.
By count class
Count class
N
Geh lt5 pct
Geh lt10 pct
Median geh
Sum ratio
Rmse
Pct rmse
count_defect
0
—
—
—
—
—
—
reversible_closed
0
—
—
—
—
—
—
express_hov
0
—
—
—
—
—
—
wrong_road
0
—
—
—
—
—
—
out_of_area
6
83.3
83.3
2.32
1.52
59.4
105
clean
26
3.85
11.5
19.4
0.45
653
74.7
Clean counts by road type
Road type
N
Geh lt5 pct
Geh lt10 pct
Median geh
Sum ratio
Rmse
Pct rmse
freeway
10
0.000
0.000
25.3
0.43
943
61.6
smaller_arterial
16
6.25
18.8
18.0
0.48
369
79.5
Headline uses the clean counts; all counts together: 0.46×.
Sources
Traffic counts (2025-06-04 to 2025-06-25; 32 counts)
Percent RMSE by volume group
Independent check
75 %
Typical error at a single count location, as a share of the average count.
Published guideline. FHWA (2010) Figure 9.8 compiles %RMSE targets by volume group from Florida, Ohio and Oregon: from about 100 % on links under 5,000 vehicles/day down to about 20 % above 50,000/day (values read off the published chart). Those groups are daily volumes (AADT); this model's counts are one analysis hour, so its groups are not the same links and the comparison is indicative only.
FHWA, Travel Model Validation and Reasonableness Checking Manual, 2nd ed. (FHWA-HEP-10-042, 2010), §9.1.4, Figure 9.8
Technical detail
Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.
Clean counts
Volume group (veh/h)
Counts
%RMSE
RMSE (veh/h)
Modelled ÷ counted
Median GEH
<500
9
104
251
0.61
18.8
500-1000
7
53.4
366
0.48
17.2
1000-2000
9
59.9
841
0.41
25.5
2000-4000
1
57.1
1,789
0.43
37.8
>=4000
0
—
—
—
—
All counts
Volume group (veh/h)
Counts
%RMSE
RMSE (veh/h)
Modelled ÷ counted
<500
15
118
198
0.73
500-1000
7
53.4
366
0.48
1000-2000
9
59.9
841
0.41
2000-4000
1
57.1
1,789
0.43
>=4000
0
—
—
—
Sources
Traffic counts (2025-06-04 to 2025-06-25; 32 counts)
Modelled vs observed, by count
Independent check
32 counts: 6 out of study area, 26 clean.
Clean countsOther classes (left out of the headlines)
Technical detail
Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.
Modelled side: assigned demand + background preload (truck_and_band); two-way counts sum both directions.
Count classes (each count gets the first that applies)
Count class
Counts
What it means
Faulty count
0
The count's own records show a fault: one lane reads 0 in every hour, so the total misses it, or the count is doubled (every value even, or more vehicles in one lane than a lane can carry).
Reversible lane, closed
0
A lane closed in this direction at the count hour, such as a reversible express lane, so there is no traffic to compare.
Express / HOV lane
0
On or beside an express or HOV lane. Their traffic depends on tolls and occupancy rules, so they are scored apart.
Matched to the wrong road
0
Matched to the wrong road: more traffic than the matched road can carry, or a station whose directions matched different roads.
Out of study area
6
Outside the study area, where most traffic comes from outside the model.
Clean
26
Everything else: the fair test of the model, used in the headlines.
Counts left out of the headlines (6)
Count
Class
Why it is left out
Counted
Modelled
tmas:00D059:204353209-7799470942-0
Out of study area
0.6 km outside the study area
20
30.3
tmas:00D059:7799470942-204353209-0
Out of study area
0.6 km outside the study area
24.8
20.6
tmas:00L057:204676392-592437718-0
Out of study area
5.7 km outside the study area
134
198
tmas:00L057:592437718-204676392-0
Out of study area
5.7 km outside the study area
85.2
214
tmas:00W055:204976350-6262780318-0
Out of study area
1.6 km outside the study area
32
25
tmas:00W055:6262780318-204976350-0
Out of study area
1.6 km outside the study area
44.5
28.9
Largest differences (GEH)
Count
Counted
Modelled
GEH
Class
tmas:00D091:204497213-204372847-0
3,133
1,343
37.8
clean
tmas:00D091:204377692-204366042-0
1,678
704
28.2
clean
tmas:00W089:11238339453-204891587-0
1,195
228
36.3
clean
tmas:00W089:205005860-204932804-0
1,062
134
37.9
clean
tmas:00D893:204562003-204590342-0
1,170
259
34.1
clean
Sources
Traffic counts (2025-06-04 to 2025-06-25; 32 counts)
Peak-direction share at paired stations
Independent check
68.2 % vs 61.2 %
At 13 count stations with both directions clean, 61.2 % of the counted traffic runs in the busier (peak) direction; the model puts 68.2 % there. Typical station error 10.1 percentage points.
ModelCounted
Technical detail
Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.
Difference, model − counted
7.0 pp
Modelled ÷ counted, peak direction
0.50×
Modelled ÷ counted, counter-peak direction
0.37×
Two-way counts (no split)
0
The peak direction is the one with more counted traffic. The two ratios tell a level error (both off together) from a split error (the two apart).
By road type
Road type
Stations
Counted peak share
Modelled peak share
Difference (pp)
Median station error (pp)
Modelled ÷ counted, peak
Modelled ÷ counted, counter-peak
Freeways
5
61.8 %
68.1 %
6.31
5.28
0.47
0.36
Arterials
8
59.8 %
68.3 %
8.42
15.9
0.55
0.38
Paired stations
Station
Road type
Peak / counter
Counted peak
Counted counter
Modelled peak
Modelled counter
Counted peak share (%)
Modelled peak share (%)
Error (pp)
00D040
Arterials
5 / 1
653
399
474
102
62
82.3
20.2
00D061
Arterials
7 / 3
482
401
148
33.4
54.6
81.6
27
00D091
Freeways
5 / 1
3,133
1,678
1,343
704
65.1
65.6
0.50
00D092
Freeways
5 / 1
1,828
712
1,061
312
72
77.3
5.30
00D099
Freeways
7 / 3
1,862
1,384
963
584
57.4
62.3
4.90
00D132
Arterials
1 / 5
65
35.8
392
290
64.5
57.4
-7.10
00D277
Arterials
1 / 5
648
642
373
246
50.2
60.3
10.1
00D530
Arterials
7 / 3
978
631
575
266
60.8
68.4
7.60
00D531
Arterials
7 / 3
238
212
35.8
112
52.9
24.2
-28.7
00D853
Freeways
1 / 5
1,446
1,014
874
358
58.8
70.9
12.1
00D893
Arterials
5 / 1
1,170
526
259
62.3
69
80.6
11.6
00F029
Arterials
5 / 1
205
132
187
24.9
60.9
88.2
27.3
00W089
Freeways
5 / 1
1,195
1,062
228
134
52.9
62.9
10
Directional stations not paired (3)
Source
Station
Why it is not paired
tmas
00D059
partner not clean (out_of_area)
tmas
00L057
partner not clean (out_of_area)
tmas
00W055
partner not clean (out_of_area)
Sources
Traffic counts (2025-06-04 to 2025-06-25; 32 counts)
GEH statistic
Independent checkAdvanced
4 % below 5
Share of clean count locations whose GEH is below 5.
Published guideline. GEH below 5 on more than 85 % of links. TAG applies this to models whose trip tables may have been adjusted to counts (matrix estimation); a demand model without that step should not be expected to reach it.
UK DfT, TAG Unit M3.1 Highway Assignment Modelling (May 2026), §3.3.10–3.3.11, Table 2
Technical detail
Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.
GEH below 10, clean
12 %
Median GEH, clean
19.4
By count class
Count class
N
Geh lt5 pct
Geh lt10 pct
Median geh
Sum ratio
Rmse
Pct rmse
count_defect
0
—
—
—
—
—
—
reversible_closed
0
—
—
—
—
—
—
express_hov
0
—
—
—
—
—
—
wrong_road
0
—
—
—
—
—
—
out_of_area
6
83.3
83.3
2.32
1.52
59.4
105
clean
26
3.85
11.5
19.4
0.45
653
74.7
Clean counts by road type
Road type
N
Geh lt5 pct
Geh lt10 pct
Median geh
Sum ratio
Rmse
Pct rmse
freeway
10
0.000
0.000
25.3
0.43
943
61.6
smaller_arterial
16
6.25
18.8
18.0
0.48
369
79.5
Sources
Traffic counts (2025-06-04 to 2025-06-25; 32 counts)
TAG flow criteria
Independent checkAdvanced
4 %
Share of the 26 clean counts within TAG's flow criteria.
Published guideline. More than 85 % of links within 100 veh/h below 700 veh/h, within 15 % at 700–2,700 veh/h, and within 400 veh/h above 2,700 veh/h. Defined on hourly flows, the model's own unit. As with GEH, TAG's models may have had matrix estimation.
UK DfT, TAG Unit M3.1 Highway Assignment Modelling (May 2026), §3.3.10–3.3.11, Table 2
Technical detail
Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.
All counts
19 %
Sources
Traffic counts (2025-06-04 to 2025-06-25; 32 counts)
Vehicle-miles travelled
Regional VMT vs HPMS
Independent check
0.62×
Modelled vehicle-km on freeways, arterials and major collectors, against federal HPMS traffic volumes.
Published guideline. Florida's guideline, quoted by FHWA (2010): regional VMT within ±5 % (acceptable) or ±2 % (preferable). Freeways, arterials and major collectors only; local roads and ramps have no HPMS counterpart.
FHWA, Travel Model Validation and Reasonableness Checking Manual, 2nd ed. (FHWA-HEP-10-042, 2010), §9.1.4
Technical detail
Independent check: HPMS traffic volumes are not used to build or calibrate travel demand. They set the background through-traffic and truck layer, which is counted on both sides.
Model
229,461 veh-km per analysis hour
HPMS
371,825 veh-km per analysis hour
Passenger demand only
0.55×
Observed side uses TMAS-measured per-cell K-factors from this study's own volumes (provenance.hour_factor). Trucks still carry the assumed truck_hour_factor. Caveat: the measurement uses whatever months observed_counts fetched — early summer on studies to date — and summer mornings carry no school traffic, so school-year K is likely somewhat higher. Direction known, size unmeasured.
Sources
FHWA Highway Performance Monitoring System (HPMS) (vintage 2024)
Hour factor (AADT → analysis hour) (measured)
VMT by road type vs HPMS
Independent check
0.40×, 0.76×
Modelled ÷ HPMS vehicle-km, by road type (freeways and expressways; arterials and major collectors), on the matched basis; the thinner bar is the netted ratio.
Matched (headline)Netted
* No HPMS counterpart for most of its length; left out of the regional ratio.
Published guideline. FHWA (2010) Table 9.2 compiles VMT guidelines by functional class from five agencies: freeways within 6–7 %; principal arterials 7–15 %; minor arterials 10–15 %; collectors 15–25 %. This comparison groups principal arterials, minor arterials and major collectors into one cell, so its guideline is the range across those classes. HPMS does not inventory local roads; that row is a coverage artifact, not a finding.
FHWA, Travel Model Validation and Reasonableness Checking Manual, 2nd ed. (FHWA-HEP-10-042, 2010), §9.1.4, Table 9.2
Technical detail
Independent check: HPMS traffic volumes are not used to build or calibrate travel demand. They set the background through-traffic and truck layer, which is counted on both sides.
Road type
Model (veh-km per analysis hour)
HPMS
Ratio
Netted
Gross
Passenger only
Freeways and expressways
60,553
146,461
0.40
0.41
0.39
0.37
Arterials and major collectors
177,439
225,364
0.76
0.79
0.79
0.79
Local roads and unclassified
17,821
4,350
0.37
4.10
3.73
4.22
Observed side uses TMAS-measured per-cell K-factors from this study's own volumes (provenance.hour_factor). Trucks still carry the assumed truck_hour_factor. Caveat: the measurement uses whatever months observed_counts fetched — early summer on studies to date — and summer mornings carry no school traffic, so school-year K is likely somewhat higher. Direction known, size unmeasured.
N sections cached
N sections in area
N edges model
N edges wholly outside area
Model vmt share in area
N edges bound
N edges full basis
N edges pruned floor
N edges eligible
Bind rate
Preload available
Disposition available
Coverage
10,299
6,386
14,856
2,338
0.79
5,138
23,056
8,200
5,758
0.89
yes
yes
Sources
FHWA Highway Performance Monitoring System (HPMS) (vintage 2024)
Hour factor (AADT → analysis hour) (measured)
Congestion
Volume-to-capacity distribution
No observed dataAdvanced
0.0 %
Share of directional road-km at a volume-to-capacity ratio of 0.85 or more in the analysis hour. A reasonableness check: no observed data.
Technical detail
No observed data: A reasonableness check: there is no observed side.
Road type
Directional km
V/C ≥ 0.85
V/C ≥ 1.0
collector
1,353
0.000
0.000
freeway
178
0.000
0.000
local
2,904
0.000
0.000
multilane_arterial
53.7
0.000
0.000
ramp
10.3
0.000
0.000
smaller_arterial
594
0.000
0.000
all
5,092
0.000
0.000
directional road-km; assigned + background vehicles over directional capacity.
Trip lengths
Work trip lengths vs LODES
UnavailableAdvanced
The model's work trips by origin and destination were not saved for this study ([demand.balancing] is off).
Commute mode share
Commute mode share across the region
Partly independent
r = 0.88
How closely the model's car share of work trips follows the ACS from tract to tract (correlation weighted by commuters). Calibration fits the region's shares, not this pattern.
Car r = 0.88
Transit r = 0.63
Walk r = 0.82
Bicycle r = 0.60
Work-trip mode share by residential density (people per sq mi), model vs ACS, with the ACS 90 % margin of error.
ModelACS
Technical detail
Partly independent: Calibration fits the region's work mode shares to this same ACS table, so the regional level is consistency-only. How the shares vary from tract to tract is not fitted.
Mode
Model modes
Weighted correlation
Tracts
car
CAR, CAR_NONHH
0.88
49
transit
TRANSIT
0.63
49
walk
WALK
0.82
49
bicycle
BICYCLE
0.60
49
Mode
People / sq mi
Tracts
Model share
ACS share
ACS ± (90 %)
ACS commuters
car
<1,000
29
0.97
0.95
0.052
32,077
car
1,000–2,500
6
0.84
0.88
0.087
15,871
car
2,500–5,000
6
0.82
0.85
0.091
10,407
car
5,000–10,000
5
0.73
0.71
0.035
9,683
car
10,000–20,000
3
0.56
0.58
0.054
5,117
car
≥20,000
0
—
—
—
0.000
transit
<1,000
29
0.006
0.008
0.004
32,077
transit
1,000–2,500
6
0.033
0.019
0.008
15,871
transit
2,500–5,000
6
0.038
0.044
0.019
10,407
transit
5,000–10,000
5
0.052
0.044
0.016
9,683
transit
10,000–20,000
3
0.055
0.055
0.026
5,117
transit
≥20,000
0
—
—
—
0.000
walk
<1,000
29
0.015
0.028
0.007
32,077
walk
1,000–2,500
6
0.076
0.063
0.021
15,871
walk
2,500–5,000
6
0.092
0.057
0.018
10,407
walk
5,000–10,000
5
0.16
0.22
0.037
9,683
walk
10,000–20,000
3
0.32
0.26
0.051
5,117
walk
≥20,000
0
—
—
—
0.000
bicycle
<1,000
29
0.012
0.005
0.003
32,077
bicycle
1,000–2,500
6
0.046
0.032
0.013
15,871
bicycle
2,500–5,000
6
0.049
0.047
0.031
10,407
bicycle
5,000–10,000
5
0.056
0.031
0.018
9,683
bicycle
10,000–20,000
3
0.061
0.094
0.031
5,117
bicycle
≥20,000
0
—
—
—
0.000
Car
ACS drove alone + carpooled + motorcycle vs model car + car as passenger (CAR_NONHH)
Weighting
tract ACS commuters × share of the tract inside the zones
Density
2020 Census population per square mile of land
Sources
ACS 5-year estimates, census tracts (B08301) (vintage 2018–2022)
2020 Census block population (density, crosswalk) (vintage 2020)
Transit
Transit boardings vs NTD
Unavailable
The NTD agency match for this study has not been confirmed. `python -m connectome.cli ntd-match <study>` proposes one; the run-study pre-flight step confirms it into ntd_match.toml.
Transit passenger-miles vs NTD
Unavailable
The NTD agency match for this study has not been confirmed. `python -m connectome.cli ntd-match <study>` proposes one; the run-study pre-flight step confirms it into ntd_match.toml.
Average transit trip length
UnavailableAdvanced
The NTD agency match for this study has not been confirmed. `python -m connectome.cli ntd-match <study>` proposes one; the run-study pre-flight step confirms it into ntd_match.toml.
Commute time
Commute time
Independent check
17.3 vs 21.2 min
Mean work trip 17.3 min in the model; 21.2 min ± 0.7 reported in the ACS. Coincidence of the two distributions: 0.68.
ModelACS
Technical detail
Independent check: the model never saw this data.
Minutes
Model share
ACS share
ACS ± (90 %)
<5
0.085
0.031
0.006
5–9
0.17
0.12
0.010
10–14
0.23
0.17
0.012
15–19
0.20
0.19
0.011
20–24
0.12
0.17
0.012
25–29
0.083
0.068
0.007
30–34
0.047
0.11
0.009
35–39
0.023
0.027
0.004
40–44
0.012
0.031
0.005
45–59
0.014
0.044
0.006
60–89
0.008
0.027
0.005
≥90
0.002
0.012
0.003
ACS mean basis
county (B08136 ÷ B08303), coverage-weighted
Model work trips in the period
8,492
Sources
ACS 5-year estimates, census tracts (B08303) (vintage 2018–2022)
ACS commute times are self-reported and door to door. They cluster at 15, 30 and 60 minutes and run above a network travel time, which for car leaves out parking search and the walk at each end. Read the shape before the level.
ACS car is car, truck or van (drove alone and carpooled); its bicycle time is lumped with taxi, motorcycle and other means, so there is no bicycle row. The ACS means are county-level: tract-level aggregate times are mostly suppressed.
Share of work trips by commute time, per mode. Coincidence is the overlap of the two distributions (1 = identical).
Car coincidence 0.62
Transit coincidence 0.44
Walk coincidence 0.65Consistency only
ModelACS
Technical detail
Partly independent: Calibration fits work walk durations to this same ACS table, so the walk rows are consistency-only; car and transit are independent.
Mode
Coincidence
ACS workers
Independence
car
0.62
63,041
independent
transit
0.44
1,733
independent
walk
0.65
5,912
consistency
ACS commute times are self-reported and door to door. They cluster at 15, 30 and 60 minutes and run above a network travel time, which for car leaves out parking search and the walk at each end. Read the shape before the level.
Sources
ACS 5-year estimates, census tracts (B08134) (vintage 2018–2022)
Run quality
Assignment convergence
No observed data
9.7×10-7
Relative gap when the assignment stopped, after 2 iterations of tapb; target 1.0×10-5. Converged.
Published guideline. A relative gap of 10⁻⁴ (0.01 %) for link flows stable enough to compare scenarios; TAG asks for below 0.1 % on a base-year model.
Boyce, Ralevic-Dekic & Bar-Gera, "Convergence of Traffic Assignments: How Much Is Enough?", Journal of Transportation Engineering 130(1):49–55 (2004); UK DfT, TAG Unit M3.1 Highway Assignment Modelling (May 2026), Table 4
Technical detail
No observed data: a check on the run itself.
Relative gap as the assignment solver reports it: total travel time on the loaded network minus the travel time if every trip took its current shortest path, divided by the former. Zero is a perfect user equilibrium.
Demand-feedback convergence
No observed data
Converged at iteration 2
Demand and assignment re-run until travel times and the trip table stop changing.
Change in the trip table between outer iterations (RMSE ÷ mean), log scale.
Technical detail
No observed data: a check on the run itself.
Iteration
Assignment gap
Travel-time change
Trip-table change
Total-trip change
Criteria met
1
0.000
0.000
0.009
0.001
no
2
0.000
0.000
0.000
0.000
yes
Time gap tol
0.025
Od rmse tol
0.005
Total trips tol
0.005
Max outer
10
Min outer
2
Noise floor
Unavailable
The noise floor was not measured for this study ([assignment.feedback] measure_noise_floor is off).
Mode-choice calibration residual
Consistency only
residual 0.015
Converged at iteration 2 of 2; tolerance 0.050. Detail on the Calibration page.
Technical detail
Consistency only: Calibration fits these targets by construction.
Coverage
No observed data
312 zones
83 of 395 zones were too sparse to model; their residents and destinations moved to a neighbouring zone, not lost. Every home-based trip was assigned a destination.