Benchmark WI-VAL · version 1.1.0 · released 2026-08-01
Where the model is right, and where it is wrong
A model that is never checked against reality is an opinion with equations. WI-SL v1.2.0 is run blind against published field trials, standardised durability tests and in-service inspections, and every residual is published — including the ones that embarrass us.
Headline
Aggregate accuracy across the benchmark
- Cases
- 24
- External observations
- Bias
- -0.5 yr
- Model runs conservative
- MAE
- 8.5 yr
- Mean absolute error
- RMSE
- 11.2 yr
- Penalises large misses
- Coverage
- 87.5%
- Inside P10–P90 · target 80%
- r
- 0.63
- Predicted vs observed
Coverage is the honest number to watch. If far fewer than 80% of observations land inside the modelled P10–P90 band, the declared uncertainty is too narrow and the intervals are overconfident. If far more, the model is hedging and the bands carry little information. The widest single miss in this release is European Oak · Stockholm, Sweden at -22.5 years.
Current status, stated plainly: the headline row above is in-sample — the WI-CAL correction was fitted on these same cases, so it flatters the model by construction. The number to judge us by is the blind leave-one-out row below, where each case is predicted with the calibration refitted without it. Specify against the P10, not the P50.
Calibration WI-CAL v1.1.0
What the correction changed, and what it cost
| Model state | Bias (yr) | MAE (yr) | RMSE (yr) | MAPE | Coverage | r |
|---|---|---|---|---|---|---|
| Uncalibrated (WI-SL 1.1.0 physics only) | +19.6 | 22.3 | 26.5 | 121.4% | 0% | 0.36 |
| Blind — leave-one-out calibration | 0 | 9.5 | 12.6 | 37.5% | 83.3% | 0.56 |
| In-sample calibrated (WI-SL 1.2.0) | -0.5 | 8.5 | 11.2 | 33.1% | 87.5% | 0.63 |
- Estimator
- log-space mean
- log-space mean residual, shrunk to the global factor
- Global factor
- ×0.547
- Applied where a zone is unfitted
- Zone factors
- A 0.3268 · B 0.441 · C 0.7014 · D 0.6906 · E 0.547
- Shrunk with 2 pseudo-observations
- σ inflation
- ×4.5
- Residual spread needed to reach 80% coverage
Two honest readings of this table. First: the physics-only model was optimistic by a factor of roughly 1.8, and a flat multiplicative correction removes almost all of that bias without touching a single equation — so the error was in the level, not the ranking. Second: reaching real 80% interval coverage required inflating the residual σ by ×4.5. The declared input uncertainties alone understated true scatter roughly fourfold. Wide bands are the truth here; narrow ones were fiction.
Service life only. Maintenance interval, silvering and performance score are uncalibrated. Correlation (r) stays modest, which is the remaining open problem: the correction fixes the average, not the case-by-case ordering. That needs more field records, not more maths.
By climate zone
Accuracy is not uniform across the planet
| Köppen zone | n | Bias | MAE | RMSE | MAPE | Coverage |
|---|---|---|---|---|---|---|
| A — tropical | 4 | +2.6 | 3.5 | 4.7 | 37.1% | 75% |
| B — arid | 5 | +0.9 | 3.8 | 5.4 | 21.5% | 100% |
| C — temperate | 9 | -0.6 | 12.1 | 13.4 | 41.7% | 88.9% |
| D — continental | 6 | -3.6 | 10.5 | 14 | 27% | 83.3% |
Zones with one or two cases are reported for transparency, not for confidence. Treat any row with n < 4 as anecdote. The tropical and arid rows are where the method most needs field data — if you hold inspection records from those zones, we want them.
By species
Where the species parameters, not the climate, are wrong
| Species | n | Bias | MAE | RMSE | MAPE | Coverage |
|---|---|---|---|---|---|---|
| Thermally Modified Ash | 5 | +15 | 15 | 15.7 | 72.5% | 100% |
| European Oak | 3 | -8.1 | 8.1 | 13 | 15.6% | 100% |
| Siberian Larch | 4 | -6.7 | 8.4 | 11.2 | 30.9% | 75% |
| Norway Spruce | 4 | -6.4 | 7.5 | 10.1 | 33.9% | 50% |
| Acetylated Radiata Pine | 4 | -2.9 | 2.9 | 3.6 | 8.2% | 100% |
| Western Red Cedar | 4 | +0.2 | 7.6 | 8.9 | 23.1% | 100% |
The calibration layer corrects by climate zone only, so any bias left in this table is a species-parameter problem: a resistance value, a modification credit or a coating assumption that does not hold. Rows are sorted by absolute bias, worst first. We publish this before we fix it, because the ordering tells a specifier which of our numbers to distrust today.
Case by case
Every observation, every residual
- val-01 · field trialinside P10–P90
Norway Spruce — Uppsala / central Sweden, above-ground field station
Mapped to Stockholm, Sweden (Dfb — humid continental) · south · sheltered · untreated
- Observed
- 12 yr
- source range 9–16 yr
- Modelled P50
- 13.3 yr
- P10–P90 5–23.5 yr
- Blind (LOO)
- 13.4 yr
- uncalibrated 19.2 yr · zone factor ×0.6906
- Residual
- +1.3 yr
- +10.8% of observed
- Observation ran
- 10 yr
- source extrapolated beyond exposure
Untreated spruce cladding, ventilated cavity, no coating. Failure criterion: onset of decay rating 2 on the EN 252 scale.
Source (institute report): EN 252 / above-ground double-layer field testing, Nordic stations — RISE / Nordic Wood Preservation Council field test summaries for untreated Picea abies, above-ground exposure
- val-02 · in service surveyoutside P10–P90
Siberian Larch — Stockholm archipelago, unpainted larch façade
Mapped to Stockholm, Sweden (Dfb — humid continental) · south · fully exposed · untreated
- Observed
- 42 yr
- source range 35–55 yr
- Modelled P50
- 22.5 yr
- P10–P90 7–40.7 yr
- Blind (LOO)
- 19.6 yr
- uncalibrated 32.4 yr · zone factor ×0.6906
- Residual
- -19.5 yr
- -46.4% of observed
- Observation ran
- 40 yr
- source extrapolated beyond exposure
Heartwood-selected boards, 25 mm cavity. Silvered within 2 years; replacement driven by board cupping, not decay.
Source (institute report): Nordic in-service façade inspections, untreated Larix sibirica — Documented Swedish/Finnish façade inspection surveys of unpainted Siberian larch cladding
- val-03 · in service surveyinside P10–P90
Western Red Cedar — Vancouver region, coastal Pacific North-West
Mapped to Pacific North-West, USA (Csb — warm-summer mediterranean) · north · sheltered · untreated
- Observed
- 45 yr
- source range 30–60 yr
- Modelled P50
- 33.3 yr
- P10–P90 10–60.7 yr
- Blind (LOO)
- 33 yr
- uncalibrated 48.6 yr · zone factor ×0.7014
- Residual
- -11.7 yr
- -26% of observed
- Observation ran
- 45 yr
- full observation period
Vertical grain heartwood. Long life in its native climate; sapwood-rich boards perform markedly worse.
Source (institute report): Western red cedar in-service performance, coastal British Columbia — FPInnovations / regional building-envelope surveys of Thuja plicata cladding
- val-04 · field trialinside P10–P90
Thermally Modified Ash — North-west European maritime coast, thermally modified hardwood cladding
Mapped to Coastal Scotland (Cfb — temperate oceanic) · east-west · fully exposed · untreated
- Observed
- 27 yr
- source range 20–35 yr
- Modelled P50
- 45 yr
- P10–P90 11.2–80 yr
- Blind (LOO)
- 45.7 yr
- uncalibrated 65.2 yr · zone factor ×0.7014
- Residual
- +18 yr
- +66.7% of observed
- Observation ran
- 12 yr
- source extrapolated beyond exposure
Source extrapolates from 12 years of exposure. Extrapolated observations carry more weight of assumption than the model does.
Source (peer reviewed): Thermally modified timber above-ground durability trials — Above-ground field trials of thermally modified hardwoods (Thermowood class D), maritime temperate exposure
- val-05 · in service surveyinside P10–P90
European Oak — Scottish east coast, untreated oak boarding
Mapped to Coastal Scotland (Cfb — temperate oceanic) · south · fully exposed · untreated
- Observed
- 35 yr
- source range 25–50 yr
- Modelled P50
- 33.7 yr
- P10–P90 10.4–59.7 yr
- Blind (LOO)
- 34 yr
- uncalibrated 48.3 yr · zone factor ×0.7014
- Residual
- -1.3 yr
- -3.7% of observed
- Observation ran
- 35 yr
- full observation period
Heartwood only. Tannin run-off staining is a serviceability issue long before decay is.
Source (institute report): European oak cladding, UK maritime in-service records — TRADA / BRE durability guidance and inspection records for Quercus robur external boarding
- val-06 · standardised testinside P10–P90
Acetylated Radiata Pine — Netherlands / UK coastal, acetylated radiata pine
Mapped to Coastal Scotland (Cfb — temperate oceanic) · east-west · fully exposed · untreated
- Observed
- 50 yr
- source range 40–60 yr
- Modelled P50
- 44.1 yr
- P10–P90 13.4–80 yr
- Blind (LOO)
- 45.8 yr
- uncalibrated 67.8 yr · zone factor ×0.7014
- Residual
- -5.9 yr
- -11.8% of observed
- Observation ran
- 20 yr
- source extrapolated beyond exposure
Manufacturer-independent trials confirm class 1 durability; the 50-year figure is a warranty-anchored expectation, not a completed observation.
Source (peer reviewed): Acetylated wood above-ground durability, EN 350 durability class 1 assessment — Field and laboratory durability assessments of acetylated Pinus radiata, above-ground use class 3
- val-07 · service life databaseoutside P10–P90
Norway Spruce — UK maritime, painted spruce cladding with maintained coating
Mapped to Coastal Scotland (Cfb — temperate oceanic) · north · sheltered · maintained coating
- Observed
- 30 yr
- source range 20–40 yr
- Modelled P50
- 13.1 yr
- P10–P90 5–23.9 yr
- Blind (LOO)
- 12 yr
- uncalibrated 19.6 yr · zone factor ×0.7014
- Residual
- -16.9 yr
- -56.3% of observed
- Observation ran
- 30 yr
- full observation period
Conditional on a maintained coating system. Without recoating at interval, observed life collapses toward the untreated case.
Source (database): Component service-life data for maintained painted softwood cladding — ISO 15686-8 factor-method reference service lives for coated softwood external cladding
- val-08 · field trialinside P10–P90
Siberian Larch — Equatorial South-East Asia, temperate softwood in tropical exposure
Mapped to Singapore (Af — tropical rainforest) · south · fully exposed · untreated
- Observed
- 7 yr
- source range 4–11 yr
- Modelled P50
- 9.4 yr
- P10–P90 5–17.1 yr
- Blind (LOO)
- 10.1 yr
- uncalibrated 30 yr · zone factor ×0.3268
- Residual
- +2.4 yr
- +34.3% of observed
- Observation ran
- 6 yr
- source extrapolated beyond exposure
The hardest case for the model. Tropical bio-load dominates and station density is poor — the zone σ multiplier is 1.45 for a reason.
Source (peer reviewed): Tropical above-ground decay hazard trials, imported temperate species — Tropical field trials showing 5–10× decay-rate acceleration versus temperate stations for the same species
- val-09 · in service surveyinside P10–P90
European Oak — Arabian interior, shaded hardwood screening
Mapped to Riyadh, Saudi Arabia (BWh — hot desert) · south · fully exposed · untreated
- Observed
- 18 yr
- source range 12–25 yr
- Modelled P50
- 17.6 yr
- P10–P90 5.7–31.3 yr
- Blind (LOO)
- 18.1 yr
- uncalibrated 41.8 yr · zone factor ×0.441
- Residual
- -0.4 yr
- -2.2% of observed
- Observation ran
- 15 yr
- source extrapolated beyond exposure
Decay is near-absent; failure is UV, checking and thermal movement. The model's decay-led framing under-predicts this failure mode.
Source (institute report): Hot-arid exterior timber inspections, Gulf region — Gulf-region building-envelope inspections of exterior hardwood screens and shading elements
- val-10 · in service surveyinside P10–P90
Acetylated Radiata Pine — Gulf coastal, acetylated pine louvres
Mapped to Dubai, UAE (BWh — hot desert, coastal) · east-west · fully exposed · untreated
- Observed
- 30 yr
- source range 22–40 yr
- Modelled P50
- 26.3 yr
- P10–P90 8.1–47.8 yr
- Blind (LOO)
- 25.3 yr
- uncalibrated 61.2 yr · zone factor ×0.441
- Residual
- -3.7 yr
- -12.3% of observed
- Observation ran
- 12 yr
- source extrapolated beyond exposure
Stainless fixings throughout. Where galvanised fixings were used, elements failed at the fastener 10–15 years earlier.
Source (institute report): Coastal Gulf in-service inspections of modified wood elements — Documented inspections of acetylated softwood louvres and decking in chloride-laden coastal exposure
- val-11 · in service surveyinside P10–P90
Western Red Cedar — Southern Scandinavia, imported cedar cladding
Mapped to Stockholm, Sweden (Dfb — humid continental) · south · sheltered · untreated
- Observed
- 33 yr
- source range 25–45 yr
- Modelled P50
- 35.7 yr
- P10–P90 11.1–65.5 yr
- Blind (LOO)
- 35.4 yr
- uncalibrated 52.1 yr · zone factor ×0.6906
- Residual
- +2.7 yr
- +8.2% of observed
- Observation ran
- 30 yr
- source extrapolated beyond exposure
Performs below its native-climate life; extractive leaching under higher driving rain shortens the plateau.
Source (institute report): Imported Thuja plicata cladding, Scandinavian inspection records — Scandinavian façade inspections of imported western red cedar, ventilated rainscreen build-ups
- val-12 · field trialinside P10–P90
Thermally Modified Ash — Coastal Oregon, modified hardwood rainscreen
Mapped to Pacific North-West, USA (Csb — warm-summer mediterranean) · east-west · fully exposed · untreated
- Observed
- 24 yr
- source range 18–32 yr
- Modelled P50
- 44.7 yr
- P10–P90 13–80 yr
- Blind (LOO)
- 50.1 yr
- uncalibrated 68.1 yr · zone factor ×0.7014
- Residual
- +20.7 yr
- +86.3% of observed
- Observation ran
- 10 yr
- source extrapolated beyond exposure
Modification reduces decay but not surface checking; the model tracks the first and only partly the second.
Source (peer reviewed): North American above-ground trials of thermally modified hardwood — Above-ground stake and cladding trials of thermally modified hardwoods, wet-temperate exposure
- val-13 · field trialoutside P10–P90
Norway Spruce — Equatorial South-East Asia, untreated temperate softwood
Mapped to Singapore (Af — tropical rainforest) · north · sheltered · untreated
- Observed
- 4 yr
- source range 2–6 yr
- Modelled P50
- 5 yr
- P10–P90 5–8.3 yr
- Blind (LOO)
- 5 yr
- uncalibrated 14.1 yr · zone factor ×0.3268
- Residual
- +1 yr
- +25% of observed
- Observation ran
- 5 yr
- full observation period
Termite attack co-occurs with fungal decay; the model treats bio-load as a single term and cannot separate the two.
Source (peer reviewed): Tropical above-ground decay hazard trials, non-durable softwoods — Above-ground tropical field trials of non-durable softwood, EN 252 / use class 3 equivalents
- val-14 · in service surveyinside P10–P90
Acetylated Radiata Pine — Singapore, acetylated pine louvres and screens
Mapped to Singapore (Af — tropical rainforest) · east-west · fully exposed · untreated
- Observed
- 22 yr
- source range 15–30 yr
- Modelled P50
- 20.2 yr
- P10–P90 6.1–36.4 yr
- Blind (LOO)
- 20.8 yr
- uncalibrated 63.7 yr · zone factor ×0.3268
- Residual
- -1.8 yr
- -8.2% of observed
- Observation ran
- 11 yr
- source extrapolated beyond exposure
Still short of the observation window it claims; the upper bound is a projection from 11 years of service.
Source (institute report): Tropical in-service inspections of acetylated softwood — Documented South-East Asian inspections of acetylated Pinus radiata external elements
- val-15 · field trialinside P10–P90
Thermally Modified Ash — Tropical rainscreen, thermally modified hardwood
Mapped to Singapore (Af — tropical rainforest) · south · fully exposed · untreated
- Observed
- 11 yr
- source range 7–16 yr
- Modelled P50
- 19.9 yr
- P10–P90 5.6–35.3 yr
- Blind (LOO)
- 22.9 yr
- uncalibrated 61.8 yr · zone factor ×0.3268
- Residual
- +8.9 yr
- +80.9% of observed
- Observation ran
- 8 yr
- source extrapolated beyond exposure
Thermal modification loses much of its relative advantage where wetting is near-continuous.
Source (peer reviewed): Thermally modified hardwood in tropical above-ground exposure — Comparative above-ground trials of thermally modified hardwoods, humid tropical stations
- val-16 · in service surveyinside P10–P90
Western Red Cedar — Arabian interior, cedar soffits under deep shading
Mapped to Riyadh, Saudi Arabia (BWh — hot desert) · north · sheltered · untreated
- Observed
- 26 yr
- source range 18–35 yr
- Modelled P50
- 22.8 yr
- P10–P90 5.6–40.2 yr
- Blind (LOO)
- 22.4 yr
- uncalibrated 52.3 yr · zone factor ×0.441
- Residual
- -3.2 yr
- -12.3% of observed
- Observation ran
- 20 yr
- source extrapolated beyond exposure
Shading removes the dominant UV term; remaining failure is dimensional movement at fixings.
Source (institute report): Hot-arid in-service inspections of softwood soffits — Gulf-region envelope inspections of shaded softwood soffits and ceilings
- val-17 · in service surveyinside P10–P90
Thermally Modified Ash — Gulf coastal, modified hardwood screens
Mapped to Dubai, UAE (BWh — hot desert, coastal) · south · fully exposed · untreated
- Observed
- 15 yr
- source range 10–22 yr
- Modelled P50
- 26 yr
- P10–P90 7.8–46.9 yr
- Blind (LOO)
- 27.6 yr
- uncalibrated 58.5 yr · zone factor ×0.441
- Residual
- +11 yr
- +73.3% of observed
- Observation ran
- 12 yr
- source extrapolated beyond exposure
Abrasive dust accelerates surface loss; the model has no abrasion term and over-predicts here.
Source (institute report): Coastal Gulf inspections of thermally modified hardwood — Inspection records of thermally modified hardwood screens in chloride and dust-laden exposure
- val-18 · in service surveyinside P10–P90
Siberian Larch — Gulf coastal, untreated larch decking and screens
Mapped to Dubai, UAE (BWh — hot desert, coastal) · east-west · fully exposed · untreated
- Observed
- 12 yr
- source range 8–18 yr
- Modelled P50
- 12.9 yr
- P10–P90 5–23.4 yr
- Blind (LOO)
- 12.7 yr
- uncalibrated 29.8 yr · zone factor ×0.441
- Residual
- +0.9 yr
- +7.5% of observed
- Observation ran
- 12 yr
- full observation period
Failure is checking and cupping under thermal cycling long before any decay is measurable.
Source (institute report): Temperate softwood in hot-arid coastal service — Gulf-region inspections of imported Larix sibirica external elements
- val-19 · in service surveyinside P10–P90
European Oak — Central Sweden, untreated oak boarding and structure
Mapped to Stockholm, Sweden (Dfb — humid continental) · east-west · fully exposed · untreated
- Observed
- 55 yr
- source range 40–80 yr
- Modelled P50
- 32.5 yr
- P10–P90 9.9–58.6 yr
- Blind (LOO)
- 30.6 yr
- uncalibrated 48.5 yr · zone factor ×0.6906
- Residual
- -22.5 yr
- -40.9% of observed
- Observation ran
- 55 yr
- full observation period
Heartwood only, with ventilated backs and no ground contact. Sapwood-bearing boards fail decades earlier.
Source (institute report): Nordic in-service records for Quercus robur heartwood exterior — Swedish inspection records of untreated oak boarding and exposed oak structure
- val-20 · service life databaseinside P10–P90
Norway Spruce — Coastal Oregon / Washington, painted spruce cladding
Mapped to Pacific North-West, USA (Csb — warm-summer mediterranean) · north · sheltered · maintained coating
- Observed
- 25 yr
- source range 18–35 yr
- Modelled P50
- 14.1 yr
- P10–P90 5–25.5 yr
- Blind (LOO)
- 12.9 yr
- uncalibrated 19.9 yr · zone factor ×0.7014
- Residual
- -10.9 yr
- -43.6% of observed
- Observation ran
- 25 yr
- full observation period
Conditional on recoating at interval; the database value assumes maintenance is actually carried out.
Source (database): Component service-life data for coated softwood in wet-temperate exposure — ISO 15686-8 factor-method reference service lives, coated softwood cladding, high driving-rain regions
- val-21 · in service surveyinside P10–P90
Siberian Larch — Scottish west coast, unpainted larch rainscreen
Mapped to Coastal Scotland (Cfb — temperate oceanic) · north · fully exposed · untreated
- Observed
- 30 yr
- source range 22–40 yr
- Modelled P50
- 19.4 yr
- P10–P90 7.1–35.4 yr
- Blind (LOO)
- 19.4 yr
- uncalibrated 29 yr · zone factor ×0.7014
- Residual
- -10.6 yr
- -35.3% of observed
- Observation ran
- 28 yr
- source extrapolated beyond exposure
Algal colonisation within two years on north elevations; a serviceability, not a structural, limit.
Source (institute report): UK maritime in-service inspections of untreated larch — TRADA / UK envelope inspection records for Larix cladding in high driving-rain exposure
- val-22 · in service surveyinside P10–P90
Western Red Cedar — UK maritime, imported cedar cladding with translucent coating
Mapped to Coastal Scotland (Cfb — temperate oceanic) · south · fully exposed · maintained coating
- Observed
- 28 yr
- source range 20–38 yr
- Modelled P50
- 40.8 yr
- P10–P90 11.9–71.9 yr
- Blind (LOO)
- 43.5 yr
- uncalibrated 59.5 yr · zone factor ×0.7014
- Residual
- +12.8 yr
- +45.7% of observed
- Observation ran
- 26 yr
- source extrapolated beyond exposure
Coating failure at 6–9 year intervals drives the maintenance cost long before the substrate is at risk.
Source (institute report): Coated Thuja plicata cladding, UK maritime inspection records — UK inspection records of translucent-coated western red cedar rainscreens
- val-23 · service life databaseinside P10–P90
Acetylated Radiata Pine — Nordic continental, acetylated pine windows and cladding
Mapped to Stockholm, Sweden (Dfb — humid continental) · south · fully exposed · untreated
- Observed
- 45 yr
- source range 35–60 yr
- Modelled P50
- 44.8 yr
- P10–P90 13.9–80 yr
- Blind (LOO)
- 46.3 yr
- uncalibrated 66.3 yr · zone factor ×0.6906
- Residual
- -0.2 yr
- -0.4% of observed
- Observation ran
- 18 yr
- source extrapolated beyond exposure
Warranty-anchored again. 18 years of real service; the remainder is projection and is flagged as such.
Source (database): Nordic service-life records for acetylated softwood joinery — Component service-life records for acetylated Pinus radiata joinery and cladding, continental climate
- val-24 · field trialinside P10–P90
Thermally Modified Ash — Central Sweden, thermally modified hardwood cladding
Mapped to Stockholm, Sweden (Dfb — humid continental) · north · sheltered · untreated
- Observed
- 30 yr
- source range 22–40 yr
- Modelled P50
- 46.6 yr
- P10–P90 16.1–80 yr
- Blind (LOO)
- 48.4 yr
- uncalibrated 68.5 yr · zone factor ×0.6906
- Residual
- +16.6 yr
- +55.3% of observed
- Observation ran
- 14 yr
- source extrapolated beyond exposure
Sheltered north exposure is the modification's best case; do not read it as a general figure.
Source (peer reviewed): Nordic above-ground trials of thermally modified hardwood — Scandinavian above-ground field trials of Thermowood class D hardwood cladding
Limits
What this benchmark cannot tell you
- The sample is small and skewed to northern Europe. Aggregate metrics are indicative only.
- Observations are digitised from published tables and figures; where a range was given, the midpoint is used.
- Site climate is mapped to the nearest Köppen-equivalent reference climate, not the exact station. That mapping is part of the reported error.
- Several sources extrapolate beyond their own exposure period. Those cases test the source's assumptions as much as the model's.
- Failure criteria differ between sources — decay onset, serviceability, and replacement are not the same event.
- The model is decay-led. In hot-arid zones the real failure mode is UV, checking and movement, which it under-weights.
The benchmark is versioned separately from the method so that accuracy can be tracked as both change. Corrections, additional field records and challenges to any case are welcome and will be credited in the next release.