feat(smoking): Add smoking variable harmonization (CEP-002) #163

DougManuel · 2026-01-22T15:20:31Z

Summary

Adds smoking variable harmonization for CCHS cycles 2001-2023, extending coverage to Master files and introducing unified derived variable functions using a new 3-step architecture.

Variables added (19 new)

Unified feeders: cigs_per_day, SMK_202, SMK_203, SMK_207
Cessation: SMK_09A, SMK_10_gate, SMK_10A_A, SMK_10A_B, SMK_10A_cont, quit_pathway
Status: SMKDSTY, SMKDGSTP, SMKDGSTP_cont, SMKDVSTP
Initiation: SMK_01C, SMK_040, SMK_06C, SMK_09C, SMKG09C_cont

Variables updated (34 existing)

Extended cycle coverage to 2019-2023 PUMF and Master
Added Master file support (2001-2023)
Improved variableStart mappings for era-specific source variable names
Updated: pack_years_cat, pack_years_der, time_quit_smoking, SMKDSTY_A, SMKDSTY_B, SMKDSTY_cat3, SMKDSTY_cat5, SMKG040, SMKG040_cont, SMKG203_A/B/cont, SMKG207_A/B/cont, SMK_005, SMK_01A, SMK_030, SMK_05B, SMK_05C, SMK_05D, SMK_06A_A/B/cont, SMK_09A_A/B/cont, SMK_204, SMK_208, SMKG01C_A/B/cont, SMKG06C, SMKG09C

Cycle coverage

File type	Cycles
PUMF	2001-2023
Master	2001-2023

3-step derived variable architecture

New modular pattern for derived variables:

Clean inputs - clean_variables() preprocesses and validates inputs
Domain logic - Status-based calculations with clear routing
Clean output - Validate output bounds and apply missing codes

Supporting infrastructure added:

R/clean-variables.R, R/missing-data-functions.R, R/missing-pattern-cache.R
R/worksheet-getters.R, R/worksheet-loaders.R, R/variable-discovery.R

Tests added

Test file	Coverage
`test-age_start_smoking.R`	Initiation age routing and bounds
`test-cigs_per_day.R`	Intensity routing by smoking status
`test-pack_years.R`	Cumulative exposure calculation
`test-time_quit_smoking.R`	Cessation timing validation

Documentation

Full specification: CEP-002 Smoking Variables

Source QMD files included in ceps/cep-002-smoking/

Test plan

Review worksheet diffs (variables.csv, variable_details.csv)
Run R CMD check
Run smoking unit tests: testthat::test_file("tests/testthat/test-pack_years.R")
Verify no regressions in existing tests

Foundation for modular derived variable calculations: - clean_variables(): Step 1 & 3 preprocessing/validation - missing-data-functions.R: any_missing(), get_priority_missing() - missing-pattern-cache.R: Pattern detection for PUMF/Master codes - parse-range-notation.R: Range parsing for validation bounds - worksheet-getters.R: get_variable_details() metadata access - worksheet-loaders.R: load_worksheet_metadata() - file-sourcing.R: source_r_robust() for dependency loading - variable-discovery.R: Variable lookup utilities This infrastructure supports the new 3-step pattern: 1. clean_variables() - preprocess inputs 2. Domain logic - status-based calculations 3. clean_variables() - validate output bounds

Primary recommended variables using 3-step architecture: - smoking-status.R: calculate_SMKDSTY_A(), calculate_SMKDSTY_cat6() - smoke-start.R: calculate_age_start_smoking() - unified initiation age - smoking-cessation.R: calculate_time_quit_smoking() - years since quit - smoke-intensity.R: calculate_cigs_per_day() - routes SMK_204/SMK_208 - smoke-pack-years.R: calculate_pack_years() - cumulative exposure - smoke-stop.R: Supporting cessation logic - smoking-validation-constants.R: PACK_YEARS_CONSTANTS Key design decisions: - Single calculate_pack_years() works for both PUMF and Master - Unified feeders (age_start_smoking, cigs_per_day, time_quit_smoking) handle PUMF vs Master routing internally - PUMF has ~15-20% relative error due to midpoint estimation - Era-agnostic: handles 2001-2023 variable naming variations See ceps/cep-002-smoking/ for full specification.

Worksheets for smoking variable harmonization: - smoking_variables.csv: Variable definitions for smoking domain - smoking_variable_details.csv: Recoding rules and mappings Covers 5 subgroups: - 01-status: SMKDSTY_cat6 (6-category smoking status) - 02-initiation: age_start_smoking (age started daily) - 03-cessation: time_quit_smoking (years since quit) - 04-intensity: cigs_per_day (cigarettes per day) - 05-pack-years: pack_years_der (cumulative exposure) Supports all PUMF cycles 2001-2022.

Tests for primary recommended smoking variables: - test-age_start_smoking.R: Initiation age routing and bounds - test-cigs_per_day.R: Intensity routing by smoking status - test-pack_years.R: Cumulative exposure calculation - test-time_quit_smoking.R: Cessation timing validation Tests verify: - Correct routing based on SMKDSTY_A status - Valid output ranges per variable_details.csv bounds - Missing value handling (tagged_na vs numeric) - Universe validation (correct NA for out-of-scope respondents)

Quarto documentation for smoking variable harmonization: Main documents: - cep-002-smoking.qmd: Methodology and rationale - 00-variable-summary.qmd: Variable overview and recommendations - derived-functions.qmd: DV function specifications Subgroup specifications (QMD + worksheet CSVs): - 01-status: SMKDSTY_cat6 (6-category smoking status) - 02-initiation: age_start_smoking - 03-cessation: time_quit_smoking - 04-intensity: cigs_per_day - 05-pack-years: pack_years_der Rendered site: https://dmanuel.quarto.pub/cep-002-smoking-variables

Updates variables.csv and variable_details.csv with smoking harmonization: - 34 existing smoking variables updated with v3 definitions - 19 new smoking variables added - Extends coverage to 2022-2023 Master files - Adds unified feeder variables (age_start_smoking, cigs_per_day, etc.) Removes separate smoking_*.csv files (now merged into main worksheets). Variables: 360 → 379 (+19 new) Variable details: 3468 → 3678 (+210 net, replacing 500 with 710 improved rows)

Copilot

Pull request overview

This PR adds comprehensive smoking variable harmonization for CCHS cycles 2001-2023, implementing CEP-002. It introduces 19 new variables and updates 34 existing ones, extending coverage to Master files and establishing a new 3-step derived variable architecture.

Changes:

New smoking variables across 5 subgroups (status, initiation, cessation, intensity, pack-years)
Extended cycle coverage to PUMF 2001-2023 and Master 2001-2023
New unified derived variable functions with standardized architecture
Comprehensive test suite for derived variables
Full CEP documentation in Quarto format

Reviewed changes

Copilot reviewed 38 out of 42 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/testthat/test-time_quit_smoking.R	Unit tests for cessation timing calculation (337 lines)
tests/testthat/test-pack_years.R	Unit tests for pack-years derivation (386 lines)
tests/testthat/test-cigs_per_day.R	Unit tests for cigarettes per day routing (632 lines)
tests/testthat/test-age_start_smoking.R	Unit tests for initiation age calculation (353 lines)
ceps/cep-002-smoking/derived-functions.qmd	Function reference documentation (448 lines)
ceps/cep-002-smoking/cep-002-smoking.qmd	Main methodology and rationale (598 lines)
ceps/cep-002-smoking/05-pack-years.qmd	Pack-years variable documentation (276 lines)
ceps/cep-002-smoking/04-intensity.qmd	Intensity variable documentation (497 lines)
ceps/cep-002-smoking/03-cessation.qmd	Cessation variable documentation (555 lines)
ceps/cep-002-smoking/02-initiation.qmd	Initiation variable documentation (808 lines)
ceps/cep-002-smoking/00-variable-summary.qmd	Variable summary table (266 lines)
CSV worksheets (multiple)	Variable definitions and recoding rules
R/smoking-validation-constants.R	Validation constants for smoking functions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rafdoodle · 2026-01-24T03:52:22Z

inst/extdata/variables.csv

+"REP_5A","Repetitive strain injury - walking","Repetitive strain injury - type of activity - walking","Categorical","cchs2009_2010_p, cchs2010_p, cchs2011_2012_p, cchs2012_p, cchs2013_2014_p, cchs2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2009_s, cchs2010_s, cchs2012_s","cchs2015_2016_p::INJ_020A, cchs2017_2018_p::INJ_020A, [REP_5A]","Repetitive strain injury","Health status","N/A",NA,"","2.2.0","2025-06-30","Variable metadata completed","",NA,"active",NA
+"REP_5B","Repetitive strain injury - sport/physical activity","Repetitive strain injury - type of activity - sports or physical exercise","Categorical","cchs2001_p, cchs2003_p, cchs2005_p, cchs2007_2008_p, cchs2009_2010_p, cchs2010_p, cchs2011_2012_p, cchs2012_p, cchs2013_2014_p, cchs2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2009_s, cchs2010_s, cchs2012_s","cchs2001_p::REPA_4A, cchs2003_p::REPC_4A, cchs2005_p::REPE_4A, cchs2007_2008_p::REP_4A, cchs2015_2016_p::INJ_020B, cchs2017_2018_p::INJ_020B, [REP_5B]","Repetitive strain injury","Health status","N/A",NA,"","2.2.0","2025-06-30","Variable metadata completed","",NA,"active",NA
+"REP_5C","Repetitive strain injury - leisure/hobby","Repetitive strain injury - type of activity - leisure or hobby","Categorical","cchs2001_p, cchs2003_p, cchs2005_p, cchs2007_2008_p, cchs2009_2010_p, cchs2010_p, cchs2011_2012_p, cchs2012_p, cchs2013_2014_p, cchs2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2009_s, cchs2010_s, cchs2012_s","cchs2001_p::REPA_4B, cchs2003_p::REPC_4B, cchs2005_p::REPE_4B, cchs2007_2008_p::REP_4B, cchs2015_2016_p::INJ_020C, cchs2017_2018_p::INJ_020C, [REP_5C]","Repetitive strain injury","Health status","N/A",NA,"","2.2.0","2025-06-30","Variable metadata completed","",NA,"active",NA
+"REP_5D","Repetitive strain injury -


SMK_09C (Years since stopped smoking daily - former daily) available as SMK_090 for 2015-2016 and 2017-2018

Thanks for catching this.

Issue confirmed: The 2015 CCHS redesign renamed the cessation timing variables:

SMK_09C → SMK_090 (years since stopped smoking daily)

Also fixed:
SMK_06C → SMK_070 (years since stopped smoking - former occasional)

The worksheets were using [SMK_09C] and [SMK_06C] bracket fallback notation, which only works when the source variable name matches. For 2015+ cycles, this would fail silently because the variables are named SMK_090 and SMK_070.

Fix applied: Added explicit mappings for 2015-2021 cycles to both variables in variables.csv:

SMK_09C: cchs2015_2016_m::SMK_090, cchs2017_2018_m::SMK_090, cchs2019_2020_m::SMK_090, cchs2021_m::SMK_090
SMK_06C: cchs2015_2016_m::SMK_070, cchs2017_2018_m::SMK_070, cchs2019_2020_m::SMK_070, cchs2021_m::SMK_070

Validation: The fixes have been verified against the source worksheet structure. The [SMK_09C] and [SMK_06C] fallback now only applies to 2007-2014 cycles where those names are correct.

Prevention: Updated the skill documentation with the 2015 cessation variable rename patterns and added a mandatory validation step to catch this class of error in future worksheet authoring.

The 2015 CCHS redesign renamed cessation timing variables: - SMK_09C -> SMK_090 (years since stopped daily) - SMK_06C -> SMK_070 (years since stopped occasional) The [SMK_09C] and [SMK_06C] bracket fallback doesn't work for 2015+ cycles where the variables have different names. Added explicit mappings for cchs2015_2016_m through cchs2021_m. Fixes issue identified in PR #163 review.

Rewrote CSV files with proper quoting using readr::write_csv() to fix "excessive quoting" errors in the CSV formatting check.

rafdoodle · 2026-01-28T18:50:00Z

ceps/cep-002-smoking/03-cessation/variables_smk_cessation.csv

+"SMK_10A_cont","Yrs quit (reducer)","Years since quit completely (former daily who continued occasional, continuous)","Continuous","cchs2003_p, cchs2005_p, cchs2007_2008_p, cchs2009_2010_p, cchs2011_2012_p, cchs2013_2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2003_m, cchs2005_m, cchs2007_2008_m, cchs2009_2010_m, cchs2011_2012_m, cchs2013_2014_m, cchs2015_2016_m, cchs2017_2018_m, cchs2019_2020_m, cchs2021_m, cchs2023_m","cchs2003_p::SMKC_10A, cchs2003_m::SMKC_10A, cchs2005_p::SMKE_10A, cchs2005_m::SMKE_10A, cchs2015_2016_p::SMK_100, cchs2015_2016_m::SMK_100, cchs2017_2018_p::SMK_100, cchs2017_2018_m::SMK_100, cchs2019_2020_m::SMK_100, cchs2021_m::SMK_100, cchs2023_m::SPU_35, [SMK_10A]","smoking","Health behaviour","years","Universe: former daily who continued occasional. Converts categorical SMK_10A/SPU_35 to pseudo-continuous using midpoint. Not available 2001, 2022 (no categorical - uses SPU_35A/B month/year). Use SMKDVSTP for 2022.","Pseudo-continuous years since quit completely for former daily who reduced to occasional. Derived from categorical using midpoint conversion.","3.0.0-alpha","2026-01-11","Removed cchs2022_m - 2022 uses month/year (SPU_35A/B) not categorical. Added cchs2023_m::SPU_35.",,,"active"
+"SMK_10_gate","Quit gate","Quit completely when stopped daily (gate variable)","Categorical","cchs2003_p, cchs2005_p, cchs2007_2008_p, cchs2009_2010_p, cchs2011_2012_p, cchs2013_2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2003_m, cchs2005_m, cchs2007_2008_m, cchs2009_2010_m, cchs2011_2012_m, cchs2013_2014_m, cchs2015_2016_m, cchs2017_2018_m, cchs2019_2020_m, cchs2021_m, cchs2022_m, cchs2023_m","cchs2003_p::[SMKC_10], cchs2005_p::[SMKE_10], cchs2007_2008_p::[SMK_10], cchs2009_2010_p::[SMK_10], cchs2011_2012_p::[SMK_10], cchs2013_2014_p::[SMK_10], cchs2015_2016_p::[SMK_095], cchs2017_2018_p::[SMK_095], cchs2003_m::[SMKC_10], cchs2005_m::[SMKE_10], cchs2007_2008_m::[SMK_10], cchs2009_2010_m::[SMK_10], cchs2011_2012_m::[SMK_10], cchs2013_2014_m::[SMK_10], cchs2015_2016_m::[SMK_095], cchs2017_2018_m::[SMK_095], cchs2019_2020_m::[SMK_095], cchs2021_m::[SMK_095], cchs2022_m::[SPU_30], cchs2023_m::[SPU_30]","smoking","Health behaviour","N/A","Universe: former daily smokers. Gate for quit pathway: 1=quit when stopped daily, 2=continued occasional. Not available 2001.","Determines if respondent quit completely when they stopped daily smoking or continued occasional smoking.","3.0.0-alpha","2026-01-04","Labels updated 2026-01-04. Universe context added.",,,"active"
+"quit_pathway","Quit pathway","Smoking cessation pathway (direct, gradual, or former occasional)","Categorical","cchs2003_p, cchs2005_p, cchs2007_2008_p, cchs2009_2010_p, cchs2011_2012_p, cchs2013_2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2003_m, cchs2005_m, cchs2007_2008_m, cchs2009_2010_m, cchs2011_2012_m, cchs2013_2014_m, cchs2015_2016_m, cchs2017_2018_m, cchs2019_2020_m, cchs2021_m, cchs2022_m, cchs2023_m","DerivedVar::[SMKDSTY_cat5, SMK_10_gate]","smoking","Health behaviour","N/A","Derived from SMKDSTY_cat5 and SMK_10_gate. Not available 2001 (no SMK_10_gate). {recommended:secondary}","Categorical indicator: 1=direct quit (quit when stopped daily), 2=gradual quit (continued occasional then quit), 3=former occasional (never daily)","3.0.0-alpha","2026-01-04","Labels updated 2026-01-04",,,"active"
+"time_quit_smoking","Yrs quit smoking*","Years since quit smoking","Continuous","cchs2001_p, cchs2003_p, cchs2005_p, cchs2007_2008_p, cchs2009_2010_p, cchs2011_2012_p, cchs2013_2014_p, cchs2015_2016_p, cchs2017_2018_p, cchs2019_2020_p, cchs2001_m, cchs2003_m, cchs2005_m, cchs2007_2008_m, cchs2009_2010_m, cchs2011_2012_m, cchs2013_2014_m, cchs2015_2016_m, cchs2017_2018_m, cchs2019_2020_m, cchs2021_m, cchs2022_m, cchs2023_m","DerivedVar::[SMK_09A_cont, SMK_06A_cont, SMKDVSTP]","smoking","Health behaviour","years","{recommended:primary} {sub_subject:cessation} Universe: all former smokers. Routes: PUMF uses SMK_09A_cont/SMK_06A_cont (midpoint-converted); Master 2003+ uses SMKDVSTP (StatCan derived continuous).","Unified continuous years since quit smoking. PUMF: midpoint from categorical; Master: SMKDVSTP pass-through.","3.0.0-alpha","2026-01-11","Fixed: Changed SPU_25 to SMKDVSTP for Master 2022/2023 (SPU_25 is categorical, SMKDVSTP is continuous).",,,"active"


time_quit_smoking only exists for PUMF data at the moment

DougManuel added 6 commits January 22, 2026 10:17

Copilot AI review requested due to automatic review settings January 22, 2026 15:20

Copilot started reviewing on behalf of DougManuel January 22, 2026 15:20 View session

Copilot AI reviewed Jan 22, 2026

View reviewed changes

DougManuel assigned yulric, mdiaspar, rafdoodle, caitlink12 and karimhalal Jan 22, 2026

rafdoodle reviewed Jan 24, 2026

View reviewed changes

This was referenced Jan 27, 2026

Smoking variable updates #152

Closed

v3.0.0 infrastructure with CCHS master file harmonization #137

Closed

Fix CSV formatting for GitHub Actions

62fa609

Rewrote CSV files with proper quoting using readr::write_csv() to fix "excessive quoting" errors in the CSV formatting check.

rafdoodle reviewed Jan 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(smoking): Add smoking variable harmonization (CEP-002) #163

feat(smoking): Add smoking variable harmonization (CEP-002) #163

DougManuel commented Jan 22, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

rafdoodle Jan 24, 2026 •

edited

Loading

Uh oh!

DougManuel Jan 26, 2026

Uh oh!

rafdoodle Jan 28, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

feat(smoking): Add smoking variable harmonization (CEP-002) #163

Are you sure you want to change the base?

feat(smoking): Add smoking variable harmonization (CEP-002) #163

Conversation

DougManuel commented Jan 22, 2026

Summary

Variables added (19 new)

Variables updated (34 existing)

Cycle coverage

3-step derived variable architecture

Tests added

Documentation

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

rafdoodle Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DougManuel Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

rafdoodle Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

rafdoodle Jan 24, 2026 •

edited

Loading

rafdoodle Jan 28, 2026 •

edited

Loading