Issue125 decompile by ablack3 · Pull Request #126 · OHDSI/Capr

ablack3 · 2026-02-09T02:28:32Z

I just saw you were working on coercing Atlas json into Capr objects. If this PR works it might get us there too. I'm trying to implement a jsonToCapr function that takes as input an Atlas json and returns Capr R code. I'm working through the OHDSI phenotype library and checking that the capr code that is produced matches the original json and the SQL generated by both the original and new json files is equivalent.

I had to add a few missing attribute constructors but overall I'm making good progress.

library(Capr)
jsonFolder <- system.file("cohorts", package = "PhenotypeLibrary", mustWork = TRUE)

jsonFiles <- list.files(jsonFolder, pattern = "\\.json$", full.names = TRUE)

cat(jsonToCapr(jsonFiles[[1]]))
#> # Generated by jsonToCapr()
#> # Source JSON: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/library/PhenotypeLibrary/cohorts/10.json
#> 
#> library(Capr)
#> 
#> # --- concept sets ---
#> csNauseaOrVomiting3 <- cs(descendants(31967, 4158495, 4101344, 4012500, 201218), name = "Nausea or Vomiting", id = 3)
#> 
#> attritionObj <- attrition(
#>   expressionLimit = "All"
#> )
#> 
#> # --- cohort ---
#> cohortDef <- cohort(
#>   entry = entry(
#>     conditionOccurrence(csNauseaOrVomiting3, conditionTypeExclude(FALSE)),
#>     observation(csNauseaOrVomiting3),
#>     observationWindow = continuousObservation(priorDays = 0L, postDays = 0L),
#>     primaryCriteriaLimit = "All",
#>     qualifiedLimit = "First"
#>   ),
#>   attrition = attritionObj,
#>   exit = exit(
#>     endStrategy = fixedExit(index = "endDate", offsetDays = 14L)
#>   ),
#>   era = era(eraDays = 0L)
#> )

^{Created on 2026-02-09 with reprex v2.1.1}

I added a helper insertCaprCode(jsonFiles[[1]]) that will use rstudioapi to insert the code in your current R file for interactive use.

So far many of the phenotype library json are passing the "roundtrip" test but not all. Still working through them. One thing I'd like your input on is if we could avoid requiring a connection to a vocabulary to create the concept attributes. I think we could get what we need from the json without going to the vocabulary but let me know what you think. The relevant line is here:

Capr/R/attributes-concept.R

Line 119 in 7d18310

    
           buildConceptAttribute <- function(ids, attributeName, connection, vocabularyDatabaseSchema) {

#125

Copilot

Pull request overview

Adds a decompiler path to Capr so Atlas/Circe cohort JSON can be converted back into equivalent Capr R code, and introduces regression tests/fixtures to validate JSON ↔ Capr round-trips (including SQL equivalence checks via CirceR when available).

Changes:

Add jsonToCapr() / jsonToCaprFile() to decompile Circe cohort JSON into Capr code (concept sets + cohort() construction).
Extend query/attribute support to better preserve Circe semantics (e.g., conditionTypeExclude(FALSE), omit CodesetId when concept set is empty).
Add testthat round-trip + feature tests and minimal JSON fixtures; add PhenotypeLibrary as a suggested dependency for broader regression coverage.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`R/jsonToCapr.R`	New decompiler implementation for Circe JSON → Capr code.
`R/query.R`	Adds `doseEra()` and changes Query JSON serialization to omit `CodesetId` for empty concept sets.
`R/attributes-logic.R`	Adds a key-value attribute class and `conditionTypeExclude()` constructor.
`tests/testthat/test-jsonToCapr.R`	Adds round-trip + SQL/semantic equivalence tests and feature-specific assertions.
`tests/testthat/resources/*.json`	Adds minimal Circe JSON fixtures for targeted decompile behaviors.
`NAMESPACE`	Exports newly added public functions/constructors.
`DESCRIPTION`	Updates roxygen note and adds `PhenotypeLibrary` + remote for suggested regression testing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/testthat/test-jsonToCapr.R

R/jsonToCapr.R

Copilot · 2026-02-09T02:39:06Z

NAMESPACE

 # Generated by roxygen2: do not edit by hand

-S3method(compile,Cohort)
 export(age)
 export(as.json)
 export(atLeast)


NAMESPACE no longer registers S3method(compile, Cohort) (it was removed in this PR), but the codebase still imports generics::compile and defines compile.Cohort with @exportS3Method compile Cohort in R/cohort.R. Without the S3method registration, generics::compile() won’t dispatch to compile.Cohort. Either restore the S3method registration (or adjust roxygen so it’s generated) or remove the S3method-oriented import/docs if S4-only dispatch is intended.

remove the S3 generic. Stick to S4 only.

…support "At least 2 events" (any two rows), and "At least 2 distinct values of some column" (e.g. 2 distinct drug concepts) in Circe/Atlas.

…ttributes like "ConditionTypeExclude" : false. These are almost always false in practice and might be from legacy Circe functionality. Nevertheless it helps to have these so we can match the Circe json output exactly.

…s with comparing Circe json to Capr generated json because the order matches.

…ject extra concept sets into the CIRCE JSON produced by compile.Cohort, even if the cohort definition doesn’t reference them. This occurs in Atlas json and is necessary to regenerate the same json/sql as the input.

…ws on a list of dataframes.

ablack3 · 2026-02-09T04:30:32Z

I think it's working pretty well now. In the thatthat test I picked 5 random phenotype libraries to run the roundtrip test on. But almost all of the phenotype library cohorts now pass the roundtrip test for both json and sql. I still need to review the copilot comments. Ready for review.

Copilot

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-09T04:38:04Z

R/jsonToCapr.R

+#' Measurement/DrugExposure attributes, exit strategies (\code{fixedExit},
+#' \code{drugExit}, \code{observationExit}), censoring criteria, and era collapse.
+#'
+#' @section Unsupported (fail in strict mode or skip with \code{mode = "skip"}):
+#' Domains: Specimen, VisitDetail, DoseEra; VisitOccurrence.ProviderSpecialty;


The roxygen docs list DoseEra as unsupported, but domainKeyToQueryFun() maps DoseEra to doseEra() and doseEra() is exported. Update the Supported/Unsupported sections so they match the actual implementation (or remove the mapping if it’s truly unsupported).

Suggested change

#' Measurement/DrugExposure attributes, exit strategies (\code{fixedExit},

#' \code{drugExit}, \code{observationExit}), censoring criteria, and era collapse.

#'

#' @section Unsupported (fail in strict mode or skip with \code{mode = "skip"}):

#' Domains: Specimen, VisitDetail, DoseEra; VisitOccurrence.ProviderSpecialty;

#' Measurement/DrugExposure attributes, DoseEra domains, exit strategies (\code{fixedExit},

#' \code{drugExit}, \code{observationExit}), censoring criteria, and era collapse.

#'

#' @section Unsupported (fail in strict mode or skip with \code{mode = "skip"}):

#' Domains: Specimen, VisitDetail; VisitOccurrence.ProviderSpecialty;

Copilot · 2026-02-09T04:38:05Z

R/collectCodesetId.R

+    id2 <- collectGuid(x@attributes[[ii]]@group)
+    # id2 is a list of tibbles; do not flatten (would turn tibbles into columns)
+    dfs <- purrr::keep(id2, is.data.frame)
+    if (length(dfs) > 0L) ids <- dplyr::bind_rows(ids, dfs)
  }


collectGuid() for Query now only keeps top-level data.frames from collectGuid(x@attributes[[ii]]@group). If the correlated criteria group contains nested groups, collectGuid(Group) returns nested lists and those nested tibbles will be dropped here, causing missing GUIDs / incorrect codeset ID remapping. Consider recursively collecting data.frames from the nested list structure (without flattening data.frames).

I'm not sure we have nested lists though...

Copilot · 2026-02-09T04:38:05Z

tests/testthat/test-jsonToCapr.R

+      items <- cs$expression$items %||% list()
+      for (j in seq_along(items)) {
+        cpt <- items[[j]]$concept


%||% is used throughout this test file but is not defined/imported here. Since %||% is imported from rlang but not exported by Capr, these tests will fail unless the operator is explicitly referenced (e.g., rlang::%||%) or defined locally.

import %||% from rlang

Copilot · 2026-02-09T04:38:05Z

tests/testthat/test-jsonToCapr.R

+  tryCatch(
+    jsonToCaprFile(jsonPath, rPath, mode = "skip"),
+    error = function(e) return(list(ok = FALSE, name = name, stage = "jsonToCapr", msg = conditionMessage(e)))
+  )
+  if (!file.exists(rPath)) {
+    return(list(ok = FALSE, name = name, stage = "write", msg = "R file was not created"))
+  }


The tryCatch(...) result from jsonToCaprFile() isn’t captured/checked. A failure in jsonToCaprFile() will return a list from the error handler, but the function will keep going and report a misleading stage = "write" error instead of stage = "jsonToCapr". Capture the tryCatch result and return() it when it’s an error list.

R/jsonToCapr.R

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…ts with nested criteria.

- Add remapCirceCodesetIdsToOriginal() to map internal codeset ids (0,1,2,...) from replaceCodesetId back to original ids from the cohort/includeConceptSets, so round-trip JSON keeps original concept set numbering. - When includeConceptSets is provided, use it as the full ConceptSets list (order and ids) instead of merging and re-numbering; then remap all CodesetId / *SourceConcept references in the CIRCE structure to those ids. - Fixes round-trip for cohorts that rely on specific ids or duplicate concept sets.

- Extract codesetId once (CodesetId or CodesetID) in criterionNodeToCapr. - When a criterion has no CodesetId and no SourceConcept, treat as "any" event (e.g. inclusion rule "any condition") and emit conceptSetVar = "NULL" instead of skipping/erroring. - When AdditionalCriteria is present but empty, emit withAll() so round-trip preserves AdditionalCriteria and CIRCE/SQL matches (e.g. outcome_death QualifiedLimit ordinal).

ablack3 added 5 commits February 9, 2026 02:02

Add keyValueAttribute and conditionTypeExclude attribute

d559a6a

inital version of jsonToCapr

b93639a

add doseEra attribute

cf28659

add test for jsonToCapr

d4bde3a

Update documentation. Suggest PhenotypeLibrary for testing.

460b8a1

ablack3 requested a review from Copilot February 9, 2026 02:28

Copilot started reviewing on behalf of ablack3 February 9, 2026 02:29 View session

Copilot AI reviewed Feb 9, 2026

View reviewed changes

ablack3 added 19 commits February 9, 2026 02:43

fix jsonToCapr output for console

23902eb

bugfix where empty lists were serialized as [] instead of {}

804d3a5

This line was causing an issue in r check

6e63e02

add isDistinct and countColumn slots to Occurence. This is so we can …

85e14ad

…support "At least 2 events" (any two rows), and "At least 2 distinct values of some column" (e.g. 2 distinct drug concepts) in Circe/Atlas.

add library(Capr) to generate Capr code

12ee55a

update docs

b2f6988

Order the attributes to match Circe. This is purely cosmetic but help…

529bc87

…s with comparing Circe json to Capr generated json because the order matches.

bugfix in jsonToCapr

b87a169

Make sure criteria list has domains as keys to match Circe

631a22d

Small improvements to collectCodesetId. Make sure we only run bind_ro…

99be0c8

…ws on a list of dataframes.

update jsonToCapr

1823086

update test for jsonToCapr

1716a62

use camelCase

3387fc3

update docs

2c47c39

add debugging tools in extras folder

4c14ff4

fix test failures and warnings

c161c4b

fix rcheck warnings and notes

438bf5f

ablack3 requested review from Copilot and mdlavallee92 February 9, 2026 04:29

Copilot started reviewing on behalf of ablack3 February 9, 2026 04:29 View session

Copilot AI reviewed Feb 9, 2026

View reviewed changes

ablack3 and others added 27 commits February 9, 2026 04:49

clean up extras/tools

2d5c419

add validation of Atlas json inputs using jsonvalidate

893c12e

Update tests/testthat/test-jsonToCapr.R

32b176e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/testthat/test-jsonToCapr.R

104d656

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update R/jsonToCapr.R

30966ad

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update R/jsonToCapr.R

22aaa7c

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update R/jsonToCapr.R

583393c

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

remove generics since these are S3 generics that we don't need.

c64231c

support more domains

bd3c58b

add end to end test tools

67a4bf8

fix bug suggested by copilot. I think I did see this come up in cohor…

722627a

…ts with nested criteria.

update docs

4be5701

update tests

d869279

ignore debugging files

812a742

add missing attributes and fix serialization bug

b742057

update end to end test tools. ignore concept ordering.

2e2924f

add missing circe attrubute and add specime domain query

20839bf

add missing visit detail and visit type attribute constructors

589b158

add missing functions

9a68ad1

add test for observation period

079a641

update docs

9899f69

docs

6d4e48d

support missing attributes

f118ec9

criteria list can be empty

90d293f

add edge case test

a1e2dec

Conversation

ablack3 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

ablack3 Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

ablack3 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

ablack3 Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

ablack3 Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ablack3 commented Feb 9, 2026 •

edited

Loading

ablack3 commented Feb 9, 2026 •

edited

Loading