SomaDataIO 6.5.0

Function and Object Improvements

• Updated preProcessAdat() to improve clarity
  • filter.qc parameter has been renamed to filter.rowcheck
  • for backward compatibility, filter.qc is still accepted but will generate
    a deprecation warning and will be removed in a future version
  • removed language that discusses implication of ColCheck SeqIds not
    being removed from preProcessAdat() output
• Updated read_annotations() function and ver_dict object to
  recognize latest menu annotations files from Q3 2025

Documentation Updates

• updated statistical analysis workflow articles with clearer
  messaging comments around preProcessAdat()
• updated two-group and three-group analysis workflow articles to
  set center.scale = FALSE in preProcessAdat() to align with guidance
  to users for univariate analysis

Internal 🚧

• Updated dplyr verb tests to no longer explicitly test
  for ordering of attributes (#165)

SomaDataIO 6.4.0

New Functions

• Added updateColMeta() function

  • added utility function to update the column metadata in an
    ADAT to match an annotations object from read_annotations()

• Added is.AptName() to check for AptName format

  • is.apt() may return TRUE for both SeqIds and AptNames
  • wrote new function, is.AptName(), to explicitly check
    if a SomaScan identifier is an AptName
  • is.AptName() code and tests based on code/tests from is.SeqId()

Function and Object Improvements

• Relaxed file version requirement in read_annotations()

  • now returns warning instead of error if annotations
    excel file version does not match specific values

• Updated preProcessAdat() to handle missing ColCheck

  • added logic to preProcessAdat() to handle adats
    that are missing ColCheck in column annotation data

Documentation Updates

• Show package loading in stat workflow vignettes
  • move library() calls loading third-party packages to new
    code chunk that now appears in analysis workflow articles

SomaDataIO 6.3.0

New Functions

• Added preProcessAdat() function

  • added new function preProcessAdat() to filter features,
    filter samples, generate data QC plots of normalization
    scale factors by covariates, and perform standard analyte
    RFU transformations including log10, centering, and scaling

• Added calcOutlierMap() function

  • added calcOutlierMap() and its print and plot
    S3 methods, along with getOutlierIds() for
    identifying sample level outliers from outlier map object
  • added ggplot2 as a package dependency

Function and Object Improvements

• Added ex_clin_data object
  • a tibble object with additional sample annotation fields
    smoking_status and alcohol_use to demonstrate merging to a
    soma_adat object

Documentation Updates

• Added pre-processing vignette article

  • includes guidance on pre-processing SomaScan data for
    a typical analysis
  • provides an example of recommended workflow of
    filtering features, filtering samples, performing data
    QC checks, and transformations of RFU features
  • introduces usage of the preProcessAdat() function

• Improved adat ingest documentation in README

  • added comments to clarify file path input to read_adat()
    example in README

• Updated stat workflow articles to begin with reading in adat

  • updated data preparation chunks with comments about
    how to download and read in the the example_data.adat object
  • data preparation chunks now use the preProcessAdat()
    function for pre-processing

• Added sample annotation merging guidance

  • updated README and loading and wrangling vignette
    article with section including code to join the
    ex_clin_data object to the example_data adat

Internal 🚧

• Added helper utility functions for snapshot plot unit tests
  • added helper utility functions figure(), close_figure(),
    save_png(), and expect_snapshot_plot() for saving plot
    snapshot output to testthat/helper.R
  • added snapshot unit tests for preProcessAdat()
    messaging, print and QC plot output

SomaDataIO 6.2.0

New Functions

• Added calc_eLOD() function (#131)
  • calculates the estimated limit of detection (eLOD)
    for SeqId columns of an input soma_adat or data.frame

Bug Fixes

• Fixed crayon bug and ui_bullet() issue (#129, #130)

  • removed crayon and usethis as dependencies in favor of cli
  • fixed bug in R version 4.4.1 with ui_bullet() internal
    calls within loadAdatsAsList() and write_adat()

• Fixed bug in Summary.soma_adat() operations (#121)

  • these operations: min(), max(), any(), range(), etc.
    would return the incorrect value due to an as.matrix()
    conversion under the hood
  • now skips that conversion, trips a warning, and carries on
  • triggers an error if non-numerics are passed
    as part of the '...' outside of a soma_adat,
    just like Summary.data.frame()

Function and Object Improvements

• collapseAdats() now maintains Cal.Set entries of Col.Meta (#113)

  • collapsing ADATs can be problematic for the attributes,
    especially for large numbers of ADATs
  • collapseAdats() now attempts to smartly
    merge the (potentially numerous elements) Col.Meta
    attribute in the final object, preserving
    the "Cal.Set" and "ColCheck" columns in particular
  • the resulting Col.Meta attribute is a combined product
    of the individual ADAT elements, and the intersect of the
    analyte features (as is the case for the rbind() that is called)

• Updated checksums and versions for Annotations Excel files (#116)

  • updated the 7k and 11k file versions and md5sum checksums
  • now allows read_annotations() to load
    the individual Excel files

• Updated lift_master object to alpha sort columns

Documentation Updates

• Updated company name, license year, and maintainer (#137)

  • SomaLogic Operating Co., Inc is now Standard BioTools, Inc.
  • updated license and copyright year to 2025
  • updated package maintainer to Caleb Scheidel

• Updated article links in README, intro vignette (#123)

  • updated links to articles in README and introduction
    vignette to URLs to pkgdown website rather than
    vignette() code references
  • added clarification to above documents that articles are
    available on website only rather than traditional vignettes
    included with package

• Updates to example documentation

  • read_annotations() example documentation now points to the
    most recent 11k Excel annotations file
  • parseHeader() example now prints list elements separately,
    rather than full object, which slowed website rendering

Internal 🚧

• Updates to GitHub Action workflows

  • added rhub.yaml configuration file to comply with rhub v2
  • updated macOS version in pkgdown.yaml to macOS-14
  • added write permission to pkgdown.yaml file to enable deployment
  • changed GitHub Action R checks to MacOS and Windows only
    • ubuntu machine was taking too long to build

• Increased package test coverage

  • added unit tests for getSomaScanLiftCCC(),
    parseCheck() and release utilities which
    were previously untested
  • increased test coverage for pivotExpressionSet()

• Added missing package anchors to .Rd files (#139)

  • fixed note from remote windows check
    related to Rd \link{} targets missing
    package anchors

• Updated README badge (#109)

  • now shows 'downloads' per month over total downloads

• Fixed link in DESCRIPTION; master -> main (#107)

SomaDataIO 6.1.0

Lifting Code 🚀

• Major restructure of lift_adat() functionality (@stufield, #81, #78)

  • lift_adat() now takes a bridge = argument,
    replacing the anno.tbl = argument (which is now deprecated). Lifting
    is now performed internally for a better (and safer)
    user experience, without the necessity of an
    external annotations (Excel) file.
  • the majority of this refactoring was internal
    and the user should not experience a major
    disruption to the API.
  • much improved lifting/bridging documentation (#82)

• Added a new lifting and bridging vignette (@stufield, #77)

  • in addition to the improved lifting documentation
    this new vignette provides additional context,
    explanation, clear examples, and lifting guidance.

New Functions ✨

• is_lifted() is new and returns a boolean according to
  whether the signal space (RFU) has been previously lifted

• Lifting accessor function for Lin's CCC values (#88)

  • getSomaScanLiftCCC() accesses the lifting correlations between
    SomaScan versions for each analyte
  • returns a tibble split by sample matrix (serum or plasma)

• merge_clin() is newly exported (#80)

  • a thin wrapper that allows users to merge
    clinical variables to soma_adat objects easily
  • previously users had to either use the CLI merge tool
    or merge in clinical variables themselves with dplyr

• Newly exported ADAT "get**" helpers (#83)

  • functions to access properties of ADATs
    • getAdatVersion()
    • getSomaScanVersion()
    • getSignalSpace()
    • checkSomaScanVersion()
  • getAdatVersion() gets a new S3 method (#92)
    • this enables passing of different objects
    • namely soma_adat or list depending on the situation

• Newly exported functions that were previously internal only:

  • addAttributes()
  • addClass()
  • cleanNames()

New Vignettes 🤓

• The package README is now simplified (#35)

  • example analysis workflows are now split out
    into their own vignettes/articles
    and cross-linked in the README

• Reorganization and expansion of statistical vignettes (#35, #47)

  • moved 3 existing statistical examples from
    README into their own vignettes
  • resulting in four new "Statistical Workflow" vignettes/articles:
    • Binary classification via logistic regression
    • Linear regression for continuous variables
    • Two-group comparison via t-test
    • Three-group analysis ANOVA

• Added new general analysis workflow vignettes

  • articles for the pkgdown website have been built out
  • new articles on:
    • safely mapping values among variables
    • safely renaming a data frame
    • loading-and-wrangling
    • typical train and test data splits
    • beginning the FAQs and/or Coming Soon pages

• Added a new vignette describing how to use the
  command-line interface merge tool (#45)

  • the new CLI merge tool used to add
    new clinical data to existing ADAT file

Updates and Improvements 🔨

• collapseAdats() better combines HEADER information (#86)

  • certain information, e.g. PlateScale and Cal*,
    are better maintained in the final collapsed ADAT
  • other entries are combined by pasting into a single string
  • should result in less duplication of superfluous entries and
    retention of more "useful" HEADER information
    in the resulting (collapsed) soma_adat

• Update read_annotations() with 11k content (#85)

• Update transform() and scaleAnalytes()

  • scaleAnalytes() (internal) now skips missing references
    and is much more like a "step" in the recipes package
  • transform() gets edge case protection with drop = FALSE
    in case a single-analyte soma_adat is scaled.

• New row.names() S3 method support for soma_adat class

  • dispatched on calls to rownmaes()
  • rather than calling NextMethod() which normally
    would invoke data.frame, we now force the data.frame
    method in case there are tbl_df or grouped_df
    classes present that would be dispatched.
    Those are bypassed in favor of the data.frame
    because tbl_df 1) can nuke the attributes, 2)
    triggers a warning about adding rownames to a tibble.

• New grouped_df S3 print support for the grouped soma_adat

  • now displays Grouping information from a call to
    the S3 print method for soma_adat class

• New grouped_df S3 method support for soma_adat class (#66)

  • grouped_df data objects previously unsupported and were
    interfering with downstream S3 methods for dplyr verbs
    once NextMethod() was called
  • this support now ensures that the group
    methods are maintained, as well as the soma_adat
    class itself (and most importantly, with its attributes intact)

• tidyr::separate.soma_adat() S3 method was simplified (#72)

  • now uses %||% helper internally
  • expanded error messages inside stopifnot() to be more informative

• is_intact_attr() is now much quieter, signaling only when called indirectly (#71)

  • new conditional logic to silences signaling messages when
    called from within another function (indirectly)
  • these previously lead to confusing messages
    when they appear in wrappers, where is_intact_attr()
    can be, sometimes deeply, nested in the call stack

• Development and improvements to the pkgdown website

  • added new links and improved clarity in YAML
  • added new logo at footer
  • restyled side bar for easier hyperlinking and getting help
  • clicking on the SomaLogic logo in the GitHub README
    now links to the pkgdown website
  • new "Coming Soon" drop-down section in the website header
    to let users know about active progress (but not yet ready
    for external publication)

• SomaDataIO no longer depends on desc package

  • to generate the README.md

Internal 🚧

• Internal rowname helpers were upgraded

  • they now use internal cross-functions
    as originally intended to avoid redundancy, efficiency,
    and improved debugging

• sysdata.rda no longer contains non-exported functions (#59)

  • new internal helper functions:
    • convertColMeta()
    • genRowNames()
    • parseCheck()
    • syncColMeta()
    • scaleAnalytes()

• Bug-fix for corner-case writing a single-analyte ADAT (#51)

  • RFU values are rounded to 1 decimal place when written by
    write_adat(), via a call to apply(), which expects a 2-dim object
    when replacing those values.
  • write_adat() no longer uses apply() and instead converts
    the entire RFU data frame to a matrix (maintains original dimensions),
    and use vectorized format conversion via sprintf()
  • in theory this should be faster because sprintf()
    is only called once on a long vector, rather than
    1000s of times on shorter vectors (inside apply()).

SomaDataIO 6.0.0

We are now on CRAN! 🥳

New changes

• New clinical data merge CLI tool (#25)
  • Rscript --vanilla merge_clin.R for merging clinical variables
    into existing *.adat SomaScan data files
  • added 2 new example meta.csv and meta2.csv files
    to run examples with random data but with valid index keys
  • see dir(system.file("cli", "merge", package = "SomaDataIO"))

Package data objects

• general reorganization of package objects, trimming
  and slimming, renaming, and file structure (#32)
• example_data.adat was reduced in size to n = 10 samples (from 192)
  to conform to CRAN size requirements (< 5MB)
• the current file was renamed example_data10.adat to reflect this change
• this likely has far-reaching consequences for users who access
  this flat file via system.file()
• the example_data object itself however remains true to its original
  file (https://github.com/SomaLogic/SomaLogic-Data/blob/master/example_data.adat)
• the directory location inst/example/ was renamed inst/extdata/
  to conform to CRAN package standard naming conventions
• the file single_sample.adat was removed from package data
  as it is now redundant (however still used in unit testing)
• SomaDataObjects was renamed and is now SomaScanObjects

Gradual deprecation

• read.adat() is now soft-deprecated; please use read_adat() instead
• lifecycle for soft-deprecated warn() -> stop() for functions
  that have been been soft deprecated since v5.0.0
  • getSomamers()
  • getSomamerData()
  • meltExpressionSet()

S3 Methods

• New S3 print method default
  • tibble has new max_extra_cols = argument, which
    is set to 6 for the print.soma_adat method
• New S3 merge method (#31)
  • calling base::merge() on a soma_adat is strongly discouraged
  • we now redirect users to use dplyr::*_join() alternatives
    which are designed to preserve soma_adat attributes

Internal upgrades

• Code hardening for prepHeaderMeta()

  • some ADATs do not have CreatedDate and CreatedBy
    in the HEADER entry. This currently breaks the writer
  • simplified to make more robust but also refactor
    to be more convenient (for abnormal ADATs not generated
    by standard SomaScan processing)
  • CreatedDateHistory was removed as an entry from written ADATs
  • CreatedByHistory was combined and dated for written ADATs
  • NULL behavior remains if keys are missing
  • CreatedBy and CreatedDate will be generated either
    as new entries or over-written as appropriate

• Numerous non-user-facing (API) changes internal package
  maintenance, efficiency, and structural upgrades were included

SomaDataIO 5.3.1

This bug-fix release primarily includes fixes related to write_adat() and its functionality:

• fixed bug in write_adat() that resulted from
  adding/removing clinical (non-SomaScan) variables to an
  ADAT. Export via write_adat() resulted in a broken ADAT file (#18)

• write_adat() now has much higher fidelity to original
  text file (*.adat) in full-cycle read-write-read operations;
  particularly in presence of bangs (!) in the Header
  section and in floating point decimals in the ?Col.Meta section (#24)

• write_adat() no longer converts commas (,) to
  semi-colons (;) in the ?Col.Meta block

• write_adat() no longer concatenates written ADATs,
  when writing to the same file. Data is over-written
  to file to avoid mangled ADATs resulting from re-writing
  to the same connection and to match the default behavior
  of write.table(), write.csv(), etc.

• read_adat() now has more consistent character type
  the Barcode2 variable in standard ADATs

• Decreased dependency of magrittr pipes (%>%) in favor of the native R pipe (|>)

• Migration to the default branch in GitHub from master -> main (#19)

SomaDataIO 5.3.0

Minor version

• updates from internal SomaLogic Operating Co. Inc. code base
• many code simplifications
• reduction in package dependencies have been reduced (in favor of base R implementation)
• speed/implementation improvements
• a minor bug-fix for the dplyr::mutate.soma_adat() S3 method for dplyr v1.0.8
  which included a breaking change that stripped soma_adat class attributes.
• Enjoy! 🎉

SomaDataIO 5.2.0

Lifting suite

• incorporates SomaScan "lifting" code that allows conversion between SomaScan v4 <-> v4.1
• new functions:
  • lift_adat()
  • read_annotations()
• minor internal structural-refactoring.

SomaDataIO 5.1.0

Minor release version

• internal structural-refactoring changes
• some exported functions were renamed (with redirects for backward compatibility)