Failed to locate civicrm.settings.php - prevents civix from being used as a tool in workflows where there is no CMS/CiviCRM site, this is a major bummer 😢 and limits how this tool can be used

[agileware-justin]

Failed to locate civicrm.settings.php - prevents civix from being used as a tool in workflows where there is no CMS/CiviCRM site, this is a major bummer 😢 and limits how this tool can be used.

For example, a github action which runs to perform a civix upgrade on an extension is non-functional because the workflow aborts as follows.

Incremental upgrades
====================

Current civix format is v23.02.1.

Upgrade v23.02.1 => v24.09.0
----------------------------

Executing upgrade script /home/runner/.composer/vendor/civicrm/civix/upgrades/24.09.0.up.php

 ! [NOTE] This update adds a new helper, E::schema(), which requires the library
 !        civimix-schema@5. To enable support for older versions of CiviCRM     
 !        (<5.73), the update will:                                             
 !                                                                              
 !        1. Create mixin/lib/pathload-0.php                                    
 !        2. Create mixin/lib/civimix-schema@5.80.2.phar                        

Set civix format to 24.09.0 in info.xml
Write info.xml

Upgrade v24.09.0 => v24.09.1
----------------------------

Executing upgrade script /home/runner/.composer/vendor/civicrm/civix/upgrades/24.09.1.up.php

Entity Framework v1 => v2
=========================

 ! [NOTE] This update converts data-storage from Entity Framework v1 (EFv1) to  
 !        Entity Framework v2 (EFv2).                                           
 !                                                                              
 !        EFv2 stores schema as *.php. It has simpler workflows and less        
 !        boilerplate. SQL is generated during installation. (More details:     
 !        https://github.com/totten/civix/wiki/Entity-Templates)                
 !                                                                              
 !        The upgrader will make the following changes:                         
 !                                                                              
 !        1. Update info.xml to use CiviMix\Schema\Outlook365\AutomaticUpgrader 
 !        (which delegates to CRM_Outlook365_Upgrader)                          
 !                                                                              
 !        This should work for many extensions, but it should be tested. You may
 !        encounter issues if any of these scenarios apply:                     
 !                                                                              
 !        1. Your target environment is CiviCRM v5.44 or earlier.               
 !        2. The SQL files include custom/manual statements.                    
 !        3. The class CRM_Outlook365_Upgrader overrides any internal plumbing  
 !        (e.g. setCurrentRevision(), appendTask(), or getRevisions())          
 !                                                                              
 !        You may skip this update. However, going forward, civix will only     
 !        support EFv2. You will be responsible for maintaining any boilerplate 
 !        for EFv1.                                                             

 Should we apply the update? [Yes, update to Entity Framework v2]:
  [y] Yes, update to Entity Framework v2
  [n] No, stay on Entity Framework v1
  [a] Abort
 > 

Review Entity Framework v1
--------------------------

 ! [NOTE] Before converting, please review the existing entity configuration    
 !        files.                                                                
 !                                                                              
 !        Every entity should normally have a pair of files (XML+PHP).          
 !                                                                              
 !        +--------+--------+-----+-----+                                       
 !        | Entity | Folder | XML | PHP |                                       
 !        +--------+--------+-----+-----+                                       
 !                                                                              
 !        The converter will read any XML files and generate new (consolidated) 
 !        PHP files in ./schema.                                                

Write info.xml

In Bootstrap.php line [178]
                                                                               
  Failed to locate civicrm.settings.php. By default, this tool searches the p  
  arent directories for a standard CMS (Drupal, WordPress, etal) and standard  
   civicrm.settings.php. Symlinks and multisite configurations may interfere.  
   To customize, set variable CIVICRM_SETTINGS to point to the preferred civi  
  crm.settings.php.                                                            
                                                                               

upgrade [--start START] [--yes]

Error: Process completed with exit code 1.

[totten]

Well, several parts of civix do run without a copy of core -- but a few do require it.

For a while, I tried to keep a firmer boundary. However, this meant the overall posture was inconsistent. (Sometimes civix can call Civi helpers; sometimes it can't; and when someone writes a PR, you have to figure out how much to push-back on whether a core-reliance is really needed.) It's a lot simpler to just require it.

OTOH, what you want is not that far away. If someone wanted to put money or time into relaxing it further, that's fine. (My POV - maybe a day's work?) A quick grep turns up these things to target:

-- These might require something clever. They are edge-cases wherein the
-- need for civicrm-core depends on choices made by user. There may be
-- a way to do a soft-boot/conditional-boot that pleases everyone.

src/CRM/CivixBundle/Builder/Info.php:    [$major, $minor] = explode('.', \CRM_Utils_System::version());
src/CRM/CivixBundle/Command/InitCommand.php:      [$verMajor, $verMinor] = explode('.', \CRM_Utils_System::version());
src/CRM/CivixBundle/Command/InitCommand.php:      $siteName = \CRM_Utils_System::baseURL();

-- These would require replacing core-helpers with non-core-helpers.
-- New helpers need to go in `civix` itself or arguably into `civimix-schema`.
-- (My guess is that `civix.git:/src/` is the path of least resistance -- but it's debatable.)

src/CRM/CivixBundle/Command/AddEntityCommand.php:        'title_plural' => \CRM_Utils_String::pluralize($entityNameCamel),
src/CRM/CivixBundle/Command/AddCaseTypeCommand.php:      $input->setArgument('<Name>', \CRM_Case_XMLProcessor::mungeCasetype($input->getArgument('<Label>')));
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:      [$xml, $error] = \CRM_Utils_XML::parseFile($fileName);
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:    [$xml, $error] = \CRM_Utils_XML::parseFile($fileName);
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:      'title_plural' => self::toString('titlePlural', $xml) ?: \CRM_Utils_String::pluralize($title),
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:    $utilSchem = class_exists('CRM_Utils_Schema') ? \CRM_Utils_Schema::class : SchemaBackport::class;
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:      if ($typeAttributes['crmType'] == 'CRM_Utils_Type::T_BOOLEAN') {
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:        $fields[$name]['data_type'] = \CRM_Utils_Type::typeToString(constant($typeAttributes['crmType']));
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:        $fields[$name]['serialize'] = 'CRM_Core_DAO::SERIALIZE_' . $fieldXml->serialize;
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:          'entity' => $thisTables[$fkTable] ?? \CRM_Core_DAO_AllCoreTables::getEntityNameForTable($fkTable),
src/CRM/CivixBundle/Command/ConvertEntityCommand.php:    return \CRM_Utils_Array::rekey($arr, ['CRM_Utils_String', 'convertStringToSnakeCase']);
src/CRM/CivixBundle/Utils/SchemaBackport.php:use CRM_Utils_Type;
src/CRM/CivixBundle/Utils/SchemaBackport.php:        return CRM_Utils_Type::T_STRING;
src/CRM/CivixBundle/Utils/SchemaBackport.php:        return CRM_Utils_Type::T_DATE + CRM_Utils_Type::T_TIME;
src/CRM/CivixBundle/Utils/SchemaBackport.php:        return CRM_Utils_Type::T_MONEY;
src/CRM/CivixBundle/Utils/SchemaBackport.php:        return CRM_Utils_Type::T_FLOAT;
src/CRM/CivixBundle/Utils/SchemaBackport.php:        return CRM_Utils_Type::T_INT;
src/CRM/CivixBundle/Utils/SchemaBackport.php:        return constant('CRM_Utils_Type::T_' . strtoupper($type));

-- These probably aren't worth changing.

src/CRM/CivixBundle/Builder/CustomDataXML.php:    $this->export = new \CRM_Utils_Migrate_Export();
src/CRM/CivixBundle/Command/TestRunCommand.php:      $template_vars['civicrm_setting']['Directory Preferences']['extensionsDir'] = \CRM_Core_BAO_Setting::getItem('Directory Preferences', 'extensionsDir');
src/CRM/CivixBundle/Command/TestRunCommand.php:      $template_vars['civicrm_setting']['URL Preferences']['extensionsURL'] = \CRM_Core_BAO_Setting::getItem('URL Preferences', 'extensionsURL');
src/CRM/CivixBundle/Command/TestRunCommand.php:      $template_vars['civicrm_setting']['Test']['test_extensions'] = array_keys(\CRM_Core_PseudoConstant::getExtensions());

I suspect you're mostly hitting on the second group (the schema helpers).

[agileware-justin]

Thanks @totten much appreciated.

Thought bubble - could civix be configured to call a remote CiviCRM site or a web service (to get the CiviCRM bits n pieces it needs)?

[agileware-justin]

OTOH, what you want is not that far away. If someone wanted to put money or time into relaxing it further, that's fine. (My POV - maybe a day's work?) A quick grep turns up these things to target:

Thanks @totten also for the estimate of work, that gives it some weighting. Have a good one!