[SD-1448] Support partial indexing success and improve bulk error logging

[vincent-gao]

Jira

https://digital-vic.atlassian.net/browse/SD-1448

Problem/Motivation

Currently, the indexItems() method operates on an "all-or-nothing" basis. If a single document in a batch fails, the module throws a generic SearchApiException.

• This causes two significant issues in our projects.
  • The entire batch fails.
  • The thrown exception message is generic ("An error occurred during indexing"). The actual error details returned by the Elasticsearch bulk API (reason, type, caused_by) are not logged, making it difficult to identify the problematic entity.

Fix

Refactor indexItems() to handle the Elasticsearch bulk response more gracefully.

Test and Screenshots

use a hook_search_api_index_items_alter to alter the data, to break the search.

/**
 * Implements hook_search_api_index_items_alter().
 */
function reference_sdp_core_search_api_index_items_alter(\Drupal\search_api\IndexInterface $index, array &$items,) {
  foreach ($items as $item_id => $item) {
    error_log($item_id);
    /** @var \Drupal\search_api\Item\ItemInterface $item */
    if (strpos($item_id, 'node/175:') !== false) {
      if ($field = $item->getField('created')) {
        $field->setValues(['I am not a timestamp, I will break the search.']);
      }
    }
  }
}

check the log

[MdNadimHossain]

wonderful. It will not stop when it finds a error, awesome. Lets test it though before merging it.

[anthony-malkoun]

Yo @vincent-gao completely fine with this approach. Is it also worth submitting an issue and patching https://git.drupalcode.org/project/elasticsearch_connector/-/blob/8.0.x/src/SearchAPI/BackendClient.php#L99

[vincent-gao]

Yo @vincent-gao completely fine with this approach. Is it also worth submitting an issue and patching https://git.drupalcode.org/project/elasticsearch_connector/-/blob/8.0.x/src/SearchAPI/BackendClient.php#L99

hi @anthony-malkoun ! I will submit a patch to Drupalcore and see what their feedback is!