Changing Mapping with Zero Downtime

At some point, you might want to change the mapping of an existing index. As you probably already know, it's not possible to modify an existing field type, you can only add a new field. This problem is usually solved in few steps:

1. Create a new index with the desired mapping.
2. Import the data to the new index.
3. Create an alias for the new index with the same name as the old index.
4. Drop the old index.

Let's see how this can be done with Elastic Migrations and Elastic Scout Driver Plus. Imagine, that we want to alter a text field title of the index books to a keyword type. The first thing we need to do is to prepare our Book model:

class Book extends Model
{
    use Searchable;
    
    public static $searchableAs;

    public function searchableAs()
    {
        return config('scout.prefix') . (self::$searchableAs ?? $this->getTable());
    }
}

This step is necessary to be able to switch the index on the fly. You can extract this functionality in a trait or use a different, more eloquent way to achieve the same result.

The second and the last preparation we need to do is to create a migration file:

php artisan elastic:make:migration zero_downtime

Now we can start reproducing the steps mentioned above. The first step is to create a new books_v2 index:

public function up(): void
{
    // step 1
    Index::create('books_v2', static function (Mapping $mapping, Settings $settings) {
        $mapping->keyword('title');
    });
}

Then we need to import the data to the new index:

public function up(): void
{
    // step 1
    Index::create('books_v2', static function (Mapping $mapping, Settings $settings) {
        $mapping->keyword('title');
    });

    // step 2
    Book::$searchableAs = 'books_v2';
    Book::makeAllSearchable();
}

Note that we switched the searchable index to books_v2. If we wouldn't do that, then models would be reindexed to the old books index.

If scout queues are enabled, then we can't proceed to the next step until all models are reindexed:

public function up(): void
{
    // step 1
    Index::create('books_v2', static function (Mapping $mapping, Settings $settings) {
        $mapping->keyword('title');
    });

    // step 2
    // memorize the number of documents in the old index
    $targetCount = Book::matchAllSearch()->size(0)->execute()->total();

    Book::$searchableAs = 'books_v2';
    Book::makeAllSearchable();

    // wait until all documents are reindexed to the new index
    while (true) {
        $currentCount = Book::matchAllSearch()->size(0)->execute()->total();
        
        if ($currentCount >= $targetCount) {
            break;
        }

        sleep(1);
    }
}

As mentioned before, this is only necessary if scout.queue configuration is enabled. Alternatively, you can split the migration into 2 parts and run the second migration when all documents are reindexed.

Finally, we need to create an alias and drop the initial index:

public function up(): void
{
    // step 1
    Index::create('books_v2', static function (Mapping $mapping, Settings $settings) {
        $mapping->keyword('title');
    });

    // step 2
    // memorize the number of documents in the old index
    $targetCount = Book::matchAllSearch()->size(0)->execute()->total();

    Book::$searchableAs = 'books_v2';
    Book::makeAllSearchable();

    // wait until all documents are reindexed to the new index
    while (true) {
        $currentCount = Book::matchAllSearch()->size(0)->execute()->total();
        
        if ($currentCount >= $targetCount) {
            break;
        }

        sleep(1);
    }

    // step 3
    Index::drop('books');

    // step 4
    Index::putAlias('books_v2', 'books');
}

Please note, that zero-downtime migration is not a trivial process and you might need to adjust some steps in your project. Use this guide as a reference only.