Update getmeili/meilisearch Docker tag to v1.36.0

[renovate]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Update	Change
getmeili/meilisearch	minor	v1.24.0 → v1.36.0

---

Release Notes

meilisearch/meilisearch (getmeili/meilisearch)

v1.36.0: 🐙

Compare Source

Version v1.36.0 introduces an exciting update to the ranking rules to improve the engine's relevance. It's actually the first time we've made such a change since v1.0, and we're really happy about this improvement!

✨ Enhancement

• Introduce the attributeRank and wordPosition criteria by @​Kerollmops in #​6154, #​6155, and #​6164

  We released two new ranking rules that Meilisearch had already been using internally for the attribute one, which is basically both ranking rules applied one after the other:

  • attributeRank: A document is considered better if the query words match in a higher searchable attribute. It ignores the position of the query words in this attribute.
  • wordPosition: A document is considered better if the query words match closer to the beginning of an attribute. The attribute rank is ignored by this rule.

  We continue our policy of migrating everyone to use a homemade HNSW by introducing a new dumpless upgrade step that migrates index uses the old annoy vector store to the new Hannoy one. Changing the vector store backend affects the ranking score. This step can take a couple of minutes when the number of embeddings is high, and we recommend changing the vector store backend beforehand to gain more control if needed. To do so, you must enable the vectorStoreSetting experimental feature and set the vectorStore root setting to experimental.

🪲 Bug fixes

• Fix search failing when using attributesToSearchOn on an empty index by @​zen-zap in #​5944

🔒 Security

• Bump bytes from 1.11.0 to 1.11.1 by @​dependabot[bot] in #​6142
• Bump jsonwebtoken from 9.3.1 to 10.3.0 by @​dependabot[bot] in #​6141
• Bump time from 0.3.44 to 0.3.47 by @​dependabot[bot] in #​6144
• Bump rsa from 0.9.9 to 0.9.10 by @​dependabot[bot] in #​6149

🔩 Miscellaneous

• Improve OpenAPI docs and CIs to improve documentation experience by @​curquiza in #​6145

[!WARNING]
Breaking change: the meilisearch-openapi-mintlify.json file will not be available in the release assets anymore. If you were using it, please refer to the one that is now available in our public documentation repository.

• GenAI contributing and PR template changes by @​dureuill in #​6140
• Fix "test with almost all features" by @​dureuill in #​6139 and #​6148
• Allow any IP for SDK tests by @​Strift in #​6151
• Improve output of the declarative tests, also in #​6164

❤️ Thanks to @​zen-zap for contributing to this release!

v1.35.1

Compare Source

🦋 Bug fixes

Meilisearch v1.35.1 fixes a possible task database corruption issue that would occur when using the S3 streaming snapshot method.

• Users of the Community Edition do not need to upgrade as this feature is Enterprise Edition only
• Affected users of the Cloud have been contacted.

Details

While snapshotting the task database, sending a task to Meilisearch could corrupt it. Index DBs were not affected, so manually rebuilding the task database fixes the corruption.

By @​dureuill in #​6160

Full Changelog: meilisearch/meilisearch@v1.35.0...v1.35.1

v1.35.0

Compare Source

Meilisearch v1.35.0 contains improvements to the observability of the search performance, a breaking change to the recently introduced POST /indexes/<index_uid>/fields route, and other bugfixes and improvements.

🌈 Improvements

Observe the search performance

The search routes accept a new field named showPerformanceDetails. When set to true, the search response contains a performance trace, allowing the user to better
understand what takes time during a search query.

impacted routes:

• POST /indexes/<index_uid>/search
• GET /indexes/<index_uid>/search
• POST /multi-search
• POST /indexes/<index_uid>/similar
• GET /indexes/<index_uid>/similar

Search

routes: POST /indexes/<index_uid>/search and GET /indexes/<index_uid>/search

Request

new request parameters:

• showPerformanceDetails: true/false (boolean)

example

{
	"q": "glass",
	"showPerformanceDetails": true
}

Response

new response field:

• performanceDetails: {"<span>": "<human_duration>", .. } (map)

example

{
	"hits": [
		// hits ..
	],
	"query": "glass",
	"processingTimeMs": 5,
	"limit": 20,
	"offset": 0,
	"estimatedTotalHits": 1,
	"requestUid": "<uuid>",
	"performanceDetails": {
		"wait for permit": "295.29µs",
		"search > tokenize": "436.67µs",
		"search > resolve universe": "649.00µs",
		"search > keyword search": "515.71µs",
		"search > format": "288.54µs",
		"search": "3.56ms"
	}
}

Multi-search

route: POST /multi-search

Request

new request parameters:

• queries.showPerformanceDetails: true/false (boolean)

example

"queries": [
	{
		"indexUid": "<index_uid>",
		"q": "glass",
		"showPerformanceDetails": true
	}
]

Response

new response field:

• results.performanceDetails: {"<span>": "<human_duration>", .. } (map)

example

{
	"results": [
		{
			"indexUid": "<index_uid>",
			"hits": [
				// hits ..
			],
			"query": "glass",
			"processingTimeMs": 5,
			"limit": 20,
			"offset": 0,
			"estimatedTotalHits": 1,
			"requestUid": "<uuid>",
			"performanceDetails": {
				"wait for permit": "295.29µs",
				"search > tokenize": "436.67µs",
				"search > resolve universe": "649.00µs",
				"search > keyword search": "515.71µs",
				"search > format": "288.54µs",
				"search": "3.56ms"
			}
		}
	]
}

Federated Search

route: POST /multi-search

Request

new request parameters:

• federation.showPerformanceDetails: true/false (boolean)

example

{
	"federation": { "showPerformanceDetails": true },
	"queries": [
		{
			"indexUid": "<index_uid>",
			"q": "glass"
		}
	]
}

Response

new response field:

• performanceDetails: {"<span>": "<human_duration>", .. } (map)

example

{
	"hits": [
		// hits ..
	],
	"query": "glass",
	"processingTimeMs": 5,
	"limit": 20,
	"offset": 0,
	"estimatedTotalHits": 1,
	"requestUid": "<uuid>",
	"performanceDetails": {
		"wait for permit": "213.83µs",
		"search > tokenize": "171.67µs",
		"search > resolve universe": "257.63µs",
		"search > keyword search": "577.71µs",
		"search > format": "114.96µs",
		"search > federation > wait for remote results": "62.71µs",
		"search > federation > merge results": "120.04µs",
		"search > federation > merge facets": "53.42µs",
		"search > federation": "237.04µs",
		"search": "289.08ms"
	}
}

Similar

routes: POST /indexes/<index_uid>/similar and GET /indexes/<index_uid>/similar

Request

new request parameters:

• showPerformanceDetails: true/false (boolean)

example

{
	"id": 143,
	"embedder": "manual",
	"showPerformanceDetails": true
}

Response

new response field:

• performanceDetails: {"<span>": "<human_duration>", .. } (map)

example

{
	"hits": [
		// hits ..
	],
	"id": "143",
	"processingTimeMs": "[duration]",
	"limit": 20,
	"offset": 0,
	"estimatedTotalHits": 4,
	"performanceDetails": {
		"search > format": "244.92µs",
		"search": "1.25ms"
	}
}

By @​ManyTheFish in #​6132

Stabilize multithreaded post-processing of facets and prefixes

Multithreaded post-processing of facets and prefixes speeds-up indexing on multi-core machines.

Before v1.35.0, it could be disabled with an experimental feature.

Meilisearch v1.35.0 removes that experimental feature, resulting in the multithreaded post-processing being always enabled.

Cloud users upgrading to v1.35.0 will benefit from the speed-up.

By @​ManyTheFish in #​6130

🦋 Fixes

Breaking change: return pagination fields in POST /indexes/<index_uid>/fields route

[!WARNING]
Breaking change: The POST /indexes/<index_uid>/fields route now returns an object containing the results, offset, limit, and total fields instead of returning an array of results. This change allows to know how many fields match a given filter

Example: before/after

Before:

[
  // fields ..
]

After:

{
  "results": [
    // [fields .. ]
  ],
  "offset": 0,
  "limit": 20,
  "total": 0
}

By @​ManyTheFish in #​6126

POST /indexes/<index_uid>/fields: Fix pattern filtering when a parent field was matching a child pattern: title was matching the pattern title.to*.

Example: before/after

For the following request:

{
  "filter": {
    "attributePatterns": [
      "title.en"
    ]
  }
}

Before Response, title field is returned:

{
  "results": [
    {
      "name": "title",
      //metadata...
    },
    
  ],
  "offset": 0,
  "limit": 20,
  "total": 1
}

After Response, nothing returned:

{
  "results": [],
  "offset": 0,
  "limit": 20,
  "total": 0
}

By @​ManyTheFish, also in #​6126

🐛 Other fixes

• Bump mini-dashboard version to fix display of image in local UI interface by @​Strift in #​6129
• Fix mTLS compatibility with Go by @​curquiza in #​6133

🔩 Miscellaneous

• Add AI usage disclosure guidelines to CONTRIBUTING.md by @​meili-bot in #​6114
• Expose SettingsDelta and related types for external crate usage by @​MkDev11 in #​6119

New Contributors

• @​MkDev11 made their first contribution in #​6119 ❤️

Full Changelog: meilisearch/meilisearch@v1.34.3...v1.35.0

v1.34.3

Compare Source

🐛 Bug Fixes

• Increase the auth store database size to 2 GiB so that we can index more keys by @​Kerollmops in #​6134

Full Changelog: meilisearch/meilisearch@v1.34.2...v1.34.3

v1.34.2

Compare Source

This patch fixes an accidental breaking change in v1.34.1 where Meilisearch would not start with a configuration file if experimental_allowed_ip_networks was not defined.

• Meilisearch Cloud users do not need to update as they were not affected by this regression
• We recommend that OSS users upgrade to v1.34.2

Full Changelog: meilisearch/meilisearch@v1.34.1...v1.34.2

v1.34.1

Compare Source

In Meilisearch v1.34.1, we released a security fix that affect web queries initiated by Meilisearch.

🔐 Security

• Cloud users: no action is needed. We found no evidence of exploitation on Meilisearch Cloud and the Cloud is protected at the infrastructure level.
• Open-source users: if you allow untrusted users to configure webhooks, embedders or network of machines, we recommend you update to Meilisearch v1.34.1

In Meilisearch versions v1.8 to v1.34.0, a user with an API key with write permissions to the configuration of the Meilisearch instance could set up Meilisearch to send POST or GET requests to domains and IPs local to the private network of the Meilisearch instance, effectively bypassing the firewall. The vulnerability has not yet been assigned a CVE number; it has CVE Request 1975471 for CVE ID.

Meilisearch v1.34.1 fixes the vulnerability by forbidding Meilisearch from making any requests to a host resolving to a non-global IP, in the sense of the IANA IPv4 Special-Purpose Address Registry or the IANA IPv6 Special-Purpose Address Registry. If you need this functionality in your Meilisearch instance, you can set private IP networks as allowed with the --experimental-allow-ip-networks parameter after reviewing the security implications.

This is a breaking change, which is allowed for security reasons.

Allowing non-global IP networks

The --experimental-allow-ip-networks CLI flag and the MEILI_EXPERIMENTAL_ALLOW_IP_NETWORKS environment variable control the behavior of Meilisearch with regards to non-global IP networks, with the CLI flag taking precedence over the environment variable when both are specified.

• When missing, the default is to reject all web requests resolving to a non-global IP.
• When set to a comma-separated list of CIDR-formatted networks (e.g. 192.168.0.0/16,10.0.0.0), then web requests to the networks from the list will be permitted. Web requests to the non-global IPs not part of the list will still be rejected.
• When set to any, all web requests will be permitted regardless of the target IP, similar to the behavior of Meilisearch v1.34.0 and lower. Use this option when you control both the machine and the configuration of the deployed Meilisearch instance.

Acknowledgments

Thanks to Gabriel Rodrigues (aka Texugo), for reporting this vulnerability and for helping us improve the security of Meilisearch.

v1.34.0

Compare Source

🌈 Improvements

Easy search over your network of machines

useNetwork field in POST /indexes/{:indexUid}/search

The search query object passed in the body of POST /indexes/{:indexUid}/search now accepts an optional boolean useNetwork. When present and set to true, the search is executed "as-if" it was a remote federated search over all remotes in the network.

That is, the following:

Search request

// POST /indexes/movies/search
{
  "q": "Batman dark knight returns 1",
  "filter": "genres IN [Action, Adventure]",
  "facets": ["genres"],
  "useNetwork": true, // ✨ NEW
  "limit": 5
}

Is executed by Meilisearch as if it was the following, assuming a network of 3 Meilisearch instances with names "0", "1" and "2":

Equivalent multi-search request

// POST /multi-search
{
    "federation": {
        "limit": 5,
        "facetsByIndex": {
            "movies": [
                "genres"
            ]
        },
        "merge": {}
    },
    "queries": [
        {
            "indexUid": "movies",
            "federationOptions": {
                "remote": "0"
            },
            "q": "Batman dark knight returns 1",
            "filter": "genres IN [Action, Adventure]"
        },
        {
            "indexUid": "movies",
            "federationOptions": {
                "remote": "1"
            },
            "q": "Batman dark knight returns 1",
            "filter": "genres IN [Action, Adventure]"
        },
        {
            "indexUid": "movies",
            "federationOptions": {
                "remote": "2"
            },
            "q": "Batman dark knight returns 1",
            "filter": "genres IN [Action, Adventure]"
        }
    ]
}

Resulting in:

Search Response

{
  "hits": [
    {
      "id": 123025,
      "title": "Batman: The Dark Knight Returns, Part 1",
      "overview": "Batman has not been seen for ten years. A new breed of criminal ravages Gotham City, forcing 55-year-old Bruce Wayne back into the cape and cowl. But, does he still have what it takes to fight crime in a new era?",
      "genres": [
        "Action",
        "Animation",
        "Mystery"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/kkjTbwV1Xnj8wBL52PjOcXzTbnb.jpg",
      "release_date": 1345507200,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 2,
        "weightedRankingScore": 0.9894586894586894,
        "remote": "2"
      }
    },
    {
      "id": 142061,
      "title": "Batman: The Dark Knight Returns, Part 2",
      "overview": "Batman has stopped the reign of terror that The Mutants had cast upon his city.  Now an old foe wants a reunion and the government wants The Man of Steel to put a stop to Batman.",
      "genres": [
        "Action",
        "Animation",
        "Mystery"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/arEZYd6uMOFTILne9Ux0A8qctMe.jpg",
      "release_date": 1357171200,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 1,
        "weightedRankingScore": 0.9894558963186414,
        "remote": "1"
      }
    },
    {
      "id": 16234,
      "title": "Batman Beyond: Return of the Joker",
      "overview": "The Joker is back with a vengeance, and Gotham's newest Dark Knight needs answers as he stands alone to face Gotham's most infamous Clown Prince of Crime.",
      "genres": [
        "Animation",
        "Family",
        "Action",
        "Science Fiction"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/7RlBs0An83fqAuKfwH5gKMcqgMc.jpg",
      "release_date": 976579200,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 1,
        "weightedRankingScore": 0.9427964918160996,
        "remote": "1"
      }
    },
    {
      "id": 155,
      "title": "The Dark Knight",
      "overview": "Batman raises the stakes in his war on crime. With the help of Lt. Jim Gordon and District Attorney Harvey Dent, Batman sets out to dismantle the remaining criminal organizations that plague the streets. The partnership proves to be effective, but they soon find themselves prey to a reign of chaos unleashed by a rising criminal mastermind known to the terrified citizens of Gotham as the Joker.",
      "genres": [
        "Drama",
        "Action",
        "Crime",
        "Thriller"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/qJ2tW6WMUDux911r6m7haRef0WH.jpg",
      "release_date": 1216166400,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 1,
        "weightedRankingScore": 0.5784178187403994,
        "remote": "1"
      }
    },
    {
      "id": 49026,
      "title": "The Dark Knight Rises",
      "overview": "Following the death of District Attorney Harvey Dent, Batman assumes responsibility for Dent's crimes to protect the late attorney's reputation and is subsequently hunted by the Gotham City Police Department. Eight years later, Batman encounters the mysterious Selina Kyle and the villainous Bane, a new terrorist leader who overwhelms Gotham's finest. The Dark Knight resurfaces to protect a city that has branded him an enemy.",
      "genres": [
        "Action",
        "Crime",
        "Drama",
        "Thriller"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/vzvKcPQ4o7TjWeGIn0aGC9FeVNu.jpg",
      "release_date": 1342396800,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 2,
        "weightedRankingScore": 0.5772657450076805,
        "remote": "2"
      }
    }
  ],
  "query": "Batman dark knight returns 1",
  "processingTimeMs": 173,
  "limit": 5,
  "offset": 0,
  "estimatedTotalHits": 47,
  "facetDistribution": {
    "genres": {
      "Action": 46,
      "Adventure": 15,
      "Animation": 34,
      "Comedy": 3,
      "Crime": 14,
      "Drama": 6,
      "Family": 15,
      "Fantasy": 8,
      "Horror": 1,
      "Mystery": 4,
      "Romance": 1,
      "Science Fiction": 14,
      "TV Movie": 4,
      "Thriller": 4,
      "Western": 1
    }
  },
  "facetStats": {},
  "requestUid": "019bbcf4-a609-7701-8d82-d370611adfb3",
  "remoteErrors": {}
}

useNetwork requires the network experimental feature to be enabled.

useNetwork query parameter in GET /indexes/{:indexUid}/search

Passing useNetwork=true as a query parameter to GET /indexes/{:indexUid}/search has the same effect as passing useNetwork: true as a field parameter to POST /indexes/{:indexUid}/search

.queries[].useNetwork field in POST /multi-search

• useNetwork can also be passed as a field of the individual queries inside of a multi-search request.
• When used on a query in a non-federated search request, it has the same effect as on POST /indexes/{:indexUid}/search for that query
• When used on a query in a federated search request, the request is executed "as-if" one query per remote of the network had been performed.

Federated search example:

Multi-search request

{
    "federation": {
        "limit": 5
    },
    "queries": [
        {
            "q": "Batman returns",
            "indexUid": "mieli",
            "useNetwork": true
        },
        {
            "q": "Superman returns",
            "indexUid": "mieli",
            "useNetwork": true
        }
    ]
}

Multi-search response

{
  "hits": [
    {
      "id": 364,
      "title": "Batman Returns",
      "overview": "While Batman deals with a deformed man calling himself the Penguin, an employee of a corrupt businessman transforms into the Catwoman.",
      "genres": [
        "Action",
        "Fantasy"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/jKBjeXM7iBBV9UkUcOXx3m7FSHY.jpg",
      "release_date": 708912000,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 0,
        "weightedRankingScore": 1.0,
        "remote": "1"
      }
    },
    {
      "id": 1452,
      "title": "Superman Returns",
      "overview": "Superman returns to discover his 5-year absence has allowed Lex Luthor to walk free, and that those he was closest to felt abandoned and have moved on. Luthor plots his ultimate revenge that could see millions killed and change the face of the planet forever, as well as ridding himself of the Man of Steel.",
      "genres": [
        "Science Fiction",
        "Action",
        "Adventure"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/qIegbn6DSUYmggfwxOBNOVS35q.jpg",
      "release_date": 1151452800,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 1,
        "weightedRankingScore": 1.0,
        "remote": "0"
      }
    },
    {
      "id": 324249,
      "title": "Requiem for Krypton: Making 'Superman Returns'",
      "overview": "A detailed behind-the-scenes documentary on the making of Superman Returns.",
      "genres": [
        "Documentary"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/eC1XKswKSoyDyJXXZszLTuwUHli.jpg",
      "release_date": 1164672000,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 1,
        "weightedRankingScore": 0.9907407407407408,
        "remote": "1"
      }
    },
    {
      "id": 294294,
      "title": "Saltwater",
      "overview": "This American Indie drama follows several endearing characters as they wade through life seeking happiness, peace and ultimately, love. Will (Ronnie Kerr, Vampire Boys 2, Shut Up and Kiss Me) leaves the Navy after many years, soon reunites old friends and begins to start his new civilian life. His friend Rich (Bruce L Hart) tries to set him up with ruggedly handsome Josh (Ian Roberts-a former Australian professional rugby player, actor and model-Cedar Boys, Superman Returns, Little Fish). While there is immense chemistry between the two, timing and certain ideals never seem to align. When a shocking tragedy happens the two are paired up to pick up the pieces and sort through the after effects. Saltwater is a story about men of all ages, finding love, losing friends, navigating their way through life and knowing it's the journey rather then the destination that's important.",
      "genres": [
        "Romance",
        "Drama"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/bDnLdYqpH9abHo4ASMPKiInx8dm.jpg",
      "release_date": 1342310400,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 1,
        "weightedRankingScore": 0.966931216931217,
        "remote": "1"
      }
    },
    {
      "id": 142061,
      "title": "Batman: The Dark Knight Returns, Part 2",
      "overview": "Batman has stopped the reign of terror that The Mutants had cast upon his city.  Now an old foe wants a reunion and the government wants The Man of Steel to put a stop to Batman.",
      "genres": [
        "Action",
        "Animation",
        "Mystery"
      ],
      "poster": "https://image.tmdb.org/t/p/w500/arEZYd6uMOFTILne9Ux0A8qctMe.jpg",
      "release_date": 1357171200,
      "_federation": {
        "indexUid": "mieli",
        "queriesPosition": 0,
        "weightedRankingScore": 0.8697089947089947,
        "remote": "1"
      }
    }
  ],
  "processingTimeMs": 247,
  "limit": 5,
  "offset": 0,
  "estimatedTotalHits": 97,
  "requestUid": "019bbd3a-5106-70e0-94fc-f58b2f0c28c8",
  "remoteErrors": {}
}

Limitations

• Facet search referencing useNetwork are not supported
• The chat route cannot use useNetwork at the moment: doing so is not trivial implementation-wise, because chat route expects to be able to open the index (to fetch chat configs), but federated search only opens the indexes once during a short critical section.

By @​dureuill in #​6101

Federated search supports page and hitsPerPage

Pass federation.page and federation.hitsPerPage with the same meaning as in a regular search request to use exhaustive pagination in the federated search

By @​dureuill in #​6101

Speed up settings changes when removing searchable

The settings indexer is more efficient when users are removing searchable attributes from the searchable fields.

By @​VedantMadane in #​6109

🔒 Security

Solves a low-severity timing attack vulnerability on key comparison by using constant-time comparison

By @​curquiza in #​6077

🔩 Maintenance

Remove some unwanted dependencies

• Remove openssl from deps by @​dureuill in #​6117
• Fix geo-types version to avoid multiple rstar deps by @​dureuill in #​6118

New Contributors

❤️ Thanks again @​VedantMadane for the contribution to this release!

Full Changelog: meilisearch/meilisearch@v1.33.1...v1.34.0

v1.33.1: 🐞

Compare Source

In Meilisearch v1.33.1, we released a security fix that affected the dump import, improvements to vector store relevance in massive databases, and a fix related to ranking scores.

🔐 Security Fix

• Cloud users: you don't need to do anything. We found no evidence of exploitation of this vulnerability on Meilisearch Cloud.
• Open-source users: if you allow importing dumps from an untrusted source, we recommend you update to v1.33.1

All versions of Meilisearch before v1.33.0 are vulnerable to a path traversal vulnerability involving the dump import functionality.

Importing a specially crafted dump could result in giving access to the Meilisearch instance to arbitrary, specifically formatted files, present on the file system of the Meilisearch instance.

✨ Enhancements

• We updated the vector store to trigger linear scanning even on bigger databases, leading to improved performance and so better result quality when the search cutoff is reached. This applies in particular when the number of filtered candidates is small relative to the number of documents in the index by @​Kerollmops in #​6113.

🐛 Bug Fixes

• We fixed a bug where only the first non-blocking buckets were taken for the non-final ranking rules. This improves the quality of search results when the search cutoff triggers, especially when vector search and a sort are involved by @​Kerollmops in #​6113.

Full Changelog: meilisearch/meilisearch@v1.33.0...v1.33.1

v1.33.0: 🐞

Compare Source

✨ Enhancement

• Add /fields route to get all the fields of an index by @​YoEight in #​6082
  Adds a new POST /indexes/{indexUid}/fields endpoint that returns detailed metadata about all fields in an index. This endpoint provides comprehensive information about each field's configuration, including display, search, filtering, and localization settings.

• Implement parallel cleanup of old field IDs by @​Kerollmops in #​6100
  We reduce the time required to perform the dumpless upgrade for instances before v1.32.0 by multi-threading database fetches. By doing that, we noticed improvements from 2 hours and 50 minutes to a bit less than 7 minutes.

• Bump hannoy to 0.1.4-nested-rtxns by @​Kerollmops in #​6103
  We updated our internal vector store to speed up the dumpless upgrade by improving the graph rebuilding and improving the speed and relevance of the search by using the explore factor as a limit to stop document searches rather than the query limit.

🪲 Bug fixes

• Meilisearch can now rescale when the index reaches 2 TB by @​Vipul-045 in #​6095

🔒 Security

• Upgrade mini-dashboard to benefit security vulnerability fixes by @​curquiza in #​6091

🔩 Miscellaneous

• Fix tests by @​dureuill in #​6090
• Add verbose OpenAPI schema descriptions for documentation by @​qdequele in #​6071
• Add code samples to the generated openAPI file by @​curquiza in #​6028

Thank you @​Vipul-045 for your first contribution ❤️

v1.32.2: 🐟

Compare Source

🐛 Bug fixes

• Do not panic when doing a dumpless upgrade on empty indexes with configured embeddings by @​Kerollmops in #​6102

Full Changelog: meilisearch/meilisearch@v1.32.1...v1.32.2

v1.32.1: 🐟

Compare Source

🌈 Improvements

Skip cleaning up the field-ID-based databases

Introduce a MEILI_EXPERIMENTAL_DISABLE_FID_BASED_DATABASES_CLEANUP env var to opt out of the field ID-based database cleanup when upgrading a Meilisearch from versions inferior to 1.32.0.

by @​ManyTheFish in #​6096

Full Changelog: meilisearch/meilisearch@v1.32.0...v1.32.1

v1.32.0: 🐟

Compare Source

🌈 Improvements

Log Search performance trace

Introduces comprehensive progress tracking and logging for search operations in Meilisearch. It adds detailed timing information for each step of the search process, enabling better observability and performance analysis.

by @​ManyTheFish in #​6072

Extract document operations from payloads in parallel

We accelerate document indexing by processing a large number of tasks in batches or a large number of records in parallel. We expedited the preparation of the payloads by extracting the various changes and assigning internal IDs in parallel. We achieved a 7x speedup on a four-million-document insertion using four CPUs, and the performance scales with the number of CPUs.

The indexedDocuments field in tasks using skipCreation no longer precisely reflects the number of document operations performed, specifically for POST and PUT operations. This count may be higher than the actual number of operations, but it doesn't affect the computation; only the reported count is impacted. We prioritize speed over perfect accuracy here, and the documents are still correctly indexed as before.

by @​Kerollmops in #​6080

🐛 Bug fixes

Vector sort: Bucket documents with same similarity

Fixed vector sort bucketing so documents with identical similarity scores are grouped together, ensuring subsequent ranking rules are applied correctly.

by @​dureuill in #​6081

Properly Delete Documents from FID-Based Databases

Fixes a bug where changing searchableAttributes from ["*"] to a subset of fields left orphaned data in fid-based databases, causing corruption and warnings during search.

by @​ManyTheFish in #​6076

Rebuild the graph links when dumpless-upgrading

Bumps hannoy to v0.1.3-nested-rtxns, which fixes graph-related recall issues and adds a method to rebuild graph links to recover previously malformed graphs. Also fixes a minor issue in the dumpless upgrade flow where the upgrade description was not displayed correctly and related operations were not properly associated with the upgrade.

by @​Kerollmops in #​6055

🛠️ Maintenance and Misc.

Update JS SDKs tests to use pnpm instead of yarn

Updated the JavaScript SDK tests to use pnpm instead of yarn in CI workflows, switching the package manager across test configurations to ensure the SDK test suite runs correctly and consistently with the current tooling.

by @​Strift in #​6075

Adapt JS tests in SDK tests CI

Updated the SDK tests CI workflow for the JavaScript SDKs

by @​curquiza in #​6050

Bump lru from 0.16.2 to 0.16.3

Fix Stacked Borrows violation in IterMut.

by @​dependabot[bot] in #​6087

Full Changelog: meilisearch/meilisearch@v1.31.0...v1.32.0

v1.31.0: 🦃

Compare Source

🌈 Improvements

🗄️ [Enterprise Edition] Make the S3-streaming snapshots an Enterprise Edition feature

The recently introduced S3-streaming snapshots feature is now available as an Enterprise Edition feature. From now on, you'll need a license to use this feature when using Meilisearch as a self-hosted solution. Note that this version introduces a breaking change regarding the S3-streaming snapshot feature, which is no longer available in the Community Edition; however, on-disk snapshots remain accessible. You can read our BUSL license for contact information, if you like. Note that if you are using the Community Edition of Meilisearch between version 1.25 and this release, you can freely use the S3 Streaming feature without an Enterprise Edition license.

by @​Kerollmops in #​6057

🔏 [Enterprise Edition] Support AWS IRSA to authenticate to S3 for snapshotting

We just introduced support for IRSA authentication to do snapshots on AWS. IRSA allows the use of short-lived access and secret keys to upload snapshots through S3. This feature is available under the Enterprise Edition and can be accessed through two experimental CLI parameters.

by @​paulden in #​6044

✍️ Allow strict document update without creating missing documents

Adds an optional skipCreation boolean query parameter to POST and PUT on /indexes/{index}/documents. When set to true, documents that don't exist in the index are silently ignored rather than created. Default is false, preserving existing behavior.

by @​YoEight in #​6053

🐛 Bug fixes

• Fix actix payload error handling by @​Kerollmops in #​6052
• Bump hannoy to v0.1.2 by @​Kerollmops in #​6063

🛠️ Maintenance and Misc.

• Run windows in the PR CI by @​Kerollmops in #​6058
• Make CI test titles more human friendly by @​curquiza in #​6062

v1.30.1: 🐸

Compare Source

What's Changed

🐛 Bug Fixes

Fix task attribution during index swap to prevent cross-index task loss by @​YoEight in #​6059

This change fixes the parenting of the tasks when doing an index swap.

Hotfix: log a warning when FieldidsWeightsMap is missing entry by @​ManyTheFish in #​6064

This hotfix avoids the encounter of following error during a search request:

{
  "message": "Inside `<query>`: Index `<index_uid>`: internal: missing <field_id> in the fieldids weights mapping.",
  "code": "internal",
  "type": "internal",
  "link": "https://docs.meilisearch.com/errors#internal"
}

🫂 New Contributors

• @​YoEight made their first contribution in #​6059

v1.30.0: 🐸

Compare Source

🌈 Improvements

Up and down network scaling

Since v1.19.0, Meilisearch Enterprise Edition allows the automatic sharding of documents over multiple Meilisearch instances, enabling scaling to more documents than a single instance would accommodate.

Meilisearch v1.30.0 adds the ability to modify the number of participants in sharding, without having to start over sending documents to a new cluster containing the number of desired machines.

To make this possible, Meilisearch v1.30.0 introduces breaking changes. These breaking changes only affect the users of the experimental network feature who enabled automatic sharding (network.sharding = true). Users of the stable features are not affected.

Usage

Setting up the initial network

1. Pick a leader machine that will receive all tasks. Calling a route that creates a dcument, settings or network-related task on a machine that is not the leader will now return an error.
2. Send your network topology to the leader by calling PATCH /network:

// PATCH /network
{
  "self": "ms0",
  "leader": "ms0", // must be equal to `self`
  "remotes": {
    "ms0": {
      "url": "URL_OF_MS0",
      "searchApiKey": "SEARCH_API_KEY_OF_MS0",
      "writeApiKey": "WRITE_API_KEY_OF_MS0",
    },
    "ms1": {
      "url": "URL_OF_MS1",
      "searchApiKey": "SEARCH_API_KEY_OF_MS1",
      "writeApiKey": "WRITE_API_KEY_OF_MS1",
    }
  }
}

3. The network is automatically propagated to other members of the network.
4. Send settings and documents as usual, but exclusively to the leader. They will be propagated to all participants in the network, and each participant will process a piece of the documents.

Details

Upgrading from v1.29 or earlier

1. We recommend using the experimental dumpless upgrade feature to go from Meilisearch v1.13+ to Meilisearch v1.29.
2. When using the experimental dumpless upgrade and the Meilisearch instance already has a network instance with sharding set to true, then the network object will be modified so that the leader is the first remote in alphabetic order. For instance, if you network contains remotes: A, B, C, the leader will be set to A.

Adding a new remote

1. Call PATCH /network on the leader with the information about the new remote:

{
  "remotes": {
  // add information about the new remote
  "ms2": {
      "url": "URL_OF_MS2",
      "searchApiKey": "SEARCH_API_KEY_OF_MS2",
      "writeApiKey": "WRITE_API_KEY_OF_MS2",
    }
  // information about existing remotes does not need to be repeated
}

4. The new network will be propagated from the leader to all remotes (including the new remote ms2).
5. All remotes will register a new networkTopologyChange task that will "rebalance" the documents between the existing remotes and the new remote, that is, ms0 and ms1 will send parts of their documents to ms2

Removing a remote

1. Call PATCH /network on the leader by setting any removed remote to null:

{
  "remotes": {
  // set removed remote to null
  "ms2": null
  // information about existing remotes does not need to be repeated
}

2. The new network will be propagated from the leader to all remotes (including to the old remote ms2 that will then no longer participate in the network).
3. All remotes will register a new networkTopologyChange task that will "rebalance" the documents between the remaining remotes, that is, ms2 will send its documents between ms0 and ms1

List of changes

List of changes, some of which are breaking

1. Breaking change: The Network object returned or edited by the /network route is modified in the following way:
   • the sharding boolean is removed
   • a leader field is added as an optional string. When it is not null, it has the same effect (and more) than having sharding: true in the previous iteration of the Network object. The leader is used as a check when receiving task creation requests.
   • a version field is added as a UUID, defaulting to the null UUID. The version is used when processing tasks.
2. Breaking change: When a network.leader is set, calling one of the following routes will fail with not_a_leader error if the target's network.self is not the same as its network.leader:
   • POST /indexes
   • PATCH/DELETE /indexes/{:indexUid}
   • POST/PUT/DELETE /indexes/{:indexUid}/documents
   • POST /indexes/{:indexUid}/documents/delete
   • POST /indexes/{:indexUid}/documents/delete-batch
   • POST /indexes/{:indexUid}/documents/edit
   • PATCH/DELETE /indexes/{:indexUid}/settings and settings sub-routes
   • PATCH /network if the target is the new leader
   • POST /swap-indexes
3. Breaking change: when a leader is set, PATCH /network no longer returns a Network object. Rather, it spawns a new NetworkTopologyChange task, and returns the summarized task view.
4. Breaking change: Tasks are duplicated by the leader to the entire network when calling the following routes:
   • POST /indexes (new to this PR)
   • PATCH/DELETE /indexes/{:indexUid} (new to this PR)
   • POST/PUT/DELETE /indexes/{:indexUid}/documents (was already the case before this PR)
   • POST /indexes/{:indexUid}/documents/delete (was already the case before this PR)
   • POST /indexes/{:indexUid}/documents/delete-batch (was already the case before this PR)
   • POST /indexes/{:indexUid}/documents/edit (was already the case before this PR)
   • PATCH/DELETE /indexes/{:indexUid}/settings and settings sub-routes (new to this PR)
   • PATCH /network if the target is the new leader (new to this PR)
   • POST /swap-indexes (new to this PR)
5. New NetworkTopologyChange tasks that perform the following:
   1. Execute any remaining task to process with a network.version lower than the network task's version
   2. Iterate over all documents in all indexes, determine their new shard, and send the document to the remote that must now have it in the new version , deleting it from the local DB
      • The export route code has been factored and specialized to allow this
      • Should the export to a remote fail, the corresponding documents are kept locally
      • If there are no documents to send for an index, still call the documents addition with an empty payload and appropriate headers containing the expected metadata
      • If there are no documents to send for an entire remote, call the network route with specific headers containing the expected metadata
   3. Wait for and process tasks from the remotes of the previous version of the network.
6. Breaking change: When importing dumps, we drop the self and leader from the network
7. Network topology change tasks can be cancelled. In this case the state will be the current one (any moved documents will stay that way). Cancellation needs to happen on all machines.

by @​dureuill in #​6000

🛠️ Maintenance

• Fix macos-amd64 compilation by @​dureuill in #​6037
• Move to Rust v1.91.1 by @​Kerollmops in #​6039
• Remove risk of command injection in CI by @​curquiza in #​6041

Asset availability note

🌈 The Meilisearch binary is available again for meilisearch-enterprise-macos-amd64 and meilisearch-macos-amd64. It was not available for Meilisearch v1.29.

Full Changelog: meilisearch/meilisearch@v1.29.0...v1.30.0

v1.29.0: 🐑

Compare Source

Build compatibility note

The git binary must now be present at build time to populate the commitSha1 field of the /version field.

This change was made for build performance reasons.

by @​dureuill in #​6030

🌈 Improvements

Another piece of the settings indexer is available: searchable and exact attributes

The new settings indexer scales better, supports near-instant cancellations and displays the progress of the indexing operation.

Previously, the new settings indexer was enabled only if the only changes in a settings batch were to embedder settings. In Meilisearch v1.29.0, the new settings indexer will be enabled if the change is any combination of:

• searchableAttributes
• exactAttributes
• proximityPrecision
• embedders (as before)

Any other change to settings appearing in a batch will cause the batch to use the legacy settings indexer.
Additionally, the new settings indexer is currently disabled by default for Cloud users.

OSS users who would like to disable the new settings indexer should pass the MEILI_EXPERIMENTAL_NO_EDITION_2024_FOR_SETTINGS environment variable to true.

by @​Kerollmops in #​5983

Enable the new vector store by default for new indexes

Meilisearch v1.21.0 introduced a new vector store backend providing better performance and relevancy.

Starting with v1.29.0, any newly created index will default to the new backend.
Existing indexes will be left unchanged.

It is still possible to explicitly choose the vector store backend, please refer to the relevant experimental feature discussion page.

by @​Kerollmops in #​6004

Support more models for huggingFace embedder

You can now select models with the XLM Roberta architecture when generating embeddings locally on CPU or GPU with the huggingFace embedder.

by @​qdequele in #​6018

🦋 Bug fixes

• Fix rare internal error in the experimental vector store by @​Kerollmops in #​6032

🔧 Maintenance and Misc.

Last week was a Quality of Life week, and while we still had improvements in the pipe, the bulk of our work was dedicated to maintenance tasks.

Most notably, the CI is now faster, going from over one hour to less than 30 minutes, and also more reliable as it automatically tests the dumpless upgrade.

• New workload to run declarative tests. This is, in particular,

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.