CloudKit Web Services QueryFilter.in() returns HTTP 400 'Unexpected input'

[leogdion]

Description

CloudKit Web Services is returning HTTP 400 "BadRequestException: Unexpected input" when using QueryFilter.in() operator to query records by an array of field values.

Environment

• MistKit Version: 1.0.0-alpha.3
• Swift Version: 6.2
• Platform: Linux (Ubuntu 24.04 in GitHub Actions), macOS (local testing)
• CloudKit Database: Public
• CloudKit Environment: Development

Reproduction Steps

Code

let guids = ["guid1", "guid2", "guid3", ...] // Array of 97 GUIDs
let filters: [QueryFilter] = [.in("guid", guids.map { FieldValue.string($0) })]
let records = try await recordOperator.queryRecords(
  recordType: "Article",
  filters: filters,
  sortBy: nil,
  limit: 200,
  desiredKeys: nil
)

Error Response

CloudKit queryRecords failed with response: badRequest(
  MistKit.Components.Responses.BadRequest(
    body: MistKit.Components.Responses.BadRequest.Body.json(
      MistKit.Components.Schemas.ErrorResponse(
        uuid: Optional("f8d8f0c2-0cec-48fa-87e6-db467ab67bfb"),
        serverErrorCode: Optional(MistKit.Components.Schemas.ErrorResponse.serverErrorCodePayload.BAD_REQUEST),
        reason: Optional("BadRequestException: Unexpected input"),
        redirectURL: nil
      )
    )
  )
)

What We've Tested

✅ Works

• Single filter with .equals() operator
• Single filter with .lessThan(), .greaterThan() operators
• Empty array passed to .in() (from QueryFilterTests)
• Small arrays (2 values) passed to .in() (from QueryFilterTests)

❌ Fails

• .in() operator with large array (97 values)
• .in() operator combined with other filters (e.g., .equals() + .in())
• .in() operator with medium arrays (20-97 values) - all sizes tested fail

🔍 Attempted Workarounds

1. Removed additional filters: Query only with .in() alone - still fails
2. Reduced batch size: Tried 20 GUIDs instead of 97 - still fails
3. In-memory filtering: Removed feedRecordName filter, filter results after query - still fails

Analysis

Possible Root Causes

1. CloudKit Web Services Limitation:

   • The .in() operator may not be fully supported by CloudKit Web Services REST API
   • May have undocumented size limits on array values
   • Native CloudKit framework may have different capabilities than Web Services

2. MistKit Serialization Issue:

   • FilterBuilder may not be correctly serializing .in() filter to CloudKit Web Services format
   • Array of FieldValues might need different JSON structure
   • OpenAPI schema might not match actual CloudKit Web Services requirements

3. Query Complexity:

   • CloudKit may reject queries that exceed certain complexity thresholds
   • Array size might contribute to query complexity metric

Temporary Workaround

We've temporarily disabled GUID-based deduplication in CelestraCloud:

// TEMPORARY: Skip GUID query due to CloudKit Web Services .in() operator issue
// TODO: Fix query or implement alternative deduplication strategy
let existingArticles: [Article] = []

Impact: Duplicate articles will be created on subsequent feed updates.

Requested Investigation

1. Review FilterBuilder Implementation:

   • Check how .in() operator is serialized to CloudKit Web Services JSON
   • Compare against CloudKit Web Services API documentation
   • Verify OpenAPI schema matches CloudKit's actual requirements

2. Test CloudKit Web Services Limits:

   • Determine maximum array size for .in() operator
   • Test if issue occurs with smaller arrays (10, 50, 100 values)
   • Check if query complexity limits apply

3. Alternative Query Strategies:

   • Individual queries per GUID (slow but reliable)
   • Cursor-based pagination with different filter approach
   • Use different operator (e.g., multiple OR conditions)

References

• CelestraCloud Issue: Add Scheduled Job CelestraCloud#19
• Failing Workflow: https://github.com/brightdigit/CelestraCloud/actions/runs/20580086167
• Working Workflow (after workaround): https://github.com/brightdigit/CelestraCloud/actions/runs/20580501682
• Related Code:
  • ArticleCloudKitService.queryArticleBatch() - CelestraCloud
  • ArticleSyncService.syncArticles() - CelestraCloud
  • QueryFilter.in() - MistKit/Sources/MistKit/PublicTypes/QueryFilter.swift:95-96

Expected Behavior

The .in() operator should work with CloudKit Web Services, allowing queries for records where a field value matches any value in a provided array (similar to SQL's IN operator).

Additional Context

This issue blocks GUID-based duplicate detection in RSS feed processing. Without .in() queries, we cannot efficiently check which articles already exist in CloudKit before creating new ones.

MistKit's QueryFilterTests show that .in() works with small arrays, suggesting this may be a size-related issue or a difference between test environment and actual CloudKit Web Services behavior.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Similar Issues

Related Issues

• Bug: Query operations don't handle pagination (200 record limit) #166
• Add KeyPath-based QueryFilter API for Type-Safe Filtering #149
• Add Foundation Predicate Support for Type-Safe CloudKit Queries (iOS 17+) #151
• Feature: Add pagination support with continuation markers for query results #145
• Add verbose error logging for failed record operations #190

🔗 Related PRs

#132 - Adding Bushel Example [merged]

👤 Suggested Assignees

• leogdion

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!