support IN clause optimization for sharding key

[Fottas]

Related to #36454

[strongduanmu]

Hi @Fottas, this PR looks very complicated. Could you please submit an issue first to describe your design?

[Fottas]

Hi @Fottas, this PR looks very complicated. Could you please submit an issue first to describe your design?

Hi @strongduanmu:
Issue Description: IN Predicate Sharding Optimization

Problem: Currently ShardingSphere sends all values in IN predicates to every shard, causing unnecessary data transfer and computation. For example, WHERE id IN (1,2,3,4) is sent as-is to all shards, even though based on sharding keys we can determine that values 1,3 belong to shard A while values 2,4 belong to shard B.
Solution: Route-aware IN predicate splitting that distributes values only to their target shards based on sharding algorithms.

Technical Design
Implementation Principle: Parse each value in IN expressions, invoke existing sharding algorithms to calculate target shards, then rewrite IN clauses for each RouteUnit to contain only values belonging to that shard.

Core Components:

1. ShardingInPredicateValue - Encapsulates IN values with their target route information
2. ShardingInPredicateToken - SQL token that rewrites IN clauses per route, filtering values and parameters
3. ShardingInPredicateTokenGenerator - Analyzes IN expressions and calculates shard distribution using existing sharding
   algorithms
4. ParameterFilterable - Cross-module decoupling interface that enables parameter filtering capability between modules, since ShardingInPredicateToken is in the sharding module while RouteSQLRewriteEngine is in the infra module

Key Features:

• Supports both standard and complex sharding strategies
• Handles parameter markers and literals
• Optimizes single-value cases to equality comparisons
• Achieves module decoupling through interface abstraction, allowing core engine to work without direct dependencies on sharding token implementations

[Fottas]

Hi @strongduanmu, I've also created issue #36454 to track this design discussion as requested.