Conflicting shard name

[rmarronnier]

cable-cr/cable#74 brought me here :
The following shard.yml triggers an error :

dependencies:
  redis:
    github: jgaskins/redis
  mosquito:
    github: hugopl/sidekiq.cr

Because the sidekiq shard has

dependencies:
  redis:
    github: stefanwille/crystal-redis
    version: ">= 2.6"

One solution would be to change the name redis to something else in

redis/shard.yml

Line 1 in 8d177ad

name: redis

I'm posting the 'mirror' issue in https://github.com/stefanwille/crystal-redis

[jwoertink]

One solution would be to change the name redis to something else in

I originally had that thought a while ago (#13), but when the other shard went dormant for over a year, I assumed this was the successor so I actually helped port both Cable and Mosquito over to this shard.

It does look like the other one is starting to pick back up, and unfortunately with how Crystal and Shards works, there's no way to include both shards at the same time. It's definitely a tricky situation.

[jgaskins]

unfortunately with how Crystal and Shards works, there's no way to include both shards at the same time

This is my biggest gripe with shards as an ecosystem. 😞 It relies on names of shards and the Crystal constants they define never having collisions but doesn't provide any ways to ensure that. There's currently no way to see the entire ecosystem of shards (one of the tradeoffs of decentralization), so you can't even go out of your way to intentionally guarantee this.

Then there's how the same problem manifests with forks of shards. For example, randomstate/openssl_ext last saw a commit in 2019, so in one of my apps I used this fork (which I guess is no longer listed as a fork?). When I later added a shard that depends on this fork (indirectly, even), since the two forks' APIs had diverged incompatibly, they could not coexist in the same project.

So, as much as I wish this were a problem with individual shards, it's more systemic and, as the ecosystem grows, this issue is likely to become more common. shards solves a technical problem, which is great and was definitely needed at the time, but there's still a sociotechnical component that it ignores completely and I'm not sure can even be solved in a backwards-compatible way.

[rmarronnier]

There's currently no way to see the entire ecosystem of shards (one of the tradeoffs of decentralization), so you can't even go out of your way to intentionally guarantee this.

made me find these two shards issues : crystal-lang/shards#137 and crystal-lang/shards#283

I'm really convinced by this type of solution : crystal-lang/shards#283 (comment)

shards coupled with a central source of truth (existing shard names) would stop the festering of avoidable conflicts AND be helpful for the shard discovery. It won't be 100% backward compatible but it'd be worth it IMO.

[jgaskins]

Thanks for the links! I'd seen (and participated in) a few discussions with a few Crystal core members about this problem before, but couldn't remember where they were. Sounds like it might be time to revisit that.