Expressions not working with global query filters

[Seramis]

I'm trying to use Expressionify together with global query filters and it seems it is not working.

To reproduce:

1. Create entity Tenant
2. Create entity User, which has Tenant and TenantId property
3. Create entity Product, which has User and UserId property
4. Create Expressionify method public static int GetTenantId(Product p) => p.User.TenantId; into Product entity
5. In DbContext.OnModelCreating(), add filter: modelBuilder.Entity<Product>().HasQueryFilter(x => x.GetTenantId() == 3);

Exception:

Message: 
System.TypeLoadException : Method 'PrecompileQuery' in type 'Clave.Expressionify.ExpressionableQueryCompiler' from assembly 'Clave.Expressionify, Version=6.6.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.

Stack Trace: 
ExpressionifyDbContextOptionsExtension.ApplyServices(IServiceCollection services)
ServiceProviderCache.ApplyServices(IDbContextOptions options, ServiceCollection services)
ServiceProviderCache.<GetOrAdd>g__BuildServiceProvider|4_1(IDbContextOptions _, ValueTuple`2 arguments)
<>c.<GetOrAdd>b__4_0(IDbContextOptions contextOptions, ValueTuple`2 tuples)
ConcurrentDictionary`2.GetOrAdd[TArg](TKey key, Func`3 valueFactory, TArg factoryArgument)
ServiceProviderCache.GetOrAdd(IDbContextOptions options, Boolean providerRequired)
DbContext.get_ContextServices()
DbContext.get_InternalServiceProvider()
IServiceProvider>.get_Instance()
InfrastructureExtensions.GetService(IInfrastructure`1 accessor, Type serviceType)
InfrastructureExtensions.GetService[TService](IInfrastructure`1 accessor)
AccessorExtensions.GetService[TService](IInfrastructure`1 accessor)
DatabaseFacade.get_Dependencies()
IDatabaseFacadeDependenciesAccessor.get_Dependencies()
RelationalDatabaseFacadeExtensions.OpenConnection(DatabaseFacade databaseFacade)
AppDbContext.ctor() line 271
EntityFilterEfTests.CreateContext() line 226
EntityFilterEfTests.Products_TenantNotSet_Returns_AllProducts() line 155
RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

[mariusGundersen]

Seems like this is the method not implemented https://github.com/dotnet/efcore/blob/e2a5f01d041d97630f4852371a6614497f69bdb1/src/EFCore/Query/Internal/IQueryCompiler.cs#L61

[mariusGundersen]

Hang on, which version of expressionify are you using? It looks like v6, but you need v9

[Seramis]

Hang on, which version of expressionify are you using? It looks like v6, but you need v9

I used the latest available from nuget.org at the time when i made this post. (about 3 days ago)

EDIT: Wait. VS has installed 6.6.0. For some reason, latest version was not the default option. I'll retest!

[Seramis]

Sorry for the confusion. The VS had 6.6.0 as a default version. I updated to 9.0.0 and got this error now:

Message: 
System.InvalidOperationException : The LINQ expression 'DbSet<Product>()
    .Where(p => p.GetTenantId() == 3)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

Stack Trace: 
QueryableMethodTranslatingExpressionVisitor.Translate(Expression expression)
QueryCompilationContext.CreateQueryExecutorExpression[TResult](Expression query)
QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
Database.CompileQuery[TResult](Expression query, Boolean async)
QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
<>c__DisplayClass11_0`1.<ExecuteCore>b__0()
CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
QueryCompiler.ExecuteCore[TResult](Expression query, Boolean async, CancellationToken cancellationToken)
QueryCompiler.Execute[TResult](Expression query)
EntityQueryProvider.Execute[TResult](Expression expression)
EntityQueryable`1.GetEnumerator()
IEnumerable<TEntity>.GetEnumerator()
List`1.ctor(IEnumerable`1 collection)
Enumerable.ToList[TSource](IEnumerable`1 source)
EntityFilterEfTests.Products_TenantNotSet_Returns_AllProducts() line 161
RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

[mariusGundersen]

I finally had time to look into this. I have created a pre-release version that maybe you can test: #35

It only works with one of the two evaluation modes, specifically the LimitedCompatibilityButCached mode (which is the faster one). The problem seems to be the order in which the query compilation decoraters are added. Maybe there is a better way to solve this, but now it seems like I need to get the existing query compiler to wrap my compiler, rather than the other way around (I wrap the existing one) which is how it's been working until now.

[Seramis]

Hi @mariusGundersen,

Sorry for delayed answer.

How could I access the pre-release version to try it out? (As nupkg or dll file for example? Or some pre-release version in nuget repository?)

Thanks!

[mariusGundersen]

Just search for pre releases when doing the nuget install, or pick the version explicitly: https://www.nuget.org/packages/Clave.Expressionify/9.2.0-global-query-filter.70

[Seramis]

Thank you!

I got some time now to try it out.
I'm getting the slightly different error as compared to earlier:

System.InvalidOperationException: 'The LINQ expression 'DbSet<Track>()
    .Where(t => t.Album == null ? null : (int?)t.Album.Artist.ArtistId == (int?)100)' could not be translated. Additional information: Translation of member 'Album' on entity type 'Track' failed. This commonly occurs when the specified member is unmapped.
Translation of member 'Album' on entity type 'Track' failed. This commonly occurs when the specified member is unmapped. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'

I currently have a test-setup with the widely distributed example database "Chinook" in SQLite: https://github.com/lerocha/chinook-database

Nuget's involved are:

• EF Core 9.0.7
• Clave.Expressionify 9.2.0-global-query-filter.70
• Clave.Expressionify.Generator 9.2.0-global-query-filter.70

I tried to add GetArtistId() on Track entity and define a fixed global filter based on that.

I made the following changes:

My DbContext (I use scaffolding, so there are some partial methods and other such thing involved):

...
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
	optionsBuilder
		.UseSqlite("Data Source=chinook.db")
		.UseExpressionify(o => o.WithEvaluationMode(ExpressionEvaluationMode.LimitedCompatibilityButCached));
}

partial void OnModelCreatingPartial(ModelBuilder modelBuilder)
{
	modelBuilder.Entity<Track>().HasQueryFilter(x => x.GetArtistId() == 100);
}
...

For Track entity made the extension class:

public static partial class TrackExtensions
{
	[Expressionify]
	public static int? GetArtistId(this Track track)
		=> track.Album == null ? null : track.Album.Artist.ArtistId;
}

And then making a query. It fails to translate the query.

var q = dbCtx.Track.Select(x => x.TrackId).Take(3);

Console.WriteLine(q.ToQueryString());

When I don't use Expressionify's capabilities and define the global filter as below (identical to the expression above), it works:

modelBuilder.Entity<Track>().HasQueryFilter(x => (x.Album == null ? null : x.Album.Artist.ArtistId) == 100);

It almost seems that because of expressionify it is unable to follow associations as the error complains it is unable to work with Track.Album property. That property is defined like this:

public int? AlbumId { get; set; }
public virtual Album? Album { get; set; }

... and OnModelCreating in DB context is like this:

modelBuilder.Entity<Track>(entity =>
{
...
	entity.HasOne(d => d.Album).WithMany(p => p.Tracks).HasForeignKey(d => d.AlbumId);
...
}

I hope I didn't miss anything. Let me know if I can provide any additional information.

EDIT:

I also noticed that even simple .Where() clause has broken with 9.2.0-global-query-filter.70, while it works properly with 9.1.0. Example query:

var q = dbCtx.Track.Where(x => x.GetArtistId() == 100).Select(x => x.TrackId).Take(3);

[mariusGundersen]

Then I'm afraid it might not be possible to implement so it works in global query filters 😕

[Seramis]

That is unfortunate.

Just for background information, EntityFrameworkCore.Projectables can do that if a certain one-liner is included based on this: EFNext/EntityFrameworkCore.Projectables#81 (And when a new version is released, it will be done automatically by the library.)

I actually found Expressionify library through the Projectables project as I wanted to try out all possible options and assess which one would be most fit for my requirements.

I'm not versed in either library's internals, so I apologize if this reference is irrelevant. But maybe there is something to be looked into in that library that could help this implementation as well.

[mariusGundersen]

I wish I had the time to look into it properly, unfortunately I don't and I'm on holiday right now. Projectables seems to have sorted it some other way, according to the commit history. That way might be implementable here as well, but I don't have the equipment to try it out while on holiday.

[Seramis]

Completely understandable.

Please, enjoy your holiday. :)