Forwarded Requests are not passing `limit` or `type`

[jason-fox]

Orion 1.10.0

I have two registrations as shown:

curl -s -iX POST 'http://localhost:1026/ngsi-ld/v1/csourceRegistrations/' \
-H 'Link: <http://context/user-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
-H 'Content-Type: application/json' \
-d '{
    "type": "ContextSourceRegistration",
    "information": [
        {
            "entities": [
                {"type": "Animal"}, {"type": "Device"}
            ]
        }
    ],
    "mode": "redirect",
    "operations": ["redirectionOps"],
    "endpoint": "http://farmer"
}'

curl -s -iX POST 'http://localhost:1026/ngsi-ld/v1/csourceRegistrations/' \
-H "NGSILD-Tenant: farmer" \
-H 'Link: <http://context/user-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
-H 'Content-Type: application/json' \
-d '{
    "type": "ContextSourceRegistration",
    "information": [
        {
            "entities": [
                {"type": "Animal"}, {"type": "Device"}
            ]
        }
    ],
    "mode": "inclusive",
    "operations": ["federationOps"],
    "endpoint": "http://devices"
}'

I make the following query on Broker A

curl -s -iX GET 'http://localhost:1026/ngsi-ld/v1/entities/?type=Animal&limit=100' 

This is forwarded to Broker B (farmer)

/ngsi-ld/v1/entities?pick=id&type=Animal on Broker B - Without passing the limit so I get the default limit of 20 Animal entities returned. There are 80+ Animal entities on Broker B

Similarly when I make the following query on Broker A:

curl -s -iX GET 'http:localhost:1026/ngsi-ld/v1/entities/urn:ngsi-ld:Device:cow001'

This is forwarded to Broker B (farmer) as:

/ngsi-ld/v1/entities/urn:ngsi-ld:Device:cow001?options=sysAttrs without passing type therefore the registration for retrieving type=Devices from Broker C is not fired and I am not getting the latest information from C (devices)

[cfreyfh]

Hi,

when you try to fetch
curl -s -iX GET 'http://localhost:1026/ngsi-ld/v1/entities/?type=Animal'

you should get a header field named NGSILD-EntityMap.

If you set this to your further requests, offset and limit should work (without type as parameter).

For example:
curl -s -iX 'http://localhost:1026/ngsi-ld/v1/entities?offset=20&limit=10' -H "NGSILD-EntityMap: urn:ngsi-ld:entity-map:19abc694-985a-11f0-95b5-79a86b89ca1e"

where urn:ngsi-ld:entity-map:19abc694-985a-11f0-95b5-79a86b89ca1e is the entityMap from your first request.

[jason-fox]

Yes that would be a workaround, but I'm not making "further requests" in this case - in a single request I want to retrieve 100 entities. The actual location of those Entities (broker A or broker B) doesn't matter to me. At it is, I have zero on A and 100 on B, but Orion 1.10 was returning only 20.

[cfreyfh]

Well, you could make a request /ngsi-ld/v1/entities?type=Animal&limit=0&count=true then you will get a header field for the entityMap and a field for the entity count:

NGSILD-EntityMap: urn:ngsi-ld:entity-map:6e807f5c-9ab7-11f0-95b5-79a86b89ca1e
NGSILD-Results-Count: 31

In the next call you then can use the entityMap for getting the number of entities you want.

[jason-fox]

This is still a bug though, since it is not compliant with 5.7.2 Query Entities in the ETSI specification - specifically this line:

• For any exclusive, redirect and inclusive Context Source Registrations, the request is forwarded for remote querying by matching endpoints.

The spec doesn't say "type and limit are removed and the request is forwarded", it just says "the request is forwarded"

[kzangeli]

Well, if I were to forward the limit param, it would stop working. The type could be forwarded but, it's not really needed. It was already forwarded when the entity map was created. Now we have the ID, and the receiving broker already has its own entity map where the corresponding registration is known (only reason for the type to be forwarded).

This is pure implementation details really.

[jason-fox]

True, you could forward a HEAD /ngsi-ld/v1/entities?type=Animal&entityMap=true, and then read the NGSILD-EntityMap header as suggested above.