API filter query returns an empty list

[jose]

Hi,

The following code returns an empty list. I was expecting to return all artifacts that match the query. Am I missing something?

# Import the DatabaseAPI
from bugswarm.common.rest_api.database_api import DatabaseAPI

# Create instance of DatabaseAPI
bugswarmapi = DatabaseAPI()

# Custom API filter query
api_filter_query = '{"lang":{"$in":["Java","Python"]},"reproduce_successes":{"$gt":0},"build_system":{"$in":["Maven","Ant","Gradle","NA"]},"classification.code":["Yes","Partial","No","NA"],"classification.build":["Yes","Partial","No","NA"],"classification.test":["Yes","Partial","No","NA"]}'

# Fetching artifacts matching query
l = bugswarmapi.filter_artifacts(api_filter_query)
print(l)

--
Best,
Jose

[hao-n]

Hi Jose,

Thanks for asking. It seems like that the query has some syntax issues for classification.* fields. You will probably need to add $in or other operators like this.

api_filter_query = '{"lang":{"$in":["Java","Python"]},"reproduce_successes":{"$gte":0},"build_system":{"$in":["Maven","Ant","Gradle","NA"]},"classification.code":{"$in":["Yes","Partial","No","NA"]},"classification.build":{"$in":["Yes","Partial","No","NA"]},"classification.test":{"$in":["Yes","Partial","No","NA"]}}'

Please let us know if you have more questions!

[jose]

Thanks @hao-n. In that case, the API Filter Query generated by the BugSwarm dataset webpage is buggy as I got it from the BugSwarm dataset webpage.

1. Go to https://www.bugswarm.org/dataset/.
2. Push the Download API Filter Query button which generates the following code

# INSTALLATION: Requires Python3
#     pip3 install bugswarm-common

# USAGE: Requires a Token
#     To receive a token contact the BugSwarm team by dropping a message here: http://www.bugswarm.org/contact/

# FILTERING AND RETRIEVING ARTIFACT METADATA
#     find_artifact: Returns the metadata about an artifact
#     list_artifacts: Returns a list of artifacts' metadata that are Java or Python and have at least one reproduce success.
#     filter_artifacts: Returns a list of artifacts' metadata based on a custom filter.

# USING API FILTER QUERY
# Import the DatabaseAPI
from bugswarm.common.rest_api.database_api import DatabaseAPI

# Import Token
from bugswarm.common.credentials import DATABASE_PIPELINE_TOKEN

# Create instance of DatabaseAPI
bugswarmapi = DatabaseAPI(token=DATABASE_PIPELINE_TOKEN)

# Custom API filter query
api_filter = '{"lang":{"$in":["Java","Python"]},"reproduce_successes":{"$gt":0},"build_system":{"$in":["Maven","Ant","Gradle","NA"]},"classification.code":["Yes","Partial","No","NA"],"classification.build":["Yes","Partial","No","NA"],"classification.test":["Yes","Partial","No","NA"]}'

# Fetching artifacts matching query
bugswarmapi.filter_artifacts(api_filter)


# For using the API to retrieve metadata for one artifact.
# bugswarmapi.find_artifact('<Insert Image Tag>')

# For using the API to retrieve metadata for all artifacts that are Java or Python and have at least one reproduce success.
# bugswarmapi.list_artifacts()

[jose]

@hao-n, as I don't have an API token, the execution of your fixed query reaches the rate limited in a couple seconds:

You are being rate limited. You can request a API token to remove this limit.
http://www.bugswarm.org/contact/
Traceback (most recent call last):
  File "/tmp/APIFilter.py", line 11, in <module>
    l = bugswarmapi.filter_artifacts(api_filter_query)
  File "/opt/homebrew/lib/python3.13/site-packages/bugswarm/common/rest_api/database_api.py", line 119, in filter_artifacts
    return self._filter(DatabaseAPI._artifacts_endpoint(), api_filter)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/bugswarm/common/rest_api/database_api.py", line 1057, in _filter
    return self._iter_pages(url)
           ~~~~~~~~~~~~~~~~^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/bugswarm/common/rest_api/database_api.py", line 1022, in _iter_pages
    next_resp.raise_for_status()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/lib/python3.13/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: http://www.api.bugswarm.org/v1/artifacts?where=%7B%22lang%22:%7B%22$in%22:%5B%22Java%22,%22Python%22%5D%7D,%22reproduce_successes%22:%7B%22$gte%22:0%7D,%22build_system%22:%7B%22$in%22:%5B%22Maven%22,%22Ant%22,%22Gradle%22,%22NA%22%5D%7D,%22classification.code%22:%7B%22$in%22:%5B%22Yes%22,%22Partial%22,%22No%22,%22NA%22%5D%7D,%22classification.build%22:%7B%22$in%22:%5B%22Yes%22,%22Partial%22,%22No%22,%22NA%22%5D%7D,%22classification.test%22:%7B%22$in%22:%5B%22Yes%22,%22Partial%22,%22No%22,%22NA%22%5D%7D%7D&page=9

Is there any way to execute this simple query without reaching the limit? (PS: I am aware that I could ask for an API token.)

[hao-n]

Hi @jose , thanks for reporting the issues!

We've fixed our website to generate correct API queries. Please try again and let us know if you encounter any other issues.

We've also updated the rate limit to ensure that a query for the full list of artifacts will not trigger 429 status. Please let us know if you have any other questions. Thank you!