Model warmup and pausing the container ready notification until the model has warmed up has the potential to increase performance by decreasing the latency per inference, and in particular on the first instance of the API's use per tf-inference container.
Need to generate model warmup data (which we can do with: https://github.com/mylesagray/tensorflow-anpr/tree/master/dataset_prep/artificial), include in the anpr-serving container image and configure TF Serving to load the warmup data on start.
ref:
Model warmup and pausing the container ready notification until the model has warmed up has the potential to increase performance by decreasing the latency per inference, and in particular on the first instance of the API's use per tf-inference container.
Need to generate model warmup data (which we can do with: https://github.com/mylesagray/tensorflow-anpr/tree/master/dataset_prep/artificial), include in the anpr-serving container image and configure TF Serving to load the warmup data on start.
ref: