Slow database load

[mdziczkowski]

Could you please optimisze the database load because it works too slow - it takes up to (at latest) several minutes before any channels would appear?

Thanks

[freearhey]

The latest update (#2140) was supposed to speed up loading times, but if you have a slow internet connection, the site may still take a long time to load.

Unfortunately, I’ve run out of ideas on how to further optimize data loading at the moment.

If anyone has any suggestions, please feel free to share them in the Discussions.

[mdziczkowski]

hmm.. did you tryed to optimize the database itself? maybe the structure or the data inside are the bottleneck...

[freearhey]

here is our entire database https://github.com/iptv-org/database you can take a look for yourself

[mdziczkowski]

here is our entire database https://github.com/iptv-org/database you can take a look for yourself

Hmm... I would have a proposal that could work and help:

Replace ".csv" files format with ".yaml". This would allow you to create organised structures with base informations like (minimal example):

• station name
• optionally: network
• reference to metadata file
• if is blocked
• if is nsfw
• if is closed

and thereof increase of speed and efficiency.

The proposed structure, could allow to load on demand small yaml files with the additional metadata when the user would click the channel name.

[freearhey]

1. YAML is up to 40% larger than CSV and its structure has no effect on performance.
2. The loading speed of the home page has nothing to do with the loading of an individual channel.

[mdziczkowski]

ad. 1. YAML is up to 40% larger than CSV and its structure has no effect on performance.

Actually, that is not exactly true (it mainly depends how it's formated), plus I tell this from a practical side because I use it in my Python3 projects and don't see the latency result of beeing larger you described.

Currently, each csv file is quite big (even GitHub says so). Due using the proposed by me solution, you could reduce the files sizes by creating a small reference yaml file containing list of channel names with minimal amount of channel informations (like proposed) and the site could load remaining metadata and stream URL's from the file that would be there listed, when the user would enter the channel informative tab, like he/she does it currently.

If you want then I can try to propose programming (code) solution for it ;-)

ad. 2. The loading speed of the home page has nothing to do with the loading of an individual channel.

You seem to have missread or missinterpreted my initial message. I had addressed the slowness to the whole database and not to single channels ;-)