Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

web matches cache json folder when have too much files returns a php timeout error #68

Open
39ma opened this issue Oct 7, 2020 · 8 comments

Comments

@39ma
Copy link
Contributor

39ma commented Oct 7, 2020

Describe the bug
When in app/cache/matches folder stucks too many match json files (like mine had ~2000), then when discord bot creating a server, web return a php error of max execution time.

Possible fix*
I fixed it by running bash cronjob to clean old .json files from that folder, and it solved a problem to me.
PHP error points to this function. I tested it, DB is answering very fastly, but checking MATCHES_CACHE takes a long time if there are a lot of files.

protected function doesMatchIdExist(int $matchId): bool
    {
        // Check whether the matchId already exists in the database.
        $query = $this->db->query('SELECT matches.matchid FROM matches WHERE matchid = :matchId', [
            ':matchId' => $matchId
        ]);

        $matches = array_filter(scandir(self::MATCHES_CACHE), function ($item) {
            return !is_dir(self::MATCHES_CACHE . "/$item");
        });

        return $query->rowCount() !== 0 || in_array("$matchId.json", $matches);
    }
@B3none
Copy link
Member

B3none commented Oct 7, 2020

This is a good solution, I'll happily accept a PR for this.

@39ma
Copy link
Contributor Author

39ma commented Oct 7, 2020

5 7 * * * find /var/www/csgo-league-web/app/cache/matches -name '*.json' -mtime +10 -delete

to crontab of Linux OS.

This is a good solution, I'll happily accept a PR for this.

@39ma
Copy link
Contributor Author

39ma commented Oct 7, 2020

idk how to PR this because its not, a part of code.

@cameronshinn
Copy link
Member

Is there a way to figure out the next match ID without having to read all the JSON files? What's even in the JSON files? I thought all the data gets stored in the matches table.

@B3none
Copy link
Member

B3none commented Oct 7, 2020

It's so that the server can get all the match information - it's pretty much just formatted data from the table with a bit of extra data bolted on.

@cameronshinn
Copy link
Member

Why does the server need all of the information from previous matches when it's creating a new one? Shouldn't only need to look at their IDs to create a new ID. Don't you use auto-incrementing keys in the matches table?

@B3none
Copy link
Member

B3none commented Oct 7, 2020

They're used for the match requests

@thboss
Copy link
Member

thboss commented Oct 12, 2020

@B3none I think using array_flip() and isset() is much faster than in_array()

    protected function doesMatchIdExist(int $matchId): bool
    {
        // Check whether the matchId already exists in the database.
        $query = $this->db->query('SELECT matches.matchid FROM matches WHERE matchid = :matchId', [
            ':matchId' => $matchId
        ]);

        $matches = array_filter(scandir(self::MATCHES_CACHE), function ($item) {
            return !is_dir(self::MATCHES_CACHE . "/$item");
        });

        $flipped_matches = array_flip($matches);

        return $query->rowCount() !== 0 || isset($flipped_matches["$matchId.json"]);
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants