`usage_data` table data flooding

[Jimmy Chu]

How to avoid data flooding in usage_data table?

• using update?
• cron job to remove data entries?
• cron job to archive data entries?

[LiAh Sheep]

• Add index for created_at

[LiAh Sheep]

• 20 hours passed, only 5 rows are added to UsageDatum table.
• Osc::AppsController#usage drops to 39.1% (cf. 80.1% in #255)

[Jimmy Chu]

so what is your proposal on fixing the eventual data flooding problem?

Currently, creating apps from osc is having issue. Once our capacity problem is fixed, and osc app creation works again, this number will grow fast.

[LiAh Sheep]

The proposal is to write a cron job that runs daily to clean up the usage_data table.

As said, a workaround is to use GUI (e.g. Sequel Pro) to clean the table from time to time. This can punt the issue to P2 so we can fix more urgent bugs first.

[LiAh Sheep]

• another 24 hours passed, 4 more rows were inserted
• Osc::AppsController#usage drops to 9.39%

On top of it are:

AppsController#show      39.1%
Osc::LogsController#list 22.8%
Osc::LogsController#show 19.8%

[LiAh Sheep]

Almost 2 weeks passed, only 43 rows in the table. Not P1 for now.