core long-term statistics should be on a separate database!

The problem

It's more like a feature request but I'm writing it here as an issue because the fact that all normal tables and long-term statistics are on the same database is highly dangerous. It happens many time to many users that database get corrupted and is deleted to restart with a clean one that's automatically created. For example I deleted the DB after the daylight savings time bug, believing that there was a problem in the DB. This way I've lost all my long term statistics with months of precious data! This way, however, all the collected beautiful data of long-term statistics are thrown away in the trash can. And that's a pity because long-term statistics is great!

The database is written and rewritten very frequently so the corruption is not an infrequent event. If the databases are two, however, the last term statistics DB would get written only once an hour when updating statistics, greatly reducing the corruption probability.

Furthermore, by dividing them, it will be possible to delete the normal DB leaving long term statistics intact. It would be also possible to backup the long term statistics and restore it in case of disaster recovering with ease.

I believe that splitting the database will be the best option we have for data integrity.

What version of Home Assistant Core has the issue?

core-2021.10.7

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Statistics

https://data.home-assistant.io/docs/statistics/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Please consider it.

In the meanwhile I'm accelerating the InfluxDB installation and use from my ToDo list as I don't want to loose data again.

Asked Jan 10 '22 03:01
avatar energywave
energywave

5 Answer:

Couldn't agree more. I always take my snapshots/backups without the db files to reduce size. The downside is that I will lose all long-term statistics when I have to use a backup. Splitting the db in two would be the perfect solution.

1
Answered Nov 04 '21 at 06:07
avatar  of Jpsy
Jpsy

Thank you for reaching out. We use GitHub for tracking issues, not for providing support or tracking feature requests.

If you want to suggest a feature, you should try our Community Forum: Feature Requests.

If you have additional questions, feel free to join our Discord chat server.

Thanks! 👍

1
Answered Nov 04 '21 at 07:36
avatar  of frenck
frenck

If you want to suggest a feature, you should try our Community Forum: Feature Requests.

Sorry Frenck, I was in doubt about that but as the feature request was born from a real issue (DB deleted = long term statistics gone) I was thinking that posting an issue was not so wrong, after all.

edit: I've deleted the link to the architecture discussion post and created a Feature Request here: https://community.home-assistant.io/t/long-term-statistics-on-a-separate-database/353280

1
Answered Nov 04 '21 at 18:37
avatar  of energywave
energywave

If you are implementing it and want to propose an implementation, yes; otherwise, the architectural repository is not meant for feature requests.

1
Answered Nov 04 '21 at 18:40
avatar  of frenck
frenck

Got it. Now I have a clear map of all resources and how to use them. Sorry for the confusion Frenck. And thank you for all the work as always.

1
Answered Nov 04 '21 at 18:44
avatar  of energywave
energywave