Storage and growth

The Storage & growth tab of every MySQL and MariaDB database shows how large it is, how fast it grows, and when it will run out of storage. The agent on the database’s collector reads the sizes from InnoDB’s own statistics, never from your data. Read Database monitoring first for collectors, grants and privacy modes.

What the tab shows

Part	Shows
Instance total	The whole database, split into Tables, General tablespaces, System, Undo, Temporary and Binary logs. Marked Estimate.
Growth per day	How fast it grows, with a band: for example 1.21 to 1.66 GiB a day (10th–90th percentile), and its size In 30 days.
Days to full	When it reaches its ceiling, and what the ceiling is.
Collection	Whether every size can be read, and when each read last ran.
Growth	A chart of the daily maximum size over 90 days or 1 year, with the projection and its band dashed to +30, +90, +180 or +365 days, and the ceiling as a line.
Schemas	Every schema with its size, share, growth per day and size in 30 days. The 50 largest have their own growth.
Top tables	The largest tables, the same columns plus Rows (estimate). A row’s chart button opens that table’s own growth chart.

The Databases list adds two columns, Size and 30-day growth. The Layers available card on the database’s Info tab shows the layer on its L3 Sizes and growth row, and the Overview’s Monitoring coverage gauge counts it as half while it is collected without row estimates.

What is measured, and how often

Read	How often	Gives
Table statistics (mysql.innodb_table_stats)	Every hour	Row estimates, and the size of tables kept in a shared tablespace
Tablespaces (InnoDB’s tablespace view)	Every 6 hours	The size of each table in its own file, general tablespaces, and the system schemas
Files (information_schema.FILES, MySQL only)	Every 6 hours	The system tablespace (ibdata1), the undo files and the temporary file

The first measurement comes 2–5 minutes after a collector starts collecting the database. Every read is capped at 5 seconds, information_schema.TABLES is never read, and a storage read that fails never marks the database down.

• Every byte counts once. A table kept in a shared tablespace (the system tablespace or a general one) is shown with (shared): it is sized from its pages, and it counts in the total through its tablespace.
• Binary logs are the size the server reports for them, every 5 minutes.
• Everything is an estimate. InnoDB sizes are space allocated, files grow in 4 MiB steps, and row counts are approximate: InnoDB refreshes them after about 10 % of a table changes.
• Only InnoDB tables are sized. InnoDB’s views do not list MyISAM or other engines’ tables.
• The 100 largest tables have their own growth, and keep it while they stay in the top 120. The rest are summed in one line. Up to 200 schemas are listed; the 50 largest have their own growth, kept while they stay in the top 60, and the others show — for growth.

The grants it needs

Grant	Without it
PROCESS (tier A)	Nothing is read for storage: the tab says No new sizes and shows the GRANT to run. The Info tab’s L3 row reads Needs PROCESS.
Tier C: SELECT on mysql.innodb_table_stats and mysql.innodb_index_stats	Sizes come from the tablespaces only: no row estimates, and tables in a shared tablespace are not sized. The tab says Sizes without row estimates and shows the two GRANTs; the L3 row reads Collected — no row estimates (needs tier C).

The grant script grants both. With both, the Collection card reads Full and the L3 row Collected. A collector whose agent predates storage collection shows No sizes yet until the agent updates itself, which it does within minutes of a release.

Growth and the forecast

The long-term store keeps each day’s maximum size for 10 years. Once a day, WatchTower15 fits the database’s growth to them:

• A robust slope. The fit is the median of the slopes between every pair of days (Theil–Sen), so a one-day spike — a long transaction, a burst of temporary tables — does not bend it.
• Up to 90 days, restarting after a drop. A sudden fall in size, such as an OPTIMIZE TABLE or a large purge, starts the fit again from the drop.
• The last 30 days when growth has changed. When growth has just started or stopped, the last 30 days are projected instead of the whole window.
• A band. The 10th and 90th percentiles of those slopes give the band around the projection.
• 7 days of history first. A new database, and a database after a drop, shows Forecast from <date> — needs 7 days of history.
• Not growing. A flat or falling database reads Not growing.

Tables, general tablespaces and the system tablespace are projected together; undo, the temporary file and the binary logs are added at their current size. Each database’s forecast is recomputed once a day.

Days to full and the ceiling

Days to full is the free space under the ceiling divided by the growth per day.

Where the database runs	Ceiling	Free space
Self-managed on EC2	The volume holding the database’s datadir on the server it runs on — the server you named at step 2 of the setup	That volume’s free space
Amazon RDS for MySQL and RDS for MariaDB, with cloud metrics	The storage AWS reports: the storage autoscaling limit, else the allocated size	AWS’s own free storage, plus the autoscaling headroom (the limit less the allocated size)
Amazon RDS for MySQL and RDS for MariaDB, without them	The Storage ceiling you enter on the Info tab	The ceiling less the instance total: an estimate, since RDS also stores logs and temporary files
Amazon Aurora MySQL	None: Aurora’s cluster volume grows by itself	—

Days to full reads	Means
A number of days	The ceiling is reached in that many days at the current growth, around the date shown
More than 10 years	Growing, but too slowly to matter
Not filling	Not growing, with free space left
0 days — No free space left	The ceiling is reached. It reads 0 days even once growth has stopped, until space is freed or the ceiling is raised
—	No ceiling, no current sizes, or not enough history yet; the card says which

On a self-managed volume, only the database’s own growth is projected: other files on the volume are not. Your volume alarms watch the volume itself.

The Storage ceiling card (RDS)

Reported by AWS. Once your collectors’ role carries the cloud metrics read policy, WatchTower15 reads the instance’s storage from the RDS API every hour, and its free storage from CloudWatch. The next hourly forecast run (at minute 37) takes them as the ceiling, before any value entered by hand. The card is then read-only: Ceiling: <size> · reported by AWS (its storage autoscaling limit), or (its allocated size) without storage autoscaling, with the free space. It follows the instance’s storage on its own, so nothing needs entering or updating.

Entered by hand, while AWS’s figures are not read: Info → Storage ceiling → Storage ceiling (GiB), then Save. In the AWS console, find it under RDS → Databases → the instance → Configuration → Storage:

• Maximum storage threshold when storage autoscaling is on;
• otherwise Allocated storage.

Enter whole GiB, from 1 to 65,536. Days to full is recomputed at once, and the change is recorded on the database’s Audit tab. Empty the field and save to clear the ceiling. Update it after you change the instance’s storage. Until a ceiling is reported by AWS or entered, the tab says No storage ceiling reported by AWS or entered and shows no days to full. Changing the ceiling needs database.edit; everyone else sees the card read-only.

• A ceiling you entered is kept, unused, while AWS’s figures are read: it applies again if they stop (the read policy removed, for example).
• AWS’s figures arriving, or changing (storage autoscaling grew the volume), queue the database for the next forecast run: until then the tab’s Days to full card reads reported by AWS, applied at the next hourly forecast run.

Self-managed: which volume

The ceiling is the volume, among those of the server the database runs on, whose mount point holds the database’s datadir; the deepest one wins. The tab names it — for example volume /var/lib/mysql on billing-01 — so check that it is the right one:

• Without a host server, set at setup step 2, there is no ceiling.
• MySQL in a container: datadir is the path inside the container, so the match may pick the wrong host volume, often /.
• A datadir reached through a symlink (for example /var/lib/mysql → /data/mysql): the match picks the link’s volume, not the one that holds the files.
• InnoDB files outside datadir — placed by innodb_data_home_dir, innodb_undo_directory or a table’s DATA DIRECTORY — are counted against the datadir volume.

The storage alarm

Database storage filling up opens a warning when days to full falls to 30 and escalates to critical at 7. It is one of the database’s default alarms, tuned like any other on the Alarms tab.

• Quiet until the first forecast, so until 7 days of history exist.
• A full database stays critical at 0 days, even once its growth has stopped, until space is freed or the ceiling is raised.
• It resolves when growth stops with free space left (Not filling), when the ceiling is raised, or when days to full climbs back above the threshold.
• It keeps its state while there is nothing to judge: sizes missing for more than 48 hours, no free-space reading, the days after a drop until the forecast is back, or a forecast that has stopped running.
• Never evaluated on Aurora, or on an RDS database with no ceiling reported by AWS or entered.
• Resolved at once, with no notification, when you remove the ceiling (clear it, or change the provider to Aurora), when the server a self-managed database runs on is cleared or deleted, or when the database goes over the table limit. The database’s Audit tab says why.

When something is missing

The tab says	Means
Sizes last measured <date>	No sizes for more than 48 hours: PROCESS revoked, the collector stopped, or the database unreachable. Days to full reads — and the alarm keeps its state until sizes return.
<read>: the read at <time> reached the 5 s limit.	That read took longer than 5 seconds.
<read>: the read at <time> failed: MySQL error <n>.	The server refused the read, with that error number (for example 1142 after a grant was revoked). Only the number is reported, never the message.
<read>: the read at <time> failed: the connection was lost.	The connection dropped during the read; the next one reconnects.
<read>: skipped while too many statements were running (pressure).	32 or more threads were running. A read skipped for pressure runs within 6 hours whatever the load.
Recomputing at the next hourly forecast run	Sizes came back after a stop, the host server changed, or AWS’s storage figures arrived or changed: days to full returns at the next forecast run, at minute 37 of the hour.

After a failed read, the parts of the total it reads keep their last reading until it is 36 hours old, then are left out of the total until the read answers again. The tab names them.

MySQL 5.7 and Aurora MySQL 2.x

On these versions, reading the tablespaces can wait for as long as tables are being created, altered or dropped. So:

• the Tablespaces and Files reads are skipped while DDL runs, temporary tables included, and run on a short connection of their own;
• under a DDL stream that never stops, a read is forced at most once a day;
• never more than one read waits on the server at a time: the monitoring user’s three connections leave room for it beside the collector’s own connection and the standby’s check. The tab says an earlier read is still waiting on the server while one does.

Names: Hashed, Clear names, Counts only

Schema and table names follow the database’s privacy mode:

Mode	The Schemas and Top tables lists show
Hashed (default)	Each schema and table as a key, such as #3f9a1c…. WatchTower15 holds only the key: a keyed hash of the name under your redaction key, which never leaves your servers.
Clear names	The names. They are kept in WatchTower15’s database only, never in the metric stores.
Counts only	No lists: the totals and the counts of schemas and tables. Per-object sizes need Hashed or Clear names.

• Switching from Clear names to another mode deletes every stored name at once; switching to Counts only also deletes the per-schema and per-table sizes.
• Accept new key gives every schema and table a new key: their sizes restart under the new keys, the old ones go after 30 days, and per-schema and per-table forecasts need 7 days again. The instance total, its forecast and days to full carry on.
• Deleting a database deletes its schema and table lists, names included.

Revealing a key on your host

Click a key, then Reveal on the host, for the command to run on one of the database’s collector servers, as root:

sudo /var/lib/wt15-server-agent/wt15-server-agent db reveal <database-uuid> <key>

It hashes the database’s current names with the redaction key on that server and prints the match — for example table `shop`.`orders` or schema `shop` — or no object with this key. It sends nothing to WatchTower15. It uses one connection, with the collector’s credentials. On MySQL 5.7 and Aurora MySQL 2.x it may answer DDL is running on this database: try again in a few minutes. The summed line of smaller tables or schemas names no single object, so it has no name to reveal.

The table limit

WatchTower15 reads the storage of a database with up to 2,048 tables by default. Above that, no storage read runs: no sizes, no growth forecast, no days to full, and an open storage alarm resolves. The tab reads:

Over the table limit. This database has <n> tables, more than its limit of <limit>. WatchTower15 does not read its storage sizes, so there is no growth forecast or days to full. Contact support to raise the limit.

The count is the tables of your own schemas: each partitioned table counts once, and the system schemas (mysql, sys, performance_schema, information_schema) and temporary tables are left out. It is taken every hour, or every 6 hours without tier C. Once the count is back under the limit, sizes return at the next read.

The limit applies to each database on its own, and only WatchTower15 support can raise it, up to 4,096 tables: contact us. A new limit reaches your collectors within a minute and applies from their next count.

Permissions

database.view reads the tab, the list columns and the Info tab’s L3 row. database.edit enters or clears an RDS storage ceiling. See Roles & permissions.