Taming MySQL 8.4 Background Histogram Lock Wait Timeouts on Amazon RDS

 MySQL 8.4 introduced several welcome optimizer enhancements, including automatic background histogram collection via the columns_histogram_background_collection feature. By proactively sampling table data in the background, MySQL can make highly accurate cardinality estimates, improving execution plans for complex workloads with zero manual intervention.

While this feature is a massive win for query performance, Amazon RDS users migrating to MySQL 8.4 are running into a frustrating side effect: severe error log bloat originating from AWS-managed system tables.

If you monitor your RDS MySQL error logs, you will likely see a flood of the following warnings:

[code] [Warning] [MY-015116] [Server] 
Background histogram update on mysql.rds_heartbeat2: Lock wait timeout exceeded; try restarting transaction[/code] 

The Architectural Clash

This behavior is not a bug in the MySQL, but rather an architectural clash between MySQL’s new background telemetry and Amazon’s RDS control plane.

To track instance health and replication metrics, Amazon RDS maintains an internal system table named mysql.rds_heartbeat2. The RDS monitoring agent continuously writes timestamps to this single-row table—often every second. As a result, the table is under a near-perpetual write lock.

Simultaneously, MySQL’s background histogram thread periodically scans the database for tables that might benefit from updated statistics. When it attempts to sample mysql.rds_heartbeat2, it collides with the constant AWS write locks. If the thread cannot obtain a lock within the innodb_lock_wait_timeout threshold, MySQL throws warning MY-015116 and skips the operation.

Because this background process runs on a loop, the result is persistent error log noise.

The Trap of Global Workarounds

Faced with log bloat, the immediate temptation is to apply a global configuration change. However, standard workarounds introduce unacceptable production trade-offs:

• Suppressing Warning MY-015116: Adding this error code to your log_error_suppression_list silences the noise, but it creates operational blindness. It masks legitimate lock wait timeouts on your actual application tables, hiding real contention issues from your team.

• Disabling Background Collection: Turning off columns_histogram_background_collection entirely fixes the logs, but sacrifices a major MySQL 8.4 optimizer enhancement, potentially degrading query performance across your application.

• Increasing the Lock Wait Timeout: Bumping innodb_lock_wait_timeout globally just to accommodate contention on a single telemetry table introduces dangerous latency constraints across your entire database instance.

The Surgical Fix

The most practical, production-grade approach is to address the conflict directly at the table level by excluding mysql.rds_heartbeat2 from histogram maintenance.

This table contains only operational heartbeat data. It is never queried by your application, nor is it involved in complex JOINs or WHERE clauses. The MySQL optimizer derives absolutely zero performance value from maintaining statistics on it.

By explicitly dropping the histogram on this table, you instruct the background thread to ignore it, instantly resolving the lock contention without touching your global parameters.

Step 1: Verify Existing Histograms

First, identify which columns the background thread has attempted to sample:

[code] 
SELECT COLUMN_NAME
FROM information_schema.column_statistics
WHERE SCHEMA_NAME = 'mysql'
  AND TABLE_NAME = 'rds_heartbeat2';
[/code]

Step 2: Drop the Histograms

Using the column names returned in Step 1 (typically id and value), explicitly drop the histograms:

[code]  
ANALYZE TABLE mysql.rds_heartbeat2 DROP HISTOGRAM ON id, value;
[/code]

The Outcome

By dropping the histogram on mysql.rds_heartbeat2, the MySQL background thread will no longer attempt to sample the AWS heartbeat table.

This surgical fix stops the recurring MY-015116 warnings, keeps your error logs clean, and preserves the performance benefits of background histogram collection for your actual application data—all without requiring a single global configuration change.

A Note for Future Automation

Ideally, addressing this behavior should not require manual intervention on every newly provisioned or upgraded MySQL 8.4 instance. Given that mysql.rds_heartbeat2 is an AWS-managed operational table with little or no optimizer value from histogram statistics, AWS could consider excluding it from background histogram collection as part of RDS MySQL provisioning or upgrade workflows.

Such a targeted exclusion would help reduce recurring MY-015116 warnings while preserving the benefits of MySQL 8.4's automatic histogram collection for customer workloads. Until such an enhancement becomes available, dropping histograms on mysql.rds_heartbeat2 remains a practical workaround for keeping error logs free from unnecessary noise.

Disclaimer: The observations and recommendations in this article are based on my personal experience working with MySQL and Amazon RDS environments, and represent my own views only. They do not reflect the opinions of my current or former employers. If AWS has already addressed this behavior, if newer MySQL releases change the underlying implementation, or if I've overlooked any relevant details, I would be grateful for corrections or additional insights in the comments or on LinkedIn.