Study what is AIO

[LiAh Sheep]

No description provided.

[LiAh Sheep]

AIO?

AIO means Async I/O, a feature that existed in Windows only, but now available to Linux. MySQL 5.5+ & InnoDB 1.1+ uses Linux’s native AIO to improve performance of InnoDB I/O threads. Basically it allows parallel, non-blocking reads/writes. [1][2]

Diagnosis

From the log listed in #240: InnoDB: Warning: io_setup() failed with EAGAIN. We see that the problem lies in Linux side – io_setup() is a linux command. MySQL could not initiate the AIO of the underlying OS, thus returning an error.

From the error message EAGAIN, we can also tell from the man page that “The specified nr_events exceeds the user's limit of available events, as defined in /proc/sys/fs/aio-max-nr.” As we planned to host 200 apps in oo6 (as opposed to 90 apps in oo5), the I/O requirement for that machine has more than doubled. Increasing the nr_events value may probably help (not tested).

Performance issue?

There seems to be no source that mentions how worse the performance will become. Here are some quotes [3]:

• Disable innodb_use_native_aio in config file. This will hit performance of MySQL, but the MySQL should work...
• There is nothing more we can do from a cartridge point of view here except provide users a way to disable Native AIO in MySQL when this issue occurs. This will make the MySQL performance a bit worse, but MySQL will still operate without any problems and so users can take a snapshot of MySQL database and remove/add the MySQL cartridge.

I believe, for now, we can still disable Native AIO. When we (and our customers) scale up and hit database I/O performance issue, we may need to revisit this.

References

[1] http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_asynchronous_io
[2] http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-aio-linux.html
[3] https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1085282

[Jimmy Chu]

I agree that we could disable AIO for now as well. Closed.