- changed milestone to *00.P1*
Study what is AIO
No description provided.
Comments (5)
-
reporter -
reporter AIO?
AIO means Async I/O, a feature that existed in Windows only, but now available to Linux. MySQL 5.5+ & InnoDB 1.1+ uses Linux’s native AIO to improve performance of InnoDB I/O threads. Basically it allows parallel, non-blocking reads/writes. [1][2]
Diagnosis
From the log listed in
#240:InnoDB: Warning: io_setup() failed with EAGAIN.
We see that the problem lies in Linux side –io_setup()
is a linux command. MySQL could not initiate the AIO of the underlying OS, thus returning an error.From the error message
EAGAIN
, we can also tell from the man page that “The specifiednr_events
exceeds the user's limit of available events, as defined in/proc/sys/fs/aio-max-nr
.” As we planned to host 200 apps in oo6 (as opposed to 90 apps in oo5), the I/O requirement for that machine has more than doubled. Increasing thenr_events
value may probably help (not tested).Performance issue?
There seems to be no source that mentions how worse the performance will become. Here are some quotes [3]:
- Disable innodb_use_native_aio in config file. This will hit performance of MySQL, but the MySQL should work...
- There is nothing more we can do from a cartridge point of view here except provide users a way to disable Native AIO in MySQL when this issue occurs. This will make the MySQL performance a bit worse, but MySQL will still operate without any problems and so users can take a snapshot of MySQL database and remove/add the MySQL cartridge.
I believe, for now, we can still disable Native AIO. When we (and our customers) scale up and hit database I/O performance issue, we may need to revisit this.
References
[1] http://dev.mysql.com/doc/innodb/1.1/en/glossary.html#glos_asynchronous_io
[2] http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-aio-linux.html
[3] https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1085282 -
reporter -
assigned issue to
-
assigned issue to
-
I agree that we could disable AIO for now as well. Closed.
-
- changed status to resolved
- Log in to comment