The full title of this blog should really be ‘SELinux is preventing mysqld (mysqld_t) “search” to ./tmp (public_content_rw_t)’ as that is the problem I’ve been having with CentOS recently (and hence my searches on the web for a solution).

The cause of the problem

I use SugarCRM for customer and project management data – and very good it is too! (Gratuitous plug – I can help your company install and use this fine software ). Except that recently, when listing my Accounts within Sugar, I would not see all of the account context. Only the account data itself would be displayed and none of the subpanels/links.

The query to retrieve more data was failing, with this error message displayed in the browser window:
mysqld: Can't create/write to file '/tmp/#08y2jw' (Errcode: 13)

In my system log (/var/log/messages), I also got multiple SELinux errors like this:
Oct 13 09:07:50 server setroubleshoot: SELinux is preventing mysqld (mysqld_t) "read" to ./tmp (public_content_rw_t). For complete SELinux messages. run sealert -l 1762c478-f3a2-4eeb-be09-bd3dc037d945

Clearly, the reason for “Errcode: 13″ was due to SELinux.

Incidentally. if you have seen a similar error on your web site, but with (Errcode: 28) instead, this is likely due to shortage of disk space. A great way of determining operating system errors like this, is to use ‘PError’, thus:
# perror 28
OS error code 28: No space left on device
# perror 13
OS error code 13: Permission denied

So there we are – two distinct and different issues.

With SELinux, resolving the permission issue can be difficult. By issuing # sealert -l 1762c478-f3a2-4eeb-be09-bd3dc037d945, as suggested above, I got the following output (trimmed and highlighted for clarity):

Summary:
SELinux is preventing mysqld (mysqld_t) “search” to ./tmp (public_content_rw_t).
Allowing Access:
Sometimes labeling problems can cause SELinux denials. You could try to restore
the default system file context for ./tmp,
restorecon -v ‘./tmp’
Additional Information:
Source Context root:system_r:mysqld_t
Target Context system_u:object_r:public_content_rw_t

First things first: issuing # restorecon -v './tmp' didn’t fix it for me. I was also surprised to see that the path to /tmp was relative to the current working directory, so I tried a slightly modified # restorecon -v '/tmp', but to no avail. After restarting mysqld, the problem persisted: MySQL was simply being refused access to /tmp. Somewhere, a policy is disallowing this.

It’s a mistake to assume the the source context and target context should be the same; they don’t have to be, as it’s entirely policy-driven.  I made bold those aspects (the file Type) above to highlight this incorrect assumption (that I previously held).

Find and fix a policy?

Although finding the troublesome policy and analysing it is a Good Thing, it’s also time-consuming and requires significant knowledge of SELinux, chiefly to avoid creating security holes. A better way, I found, was simply to relocate where mysqld tries to store temporary data.

Thanks to Surachart Opun’s blog, I learned that you can specify a new location for temporary files. In /etc/my.cnf, add or edit the following:

[mysqld]
tmpdir=/tmp # # e.g.
tmpdir=/var/lib/mysql/tmp


Now do the legwork to set up the directory properly:

First, create directory with appropriate permissions
# cd /var/lib/mysql
# mkdir tmp
# chown mysql:mysql tmp
# chmod 1750 tmp

Now set the SELinux context up:
# chcon --reference /var/lib/mysql tmp

and make the SELinuiux context permanent:
# semanage fcontext -a -t mysql_db_t "/var/lib/mysql/tmp(/.*)?"

Finally, restart mysql:
# service mysqld restart

Closing thoughts: optimisation

The methods above fixed the particular problem I was having. They didn’t, however, actually pinpoint the cause. This is one of the good things about Linux and SELinux in particular: you are forced to rethink what the system is doing and work out a solution that sits within the predefined security context – or learn how to write SELinux policies. Personally, I prefer the former

There is an additional benefit to the solution above – namely, optimisation. Because we have specified the security context with semanage, we are free to mount an external file system and use that instead for MySQL’s temporary files. In other words, we can maintain the security but increase the performance.  One such filesystem could be tmpfs. tmpfs is actually a RAM Disk, uses a fixed amount of RAM to provide file storage. It is much quicker than an on-disk filesystem and thus perfectly optimised for storing temporary, caching data. There are many resources about tmpfs on the web. A good introduction to tmpfs can be at Planet Admon.