eucalyptus-selinux: security as a first-class citizen!

Eucalyptus 4.3 development sprint is almost over. SELinux support for Eucalyptus is one of the most exciting features [EUCA-1620] for this release.

Like me, if you haven’t looked into SELinux in a while or it is a new thing to you, here are few tips and tricks that may come handy if things don’t work as expected with SELinux while you are playing with Eucalyptus nightly builds or source code during dev cycles or any other new software while SELinux is set to enforcing on the system.

[It is recommended that you try Eucalyptus 4.3 with SELinux on RHEL 7.x or CentOS 7.x.]

Recently while trying Eucalyptus when SELinux is enabled, I was having issues with starting eucanetd.service, which gave me a chance to look further into Eucalyptus SELinux. It appears that the very latest eucanetd requires to open an UDP port and since SELinux doesn’t yet know about it, it is giving an AVC (Access Vector Cache) error.

In case of situation like this or when a newly installed application fails to run/execute on an SELinux enable box, the first thing to check would be /var/log/audit/audit.log.

In the log I found the following error:

type=AVC msg=audit(1462905888.565:1432): avc: denied { name_bind } for pid=10725 comm="eucanetd" src=63822 
scontext=system_u:system_r:eucalyptus_eucanetd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=udp_socket

type=SYSCALL msg=audit(1462905888.565:1432): arch=c000003e syscall=49 success=no exit=-13 a0=3 a1=7ffef025a010 a2=10 
a3=7ffef0259d90 items=0 ppid=1 pid=10725 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) 
ses=4294967295 comm="eucanetd" exe="/usr/sbin/eucanetd" subj=system_u:system_r:eucalyptus_eucanetd_t:s0 key=(null)

To debug further and start eucanetd.service, I have installed the following tool,
yum install policycoreutils-python

This tools comes with a fantastic command called ‘audit2allow‘, which is useful for both debugging and fixing SELinux policy related issues.

The following command shows the list of SELinux failures with reasons.

audit2allow --why --all

Example:

type=AVC msg=audit(1462905222.124:825): avc:  denied  { name_bind } for  pid=2304 comm="eucanetd" src=63822 scontext=system_u:system_r:eucalyptus_eucanetd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=udp_socket
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

type=AVC msg=audit(1462905888.565:1432): avc:  denied  { name_bind } for  pid=10725 comm="eucanetd" src=63822 scontext=system_u:system_r:eucalyptus_eucanetd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=udp_socket
	Was caused by:
		Missing type enforcement (TE) allow rule.

		You can use audit2allow to generate a loadable module to allow this access.

Update:
Though the above information is okay, but if you are submitting an issue for a project or asking for SELinux related queries, ausearch is probably a better tool as it combines the SYSCALL and AVC records. Thanks Garrett Holmstrom for the suggestion.

Example:

ausearch -c eucanetd --start recent

Output:

----
time->Tue May 10 17:05:36 2016
type=SYSCALL msg=audit(1462925136.650:5237): arch=c000003e syscall=49 success=no exit=-13 a0=3 a1=7ffe13b6d920 a2=10 
a3=7ffe13b6d6a0 items=0 ppid=1 pid=6069 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) 
ses=4294967295 comm="eucanetd" exe="/usr/sbin/eucanetd" subj=system_u:system_r:eucalyptus_eucanetd_t:s0 key=(null)

type=AVC msg=audit(1462925136.650:5237): avc:  denied  { name_bind } for  pid=6069 comm="eucanetd" src=63822 
scontext=system_u:system_r:eucalyptus_eucanetd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=udp_socket

We can see that eucalyptus_eucanetd type (eucalyptus_eucanetd_t) is missing rules for access. In my case I had quite a few, but the above example gives an idea of how it might look.

Newbie tips:
Run the following command to check security context of a file on a SELinux enabled system,

# ls -ltrhZ /usr/sbin/eucanetd

-rwxr-xr-x. root root system_u:object_r:eucalyptus_eucanetd_exec_t:s0 /usr/sbin/eucanetd

selinux-penguin-new_medium

Now that we’ve identified the problem, solving it is easy. We can use audit2allow to generate modules to install necessary policies. Though, this can be a tedious job sometimes.

To generate missing policies, we can run the following command,

audit2allow -M my_eucanetd_policy < /var/log/audit/audit.log

which will generate an output like below:

******************** IMPORTANT ***********************
To make this policy package active, execute:

semodule -i my_eucanetd_policy.pp

Now load the module as it says in the output.

At this point we try to run the service again, if it’s a new service, it may fails due to not enough permission, e.g a typical service may require following access for a file { getattr read open } in multiple system calls, but generate policy with audit2allow may only detect the last error.

So, if the service fails to start, we check the audit.log, if it’s another AVC denial error, run the command again to generate module, continue this process until there is no AVC denial in the audit.log.

By this time there could be a couple of policy modules generated and we may want have one single policy file to make future usage easier. After merging the policy files, we need to compile and create policy module package and then install module package like above.

For example after merging policies, mine looked like following:
[WARNING]
this post is for debugging and learning purposes only, under no circumstance you should need to apply custom policies for Eucalyptus. Please open an issue if you run across an AVC denial.


module eucanetd_policy 1.0;

require {
        type user_tmp_t;
        type unreserved_port_t;
        type eucalyptus_cloud_t;
        type eucalyptus_eucanetd_t;
        type eucalyptus_var_lib_t;
        type node_t;
        type eucalyptus_var_run_t;
        class capability { dac_read_search dac_override };
        class file { create getattr write open };
        class udp_socket { name_bind node_bind };
        class dir { read write search add_name };
}

#============= eucalyptus_cloud_t ==============
allow eucalyptus_cloud_t self:capability { dac_read_search dac_override };
allow eucalyptus_cloud_t user_tmp_t:dir read;

#============= eucalyptus_eucanetd_t ==============
allow eucalyptus_eucanetd_t eucalyptus_var_lib_t:dir search;
allow eucalyptus_eucanetd_t eucalyptus_var_run_t:dir { write add_name };
allow eucalyptus_eucanetd_t eucalyptus_var_run_t:file { create getattr write open };

allow eucalyptus_eucanetd_t node_t:udp_socket node_bind;
allow eucalyptus_eucanetd_t unreserved_port_t:udp_socket name_bind;

Update:
If planning to merge to another selinux module, the following command could be provide outputs precisely, thanks Tony Beckham for the suggestion.

ausearch --start recent --success no | audit2allow

After merging all the policies generated by audit2allow, it needs to be compiled and then packaged to be installed as a module.

# compile policy
checkmodule -M -m eucanetd_policy.te -o eucanetd_policy.mod

# create module package
semodule_package -m eucanetd_policy.mod -o eucanetd_policy.pp

# install module
semodule -i eucanetd_policy.pp

By following the above steps, typically we should be able to avoid problems like AVC denials.

The source for eucalyptus-selinux: https://github.com/eucalyptus/eucalyptus-selinux
Report for Eucalyptus related issues: https://eucalyptus.atlassian.net

Advertisements

3 thoughts on “eucalyptus-selinux: security as a first-class citizen!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s