On Mon, Jun 29, 2015 at 04:24:55PM -0400, Paul Moore wrote:
> On Monday, June 29, 2015 07:47:29 PM Andrew Jones wrote:
> > On Mon, Jun 29, 2015 at 10:53:14AM -0400, Paul Moore wrote:
> > > On Monday, June 29, 2015 09:50:17 AM Andrew Jones wrote:
> > > > On Fri, Jun 26, 2015 at 04:26:22PM -0400, Paul Moore wrote:
> > > > > Perhaps a stupid question, but you did verify that it is cacheflush
> > > > > that
> > > > > is causing the problem?  The seccomp filter code will emit a message
> > > > > to
> > > > > syslog or the audit log, depending on your configuration, with the
> > > > > syscall number.
> > > > > 
> > > > >  #./tools/scmp_sys_resolver -a arm cacheflush
> > > > >  983042
> > > > >  #./tools/scmp_sys_resolver -a arm 983042
> > > > 
> > > > I hadn't before (didn't know about the logging). I had determined the
> > > > problem by running qemu in gdb. I just checked now though and confirmed
> > > > it
> > > > 
> > > > type=SECCOMP msg=audit(1435563996.731:2032): auid=1001 uid=1001 gid=1001
> > > > ses=157 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> > > > pid=27059 comm="qemu-system-arm"
> > > > exe="/home/drjones/code/qemu/arm-softmmu/qemu-system-arm" sig=31
> > > > arch=40000028 syscall=983042 compat=0 ip=0xb6b43164 code=0x0
> > > > 
> > > > This log was generated even with the above patch applied to qemu.
> > > 
> > > The only thing that comes to mind quickly is that the cacheflush() call is
> > > being done by a thread that was created before the seccomp filter was
> > > loaded into the kernel; although I believe you said you already checked
> > > that.
> >
> > Nope, I hadn't, but I have now ...
> 
> Actually, never mind on that, I was being stupid.  If it was a different 
> thread it wouldn't be impacted by the seccomp filter at all ...
> 
> > ... So we're calling __clear_cache from the same thread that called
> > seccomp_start, and that thread dies the moment it calls the syscall.
> > No other threads except id(2) at this time, which appears to be
> > something created by __libc_start_main before main() runs.
> 
> Hmm, so either the kernel is screwing up with the seccomp filter for this 
> particular syscall (unlikely) or libseccomp is screwing up the filter creation 
> (more likely).  I don't have an ARM system handy at the moment, but could you 
> use the seccomp_export_pfc() and seccomp_export_bpf() functions to dump the 
> PFC/BPF filter code to a file and send it out?

Attached

> 
> > > If you are using a recent kernel and libseccomp you can try enabling the
> > > SCMP_FLTATR_CTL_TSYNC attribute to apply the filter to all running threads
> > > in the process.
> > > 
> > > 	rc = seccomp_attr_set(ctx, SCMP_FLTATR_CTL_TSYNC, 1);
> > > 	if (rc)
> > > 	
> > > 		/* error */
> > 
> > I tried this, but it error'ed out with rc == -95 (EOPNOTSUPP ?)
> > My kernel version is 4.0.5-200.fc21.armv7hl+lpae
> 
> That should be a recent enough kernel, but perhaps your version of libseccomp 
> was built against an older version of the kernel that didn't have the 
> necessary support (and it was disabled at compile time)?
>

I looked at the pfc file and compared all the syscalls in it vs.  the list
in qemu-seccomp.c. The pfc file is missing cacheflush, and has an 'UNKNOWN'
instead. Also, I think there may be another problem with the filter (or pfc)
generation. Several of the syscalls have weird syscall numbers. For example,
I would expect mmap to be 90, but instead it's -10181.

And, since there was something weird, and not related to cacheflush, in the
arm32 pfc, I decided to check it on my mustang too. The output there gets
"cacheflush" for the name instead of UNKNOWN, but has the same weird
number (-10104) that the midway has. It also has several other weird
numbers. The output from the mustang is in the attached tarball as well.

Thanks,
drew