[lttng-dev] Seeking Assistance on LTTng for Kubernetes Performance Research - Benjamin Saint-Cyr via lttng-dev

From: Benjamin Saint-Cyr via lttng-dev <lttng-dev@lists.lttng.org>
To: lttng-dev@lists.lttng.org
Subject: [lttng-dev] Seeking Assistance on LTTng for Kubernetes Performance Research
Date: Thu, 06 Apr 2023 19:56:30 +0000	[thread overview]
Message-ID: <20230406195630.Horde.jhBZfsMqxFud8S3napU7CjJ@www.imp.polymtl.ca> (raw)

Dear LTTng Developers,

I am currently conducting performance research on Kubernetes for my  
master's thesis and am using LTTng for data collection and Trace  
Compass for analysis. I have a couple of questions and concerns, and I  
would appreciate your guidance to ensure I am using your tools  
effectively.

	1. In the Control Flow view, I can see the CPU throttling due to the  
quota defined in the Cgroup as the pod runs and then waits for the CPU  
periodically. However, it is unclear whether the pod is waiting due to  
a full CPU or because of timeshares. I can easily find this  
information in /sys/fs/cgroup/cpu,cpuacct/kubepods/<burstable or  
besteffort>/pod<UID>/cpu.stat if the pod is being throttled. Is there  
a way to extract this information using LTTng or Trace Compass?

	2. I am attempting to determine which PID is running a specific  
Container/Pod UID. Currently, I can deduce this by observing the  
process named runc[*:INIT], as it creates the cgroup and clones the  
process for the container. However, this approach is fragile and only  
works due to an implementation detail. It also fails if tracing starts  
after deployment. This information is easily available in  
/sys/fs/cgroup/cpu,cpuacct/kubepods/<burstable or  
besteffort>/pod<UID>/<container ID>/cgroup.procs. Is there an existing  
method within LTTng to obtain this information that I may not be aware  
of?

I have been informed that it might be possible to read the information  
in cpu.stat or cgroup.procs using a state dump at the beginning or  
end, but the process remains unclear. Alternatively, I could  
instrument the kernel or the process handling the accounting, but this  
would be quite challenging. Another option would be to use something  
similar to cAdvisor, but its efficiency is questionable since it  
dedicates an entire process to reading a few files.

I would greatly appreciate any guidance or suggestions you may have on  
improving my approach and effectively utilizing LTTng for my research.

Thank you for your time and assistance.

Best regards,

Benjamin Saint-Cyr

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev