Loading…
LinuxCon + CloudOpen + ELC-E Europe 2014 has ended
To Learn More: LinuxCon Europe | CloudOpen Europe | Embedded Linux Conference Europe.

Attendees! Please provide us feedback on the sessions you attend! Click here to submit a brief survey for each session and win a $250 Amazon gift certificate. 

>> Tracing Summit: View the Full Schedule

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Tracing Summit [clear filter]
Monday, October 13
 

9:05am

Using Tracing at Facebook Scale, Yannick Brosseau, Facebook
Facebook systems are complex and are stressing the Linux kernel in various, extreme, ways. As we move to kernels that are closer to mainline, we need tools to help us improve the kernel stability and performance. This presentation will describe how we use the various Linux tracing tools for debugging and continuous performance monitoring. We will also cover the road blocks that we faced and what improvements would be a benefit to us.

Speakers
YB

Yannick Brosseau

Production Engineer, Facebook
Yannick Brosseau is a Production Engineer on the Kernel team at Facebook. As such he works on improving the stability and performance of the kernels deployed on the Facebook infrastructure and develops testing, monitoring and deployment tools to help in this endeavor. Previously... Read More →


Monday October 13, 2014 9:05am - 9:35am
Room 2

9:40am

Perf & CTF - Jiri Olsa, Red Hat
The presentation will give an overview of both perf and CTF data storage formats and will focus on mutual comparison of both formats. I will outline (so far) considered ways of conversions from perf data file into CTF stream and discuss possible implications of mixing both perf and CTF data. As the practical side of the talk I'll introduce and describe new perf command, which allows perf to CTF data conversion.

Speakers
avatar for Jiri Olsa

Jiri Olsa

Software Engineer, Red Hat Czech, s.r.o.
Jiri works for RedHat full time on Linux as kernel generalist engineer in Brno office, Czech Republic.


Monday October 13, 2014 9:40am - 10:10am
Room 2

10:15am

User Case Study: Tracing in the QEMU Emulator - Stefan Hajnoczi, Red Hat
The QEMU machine emulator is used by projects including KVM, Xen, and Linaro to emulate hardware. Low overhead instrumentation is important both for developers and in production. This talk covers how tracing is used in QEMU. QEMU abstracts tracers using a code-generation tool called 'tracetool'. This allows the cross-platform codebase to work with DTrace, SystemTap, LTTng UST, and others. Although tracetool is part of the QEMU source tree, it could also be reused by other projects. Users often wish to get started without installing a tracer, so QEMU provides a built-in tracer called 'simpletrace'. Although it's features are modest, it offers a Python module for writing custom analysis scripts. This makes it a powerful tracer during development. Tracing has been integrated into the very core of QEMU. Recent patches make it possible to trace just-in-time compiled code.

Speakers
SH

Stefan Hajnoczi

Stefan Hajnoczi has contributed to QEMU since 2010. He maintains the tracing subsystem, as well as the net subsystem and co-maintains the block layer. Stefan has worked on disk image formats, storage migration, and I/O performance optimization in QEMU at IBM's Linux Technology Center... Read More →


Monday October 13, 2014 10:15am - 10:45am
Room 2

11:15am

From Network to Application: Understanding Your Distributed System with Trace Compass - Geneviève Bastien, École Polytechnique de Montréal; Bernd Hufmann, Ericsson

Trace Compass (previously TMF) is an extensible framework for building trace analysis and visualization tools. With the built-in CTF and Pcap parsers we will demonstrate analyses of data at network, hypervisor, operating system and application level using a simple distributed application. You will see:

  • Network traffic analysis: Correlate network traces (Pcap) with application traces,
  • Data driven analysis: Analyze and visualize recorded data in custom views without writing of a single line of Java code,
  • Virtual machine analysis: If the applications is in a virtualized environment, the host and VM's traces can be correlated and show the resource usage (CPU, VCPU). We can observe a totally different execution than the one from 2 different machines. and explain latencies in the application's execution.

Speakers
avatar for Geneviève Bastien

Geneviève Bastien

Research Associate, École Polytechnique de Montréal
Geneviève Bastien is a research associate at the Dorsal Laboratory of École Polytechnique de Montréal. She is a contributor to the Trace Compass and LTTng projects. Her mission is to make the students' life easier when in comes to prototyping cool new analyses and to make sure... Read More →


Monday October 13, 2014 11:15am - 11:45am
Room 2

11:50am

Discover What Your App is Waiting With Kernel Tracing - Francis Giraldeau, École Polytechnique de Montréal
System administrators and developers are challenged by understanding the elapsed time of heterogeneous and distributed programs in order to improve response time. Profilers are useful to identify hotspot code in an application, but they do not take the wait time into account. A task can block for a device, a local task using any inter-process communication or a task running on another computer. We present a technique using solely kernel tracing to recover the active path of a computation, crossing machines barrier. The technique uses scheduling, interrupt and network events, but does not require system call tracing nor user space instrumentation. Therefore, this technique works for any programming language, and includes interactions with kernel threads. We show the analysis result of actual use-cases, including shell scripts, RPC request, file server and web app.

Speakers
avatar for Francis Giraldeau

Francis Giraldeau

Student, Polytechnique Montreal
Francis Giraldeau is a Linux enthusiast since 15 years. He deployed large-scale and high-availability infrastructure based on open source software, and worked as software quality manager, which led him to contribute patches to many projects. He developed the support for Apache web... Read More →


Monday October 13, 2014 11:50am - 12:10pm
Room 2

12:15pm

Porting LTTng to Android for Kernel-Space and Native User-Space Tracing - Charles Briere, Samsung
Android already have some nice debugging and profiling tool, but when it comes to tracing, you mostly rely on Perf and Ftrace. From the nature of Android being mostly Java based, having LTTng on there would be greatly useful to aggregate kernel, native application and Java application traces. However, it doesn't come free as some features on standard Linux OS are not available on Android. Most important differences in this case are within IPC, base libraries and build tools. The presentation is about what have been done so far to work with those differences and what still needs to be done, followed by a quick demo.

Speakers
CB

Charles Briere

Charles grew up in Montréal, Canada, where he studied Computer Engineering at École Polytechnique de Montréal and graduated in 2013. His first steps in tracing were with LTTng for his final bachelor project. He worked under the supervision of professor Michel R. Dagenais within... Read More →


Monday October 13, 2014 12:15pm - 12:30pm
Room 2

12:15pm

Porting LTTng to Android for Kernel-Space and Native User-Space Tracing - Charles Briere, Samsung
Android already have some nice debugging and profiling tool, but when it comes to tracing, you mostly rely on Perf and Ftrace. From the nature of Android being mostly Java based, having LTTng on there would be greatly useful to aggregate kernel, native application and Java application traces. However, it doesn't come free as some features on standard Linux OS are not available on Android. Most important differences in this case are within IPC, base libraries and build tools. The presentation is about what have been done so far to work with those differences and what still needs to be done, followed by a quick demo.

Speakers
CB

Charles Briere

Charles grew up in Montréal, Canada, where he studied Computer Engineering at École Polytechnique de Montréal and graduated in 2013. His first steps in tracing were with LTTng for his final bachelor project. He worked under the supervision of professor Michel R. Dagenais within... Read More →


Monday October 13, 2014 12:15pm - 12:30pm
Room 2

12:35pm

From DTrace to Linux - Brendan Gregg, Netflix
What can Linux learn from DTrace: what went well, and what didn't go well, on its path to success? This talk will discuss not just the DTrace software, but lessons from the marketing and adoption of a system tracer, and an inside look at how DTrace was really deployed and used in production environments. It will also cover ongoing problems with DTrace, and how Linux may surpass them and continue to advance the field of system tracing. A world expert and core contributor to DTrace, Brendan now works at Netflix on Linux performance with the various Linux tracers, and will summarize his experiences and suggestions for improvements (ftrace, perf_events, eBPF, SystemTap, ktap, sysdig, LTTng, and the DTrace Linux ports). He has also been contributing to various tracers: recently promoting ftrace and perf_events adoption through articles and front-end scripts, and testing eBPF.

Speakers
avatar for Brendan Gregg

Brendan Gregg

Senior Performance Architect, Netflix
Brendan Gregg is an industry expert in computing performance and cloud computing. He is a senior performance architect at Netflix, where he does performance design, evaluation, analysis, and tuning. He is the author of Systems Performance published by Prentice Hall, and received the... Read More →


Monday October 13, 2014 12:35pm - 1:20pm
Room 2

2:30pm

Runtime Analysis of Parallel Applications for Industrial Software Development - Daniel Becker, Siemens & Markus Geimer, Jülich Supercomputing Centre
Utilizing the parallelism offered by multicore CPUs is hard, though profiling and tracing are well-established techniques to understand, debug, engineer and optimize codes. While many tools are available to capture profiles and traces, these tools are often difficult to use in industrial contexts. Tool development often started with sequential applications in mind to transition to parallelism not until later, resulting in improper feature sets and usability. In contrast, parallel tools are often targeted towards HPC with a strong focus on MPI and OpenMP. As this turns these tools less suitable for codes using alternative threading models (POSIX, Qt, and ACE), this talk presents extensions to the open-source profiling and tracing tools Score-P and Scalasca. Score-P captures detailed program execution data allowing Scalasca to perform a sophisticated performance and wait-state analysis.

Speakers
DB

Daniel Becker

Daniel Becker received his Ph.D. degree from RWTH Aachen University in 2009. He completed his Ph.D. project at the Jülich Supercomputing Centre in the area of scalable performance analysis tools. His career path alsoincludes research stays at academic and industrial organizations... Read More →
MG

Markus Geimer

Senior scientist, Forschungszentrum Jülich GmbH


Monday October 13, 2014 2:30pm - 3:00pm
Room 2

3:05pm

First Failure Data Capture for Linux - Michael Holzheu, IBM

Successful software problem determination depends heavily on the availability of debugging data such as logs, traces, and dumps. More often than not the required information is not readily available resulting in the need to perform life debugging, instrumentation, and problem reproduction. What if this approach is impracticable because the system is not accessible, a further outage is not acceptable, or the problem cannot be easily reproduced?

First Failure Data Capture (FFCD) is a concept that aims at ensuring that all relevant data is collected, retained and reported at the first occurrence of an error. It has been implemented successfully for years in core mainframe components with high availability requirements, such as system firmware or operating systems. This presentation discusses ideas on how the FFDC concept could be applied to Linux.


Speakers
avatar for Michael Holzheu

Michael Holzheu

Mr., IBM
Michael Holzheu is a Linux kernel developer at the IBM lab in Boeblingen, Germany. He studied computer science at the University of Erlangen and has worked for IBM since 1998. After a start in the z/OS UNIX Systems Services environment, he joined the Linux on z Systems team in 2000... Read More →


Monday October 13, 2014 3:05pm - 3:35pm
Room 2

3:40pm

Hardware Trace - The Ultimate Linux Performance Tuning Tool - Mathieu Poirier, Linaro; Al Grant, ARM

The talk is focused on how the hardware trace features on many SoCs can be practically used by ordinary software developers to investigate Linux performance issues. We'll describe ARM's CoreSight trace technology in enough detail to make the talk useful to people not familiar with hardware trace, but the talk will focus on the practical application of hardware trace to Linux - e.g. how trace can be set up from within the OS, how hardware instrumentation trace (STM) can be used to support existing Linux trace mechanisms such as ftrace, and how hardware-based instruction trace can be configured and visualized to solve problems out of reach of these software-based mechanisms. We'll look at some of the practical issues with using hardware trace and how these can be dealt with.

The second part of the presentation will focus on the new framework proposed by Linaro to support Coresight and hardware assisted tracing in the Linux kernel. More specifically we will go over the solution, the current state of upstreaming and the challenges still ahead. If time permits we will see an example of HW trace decoding using the framework in the Linux kernel.


Speakers
AG

Al Grant

System Performance Architect, ARM
Al Grant is a performance architect at ARM, using hardware trace and other tools to improve the efficiency of CPU designs, compilers, OSes and software. Al led the development of the first ARM 64-bit compiler.
MP

Mathieu Poirier

Mathieu Poirier is part of the Core Development group at Linaro where he is currently involved in an effort to upstream a framework and a set of drivers to support Coresight in the Linux kernel.


Monday October 13, 2014 3:40pm - 4:10pm
Room 2

4:15pm

Packet Trace Modelling and Visualization - Petru Lauric, Freescale
This presentation provides an overview of effective means for correlating and visualizing the trace data collected from a network processor. Trace data directly associated with the network packets is collected at key points in the system, using hardware and/or Linux software mechanisms such as ftrace or LTTng. An abstract model of the packet data flows in the system is used in order to represent the trace-based analysis data in an intuitive fashion. The trace data collected from multiple sources is correlated and analyzed in order to automatically build the data flow model, together with the data flow profile. The model is used to visualize the activity of the traced system along with key metrics such as network packet processing latencies, load balancing factors, packet dropout rates, processed data volume and others. The presentation discusses the trace data requirements, the trace correlation aspects and the need for networking oriented trace data analysis and visualization. The Freescale Packet Analysis Tool, a software tool implementing these trace analysis techniques, is used to demonstrate the presented concepts.

Speakers
PL

Petru Lauric

Senior Member of Technical Staff, Freescale
Petru Lauric works for Freescale where he was involved in various software analysis projects, ranging from binary code instrumentation-based performance analysis tools for Nintendo and Sony game consoles to source code instrumentation-based analysis tools for microcontrollers. More... Read More →


Monday October 13, 2014 4:15pm - 4:45pm
Room 2

4:50pm

5:30pm

BoFs: First Failure Data Capture for Linux - Michael Holzheu & Michael Müller, IBM
We will present an overview of our ideas in the FFDC presentation on the Tracing Summit (http://sched.co/1tysvUS). Let's take some more time to discuss how FFDC could be implemented in Linux.
The audience is anyone who is interested and especially people that have experience in Linux problem determination. For example, people developing embedded systems, tracing/logging system owners, or owners of software components that provide debugging data. Please share your experience and ideas with us!

Speakers
avatar for Michael Holzheu

Michael Holzheu

Mr., IBM
Michael Holzheu is a Linux kernel developer at the IBM lab in Boeblingen, Germany. He studied computer science at the University of Erlangen and has worked for IBM since 1998. After a start in the z/OS UNIX Systems Services environment, he joined the Linux on z Systems team in 2000... Read More →
MM

Michael Müller

Michael Mueller is a System Software developer at the IBM Lab in Boeblingen, Germany. He studied Computer Science at the Universities of Erlangen and Dortmund and Business Administration at the Henley Management College. He joined IBM in 1991 when he worked for the education departmenton... Read More →


Monday October 13, 2014 5:30pm - 6:30pm
Room 8