julian.kunkel/mistral-io-datasets
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Mehr text

Browse Source
This commit is contained in:
Julian M. Kunkel 2020-11-19 16:58:18 +00:00
parent 554944e4a0
commit 0a581098bb
1 changed files with 97 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

165
paper/main.tex
View File

@ -296,7 +296,7 @@ In \Cref{fig:job-L}, the mean value is mostly rounded down to 0 except for the f
\begin{subfigure}{0.8\textwidth}
\centering
\includegraphics[width=\textwidth]{job-timeseries7488914-30}
\caption{Job-L (first 30 segments of 400; remaining segments are similar)}
\caption{Job-L (first 30 segments of 400; remaining segments are zero)}
\label{fig:job-L}
\end{subfigure}
\centering
@ -734,6 +734,29 @@ The number of unique names is 19, 38, 49, and 51 for BIN\_aggzero, HEX\_phases,
The jobs that are similar according to the bin algorithms (see \Cref{fig:job-M-bin-aggzero}) differ from our expectations.
The other algorithms like HEX\_lev (\Cref{fig:job-M-hex-lev}) and HEX\_native (\Cref{fig:job-M-hex-native}) seem to work as intended:
While jobs exhibit short bursts of other active metrics even for low similarity we can eyeball a relevant similarity.
The KS algorithm working on the histograms ranks the jobs correctly on the similarity of their histograms.
However, as it does not deal with the length of the jobs, it may identify jobs of very different length.
In \Cref{fig:job-M-ks}, we see the 3rd ranked job, which profile is indeed quite similar but the time series differs but it is just running for 10min (1 segment) on 10\,nodes.
Remember, for the KS algorithm, we concatenate the metrics of all nodes together instead of averaging it in order to explore if node-specific information helps to draw further information about similarity.
\begin{figure}[bt]
\begin{subfigure}{0.5\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_5024292-out/ks-0.7863--ks-2hist7827264}
\caption{Histogram}
\end{subfigure}
\qquad
\begin{subfigure}{0.36\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_5024292-out/ks-0.7863--ks-2timeseries7827264}
\caption{Concatenated time series}
\end{subfigure}
\caption{Job-M with KS, for Rank\,3, SIM=78\%}
\label{fig:job-M-ks}
\end{figure}
\begin{figure}[bt]
@ -818,37 +841,43 @@ While jobs exhibit short bursts of other active metrics even for low similarity
\label{fig:job-M-hex-native}
\end{figure}
\begin{figure}[bt]
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.8831--1timeseries7826634}
\caption{Rank 2, SIM=88\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.7963--2timeseries5240733}
\caption{Rank 3, SIM=80\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.4583--14timeseries4244400}
\caption{Rank 15, SIM=46\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.2397--99timeseries7644009}
\caption{Rank 100, SIM=24\%}
\end{subfigure}
\caption{Job-M with HEX\_phases, selection of similar jobs}
\label{fig:job-M-hex-phases}
\end{figure}
%
% \begin{figure}[bt]
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.8831--1timeseries7826634}
% \caption{Rank 2, SIM=88\%}
% \end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.7963--2timeseries5240733}
% \caption{Rank 3, SIM=80\%}
% \end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.4583--14timeseries4244400}
% \caption{Rank 15, SIM=46\%}
% \end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_5024292-out/hex_phases-0.2397--99timeseries7644009}
% \caption{Rank 100, SIM=24\%}
% \end{subfigure}
%
% \caption{Job-M with HEX\_phases, selection of similar jobs}
% \label{fig:job-M-hex-phases}
% \end{figure}
\subsection{Job-L}
For the bin algorithms, the inspection of job names (14 unique names) leads to two prominent applications: bash and xmessy with 45 and 48 instances, respectively.
The hex algorithms identify a more diverse set of applications (18 unique names and no xmessy job), and the HEX\_phases algorithm has 85 unique names.
The KS algorithm finds 71 jobs ending with t127, which is a typical model configuration.
The bin algorithms find a low similarity (best 2nd ranked job is 17\% similar), the inspection of job names (14 unique names) leads to two prominent applications: bash and xmessy with 45 and 48 instances, respectively.
In \Cref{fig:job-L-bin-aggzero}, it can be seen that the found jobs have little in common with the reference job.
The HEX\_lev and HEX\_native algorithms identify a more diverse set of applications (18 unique names and no xmessy job).
HEX\_native \Cref{fig:job-L-hex-native} finds long jobs where the only few activity as our reference job.
The HEX\_phases algorithm finds 85 unique names but as there is only one short IO phase in the reference job, it finds many (short) jobs with 100\% similarity as seen in \Cref{fig:job-L-hex-phases}.
The KS algorithm is even more inclusive having 1285 jobs with 100\% similarity; the 100 selected ones contain 71 jobs ending with t127, which is a typical model configuration.
As expected, the histograms mimics the profile of the reference job, and thus, the algorithm does what it is expected to do.
\begin{figure}[bt]
\begin{subfigure}{0.3\textwidth}
@ -856,11 +885,11 @@ The KS algorithm finds 71 jobs ending with t127, which is a typical model config
\includegraphics[width=\textwidth]{job_similarities_7488914-out/bin_aggzeros-0.1671--1timeseries7869050}
\caption{Rank 2, SIM=17\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_7488914-out/bin_aggzeros-0.1671--2timeseries7990497}
\caption{Rank 3, SIM=17\%}
\end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_7488914-out/bin_aggzeros-0.1671--2timeseries7990497}
% \caption{Rank 3, SIM=17\%}
% \end{subfigure}
\begin{subfigure}{0.3\textwidth}
\includegraphics[width=\textwidth]{job_similarities_7488914-out/bin_aggzeros-0.1521--14timeseries8363584}
\caption{Rank 15, SIM=15\%}
@ -875,31 +904,31 @@ The KS algorithm finds 71 jobs ending with t127, which is a typical model config
\label{fig:job-L-bin-aggzero}
\end{figure}
\begin{figure}[bt]
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.9386--1timeseries7266845}
\caption{Rank 2, SIM=94\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.9375--2timeseries7214657}
\caption{Rank 3, SIM=94\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.7251--14timeseries4341304}
\caption{Rank 15, SIM=73\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.1657--99timeseries8036223}
\caption{Rank 100, SIM=17\%}
\end{subfigure}
\caption{Job-L with HEX\_lev, selection of similar jobs}
\label{fig:job-L-hex-lev}
\end{figure}
%
% \begin{figure}[bt]
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.9386--1timeseries7266845}
% \caption{Rank 2, SIM=94\%}
% \end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.9375--2timeseries7214657}
% \caption{Rank 3, SIM=94\%}
% \end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.7251--14timeseries4341304}
% \caption{Rank 15, SIM=73\%}
% \end{subfigure}
% % \begin{subfigure}{0.3\textwidth}
% % \centering
% % \includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_lev-0.1657--99timeseries8036223}
% % \caption{Rank 100, SIM=17\%}
% % \end{subfigure}
%
% \caption{Job-L with HEX\_lev, selection of similar jobs}
% \label{fig:job-L-hex-lev}
% \end{figure}
\begin{figure}[bt]
@ -917,11 +946,11 @@ The KS algorithm finds 71 jobs ending with t127, which is a typical model config
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_native-0.8708--14timeseries4936553}
\caption{Rank 15, SIM=87\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_native-0.1695--99timeseries7942052}
\caption{Rank 100, SIM=17\%}
\end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \centering
% \includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_native-0.1695--99timeseries7942052}
% \caption{Rank 100, SIM=17\%}
% \end{subfigure}
\caption{Job-L with HEX\_native, selection of similar jobs}
\label{fig:job-L-hex-native}
@ -938,10 +967,10 @@ The KS algorithm finds 71 jobs ending with t127, which is a typical model config
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_phases-1.0000--1timeseries4405671}
\caption{Rank 3, SIM=100\%}
\end{subfigure}
\begin{subfigure}{0.3\textwidth}
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_phases-1.0000--2timeseries4621422}
\caption{Rank 15, SIM=100\%}
\end{subfigure}
% \begin{subfigure}{0.3\textwidth}
% \includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_phases-1.0000--2timeseries4621422}
% \caption{Rank 15, SIM=100\%}
% \end{subfigure}
\begin{subfigure}{0.3\textwidth}
\centering
\includegraphics[width=\textwidth]{job_similarities_7488914-out/hex_phases-1.0000--99timeseries4232293}