change ILP text\
git-svn-id: https://yap.svn.sf.net/svnroot/yap/trunk@1825 b08c6af1-5177-4d33-ba66-4b1c6b8b522a
This commit is contained in:
vsc
parent
63a4ae736d
commit
570bce634d
|
|
@ -936,6 +936,12 @@ and report times in seconds.
|
||||||
|
|
||||||
\subsection{JITI Speedups} \label{sec:perf:speedups}
|
\subsection{JITI Speedups} \label{sec:perf:speedups}
|
||||||
%---------------------------------------------------
|
%---------------------------------------------------
|
||||||
|
% Our experience with the indexing algorithm described here shows a
|
||||||
|
% significant performance improvement over the previous indexing code in
|
||||||
|
% our system. Quite often, this has allowed us to tackle applications
|
||||||
|
% which previously would not have been feasible. We next present some
|
||||||
|
% results that show how useful the algorithms can be.
|
||||||
|
|
||||||
Here I already have "compress", "mutagenesis" and "sg\_cyl"
|
Here I already have "compress", "mutagenesis" and "sg\_cyl"
|
||||||
The "sg\_cyl" has a really impressive speedup (2 orders of
|
The "sg\_cyl" has a really impressive speedup (2 orders of
|
||||||
magnitude). We should keep the explanation in your text.
|
magnitude). We should keep the explanation in your text.
|
||||||
|
|
@ -943,217 +949,6 @@ and report times in seconds.
|
||||||
If time permits, we should also add some FSA benchmarks
|
If time permits, we should also add some FSA benchmarks
|
||||||
(e.g. "k963", "dg5" and "tl3" from PLDI)
|
(e.g. "k963", "dg5" and "tl3" from PLDI)
|
||||||
|
|
||||||
\subsection{JITI in ILP} \label{sec:perf:ILP}
|
|
||||||
%--------------------------------------------
|
|
||||||
The need for just-in-time indexing was originally motivated by ILP
|
|
||||||
applications. Table~\ref{tab:aleph} shows JITI performance on some
|
|
||||||
learning tasks using the ALEPH system~\cite{}. The dataset
|
|
||||||
\texttt{Krki} tries to learn rules for chess end-games;
|
|
||||||
\texttt{GeneExpression} learns rules for yeast gene activity given a
|
|
||||||
database of genes, their interactions, plus micro-array data;
|
|
||||||
\texttt{BreastCancer} processes real-life patient reports towards
|
|
||||||
predicting whether an abnormality may be malignant;
|
|
||||||
\texttt{IE-Protein\_Extraction} processes information extraction from
|
|
||||||
paper abstracts to search proteins; \texttt{Susi} learns from shopping
|
|
||||||
patterns; and \texttt{Mesh} learns rules for finite-methods mesh
|
|
||||||
design. The datasets \texttt{Carcinogenesis}, \texttt{Choline},
|
|
||||||
\texttt{Mutagenesis}, \texttt{Pyrimidines}, and \texttt{Thermolysin}
|
|
||||||
are about predicting chemical properties of compounds. The first three
|
|
||||||
datasets present properties of interest as boolean attributes, but
|
|
||||||
\texttt{Thermolysin} learns from the 3D structure of a molecule.
|
|
||||||
Several of these datasets are standard across Machine Learning
|
|
||||||
literature. \texttt{GeneExpression}~\cite{} and
|
|
||||||
\texttt{BreastCancer}~\cite{} were partly developed by some of the
|
|
||||||
authors. Most datasets perform simple queries in an extensional
|
|
||||||
database. The exception is \texttt{Mutagenesis} where several
|
|
||||||
predicates are defined intensionally, requiring extensive computation.
|
|
||||||
|
|
||||||
|
|
||||||
\begin{table}[ht]
|
|
||||||
%\vspace{-\intextsep}
|
|
||||||
%\begin{table}[htbp]
|
|
||||||
%\centering
|
|
||||||
\centering
|
|
||||||
\begin {tabular}{|l|r|r|r|r|} \hline %\cline{1-3}
|
|
||||||
& \multicolumn{2}{|c|}{\bf Time in sec.} & \bf \JITI \\
|
|
||||||
{\bf Benchs.} & \bf $A1$ & \bf JITI & \bf Ratio \\
|
|
||||||
\hline
|
|
||||||
\texttt{BreastCancer} & 1450 & 88 & 16\\
|
|
||||||
\texttt{Carcinogenesis} & 17,705 & 192 &92\\
|
|
||||||
\texttt{Choline} & 14,766 & 1,397 & 11 \\
|
|
||||||
\texttt{GeneExpression} & 193,283 & 7,483 & 26 \\
|
|
||||||
\texttt{IE-Protein\_Extraction} & 1,677,146 & 2,909 & 577 \\
|
|
||||||
\texttt{Krki} & 0.3 & 0.3 & 1 \\
|
|
||||||
\texttt{Krki II} & 1.3 & 1.3 & 1 \\
|
|
||||||
\texttt{Mesh} & 4 & 3 & 1.3 \\
|
|
||||||
\texttt{Mutagenesis} & 51,775 & 27,746 & 1.9\\
|
|
||||||
\texttt{Pyrimidines} & 487,545 & 253,235 & 1.9 \\
|
|
||||||
\texttt{Susi} & 105,091 & 307 & 342 \\
|
|
||||||
\texttt{Thermolysin} & 50,279 & 5,213 & 10 \\
|
|
||||||
\hline
|
|
||||||
\end{tabular}
|
|
||||||
\caption{Machine Learning (ILP) Datasets: Times are given in Seconds,
|
|
||||||
we give time for standard indexing with no indexing on dynamic
|
|
||||||
predicates versus the \JITI implementation}
|
|
||||||
\label{tab:aleph}
|
|
||||||
\end{table}
|
|
||||||
|
|
||||||
We compare times for 10 runs of the saturation/refinement cycle of the
|
|
||||||
ILP system. Table~\ref{tab:aleph} shows results. The \texttt{Krki}
|
|
||||||
datasets have small search spaces and small databases, so they
|
|
||||||
essentially maintain performance. The \texttt{Mesh},
|
|
||||||
\texttt{Mutagenesis}, and \texttt{Pyrimides} applications do not
|
|
||||||
benefit much from indexing in the database, but they do benefit from
|
|
||||||
indexing in the dynamic representation of the search space, as their
|
|
||||||
running times halve.
|
|
||||||
|
|
||||||
The \texttt{BreastCancer} and \texttt{GeneExpression} applications use
|
|
||||||
1NF data (that is, unstructured data). The benefit here is from
|
|
||||||
multiple-argument indexing. \texttt{BreastCancer} is particularly
|
|
||||||
interesting. It consists of 40 binary relations with 65k elements
|
|
||||||
each, where the first argument is the key, like in \texttt{sg\_cyl}. We
|
|
||||||
know that most calls have the first argument bound, hence indexing was
|
|
||||||
not expected to matter very much. Instead, the results show \JITI
|
|
||||||
running time to improve by an order of magnitude. Like in
|
|
||||||
\texttt{sg\_cyl}, this suggests that relatively small numbers of badly
|
|
||||||
indexed calls can dominate running time.
|
|
||||||
|
|
||||||
\texttt{IE-Protein\_Extraction} and \texttt{Thermolysin} are example
|
|
||||||
applications that manipulate structured data.
|
|
||||||
\texttt{IE-Protein\_Extraction} is the largest dataset we considered,
|
|
||||||
and indexing is simply critical: we could not run the application in
|
|
||||||
reasonable time without JITI. \texttt{Thermolysin} is smaller and
|
|
||||||
performs some computation per query: even so, indexing is very
|
|
||||||
important.
|
|
||||||
|
|
||||||
\begin{table*}[ht]
|
|
||||||
\centering
|
|
||||||
\begin {tabular}{|l|r|r|r|r|r||r|r|r|r|r|r|} \hline %\cline{1-3}
|
|
||||||
& \multicolumn{5}{|c||}{\bf Static Code} & \multicolumn{6}{|c|}{\bf Dynamic Code \& IDB} \\
|
|
||||||
& \textbf{Clause} & \multicolumn{4}{|c||}{\bf Indexing Code} & \textbf{Clause} & \multicolumn{5}{|c|}{\bf Indexing Code} \\
|
|
||||||
\textbf{Benchmarks} & & Total & T & W & S & & Total & T & C & W & S \\
|
|
||||||
\hline
|
|
||||||
\texttt{BreastCancer} & 60940 & 46887 & 46242 &
|
|
||||||
3126 & 125 & 630 & 14 &42 & 18& 57 &6 \\
|
|
||||||
|
|
||||||
\texttt{Carcinogenesis} & 1801 & 2678
|
|
||||||
&1225 & 587 & 865 & 13512 & 942 & 291 & 91 & 457 & 102
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Choline} & 666 & 174
|
|
||||||
&67 & 48 & 58 & 3172 & 174
|
|
||||||
& 76 & 4 & 48 & 45
|
|
||||||
\\
|
|
||||||
\texttt{GeneExpression} & 46726 & 22629
|
|
||||||
&6780 & 6473 & 9375 & 116463 & 9015
|
|
||||||
& 2703 & 932 & 3910 & 1469
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{IE-Protein\_Extraction} &146033 & 129333
|
|
||||||
&39279 & 24322 & 65732 & 53423 & 1531
|
|
||||||
& 467 & 108 & 868 & 86
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Krki} & 678 & 117
|
|
||||||
&52 & 24 & 40 & 2047 & 24
|
|
||||||
& 10 & 2 & 10 & 1
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Krki II} & 1866 & 715
|
|
||||||
&180 & 233 & 301 & 2055 & 26
|
|
||||||
& 11 & 2 & 11 & 1
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Mesh} & 802 & 161
|
|
||||||
&49 & 18 & 93 & 2149 & 109
|
|
||||||
& 46 & 4 & 35 & 22
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Mutagenesis} & 1412 & 1848
|
|
||||||
&1045 & 291 & 510 & 4302 & 595
|
|
||||||
& 156 & 114 & 264 & 61
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Pyrimidines} & 774 & 218
|
|
||||||
&76 & 63 & 77 & 25840 & 12291
|
|
||||||
& 4847 & 43 & 3510 & 3888
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Susi} & 5007 & 2509
|
|
||||||
&855 & 578 & 1076 & 4497 & 759
|
|
||||||
& 324 & 58 & 256 & 120
|
|
||||||
\\
|
|
||||||
|
|
||||||
\texttt{Thermolysin} & 2317 & 929
|
|
||||||
&429 & 184 & 315 & 116129 & 7064
|
|
||||||
& 3295 & 1438 & 2160 & 170
|
|
||||||
\\
|
|
||||||
|
|
||||||
\hline
|
|
||||||
\end{tabular}
|
|
||||||
\caption{Memory Performance on Machine Learning (ILP) Datasets: memory
|
|
||||||
usage is given in KB}
|
|
||||||
\label{tab:ilpmem}
|
|
||||||
\end{table*}
|
|
||||||
|
|
||||||
|
|
||||||
We have seen that using the \JITI does not impose a significant
|
|
||||||
overhead. Table~\ref{tab:ilpmem} discusses the memory cost . It
|
|
||||||
measures memory being spend at a point near the end of execution.
|
|
||||||
Because dynamic memory expands and contracts, we chose a point where
|
|
||||||
memory usage should be at a maximum. The first five columns show data
|
|
||||||
usage on static predicates. The leftmost sub-column represents the
|
|
||||||
code used for clauses; the next sub-columns represent space used in
|
|
||||||
indices for static predicates: the first column gives total usage,
|
|
||||||
which consists of space used in the main tree, the expanded
|
|
||||||
wait-nodes, and hash-tables.
|
|
||||||
|
|
||||||
Static data-base sizes range from 146MB to 666KB, the latter mostly in
|
|
||||||
system libraries. The impact of indexing code varies widely: it is
|
|
||||||
more than the original code for \texttt{Mutagenesis}, almost as much
|
|
||||||
for \texttt{IE-Protein\_Extraction}, and in most cases it adds at
|
|
||||||
least a third and often a half to the original data-base. It is
|
|
||||||
interesting to check the source of the space overhead: if the source
|
|
||||||
are hash-tables, we can expect this is because of highly-complex
|
|
||||||
indices. If overhead is in \emph{wait-nodes}, this again suggests a
|
|
||||||
sophisticated indexing structure. Overhead in the main tree may be
|
|
||||||
caused by a large number of nodes, or may be caused by \texttt{try}
|
|
||||||
nodes.
|
|
||||||
|
|
||||||
One first conclusion is that \emph{wait-nodes} are costly space-wise,
|
|
||||||
even if they are needed to achieve sensible compilation times. On the
|
|
||||||
other hand, whether the space is allocated to the tree or to the
|
|
||||||
hashes varies widely. \texttt{IE-Protein\_Extraction} is an example
|
|
||||||
where the indices seem very useful: most space was spent in the
|
|
||||||
hash-tables, although we still are paying much for \emph{wait-nodes}.
|
|
||||||
\texttt{BreastCancer} has very small hash-tables, because it
|
|
||||||
attributes range over small domains, but indexing is useful (we
|
|
||||||
believe this is because we are only interested in the first solution
|
|
||||||
in this case).
|
|
||||||
|
|
||||||
This version of the ILP system stores most dynamic data in the IDB.
|
|
||||||
The size of reflects the search space, and is largely independent of
|
|
||||||
the program's static data (notice that small applications such as
|
|
||||||
\texttt{Krki} do tend to have a small search space). Aleph's author
|
|
||||||
very carefully designed the system to work around overheads in
|
|
||||||
accessing the data-base, so indexing should not be as important. In
|
|
||||||
fact, indexing has a much lower space overhead in this case,
|
|
||||||
suggesting it is not so critical. On the other hand, looking at the
|
|
||||||
actual results shows that indexing is working well: most space is
|
|
||||||
spent on hashes and the tree, little space is spent on \texttt{try}
|
|
||||||
instructions. It is hard to separate the contributions of JITI on
|
|
||||||
static and dynamic data, but the results for \texttt{Mesh} and
|
|
||||||
\texttt{Mutagenesis}, where the JITI probably has little impact on
|
|
||||||
static code, suggest a factor of two from indexing on the IDB in this
|
|
||||||
case.
|
|
||||||
|
|
||||||
|
|
||||||
% Our experience with the indexing algorithm described here shows a
|
|
||||||
% significant performance improvement over the previous indexing code in
|
|
||||||
% our system. Quite often, this has allowed us to tackle applications
|
|
||||||
% which previously would not have been feasible. We next present some
|
|
||||||
% results that show how useful the algorithms can be.
|
|
||||||
|
|
||||||
Next, we present performance results for demand-driven indexing on a
|
Next, we present performance results for demand-driven indexing on a
|
||||||
number of benchmarks and real-life applications. Throughout, we
|
number of benchmarks and real-life applications. Throughout, we
|
||||||
compare performance with single argument indexing. We use YAP-5.1.2
|
compare performance with single argument indexing. We use YAP-5.1.2
|
||||||
|
|
@ -1216,128 +1011,40 @@ improves performance in the latter case only, but this does make a
|
||||||
large difference, as the WAM code has to visit all thousand clauses if
|
large difference, as the WAM code has to visit all thousand clauses if
|
||||||
the second argument is unbound.
|
the second argument is unbound.
|
||||||
|
|
||||||
The graph reachability datasets because they both use the same
|
\subsection{JITI in ILP} \label{sec:perf:ILP}
|
||||||
program, but on different databases. The t-test does not show a
|
%--------------------------------------------
|
||||||
significant difference
|
The need for just-in-time indexing was originally motivated by ILP
|
||||||
|
applications. Table~\ref{tab:aleph} shows JITI performance on some
|
||||||
} the database
|
learning tasks using the ALEPH system~\cite{}. The dataset
|
||||||
itself. The JITI brings little benefits on the linear graphs if we
|
\texttt{Krki} tries to learn rules from a small database of chess
|
||||||
call the \texttt{path/3} predicates with left or right recursion. On
|
end-games; \texttt{GeneExpression} learns rules for yeast gene
|
||||||
the other hand, it always improves performance when using the doubly
|
activity given a database of genes, their interactions, and
|
||||||
recursive version, and it always improves performance on the tree
|
micro-array gene expression data; \texttt{BreastCancer} processes
|
||||||
graph.
|
real-life patient reports towards predicting whether an abnormality
|
||||||
|
may be malignant; \texttt{IE-Protein\_Extraction} processes
|
||||||
To understand why, we first consider the simplest execution pattern,
|
information extraction from paper abstracts to search proteins;
|
||||||
given by the left-recursive procedure. The code for the LRF is:
|
\texttt{Susi} learns from shopping patterns; and \texttt{Mesh} learns
|
||||||
|
rules for finite-methods mesh design. The datasets
|
||||||
\begin{verbatim}
|
|
||||||
path1(X,Y,[X,Y]) :- arc(X,Y).
|
|
||||||
path1(X,Y,[X|P]) :- arc(X,Z),
|
|
||||||
path1(Z,Y,P),
|
|
||||||
not_member(X,P).
|
|
||||||
\end{verbatim}
|
|
||||||
\noindent
|
|
||||||
Careful inspection of the program shows that \texttt{arc/3} can be
|
|
||||||
accessed with different modes. First, given the top-level goal
|
|
||||||
$path1(X,Y,\_)$ the two clauses for \texttt{path1/3} call
|
|
||||||
\texttt{arc/3} with both arguments free. Second, the recursive call
|
|
||||||
to \texttt{path1/3} can call \texttt{arc/3} in the base clause with
|
|
||||||
\emph{both arguments bound}. If the graph is linear, the second
|
|
||||||
argument is functionally dependent on the first, and indexing on the
|
|
||||||
first argument is sufficient. But, if the graph has a branching factor
|
|
||||||
$> 1$, WAM style first argument indexing will lead to backtracking,
|
|
||||||
whereas the JITI can perform direct lookup through the hash tables.
|
|
||||||
This explains the performance improvement for the \texttt{tree}
|
|
||||||
graphs.
|
|
||||||
|
|
||||||
Do such improvements hold for real applications? An interesting
|
|
||||||
application of tabled Prolog is in program analysis, often based in
|
|
||||||
Anderson's points-to analysis~\cite{anderson-phd}. In this framework,
|
|
||||||
imperative programs are encoded as a set of facts, and properties of
|
|
||||||
interest are encoded rules. Program properties can be verified by
|
|
||||||
checking the closure of the rules. Such programs therefore have
|
|
||||||
similar properties to the \texttt{path} benchmarks, and should
|
|
||||||
generate similar performance. Table~\ref{tab:pa} shows such
|
|
||||||
applications. The first analyses a smallish program and the second the
|
|
||||||
\texttt{javac} benchmark.
|
|
||||||
|
|
||||||
\begin{table}[ht]
|
|
||||||
\centering
|
|
||||||
\begin {tabular}{|l|r|r||r|r|r||} \hline %\cline{1-3}
|
|
||||||
& \multicolumn{2}{|c||}{\bf Time in sec.} &
|
|
||||||
\multicolumn{3}{|c||}{\bf Static Space in KB.} \\
|
|
||||||
{\bf Benchs.} & \bf $A_1$ & \bf JITI & \bf Clause & \multicolumn{2}{|c||}{\bf Indices} \\
|
|
||||||
& & & & \bf $A_1$ & \bf JITI \\
|
|
||||||
\hline
|
|
||||||
\texttt{pta} & 14 & 1.7 & 845 & 318 & 351 \\
|
|
||||||
\texttt{tea} & 800 & 36.9 & 36781 & 1793 & 2848 \\
|
|
||||||
\hline
|
|
||||||
\end{tabular}
|
|
||||||
\caption{Program Analysis}
|
|
||||||
\label{tab:pa}
|
|
||||||
\end{table}
|
|
||||||
|
|
||||||
Table~\ref{tab:pa} shows total running times, and size of static
|
|
||||||
data-base in KB for a YAP run. The first column shows the size in
|
|
||||||
clauses, the other two show the size of the indices when using
|
|
||||||
single-argument indexing and the JITI.
|
|
||||||
|
|
||||||
|
|
||||||
\begin{table}[ht]
|
|
||||||
%\vspace{-\intextsep}
|
|
||||||
%\begin{table}[htbp]
|
|
||||||
%\centering
|
|
||||||
\centering
|
|
||||||
\begin {tabular}{|l|r|r|r|r|} \hline %\cline{1-3}
|
|
||||||
& \multicolumn{2}{|c|}{\bf Time in sec.} & \bf JITI \\
|
|
||||||
{\bf Benchs.} & \bf $A1$ & \bf JITI & \bf Ratio \\
|
|
||||||
\hline
|
|
||||||
\texttt{BreastCancer} & 1450 & 88 & 16\\
|
|
||||||
\texttt{Carcinogenesis} & 17,705 & 192 &92\\
|
|
||||||
\texttt{Choline} & 14,766 & 1,397 & 11 \\
|
|
||||||
\texttt{GeneExpression} & 193,283 & 7,483 & 26 \\
|
|
||||||
\texttt{IE-Protein\_Extraction} & 1,677,146 & 2,909 & 577 \\
|
|
||||||
\texttt{Krki} & 0.3 & 0.3 & 1 \\
|
|
||||||
\texttt{Krki II} & 1.3 & 1.3 & 1 \\
|
|
||||||
\texttt{Mesh} & 4 & 3 & 1.3 \\
|
|
||||||
\texttt{Mutagenesis} & 51,775 & 27,746 & 1.9\\
|
|
||||||
\texttt{Pyrimidines} & 487,545 & 253,235 & 1.9 \\
|
|
||||||
\texttt{Susi} & 105,091 & 307 & 342 \\
|
|
||||||
\texttt{Thermolysin} & 50,279 & 5,213 & 10 \\
|
|
||||||
\hline
|
|
||||||
\end{tabular}
|
|
||||||
\caption{Machine Learning (ILP) Datasets}
|
|
||||||
\label{tab:aleph}
|
|
||||||
\end{table}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
JITI was originally motivated by applications in the area of Machine
|
|
||||||
Learning that try to learn rules from databases (our compiler is used
|
|
||||||
on a number of such systems). Table~\ref{tab:aleph} shows performance
|
|
||||||
for one of the most popular such systems in some detail. The datasets
|
|
||||||
\texttt{Carcinogenesis}, \texttt{Choline}, \texttt{Mutagenesis},
|
\texttt{Carcinogenesis}, \texttt{Choline}, \texttt{Mutagenesis},
|
||||||
\texttt{Pyrimidines}, and \texttt{Thermolysin} are about predicting
|
\texttt{Pyrimidines}, and \texttt{Thermolysin} are about predicting
|
||||||
chemical properties of compounds. Most queries perform very simple
|
chemical properties of compounds. The first three datasets store
|
||||||
queries in an extensional database; \texttt{Mutagenesis} includes
|
properties of interest as tables, but \texttt{Thermolysin} learns from
|
||||||
several predicates defined as rules; and \texttt{Thermolysin} performs
|
the 3D-structure of a molecule's conformations. Several of these
|
||||||
simple 3D distance computations. \texttt{Krki} are chess end-games.
|
datasets are standard across Machine Learning literature.
|
||||||
\texttt{GeneExpression} processes micro-array data,
|
\texttt{GeneExpression}~\cite{} and \texttt{BreastCancer}~\cite{} were
|
||||||
\texttt{BreastCancer} real-life patient reports,
|
partly developed by some of the authors. Most datasets perform simple
|
||||||
\texttt{IE-Protein\_Extraction} information extraction from paper
|
queries in an extensional database. The exception is
|
||||||
abstracts that mention proteins, \texttt{Susi} shopping patterns, and
|
\texttt{Mutagenesis} where several predicates are defined
|
||||||
\texttt{Mesh} finite-methods mesh design. Several of these datasets
|
intensionally, requiring extensive computation.
|
||||||
are standard across Machine Learning literature.
|
|
||||||
\texttt{GeneExpression} and \texttt{BreastCancer} were partly
|
|
||||||
developed by the authors.
|
|
||||||
|
|
||||||
\begin{table}[ht]
|
\begin{table}[ht]
|
||||||
%\vspace{-\intextsep}
|
%\vspace{-\intextsep}
|
||||||
%\begin{table}[htbp]
|
%\begin{table}[htbp]
|
||||||
%\centering
|
%\centering
|
||||||
\centering
|
\centering
|
||||||
\begin {tabular}{|l|r|r|r|r|} \hline %\cline{1-3}
|
\begin {tabular}{|l|r|r|r|r|} \hline %\cline{1-3}
|
||||||
& \multicolumn{2}{|c|}{\bf Time in sec.} & \bf JITI \\
|
& \multicolumn{2}{|c|}{\bf Time in sec.} & \bf \JITI \\
|
||||||
{\bf Benchs.} & \bf $A1$ & \bf JITI & \bf Ratio \\
|
{\bf Benchs.} & \bf $A1$ & \bf JITI & \bf Ratio \\
|
||||||
\hline
|
\hline
|
||||||
\texttt{BreastCancer} & 1450 & 88 & 16\\
|
\texttt{BreastCancer} & 1450 & 88 & 16\\
|
||||||
|
|
@ -1354,210 +1061,161 @@ developed by the authors.
|
||||||
\texttt{Thermolysin} & 50,279 & 5,213 & 10 \\
|
\texttt{Thermolysin} & 50,279 & 5,213 & 10 \\
|
||||||
\hline
|
\hline
|
||||||
\end{tabular}
|
\end{tabular}
|
||||||
\caption{Machine Learning (ILP) Datasets}
|
\caption{Machine Learning (ILP) Datasets: Times are given in Seconds,
|
||||||
|
we give time for standard indexing with no indexing on dynamic
|
||||||
|
predicates versus the \JITI implementation}
|
||||||
\label{tab:aleph}
|
\label{tab:aleph}
|
||||||
\end{table}
|
\end{table}
|
||||||
|
|
||||||
We compare times for 10 runs of the saturation/refinement cycle of the
|
We compare times for 10 runs of the saturation/refinement cycle of the
|
||||||
ILP system. Table~\ref{tab:aleph} shows very clearly the advantages
|
ILP system. Table~\ref{tab:aleph} shows results. The \texttt{Krki}
|
||||||
of JITI: speedups range up to two orders of magnitude. Applications
|
datasets have small search spaces and small databases, so they
|
||||||
such as \texttt{BreastCancer} and \texttt{GeneExpression} manipulate
|
essentially achieve the same performance under both versions: there is
|
||||||
1NF data (that is, unstructured data). The first benefit is from
|
no slowdown. The \texttt{Mesh}, \texttt{Mutagenesis}, and
|
||||||
multiple-argument indexing. Multi-argument is available in other
|
\texttt{Pyrimides} applications do not benefit much from indexing in
|
||||||
Prolog systems~\cite{BIM,xsb-manual,ZhTaUs-small,SWI}), but using
|
the database, but they do benefit from indexing in the dynamic
|
||||||
it would require extra user information that would be hard to most ILP
|
representation of the search space, as their running times halve.
|
||||||
users: the JITI provides that for free. Just multi-argument indexing
|
|
||||||
does not explain everything. \texttt{BreastCancer} results were of
|
The \texttt{BreastCancer} and \texttt{GeneExpression} applications use
|
||||||
particular interest to us because the dataset was to a large extent
|
1NF data (that is, unstructured data). The benefit here is mostly from
|
||||||
developed by the authors. It consists of 40 binary relations which are
|
multiple-argument indexing. \texttt{BreastCancer} is particularly
|
||||||
most often used with the first argument as a key (it is almost
|
interesting. It consists of 40 binary relations with 65k elements
|
||||||
propositional learning). We did not expect a huge speedup, but the
|
each, where the first argument is the key, like in
|
||||||
results show the opposite: calls with both arguments bound, or with
|
\texttt{sg\_cyl}. We know that most calls have the first argument
|
||||||
the second argument bound may not be very frequent, but they are
|
bound, hence indexing was not expected to matter very much. Instead,
|
||||||
frequent enough to justify indexing. This would be difficult to
|
the results show \JITI running time to improve by an order of
|
||||||
predict beforehand, even to experienced Prolog programmers.
|
magnitude. Like in \texttt{sg\_cyl}, this suggests that even a small
|
||||||
|
percentage of badly indexed calls can come to dominate running time.
|
||||||
|
|
||||||
\texttt{IE-Protein\_Extraction} and \texttt{Thermolysin} are example
|
\texttt{IE-Protein\_Extraction} and \texttt{Thermolysin} are example
|
||||||
applications that manipulate structured data.
|
applications that manipulate structured data.
|
||||||
\texttt{IE-Protein\_Extraction} is a large dataset, therefore indexing
|
\texttt{IE-Protein\_Extraction} is the largest dataset we consider,
|
||||||
is simply critical: we could not run the application in reasonable
|
and indexing is simply critical: it is not possible to run the
|
||||||
time without JITI. \texttt{Thermolysin} is smaller and performs
|
application in reasonable time with one argument
|
||||||
significant computation per query: even so, indexing is very
|
indexing. \texttt{Thermolysin} is smaller and performs some
|
||||||
important.
|
computation per query: even so, indexing improves performance by an
|
||||||
|
order of magnitude.
|
||||||
Indexing is no magical bullet. On the flip side, \texttt{Mutagenesis}
|
|
||||||
is an example where indexing does help, but not by much. The problem
|
|
||||||
is that most time is spent on recursive predicates that were built to
|
|
||||||
use the first argument. \texttt{Mutagenesis} also shows a concern with
|
|
||||||
JITI: we generate large indices but we do not benefit very much.
|
|
||||||
|
|
||||||
\begin{table*}[ht]
|
\begin{table*}[ht]
|
||||||
\centering
|
\centering
|
||||||
\begin {tabular}{|l|r|r|r|r|r||r|r|r|r|r|r|} \hline %\cline{1-3}
|
\begin {tabular}{|l|r|r||r|r|} \hline %\cline{1-3}
|
||||||
& \multicolumn{5}{|c||}{\bf Static Code} & \multicolumn{6}{|c|}{\bf Dynamic Code \& IDB} \\
|
& \multicolumn{2}{|c||}{\bf Static Code} & \multicolumn{2}{|c|}{\bf Dynamic Code} \\
|
||||||
& \textbf{Clause} & \multicolumn{4}{|c||}{\bf Indexing Code} & \textbf{Clause} & \multicolumn{5}{|c|}{\bf Indexing Code} \\
|
Benchmarks & \textbf{Clause} & {\bf Index} & \textbf{Clause} & {\bf Index} \\
|
||||||
\textbf{Benchmarks} & & Total & T & W & S & & Total & T & C & W & S \\
|
% \textbf{Benchmarks} & & Total & T & W & S & & Total & T & C & W & S \\
|
||||||
\hline
|
\hline
|
||||||
\texttt{BreastCancer} & 60940 & 46887 & 46242 &
|
\texttt{BreastCancer}
|
||||||
3126 & 125 & 630 & 14 &42 & 18& 57 &6 \\
|
& 60940 & 46887
|
||||||
|
% & 46242 & 3126 & 125
|
||||||
|
& 630 & 14
|
||||||
|
% &42 & 18& 57 &6
|
||||||
|
\\
|
||||||
|
|
||||||
\texttt{Carcinogenesis} & 1801 & 2678
|
\texttt{Carcinogenesis}
|
||||||
&1225 & 587 & 865 & 13512 & 942 & 291 & 91 & 457 & 102
|
& 1801 & 2678
|
||||||
\\
|
% &1225 & 587 & 865
|
||||||
|
& 13512 & 942
|
||||||
|
%& 291 & 91 & 457 & 102
|
||||||
|
\\
|
||||||
|
|
||||||
\texttt{Choline} & 666 & 174
|
\texttt{Choline} & 666 & 174
|
||||||
&67 & 48 & 58 & 3172 & 174
|
% &67 & 48 & 58
|
||||||
& 76 & 4 & 48 & 45
|
& 3172 & 174
|
||||||
|
% & 76 & 4 & 48 & 45
|
||||||
\\
|
\\
|
||||||
\texttt{GeneExpression} & 46726 & 22629
|
\texttt{GeneExpression} & 46726 & 22629
|
||||||
&6780 & 6473 & 9375 & 116463 & 9015
|
% &6780 & 6473 & 9375
|
||||||
& 2703 & 932 & 3910 & 1469
|
& 116463 & 9015
|
||||||
|
%& 2703 & 932 & 3910 & 1469
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{IE-Protein\_Extraction} &146033 & 129333
|
\texttt{IE-Protein\_Extraction} &146033 & 129333
|
||||||
&39279 & 24322 & 65732 & 53423 & 1531
|
%&39279 & 24322 & 65732
|
||||||
& 467 & 108 & 868 & 86
|
& 53423 & 1531
|
||||||
|
%& 467 & 108 & 868 & 86
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Krki} & 678 & 117
|
\texttt{Krki} & 678 & 117
|
||||||
&52 & 24 & 40 & 2047 & 24
|
%&52 & 24 & 40
|
||||||
& 10 & 2 & 10 & 1
|
& 2047 & 24
|
||||||
|
%& 10 & 2 & 10 & 1
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Krki II} & 1866 & 715
|
\texttt{Krki II} & 1866 & 715
|
||||||
&180 & 233 & 301 & 2055 & 26
|
%&180 & 233 & 301
|
||||||
& 11 & 2 & 11 & 1
|
& 2055 & 26
|
||||||
|
%& 11 & 2 & 11 & 1
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Mesh} & 802 & 161
|
\texttt{Mesh} & 802 & 161
|
||||||
&49 & 18 & 93 & 2149 & 109
|
%&49 & 18 & 93
|
||||||
& 46 & 4 & 35 & 22
|
& 2149 & 109
|
||||||
|
%& 46 & 4 & 35 & 22
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Mutagenesis} & 1412 & 1848
|
\texttt{Mutagenesis} & 1412 & 1848
|
||||||
&1045 & 291 & 510 & 4302 & 595
|
%&1045 & 291 & 510
|
||||||
& 156 & 114 & 264 & 61
|
& 4302 & 595
|
||||||
|
%& 156 & 114 & 264 & 61
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Pyrimidines} & 774 & 218
|
\texttt{Pyrimidines} & 774 & 218
|
||||||
&76 & 63 & 77 & 25840 & 12291
|
%&76 & 63 & 77
|
||||||
& 4847 & 43 & 3510 & 3888
|
& 25840 & 12291
|
||||||
|
%& 4847 & 43 & 3510 & 3888
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Susi} & 5007 & 2509
|
\texttt{Susi} & 5007 & 2509
|
||||||
&855 & 578 & 1076 & 4497 & 759
|
%&855 & 578 & 1076
|
||||||
& 324 & 58 & 256 & 120
|
& 4497 & 759
|
||||||
|
%& 324 & 58 & 256 & 120
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\texttt{Thermolysin} & 2317 & 929
|
\texttt{Thermolysin} & 2317 & 929
|
||||||
&429 & 184 & 315 & 116129 & 7064
|
%&429 & 184 & 315
|
||||||
& 3295 & 1438 & 2160 & 170
|
& 116129 & 7064
|
||||||
|
%& 3295 & 1438 & 2160 & 170
|
||||||
\\
|
\\
|
||||||
|
|
||||||
\hline
|
\hline
|
||||||
\end{tabular}
|
\end{tabular}
|
||||||
\caption{Memory Performance on Machine Learning (ILP) Datasets}
|
\caption{Memory Performance on Machine Learning (ILP) Datasets: memory
|
||||||
|
usage is given in KB}
|
||||||
\label{tab:ilpmem}
|
\label{tab:ilpmem}
|
||||||
\end{table*}
|
\end{table*}
|
||||||
|
|
||||||
|
|
||||||
In general, one would wonder whether the benefits in time correspond
|
Table~\ref{tab:ilpmem} discusses the memory cost paid in using
|
||||||
to costs in space. Figure~\ref{tab:ilpmem} shows memory performance at
|
\JITI. The table presents data obtained at a point near the end of
|
||||||
a point near the end of execution. Numbers are given in KB. Because
|
execution. Because dynamic memory expands and contracts, we chose a
|
||||||
dynamic memory expands and contracts, we chose a point where dynamic
|
point where memory usage should be at a maximum. The first two numbers
|
||||||
memory should be at maximum usage. The first five columns show data
|
show data usage on \emph{static} predicates. Static data-base sizes
|
||||||
usage on static predicates. The leftmost sub-column represents the
|
range from 146MB (\texttt{IE-Protein|_Extraction} to less than a MB
|
||||||
code used for clause; the next sub-columns represent space used in
|
(\texttt{Choline}, \texttt{Krki}, \texttt{Mesh}). Indexing code can be
|
||||||
indices for static predicates: the first column gives total usage,
|
more than the original code, as in \texttt{Mutagenesis}, or almost as
|
||||||
which consists of space used in the main tree, the expanded
|
much, eg, \texttt{IE-Protein\_Extraction}. In most cases the YAP \JITI
|
||||||
wait-nodes, and hash-tables.
|
adds at least a third and often a half to the original data-base. A
|
||||||
|
more detailed analysis shows the source of overhead to be very
|
||||||
|
different from dataset to dataset. In \texttt{IE-Protein|_Extraction}
|
||||||
|
the problem is that hash tables are very large. Hash tables are also
|
||||||
|
where most space is spent in \texttt{Susi}. In \texttt{BreastCancer}
|
||||||
|
hash tables are actually small, so most space is spent in
|
||||||
|
\TryRetryTrust chains. \texttt{Mutagenesis} is similar: even though
|
||||||
|
YAP spends a large effort in indexing it still generates long
|
||||||
|
\TryRetryTrust chains. Storing sets of matching clauses at \jitiSTAR
|
||||||
|
nodes takes usually over 10\% of total memory usage, but is never dominant.
|
||||||
|
|
||||||
Static data-base sizes range from 146MB to 666KB, the latter mostly in
|
This version of ALEPH uses the internal data-base to store the IDB.
|
||||||
system libraries. The impact of indexing code varies widely: it is
|
The size of reflects the search space, and is to some extent
|
||||||
more than the original code for \texttt{Mutagenesis}, almost as much
|
independent of the program's static data, although small applications
|
||||||
for \texttt{IE-Protein\_Extraction}, and in most cases it adds at
|
such as \texttt{Krki} do tend to have a small search space. ALEPH's
|
||||||
least a third and often a half to the original data-base. It is
|
author very carefully designed the system to work around overheads in
|
||||||
interesting to check the source of the space overhead: if the source
|
|
||||||
are hash-tables, we can expect this is because of highly-complex
|
|
||||||
indices. If overhead is in \emph{wait-nodes}, this again suggests a
|
|
||||||
sophisticated indexing structure. Overhead in the main tree may be
|
|
||||||
caused by a large number of nodes, or may be caused by \texttt{try}
|
|
||||||
nodes.
|
|
||||||
|
|
||||||
One first conclusion is that \emph{wait-nodes} are costly space-wise,
|
|
||||||
even if they are needed to achieve sensible compilation times. On the
|
|
||||||
other hand, whether the space is allocated to the tree or to the
|
|
||||||
hashes varies widely. \texttt{IE-Protein\_Extraction} is an example
|
|
||||||
where the indices seem very useful: most space was spent in the
|
|
||||||
hash-tables, although we still are paying much for \emph{wait-nodes}.
|
|
||||||
\texttt{BreastCancer} has very small hash-tables, because it
|
|
||||||
attributes range over small domains, but indexing is useful (we
|
|
||||||
believe this is because we are only interested in the first solution
|
|
||||||
in this case).
|
|
||||||
|
|
||||||
This version of the ILP system stores most dynamic data in the IDB.
|
|
||||||
The size of reflects the search space, and is largely independent of
|
|
||||||
the program's static data (notice that small applications such as
|
|
||||||
\texttt{Krki} do tend to have a small search space). Aleph's author
|
|
||||||
very carefully designed the system to work around overheads in
|
|
||||||
accessing the data-base, so indexing should not be as important. In
|
accessing the data-base, so indexing should not be as important. In
|
||||||
fact, indexing has a much lower space overhead in this case,
|
fact, indexing has a much lower space overhead in this case,
|
||||||
suggesting it is not so critical. On the other hand, looking at the
|
suggesting it is not so critical. A more detailed analysis shows tha
|
||||||
actual results shows that indexing is working well: most space is
|
indexing is working well: most space is spent on hashes tables and on
|
||||||
spent on hashes and the tree, little space is spent on \texttt{try}
|
internal nodes of tree, and relatively little space is spent on
|
||||||
instructions. It is hard to separate the contributions of JITI on
|
\TryRetryTrust chains.
|
||||||
static and dynamic data, but the results for \texttt{Mesh} and
|
|
||||||
\texttt{Mutagenesis}, where the JITI probably has little impact on
|
|
||||||
static code, suggest a factor of two from indexing on the IDB in this
|
|
||||||
case.
|
|
||||||
|
|
||||||
Last, we discuss a natural language application, Van Noord's FSA
|
|
||||||
toolbox. This is an implementation of a set of finite state automata
|
|
||||||
for natural language tasks. The system includes a test suite with 150
|
|
||||||
tasks. We selected the 10 tasks with longer-running times in the
|
|
||||||
single argument version.
|
|
||||||
|
|
||||||
\begin{table}[ht]
|
|
||||||
\centering
|
|
||||||
\begin {tabular}{|l|r|r||r|r|r||} \hline %\cline{1-3}
|
|
||||||
& \multicolumn{2}{|c||}{\bf Time in msec.} &
|
|
||||||
\multicolumn{3}{|c||}{\bf Dynamic Space in KB.} \\
|
|
||||||
{\bf Benchs.} & \bf $A_1$ & \bf JITI & \bf Clause & \multicolumn{2}{|c||}{\bf Indices} \\
|
|
||||||
& & & & \bf $A_1$ & \bf JITI \\
|
|
||||||
\hline
|
|
||||||
\texttt{k963} & 1944 & 684 & 1348 & 26 & 40 \\
|
|
||||||
\texttt{k961} & 1972 & 652 & 1348 & 26 & 40 \\
|
|
||||||
\texttt{k962} & 1996 & 668 & 1350 & 26 & 40 \\
|
|
||||||
\texttt{drg3} & 3532 & 3641 & 649 & 19 & 35 \\
|
|
||||||
\texttt{d2ph} & 3612 & 3667 & 649 & 19 & 35 \\
|
|
||||||
\texttt{d2m} & 3952 & 3668 & 649 & 19 & 35 \\
|
|
||||||
\texttt{ld1} & 4084 & 4016 & 649 & 19 & 35 \\
|
|
||||||
\texttt{dg5} & 6084 & 1352 & 3305 & 39 & 61 \\
|
|
||||||
\texttt{g2p} & 25212& 14120 & 10373 & 47 & 67 \\
|
|
||||||
\texttt{tl3} & 74476& 14925 & 14306 & 70 & 49 \\
|
|
||||||
\hline
|
|
||||||
\end{tabular}
|
|
||||||
\caption{Performance on a Natural Language Application}
|
|
||||||
\label{tab:fsa}
|
|
||||||
\end{table}
|
|
||||||
|
|
||||||
FSA is very different from the two previous examples. These are
|
|
||||||
relatively complex algorithms, and there is relatively little
|
|
||||||
``data''. Even so, Table~\ref{tab:fsa} shows significant speedups from
|
|
||||||
using JITI. Note that Table~\ref{tab:fsa} only shows memory
|
|
||||||
performance on dynamic data: static data does not show very
|
|
||||||
significant differences. The results show two different types of
|
|
||||||
tasks. In cases such as \texttt{tl3} or \texttt{dg5} JITI gives a
|
|
||||||
significant speedup; in tasks such as \texttt{drg3} the difference
|
|
||||||
does not seem to be significant, and it some cases JITI is slower.
|
|
||||||
Analysis show that the tasks that do well are the tasks that use
|
|
||||||
dynamic predicates. In this case, indexing is beneficial. Although
|
|
||||||
there is an increase in total code, the indices are good: there is a
|
|
||||||
reduction in the code for \texttt{try} instructions, and an increase
|
|
||||||
in code for hash-tables, which indicates dynamic predicates are
|
|
||||||
indexing well. In tasks such as \texttt{drg3} and friends, the JITI
|
|
||||||
does not bring much benefits, whereas it spends extra time compiling
|
|
||||||
and takes extra space.
|
|
||||||
|
|
||||||
|
|
||||||
\section{Concluding Remarks}
|
\section{Concluding Remarks}
|
||||||
|
|
|
||||||
Reference in New Issue