476 lines
17 KiB
Plaintext
476 lines
17 KiB
Plaintext
\documentclass[11pt]{article}
|
|
\usepackage{pl}
|
|
\usepackage{html}
|
|
\usepackage{times}
|
|
|
|
\onefile
|
|
\htmloutput{html} % Output directory
|
|
\htmlmainfile{index} % Main document file
|
|
\bodycolor{white} % Page colour
|
|
|
|
\newcommand{\elem}[1]{{\tt\string<#1\string>}}
|
|
|
|
\begin{document}
|
|
|
|
\title{SWI-Prolog RDF parser}
|
|
\author{Jan Wielemaker \\
|
|
HCS, \\
|
|
University of Amsterdam \\
|
|
The Netherlands \\
|
|
E-mail: \email{jan@swi-prolog.org}}
|
|
|
|
\maketitle
|
|
|
|
\begin{abstract}
|
|
\url[RDF]{http://www.w3.org/RDF/} ({\bf R}esource {\bf D}escription {\bf
|
|
F}ormat) is a \url[W3C]{http://www.w3.org/} standard for expressing
|
|
meta-data about web-resources. It has two representations providing
|
|
the same semantics. RDF documents are normally transferred as XML
|
|
documents using the RDF-XML syntax. This format is unsuitable for
|
|
processing. The parser defined here converts an RDF-XML document into
|
|
the \jargon{triple} notation. The library \pllib{rdf_write} creates
|
|
an RDF/XML document from a list of triples.
|
|
\end{abstract}
|
|
|
|
\vfill
|
|
|
|
\tableofcontents
|
|
|
|
\vfill
|
|
\vfill
|
|
|
|
\newpage
|
|
|
|
\section{Introduction}
|
|
|
|
RDF is a promising standard for representing meta-data about documents
|
|
on the web as well as exchanging frame-based data (e.g. ontologies). RDF
|
|
is often associated with `semantics on the web'. It consists of a formal
|
|
data-model defined in terms of \jargon{triples}. In addition, a
|
|
\jargon{graph} model is defined for visualisation and an XML application
|
|
is defined for exchange.
|
|
|
|
`Semantics on the web' is also associated with the Prolog programming
|
|
language. It is assumed that Prolog is a suitable vehicle to reason with
|
|
the data expressed in RDF models. Most of the related web-infra
|
|
structure (e.g. XML parsers, DOM implementations) are defined in Java,
|
|
Perl, C or C+{+}.
|
|
|
|
Various routes are available to the Prolog user. Low-level XML parsing
|
|
is due to its nature best done in C or C+{+}. These languages produce
|
|
fast code. As XML/SGML are at the basis of most of the other web-related
|
|
formats we will benefit most here. XML and SGML, being very stable
|
|
specifications, make fast compiled languages even more attractive.
|
|
|
|
But what about RDF? RDF-XML is defined in XML, and provided with a
|
|
Prolog term representing the XML document processing it according to the
|
|
RDF syntax is quick and easy in Prolog. The alternative, getting yet
|
|
another library and language attached to the system, is getting less
|
|
attractive. In this document we explore the suitability of Prolog for
|
|
processing XML documents in general and into RDF in particular.
|
|
|
|
|
|
\section{Parsing RDF in Prolog}
|
|
|
|
We realised an RDF compiler in Prolog on top of the {\bf sgml2pl}
|
|
package (providing a name-space sensitive XML parser). The
|
|
transformation is realised in two passes.
|
|
|
|
The first pass rewrites the XML term into a Prolog term conveying the
|
|
same information in a more friendly manner. This transformation is
|
|
defined in a high-level pattern matching language defined on top of
|
|
Prolog with properties similar to DCG (Definite Clause Grammar).
|
|
|
|
The source of this translation is very close to the BNF notation used by
|
|
the \url[specification]{http://www.w3.org/TR/REC-rdf-syntax/}, so
|
|
correctness is `obvious'. Below is a part of the definition for RDF
|
|
containers. Note that XML elements are represented using a term of the
|
|
format:
|
|
|
|
\begin{quote}
|
|
\term{element}{Name, [AttrName = Value...], [Content ...]}
|
|
\end{quote}
|
|
|
|
\begin{code}
|
|
memberElt(LI) ::=
|
|
\referencedItem(LI).
|
|
memberElt(LI) ::=
|
|
\inlineItem(LI).
|
|
|
|
referencedItem(LI) ::=
|
|
element(\rdf(li),
|
|
[ \resourceAttr(LI) ],
|
|
[]).
|
|
|
|
inlineItem(literal(LI)) ::=
|
|
element(\rdf(li),
|
|
[ \parseLiteral ],
|
|
LI).
|
|
inlineItem(description(description, _, _, Properties)) ::=
|
|
element(\rdf(li),
|
|
[ \parseResource ],
|
|
\propertyElts(Properties)).
|
|
inlineItem(LI) ::=
|
|
element(\rdf(li),
|
|
[],
|
|
[\rdf_object(LI)]), !. % inlined object
|
|
inlineItem(literal(LI)) ::=
|
|
element(\rdf(li),
|
|
[],
|
|
[LI]). % string value
|
|
\end{code}
|
|
|
|
Expression in the rule that are prefixed by the \verb$\$ operator acts
|
|
as invocation of another rule-set. The body-term is converted into
|
|
a term where all rule-references are replaced by variables. The
|
|
resulting term is matched and translation of the arguments is achieved
|
|
by calling the appropriate rule. Below is the Prolog code for the
|
|
{\bf referencedItem} rule:
|
|
|
|
\begin{code}
|
|
referencedItem(A, element(B, [C], [])) :-
|
|
rdf(li, B),
|
|
resourceAttr(A, C).
|
|
\end{code}
|
|
|
|
Additional code can be added using a notation close to the Prolog
|
|
DCG notation. Here is the rule for a description, producing
|
|
properties both using {\bf propAttrs} and {\bf propertyElts}.
|
|
|
|
\begin{code}
|
|
description(description, About, BagID, Properties) ::=
|
|
element(\rdf('Description'),
|
|
\attrs([ \?idAboutAttr(About),
|
|
\?bagIdAttr(BagID)
|
|
| \propAttrs(PropAttrs)
|
|
]),
|
|
\propertyElts(PropElts)),
|
|
{ !, append(PropAttrs, PropElts, Properties)
|
|
}.
|
|
\end{code}
|
|
|
|
|
|
\section{Predicates}
|
|
|
|
The parser is designed to operate in various environments and therefore
|
|
provides interfaces at various levels. First we describe the top level
|
|
defined in \pllib{rdf}, simply parsing a RDF-XML file into a list of
|
|
triples. Please note these are {\em not} asserted into the database
|
|
because it is not necessarily the final format the user wishes to reason
|
|
with and it is not clean how the user wants to deal with multiple RDF
|
|
documents. Some options are using global URI's in one pool, in Prolog
|
|
modules or using an additional argument.
|
|
|
|
\begin{description}
|
|
\predicate{load_rdf}{2}{+File, -Triples}
|
|
Same as \term{load_rdf}{File, Triples, []}.
|
|
|
|
\predicate{load_rdf}{3}{+File, -Triples, +Options}
|
|
Read the RDF-XML file \arg{File} and return a list of \arg{Triples}.
|
|
\arg{Options} defines additional processing options. Currently defined
|
|
options are:
|
|
|
|
\begin{description}
|
|
\termitem{base_uri}{BaseURI}
|
|
If provided local identifiers and identifier-references are globalised
|
|
using this URI. If omited or the atom \verb$[]$, local identifiers are
|
|
not tagged.
|
|
|
|
\termitem{blank_nodes}{Mode}
|
|
If \arg{Mode} is \const{share} (default), blank-node properties (i.e.\
|
|
complex properties without identifier) are reused if they result in
|
|
exactly the same triple-set. Two descriptions are shared if their
|
|
intermediate description is the same. This means they should produce the
|
|
same set of triples in the same order. The value \const{noshare} creates
|
|
a new resource for each blank node.
|
|
|
|
\termitem{expand_foreach}{Boolean}
|
|
If \arg{Boolean} is \const{true}, expand \const{rdf:aboutEach} into
|
|
a set of triples. By default the parser generates
|
|
\term{rdf}{each(Container), Predicate, Subject}.
|
|
|
|
\termitem{lang}{Lang}
|
|
Define the initial language (i.e.\ pretend there is an \const{xml:lang}
|
|
declaration in an enclosing element).
|
|
|
|
\termitem{ignore_lang}{Bool}
|
|
If \const{true}, \const{xml:lang} declarations in the document are
|
|
ignored. This is mostly for compatibility with older versions of
|
|
this library that did not support language identifiers.
|
|
|
|
\termitem{convert_typed_literal}{:ConvertPred}
|
|
If the parser finds a literal with the \const{rdf:datatype}=\arg{Type}
|
|
attribute, call \term{ConvertPred}{+Type, +Content, -Literal}.
|
|
\arg{Content} is the XML element contentas returned by the XML
|
|
parser (a list). The predicate must unify \arg{Literal}
|
|
with a Prolog representation of \arg{Content} according to
|
|
\arg{Type} or throw an exception if the conversion cannot be made.
|
|
|
|
This option servers two purposes. First of all it can be used
|
|
to ignore type declarations for backward compatibility of this
|
|
library. Second it can be used to convert typed literals to
|
|
a meaningful Prolog representation. E.g.\ convert '42' to the
|
|
Prolog integer 42 if the type is \const{xsd:int} or a related
|
|
type.
|
|
|
|
\termitem{namespaces}{-List}
|
|
Unify \arg{List} with a list of \arg{NS}=\arg{URL} for each
|
|
encountered \const{xmlns}:\arg{NS}=\arg{URL} declaration found
|
|
in the source.
|
|
|
|
\termitem{entity}{+Name, +Value}
|
|
Overrule entity declaration in file. As it is common practice
|
|
to declare namespaces using entities in RDF/XML, this option
|
|
allows for changing the namespace without changing the file.
|
|
Multiple of these options are allowed.
|
|
\end{description}
|
|
|
|
The \arg{Triples} list is a list of \term{rdf}{Subject, Predicate,
|
|
Object} triples. \arg{Subject} is either a plain resource (an atom),
|
|
or one of the terms \term{each}{URI} or \term{prefix}{URI} with the
|
|
obvious meaning. \arg{Predicate} is either a plain atom for
|
|
explicitely non-qualified names or a term
|
|
\mbox{\arg{NameSpace}{\bf :}\arg{Name}}. If \arg{NameSpace} is the
|
|
defined RDF name space it is returned as the atom \const{rdf}.
|
|
Finally, \arg{Object} is a URI, a \arg{Predicate} or a term of the
|
|
format \term{literal}{Value} for literal values. \arg{Value} is
|
|
either a plain atom or a parsed XML term (list of atoms and elements).
|
|
\end{description}
|
|
|
|
|
|
\subsection{RDF Object representation} \label{sec:rdfobject}
|
|
|
|
The \emph{Object} (3rd) part of a triple can have several different
|
|
types. If the object is a resource it is returned as either a plain
|
|
atom or a term \mbox{\arg{NameSpace}{\bf :}\arg{Name}}. If it is a
|
|
literal it is returned as \term{literal}{Value}, where \arg{Value}
|
|
takes one of the formats defined below.
|
|
|
|
\begin{itemlist}
|
|
\item [An atom]
|
|
If the literal \arg{Value} is a plain atom is a literal value not
|
|
subject to a datatype or \const{xml:lang} qualifier.
|
|
|
|
\item [\term{lang}{LanguageID, Atom}]
|
|
If the literal is subject to an \const{xml:lang} qualifier
|
|
\arg{LanguageID} specifies the language and \arg{Atom} the
|
|
actual text.
|
|
|
|
\item [A list]
|
|
If the literal is an XML literal as created by
|
|
\mbox{parseType="Literal"}, the raw output of the XML parser for the
|
|
content of the element is returned. This content is a list of
|
|
\term{element}{Name, Attributes, Content} and atoms for CDATA parts as
|
|
described with the SWI-Prolog \url[SGML/XML
|
|
parser]{http://www.swi-prolog.org/packages/sgml2pl.html}
|
|
|
|
\item [\term{type}{Type, StringValue}]
|
|
If the literal has an \verb$rdf:datatype=$\arg{Type} a term of this
|
|
format is returned.
|
|
\end{itemlist}
|
|
|
|
|
|
\subsection{Name spaces}
|
|
|
|
XML name spaces are identified using a URI. Unfortunately various URI's
|
|
are in common use to refer to RDF. The \file{rdf_parser.pl} module
|
|
therefore defines the namespace as a multifile/1 predicate, that can be
|
|
extended by the user. For example, to parse the \url[Netscape
|
|
OpenDirectory]{http://www.mozilla.org/rdf/doc/inference.html}
|
|
\file{structure.rdf} file, the following declarations are used:
|
|
|
|
\begin{code}
|
|
:- multifile
|
|
rdf_parser:rdf_name_space/1.
|
|
|
|
rdf_parser:rdf_name_space('http://www.w3.org/TR/RDF/').
|
|
rdf_parser:rdf_name_space('http://directory.mozilla.org/rdf').
|
|
rdf_parser:rdf_name_space('http://dmoz.org/rdf').
|
|
\end{code}
|
|
|
|
The initial definition of this predicate is given below.
|
|
|
|
\begin{code}
|
|
rdf_name_space('http://www.w3.org/1999/02/22-rdf-syntax-ns#').
|
|
rdf_name_space('http://www.w3.org/TR/REC-rdf-syntax').
|
|
\end{code}
|
|
|
|
|
|
\subsection{Low-level access}
|
|
|
|
The above defined load_rdf/[2,3] is not always suitable. For example, it
|
|
cannot deal with documents where the RDF statement is embedded in an XML
|
|
document. It also cannot deal with really large documents (e.g.\ the
|
|
Netscape OpenDirectory project, currently about 90 MBytes), without huge
|
|
amounts of memory.
|
|
|
|
For really large documents, the {\bf sgml2pl} parser can be programmed
|
|
to handle the content of a specific element (i.e. \elem{rdf:RDF})
|
|
element-by-element. The parsing primitives defined in this section
|
|
can be used to process these one-by-one.
|
|
|
|
\begin{description}
|
|
\predicate{xml_to_rdf}{3}{+XML, +BaseURI, -Triples}
|
|
Process an XML term produced by load_structure/3 using the
|
|
\term{dialect}{xmlns} output option. \arg{XML} is either
|
|
a complete \elem{rdf:RDF} element, a list of RDF-objects
|
|
(container or description) or a single description of container.
|
|
|
|
\predicate{process_rdf}{3}{+Input, :OnTriples, +Options}
|
|
|
|
Exploits the call-back interface of {\bf sgml2pl}, calling
|
|
\term{\arg{OnTriples}}{Triples, File:Line} with the list of triples
|
|
resulting from a single top level RDF object for each RDF element in the
|
|
input as well as the source-location where the description started.
|
|
\arg{Input} is either a file name or term \term{stream}{Stream}. When
|
|
using a stream all triples are associated to the value of the
|
|
\const{base_uri} option. This predicate can be used to process arbitrary
|
|
large RDF files as the file is processed object-by-object. The example
|
|
below simply asserts all triples into the database:
|
|
|
|
\begin{code}
|
|
assert_list([], _).
|
|
assert_list([H|T], Source) :-
|
|
assert(H),
|
|
assert_list(T, Source).
|
|
|
|
?- process_rdf('structure,rdf', assert_list, []).
|
|
\end{code}
|
|
|
|
\arg{Options} are described with load_rdf/3. The option
|
|
\const{expand_foreach} is not supported as the container may be in a
|
|
different description. Additional it provides \const{embedded}:
|
|
|
|
\begin{description}
|
|
\termitem{embedded}{Boolean}
|
|
The predicate process_rdf/3 processes arbitrary XML documents, only
|
|
interpreting the content of \const{rdf:RDF} elements. If this option
|
|
is \const{false} (default), it gives a warning on elements that are
|
|
not processed. The option \term{embedded}{true} can be used to
|
|
process RDF embedded in \jargon{xhtml} without warnings.
|
|
\end{description}
|
|
|
|
\end{description}
|
|
|
|
|
|
|
|
|
|
\section{Writing RDF graphs}
|
|
|
|
The library \pllib{rdf_write} provides the inverse of load_rdf/2 using
|
|
the predicate rdf_write_xml/2. In most cases the RDF parser is used in
|
|
combination with the Semweb package providing \pllib{semweb/rdf_db}.
|
|
This library defines rdf_save/2 to save a named RDF graph from the
|
|
database to a file. This library writes a list of rdf terms to a stream.
|
|
It has been developed for the SeRQL server which computes an RDF graph
|
|
that needs to be transmitted in an HTTP request. As we see this as a
|
|
typical use-case scenario the library only provides writing to a stream.
|
|
|
|
\begin{description}
|
|
\predicate{rdf_write_xml}{2}{+Stream, +Triples}
|
|
Write an RDF/XML document to \arg{Stream} from the list of \arg{Triples}.
|
|
\arg{Stream} must use one of the following Prolog stream encodings:
|
|
\const{ascii}, \const{iso_latin_1} or \const{utf8}. Characters that
|
|
cannot be represented in the encoding are represented as XML entities.
|
|
Using ASCII is a good idea for documents that can be represented almost
|
|
completely in ASCII. For more international documents using UTF-8 creates
|
|
a more compact document that is easier to read.
|
|
|
|
\begin{code}
|
|
rdf_write(File, Triples) :-
|
|
open(File, write, Out, [encoding(utf8)]),
|
|
call_cleanup(rdf_write_xml(Out, Triples),
|
|
close(Out)).
|
|
\end{code}
|
|
\end{description}
|
|
|
|
|
|
\section{Testing the RDF translator}
|
|
|
|
A test-suite and driver program are provided by \file{rdf_test.pl} in
|
|
the source directory. To run these tests, load this file into Prolog in
|
|
the distribution directory. The test files are in the directory
|
|
\file{suite} and the proper output in \file{suite/ok}. Predicates
|
|
provided by \file{rdf_test.pl}:
|
|
|
|
\begin{description}
|
|
\predicate{suite}{1}{+N}
|
|
Run test \arg{N} using the file \file{suite/tN.rdf} and display the
|
|
RDF source, the intermediate Prolog representation and the resulting
|
|
triples.
|
|
\predicate{passed}{1}{+N}
|
|
Process \file{suite/tN.rdf} and store the resulting triples in
|
|
\file{suite/ok/tN.pl} for later validation by test/0.
|
|
\predicate{test}{0}{}
|
|
Run all tests and classify the result.
|
|
\end{description}
|
|
|
|
\appendix
|
|
|
|
\section{Metrics}
|
|
|
|
It took three days to write and one to document the Prolog RDF parser.
|
|
A significant part of the time was spent understanding the RDF
|
|
specification.
|
|
|
|
The size of the source (including comments) is given in the table
|
|
below.
|
|
|
|
\begin{center}
|
|
\begin{tabular}{|rrr|l|l|}
|
|
\hline
|
|
\bf lines & \bf words & \bf bytes & \bf file & \bf function \\
|
|
\hline
|
|
109 & 255 & 2663 & rdf.pl & Driver program \\
|
|
312 & 649 & 6416 & rdf_parser.pl & 1-st phase parser \\
|
|
246 & 752 & 5852 & rdf_triple.pl & 2-nd phase parser \\
|
|
126 & 339 & 2596 & rewrite.pl & rule-compiler \\
|
|
\hline
|
|
793 & 1995 & 17527 & total & \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
|
|
|
|
We also compared the performance using an RDF-Schema file generated by
|
|
\url[Protege-2000]{http://www.smi.stanford.edu/projects/protege/} and
|
|
interpreted as RDF. This file contains 162 descriptions in 50 Kbytes,
|
|
resulting in 599 triples. Environment: Intel Pentium-II/450 with
|
|
384 Mbytes memory running SuSE Linux 6.3.
|
|
|
|
The parser described here requires 0.15 seconds excluding 0.13 seconds
|
|
Prolog startup time to process this file. The \url[Pro
|
|
Solutions]{http://www.pro-solutions.com/rdfdemo/} parser (written in
|
|
Perl) requires 1.5 seconds exluding 0.25 seconds startup time.
|
|
|
|
|
|
\section{Installation}
|
|
|
|
\subsection{Unix systems}
|
|
|
|
Installation on Unix system uses the commonly found {\em configure},
|
|
{\em make} and {\em make install} sequence. SWI-Prolog should be
|
|
installed before building this package. If SWI-Prolog is not installed
|
|
as \program{pl}, the environment variable \env{PL} must be set to the
|
|
name of the SWI-Prolog executable. Installation is now accomplished
|
|
using:
|
|
|
|
\begin{code}
|
|
% ./configure
|
|
% make
|
|
% make install
|
|
\end{code}
|
|
|
|
This installs the Prolog library files in \file{$PLBASE/library}, where
|
|
\file{$PLBASE} refers to the SWI-Prolog `home-directory'.
|
|
|
|
\subsection{Windows}
|
|
|
|
Run the file \file{setup.pl} by double clicking it. This will install
|
|
the required files into the SWI-Prolog directory and update the
|
|
library directory.
|
|
|
|
\end{document}
|
|
|
|
|