16.2. SPARQL (original) (raw)

16.2.1. SPARQL Implementation Details

Virtuoso's RDF support includes in-built support for the SPARQL query language. It also includes a number of powerful extensions that cover path traversal and business intelligence features. In addition, there is in-built security based on Virtuoso's support for row level policy-based security, custom authentication, and named graphs.

The current implementation does not support some SPARQL features:

On the other hand, Virtuoso implements some extensions to SPARQL:

The following listing shows the SPARQL grammar expressed in BNF, including all Virtuoso extensions but excluding rules for the syntax of each lexical element. Rule numbers in square brackets are from W3C normative SPARQL grammar. An asterisk indicates that the rule differs from the W3C grammar due to Virtuoso extensions -[Virt] means that the rule is Virtuoso-specific,[DML] indicates a data manipulation language extension from SPARUL.

[1]* Query ::= Prolog ( QueryBody | SparulAction* | ( QmStmt ('.' QmStmt)* '.'? ) ) [1] QueryBody ::= SelectQuery | ConstructQuery | DescribeQuery | AskQuery [2]* Prolog ::= Define* BaseDecl? PrefixDecl* [Virt] Define ::= 'DEFINE' QNAME (QNAME | Q_IRI_REF | String ) [3] BaseDecl ::= 'BASE' Q_IRI_REF [4] PrefixDecl ::= 'PREFIX' QNAME_NS Q_IRI_REF [5]* SelectQuery ::= 'SELECT' 'DISTINCT'? ( ( Retcol ( ','? Retcol )* ) | '' ) DatasetClause WhereClause SolutionModifier [6] ConstructQuery ::= 'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier DatasetClause* WhereClause? SolutionModifier [8] AskQuery ::= 'ASK' DatasetClause* WhereClause [9] DatasetClause ::= 'FROM' ( DefaultGraphClause | NamedGraphClause ) [10]* DefaultGraphClause ::= SourceSelector SpongeOptionList? [11]* NamedGraphClause ::= 'NAMED' SourceSelector SpongeOptionList? [Virt] SpongeOptionList ::= 'OPTION' '(' ( SpongeOption ( ',' SpongeOption )* )? ')' [Virt] SpongeOption ::= QNAME PrecodeExpn [Virt] PrecodeExpn ::= Expn (* Only global variables can occur in Expn, local cannot ) [13] WhereClause ::= 'WHERE'? GroupGraphPattern [14] SolutionModifier ::= OrderClause? ((LimitClause OffsetClause?) | (OffsetClause LimitClause?))? [15] OrderClause ::= 'ORDER' 'BY' OrderCondition+ [16] OrderCondition ::= ( 'ASC' | 'DESC' )? ( FunctionCall | Var | ( '(' Expn ')' ) | ( '[' Expn ']' ) ) [17] LimitClause ::= 'LIMIT' INTEGER [17] LimitClause ::= 'LIMIT' INTEGER [18] OffsetClause ::= 'OFFSET' INTEGER [18] OffsetClause ::= 'OFFSET' INTEGER [19]* GroupGraphPattern ::= '{' ( GraphPattern | SelectQuery ) '}' [20] GraphPattern ::= Triples? ( GraphPatternNotTriples '.'? GraphPattern )? [21]* GraphPatternNotTriples ::= QuadMapGraphPattern | OptionalGraphPattern | GroupOrUnionGraphPattern | GraphGraphPattern | Constraint [22] OptionalGraphPattern ::= 'OPTIONAL' GroupGraphPattern [Virt] QuadMapGraphPattern ::= 'QUAD' 'MAP' ( IRIref | '' ) GroupGraphPattern [23] GraphGraphPattern ::= 'GRAPH' VarOrBlankNodeOrIRIref GroupGraphPattern [24] GroupOrUnionGraphPattern ::= GroupGraphPattern ( 'UNION' GroupGraphPattern ) [25]* Constraint ::= 'FILTER' ( ( '(' Expn ')' ) | BuiltInCall | FunctionCall ) [26]* ConstructTemplate ::= '{' ConstructTriples '}' [27] ConstructTriples ::= ( Triples1 ( '.' ConstructTriples )? )? [28] Triples ::= Triples1 ( '.' Triples? )? [29] Triples1 ::= VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList [30] PropertyList ::= PropertyListNotEmpty? [31] PropertyListNotEmpty ::= Verb ObjectList ( ';' PropertyList )? [32]* ObjectList ::= ObjGraphNode ( ',' ObjectList )? [Virt] ObjGraphNode ::= GraphNode TripleOptions? [Virt] TripleOptions ::= 'OPTION' '(' TripleOption ( ',' TripleOption )? ')' [Virt] TripleOption ::= 'INFERENCE' ( QNAME | Q_IRI_REF | SPARQL_STRING ) [33] Verb ::= VarOrBlankNodeOrIRIref | 'a' [34] TriplesNode ::= Collection | BlankNodePropertyList [35] BlankNodePropertyList ::= '[' PropertyListNotEmpty ']' [36] Collection ::= '(' GraphNode* ')' [37] GraphNode ::= VarOrTerm | TriplesNode [38] VarOrTerm ::= Var | GraphTerm [39]* VarOrIRIrefOrBackquoted ::= Var | IRIref | Backquoted [40]* VarOrBlankNodeOrIRIrefOrBackquoted ::= Var | BlankNode | IRIref | Backquoted [Virt] Retcol ::= ( Var | ( '(' Expn ')' ) | RetAggCall ) ( 'AS' ( VAR1 | VAR2 ) )? [Virt] RetAggCall ::= AggName '(', ( '' | ( 'DISTINCT'? Var ) ) ')' [Virt] AggName ::= 'COUNT' | 'AVG' | 'MIN' | 'MAX' | 'SUM' [41] Var ::= VAR1 | VAR2 | GlobalVar | ( Var ( '+>' | '>' ) IRIref ) [Virt] GlobalVar ::= QUEST_COLON_PARAMNAME | DOLLAR_COLON_PARAMNAME | QUEST_COLON_PARAMNUM | DOLLAR_COLON_PARAMNUM [42] GraphTerm ::= IRIref | RDFLiteral | ( '-' | '+' )? NumericLiteral | BooleanLiteral | BlankNode | NIL | Backquoted [Virt] Backquoted ::= '' Expn '' [43] Expn ::= ConditionalOrExpn [44] ConditionalOrExpn ::= ConditionalAndExpn ( '||' ConditionalAndExpn )* [45] ConditionalAndExpn ::= ValueLogical ( '&&' ValueLogical )* [46] ValueLogical ::= RelationalExpn [47]* RelationalExpn ::= NumericExpn ( ( ('='|'!='|'<'|'>'|'<='|'>='|'LIKE') NumericExpn ) | ( 'IN' '(' Expns ')' ) )? [49] AdditiveExpn ::= MultiplicativeExpn ( ('+'|'-') MultiplicativeExpn )* [50] MultiplicativeExpn ::= UnaryExpn ( (''|'/') UnaryExpn ) [51] UnaryExpn ::= ('!'|'+'|'-')? PrimaryExpn [58] PrimaryExpn ::= BracketedExpn | BuiltInCall | IRIrefOrFunction | RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | Var [55] IRIrefOrFunction ::= IRIref ArgList? [52]* BuiltInCall ::= ( 'STR' '(' Expn ')' ) | ( 'IRI' '(' Expn ')' ) | ( 'LANG' '(' Expn ')' ) | ( 'LANGMATCHES' '(' Expn ',' Expn ')' ) | ( 'DATATYPE' '(' Expn ')' ) | ( 'BOUND' '(' Var ')' ) | ( 'sameTERM' '(' Expn ',' Expn ')' ) | ( 'isIRI' '(' Expn ')' ) | ( 'isURI' '(' Expn ')' ) | ( 'isBLANK' '(' Expn ')' ) | ( 'isLITERAL' '(' Expn ')' ) | RegexExpn [53] RegexExpn ::= 'REGEX' '(' Expn ',' Expn ( ',' Expn )? ')' [54] FunctionCall ::= IRIref ArgList [56]* ArgList ::= ( NIL | '(' Expns ')' ) [Virt] Expns ::= Expn ( ',' Expn )* [59] NumericLiteral ::= INTEGER | DECIMAL | DOUBLE [60] RDFLiteral ::= String ( LANGTAG | ( '^^' IRIref ) )? [61] BooleanLiteral ::= 'true' | 'false' [63] IRIref ::= Q_IRI_REF | QName [64] QName ::= QNAME | QNAME_NS [65]* BlankNode ::= BLANK_NODE_LABEL | ( '[' ']' ) [DML] SparulAction ::= CreateAction | DropAction | LoadAction | InsertAction | InsertDataAction | DeleteAction | DeleteDataAction | ModifyAction | ClearAction [DML]* InsertAction ::= 'INSERT' ( ( 'IN' | 'INTO ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )? [DML]* InsertDataAction ::= 'INSERT' 'DATA' ( ( 'IN' | 'INTO ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn ConstructTemplate [DML]* DeleteAction ::= 'DELETE' ( 'FROM' 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )? [DML]* DeleteDataAction ::= 'DELETE' 'DATA' ( 'FROM' 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn ConstructTemplate [DML]* ModifyAction ::= 'MODIFY' ( 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn? 'DELETE' ConstructTemplate 'INSERT' ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )? [DML]* ClearAction ::= 'CLEAR' ( 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn )? [DML]* LoadAction ::= 'LOAD' PrecodeExpn ( ( 'IN' | 'INTO' ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn )? [DML]* CreateAction ::= 'CREATE' 'SILENT'? 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn [DML]* DropAction ::= 'DROP' 'SILENT'? 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn [Virt] QmStmt ::= QmSimpleStmt | QmCreateStorage | QmAlterStorage [Virt] QmSimpleStmt ::= QmCreateIRIClass | QmCreateLiteralClass | QmDropIRIClass | QmDropLiteralClass | QmCreateIRISubclass | QmDropQuadStorage | QmDropQuadMap [Virt] QmCreateIRIClass ::= 'CREATE' 'IRI' 'CLASS' QmIRIrefConst ( ( String QmSqlfuncArglist ) | ( 'USING' QmSqlfuncHeader ',' QmSqlfuncHeader ) ) [Virt] QmCreateLiteralClass ::= 'CREATE' 'LITERAL' 'CLASS' QmIRIrefConst 'USING' QmSqlfuncHeader ',' QmSqlfuncHeader QmLiteralClassOptions? [Virt] QmDropIRIClass ::= 'DROP' 'IRI' 'CLASS' QmIRIrefConst [Virt] QmDropLiteralClass ::= 'DROP' 'LITERAL' 'CLASS' QmIRIrefConst [Virt] QmCreateIRISubclass ::= 'IRI' 'CLASS' QmIRIrefConst 'SUBCLASS' 'OF' QmIRIrefConst [Virt] QmIRIClassOptions ::= 'OPTION' '(' QmIRIClassOption (',' QmIRIClassOption)* ')' [Virt] QmIRIClassOption ::= 'BIJECTION' | 'DEREF' | 'RETURNS' STRING ('UNION' STRING)* [Virt] QmLiteralClassOptions ::= 'OPTION' '(' QmLiteralClassOption (',' QmLiteralClassOption)* ')' [Virt] QmLiteralClassOption ::= ( 'DATATYPE' QmIRIrefConst ) | ( 'LANG' STRING ) | ( 'LANG' STRING ) | 'BIJECTION' | 'DEREF' | 'RETURNS' STRING ('UNION' STRING)* [Virt] QmCreateStorage ::= 'CREATE' 'QUAD' 'STORAGE' QmIRIrefConst QmSourceDecl* QmMapTopGroup [Virt] QmAlterStorage ::= 'ALTER' 'QUAD' 'STORAGE' QmIRIrefConst QmSourceDecl* QmMapTopGroup [Virt] QmDropStorage ::= 'DROP' 'QUAD' 'STORAGE' QmIRIrefConst [Virt] QmDropQuadMap ::= 'DROP' 'QUAD' 'MAP' 'GRAPH'? QmIRIrefConst [Virt] QmDrop ::= 'DROP' 'GRAPH'? QmIRIrefConst [Virt] QmSourceDecl ::= ( 'FROM' QTABLE 'AS' PLAIN_ID QmTextLiteral* ) | ( 'FROM' PLAIN_ID 'AS' PLAIN_ID QmTextLiteral* ) | QmCondition [Virt] QmTextLiteral ::= 'TEXT' 'XML'? 'LITERAL' QmSqlCol ( 'OF' QmSqlCol )? QmTextLiteralOptions? [Virt] QmTextLiteralOptions ::= 'OPTION' '(' QmTextLiteralOption ( ',' QmTextLiteralOption )* ')' [Virt] QmMapTopGroup ::= '{' QmMapTopOp ( '.' QmMapTopOp )* '.'? '}' [Virt] QmMapTopOp ::= QmMapOp | QmDropQuadMap | QmDrop [Virt] QmMapGroup ::= '{' QmMapOp ( '.' QmMapOp )* '.'? '}' [Virt] QmMapOp ::= ( 'CREATE' QmIRIrefConst 'AS' QmMapIdDef ) | ( 'CREATE' 'GRAPH'? QmIRIrefConst 'USING' 'STORAGE' QmIRIrefConst QmOptions? ) | ( QmNamedField+ QmOptions? QmMapGroup ) | QmTriples1 [Virt] QmMapIdDef ::= QmMapTriple | ( QmNamedField+ QmOptions? QmMapGroup ) [Virt] QmMapTriple ::= QmFieldOrBlank QmVerb QmObjField [Virt] QmTriples1 ::= QmFieldOrBlank QmProps [Virt] QmNamedField ::= ('GRAPH'|'SUBJECT'|'PREDICATE'|'OBJECT') QmField [Virt] QmProps ::= QmProp ( ';' QmProp )? [Virt] QmProp ::= QmVerb QmObjField ( ',' QmObjField )* [Virt] QmObjField ::= QmFieldOrBlank QmCondition* QmOptions? [Virt] QmIdSuffix ::= 'AS' QmIRIrefConst [Virt] QmVerb ::= QmField | ( '[' ']' ) | 'a' [Virt] QmFieldOrBlank ::= QmField | ( '[' ']' ) [Virt] QmField ::= NumericLiteral | RdfLiteral | ( QmIRIrefConst ( '(' ( QmSqlCol ( ',' QmSqlCol )* )? ')' )? ) | QmSqlCol [Virt] QmCondition ::= 'WHERE' ( ( '(' SQLTEXT ')' ) | String ) [Virt] QmOptions ::= 'OPTION' '(' QmOption ( ',' QmOption )* ')' [Virt] QmOption ::= ( 'SOFT'? 'EXCLUSIVE' ) | ( 'ORDER' INTEGER ) | ( 'USING' PLAIN_ID ) [Virt] QmSqlfuncHeader ::= 'FUNCTION' SQL_QTABLECOLNAME QmSqlfuncArglist 'RETURNS' QmSqltype [Virt] QmSqlfuncArglist ::= '(' ( QmSqlfuncArg ( ',' QmSqlfuncArg )* )? ')' [Virt] QmSqlfuncArg ::= ('IN' | QmSqlId) QmSqlId QmSqltype [Virt] QmSqltype ::= QmSqlId ( 'NOT' 'NULL' )? [Virt] QmSqlCol ::= QmSqlId | spar_qm_sql_id [Virt] QmSqlId ::= PLAIN_ID | 'TEXT' | 'XML' [Virt] QmIRIrefConst ::= IRIref | ( 'IRI' '(' String ')' )

Example: Using OFFSET and LIMIT

Virtuoso uses a zero-based index for OFFSET. Thus, in the example below, the query returns 1000 rows starting from, and including, record 9001 of the result set. Note that the default value of the MaxSortedTopRows parameter in the [Parameters] section of the virtuoso.ini configuration file defaults to 10000, so in this example its value will need to have been increased beforehand.

SQL>SELECT ?name ORDER BY ?name OFFSET 9000 LIMIT 1000

LIMIT applies to the solution resulting from the graph patterns specified in the WHERE CLAUSE. This implies that SELECT and CONSTRUCT/DESCRIBE queries will behave a little differently. In the case of a SELECT, there is a straight translation i.e. LIMIT 4 implies 4 records in the result set. In the case of CONSTRUCTs where the solution is a graph (implying that the existence of duplicates and/or unbound variables is common) LIMIT is basically a maximum triples threshold of: [Solution Triples] x [LIMIT].

Example query:

SQL>SPARQL prefix dct:http://purl.org/dc/terms/ prefix rdfs:http://www.w3.org/2000/01/rdf-schema#

CONSTRUCT { ?resource dct:title ?title ; a ?type }

FROM http://msone.computas.no/graphs/inferred/classification FROM http://msone.computas.no/graphs FROM http://msone.computas.no/graphs/instance/nfi FROM http://msone.computas.no/graphs/instance/mo FROM http://msone.computas.no/graphs/ontology/mediasone FROM http://msone.computas.no/graphs/vocab/mediasone FROM http://msone.computas.no/graphs/inferred/nfi/realisation1 FROM http://msone.computas.no/graphs/inferred/mo/realisation1 FROM http://msone.computas.no/graphs/inferred/nfi/realisation2 FROM http://msone.computas.no/graphs/inferred/mo/realisation2 FROM http://msone.computas.no/graphs/inferred/agent-classification FROM http://msone.computas.no/graphs/ontology/mediasone/agent

WHERE { { ?resource a ?type . FILTER (?type = http://www.w3.org/2002/07/owl#Class ) . ?resource rdfs:label ?title . } UNION { ?resource a ?type . FILTER (?type in ( http://musicbrainz.org/mm/mm-2.1#Track , http://www.csd.abdn.ac.uk/~ggrimnes/dev/imdb/IMDB#Movie , http://xmlns.com/foaf/0.1/Image , http://www.computas.com/mediasone#Text ) ) . ?resource dct:title ?title . } FILTER regex(?title, "turi", "i") } ORDER BY ?title LIMIT 4 OFFSET 0

Example: Prevent Limits of Sorted LIMIT/OFFSET query

The DBpedia SPARQL endpoint is configured with the following INI setting:

MaxSortedTopRows = 40000

The setting above sets a threshold for sorted rows. Thus, when using basic SPARQL queries that include OFFSET and LIMIT the following query will still exist the hard limit set in the INI:

DEFINE sql:big-data-const 0 SELECT DISTINCT ?p ?s FROM http://dbpedia.org WHERE { ?s ?p http://dbpedia.org/resource/Germany } ORDER BY ASC(?p) OFFSET 40000 LIMIT 1000

returns the following error on execution:

HttpException: 500 SPARQL Request Failed

Virtuoso 22023 Error SR353: Sorted TOP clause specifies more then 41000 rows to sort. Only 40000 are allowed. Either decrease the offset and/or row count or use a scrollable cursor

To prevent the problem outlined above you can leverage the use of subqueries which make better use of temporary storage associated with this kind of quest. An example would take the form:

SELECT ?p ?s WHERE { { SELECT DISTINCT ?p ?s FROM http://dbpedia.org WHERE { ?s ?p http://dbpedia.org/resource/Germany } ORDER BY ASC(?p) } } OFFSET 50000 LIMIT 1000

SPARQL and XQuery Core Function Library

In the current implementation, the XQuery Core Function Library is not available from SPARQL.

As a temporary workaround, string parsing functions are made available, because they are widely used in W3C DAWG examples and the like. They are:

xsd:boolean (in strg any) returns integer xsd:dateTime (in strg any) returns datetime xsd:double (in strg varchar) returns double precision xsd:float (in strg varchar) returns float xsd:integer (in strg varchar) returns integer

(assuming that the query contains the declaration: 'PREFIX xsd: http://www.w3.org/2001/XMLSchema#')