A Decision Matrix for the Evaluation of Triplestores for Use in a Virtual Research Environment

Authors: Tristan O’Neill, Trina Myers, Jarrod Trevathan

Abstract:

The Tropical Data Hub (TDH) is a virtual research environment that provides researchers with an e-research infrastructure to congregate significant tropical data sets for data reuse, integration, searching, and correlation. However, researchers often require data and metadata synthesis across disciplines for cross-domain analyses and knowledge discovery. A triplestore offers a semantic layer to achieve a more intelligent method of search to support the synthesis requirements by automating latent linkages in the data and metadata. Presently, the benchmarks to aid the decision of which triplestore is best suited for use in an application environment like the TDH are limited to performance. This paper describes a new evaluation tool developed to analyze both features and performance. The tool comprises a weighted decision matrix to evaluate the interoperability, functionality, performance, and support availability of a range of integrated and native triplestores to rank them according to requirements of the TDH.

Keywords: Virtual research environment, Semantic Web, performance analysis, tropical data hub.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1088782

References:

[1] T. Myers, J. Trevathan, and I. Atkinson, "The Tropical Data Hub: A Virtual Research Environment for tropical science knowledge and discovery," International Journal of Environmental, Cultural, Economic and Social Sustainability, January 2012.
[2] G. Antoniou and F. van Harmelen, A Semantic Web primer (2nd edition), 2nd ed. Cambridge, MA, USA: The MIT Press, 2008.
[3] D. Allemang and J. Hendler, Semantic Web for the working ontologist, 2nd Edition: effective modeling in RDFS and OWL. Burlington, MA, USA: Morgan Kaufmann, 2011.
[4] W3C. (2012) The Linking Open Data Project (Online). Available: http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData
[5] T. Heath and C. Bizer, "Linked Data: Evolving the Web into a Global Data Space," in Synthesis Lectures on the Semantic Web: Theory and Technology. vol. 1, J. Hendler and F. vanHarmelen, Eds.: Morgan & Claypool Publishers, 2011, pp. 1-136.
[6] H. Alani, W. Hall, K. O'Hara, N. Shadbolt, M. Szomszor, and P. Chandler, "Building a pragmatic Semantic Web," IEEE Intell. Syst., vol. 23, pp. 61-68, May-June 2008.
[7] T. O'Neill. (2012) Weighted decision matrix triplestore evaluation model (Online). Available: https://docs.google.com/folder/d/0B7rjeZeCV0l-Z3VxckphM2tQVzQ/edit
[8] W3C. (2012) RDF Store Benchmarking (Online). Available: http://www.w3.org/wiki/RdfStoreBenchmarking
[9] M. Mohamed, L. Jens, A. ren, and N. Axel-Cyrille Ngonga, "DBpedia SPARQL benchmark: performance assessment with real queries on real data," in Proceedings of the 10th international conference on The Semantic Web (ISWC2011). vol. 1 Bonn, Germany: Springer-Verlag, 2011, pp. 454-469.
[10] C. R. Rivero, A. Schultz, C. Bizer, and D. Ruiz, "Benchmarking the Performance of Linked Data Translation Systems," in Linked Data on the Web (LDOW2012) Workshop at WWW2012. Lyon, France., 2012.
[11] G. Yuanbo, "A Requirements Driven Framework for Benchmarking Semantic Web Knowledge Base Systems," IEEE Transactions on Knowledge and Data Engineering, vol. 19, pp. 297-309, 2007.
[12] F. Baader, D. Calvanese, D. L. McGuinness, D. Nardi, and P. F. Patel-Schneider, "The Description Logic handbook theory, implementation and applications (2nd ed.)," 2nd ed. ed, F. Baader, D. Calvanese, D. L. McGuinness, D. Nardi, and P. F. Patel-Schneider, Eds. Cambridge, USA: Cambridge University Press, 2007, p. 545.
[13] T. Myers, J. Trevathan, R. Johnstone, and I. Atkinson, "Supporting dynamic hypothesis modelling and alerts in marine environments.," in Coast to Coast 2012 Brisbane, Australia, 2012.
[14] S. Liang, P. Fodor, H. Wan, and M. Kifer, "OpenRuleBench: an analysis of the performance of rule engines," in 18th international conference on World Wide Web (WWW'09) Madrid, Spain: ACM, 2009.
[15] J. Trevathan, R. Johnstone, T. Chiffings, I. Atkinson, N. Bergmann, W. Read, S. Theiss, T. Myers, and T. Stevens, "SEMAT – The next generation of inexpensive marine environmental monitoring and measurement systems," Sensors, vol. 12, pp. 9711-9748, 2012.
[16] AIMS. (2012) AIMS Data Centre - Weather Observing System (Online). Available: http://data.aims.gov.au/awsqaqc/do/start.do
[17] J. Vanderwal. (2012) Australia decadal climate change predictions (~5km resolution) (Online). Available: https://eresearch.jcu.edu.au/ tdh/data/a308884e-87a1-49f4-b7f2-be19d3123018