Smart Discovery of Cultural and Natural Tourist Routes

This paper presents a system designed to utilize innovative spatial interconnection technologies for sites and events of environmental, cultural and tourist interests. The system will discover and consolidate semantic information from multiple sources, providing the end-user the ability to organize and implement integrated and enhanced tours. The system, called e-xnilatis, will extend existing innovative techniques, including: semi-automated searching and extraction of real-time knowledge from online resources, automated discovery of points of interest as well as events, semantic integration, classification and hierarchy of information from various sources, spatial representation of content, personalized user experience and Augmented Reality (AR) for the interconnection of the digital with the natural environment. A complex online platform and applications for smart devices will be developed so that the user manages and receives the optimum route information along with AR experience when applicable. The platform will be an open architecture tool that, with the appropriate time-space constraints, will be able to create adaptive content. A typical example of this is the Egnatia motorway, where the e-xnilatis platform will be evaluated.


INTRODUCTION
Even though a wide variety of route recommendation systems has existed for years [17] [32], only until recently the hardware evolution [34], advanced data mining techniques [20] and studies upon human computer interaction [9] enabled those systems to incorporate more aspects than mere route suggestions to the user, such as personalized interfaces and content, augmented/virtual reality features, and so on. The technological advancements in multiple research fields led to the ambiguous era of the Internet of Things (IoT), rendering nowadays grandiose interdisciplinary projects plausible.
Within the scope of e-xnilatis various research interests emerge, from initial data aggregation and analysis to the final product served to the end user. First major issue constitutes the necessity for locating online content in a semi-automated manner and extracting information in real-time to aid in detecting places of particular interest (POIs), time-limited events and general environmental issues, such as traffic or weather. Apart from qualitative details ascribed to the established POIs, intentions of including geospatial data, for each POI, in order to semantically represent them are eligible. Consequently, the diversity of all the incoming information leads the implementation towards contriving innovative techniques upon semantic unification during mapping in a single unique ontological model, concerning all the indispensable knowledge proffered in online sources and personal information flowing from the end user to the system. An additional aspect of the e-xnilatis platform is the provision of augmented reality experience when applicable, including the representation of 2D and 3D artifacts in the user's smart-phone screen, baring in mind the geolocation details of the user also, such as the orientation, the distance from the object and the GPS coordinates. On top of the aforementioned technologies, a supplementary characteristic, which attaches added value to the concept, is that the user experience of the e-xnilatis platform is not intended to be immutable, on the contrary, personalization techniques will be taken into consideration to procure variable exposure to the user interface and supply recommended ranked routes according to personal traits such as profile details complemented by the user himself, queries to the system containing limitations about time or distance availability and interests at the moment, usage history and similarities with other users.
The e-xnilatis platform will be evaluated in pilot trials that will take place in the axis of Egnatia motorway, the longest motorway of Greece, with a total length of 670 Km. The Egnatia motorway crosses the regions of Epirus, Macedonia and Thrace, as well as all major cities, ports and airports in northern Greece. Moreover, it is linked to the neighboring countries (Albania, North Macedonia, Bulgaria and Turkey). It is indicative that the area of service and influence of the motorway accounted for is 36 % of the country's total population and 33% of the total gross national product. Additionally, along of the Egnatia motorway axis, an individual can find crowded sites and events of varying interest (environmental, cultural, historical, archaeological, folklore, religious, sports, artistic) that can form an integrated tourist route, and unique local tours.
The rest of the paper is organized as follows. In Section II, the state-of-the-art related work with this paper is mentioned. In Section III the architecture of the proposed system is presented and analyzed. In Section IV, the primary technological components of the data collection and analysis procedure are described. In Section V the augmented reality module and the personalization techniques are explained thoroughly. The distinct use case scenarios can be found in Section VI. Summarizing, Section VII elaborates on the future work and concludes the paper.

RELATED WORK
The project's intended deployments are not based on a single technology but rather on the integration of multiple technologies. The research process of designing an interconnected infrastructure for smart spatiotemporal integration of routes with additional features, comprises various components. The first components include the sensors interacting with the physical world as well as the available web data, the communication between server platforms and data protocols. The second category components in this project includes data representation, ontologies and reasoning mechanisms. The third stage components concern the route recommendations and the last one the augmented reality features offered to the end-user.
The continuous advancement in electronics allowed many devices e.g. smart phones and tablets to include multiple sensors. These devices through the incorporated sensors can measure different physical parameters and are deployed in different applications. The measured physical parameters can be from motion, position sensors to optical sensors and so on. It should be kept in mind though that those sensors can also be exploited independently as objects connected directly to a large integrated system [13].
Another source of useful content are the social media where numerous approaches have recently appeared in the literature on the problems of event detection tracking and summarization. There are two generalized event detection and tracking approaches: document-pivot [3] [24] and feature-pivot methods [8]. The former intends to cluster items related to the same event whereas the latter intends to first recognize the representative features of the underlying events in the collection and then detect events by leveraging these representative features. The core idea of a plethora of works in event summarization is the segmentation of items set into coherent topics or sub-topics and the selection of the most representative documents in each segment. For event segmentation several approaches have been proposed, ranging from Hidden Markov Models [7] to LDA extensions [4]. Regarding the selection of representative documents, either centroid-based [26] or graph-based ranking approaches [10] have been considered.
When it comes to websites, web data extraction techniques allow collecting large amounts of data from the Web. To extract data from a particular Web source by using a web wrapper, a procedure occurs where it seeks a set of data of interest and automatically extracts it from the page into a structured format [11]. The Web wrapper life cycle consists of three stages: (1) wrapper generation, in which the wrapper is fully constructed for a particular source of interest; (2) wrapper execution, in which the wrapper is executed on a set of target pages and data are extracted and stored; (3) wrapper maintenance, where the wrapper is updated due to changes in the HTML structure of the Web source.
Ontologies and technologies of Semantic Web are being widely used for the representation of the data. Ontologies are a means for a structured detailed description of an object or a concept. Its use focuses on semantically enhancing resources, through facilitating the understanding of the meaning of the metadata [31].
Event was one core ontology for event annotation [27]. This ontology was focused around the concept of event, seen here as the way by which cognitive agents classify arbitrary time/space regions. The agent class was derived from the "Friend of a Friend" (FOAF) ontology [6], a core ontology for social interconnections. In [30] researchers developed a more specialized ontology (LODE) based on the event ontology for publishing descriptions of historical events as Linked Data, and for mapping between other event-related vocabularies and ontologies. In [14] , an ontology of temporal concepts was developed, for describing the temporal properties of resources in the world or in Web pages. This ontology provides a vocabulary for expressing facts about topological (ordering) relations among instants and intervals, together with information about durations, and about temporal position including date-time information. Time positions and durations may be expressed using either the conventional (Gregorian) calendar and clock, or using another temporal reference system such as Unix-time, geologic time, or different calendars. Furthermore, the GeoSPARQL ontology [2] is a standard for representation and querying of geospatial linked data for the Semantic Web from the Open Geospatial Consortium (OGC). The definition of a small ontology based on well-understood OGC standards is intended to provide a standardized exchange basis for geospatial RDF data which can support both qualitative and quantitative spatial reasoning and querying with the SPARQL database query language. On the framework of this proposed project new ontologies have to be deployed that are not incorporated in the above-mentioned ontologies. These ontologies must annotate every aspect of available data, from incoming unstructured web information to points of interest.
Route recommendation systems are a task of increasing interest and of significant relativity to e-xnilatis project. Initial studies decomposed the problem in three steps [25]. In a first step, a quantity of first travel routes is determined on the basis of an optimization criterion, and location-independent and situation-specific user preferences derived therefrom. In the second step, the first travel routes are recalculated in sub-segments on the basis of location-dependent and situation-specific user preferences and finally, in the third step, the determined travel routes are prioritized on the basis of at least one rating criterion. Afterwards, attempts to include other variables when evaluating routes such as traffic [35] and personal interests [18] were of great success.
Apart from data extraction, representation and automatic content generation technologies, there is an increasing interest in visualization technologies and techniques to maximize user experience, like Virtual Reality (VR) and Augmented Reality (AR). Those technologies have been successfully used to visualize reconstructed archaeological sites [5] [21], virtual museums [33] [19] and other sites of digital heritage.
AR applications can create memorable user experiences by mixing real world information with synthetic data. Many system designers and digital marketers now perceived AR as a key means of increasing engagement with consumers and visitors. There are two ways that AR is being introduced in every day life: 1. through the use of mobile devices like smartphones and tablets, and 2. through wearable devices like light-weight see-through Hololens (by Microsoft), Google Glass, and SmartEyeglass (by Sony). Real-world objects are enhanced with additional information projected in place using Markers (small black and white boards like QR codes) and on demand leading to Ambient Intelligence (AI), a term used to describe ubiquitous computing plus social and intelligent interfaces. The combination of those technologies (mobile AR, AI) in the environment we live allow a wider sharing of digital resources. This kind of user experience and interaction necessitates the convergence of multiple research and technology areas. These include indoors and outdoors location tracking using Markerless object recognition [22], activity recognition, semantic representation, context-aware AR visualization and human-objects interaction [1]. Last but not least, the above technologies when combined with Personalization can increase the user satisfaction (personalized augmentation) [28].

OVERVIEW OF SYSTEM ARCHITECTURE
The proposed system architecture is presented in a high level design in Fig. 1. It consists of several modules, each containing clustered services that determine its functionality. It is intended to use Java for building the back-end services, PHP for certain service calls and Python for the front-end platform. For storing data and triplets, MongoDB and GraphDB constitute a viable solution. In addition, certain frameworks will be used such as Django, OpenLayers, Nominatim, etc. UTF-8 encoding will be used and enforced throughout the e-xnilatis platform to ensure correct and consistent handling of textual content. Furthermore, the preferred exchange format for structured data will be JSON, but in the extreme case where tools and/or modules do not support JSON, or when deemed appropriate, XML will also be a viable choice. For semantic data representation, the OWL 2 language will be the default option; the Turtle syntax will be utilized to serialise RDF graphs. A more detailed architecture of the system services data flow described in this chapter within each module is presented in Fig. 2. In more detail, the online data input module consists of various services accountable for aggregating targeted information from multiple varying sources. The social media crawling service will be transferring data from verified social media accounts in real-time.
In the same manner, the scrapping service will be responsible for scrapping content from trusted websites whereas the geoportal data collector API will be fetching data from sensors semi-automatically. Incorporated in this module may be considered the user feedback collector service, which will serve standardized evaluations from the end users to our system, as well as the user profile similarity calculator service, triggered when a novel user utilizes the e-xnilatis platform for the first time and expects recommendations based on activities of users with coherent interests.
The services mentioned above will contact directly the backend platform which will be under unremitting human supervision. Within this component all the necessary analysis and semantic unification will occur and when enabled by the administrator the structured now data will be passed on to populate the knowledge base. However, it is worth mentioning that the direct connection from the front-end platform towards the back-end platform implies that information regarding each user's unique profile and personal queries to the system will flow, be analyzed and mapped to suffice the ontological model.
The knowledge base is referred distinctly as it constitutes an intelligent means of storing mapped incoming streams of structured information in the form of triplets, abiding by the strict specifications defined during the design of the unique ontological model. As a smart type of server containing and representing semantically all the available data, the reasoning service resides on top of it, being able to infer additional knowledge ordinarily missed by a human, a procedure whose products follow a recycling route and conclude in being stored also in the knowledge base.
The rules of reasoning compose a specialized technique of querying the knowledge base and may vary from primitive requests to more sophisticated ones. The byproducts of these proceedings are utilized by both the personalization module and the route planning module. The profile-based recommendation service will be applying personalization algorithms on top of the reasoning results to achieve population reduction within the pool of POIs. Then the remaining ones will be stored once again in the knowledge base but this time they will be associated with a particular user. Serially, a further confinement will be enforced to the pluralism of POIs as additional constraints will be applied, such as time or distance. The few remaining POIs will be moved forward to the route planning module which by using a modified Dijkstra's shortest path algorithm based on distance and/or time will frame several ranked recommended routes shown in the user's smart device via the front-end platform.
The front-end platform will act as an intermediate between the complete back-end system and the end user, where basic authentication with credentials will provide access at user level. Personalized interfaces will be designed according to needs or circumstances which will enable the user to dispatch and receive information with the system as well as with other users since provision of social interactions through a platform is planned. The front-end platform will also provide the device's orientation, GPS location and camera feed to the augmented reality (AR) module.
As the augmented reality module receives the user tracking details compiled within the front-end platform, it defines whether the mode of the experience will be 2D or 3D depending on object availability. More specifically, algorithms regarding object recognition, based on GPS coordinates and/or visual live feed from a smart device, and 2D/3D object retrieval from a server will be investigated, extended and implemented. Afterwards, the AR service is held responsible for presenting the object on the smart device properly according to positioning and distance between the two of them. The latest procedure might prove tremendously resource consuming for the user's average commercial hardware, however, algorithmic optimization techniques will be developed thoroughly as a subsystem of the AR service.
In conclusion, apart from the smart devices the end user should possess to witness the complete experience, for instance smartphones and tablets with constant access to the internet, an onsite smart installation with Hololens glasses will be established across the Egnatia motorway with the exact location remaining undecided to this moment. A mobile version of the e-xnilatis platform's user capabilities will be designed and depicted on the user's device via a browser while, given the opportunity, an AR mobile application will be executed automatically and separately.

DATA COLLECTION AND ANALYSIS
A controlled set of technologies will be employed to implement the project's components, having as a goal to provide a robust, maintainable and manageable system. The technology stack that will be employed has to be defined by taking into consideration the compatibility between the system components. Although, the e-xnilatis platform includes independent components there are dependencies for a subset of modules which impose the use of a certain technology. The technology stack could determine the success of a project and there are a couple of aspects that have to be taken into account. Sometimes, the selection of the technology stack is determined by the type of a module that you aim to build. For example, it is well-known that Python is an effective and powerful language for Machine Learning and Big Data services [23].

Sensors and Smart Devices
The infrastructure takes advantage of different sensor types along the Egnatia Motorway and others found in almost every mobile device: position and optical sensors. Position sensors are based on radio navigation systems and, more specific, on GPS (Global Positioning System) which is the system used in every modern mobile device. Mobile applications will acquire the location of the personnel and send the data to the system. Communication between the platform and the mobile devices is achieved using both Wi-Fi and cellular networks (i.e. LTE).

Social Media and Websites
As a social media mining tool we will utilize and heavily modify a generic open-source framework for monitoring, analyzing and retrieving content from multiple social media platforms [29]. End users of the tool can define collections that represent their information needs through a REST API or by using a user interface on top of it. These collections can contain (1) keywords, (2) accounts or (3) locations. The tool uses that input to collect relevant content and expose analytics over it.
As a web data extraction tool we modified and used the eas-IE framework [12] to gather data from verified sites. It enables users with limited programming skills to contribute more actively to the process of data collection by only defining a configuration file for each Web source of interest. The framework then generates and executes a wrapper for each source and achieves high accuracy and adaptivity to any Web page structure.
Except for POIs with immutable characteristics, the aforementioned technologies will be able to track and locate time-limited events, which is the most frightful and fruitful problem in data aggregation to tackle. Techniques based on trending topics with hashtags on social media or on usage of specialized APIs serving time-dependent events will be developed to extract additional temporary POIs. Another matter when consuming data and metadata from the web is the existence of duplicated instances from multiple sources. To identify and recognize the same POI from different web sources, strategies based on geolocation, taking into consideration areas/masks of proximity, are being developed.

Knowledge Representation and Reasoning
In order to enable the use of deductive reasoning over the collected data to build personalized tours, we designed the e-xnilatis ontology. It is an ontology that leverages concepts from SSN and SOSA [16], LODE [30], owlTime [14], GeoSPARQL [2] and FOAF [6]. The entities of e-xnilatis ontology combined with the aforementioned ontologies, covers most of the concepts that are needed for recommending personalized routes.
The main entities/concepts that we use for representing data in e-xnilatis platform are the following: • SOSA: Sensor represents the devices or even agents that make observations. • SOSA: Observation represents the data that are produced by the Sensor class. • LODE: Event represents the events that are generated via the data gathering. The interpretation layer provides a reasoning framework over the ontology representation layer for achieving the extraction of few personal points of interest. This is achieved through the combination of the OWL 2 reasoning paradigm and the execution of SPARQL rules [15] in terms of CONSTRUCT query patterns over RDF graphs. Although SPARQL is mostly known as a query language for RDF, by using the CONSTRUCT graph pattern, it is able to define SPARQL rules that can create new RDF data, combining existing RDF graphs into larger ones. Such rules are defined in the interpretation layer in terms of a CONSTRUCT and a WHERE clause: the former defines the graph patterns, i.e. the set of triple patterns that should be added to the underlying RDF graph upon the successful pattern matching of the graphs in the WHERE clause.

HUMAN-COMPUTER INTERACTION AND PERSONALIZATION 5.1 Augmented Reality
The application of AR to daily-life activities as a new intuitive and unobtrusive interface to interact with one's surroundings and in particular with 'smart' devices and appliances is poised to further boost the deployment of AR and thus enlarge the potential market. E-xnilatis is following a post-desktop approach of Human-Computer Interaction (HCI), having AR as the component to project in place and on time of demand various archaeological sites and The brief architecture of the AR component is presented in the Fig. 3 below. Unity will be used for the development of the AR app, while .OBJ files from existing 3D model databases will be used to serve the contents to the mobile AR app. For outdoors activities (e.g. visiting archaeological sites), a markerless technique will be used to avoid the need to have markers on site. This markerless approach can use images which are part of the physical environment as long as they contain detailed features and have a reasonable high resolution to allow them be used for marking purposes. On the other hand, for indoors activities (e.g. visiting a museum or gallery, or viewing a maquette) AR markers can be used to allow a more stable positioning of the artificial component on the AR canvas.

Personalization
Personalization is a process to be applied both on Platform contents and the user interfaces (web and mobile app) in order to tailor the services on offer to fulfill the needs of specific individuals, or groups of users who share similar characteristics like visual, acoustic or motor impairments. This will be dynamic process performed at the time and place of the service. The contents and UI adaptions will be based on the user needs, environmental parameters and finally on the software and hardware profiles of end devices.
For adapting the UI, a set of CSS parameters will be defined first (e.g. font size and color, background image, layout, etc.). Linking the user models with UI parameters with logical descriptors (rules) comes after, and finally recommendations for UI adaptation will automatically generated.
On the other hand, the content adaptation will be more complicated. E-xnilatis will focus on making automatic content adaptations in order to present content according to user's personal preferences, and context of use. While drawing a route on the map for example, the system will filter out all contents (POIs) that do not lie in the user's interests or within available time and distance. Personal preferences given by the user, as well as the interaction history (log files) will provide valuable input to the recommendations generation algorithm. The output will be a list of POIs estimated to be of high interest for the user and a list of proposed nearby routes which include stops in POIs.
Overall, the adaptation mechanism will adjust: 1. the expected duration of each activity according to the user needs and abilities, 2. the list of proposed routes and POIs according to the personal preferences and the popularity of the proposals as they were evaluated by other users with similar profile, 3. the appearance of the interface (UI) according to the user characteristics, the social and physical settings.

USE CASES
The Piraeus Bank Group Cultural Foundation (PIOP), as an institution that manages two thematic museums at the two edges of the Egnatia motorway will contribute to the actual realization of the project at a cultural level, whereas 3 protected area management bodies (Axios-Loudias-Aliakmonas Delta, Koronia-Volvia and Pamvotis lakes) have already committed themselves to implementing the proposed system for tours of environmental interest.
The development and installation of the e-xnilatis platform will be made for its use along the Egnatia Motorway, while the pilot application of POIs will initially include web content that will be automatically discovered, as well as content from the PIOP museums and environmental POIs from the partner organizations.
Assessment of prototypes and final system will be implemented in real-world conditions. The pilot tests will apply to 2 collaborating folk museums and at least 3 areas of environmental interest.

CONCLUSION AND FUTURE WORK
The work in progress described in this paper encompasses the development currently underway for all the main modules of the e-xnilatis platform. Each module caters to different technological needs, which compose a system designed with the goal to facilitate human-machine interaction in the fields of route recommendation and personalized experience. Professionals and end-users will evaluate the prototypes via extensive trials that will be carried out during the prototypes' testing.
In the case the final system's results are encouraging enough, intentions of adopting and adapting the infrastructure to other transport networks throughout the globe are existent, thus e-xnilatis is planned as a project based on open source technologies with easy scalability kept in mind, to boost mayhaps future avocation.