Integrating external systems with TopBraid EDG – Example Scenarios

The following section will walk you through the integration scenarios below:  

  • A user in an external system suggests a new concept to be added to EDG. A concept could potentially belong to any of taxonomies managed by EDG. Curator, after review, decides to accept the proposal, determines the appropriate target collection, and moves the concept there. These steps are performed without using workflows.
  • A user in an external system suggests an alternative name for a concept in EDG. A workflow to manage the review and approval process is created though an API call. We describe the use of the built-in EDG workflow template as well as provide steps for creating a new workflow template dedicated to review of proposed alternative names.

In these scenarios we will use the Geography Taxonomy example provided for download from TopQuadrant’s website as part of  the “samples project”. While this document focuses on Taxonomies and Taxonomy Concepts, the same approach can be used for any type of asset collection and any type of assets.  

A new concept is proposed in an external system 

  1. Create an ontology with class and properties that will be used by the external integrating system.

In order to allow newly proposed concepts from downstream systems, we need to define a lightweight ontology in EDG (Ontologies > Create New Ontology button). Include SKOS Shapes (Settings > Includes). Create a class :ProposedConcept as a sub-class of skos:Concept. The following properties will be used for traceability:

  • :proposer – who proposed this concept (datatype: xsd:string, sh:minCount 1, sh:maxCount 1);
  • :when – the date the concept was proposed (datetype: xsd:date, sh:minCount 1, sh:maxCount 1);
  • :fromWhere – which downstream system this concept came from (datatype: xsd:string, sh:minCount 1, sh:maxCount 1).

The newly created ontology must be included in the relevant taxonomies (i.e. the taxonomies that will receive approved proposed concepts). This can be done by clicking on the Settings tab and then selecting Includes.

  1. Create a taxonomy to store proposed concepts.

This Proposed Concepts taxonomy will be used to temporarily store all proposals (Taxonomies > Create New Taxonomy button). Once approved, a new concept may be moved to an appropriate permanent taxonomy. This taxonomy designed to store new proposals should include (Settings > Includes) (1) the newly created ontology model and, (2), optionally, the model(s) used for the target taxonomy (taxonomies). The latter inclusion is necessary if the target taxonomy(s) extend SKOS with custom classes – as it is the case with the Geography Taxonomy. In this case, the data curator will change the generic proposed concept types to the more specific types. Typically, concepts are organized in a taxonomy using hierarchical relationships starting with a concept scheme. Our Proposed Concepts taxonomy is more of a folksonomy. It will not contain any hierarchical relationships, so for viewing purposes, we will switch its default view to a tabular display (Manage > Set Default App, select Tabular taxonomy app). Creating a new taxonomy creates a new GraphQL webservice that can be used from external applications: http://{server_address}:8083/tbl/graphql/{name_of_proposed_concepts_taxonomy}/{schema_prefix}%3AProposedConcept

  • {server_address} is replaced by the address of the EDG server. If using this in TopBraid Composer, replace with localhost.
  • {name_of_proposed_concepts_taxonomy} is replaced by the taxonomy ID.
  • {schema_prefix} is replaced by the prefix of the ontology created in step 1. Prefixes can be found under Namespaces and Prefixes option in the Settings

Example: http://localhost:8083/tbl/graphql/folksonomy_for_proposals/folksonomy_ontology%3AProposedConcept

  1. Proposing a new concept from an external system (in external system to integrate with EDG).

EDG provides a number of APIs for easy integration with external systems, however in order to perform write updates in EDG, access has to be granted to the integrating middleware and to the taxonomy with proposed concepts, by creating a service register account on EDG. In order to propose a new concept you need to do a POST request to the GraphQL webservice. The following GraphQL mutation computation should be passed, either as an HTML encoded alongside the query parameter, or as part of the POST’s body. For the latter, the GraphQL mutation should be passed as a value to the query key.

mutation{
createProposedConcept(input:{
uri:”http://topquadrant.com/ns/examples/geography#Birkirkara”
prefLabel:{
lang: “en”
string:”Birkirkara”
}
label:”Birkirkara”
fromWhere:”System ABC”
definition: “Birkirkara is a city in Malta”
proposer:”Jeremy”
when:”2019-08-22″
})
report{
conforms
results {
message
path
pathLabel
}
}
}

The above snippet creates a “preview” of the proposed concept. The createProposedConcept function creates the concept (in this case City), whilst the report function gives indication of whether the proposed concept is valid. If not valid (“conforms”: false), the web service identifies the validation errors. This can be used within the external system or the middleware to highlight problems that can be fixed prior to submitting to EDG. In our example, in the external system, the user only provides a label, the language of the label, and a description for the proposed concept. These values are highlighted in green in the snippet above. The rest of the values and the structure should be hard-coded in the middleware. The more complex values are described below:

  •  uri

Pattern: {taxonomy_namespace}{label} Example: http://topquadrant.com/ns/examples/geography#Birkirkara Description: The taxonomy namespace is the namespace of the main taxonomy where the concept has to be ultimately stored. This can be found under the Default namespace section in the taxonomy asset in EDG (Settings tab). The second part is the proposed concept’s label

  •  prefLabel

Type: JSON Object with keys lang and string   Description: The prefLabel is required in SKOS taxonomies to identify the main label for the concept. The label given by the user is the value for the key string, whilst the defined language is the value for the lang key. ISO 639-1 code (2 letter language codes) are used for this value.

  • when

Pattern: YYYY-MM-DD In order to push the proposed concept to EDG, the commit keyword is added to the mutation computation as shown below:

mutation{
createProposedConcept(input:{
# Properties and values for the proposed concept
})
commit
}
  1. Moving proposed concepts into the target taxonomy in EDG.

After a concept is proposed, the data curator reviews it and, if acceptable, approves and moves the newly created concept to the target taxonomy. The data curator can find the newly proposed concept in EDG, by opening the taxonomy with proposed concepts. Selecting the concept, displays its information. If the curator decides that the concept should be added to the target taxonomy, they click on the type field and a drop down box with all possible types is displayed. Curator can then change the type to (in our example) City and click on save changes button.   Once the type statement is changed, the data curator can safely move the new concept to the target taxonomy in EDG. EDG provides the functionality of moving instance data across different assets. Navigate to the target taxonomy and then click the Transform tab. Click on Move under the ‘Copy or Move Instances from Other Asset Collection’.   From the drop down box, choose the collection where the proposed concepts are stored. Then chose the type of the concepts to move, and finally click the Finish button.   Once all concepts are moved to the target taxonomy, the curator can arrange them in the hierarchy. Several approaches are possible, depending on curator’s preference and a number of proposed concepts that need to be arranged taxonomically: Select a concept you want to make a parent of a proposed concept, click on the narrower concept field and start typing to find the desired proposed concept .

  1. Switch to the tabular view and search for proposed concepts using Filters. They can be found by searching on concepts without “broader” relationship. Select a concept, click on the broader concept field and start typing to find the desired parent concept
  2. Run Problems and Suggestions report (Reports > Problems and Suggestions). It will identify all concepts that do not have a broader parent and are not top concepts of a scheme.

Curator can also adjust concept definition if needed. This could be done pre or post move depending on the traceability requirements.  

Alternative name for an existing concept

 

  1. Call EDG API to find the edited concept’s home collection graph 

This step is necessary if there are multiple possible target taxonomies and the source system does not store information about target graphs.  In order to find the asset collection that defines the concept we need to query EDG using the concept’s URI. For this, the SPARQL endpoint is used. The generic EDG SPARQL endpoint URL is: http://{server_address}:8083/tbl/sparql The SPARQL endpoint can be invoked as a restful API. The post request only requires a query key to be passed in the call’s body. The content type should be application/x-www-form-urlencoded. The SPARQL query should be the following:  

SELECT ?masterGraph
WHERE {
() teamwork:readableGraphsUnderTeamControl (?masterGraph ?teamGraph) .
GRAPH ?masterGraph {
<concept_uri> a ?any .
}
}

This query will retrieve the home collection (or the graph) where the concept is defined. concept_uri is replaced with the URI of the concept in question. Therefore, in our example, if we would like to propose an alternative name for the concept Dublin, our query would look as follows:  

SELECT ?masterGraph
WHERE {
() teamwork:readableGraphsUnderTeamControl (?masterGraph ?teamGraph) .
GRAPH ?masterGraph {
<http://topquadrant.com/ns/examples/geography#Dublin> a ?any .
}
}

Once invoked, results will be returned. EDG provides response type formats (see: Pre-built RESTful Web Services ); however, for simplicity in this example we use the application/sparql-results+json-simple. To use this format, the POST request should add this type as the value for the Accept header. The following are the results we get for the above mentioned SPARQL query:  

[
{
“masterGraph”: “urn:x-evn-master:geography_3”
}
]

This means that the concept Dublin is defined in the graph urn:x-evn-master:geography_3urn:x-evn-master: is the graph’s URL prefix whilst geography_3 is the ID. The latter is used in the next step.  

  1. Starting a workflow for the taxonomy

A workflow can be programmatically started on EDG via a restful API call. The API’s URL (POST request) is:   http://{server_address}:8083/tbl/swp?_viewClass=teamwork:AddTagService   This API requires a number of parameters that need to be passed as part of the query string: name – the name of the working copy (or instance of the workflow), such as a date stamp:

  • projectGraph – the URI of the concept’s collection (the home collection) as returned in the previous step (1);
  • workflow (optional) – if the taxonomy has a defined workflow specific for externally invoked changes (see scenario 3), the URI of the defined workflow for such external changes is required. In this example we have only one workflow;
  • editedResource (optional) – the value for this parameter is the URI of the concept that will be edited.

The name of the workflow can be a date stamp set by the middleware which will later act as the ID for the GraphQL web services call. Furthermore, assigning date stamps ensures that only one workflow a day is initiated on EDG, and all suggested changes for that day are stored in the initiated working copy. In EDG, the ID cannot start with a number and using characters such as / and – would require some further string manipulation in the middleware. Therefore, the suggested IDs are: workflow_{YYYYMMDD} or workflow{YYYYMMDD}, for example workflow_20190821 or workflow20190821.   The API call URL would look similar to:   http://localhost:8083/tbl/swp?_viewClass=teamwork:AddTagService&name=workflow20190821&projectGraph=urn:x-evn-master:geography_3   If successful, EDG will respond with the following JSON:  

{
“changed”: true,
“added”: 7,
“deleted”: 0,
“rootResource”: “urn:x-tags:workflow20190822”,
“changes”: {
“urn:x-tags:workflow20190822”: {
“http://www.w3.org/1999/02/22-rdf-syntax-ns#type”: [
“http://topbraid.org/teamwork#Tag”
],
“http://www.w3.org/2000/01/rdf-schema#label”: [
“\”workflow20190822\””
],
“http://topbraid.org/teamwork#status”: [
“http://topbraid.org/teamwork#Uncommitted”
],
“http://purl.org/dc/terms/created”: [
“\”2019-08-22T11:00:14.066+01:00\”^^xsd:dateTime”
],
“http://rdfs.org/sioc/ns#has_creator”: [
“urn:x-tb-users:SystemXYZ”
],
“http://topbraid.org/teamwork#manager”: [
“urn:x-tb-users:Administrator”
],
“http://topbraid.org/teamwork#workflowTemplate”: [
“urn:x-evn-workflows:SuggestAlternativeNameWorkflow”
]
}
}
}

EDG will automatically create a web service for the workflow’s GraphQL interface. Any changes and queries done on this web service will be executed against the workflow’s working copy and not the original taxonomy. The following API will become available:    http:// {server_address}:8083/tbl/graphql/{home_collection_graph}.{workflow_id}   where, {home_collection_graph} is the id of the home collection graph (in this case geography_3) and the {workflow_id} is the ID given to the workflow by EDG, in this case the ID set by the middleware (i.e. workflow20190822). Submit an alternative label for the concept In order to submit an alternative label using the GraphQL web service, we now need to do a mutation in the working copy: http:// {server_address}:8083/tbl/graphql/geography_3.workflow20190822 In order to propose an alternative label you need to do a POST request to the GraphQL webservice. The following mutation computation should be passed, HTML encoded, alongside the query parameter:

mutation{
addToCity(input:{
uri:”http://topquadrant.com/ns/examples/geography#Dublin”
skos_altLabel:{
lang: “ga”
string:” Baile átha Cliath”
}
})
commit
}
  • Once the change (or changes) are made, the data curator needs to move through the states in the workflow. In EDG, the curator will see the workflows he needs to act on by clicking on My Workflows. This page will display a list of all workflows in progress that a curator needs to act on.

The curator has to choose the workflow where the external changes were done and click the Go to Workflow button. From there, the curator needs to find the modified concept (Taxonomy tab), and following a review the curator can commit to production (Gear Tab > Commit to Production). In cases where the workflow is assigned to one particular concept (i.e. a value for editedResource was defined in the parameters), the data curator can instead select the Go to Asset button, so that EDG will navigate to and display the asset in question. This way, the curator will see the changed concept immediately without having to look it up in the taxonomy.   In a workflow, there might be an optional step that allows the data curator to freeze the workflow. It is typically used when the curator needs to ask other users to review the proposed change. When in this state, no user can make changes, but they can leave comments. The data curator will then decide whether to approve the change. If approved, the taxonomy is updated with the new changes.  

Define custom workflow in TBC for external edits

Editing an existing concept in a taxonomy from an external source could require a custom workflow different from the workflows shipped with EDG. In order to define a new workflow, the following has to be done:

  1. Download all existing workflows from EDG (Workflow Templates Home Tab > Download as Turtle file), import this file in the TBC project workspace (this can be done by simply dragging the downloaded file into the workspace) and open.
  2. In TBC, create a new workflow template instance. Use one of the following types (classes): teamwork:TagWorkflowTemplate (a generic workflow template), teamwork:ExistingResourceTagWorkflowTemplate, and teamwork:NewResourceTagWorkflowTemplate. You will find them in the Classes In our example, we will create an instance of the type teamwork:ExistingResourceTagWorkflowTemplate.

Right click on the type and then click on ‘Create instance…’. Once the instance is created, add rdfs:comment and rdfs:label for the new workflow instance. These will be displayed on the UI. This is how the instance will look like (TURTLE syntax):  

<urn:x-evn-workflows:SuggestAlternativeNameWorkflow>
rdf:type teamwork:ExistingResourceTagWorkflowTemplate ;
rdfs:comment “A workflow that allows external users to propose alternative names to concept in the taxonomy” ;
rdfs:label “Suggest Alternative Name Workflow” .

  3.  If needed, the newly defined workflow can be declared the default workflow for all asset collections of a given type e.g., all taxonomies. In order to do this, add a value to the predicate teamwork:defaultTagWorkflowTemplateForProjectType. The value to use for the taxonomy asset collections is taxonomies:ProjectType.   4. The next task is to identify the workflow stages/transitions, starting from the initial uncommitted status (i.e the stage when a new workflow is initiated), till the end stage. Let us consider the following workflow: First, we have to identify the initial state of the workflow. This value has to be defined using the predicate teamwork:initialStatus. In this example, for our initial state we can use EDG’s predefined value of teamwork:Uncommitted.

We then need to define all remaining transitions by creating instances (one for each transition) of type teamwork:TagStatusTransition as values attached to the property teamwork:transition. In TBC, these can be easily done by clicking on the arrow next to the property (the workflow instance defined in B) and then clicking on Create blank node. In this form we need to fill out the following predicates:

teamwork:fromStatus – the current workflow state (value: teamwork:TagStatus); 

teamwork:toStatus – the next workflow state (value: teamwork:TagStatus); 

teamwork:transitionLabel (optional) – a human readable label for the transition (value: xsd:string); 

teamwork:requiredGoverananceRole – the responsible person for moving forward from this state (value: teamwork:WorkflowParticipantProperty). 

By default, EDG provides users with a number of predefined tag statuses (in TBC you can search for these by typing teamwork:TagStatus in the Classes view and then click the Instances view), and governance role. Taking our previous example, the workflow transitions will look like the following (TURTLE syntax):   #… other predicates for resource

teamwork:initialStatus teamwork:Uncommitted ;
teamwork:transition ;
rdf:type teamwork:TagStatusTransition ;
teamwork:fromStatus teamwork:FrozenForReview ;
teamwork:requiredGovernanceRole edg:dataSteward ;
teamwork:toStatus teamwork:Committed ;
teamwork:transitionLabel “Accept changes” ;
] ;
teamwork:transition [
rdf:type teamwork:TagStatusTransition ;
teamwork:fromStatus teamwork:FrozenForReview ;
teamwork:requiredGovernanceRole edg:dataSteward ;
teamwork:toStatus teamwork:Rejected ;
teamwork:transitionLabel “Alternative name for concept is rejected” ;
] ;
teamwork:transition [
rdf:type teamwork:TagStatusTransition ;
teamwork:fromStatus teamwork:Uncommitted ;
teamwork:requiredGovernanceRole edg:dataSteward ;
teamwork:toStatus teamwork:FrozenForReview ;
teamwork:transitionLabel “Changes done, awaiting for approval or rejection from data curator” ; 
] ;

Once finished, upload the edited workflow turtle file back into EDG (Workflow Templates Home Tab > Upload Turtle file) More details on creating workflows can be found in Adding Custom Workflow Templates and Workflows.

Importing Excel spreadsheets into EDG assets using pre-defined templates

The purpose of this example is to showcase how Excel sheets can be imported into EDG assets using external tools such as Python or using RESTful APIs. 

Prerequisites

  • An existing asset collection in EDG;
  •  Defined spreadsheet mapping template in EDG
  • This mapping is created in the EDG asset collection by clicking on the Import tab, and then selecting the Import Spreadsheet using Pattern option;

Hint: EDG automatically assigns an ID to newly created mappings. This ID is used during the importing of spreadsheets into EDG that uses the particular mapping. In this document, we will dynamically access EDG to retrieve this ID, therefore, it is suggested that the template name is an identifier that is meaningful to you.

Importing a spreadsheet – Steps

A two-step approach is taken in order to import data from an Excel spreadsheet, convert it into EDG-ready data, and finally store in the required asset collection. As a first step the Excel file is uploaded and an ID is generated. The second step performs the actual mapping and conversion.

Step 1: Uploading Excel file

REST API URL: /evnimportFileUpload
HTTP Method: POST
Encoding: multipart/form-data
Payload Parameters:

Parameter

Description

Example

sheetIndex

This parameter identify the sheet that needs to be imported. Excel sheets are numbered starting at 1 for the leftmost sheet. Only one sheet can be imported at a time.

1

_base

The collection graph URI where the the spreadsheet data will be imported to.

urn:x-evn-master:asset_collection_a

_progressId

A randomly generated identifier that is attached to the importer process. (Optional)

123-456-789

projectGraph

The collection graph URI where the the spreadsheet data will be imported to.

urn:x-evn-master:asset_collection_a

record

A boolean parameter, which, if set as true, EDG will record each new triple in change history. This is not recommended for large imports.

false

template

The mapping template ID to be used during the import and conversion into EDG. Section Finding mapping template ID describes how the ID is retrieved via a SPARQL query.

urn:x-evn:Mapping1597045916622

overwrite

A boolean parameter, which, if set as true, the import will overwrite existing values in the asset collection.

true

Files Parameters: The multipart/form-data encoding type expects the Files parameter as follows:

files = [
  (‘file’, open(‘/path/to/file.xlsx’,’rb’))
]

Response: The REST API call returns a JSON object, which values will be used in the second step. The response will include: – id – an identifier generated by EDG after uploading the excel spreadsheet; – _base – the asset collection graph URI where the spreadsheet data will be imported to; – _progressId – the randomly generated identified attached to the imported process.

For example:

{
  “id” : 26 ,
  “_base” : “urn:x-evn-master:asset_collection_a” ,
  “_progressId” : “123-456-789”
}

Step 2: Perform Mapping

REST API URL: /evnimportMapping
HTTP Method: GET
Payload Parameters:

Parameter

Description

Example

importId

The identified generated by EDG after uploading the excel spreadsheet.

26

id

The identified generated by EDG after uploading the excel spreadsheet. This value is the same as importId.

26

_base

The collection graph URI where the the spreadsheet data will be imported to.

urn:x-evn-master:asset_collection_a

_progressId

A randomly generated identifier that is attached to the importer process. (Optional)

123-456-789

projectGraph

The collection graph URI where the the spreadsheet data will be imported to.

urn:x-evn-master:asset_collection_a

record

A boolean parameter, which, if set as true, EDG will record each new triple in change history. This is not recommended for large imports.

false

template

The mapping template ID to be used during the import and conversion into EDG. See Finding mapping template ID.

urn:x-evn:Mapping1597045916622

overwrite

A boolean parameter, which, if set as true, the import will overwrite existing values in the asset collection.

true

tag

Should the data be imported in a workflow, the URI of the asset’s workflow is defined here. (Optional)

urn:x-tags:test_workflow

Response: This REST API call returns an HTML page. In general, a status code 200 would mean that the import was successful.

Finding mapping template ID

In order to identify the EDG defined ID for a previous spreadsheet importer mapping, we need to execute a SPARQL query. This can be done via the available SPARQL endpoint: http://<host_url>/tbl/sparql, where <host_url> should be replaced with EDG’s URL.

The query is:

SELECT ?baseURI
WHERE {
  rdf:nil teamwork:tableMapFiles ( ?baseURI ?fileLabel ) .
  FILTER(?fileLabel = “MyTemplateLabel-10082020”)
}

A POST request, can be executed against the SPARQL endpoint, with the following parameters: – query – A string SPARQL query; – format – (Optional) The response format could be one of the following: xml (default), json, csv, tsv.

A typical request using HTTPie in shell:

http –follow –timeout 3600 GET localhost:8083/tbl/sparql ‘query’==’SELECT ?baseURI WHERE { rdf:nil teamwork:tableMapFiles ( ?baseURI ?fileLabel ) . FILTER(?fileLabel = “Triple Stores”) }’ ‘format’==’json’ Cookie:’username=Administrator;’

A typical request using Python requests library:

import requests

url = “http://localhost:8083/tbl/sparql?query=SELECT ?baseURI WHERE { rdf:nil teamwork:tableMapFiles ( ?baseURI ?fileLabel ) . FILTER(?fileLabel = \”Triple Stores\”) }&format=json”

payload = {}
headers = {
  ‘Cookie’: ‘username=Administrator’
}

response = requests.request(“GET”, url, headers=headers, data = payload)

print(response.text.encode(‘utf8’))

Importing spreadsheet content into EDG using Python as a connector

In this section, we will walkthrough a sample python application that creates a connection session to EDG and programmatically imports a spreadsheet file into an asset collection. This example assumes python 3.x.x and the requests library.

In this example the following libraries are imported:

import requests
import json
import os
import uuid

and the following global variables are used:

# Global Variables
host = “http://localhost:8083/tbl/”
home = host + “swp?_viewName=home”

The host variable has to be changed to your EDG’s URL.

Setting up an EDG session

There are different ways to authenticate into EDG. The following code snippets should be modified accordingly.

No password authentication method:

# 1. Create Session
session = requests.session()

# 1a. Create Cookie – This is only required for authentication using TBC
cookie_obj = requests.cookies.create_cookie(domain=“localhost”,name=‘username’,value=‘Administrator’)
session.cookies.set_cookie(cookie_obj)

# 2. Initialise session
r = session.get(home)

In this case, a cookie needs to be set. The domain and the value fields need to be changed. The variable session can now be used for further requests.

Tomcat basic authentication:

username = “username”
password = “password”
login_url = host + “j_security_check”

# post fields for logging in
post_fields = {
    ‘j_username’: username,
    ‘j_password’: password
}

# 1. Create Session
session = requests.session()

# 2. Initialize session
r = session.get(home)

# 3. Log in
r = session.post(login_url, post_fields)

Basic authentication requires a security check, and thus both username and password need to be assigned. The variable session can now be used for further requests.

Preparing the data request

After creating a connection between our python application and EDG, we can perform further requests including querying a SPARQL endpoint and importing spreadsheets.

Based on the parameters required for the API calls described in the section Importing a spreadsheet – Steps, we define the following variables:

# Hard-coded variables – change these according to your needs
file_path = ‘/path/to/file/MyFile.xlsx’
file_name = ‘MyFile.xlsx’
sheet_index = ‘1’
graph = ‘urn:x-evn-master:asset_collection_a’
template_name = ‘MyTemplateLabel-10082020’
overwrite_previous = True

# Get the template ID
template_id_objects = convert_sparql_results(retrieve_template_id(session, template_name))

# Checks if mapping template exists
if (len(template_id_objects) > 0):
  template_id = template_id_objects[0][“baseURI”][“value”]
  # … Execute import request
else:
  print(“No template with name “+ template_name)

The first six variables, are straight forward. These can be hard-coded or passed through by another service. In order to assign a value for template_id, we need to query EDG with the template_name. Refer to Finding Mapping Template ID.

The function retrieve_template_id(session, template_name) is defined as follows:

def retrieve_template_id(session, template_name):
    query = “””SELECT ?baseURI
    WHERE {
      rdf:nil teamwork:tableMapFiles ( ?baseURI ?fileLabel ) .
      FILTER(?fileLabel = “{template_name}“)
    }”””
    query = query.replace({template_name}, template_name)

    data = {
        ‘query’: query,
        ‘format’: ‘json’
    }

    response = session.post(host+“sparql”, data=data)

    if (response.status_code == 200):
        return response.text
    else:
        print(response.text)
        return None

The parameters are the initialised session and the template_name. The function returns the SPARQL results in a JSON format.

convert_sparql_results(sparql_results) is a function defined for convenience, in order to convert the SPARQL results into parsable python objects. The function takes the returned SPARQL results in a JSON format and is defined as follows:

def convert_sparql_results(sparql_results):
    return_list = []
    json_obj = json.loads(sparql_results)

    for result in json_obj[‘results’][‘bindings’]:
        return_list.append(result)

    return return_list

Executing the Import request

If the SPARQL endpoint returns an ID corresponding to the mapping label, the application will proceed in creating another request in EDG, this time to import data of an excel spreadsheet into a particular asset collection using the required mapping.

  # … Execute import request
  status_code = import_excel_file(session, file_path, file_name, sheet_index, template_id, overwrite_previous, graph)
  if(status_code == 200):
      print(“Upload Successful”)
  else:
      print(“Upload Unsuccessful”)

In this sample application we define a function import_excel_file(session, file_path, file_name, sheet_index, template_id, overwrite_previous, graph) using the variables assigned previously. The function import_excel_file create requests to the previously described REST APIs:

# Imports excel sheet into an EDG asset. Returns the status code of the request
def import_excel_file(session, file_path, file_name, sheet_index, template_id, overwrite_previous, graph):
    # Step 1 – Upload the excel sheet to EDG
    files = {
        ‘file’: (file_name, open(file_path, ‘rb’))
    }

    data = {
        ‘sheetIndex’: sheet_index,
        ‘_base’: graph,
        ‘_progressId’: str(uuid.uuid4()),
        ‘projectGraph’: graph,
        ‘record’: ‘false’,
        ‘template’: template_id,
        ‘overwrite’: overwrite_previous
    }
    response = session.post(host + ‘evnimportFileUpload’, data=data, files=files)

    # Check if the session response or EDG produced an error. If the status code is 200, proceed to the second step.
    if (response.status_code == 200):
        # Step 2 – Ask EDG to do the actual mapping and importing to the graph
        response_dict = eval(response.text)

        payload = {
            ‘importId’: response_dict[“id”],
            ‘projectGraph’: graph,
            ‘tag’: ,
            ‘template’: template_id,
            ‘id’: response_dict[“id”],
            ‘_base’: graph,
            ‘_progressId’: response_dict[“_progressId”],
            ‘record’: False,
            ‘overwrite’: overwrite_previous
        }
        response_get_request = session.get(host + ‘evnimportMapping’, params=payload)
        return response_get_request.status_code
    else:
        return response.status_code

If successful, this method will return a 200 status code.

Importer.py

Download example python script below:

importerTBC