Installation of the EDG Server

 

Overview

This document covers installing the TopBraid Enterprise Data Governance (EDG) server into its platform environment and integrating it with systems services such as LDAP, SAML, etc. To administer the EDG application user functions, see EDG Server Administration.

The EDG server uses Java servlets deployed to an Apache Tomcat web server and servlet container. Installation involves deploying the EDG WAR (Web Application Archive) file in the application server. Customers receive a link and login/password for the website to download the WAR file for installation. The installation files consist of the EDG server WAR file (edg.war), setup properties file and authentication valve. 

Installation of the EDG server differs slightly for different operating systems. Detailed instructions for Tomcat, Linux and Windows are provided in the sections below. In general, installation consists of:

  1.  Deploying the EDG WAR file in the application server. For Tomcat, this involves copying the edg.war file to the Tomcat webapps directory, or using Tomcat Manager to deploy edg.war.
  2. Configuring authentication
  3.  Setting configurations through the setup file or interactive
  4. Server Administration 

EDG Server Platform Considerations

These sections describe some aspects of the server platform to consider prior to installing the EDG server. 

Application Server and Supported Platforms 

The EDG server uses Apache Tomcat running on Java. For the details of your EDG server version, see the  Supported Platforms page for compatible platform components (e.g., Java, database, etc.). This also includes sections with details and suggested configurations for installing on Linux or Windows Server. 

Suggested Java Configuration

If you are using both TopBraid EDG and another TopBraid server application such as Explorer on the same machine, we recommend that you run each in a separate JVM by running them with separate instances of Tomcat.

It is necessary to ensure that you have adequate physical memory to run EDG and that you have configured your Web Application Server to use it. During the purchasing process you should have received a guide to Server Sizing Specifications. Please set these appropriate to your data and users. If you did not receive this, please contact TopQuadrant support.

See the installation instructions below for your operating system for more details concerning how to set the parameters.

Operating Systems

 

Installation on Linux

This section describes details and suggested configurations when using Linux as the base operating system and Tomcat as the Web Application Server.

A suggested configuration for Enterprise Data Governance is as follows:

Operating Systems: Current version of Linux with 2.6 or higher kernel
Java Runtime Environment: Java 11
Web Application Server

 Apache Tomcat 9 latest

Apache Tomcat 8.5.latest

 see supported-platforms for specific versions 

Permissions Full access to the Tomcat configuration area is required.

 

Enterprise Data Governance also requires that the nofile limit be increased as it opens a large number of JAR files, and each TDB database also has a large number of files. The default is 1024 open files and sockets. Reset this by adding the following two lines to the /etc/security/limits.conf configuration file:

 

tomcat8         soft    nofile          2048
tomcat8         hard    nofile          8192

 

Installation on Windows Server

 

This section describes details and suggested configurations when using Windows Server as the base operating system and Tomcat as the Web Application Server.

A suggested configuration for Enterprise Data Governance is as follows:

Operating Systems:

 

Windows Server, (e.g. 2008, 2012, 2016)

 

Administrator access is required for installation.

 

Java Runtime Environment: Java 11
Web Application Server

 Apache Tomcat 9 latest

Apache Tomcat 8.5.latest

 see supported-platforms for specific versions

 

 

Authentication for TopBraid EDG - background

 

All authentication for TopBraid are handled by using standard Java EE web container methods. Once inside the container, users are free to design their own RBAC designs, such as TopBraid EDG’s User Roles management or Governance Roles for vocabulary or asset models. Some of the details will depend on the authentication Realm set up for Tomcat – i.e. LDAP, AD, etc. However, authentication is always a handshake between the entity requiring access and the web container (e.g. Tomcat), not TopBraid Suite applications. Please see sections below for details on LDAP, SAML and OAuth authentication.

Organization-specific IT policies will largely dictate interactions with the web container to authenticate users and services. Therefore, TopQuadrant can only play an advisory role for getting started with authentication issues.

Consult Java EE tutorials for a good starting point on authentication methods. TopBraid supports Form, Basic and no authentication: 

  1. No Authentication. No user id or password is required to invoke services. This is useful for Linked Open Data applications and applications are are used openly behind firewalls. This should only be used for read-only servers and the administrator should ensure that the Enable SPARQL Updates parameter in the Server Administration page is set to false, which is the default that blocks updates.
  2. Form-based authentication. This is the best choice for all UI-based applications, such as EDG and SWA/SWP-based applications. Form-based authentication will display a form for entering authentication, pass the challenge to the authentication agent, and respond with the challenge result. On logout, the user is logged out from the container.
  3. Basic authentication. This should be used for access via 3rd party entities, such as web services. It is not convenient for user-based authentication because the user cannot log out of the system without closing the browser. It is recommended to use Basic authentication when the server is used for web services and an administrator needs to occasionally log in to perform maintenance.* When using SAML, the IDP performs the authentication unless an API request with BASIC authorization header, which will need to use BASIC as the auth-method.

End User Authentication

If user logout is required, then it is best to use Form authentication. In this case the application controls the login and logout pages that are displayed to the user. When a user logs out, they are logged out from the web application container.

Some user information is cached in the graph <urn:x-evn-user-data> of the active TopBraid workspace. This information is passed to TopBraid from the web application container at login. Helper functions are available from any SPARQL context, including smf:currentUserName()smf:hasCurrentUser(), and smf:userWithName().

 

Role-Based Access Control (RBAC) in TopBraid

TopBraid does not directly handle user authentication, which is performed by an authentication layer, such as LDAP or Active Directory, that interacts with the web application container (Tomcat). Once a user has passed the authentication challenge from the web application, the user id and roles are cached in TopBraid. An important implication is that the TopBraid infrastructure cannot know about the existence of a user until they have successfully logged in to the web container.

Once a user has successfully passed authentication by the web container, user information is cached in the urn:x-evn-user-data graph in the TopBraid workspace. Useful SPARQL functions for access to user information accessible by TopBraid include the following:

  • smf:currentUserName: Gets the name of the user that is currently logged into TopBraid. Should be preceded by smf:hasCurrentUser to avoid exceptions.
  • smf:userWithName: Converts a user name into a URI resource, following the default settings in TopBraid. Often used in conjunction with smf:currentUserName().
  • teamwork:currentUserHasPrivilege: Checks whether the currently logged in user has a given privilege, specified by a role property. The current user must have that role or a sub-property thereof, for the given governed resource. The query will be executed on the given team graph.

 

Web Service Authentication

Web services invocation on servers requiring authentication can use Basic or Form-based authentication. Basic authentication using HTTPS encryption is recommended. SAML SSO cannot be used for web services. Please see SAML below for more information.

Basic Authentication

Basic authentication should be used for access by web services. Basic authentication relies on a Base64 encoded ‘Authorization’ header whose value consists of the word ‘Basic’ followed by a space followed by the Base64 encoded name:password. This is better suited for service-only access because the only way a user can log out is to shut down their browser. The URL with header information can be submitted with authentication information. Here’s an example using cURL to access a SPARQLMotion service named ‘DisplaySimpleHtml’ in TopBraid:

curl -H “Authorization: Basic c2NvdHQ6MTIzNDU=” -X POST “http://localhost:8080/edg/tbl/sparqlmotion?id=DisplaySimpleHtml”

GET works as well. The base 64 in the middle translates to the uid:pwd string “scott:12345”.

 

Some other examples of service calls with parameters (be sure to encode the URL before submitting as a web service call) :

curl -H “Authorization: Basic c2NvdHQ6MTIzNDU=” -X POST “http://localhost:8080/edg/tbl/sparql?default-graph-uri=http://topbraid.org/examples/kennedys&format=application/sparql-results+json&query=SELECT ?s WHERE {?s a <http://www.w3.org/2002/07/owl#Class>}”

curl -H “Authorization: Basic c2NvdHQ6MTIzNDU=” -X POST “http://localhost:8080/edg/tbl/sparql?query=SELECT ?s WHERE { GRAPH <http://topbraid.org/examples/kennedys> {?s a <http://www.w3.org/2002/07/owl#Class>} }”

wget has a similar protocol:

wget –save-cookies=”/tmp/cookies” –header “Content-type: application/x-www-form-urlencoded” –post-data=”j_username=scott&j_password=tomcat” http://localhost:8080/edg/j_security_check

For secure access, web services should use HTTPS encryption.

 

Form-Based Authentication

Access using Form-based authentication, while not recommended, is possible using cookies generated by the server. One method is to respond to the challenge with a hardcoded URL with a valid user id and password. The general form of this response is:

 

http://[host]:[port]/edg/j_security_check?j_username=[username]&j_password=[password]

Another method is to request an HTTP cookie that can be used in subsequent requests. The following is an example script for accessing a TopBraid form-based server. In summary, the script interaction has three parts:

 

  1. The client requests a cookie, e.g., loginRequestCookie, to use for logging in using GET or POST.
  2. The client logs in, and if successful, the client will receive another cookie, e.g., loginSuccessCookie, which is saved for subsequent requests.
  3. The client uses the loginSuccessCookie for subsequent TopBraid service requests, such as the SPARQL endpoint call in the example.

Assuming the following is defined in a file named authenticateCallService.sh:

#!/bin/bash

#The script needs the servername:portnumber, username and password respectively to work correctly.

if [[ $# -ne 3 ]] ; then
echo ‘Invalid command. Please run authenticate script in following format ./authenticateCallService <servername:port> <username> <password>’
exit 1
fi

curl -c ~/loginRequestCookie -X POST “http://$1/edg/tbl” -D ~/firstReqHeaders > /dev/null

#The server sends a new cookie as a response to this request. Use that cookie for subsequent requests.
curl -b ~/loginRequestCookie -X POST “http://$1/edg/tbl/j_security_check” -H “Context-Type: application/x-www-form-urlencoded” –data “j_username=$2&j_password=$3” -L “http://$1/edg/tbl” -c ~/loginSuccessCookie -D ~/secReqHeaders

curl -b ~/loginSuccessCookie -X POST “http://$1/edg/tbl/sparql” -G –data-urlencode “query=SELECT * WHERE{?s a <http://www.w3.org/2002/07/owl#Class>} ” –data-urlencode “default-graph-uri=http://topbraid.org/examples/kennedys” –data “format=json” -D ~/thirdReqHeaders

#Logout when done so as not to exhaust the user limit on the license
curl -b ~/loginSuccessCookie -X POST “http://$1/edg/tbl/purgeuser” -D ~/logoutReqHeaders

 

The script can be invoked by the command:

 

./authenticateCallService <servername:port> <username> <password>

 

For secure access, web services should use HTTPS encryption.

 

Authenticating calls that use SPARQL’s SERVICE keyword

To enable the use of the SPARQL SERVICE keyword to retrieve data from SPARQL endpoints that require authentication, add an entry to Password Management in Server Administration of the following form:

 

username@localhost:8080/edg/tbl/sparql : password

 

For a query like the following, TopBraid EDG will check whether there is a user@uri key in storage that ends with the given URI, and returns the first one. 

 

SELECT ?fname
WHERE {
	SERVICE <https://localhost:8080/edg/tbl/sparql> {
		GRAPH <http://topbraid.org/examples/kennedys> {
			?subject <http://topbraid.org/examples/kennedys#firstName> ?fname
		}
	}
}

 

This is used to generate the credentials for basic authentication.

See Password Management for more information about managing passwords in storage.

 

 

EDG Server Installation

If setup was done incorrectly or is incomplete you will get a setup page instead of the EDG home page. You can always navigate to “server”/edg/tbl/setup” to see your configurations.

The following sections detail specific instructions for running TopBraid servers on Tomcat. There are 2 ways to configure TopBraid EDG and Explorer. First using the setup properties file and second the interactive method. Please choose which ever you feel most comfortable with.

EDG installation using the setup.properties file

Note: These are not upgrade steps. These are for new installations.

1. Ensure you are running on Java 11

2. Configure authentication for user access. See LDAP, SAML, OAuth, and Tomcat users.

3. Copy the file called edg-setup.properties from the topbraid-edg7.0.zip download into the tomcat root directory (or location readable by tomcat). Modify the parameters in the file per the file instructions.

Field Description example
workspacePath The workspace houses all the data for EDG. This needs to be persistent and have read/write access for Tomcat. /var/lib/tomcat9/Workspace
authMethod See authentication above. For SAML SSO, select Basic.
authRealm For basic, sets the name in the header TopBraid EDG
securityRoles Security groups that align with LDAP or ADFS or Tomcat users  admin,poweruser,manager,editor,viewer
vaultPath Location of encrypted data for EDG ./vault.properties
vaultPassword Master password used to decrypt and encrypt  storage
vaultEncryption Encryption algorithm used for encrypted storage PBEWithHMACSHA512AndAES_256
licenseFile Path to license file. Do not enter this if you want your Administrators to be able to update the license in the UI. If entered then this file will need to be updated when new licenses are provided by TopQuadrant and the server restarted. ./edg-license.lic
sessionTimeout Tomcat session timeout in minutes 90
databaseType Choice of database storage option, TDB, RDBMS or Data Platform. Additional fields will need to be entered when choosing DP or RDBMS. Please see descriptions below for more details. Note: If you are using RDMBS please restart your server once setup is complete. SharedTDB

4. Set the system property edg.setup for Tomcat to the full path of edg-setup.properties. This can be done in various ways, for example by editing Tomcat’s conf/catalina.properties file. For example: edg.setup=/var/lib/tomcat9/edg-setup.properties

Or you can also modify the command line used to start Tomcat such as:
-Dedg-setup=/var/lib/tomcat9/edg-setup.properties

5. Place the edg.war into the webapps directory in tomcat

6. Start Tomcat

8. In EDG, Enter license if location not specified in the setup file.

9. Navigate to Server Administration, Rights Management, make sure security roles are correctly set for ANY_ROLE, administrator, etc. Initially, all users will have permissions to all EDG resources and functions via the default assignment of AdministratorGroup to ANY_ROLE.  An administrator’s initial task should be to use Access Control > Rights Management to transfer the AdministratorGroup to their organization’s defined administrative role(s) and then remove it from ANY_ROLE.

11. Now navigate to the home page of EDG. Setup is complete. From now on this file will act as the source of your configuration settings. An administrator should now go over the guidance here to make any additional adjustments to EDG. If setup was done incorrectly or is incomplete you will get a setup page instead of the EDG home page. You can always navigate to “server”/edg/tbl/setup” to see your configurations.

 

EDG installation using the interactive deployment wizard

1. Ensure you are running on Java 11

2. Configure authentication for user access. See LDAPSAMLOAuth, and Tomcat users.

3. Place the edg.war into the webapps directory in tomcat

4. Start Tomcat

5. Navigate to EDG, this should detect you need to setup configuration and present a page with an option to use the interactive setup. Launch this option.

6. Fill out all the configuration parameters

Field Description example
Workspace The workspace houses all the data for EDG. This needs to be persistent and have read/write access for Tomcat.
License Upload your license provided by support
Authentication Configuration See authentication above. For SAML SSO, select Basic.
Roles to be added Security groups that align with LDAP or ADFS or Tomcat users. Map these to permissions such as Manager, Editor, Viewer in the dropdowns.  admin,poweruser,manager,editor,viewer
Vault Configuration Location of encrypted data for EDG
Vault Password Master password used to decrypt and encrypt  storage
License file Enter the license provided by TopQuadrant
Session Timeout Tomcat session timeout in minutes 90
Application Data Storage Choice of database storage option, TDB, RDBMS or Data Platform. Additional fields will need to be entered when choosing DP or RDBMS. Please see descriptions below for more details.

7. Click the Restart EDG button after submitting the form

8. Navigate to EDG home once the application restarts to log in. Initial deployment and setup is now complete. If setup was done incorrectly or is incomplete you will get a setup page instead of the EDG home page. You can always navigate to “server”/edg/tbl/setup” to see your configurations.

9.  An administrator should now go over the guidance here to make any additional adjustments to EDG.

Database Type: Application data storage

 

The Database Type parameter is the application storage for your data. When using RDBMS or Data Platform, you will have additional information to configure.

EDG offers three types of persistence technology: (1) relational database, (2) Jena TDB files, or (3) Data Platform.

App data storage type Description File Extension
In-memory + RDBMS persistence Choice of relational DB: Oracle, Microsoft SQL Server, or MySQL, which requires further RDBMS Configuration below, below. .sdb
TDB (One database per graph) Apache Jena TDB, configured for each graph to use its own TDB database.  For EDG instances with application storage type of TDB (Each graph stored in its own, separate database), setting ulimit to unlimited is recommended. This will prevent EDG from reaching too many open files on your instance. .tdb
TDB (Shared graph database) Apache Jena TDB, configured for all graphs to share a single TDB database, data will be stored in the _Data folder at the root of the workspace.
.xdb

 

Data Platform Data Platform as a data store. This will enable all EDG collections to be synced between EDG nodes. See Data Platform documentation for specific instructions on setting up Data Platform. .dpc

Because EDG’s own system graphs also depend on the data storage type, changing the Application data storage should be considered as tantamount to a new installation, especially if the original installation used a non-TDB, remote data-store. Although existing graph data is not directly affected, changing the data storage type’s remote store could entail the need to migrate data from the old source.

The TDB options require no additional setup or parameters. RDBMS each have additional required configuration, as described below.

NOTE: The choice of back-end storage is mainly a customer preference until you are getting into large scale data of over 30 million triples. With the TDB options, the database lives in the workspace file system on the server. With TDB – one database per graph (gTDB), each graph will have it’s own database. With TDB – shared graph database (xTDB), one database contains all the graphs. You will see the extensions in the connector files in your Base URI Management page as “graph name”.tdb and .xdb. Either TDB does not use as much memory as a RDBMS option. It also does not load all the data into cache at server startup. You will notice significantly quicker startup times with TDB. With RDBMS you will have the data residing in a database on another server and the connector files in the workspace. This extension will be .sdb. The difference between Oracle, MySQL and SQL Server is minimal as far as EDG is concerned. They have different ways of processing the reads/writes so performance may differ slightly with large amounts of data. If choosing RDBMS, you should choose what your DBA’s are most comfortable maintaining and tuning.

Even organizations that are expecting to have a relatively small number of triples will often choose a TDB option over RDBMS in order to get up and running quicker and have less moving parts.

With any option you choose, it’s important to keep your workspace regularly backed up or use server snapshots.

 

RDBMS Configuration Parameters

 

For relational RDBMS parameters, the corresponding database must already exist before a user can use the web-based EDG interface to create a new vocabulary in that database.

 

Parameter Default Description
RDBMS URL The URL of the relational database. For example, jdbc:oracle:thin:@localhost:1521:delphi, where delphi is the name of the instance, or jdbc:mysql://localhost:3306/myDatabase. The database with that name must already exist on the database server. (In the latter case, the myDatabase database must already exist on the MySQL system.)

Common formats for the RDBMS URL include:

jdbc:mysql://<server>/<database>
jdbc:oracle:thin:@//<server>:<port>/<service>
jdbc:oracle:thin:@<server>:<port>:<SID>
jdbc:sqlserver://<server>[:<port>][/database][;property=value]

NOTE for SQL Server: A single backslash “\” in the URL string may cause a problem in the vault storage file for the password. Alternatives are (1) to use double-backslashes “\\” or (2) store the password using Password Management or (3) replace the backslash “\” element by a keyword assignment, e.g., “…;instanceName=myInstance;…” instead of “…\myInstance;…”.

RDBMS database type Select the supported type of relational database being used.
RDBMS user name Login name for the database.
RDBMS Update Batch Size 1000 OPTIONAL: This is the number of rows written to the SQL database in each batch. If unset, then 1000 is used. Adjusting it might improve bulk insert performance.
RDBMS Update Fetch Size OPTIONAL: The number of rows returned from the SQL database on each network round trip. Certain values have certain meaning to difference database types. Not all databases use this value.

NOTE: Leaving the Batch and Fetch sizes unset should generally yield acceptable loading/caching performance. Each can be fine-tuned for a particular application by adjusting it up or down and observing the performance changes.

RDBMS password

 

 

    Configure Logging

     

    EDG uses slf4j/log4j as the logging library. The log level can set by the customer as well as the rotation and format. Some of the logging can be accessed in the user interface by administrators; errors, warnings and information. You can filter, send the log to TopQuadrant support or download it from the user interface. Additional logging is available through the container, Tomcat. This will include access and startup logs

    To configure the log size and rotation, modify the log4j2.xml file found here “tomcat”/edg/web-inf/setupdata.

     

    If you encounter server startup problems, the following settings in Apache Tomcat’s conf/logging.properties file can give you a better picture of what’s going on:

    handlers = 1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli.FileHandler, 
      3manager.org.apache.juli.FileHandler, 4host-manager.org.apache.juli.FileHandler, 
      5edg.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler  
    5edg.org.apache.juli.FileHandler.level = FINE 
    5edg.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
    5edg.org.apache.juli.FileHandler.prefix =edg.  
    org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/edg].level = INFO 
    org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/edg].handlers = 5edg.org.apache.juli.FileHandler

     

    Keep in mind that the handlers setting would replace the existing one. It is shown as three lines above, but should be one line in your logging.properties file.

    Tomcat Settings – other

    • A tomcat SSL connector, without an exclusion rule will require some characters be encoded where a flat http connector will not. This can cause problems with embedded links.
    • Set the following in Tomcat relaxedPathChars=”&lt;&gt;” relaxedQueryChars=”&lt;&gt;”

    LDAP Configuration

    This section describes how to use the EDG server with LDAP authentication.

     

    Integration Instructions

    Authentication is the process by which users log on to EDG. It is possible, but not advised, to run EDG without authentication. In this case, all users get full administrative privileges.

    EDG uses the Tomcat web-app server for authentication. This is intended to help implementation of policies such as single sign-on. EDG provides no further features for single sign-on. If this is a policy mandated for your environment, first find some other Java EE based service and understand which Web Application Server they are using, and how they authenticate users in accordance with the single sign-on policy. Then contact TopQuadrant with this information for advice as to how to configure EDG in a similar way.

    This section provides instructions for integrating EDG with LDAP in Linux and Active Directory in Windows Server.

     

    LDAP Configuration within Tomcat

    These instructions cover the configuration of LDAP within Tomcat. For any other web application server, there could be some differences. Note that the exact formats used will depend on your LDAP configuration. Also, although the description below has a step to initiate an LDAP service, normally this is not required because you will be using a pre-existing LDAP service for configuring Tomcat and EDG.

     

    Identifying/Creating a Group and at Least One User in That Group

    In the sample configuration routine shown below, we will set the security to permit a specific group to access EDG. This group needs a name, which is configured within LDAP. You may already have an appropriate group.

     

    Configuring Tomcat to Use LDAP

    Next, we need to configure tomcat to connect to LDAP. This is specified in this document : https://tomcat.apache.org/tomcat-8.5-doc/realm-howto.html#JNDIRealm

    This is the difficult step. It requires adding an entry to server.xml, found in the conf folder for Tomcat. For our example, the entry is as follows:

     

    <Realm   className="org.apache.catalina.realm.JNDIRealm" debug="99"
       connectionURL="ldap://localhost:389"
       userPattern="uid={0},ou=people,dc=tqinc,dc=info"
       roleBase="ou=groups,dc=tqinc,dc=info"
       roleName="cn"
       roleSearch="(memberUid={1})"
       />

     

    inside of the Engine element. Note that in this expression, {0} and {1} both stand in for the username (they are written literally as {0} {1} in the web.xml file), but the actual expressions such as uid={0},ou=people,dc=tqinc,dc=info will be different depending on your LDAP service.

    For reference, this is for group configuration that looks like this in LDAP:

     

    keefe@keefetq:~$ ldapsearch -xLLL -b "dc=tqinc,dc=info" -s sub "(memberUid=keefe)"
    dn: cn=acme,ou=groups,dc=tqinc,dc=info
    objectClass: posixGroup
    cn: acme
    gidNumber: 10001
    description: Group account
    memberUid: keefe
    

     

    If your LDAP directory is setup in a straightforward way, it should be easy to modify the search expression appropriately. However, you may need assistance from someone who knows your LDAP system.

    Three pieces of information are needed:

    1. A base pattern to identify groups in LDAP, (in this case: ou=groups,dc=tqinc,dc=info).
    2. A property at which the rolename we will use in securing EDG may be found, in this case:cn
    3. A search filter that indicates role membership, in this case : (memberUid={1}) where {1} is the username.

    Active Directory Integration within Tomcat

    These instructions concern integrating Active Directory within Tomcat using the LDAP protocol. Note that the exact format depends on your LDAP configuration.

     

    Populate LDAP

    The security to permit a specific group of users to must be set to access EDG. This group needs a name, which is configured within LDAP. You may already have an appropriate group.

    1. Create users in Active Directory for EDG use. If existing users need to use the EDG, then ignore this step.
    2. Create a TopBraid EDG group named acme and associate users with this group in Active Directory using the wizard.

    Configuring Tomcat to Use LDAP

    Next, configure tomcat to connect to LDAP. This is specified in this document : https://tomcat.apache.org/tomcat-8.5-doc/realm-howto.html#JNDIRealm

    This is the difficult step. It requires adding an entry to server.xml, found in the conf folder for Tomcat. For our example, the entry is as follows:

     

    <Realm className="org.apache.catalina.realm.JNDIRealm"
    
    		connectionURL="ldap://localhost:3268"
    		authentication="simple"
    		referrals="follow"
    
    		connectionName="jeremy@tblive.tqinc.info"
    		connectionPassword="enter password here"
    
    		userSearch="(sAMAccountName={0})"
    		userBase="CN=Users,DC=tblive,DC=tqinc,DC=info"
    		userSubtree="true"
    
    		roleSearch="(member={0})"
    		roleName="cn"
    		roleSubtree="true"
    		roleBase="CN=Users,DC=tblive,DC=tqinc,DC=info"
    
    		debug="99"
    	/>

     

    inside of the Engine element.

    In the userSearch expression, {0} stands for the username as typed by the end-user, whereas in the roleSearch expression, {0} stands for the distinguished name found from the user look up. (These are written literally as {0} in the server.xml file), but the actual expressions such as (sAMAccountName={0}) and CN=Users,DC=tblive,DC=tqinc,DC=info will be different depending on your LDAP service. With Active Directory, (sAMAccountName={0}) and roleSearch="(member={0})" should generally be correct.

    If your LDAP directory is setup in a straightforward way, it should be easy to modify the search expression appropriately. However, you may need assistance from someone who knows your LDAP system.

    Three pieces of information are needed:

    1. A base pattern to identify groups in ldap, (in this case: CN=Users,DC=tblive,DC=tqinc,DC=info).
    2. A property at which the rolename we will use in securing EDG may be found, in this case:cn
    3. A search filter that indicates role membership, in this case : (member={0}) where {0}is the distinguishedName (and {1} is the username).

    This can also be seen using an LDAP explorer tool something similar to JXplorer

    For reference, the screen-shot below shows how the Active Directory group configuration looks like in JXplorer:

    Connecting to LDAP:

     

     

    User record for user Scott:

     

     

    Group record belonging to user Scott:

     

     

    Configuring EDG to Use LDAP

    Finally, add a security constraint to EDG in the web.xml file:

     

    <security-constraint>
      <display-name>Example Security Constraint</display-name>
    
      <web-resource-collection>
      <web-resource-name>Protected Area</web-resource-name>
    <!-- Define the context-relative URL(s) to be protected --> 
      <url-pattern>/*</url-pattern>
    <!-- If you list http methods, only those methods are protected -->
      <http-method>DELETE</http-method>
    
      <http-method>GET</http-method>
      <http-method>POST</http-method>
      <http-method>PUT</http-method>
      </web-resource-collection>
    
      <auth-constraint>
    <!-- Anyone with one of the listed roles may access this area -->
      <role-name>acme</role-name>
      </auth-constraint>
    </security-constraint>
    
    <!-- Default login configuration uses basic authentication -->
    
    <login-config>
      <auth-method>BASIC</auth-method>
      <realm-name>TopBraid</realm-name>
    </login-config>
       
    <!-- Security roles referenced by this web application -->
    
    <security-role>
      <role-name>acme</role-name>
    </security-role>

    Starting an LDAP Service in Linux

     This is intended as a quick guide. More detailed instructions can be found elsewhere: 

    1. Install LDAP. For an example of installation documentation, see the documentation for the Ubuntu OpenLDAP Server.
    2. Configure LDAP by running through the setup wizard and setting up a domain name and related information. For example:
      domain = tqinc.info
      organization name = tq
      
    3. Populate LDAPThe example below, which uses the ldapadd utility, is based Ubuntu’s OpenLDAP configuration guide, replacing all example.com references with the domain specified above: tqinc.info.
      keefe@keefetq:~$ ldapadd -x -D cn=admin,dc=tqinc,dc=info -W -f test.ldif 
      Enter LDAP Password:
      adding new entry "ou=people,dc=tqinc,dc=info"
      adding new entry "ou=groups,dc=tqinc,dc=info"
      adding new entry "uid=john,ou=people,dc=tqinc,dc=info"
      adding new entry "cn=example,ou=groups,dc=tqinc,dc=info"
      

    Identifying/Creating a Group and at Least One User in that Group

     In this stage, we’ll set the security to permit one or more specific groups to access EDG. This group needs a name, which is configured within LDAP.

    First we install the LDAP scripts and add a group and a user,remembering to configure the LDAP scripts first. With a different LDAP server, this step will be different.

    keefe@keefetq:~$ sudo ldapaddgroup acme
    Error adding group acme to LDAP
    

     

    The error message shows that the LDAP scripts must be configured first:

    keefe@keefetq:~$ sudo gedit /etc/ldapscripts/ldapscripts.conf 
    keefe@keefetq:~$ sudo sh -c "echo -n 'monkey' > /etc/ldapscripts/ldapscripts.passwd"
    keefe@keefetq:~$ sudo chmod 400 /etc/ldapscripts/ldapscripts.passwd
    keefe@keefetq:~$ sudo ldapaddgroup acme
    Successfully added group acme to LDAP
    keefe@keefetq:~$ sudo ldapadduser keefe acme 
    Successfully added user keefe to LDAP 
    keefe@keefetq:~$ sudo ldapsetpasswd keefe 
    Changing password for user uid=keefe,ou=people,dc=tqinc,dc=info
    New Password:
    Retype New Password:
    Successfully set password for user uid=keefe,ou=people,dc=tqinc,dc=info
    keefe@keefetq:~$
    

    LDAP Servers (Service Providers)

    The preceding LDAP sections configure EDG to use of LDAP for authentication: user login identity and roles. To further configure EDG for using additional LDAP information, e.g., email, see Server Administration – Server Configuration – LDAP Parameters

    SAML Authentication

    This section describes how to use the EDG server with SAML authentication. When choosing this method of authentication, be sure that you have a technical resource familiar with your Identity Provider to assist with configuration. TopQuadrant is unable to assist with specific configuration options for each customers Identity Provider.

    The TopBraid SAML valve supports SP-Initiated Browser based SAML 2.0 Single Sign On (SSO). HTTPS is required to be used by the Tomcat instance and the Tomcat will need to be supplied the certificates of the IDP to allow trusted HTTP communication. Likewise, the IDP will need to have the certificates of the Tomcat instance. The TopBraid SAML valve requires at least 2 attributes from the IDP, one to be mapped to the user object and one to be mapped to the role object. While the valve is also capable of detecting authorization headers – ie: basic auth included, those are routed to the authentication providers of the VM, eg: JNDI LDAP and/or local tomcat-users.xml.

    If a BASIC authorization header is missing, then the valve will proceed to the IDP login process.

    – SAML will only work with secure connections (HTTPS).
    – API clients will need to use BASIC or OAuth if they want to use SAML for Browser SSO.
    – API clients will need to connect to EDG using HTTPS if EDG is using SAML.
    – Any SPARQLMotion modules that use URLs to connect to EDG will need to use HTTPS if EDG is using SAML.

    The valve consists of the following (provided in the zip file that also contains the edg.war):
    -SAML valve jar and dependencies
    -modified context.xml

    Place the Auth jars in the lib directory of your Tomcat instance.

    Place the context.xml in your webapp/edg/META-INF directory. It is a good idea to backup the original context.xml

    Context.xml Changes

    1. Modify the context.xml to match the path to your IDP Federated Metadata URL.
    – The path to the metadata xml can be either file:/// or http:// paths.
    – file:/// paths should be absolute filesystem paths.
    – if there is latency to the http:// path, the xml can be downloaded to the local filesystem to improve performance.

    2. Modify the context.xml to match the attribute mappings sent by your IDP to the username and role fields used by EDG.
    – Map username/login ID URI =>name     * required
    – Map group or role URI =>role      * required for RBAC

    3. Additional attributes can be added for email address and display name. These will then be used in EDG’s user directory:
    -Map email URI =>http://edg.topbraid.solutions/model/email
    -Map display name URI=>http://edg.topbraid.solutions/model/name

    4. Define the Service Provider
    -When defining the Service Provider in the IDP, the assertion consumer URL will be http://yourserver/edg/saml

    An appropriate SSO session timeout will be determined from the assertion sent from the IDP, if possible. Alternately, you can configure sessionTimeout (in minutes) in context.xml

    Debugging

    If you need to debug the SAML Response or assertion attributes, you can place the following at the end of your logging.properties file in Tomcat and restart Tomcat.
    java.util.logging.ConsoleHandler.level = FINEST
    org.topbraidlive.auth.adapters.saml.tomcat.level = FINEST

    This will give the entire SAML response and attributes in the catalina log file.

    Detailed instructions are provided upon request to support@topquadrant.com.

    OAuth Authentication

    Integrating APIs with EDG using OAuth

    TopBraid EDG supports OAuth for client APIs. Interactive users will need to use another authentication method (SAML, Basic, or Forms).Unzip auth-7.0.2.zip into the /lib directory of your Tomcat installation.  E.g. /var/lib/tomcat8.5/lib
    • cd /var/lib/tomcat8.5
    • unzip auth-7.0.2.zip -d lib
    • If you had TopBraid SAML or OAuth from a previous version of EDG, you will need to remove the old jars

     

    Add Oauth config to EDG context.xml

    • Edit webapps/edg/META-INF/context.xml
    • If you previously had TopBraid SAML from a version of EDG prior to 6.4.2, then you need to adjust the className for SAML to  org.topbraidlive.auth.adapters.saml.tomcat.SamlValve
    • Make sure the OAuth valve comes before the SAML valve

     

    <?xml version=’1.0′ encoding=’utf-8′?>

    <!– Copyright (c) 2016 TopQuadrant, Inc. All rights reserved. –>

    <Context path=”/edg”>

        <!–

          By default, Tomcat 8 tries to scan our jars for tag library

          definitions.  We don’t have any, so skipping these jars will

          improve startup and JSP compile times.

        –>

        <JarScanner>

            <JarScanFilter defaultTldScan=”false”/>

        </JarScanner>

     

        <Resources cacheMaxSize=”40960″/>

            <!–  Disable session persistence per PLAT-769 because Java8/Mars seems to have broken it.  –>

            <Manager pathname=”” />

     

      <!–

       TopBraid adapter for OAuth

      –>

      <Valve className=”org.topbraidlive.auth.adapters.oauth.tomcat.OAuthValve”

          attributeMappings=”sub=>name;appid=>name;role=>role;apptype=>role”

          authorizationServer=”https://oauth-idp.topquadrant.com/adfs”

          resourceURI=”https://oauth-edg.topquadrant.com/edg/”

          tokenRedirectURI=”https://oauth-edg.topquadrant.com/edg/oauth/getTokens”

          tokenRequestURI=”/oauth/getTokens”/>

     

      <!–

       TopBraid adapter for SAML

      –>

      <Valve className=”org.topbraidlive.auth.adapters.saml.tomcat.SamlValve”

               assertionConsumerService=”/saml”           attributeMappings=”http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress=>name;http://schemas.microsoft.com/2012/12/certificatecontext/field/subjectname=>role”           federationMetadata=”https://saml-idp.topquadrant.com/FederationMetadata/2007-06/FederationMetadata.xml”

               serviceProvider=”https://saml-edg.topquadrant.com/edg/saml”/>

    </Context>

     

    Browse to your EDG webapp UI.  Navigate to Server Administration -> Server Configuration

     

    Under the OAuth Parameters, fill in the following:

      • Authorization server URL  (eg, ADFS https://server/adfs/oauth2/token)
      • Client ID obtained by your SSO administrator for this Application
      • Client secret obtained by your SSO administrator for this Application

     OAuth workflows

    The authorization code workflow involves the following steps: 

    The OAuth client initiates the flow when it directs the user agent of the resource owner to the authorization endpoint. … The OAuth client requests an access token from the authorization server through the token endpoint.  This is likely where you will utilize internal scripts/apis to access EDG.

     

    The client credentials workflow involves EDG-to-EDG applications, such as Publishing to Explorer, Send Projects or Federated Sparql queries.  The system authenticates and authorizes the app (EDG) rather than a user. For this scenario, typical authentication schemes like username + password or social logins don’t make sense. Instead, uses the Client Credentials Flow in which they pass along their Client ID and Client Secret to authenticate themselves and get a token.

    1. Your app authenticates with the Authorization Server using its Client ID and Client Secret (/https://server/adfs/oauth2/token endpoint).
    2. Your Authorization Server validates the Client ID and Client Secret.
    3. Your Authorization Server responds with an Access Token.
    4. Your application can use the Access Token to call an API on behalf of itself.
    5. The API responds with requested data.

     

    The Password grant workflow is a way to exchange a user’s credentials for an access token. Because the client application has to collect the user’s password and send it to the authorization server, it is not recommended that this grant be used at all anymore.

     

    This flow provides no mechanism for things like multifactor authentication or delegated accounts, so is quite limiting in practice.

     

    The latest OAuth 2.0 Security Best Current Practice disallows the password grant entirely.

     

    These 4 attributes are needed to fulfil OAuth authorization requirements in EDG.

    •         appid
    •         sub
    •         role
    •         apptype

     * a role of Confidential must be placed in web.xml for ADFS OAuth.

    If you need to test programs against EDG, you can do so with the tool included at https://yourserver/edg/oauth/getTokens

     

     

    Tomcat users authentication

    Users can be specficied in the user realm in tomcat-users.xml.

    An example is below:

     <role rolename=”Admin”/>
    <role rolename=”Manager”/>
    <role rolename=”Editor”/>
    <role rolename=”Viewer”/>
    <user username=”Admin_user” password=”password32″ roles=”Admin,Manager”/>
    <user username=”Editor_user” password=”password54″ roles=”Editor”/>
    <user username=”Guest” password=”password76″ roles=”Viewer”/>

     

    It is imperative that your web.xml security roles match the roles defined in this tomcat-users.xml file, case sensative.

     

     

    Accessing the EDG application

    For access to the EDG Console, send your browser to the URL http://[Web-application-server-host]:[Web-application-server-port]/edg/tbl/

    The project server.topbraidlive.org and several others will be created in the workspace. If not, there were initialization problems and the TopBraid server will not work properly.The most common source of problems are folder permissions for the process running the Web Application Server. Try removing the workspace folder and reloading web.xml, making sure that the process has write access to the parent folder of the workspace folder.

     

    License registration

    As part of the deployment/setup wizard, the license file will be added to EDG. If this is incorrect, expired or missing, a message will appear that points to the registration page, as shown in the following screen shot:

     

     

    1. Click on the link. A page will appear stating that there are “No currently registered products!”
    2. Click on the Change or Update Registration Information link. Use the Choose File button to choose the license file (.lic file) sent by TopBraid support when the license was issued.

    A dialog will appear, verifying that the license was updated. The licenses and their expiration date will be displayed. This page is always available through the Server Administration tab and choosing the Server Administration then the Product Registration links.

    The TopBraid EDG license will control the number of concurrent users, human and API, that can be using EDG at any one time. For form based authentication, a license seat is used by each unique session. This means that many tabs in one browser are all counted in the same user session/license seat but unique browsers by the same user are counted as different sessions using their own license seat. API calls authenticate into EDG as well using a license seat during processing. For APIs if using form authentication, be sure to log out after completion. Sessions expire upon timeout (set in web.xml in Tomcat), thus releasing the license seat held by the session. 

    To track user licenses with form authentication, follow the steps below. This will add details to the log files for user log in, log out, and expired sessions.

    1. Navigate to your workspace.

    2. Edit the file log4j2.xml.

    3.  Add the following to the Loggers section:

    <Logger name="org.topbraid.core.license.FormsAuthConcurrentUsers" level="trace" />

     

    Sample Data

    The EDG server-based application does not include the sample data that is included in TopBraid Composer-Maestro Edition (TBC-ME). If desired, the examples can be deployed from TBC-ME to the server by clicking on the sample project in TBC-ME’s Navigator window and choosing Export… > TopBraid Composer > Deploy Project to TopBraid Live Server. Permission will default to “Administrator” for the sample project. Please update this in the role management page if needed.