Setting Up FME Flow with Ansible

Setup the FME Flow Core, FME Flow Database and FME Flow System Share on MS Windows using Ansible.

Projects: c2platform/rws/ansible-gis, c2platform.wincore, c2platform.gis


Here’s an overview of the process to set up FME Flow Core, FME Flow Database, and FME Flow System Share using Vagrant and Ansible. For more information about these components, refer to FME Flow Architecture  .

  1. Vagrant creates two VirtualBox Windows VMs: gsd-fme-core, gsd-ad, and a LXD container gsd-db1.
  2. Vagrant utilizes the Vagrant Windows Sysprep Provisioner  on gsd-fme-core and gsd-ad.
  3. Vagrant runs the Ansible provisioner in the following order on nodes:
    1. On gsd-ad, the c2platform.wincore.ad collection is used to configure the AD domain controller for the domain ad.c2platform.org.
    2. PostgreSQL 14 is installed on gsd-db1, along with a database and database user with necessary privileges.
    3. On gsd-fme-core, Ansible performs the following steps:
      1. Joins the node to the Windows domain ad.c2platform.org.
      2. Installs Java using the c2platform.gis.java role.
      3. Installs Tomcat using the c2platform.gis.tomcat role.
      4. Installs FME Flow Core using the c2platform.gis.fme role.

The diagram below illustrates the setup achieved with Vagrant, excluding the reverse proxy gsd-rproxy1.

Prerequisites

Before proceeding, make sure you have completed the steps to Setup the RWS Development Environment on Ubuntu 22

Setup

Use the following commands to run the FME play and create the gsd-ad, gsd-db1, and gsd-core-fme nodes. Running the vagrant up command will take approximately 25 minutes to complete.

export BOX="gsd-ad gsd-db1 gsd-core-fme"
export PLAY="plays/gis/fme.yml"
vagrant up $BOX | tee provision.log

Verify

gsd-ad

  1. Log in to gsd-ad and execute systeminfo | Select-String "Domain". This should return ad.c2platform.org.

    Show me
    PS C:\Users\vagrant> systeminfo | Select-String "Domain"
    
    OS Configuration:          Primary Domain Controller
    Domain:                    ad.c2platform.org
    
    
    PS C:\Users\vagrant> nslookup ad.c2platform.org
    Server:  ip6-localhost
    Address:  ::1
    
    Name:    ad.c2platform.org
    Addresses:  1.1.8.108
              10.0.2.15
    
    PS C:\Users\vagrant>
    
  2. Open the DNS Manager and check the properties of the DNS server GSD-AD. Only 1.1.8.108 should be enabled as a listening IP address.

    Show me
  3. On your Ubuntu laptop run dig @1.1.8.108 ad.c2platform.org. This should resolve ad.c2platform.org to 1.1.8.108.

    Show me
    onknows@io3:~$ dig @1.1.8.108 ad.c2platform.org
    
    ; <<>> DiG 9.18.12-0ubuntu0.22.04.3-Ubuntu <<>> @1.1.8.108 ad.c2platform.org
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27806
    ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4000
    ;; QUESTION SECTION:
    ;ad.c2platform.org.		IN	A
    
    ;; ANSWER SECTION:
    ad.c2platform.org.	600	IN	A	1.1.8.108
    ad.c2platform.org.	600	IN	A	10.0.2.15
    
    ;; Query time: 0 msec
    ;; SERVER: 1.1.8.108#53(1.1.8.108) (UDP)
    ;; WHEN: Wed Oct 25 09:40:07 CEST 2023
    ;; MSG SIZE  rcvd: 78
    

gsd-fme-core

Log in to gsd-fme-core.

  1. Verify that the computer is part of the domain ad.c2platform.org:

    vagrant ssh gsd-fme-core
    powershell
    systeminfo.exe | Select-String "Domain"
    
    Show me
    vagrant@GSD-FME-CORE C:\Users\vagrant>powershell
    Windows PowerShell
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows
    
    PS C:\Users\vagrant> systeminfo.exe | Select-String "Domain"
    
    Domain:                    ad.c2platform.org
    
    
    PS C:\Users\vagrant>
    

Login gis-backup-operator

Login as gis-backup-operator using remote desktop to gsd-fme-core to confirm that the user has been created with the correct password.

Database

On gsd-fme-core, connect using remote desktop using the Vagrant user, start pgAdmin, and import the settings in file C:\Users\Public\Desktop\pgadmin.json.

Connect to the server using password secret. Drill down to the tables of the fmeserver database, you should see the database tables of FME Flow like fme_action etc. This verifies that the database has been created.

Review

In the RWS Ansible Inventory project c2platform/rws/ansible-gis review specific plays and configurations.

FME Play

Review the play plays/gis/fme.yml. Pay attention to the use of the when condition, ensuring that the installation of Java and Tomcat is restricted to gsd-fme-core.

This play utilizes various roles. Notable ones are:

  • The fme_flow and tomcat role in the c2platform.gis collection.
  • win from the c2platform.wincore collection, which is integrated into both tomcat and fme_flow roles.This integration facilitates the management of Windows resources through fme_flow_win_resources and tomcat_win_resources.

Tomcat context.xml

The variable tomcat_win_resourcesis used to manage the the context.xml file of Tomcat. In group_vars/fme_core/tomcat.yml take note of the following item:

  - path: "{{ tomcat_home }}/conf/context.xml"
    xpath: /Context
    fragment: >-
      <Valve className="org.apache.catalina.authenticator.SSLAuthenticator"
      disableProxyCaching="false" />      
    notify: Restart Tomcat service

This item adds a Valve element to the file context.xml

<Valve className="org.apache.catalina.authenticator.SSLAuthenticator"       disableProxyCaching="false" />

Tomcat web.xml

Simarly using tomcat_win_resources a security-constraint element is added to web.xml. Take note of xpath expression. This XPath expression /*[local-name()='web-app'] selects the root element only if its local name (i.e., the tag name without the namespace prefix) is ‘web-app’. This is a workaround and is generally less precise than using proper namespace handling, but it’s often used in tools that don’t support XML namespaces directly.

  - path: "{{ tomcat_home }}/conf/web.xml"
    backup: true
    xpath: /*[local-name()='web-app']
    fragment: >-
      <security-constraint>
        <web-resource-collection>
          <web-resource-name>HTTPSOnly</web-resource-name>
          <url-pattern>/*</url-pattern>
        </web-resource-collection>
        <user-data-constraint>
          <transport-guarantee>CONFIDENTIAL</transport-guarantee>
        </user-data-constraint>
      </security-constraint>      
    notify: Restart Tomcat service

FME fmeServerConfig.txt

Take note of file fmeServerConfig.txt in directory D:\Apps\FME\Flow\Server. It has been changed by Ansible, it contains the line

FME_SERVER_WEB_URL=https://GSD-FME-CORE.ad.c2platform.org:443

This line is managed by Ansible using configuration in group_vars/fme_core/main.yml. This file contains variable fme_flow_win_resources as shown below:

fme_flow_win_resources:
  core:
    - name: FME_SERVER_WEB_URL
      module: win_lineinfile
      path: "{{ fme_flow_home }}/Server/fmeServerConfig.txt"
      regex: '^FME_SERVER_WEB_URL='
      line: FME_SERVER_WEB_URL=https://{{ ansible_fqdn }}:{{ gs_tomcat_https_port }}
      notify: Restart FME Flow services

Default Install

To perform a default installation of FME Flow, which includes deploying a PostgreSQL database without a separate Tomcat instance, follow these steps:

  1. Remove or disable the fme_flow_install_command.
  2. Recreate the environment by executing the following commands:
vagrant destroy gsd-fme-core -f
vagrant up gsd-fme-core

After successfully completing these commands, you should be able to access the FME Flow interface by visiting http://localhost  , which will redirect you to http://localhost/fmeserver/  . Log in as admin with the password admin.