Install an SSL Certificate on your FileMaker Server

Koen Van Hulle, Claris Engage 2020

Menu

1. Introduction: Why SSL Certificates Matter for FileMaker Server
2. The Story of ClipFactory: Eric and Emma’s SSL Challenge
3. What Exactly Does SSL/TLS Solve?
4. The Chain of Trust: Why It’s Essential for SSL
5. Different Levels of SSL Validation Explained
6. Step-by-Step Guide to Installing an SSL Certificate on FileMaker Server
7. Troubleshooting Common SSL Certificate Issues
8. TLS and FileMaker Server: Security Pitfalls and Solutions
9. Testing Your TLS Settings: How to Achieve Maximum Security
10. Conclusion: Key Insights for Installing SSL on FileMaker Server

Introduction: Why SSL Certificates Matter for FileMaker Server #tag1

With the rising number of data breaches and cybersecurity threats, securing the connection between your FileMaker Server and your users has become essential. An SSL certificate ensures that all communication between your server and clients is encrypted, preventing potential attackers from eavesdropping, tampering, or stealing sensitive data.

However, the process of obtaining, installing, and managing SSL certificates is often viewed as complex and cumbersome. In this session, Koen Van Hulle walks us through the SSL installation process on FileMaker Server, uncovering the nuances and addressing common challenges. You’ll learn how to ensure that your server not only has a valid SSL certificate but is also fully secured with up-to-date TLS settings.

The Story of ClipFactory: Eric and Emma’s SSL Challenge #tag2

At the fictional company ClipFactory, Eric and Emma are tasked with setting up a secure FileMaker Server for managing contracts and deals for their flagship product, Clippy—the first internet-connected paperclip. While Emma oversees contracts, Eric is responsible for the server’s technical setup. But as Eric quickly learns, installing an SSL certificate on FileMaker Server is no easy feat.

Eric is relatively new to managing FileMaker Server and finds himself puzzled by the complexities of SSL certificates. Together with Emma, he embarks on a journey to properly secure their server. Along the way, they explore why SSL is necessary, how to install it correctly, and what additional security measures are required.

What Exactly Does SSL/TLS Solve? #tag3

TLS (Transport Layer Security) and its predecessor, SSL (Secure Socket Layer), are protocols designed to ensure secure communication over a network. But what specific problems do they solve?

1. Identity Validation: TLS/SSL certificates verify that the server you’re connecting to is indeed who it claims to be. For example, when you access fms.clipfactory.com, the SSL certificate ensures that you’re actually connecting to ClipFactory’s server, not an imposter’s.
2. Encryption: SSL/TLS encrypts the data being exchanged between the client and server, making it unreadable to any potential eavesdroppers. This ensures that sensitive information, such as login credentials or financial data, remains secure.
3. Integrity Verification: SSL/TLS checks that the data being transmitted hasn’t been tampered with in transit. If an attacker tries to modify the data, the recipient will be alerted that the message has been altered.
4. Authentication: The server and client authenticate each other, ensuring both parties can trust the identity of the other.

Without SSL/TLS, data transmitted between your FileMaker Server and clients would be vulnerable to various types of cyberattacks, including man-in-the-middle (MITM) attacks, where an attacker intercepts and manipulates the communication between two parties.

The Chain of Trust: Why It’s Essential for SSL #tag4

One of the key components of SSL/TLS is the Chain of Trust, a hierarchical structure that ensures the legitimacy of an SSL certificate. Understanding how this chain works is crucial to grasping how SSL certificates function.

The Chain of Trust typically involves three types of certificates:

• Root Certificate: Issued by a trusted Root Certificate Authority (CA), this is the foundation of trust. A Root CA is trusted by operating systems, browsers, and applications. For example, companies like DigiCert act as Root CAs.
• Intermediate Certificate: This certificate is issued by a Root CA and used to sign Leaf Certificates. The Intermediate CA is trusted by the Root CA but not by clients directly. It acts as a middleman between the Root and Leaf certificates.
• Leaf Certificate: Also known as the end-entity certificate, this is the SSL certificate issued to the server (in this case, fms.clipfactory.com). When a client connects to the server, the Leaf Certificate, along with the Intermediate Certificate, is presented to create a secure connection.

The Chain of Trust in Action

Think of a real-world scenario: Eric needs to sign a contract with Apple, and he’s working with an Apple representative, Jack. Jack presents his credentials, which were signed by Eddy Cue, a senior Apple executive. But to fully trust Jack, Eric also wants proof that Eddy Cue has the authority to sign Jack’s credentials, which is verified by a certificate signed by Tim Cook, Apple’s CEO. The Chain of Trust is now established, allowing Eric to trust Jack through the connections up the chain.

Similarly, when you visit a website or server with an SSL certificate, your browser or application checks the certificate’s Chain of Trust. If the chain is broken or contains an untrusted entity, you’ll receive a warning that the certificate is invalid.

Self-Signed Certificates: The Weak Link

Unlike certificates issued by trusted CAs, self-signed certificates skip the Chain of Trust. In these cases, the server essentially signs its own certificate, which means there’s no third-party validation of its legitimacy. This is why FileMaker’s default self-signed certificate is considered insecure and should only be used for testing purposes.

Different Levels of SSL Validation Explained #tag5

SSL certificates come with varying levels of validation, each offering a different degree of trust. Understanding the differences between these validation types will help you choose the right one for your FileMaker Server.

1. Domain Validation (DV): This is the most basic and affordable type of SSL certificate. The Certificate Authority (CA) simply checks whether the applicant owns the domain in question. This validation is done by confirming that the applicant has control over the domain’s DNS records. DV certificates provide encryption but don’t offer any information about the organization behind the website.
2. Organization Validation (OV): In addition to domain ownership, the CA verifies the organization’s legitimacy by checking business registration records. OV certificates display the organization’s name, providing a higher level of trust for users.
3. Extended Validation (EV): The highest level of validation, EV certificates undergo a rigorous vetting process. The CA not only verifies domain ownership and organizational legitimacy but also checks legal, physical, and operational existence. EV certificates are typically used by large enterprises, government agencies, or financial institutions and are often displayed with a green address bar in the browser.

Choosing the right SSL certificate depends on your organization’s security needs and budget. For internal FileMaker Server deployments, a Domain Validated (DV) certificate might suffice, but for public-facing applications, you might want to consider Organization Validation (OV) or Extended Validation (EV) for additional trust.

Step-by-Step Guide to Installing an SSL Certificate on FileMaker Server #tag6

Installing an SSL certificate on FileMaker Server involves several important steps. Eric’s journey provides a perfect case study for understanding the full process, from creating a Certificate Signing Request (CSR) to configuring FileMaker Server.

1. Create a Certificate Signing Request (CSR)

The first step in obtaining an SSL certificate is generating a CSR. This file contains all the necessary information about your server, including the fully qualified domain name (FQDN), organization details, and public key. There are three ways to generate a CSR:

• Using the FileMaker Server Admin Console: The easiest method is to use the FileMaker Server’s built-in commands to generate a CSR and private key: fmsadmin certificate create --name fms.clipfactory.com --output fms.clipfactory.csr
• Using OpenSSL: If you prefer using OpenSSL on a Mac or Linux machine, you can generate a CSR with the following command: openssl req -new -newkey rsa:2048 -nodes -keyout fms.clipfactory.key -out fms.clipfactory.csr
• Using IIS (Windows Server): For Windows users, IIS provides a graphical interface to generate a CSR under the Server Certificates section.

2. Order the Certificate

Once the CSR is generated, upload it to a Certificate Authority (CA) such as DigiCert, Let’s Encrypt, or Sectigo. When ordering, ensure you select the PEM format for FileMaker Server compatibility.

3. Complete the Validation Process

Depending on the type of certificate (DV, OV, or EV), the CA will initiate a validation process. For DV certificates, this might involve adding a TXT record to your DNS, uploading a file to your web server, or verifying ownership via email.

For OV and EV certificates, expect more in-depth checks, such as submitting business registration documents and receiving a phone call from the CA.

4. Install the Certificate

Once validation is complete, the CA will issue your SSL certificate. You will typically receive the following files:

• The Leaf Certificate: The actual certificate for your server.
• The Intermediate Certificate: If required, this establishes the chain of trust.
• The Private Key: This was generated along with your CSR and is essential for SSL installation.

To install the SSL certificate on FileMaker Server:

1. Upload the Leaf Certificate and Intermediate Certificate.
2. Provide the Private Key and Private Key Password (if applicable).
3. Restart FileMaker Server to activate the SSL certificate.

Troubleshooting Common SSL Certificate Issues #tag7

Even with careful planning, SSL certificate installation can sometimes encounter unexpected hurdles. Here are some common issues Eric faced and how he resolved them.

Wrong Certificate Format

FileMaker Server requires SSL certificates to be in PEM format. If your certificate is delivered in a different format, such as P7B or PFX, you can use OpenSSL to convert it.

# Convert P7B to PEM
openssl pkcs7 -print_certs -in cert.p7b -out cert.pem

# Convert PFX to PEM
openssl pkcs12 -in cert.pfx -out cert.pem -nodes

“PRE-SIGN FAILED” Error

This error often arises when your domain is restricted to only allow certificates from certain Certificate Authorities (CAs). This restriction is controlled by the CAA (Certification Authority Authorization) DNS record. If your selected CA isn’t listed in the CAA record, the certificate request will fail.

To resolve this:

1. Modify the CAA DNS record to allow the chosen CA, or
2. Use a CA that’s already authorized in the DNS record.

Missing SAN (Subject Alternative Name) Extension

Some internal Certificate Authorities require the SAN extension to be present in the CSR. This extension lists all alternative domain names that the certificate should cover. You can add the SAN extension using OpenSSL by creating a configuration file:

bashCopy code[ req ]
default_bits       = 2048
prompt             = no
default_md         = sha256
distinguished_name = dn
req_extensions     = req_ext

[ dn ]
C = US
ST = California
L = San Francisco
O = ClipFactory
OU = IT
CN = fms.clipfactory.com

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = fms.clipfactory.com
DNS.2 = webdirect.clipfactory.com

Then, generate the CSR with:

openssl req -new -key fms.clipfactory.key -out fms.clipfactory.csr -config san_config.cnf

TLS and FileMaker Server: Security Pitfalls and Solutions #tag8

After successfully installing an SSL certificate, Eric realized that his FileMaker Server was still vulnerable to certain attacks. The SSL certificate only encrypts data, but ensuring secure TLS settings is equally important.

Cipher Suites: What Are They?

A cipher suite is a combination of encryption algorithms used to secure a connection. FileMaker Server uses modern cipher suites like TLS 1.2, which is secure. However, older versions of FileMaker Server (and some default settings) may still support outdated cipher suites, such as TLS 1.0 or 3DES, which are vulnerable to attacks like POODLE or BEAST.

To mitigate these risks:

1. Always update to the latest version of FileMaker Server.
2. Manually disable older cipher suites if necessary.

FileMaker Server and Web Components

FileMaker Server has two main components that handle traffic:

1. The FileMaker Server Core: This handles connections for FileMaker Pro and Go. It supports TLS 1.2 by default.
2. The Web Server: This handles WebDirect traffic and runs on Apache (macOS/Linux) or IIS (Windows). The web server configuration affects how TLS is implemented for web-based traffic.

Eric discovered that his Windows Server was configured with outdated cipher suites. He had to modify the Windows registry to enable modern cipher suites and disable outdated ones.

For Linux/macOS, the Apache settings can be modified in the configuration files:

# /etc/httpd/conf.d/ssl.conf
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite HIGH:!aNULL:!MD5

Testing Your TLS Settings: How to Achieve Maximum Security #tag9

After configuring TLS settings, Eric wanted to verify that his server was secure. He used two online tools to check the status of his SSL/TLS configuration.

1. ImmuniWeb

This service tests your TLS settings and provides a compliance check for standards like PCI DSS, HIPAA, and NIST. It offers a detailed report on vulnerabilities and suggests ways to improve your security.

2. SSL Labs

SSL Labs is another popular tool that rates your server’s security on a scale from A+ to F. It evaluates the server’s support for secure cipher suites, protocol versions, and whether it’s vulnerable to attacks like POODLE or BEAST.

After running these tests, Eric discovered that his server had a C rating. By adjusting the cipher suites and TLS versions, he was able to raise the security rating to A+, ensuring that his server was fully compliant and secure.

Conclusion: Key Insights for Installing SSL on FileMaker Server #tag10

Eric and Emma’s journey through SSL installation on FileMaker Server highlights several crucial lessons:

1. SSL/TLS is More Than Just Encryption: It involves identity validation, communication integrity, and authentication.
2. Choose the Right SSL Certificate: Depending on your needs, select a Domain Validation (DV), Organization Validation (OV), or Extended Validation (EV) certificate.
3. Always Keep Your Server Updated: Use the latest version of FileMaker Server and ensure that your TLS settings are up to date.
4. Test Your Security Regularly: Use tools like ImmuniWeb and SSL Labs to ensure that your server’s SSL/TLS configuration is secure and compliant.

By following these steps, you can ensure that your FileMaker Server is not only encrypted but also fully secured against modern threats.

This comprehensive guide covers every aspect of installing and configuring an SSL certificate on FileMaker Server, ensuring that both your data and your users remain secure.