Modern DevOps: Making the Switch

Dimitris Kokoutsidis 24 mins0

A Comprehensive and Detailed Guide

Mislav Kos, Sep 2, 2020, Claris Engage 2020

Table of Contents

  1. Introduction
  2. What is DevOps?
  3. DevOps in FileMaker: An Overview
  4. Core DevOps Practices Explained
  5. The DevOps Loop: A Detailed Breakdown
  6. Version Control in FileMaker Development
  7. Automated Testing in FileMaker
  8. Building an Agile DevOps Workflow
  9. Automated Deployment in FileMaker
  10. Handling Multi-File Deployments
  11. Advanced Testing Techniques
  12. Monitoring, Alerts, and User Feedback
  13. Managing Hotfixes in a DevOps Environment
  14. Practical Tips for Transitioning to DevOps
  15. Summary and Final Thoughts
  16. Additional Resources and Further Reading

Introduction

The adoption of DevOps practices in FileMaker development can drastically improve the efficiency, quality, and agility of your projects. This guide dives deep into the transition from traditional development methods to a modern DevOps approach, focusing on tools, strategies, and practical tips to ensure a successful shift. Whether you’re new to DevOps or looking to refine your practices, this detailed guide covers every aspect of the journey.

What is DevOps?

The Evolution of DevOps

DevOps originated as a movement to integrate software development (Dev) and IT operations (Ops). The aim was to address the disconnect between these two traditionally separate entities, fostering a culture of collaboration and shared responsibilities. Over the years, it evolved into a methodology with practices designed to streamline software development, reduce errors, and speed up delivery.

DevOps vs. Traditional Development

Traditional development cycles, often referred to as Waterfall or Agile, involve distinct phases: design, development, testing, deployment, and maintenance. These phases are usually sequential, leading to long cycles and rigid processes. DevOps breaks down these silos by promoting:

  • Continuous Integration: Integrating code frequently.
  • Continuous Testing: Automating tests to catch bugs early.
  • Continuous Deployment: Deploying small changes rapidly.
  • Continuous Monitoring: Observing the system in real-time for improvements.

Core Benefits of DevOps

DevOps is not just about tools; it’s a cultural shift with tangible benefits:

  • Accelerated Time to Market: Faster delivery of features.
  • Increased Collaboration: Development and operations work together seamlessly.
  • Reduced Errors: Bugs are caught early and often automatically.
  • Enhanced Stability: Systems are more stable due to continuous monitoring.

DevOps in FileMaker: An Overview

Unique Challenges with FileMaker

FileMaker presents unique challenges to DevOps because:

  • Code is Embedded: Unlike other platforms, where code exists in text files, FileMaker’s code resides within the database.
  • Database-Centric: FileMaker integrates data and logic, making deployments more complex.
  • Binary Files: FileMaker solutions are stored in binary formats, complicating version control.

How FileMaker is Different from Other Platforms

Unlike traditional platforms:

  • FileMaker solutions blend data, schema, and business logic into a single file.
  • There is no direct code access like in .js, .php, or .py files.
  • Updates involve modifying the database itself, necessitating careful planning and automation.

Core DevOps Practices Explained

Continuous Integration (CI)

CI is a foundational practice in DevOps, involving frequent integration of code changes into a shared repository. Every code change is verified by an automated build, allowing teams to detect problems early.

Key CI Components for FileMaker:

  • Save as XML: Export solutions to XML format regularly for tracking changes.
  • Automated Tests: Run tests after each integration to verify stability.
  • Feedback Loop: Use tools like Slack or email notifications to alert developers of issues.

Continuous Delivery (CD)

CD extends CI by ensuring the main branch is always ready to deploy. Every change is automatically tested, validated, and made ready for deployment.

FileMaker Focus:

  • Ensure the XML version of the FileMaker solution is consistent and up-to-date.
  • Automate cloning and preparation of the database for final deployment.
  • Run QA checks on clones to ensure readiness for production.

Continuous Deployment (CD)

Continuous Deployment automates the final step of CI/CD, deploying every change that passes tests without human intervention.

Implementing in FileMaker:

  • Automate the Data Migration Tool for deployments.
  • Use Server Admin APIs to manage deployments from start to finish.
  • Set up automated checks to monitor the live environment after every deployment.

The DevOps Loop: A Detailed Breakdown

Planning and Agile Workflow

Agile planning plays a crucial role in DevOps. The development cycle starts with planning:

  • Sprint Planning: Outline work for the upcoming sprint (usually 2-4 weeks).
  • Backlogs: Maintain a clear backlog in tools like Jira, categorized by priority.
  • User Stories: Define user stories that describe features from the user’s perspective.

Development and Collaboration

Collaboration is key:

  • Branching Strategy: Use a branching strategy that aligns with Agile.
  • Peer Reviews: Conduct code reviews regularly to maintain quality.
  • Daily Standups: Keep short daily meetings to discuss progress and roadblocks.

Testing and Quality Assurance

Testing is the backbone of DevOps:

  • Unit Tests: Focus on individual components.
  • Integration Tests: Test how modules interact.
  • Acceptance Tests: Ensure the final product meets requirements.
  • Use Automated Tools for continuous testing after every code change.

Deployment Strategies

Deployments should be smooth, repeatable, and automated:

  • Data Migration Tool: Standardize deployments by migrating data from production to the latest version.
  • Zero-Downtime Deployments: Implement blue-green deployments or rolling updates for high-availability systems.

Monitoring and Feedback

Gather metrics to inform future development:

  • Performance Monitoring: Use tools like Zabbix to monitor server health.
  • Error Tracking: Implement log monitoring for error detection.
  • User Feedback: Gather feedback post-deployment to identify improvement areas.

Version Control in FileMaker Development

What is Version Control?

Version Control is the practice of tracking and managing changes to software code. It enables developers to revert to previous versions, collaborate, and maintain history.

FileMaker vs. Traditional Version Control Systems

In traditional development:

  • Code is in text files, easily managed by Git or SVN.
  • Changes are tracked line-by-line.

In FileMaker:

  • Code is embedded in the database.
  • Solutions are binary files, complicating traditional versioning.

Using XML for FileMaker Version Control

How XML Helps:

  • Export to XML: FileMaker Pro Advanced allows exporting database definitions to XML.
  • Track Changes: Use tools like BaseElements or FMPerception to compare XML versions.

Challenges:

  • XML is a single, large file—ideal would be splitting components (scripts, tables, fields) into separate files.

Comparing XML Files

Manual XML comparison can be daunting:

  • Use Diff Tools to highlight changes.
  • Identify core changes in scripts, layouts, or fields.
  • Keep a consistent structure for XML exports to reduce false positives.

Automated Testing in FileMaker

Why Manual Testing Falls Short

Manual testing, while effective, lacks consistency and speed:

  • Prone to human error.
  • Time-consuming and resource-intensive.
  • Lacks thoroughness, especially in regression scenarios.

Setting Up Automated Testing

Automating tests can catch bugs faster:

  • Use custom scripts to automate workflows.
  • Implement unit tests for business logic and calculations.
  • Utilize AppleScript or JavaScript for automated UI testing.

Unit Testing for FileMaker

Unit testing focuses on small components:

  • Create custom functions that perform isolated calculations.
  • Test each function with a range of valid and invalid inputs.
  • Automate the process with scripts that compare expected vs. actual results.

Integration Testing in FileMaker

Test interactions between components:

  • Scripted Workflows: Automate the navigation and execution of scripts.
  • Ensure data flows seamlessly between scripts, layouts, and modules.

Regression Testing and Maintaining Quality

Regression tests ensure new changes don’t break existing functionality:

  • Automate using Data Viewer scripts to check complex business logic.
  • Schedule tests after every deployment.

Automating UI Tests with Tools

UI tests are crucial but challenging:

  • Use AppleScript to simulate user interactions.
  • Explore third-party tools like AutoIt or Selenium for cross-platform automation.

Building an Agile DevOps Workflow

Scrum vs. Kanban: Which to Choose?

  • Scrum: Time-boxed sprints (2-4 weeks), structured with roles (Scrum Master, Product Owner).
  • Kanban: Continuous workflow, flexible priority-based task management.

Creating Effective Backlogs

Backlogs drive the development process:

  • Epics: Large stories broken down into smaller tasks.
  • User Stories: Simple, user-centered descriptions of desired functionality.
  • Prioritization: Regularly review and adjust priorities.

Tracking Progress with Jira

Jira can organize and track your development:

  • Boards: Visualize tasks (To-Do, In Progress, Done).
  • Sprint Reports: Track velocity and backlog health.
  • Version Control: Link tasks to commits and track progress.

Release Planning Best Practices

Organize releases effectively:

  • Create release plans with defined objectives.
  • Plan testing and deployment stages.
  • Use Jira’s release hub to track what’s included.

Automated Deployment in FileMaker

Common Deployment Models

Live Development Pitfalls

Making changes directly in production:

  • High risk of data loss and errors.
  • Impacts the end-user experience.
  • Should be avoided unless for urgent hotfixes.

Manual/Piecemeal Deployment

Step-by-step manual deployments:

  • Requires meticulous documentation.
  • Error-Prone: Easy to miss crucial steps.

Data Interface Separation Approach

Decouple data from interface:

  • Data File: Houses the database records.
  • Interface File: Holds scripts, layouts, and logic.
  • Simplifies upgrades by only replacing the interface file.

Data Migration Tool Deep Dive

What It Is: A command-line utility for migrating data between FileMaker files.

Why Use It:

  • Faster: Handles large data sets efficiently.
  • Consistency: Merges production data with development code seamlessly.

How It Works:

  1. Create a clone of the development file.
  2. Run the tool to migrate data to the new clone.
  3. Replace the production file with the updated version.

Challenges:

  • Handling complex calculations and container fields.
  • Managing external resources like documents and media.

App Upgrade Tool: Future Prospects

An emerging tool from Claris:

  • Promises incremental updates.
  • Ideal for small-scale patches.
  • Not yet production-ready but has potential.

Step-by-Step Deployment Guide

Pre-Deployment Planning

  • Review Change Logs: Ensure all changes are captured.
  • Backup Production: Always back up before starting a deployment.
  • Identify Risks: List any high-risk areas of the deployment.
  • Create a Rollback Plan: Define steps to revert if things go wrong.

Cloning Development Files

  • Use FileMaker’s Save as Clone feature.
  • Store clones in a dedicated deployment folder.
  • Double-check that clones are not opened to avoid metadata issues.

Data Migration Execution

  1. Run the Data Migration Tool using terminal or command line.
  2. Monitor progress via log files.
  3. Address any issues with calculation fields immediately.

Post-Deployment QA

  • Integration Testing: Ensure the production environment matches expectations.
  • Feature Checks: Verify that new features work correctly.
  • System Compatibility: Test external APIs, web services, and authentication mechanisms.

Managing Deployment Logs

  • Capture command-line outputs into log files.
  • Use log parsers to detect errors.
  • Document deployment issues for future improvements.

Handling Multi-File Deployments

Challenges with Multi-File Solutions

Multi-file deployments are more complex:

  • Files must be updated in a specific order.
  • Dependencies between files can cause complications.
  • File migration times vary based on size and complexity.

Concurrency Techniques for Faster Migration

Run migrations in parallel:

  • Use multiple instances of the Data Migration Tool.
  • Balance resources (CPU, memory) to avoid overload.
  • Test different concurrency levels to find the optimal configuration.

Tools for Multi-File Management

  • YADM (Yet Another Data Migration Tool): Automates the migration process for multi-file solutions.
  • Custom Scripts: Write automation scripts using FileMaker Admin API for orchestration.

Advanced Testing Techniques

Testing Workflow Automation

Automate the entire test workflow:

  • Use hooks to trigger tests on code commits.
  • Implement end-to-end tests using CI/CD pipelines.
  • Automate report generation for every test cycle.

Performance Testing for High-Load Scenarios

Simulate high-load conditions:

  • Use tools like JMeter or Apache Benchmark.
  • Focus on database stress tests, script execution times, and server load.

Security Testing and Vulnerability Scanning

Ensure the security of the FileMaker environment:

  • Implement vulnerability scanning tools.
  • Test for SQL injection, script injection, and data leaks.
  • Use behavioral analytics to detect anomalies post-deployment.

Monitoring, Alerts, and User Feedback

Application Monitoring Essentials

Track performance and system health:

  • CPU, Memory, Disk Usage: Critical metrics for server stability.
  • Error Rates: Monitor logs for script errors or failed calculations.
  • Response Times: Track how fast queries and scripts execute.

Zabbix and Other Monitoring Tools

Zabbix can be a valuable asset:

  • Alerting System: Send alerts for critical errors.
  • Dashboards: Real-time views of server health.
  • Integration: Connect with FileMaker APIs for deeper insights.

Gathering User Feedback for Continuous Improvement

User feedback is invaluable:

  • Implement feedback loops after each deployment.
  • Use surveys or in-app prompts to gather responses.
  • Analyze data for trends and areas of improvement.

Managing Hotfixes in a DevOps Environment

Hotfix Workflow Best Practices

Proper hotfix management is essential:

  • Development First: Always create fixes in the development environment first.
  • Quick Tests: Prioritize fast validation.
  • Manual Deployment: Deploy manually using the Piecemeal approach.

Minimizing Disruptions During Critical Fixes

  • Time Box: Restrict the time allocated for fixes.
  • Monitor in Real-Time: Observe the environment during and after the hotfix.
  • Document Everything: Keep detailed records of the issue, resolution, and lessons learned.

Practical Tips for Transitioning to DevOps

Step-by-Step Approach

  1. Start Small: Focus on one aspect of DevOps, like CI or automated testing.
  2. Tooling: Choose the right tools for your environment (e.g., Jira for planning, Git for version control).
  3. Team Buy-In: Ensure your team is on board with the cultural shift.
  4. Iterate: Refine your processes based on what works and what doesn’t.

Common Pitfalls and How to Avoid Them

  • All-or-Nothing Mindset: Transition gradually, not all at once.
  • Ignoring Culture: DevOps is as much about mindset as it is about tools.
  • Over-Engineering: Keep solutions simple; focus on solving problems, not adding complexity.

Summary and Final Thoughts

The journey to modern DevOps in FileMaker involves understanding the unique challenges, adopting suitable tools, and building workflows that fit your needs. From version control to deployment, testing to monitoring, each aspect of DevOps enhances your capability to deliver high-quality solutions faster. By embracing these practices, you’ll find that deployments become smoother, collaboration improves, and the end-user experience benefits.

Additional Resources and Further Reading

This guide provides a comprehensive overview of everything needed to successfully integrate DevOps into FileMaker development. Each section is expanded to give practical steps, context, and strategies for both beginners and experienced developers.

Dimitris Kokoutsidis

Related Articles