MirrorSync 6 basic setup

Looking for MirrorSync 5 documentation? See MirrorSync 5 basic setup
Looking for MirrorSync 4 documentation? See MirrorSync 4 basic setup
Looking for MirrorSync 3 documentation? See MirrorSync basic setup

Overview

MirrorSync is an elegant synchronization product that can sync between any combination of FileMaker Pro, FileMaker Go, FileMaker Server, SQL database (MySQL, Oracle, SQL Server, or any database that supports JDBC), Salesforce, Amazon DynamoDB, and Amazon RedShift. For mobile devices, users can sync the hosted file with their FMGo or Pro clients, go off network to make changes, and sync again when a connection is available. For servers, MirrorSync allows a database to run in multiple locations, or to integrate different databases and make sure that changes to each server are reflected in the other server.

MirrorSync is only installed on the server computer(s). It is not a plug-in, and does not require any software installation on client devices.

MirrorSync is extremely easy to set up. The only required database changes are:

1. Enabling the FMREST extended privileges in your security configuration
2. Making sure that every table you want to sync has a primary key, modification timestamp, and creation timestamp. If your database is created with FileMaker 17 or later, these fields are included by default.
3. Creating a layout for each table.
4. A few copy and paste operations.

Why use MirrorSync?

1. For mobile users who access the database with FileMaker Go on their iOS device, MirrorSync allows these users to work efficiently with limited or no connectivity to the server network. Each user can take a local copy on their iPad or iPhone, input the information from the field, and then sync when they regain a network connection. Depending on their workflow, they could even specify a one-way sync where their information is pushed to the server, but no data is pulled down, or vice versa for users who need fast read-only information.
2. For remote users who work on laptops, MirrorSync solves the problem of how to work productively on a hosted FileMaker database, regardless of network speed. If, for example, you have multiple users connecting to FileMaker Server over a wide area network (WAN) you could notice a substantial slowdown, as every change is immediately written back to FileMaker Server. MirrorSync allows each user to take a local copy of the database which would run on the user's machine and not be affected by the WAN performance. The user could sync when they first came into the office, work on their local copy throughout the day, and sync at specified times (or on demand) to push and pull the latest data to the server.
3. MirrorSync is a great solution when server uptime must be maintained. By installing MirrorSync on two computers running FileMaker Server, you can have your main production server constantly syncing with a backup server with a one-way server-to-server sync. In the event that the production server goes down or suffers database corruption, the backup server will be ready to immediately take over.
4. For groups of users at multiple locations, MirrorSync should be used in server-to-server mode, where an identical copy of the database is installed on FileMaker Server in each location. Each workgroup works at full LAN speed on their own FileMaker Server, and MirrorSync uses a bidirectional sync to take care of keeping the servers in sync with each other, so that each workgroup can see and edit changes made by the other group. In this configuration, MirrorSync should be set to sync very often, such as every 30-60 seconds.
5. To maximize performance and reliability, MirrorSync can be used in a clustering configuration, where two or more servers are running at the same data center and synchronized bidirectionally so that they all contain the same set of data. This means that each server only needs to work with 1/2 or less of the total users, and if one server goes down, the other server can take over the entire workload seamlessly. For a video demo of this, see https://www.youtube.com/watch?v=bQlGHrNtHCg
6. For integration with non-FileMaker databases, MirrorSync is the simplest and most efficient way to accomplish this. For instance, if you have an online storefront running with MySQL, Oracle, or DynamoDB at a remote data center, but you need to have instant access to all of the orders in a database running on FileMaker Server, MirrorSync can selectively sync certain SQL tables to FileMaker, even if the table and field names do not match. In this configuration, MirrorSync should be set to sync fairly often, such as every 5-10 minutes.

Requirements

• Server:
  • Windows Server 2012r2 or later / Mac OS X 10.12 or later / Linux running Tomcat 6 or 7.052 or later.
  • FileMaker Server 17 or later is required; FileMaker Server 18v3 or later is recommended. FileMaker Cloud is supported, although it cannot support all MirrorSync features (See special instructions for syncing with FileMaker Cloud).
    NOTE: Starting in MirrorSync 6.21, FileMaker Server 17 is no longer supported. If you have FMS 17, you will need to use MirrorSync 6.12 or earlier. For FMS 16 or earlier, you will need to use MirrorSync 5 or earlier.
  • 3 gigabytes or more of RAM. 8 gigabytes or more are recommended for large databases with hundreds of thousands of rows, or for more than 10 simultaneous sync operations.
• Configuration utility (only run by developer / administrator):
  • OS X 10.8.3 or later, or Windows Server 2012v2 or later.
  • FileMaker Pro 17 or later
• Sync Client:
  • Mac or Windows with FileMaker Pro 17 or later OR
  • FileMaker Go 17 or later running on iPad, iPhone, or iPod Touch

MirrorSync With Hosting Providers

MirrorSync supports hosting provider installations of MirrorSync.

If you would like for 360Works to host your FileMaker databases with MirrorSync, please contact us directly about this service.

Here is a complete list of companies that have indicated that they are set up or willing to configure their servers to host MirrorSync. If you'd like to be added to this list, please contact us at plugins@360works.com.

• 360Works ( http://www.360works.com/ )
• ODI Technologies ( http://www.oditech.com/ )
• FMPHost ( http://www.fmphost.com/our-services/360works-mirrorsync-and-zulu-hosting )
• Foxtail Technology ( http://www.foxtailtech.com/ )
• Michael Tinder, DataUp365 ( http://dataup365.com/DU/ )
• Andrew McCallum, Niche IT Pty Lty ( http://www.nicheit.com.au/ )
• NeoCode Software ( http://www.neocodesoftware.com )
• WorldCloud ( http://www.worldcloud.com )
• ZeroBlue Technology Solutions (http://store.zerobluetech.com/hosting/filemaker-hosting.html)
• Data Therapy LTD (http://www.datatherapy.com/hosting/)
• MC Services (http://www.mcservices.com/filemaker/filemaker-hosting/)
• Longterm Solutions (http://www.longtermsolutions.com)
• MainSpring, Inc(https://gomainspring.com/)

The following companies support MirrorSync, but only in dedicated (not shared) configurations:

• Productive Computing ( http://productivecomputing.com )
• Triple 8 ( http://triple8.net )
• Point in Space ( http://www.pointinspace.com )

If you are a hosting provider, read the Installation for hosting providers' section in the advanced documentation page.

Technical overview

MirrorSync is a web application, installed on a server (typically on FileMaker Server). A FileMaker script that is pasted into your FileMaker Solution uses the Insert from URL script step to communicate with MirrorSync. The Data API is used during configuration, and to trigger the sync script on FileMaker Server. Because the Insert from URL script step is used for most data transfer, the amount of bandwidth used by the Data API is negligible.

For FileMaker Pro/Go sync, users run the sync by triggering a regular FileMaker Script in the solution. For server-to-server sync, the syncs are run using the MirrorSync admin utility, and can be scheduled to run in the background at regular intervals.

There are no plug-ins or other software to install on user's devices.

Three ways to get started with MirrorSync

We've setup three ways for you to try MirrorSync:

1. Live demo (time to sync: Less than 1 minute): If you would like to see MirrorSync in action without needing to install anything, you can download our live MirrorSync demo database by clicking the yellow button titled 'Try live demo now' from the main MirrorSync page. Just open the file in FileMaker Pro or Go, make some changes, and then hit the sync button. To see how MirrorSync works between multiple devices, use that same download URL on a different device and sync away! Be aware that the data in this file is shared with everybody who downloads it, and also that it resets every night at midnight EST, so don't put anything important in this file.
2. Sync using the Tasks template (time to sync: Approximately 20 minutes): Using the tasks starter solution that comes with FileMaker Pro, we provide a step-by-step video walkthrough that you can follow along with. This 13 minute video starts with creating a new FileMaker database and goes through setup and sync. Just watch the video for each step, and pause while you do it on your own file. Watch the video on YouTube.
3. Configure your own file for syncing (time to sync: 30-60 minutes for most solutions): Follow the instructions below to configure MirrorSync with your own FileMaker solution.

Installing / upgrading

MirrorSync has a standard OS X and Windows installer included with it. Just double-click the installer for your platform and follow the instructions. Be aware that if you re-install FileMaker Server at some point in the future, you may need to re-run the MirrorSync installer in order to set the web server settings correctly. If you are running into other issues during installation, please go over our troubleshooting steps here.

MirrorSync can also run on Linux. Starting in MirrorSync 6.5, there is an installer packaged with MirrorSync. See the instructions here on how to run it. You can also install MirrorSync manually but it requires you to configure Tomcat yourself. Read our advanced docs for how to manually install MirrorSync on Linux

Preparing your database for use with MirrorSync

• If you have a multi-file solution (ie. separation model or migrated from older FileMaker version), make sure that there is a single main file with external table occurrences referencing the other files. If you are using a separation model, this should be the data file. When you distribute offline files to your users, you will distribute the entire set of files, which will include the main sync file.

• Be sure that the FMREST extended privilege is enabled in your solution for all accounts that will use MirrorSync. Choose this option under File > Manage > Security… Double click the FileMaker user, and choose Edit… from the Privilege Set.

• Make sure that every table in your solution has a primary key. This is a unique identifier which cannot be empty, and should never be changed once the record is created. MirrorSync can work with traditional serial number primary keys, UUID primary keys, or custom primary key strategies. It will save steps during setup (but it's not required) for your primary keys to have the 'not empty' and 'unique' validation option enabled. For a deeper discussion of how primary keys are handled when syncing, refer to 'Primary key considerations' in the advanced documentation. Do NOT create new primary keys for use with MirrorSync, unless you have tables that do not already have primary keys (read the 'primary key considerations' link for an explanation)
• If you are using an auto-entered calculation for your primary keys (such as Get(UUID)), be sure to UNCHECK the field options checkboxes that say 'Do not replace existing value of field'
• We highly recommend, but do not require, that you set primary key validation to happen 'always' and do not allow user overrides.
• If your solution uses serial numbers as user-visible data for things like job numbers, check numbers, P.O. numbers, or invoice numbers, refer to the section 'User-visible serial numbers' in the advanced documentation.

• Every table (including join tables) must have a timestamp field which is set to auto-enter a modification timestamp, as well as a creation timestamp field which auto-enters the creation timestamp. If you will have users with direct, online access to FileMaker Server who are not in the same time zone as the FileMaker Server, you will need to set up special host modification timestamps as outlined in the 'Time zone considerations' section of the advanced documentation. This requirement does not apply to offline users in different time zones, who access the server via MirrorSync.
• Please remove or rename fields that have a backslash (\) in the name. These fields are incompatible with MirrorSync.
• Host the database on FileMaker Server. Open the file from the server with FileMaker Pro Advanced, using a full access account.

Integrating MirrorSync into your solution

Here are the steps to integrate MirrorSync into your solution:

1. Install MirrorSync. You can install MirrorSync on any Mac, Windows, or Linux computer, whether or not it is running FileMaker Server. During installation, you will create a username and password that will be used to administer MirrorSync. If you choose to install MirrorSync on a computer other than your FileMaker Server, you will need to also install an extra copy of MirrorSync on your FileMaker Server if you want to use the 'download link' feature for easy distribute to offline users.
2. After installation, you will be taken to the MirrorSync home page. This is located at http://yourServerAddress/MirrorSync, or in some cases, http://yourServerAddress:42424/MirrorSync. Keep in mind that you can open this page and proceed with the rest of the integration process from your own computer - you don't need to remote desktop to the computer where MirrorSync is installed. From the home page, click the blue 'Launch MirrorSync Client' button.
3. To log in to the configuration client, the server address should match the URL you used to reach the MirrorSync home page to download the configuration client in the previous step. You can copy and paste this value from the browser into the server address field. Use the username and password you created during installation to log in to the config client, and then follow the setup instructions in the config client. You can follow this video which walks through a simple configuration set up for more guidance.
4. After completing the setup, if you will be syncing with FileMaker Pro or Go, you will need to distribute offline copies of your solution to your users. Click the 'Download Database' button in the config client and choose either 'Download' for quick testing on your own computer, or 'Create Link' for deployment to end users.
5. Once you have created an offline copy, run the 'MirrorSync 6' script to do the initial sync, and then run it again whenever you want to sync with the server. If you'd like to, you can add a button somewhere on the layout for users to run the sync easily.

Distributing new versions of your application

MirrorSync 6 makes it extremely easy to keep your offline copies up to date with changes made on the server. If you're using the new ability to sync all layouts starting with a certain prefix, it is no longer necessary, as it was in previous versions, to edit your configuration after you've added or removed new tables or fields. Just make sure that if you create a new table, you create a corresponding layout with a prefix that matches what you've configured. It is also no longer necessary to add any fields to this layout - it can be completely blank (unless you select the MirrorSync option to only include fields on the layout).

Once you've made your development changes on the hub or development copy, modify the field 'DatabaseVersion' in the MirrorSync table (see screenshot below). By changing this value (to anything you want, as long as it's different than what it was before), users will be prompted on the next sync to download your new updated solution file (screenshot below). If you're working on a separate development file (as opposed to directly on the production hub database), we recommend using 360Deploy to deploy your development copy to the hub. This works well in conjunction with MirrorSync. If you're using a separate mobile file, change the DatabaseVersion field in that separate mobile file, not the hub file. This auto-update feature will also work for FileMaker spoke databases syncing with non-FileMaker hubs, such as MySQL, SQL Server, or Oracle.

This feature also works for server-to-server syncs. Changing the DatabaseVersion field in the MirrorSync table notifies MirrorSync that it should replace the spoke database with a new copy of the database from the hub. During the configuration process, you can define an "update window", so that replacing the spoke database can be done after hours. By default, this update window is from 11pm to 3am.

When you change the value of the "DatabaseVersion" field, users will be prompted at the end of a successful sync to download the new version of the file

Users can choose "not yet" to continue using their current file or choose "ok" to update their file. If they choose "ok" then and updater window will open and the new file will download and automatically open.

Server side initial syncs

MirrorSync download URLs can now run the initial sync on the server, and then download the already synced offline copy to the user's device. This is much faster and more reliable for large, slow initial syncs.

To set this up, open the MirrorSync Config client, select the sync you want to generate a download link for, and click "Download Database". This will give you the option to disable a previously generated download link, create a download link for distribution, or download the database directly.

To generate a link to distribute to your users, click Create Link. This will then show you the below dialog. Select either full copy or empty clone, whichever is appropriate for your sync. Select whether the solution syncing is single file or multi-file. Selecting Client side for initial sync will work the same
as MirrorSync has always worked, users will download the file and the initial sync is run from their device. Selecting "Server" will trigger the initial sync before the file is downloaded from the browser

When users enter the Server side initial sync URL into a browser they will be prompted to enter in credentials for the FileMaker Database. This will ensure that only the records that they have access to are synced to the file

If you chose the option in the configuration to use hub database credentials to log in to the hub database, then users will not get this prompt.

Once authorized, the sync will start and there will be a delay while the initial sync runs before the file is downloaded.

Important warning

It is very important that once an initial sync has been completed on a file, that synced file MUST NOT BE SHARED AMONG MULTIPLE DEVICES. For instance, it is a very bad idea for an administrator to run the initial sync on a database file, and then email duplicate copies of that synced database to multiple users. It is OK to email that synced database to a single user, then run an initial sync on another database that has never been synced before, send that second file to a second user, and then repeat for each additional user.

A subtle way that this can happen is if an already-synced offline copy of the database is placed on the FileMaker Server as the new ‘master’ database. When this already synced database is copied to each offline user, that violates the rule about sharing already synced databases. If you do need to take a synced offline copy and host it on the FileMaker Server for some reason, be sure to delete all of the records in the MirrorSync table before anybody copies the database. This will help MirrorSync realize that this database should go through the initial sync process on the next sync, and be treated as a new unique copy of the database.

Advanced topics

This document should get you up and running with MirrorSync. However, there are many advanced features and customizations available, which are covered on the MirrorSync advanced topics page.

Getting help

If you encounter any problems while syncing, please go to the MirrorSync homepage and click the link titled 'report a bug'. This will send us the sync log file, which is required to solve almost all issues. Be sure to include your contact info in the bug report so that we can get in touch with you.

If for some reason the bug report feature does not work, you can still manually submit your logs via email to support@360works.com . Please make sure to include a thorough description of the issue that you are experiencing in the email and then attach your logs. By default, MirrorSync logs will be located at:

Windows
C:\Program Files\360Works\Applications\logs

Mac
/Library/360Works/Applications/logs

Mirrorsync logs follow the naming convention <instanceName>.<timestamp>.log so if your instance is called the default "MirrorSync" and the date is January 1st, 2020 then the logs will be called MirrorSync.2020-1-1.log . Be sure to get the log with the timestamp that correlates with when the issue was reproduced. In some cases your logs may be too large to email even when compressed. If that is the case, go ahead and send an email without the logs (include that you are sending them through the uploader on our website)and wait for the automated response back with your ticket number (starts with "HD"). Once you have that, you can upload your logs here. Please be sure to note your ticket number in the upload form.

There is an online discussion forum hosted by FMForums. Please look through the posts and see if your question has been posted before making a new topic.

If you have other questions, we have a dedicated support team that is ready to help. Send us an e-mail at support@360works.com. If it's related to a sync problem, it's better to use the 'report a bug' link described above, since that will attach the sync log file.